English JPEGmini – the Future of Image Compression? ע״י עידו גנוט פורסם ב 25/08/2011 2 0 An Israeli company recently unveiled what might be the next generation of image compression technology. Based on the JPEG format, the new JPEGmini is compatible with existing JPEG but capable of reducing the file size of an image by up to 5X without any visible loss of image quality. A revolution on its way? MegaPixel.co.il set out to test it and talk with the developers. Several weeks ago we accidentally came across a website of an unknown company with a bold statement – a new compression technology which can compress the file size of any image by up to 5 times with no visible loss of quality compared to the original image, all that while still being fully compatible with the current JPEG format. To be honest we were quite skeptical – attempts have been made in the past to create improved image compression technologies, and when giants such as Google (with its WebP technology) breathing down its neck, JPEGmini has a lot to prove. To learn more about the technology, we scheduled a meeting with ICVT – the start-up which has been working for the past several years on JPEGmini, located in one of Tel-Aviv’s tallest buildings (surprisingly just one floor below Google's main offices in Israel). We will get to our interview with ICVT shortly, but first some background on JPEGmini. JPEGmini uses classic JPEG technology. This has both advantages and disadvantages. The biggest disadvantage is that the theoretical compression threshold is limited by the JPEG format (we will discuss what JPEGmini was able to do about that later in the interview). This is unlike rival compression formats which are using different compression algorithms capable of reducing the file size further. On the upside JPEGmini doesn't have compatibility problems like other proprietary formats, issues which have severely restricted their use. If JPEGmini does not compress images beyond conventional JPEG, then you might ask what is the big innovation here? The answer has to do with the technology developed by ICVT which allows JPEGmini to compress any image just below the level where artifacts start to appear. The technology which is based on several patent-pending algorithms, was developed to mimic the way the human eye (and mind) perceive images and to compress each image automatically to a level just above that where artifacts would begin to appear. JPEGmini has several advantages for photographers as well as other potential users. JPEGmini can save space on the hard drive but can also save time when sending images using e-mail or other services (some of them are also limited by the size of files that can be transferred each time). In many countries upload speeds are restricted and mobile data transfer can become costly when transferring many large images. Finally online storing and backing up of images can be costly and compressing them efficiently can save those costs. Other potential clients that can benefit from the JPEGmini technology are businesses – especially those with huge repositories of images. Top among those are giants such as Facebook and Google but smaller companies might be able to benefit as well, saving money by reducing the size of their image databases and cutting costs on data transfers. Testing JPEGmini Currently you can find a free working technology demonstration on the JPEGmini website. You can upload a single JPEG file (or if you register, several files) and download the corresponding JPEGmini file. We decided to try the technology ourselves and ran some tests on different images (all the images we tested were full size images taken with a Nikon D5000 at 12MP, fine). Using the JPEGmini service we were able to reduce the size of the original image by a factor of 5.6X (i.e. from about 4MB to slightly less than 800KB). If we compare this to Adobe Photoshop compression in terms of size (on the software scale ranging from 0-12) we are talking about a compression comparable to about 6-7. Aggressive compression – you can see the "steps" in the sky Now for the million dollar question, how was the quality of the JPEGmini images? Well, we tested a variety of images (although a relatively small number) and we consulted with at least 4 different independent experts in image quality. In all cases we were not able to detect any difference in either detail or color in any of the images we tested at 100% magnification compared to the original image. A dog magnified to 100% – find the differences – the original on the right, JPEGmini on the left (Credit: Iddo Genuth) Once we tried to enlarge the image beyond 100% we did find some artifacts but this is not unique to JPEGmini (ICVT can only guarantee no visible difference up to 100% magnification anyway). You can test JPEGmini yourselves and see if it lives up to its claims. A beach photo magnified to 400% – look at the differences – the original on the right, JPEGmini on the left (Credit: Iddo Genuth) JPEGmini – Compressed interview In order to understand JPEGmini's technology better, MegaPixel conducted an exclusive interview with Dror Gil, ICVT’s CTO. We started with a few background questions (some of the answers are fairly technical and readers can feel free to skip to the main part of the interview). Q: How does the JPEG format works? A: JPEG is a lossy compression, meaning that any JPEG compression involves some loss of information. The compression consists of several steps: Color space transformation from RGB to YCBCR Reducing the spatial resolution of the CB and CR components by a factor of 2 in the horizontal axis, or both in the horizontal and the vertical axis (this step is optional). Dividing each color components to blocks of 8×8 pixels. Transforming each 8×8 block to the frequency domain using a DCT transform. Quantization (reduction of precision by representation with a lower number of bits) of each of the frequency components using a quantization matrix. This is the main stage that causes loss of information. Entropy coding (a lossless compression, similar to ZIP) on the frequency components which have undergone quantization. Q: Is saving an image in Photoshop in minimum JPEG compression (i.e. 12) lossless? A: Even at quality 12, the Photoshop quantization matrix has values which are greater than 1, so the precision of some of the frequency components is reduced, meaning that some information is lost (you can view the quantization matrixes for Photoshop’s quality 12 here). Actually, even when using a quantization matrix that has all values set to 1, there is a loss of quality due to the DCT transform and the conversion from floating point values to integer values. Q: What are the compression limits of the JPEG format? A: It varies based on the content of the image. And sometimes you increase the quality setting in Photoshop and the actual quality of the image is reduced… You can find more information on this issue, and a graph of file size vs. quality in Photoshop here. Q: Why do you think JPEG became the most widespread image format in the world? A: The JPEG format was invented in the 1990s, when digital photography started. All digital cameras used it, so it was supported by all PC and Mac software, and also in DVD players, MP3 and MP4 players, cellular phones, digital picture frames, etc. It is very difficult to break this value chain, since if you want to support a new format you need to change camera hardware, computer software, reprogram the firmware on electronic devices, etc. The JPEG-2000 format was approved in 2000, and is based on wavelet transform instead of DCT. Since the relative advantage of JPEG-2000 over JPEG in compressing high quality photos is small (around 20%), JPEG-2000 had very low market penetration, and is used today only in specific applications, mainly in the medical and military segments. A new format called JPEG-XR promises better compression than JPEG-2000, but it was approved only in 2009, so it will take a long time until it is used in mass-market applications (if at all). Q: What are the main problems associated with compression using JPEG? A: The main problem with JPEG compression is that as the compression level increases and the file size decreases, more artifacts are created in the image. These artifacts are mainly in the form of blockiness (broken lines) and “noise” around edges in the picture. To prevent these artifacts, you can lower the compression level, but then the file size increases significantly. Now we move to the main interview with Dror Gil regarding JPEGmini's technology: Q: How did you come up with the idea to develop JPEGmini? A: We met with a large company that managed a data center which provides backup for user files and data, but did not handle user photos, since they claimed the files were too big and the cost of storage was too high. Then lead us to think about a solution to reduce costs for big data centers by reducing the file size of photos without hurting their quality. We started to research various options, and recruited image processing experts who helped us reach the solution. Q: Why do we need JPEGmini if we can simply set Photoshop (or any other software) to compress to a specific level (say 6 or 8 in Photoshop)? A: The problem with setting an arbitrary compression level is that at each compression level, there will be some photos for which artifacts will be created. So you cannot set a fixed compression level (especially not a medium or relatively high compression level) for which no artifacts will be created in any photo. That’s why digital cameras set a very low compression level, which ensures that artifacts will not be created in almost all photos. This creates a significant increase in the file size of the photo, which subsequently increases the storage requirements in the memory card, in the PC used to store the photos, and in online photo hosting services. According to the tests we have conducted, it is not possible to set a fixed compression level that will give optimal results (in terms of both quality and file size) for every JPEG photo, at every resolution and every quality level of the original photo. So even if the user has photos from a single camera at a single resolution, there is a difference in the compression level that can be applied to these photos without hurting their quality, depending on the original photo quality, amount of detail, lighting, level of focus in the subject and the background, etc. Therefore, a user that will apply a fixed compression level will hurt some of the photos by creating artifacts (the compression level will be too high), while for other photos the file size will be unnecessarily large (the compression level will be too low). Furthermore, a user or family that owns several cameras and cellular phones, each one generating photos in different resolutions and qualities, and also receive photos of themselves from other users, create a photo collection on their computers which has a large variation. In this scenario, it is clear that a fixed compression level is not a suitable solution at all. Not to mention companies with large data centers such as Facebook, Flickr and Picasa who receive photos from thousands of different camera and mobile phone types. Q: How does the JPEGmini algorithm work? A: The algorithms we have developed, on which we have filed 14 patent applications, simulate the human visual system. Our technology analyzes each specific photo, and determines the maximum amount of compression that can be applied to the photo without creating any visual artifacts. In this way, the system compresses each photo to the maximum extent possible without hurting the perceived quality of the photo. Q: How much compression does JPEGmini enable? A: Our technology can reduce the file size of photos by a factor of up to 5x without hurting their quality. The amount of compression that our algorithms apply varies from photo to photo as mentioned above, buy you can estimate the average amount of compression according to the photo resolution, as described in the following table: Photo Resolution Average File Size Reduction Using JPEGmini 8MP and more 70 – 80% 5-7MP 60 – 75% 3 MP 50 – 60% 2 MP 30-50% 1024×768 25 – 35% 800×600 20 – 25% 640×480 or less 10 – 20% Q: You are using JPEG as your basis, but claim that you are using it better than others – could you explain what does that mean? A: The JPEG standard defines the format of the compressed image, but not the compression process itself. Most JPEG compressors on the market today use a common implementation of JPEG encoding called IJG. We developed our own JPEG encoder which optimizes the compression tools in the JPEG standard for each specific photo. Therefore, even for the same compression level, our JPEG compressor will create a smaller file than a common JPEG compressor, while our file is fully compatible with the JPEG standard and can be viewed by any image software or mobile device. When you combine our compression capability at a given quality level, together with the technology we have developed to analyze photos from a human perspective and set the optimal quality level for each photo, you get the smallest JPEG file which can represent each photo without hurting its perceived quality. Q: What about quality – what have you done to prove that your algorithms actually preserve the quality of the original image? A: To verify that the image quality after JPEGmini processing is unchanged, we conducted user testing according to ITU BT.500 standard for image quality testing. Results of the user testing have shown that users could not distinguish between the original image and the image that has undergone JPEGmini processing, when viewing the images on a screen at 100% zoom (each pixel on the screen represents a pixel in the image), and when the images are printed at 200 DPI. Q: You call your technology JPEGmini – how compatible is it with existing JPEG technology? A: From the moment we started to work on the technology, it was very important to us that it will generate fully standard JPEG files, which can be viewed in any software, browser or consumer device. All the algorithms we use are based on the JPEG baseline standard, with no extensions or modifications, so the resulting files are regular JPEGs. We think that attempts by other companies to improve compression using a new file format (like Google are trying to do with WebP) are doomed to failure, due to the strong JPEG value chain that was discussed above. It is also important to point out that WebP does not have any quality measure, so the user has to manually set the compression level using trial and error (increasing the compression level gradually, and examining the photos to see if any artifacts were introduced). Therefore, this format cannot be used for automatic recompression and file size reduction of JPEG photos. On the other hand, when using JPEGmini technology the quality measure is integrated into the compression process, so we can recompress millions of photos automatically, with no human intervention required, to the maximum extent possible for each photo (up to 5x) while ensuring that no photo will be damaged during this process. Q: Will users be able to identify JPEGmini files? A: We add the suffix “_mini” to the file name before the .JPG extension. We also mark in an EXIF field inside the file that it has undergone optimization by JPEGmini technology. Of course the user can rename the file or remove the EXIF field, and this will not affect the photo quality in any way. Q: How do you see your technology helping photographers especially in a time when internet bandwidth is constantly increasing and the price of data storage is decreasing – who really needs compression anyway? A: Bandwidth and storage capacity are always increasing, but so does the resolution of cameras. People are now filling their internal and external disk drives rapidly, and have to replace them often – we can delay the replacement by a few years. Uploading the photos to online services also takes a long time, since for most people the upload bandwidth is much smaller than the download bandwidth, and we can shorten this time significantly without hurting the photo quality. In addition, many users today hook up their computers to cellular phones or modems, and in cellular networks the available bandwidth is lower, and data costs are much higher. This is also true for smartphones, which have high resolution, high quality internal cameras. Smartphone users want to share their photos in real-time using social networks or online photo sharing sites, and this is also done over cellular networks of course. In this case we save the user not only time but also money, since in most countries mobile operators do not offer unlimited data packages anymore. Another significant advantage of our technology is in sending photos over email, which is still a very common method of photo sharing for unsophisticated users. Despite the reduction in storage and bandwidth costs, the maximum size of files that can be attached to a Gmail message, for example, is still limited to 25 Megabytes. Therefore, only 10-15 high quality photos can be attached to a single message, and we are all familiar with the situation of users who share a photo album by splitting it into several emails, or reducing the photo resolution which significantly affects the user experience for the recipient. With JPEGmini you can transfer up to 5 times more photos in an email message, with their original resolution and quality, which in most cases enables sending a full album in a single email message. Q: What other applications beside photography do you see as relevant for JPEGmini and who are your potential clients? A: Our main potential clients are data centers of large web companies such as Google, Yahoo, Facebook, Microsoft, Amazon and Apple who store hundreds of millions and sometime billions of photos. We can significantly reduce their storage costs. Other potential clients are Content Delivery Networks (CDNs), who can save bandwidth and costs for themselves and their customers using our technology. Photo-rich websites can improve the user experience, since reducing the photo file size results in reduced page load time. This advantage is also relevant to developers of flash applications and games, who can reduce the size of images in the app, and consequently reduce the time it takes to download the app to the browser. Q: What about security and military related applications? A: We are currently in contact with potential customers in this field, but we cannot elaborate on this issue for obvious reasons. Q: Where do you stand now – what kind of services do you currently offer and when are you going to launch new services? A: In mid July we launched a web service, which enables any user to upload an unlimited number of photo albums to our website, and download the compressed version of those albums after JPEGmini processing. You can also test the technology by uploading a single photo, and comparing the quality of the original and the JPEGmini versions online. The service is currently provided for free, to give our technology maximum exposure. Q: Did you consider for example developing plug-ins for different applications? A: We are currently evaluating the business case for several other products and services, including a Web API for our service, which will enable companies and websites to send us photos directly (without using the GUI on the site); offline desktop software for local JPEGmini processing without having to upload the pictures to our site (we may implement this as a Photoshop plug-in); and various mobile applications. This article was adapted from the original Hebrew version.