Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Image processor
open-in-new
<h2 id="function">Function</h2> <h3>Bayer transformation</h3> <p>The <a href="/facts/Photodiode/152Rpk06">photodiodes</a> employed in an <a href="/facts/Image_sensor/j3GNrRln">image sensor</a> are color-blind by nature: they can only record <a href="/facts/Shades_of_gray/0Kdytv1P">shades of grey</a>. To get <a href="/facts/Color/XDvW7ESP">color</a> into the picture, they are covered with different color filters: <a href="/facts/Red/gLnFAVx4">red</a>, <a href="/facts/Green/gFTac7St">green</a> and <a href="/facts/Blue/DHWb8MUa">blue</a> (<a href="/facts/RGB/3Be0zwCj">RGB</a>) according to the pattern designated by the <a href="/facts/Bayer_filter/yC8UPoVU">Bayer filter</a>. As each photodiode records the color information for exactly one <a href="/facts/Pixel/UzwBvXBV">pixel</a> of the image, without an image processor there would be a green pixel next to each red and blue pixel. </p><p>This process, however, is quite complex, and involves a number of different operations. Its quality depends largely on the effectiveness of the <a href="/facts/Algorithm/fnl5NmRt">algorithms</a> applied to the raw data coming from the sensor. The mathematically manipulated data becomes the recorded photo file. </p> <h3>Demosaicing</h3> <p>As stated above, the image processor evaluates the color and <a href="/facts/Brightness/e93R5I4b">brightness</a> data of a given pixel, compares them with the data from neighboring pixels, and then uses a <a href="/facts/Demosaicing/sbtIMmN6">demosaicing</a> algorithm to produce an appropriate color and brightness value for the pixel.<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> The image processor also assesses the whole picture to guess at the correct distribution of <a href="/facts/Contrast_(vision)/zD9F91Je">contrast</a>. By adjusting the <a href="/facts/Gamma_correction/cApBLTYj">gamma</a> value (heightening or lowering the contrast range of an image's mid-tones), subtle tonal gradations, such as in <a href="/facts/Human_skin/cfRXQwSs">human skin</a> or the blue of the <a href="/facts/Sky/wwvQ84J1">sky</a>, become much more realistic. </p> <h3>Noise reduction</h3> <p><a href="/facts/Image_noise/zwhrU8lc">Noise</a> is a phenomenon found in any <a href="/facts/Electronic_circuit/IRfQ6Cc2">electronic circuitry</a>. In <a href="/facts/Digital_photography/a2UfweJh">digital photography</a> its effect is often visible as random spots of obviously wrong color in an otherwise smoothly-colored area. Noise increases with temperature and <a href="/facts/Exposure_(photography)/Rzr0aGMd">exposure</a> times. When higher <a href="/facts/ISO_5800%3a1987/HjnLPrIb">ISO</a> settings are chosen the electronic signal in the image sensor is amplified, which at the same time increases the noise level, leading to a lower <a href="/facts/Signal-to-noise_ratio/qohClhyG">signal-to-noise ratio</a>. The image processor attempts to separate the noise from the image information and to remove it. This can be quite a challenge, as the image may contain areas with fine textures which, if treated as noise, may lose some of their definition. </p> <h3>Image sharpening</h3> <p>As the color and brightness values for each pixel are <a href="/facts/Interpolated/8PyetX4f">interpolated</a> some <a href="/facts/Image_sharpening/l5LvzVBa">image sharpening</a> is applied to even out any fuzziness that has occurred. To preserve the impression of <a href="/facts/Depth_perception/R6FK2z6o">depth</a>, clarity and fine details, the image processor must sharpen edges and contours. It therefore must <a href="/facts/Edge_detection/9VplpIcS">detect edges</a> correctly and reproduce them smoothly and without over-sharpening. </p> <h2 id="models">Models</h2> <p>Image processor users are using industry standard products, application-specific standard products (ASSP) or even <a href="/facts/Application-specific_integrated_circuit/YGUXpX50">application-specific integrated circuits</a> (ASIC) with trade names: Canon's is called <a href="/facts/DIGIC/KCnwQKNY">DIGIC</a>, Nikon's <a href="/facts/Expeed/7HUMwSqm">Expeed</a>, Olympus' TruePic, Panasonic's <a href="/facts/Venus_Engine/hQWIDYoX">Venus Engine</a> and Sony's <a href="/facts/Bionz/vVaYDYey">Bionz</a>. Some are known to be based on the <a href="/facts/Fujitsu/6e5gvNSa">Fujitsu</a> <a href="/facts/Milbeaut/h6l1SuFI">Milbeaut</a>, the <a href="/facts/Texas_Instruments/88NUhuHF">Texas Instruments</a> <a href="/facts/OMAP/3oz3mLlT">OMAP</a>, <a href="/facts/Panasonic/TqkBQmgj">Panasonic</a> <a href="/facts/MN103/f7Po8QCr">MN103</a>, <a href="/facts/Zoran_Corporation/B118jFQ0">Zoran</a> Coach, Altek Sunny or <a href="/facts/Sanyo/kk7BSfKF">Sanyo</a> image/video processors. </p><p><a href="/facts/ARM_architecture/SALsYA79">ARM architecture</a> processors with its <a href="/facts/ARM_NEON/SALsYA79">NEON SIMD</a> <a href="/facts/Media_Processing_Engine/WYCcoTSt">Media Processing Engines</a> (MPE) are often used in <a href="/facts/Mobile_phone/H3EcqD9h">mobile phones</a>. </p> <h3>Processor brand names</h3> <ul><li>ATI - <a href="/facts/Imageon/RbWr5jRy">Imageon</a> (graphics co-processor used in many early mobile photos to offer camera image signal processing<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a>)</li> <li>Canon - <a href="/facts/DIGIC/KCnwQKNY">DIGIC</a> (based on Texas Instruments <a href="/facts/OMAP/3oz3mLlT">OMAP</a>)<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a></li> <li>Casio - EXILIM engine</li> <li>Epson - EDiART</li> <li>Fujifilm - EXR III or X Processor Pro</li> <li>Google - <a href="/facts/Pixel_Visual_Core/lsI1GocM">Pixel Visual Core</a><a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a></li> <li>HTC - ImageSense</li> <li>Intel - IPU<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a></li> <li>MediaTek - Imagiq</li> <li>Minolta / Konica Minolta - <a href="/facts/SUPHEED/kXp2j6xQ">SUPHEED</a> with CxProcess</li> <li>Leica - MAESTRO (based on Fujitsu <a href="/facts/Milbeaut/h6l1SuFI">Milbeaut</a>)<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a></li> <li>Nikon - <a href="/facts/Expeed/7HUMwSqm">Expeed</a> (based on Fujitsu <a href="/facts/Milbeaut/h6l1SuFI">Milbeaut</a>)<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a></li> <li>Olympus - TruePic (based on Panasonic <a href="/facts/MN103/f7Po8QCr">MN103</a>/MN103S)</li> <li>OPPO - MariSilicon X</li> <li>Panasonic - <a href="/facts/Venus_Engine/hQWIDYoX">Venus Engine</a> (based on Panasonic <a href="/facts/MN103/f7Po8QCr">MN103</a>/MN103S)</li> <li>Pentax - PRIME (Pentax Real IMage Engine) (newer variants based on Fujitsu <a href="/facts/Milbeaut/h6l1SuFI">Milbeaut</a>)</li> <li>Qualcomm - <a href="/facts/Qualcomm_Spectra/v54S3UgA">Qualcomm Spectra</a> (based on <a href="/facts/Qualcomm_Snapdragon/FWJM1F0x">Qualcomm Snapdragon</a>)</li> <li>Ricoh - GR engine (GR digital), Smooth Imaging Engine</li> <li>Samsung - DRIMe (based on <a href="/facts/Samsung_Electronics/STxnmHwl">Samsung</a> <a href="/facts/Exynos/VSMI3vDf">Exynos</a>)</li> <li>Sanyo - Platinum engine</li> <li>Sigma - True</li> <li>Sharp - ProPix</li> <li>Socionext - <a href="http://socionextus.com/products/image-processors-milbeaut/">Milbeaut</a> Family of ISPs - SC2000 (M-10V), SC2002 (M-11S)</li> <li>Sony - <a href="/facts/Bionz/vVaYDYey">Bionz</a></li> <li>THine - THP series <a href="https://www.thine.co.jp/en/products/camera_processor/Image-Signal-Processor/">[1]</a> with compatible SDK Kit for developing firmware <a href="https://www.thinesolutions.com/camera-development-kit-cdk">[2]</a></li> <li>UNISOC - Vivimagic</li></ul> <h3>Speed</h3> <p>With the ever-higher pixel count in image sensors, the image processor's speed becomes more critical: <a href="/facts/Photographer/rmSPgLfE">photographers</a> don't want to wait for the camera's image processor to complete its job before they can carry on shooting - they don't even want to notice some processing is going on inside the camera. Therefore, image processors must be optimised to cope with more data in the same or even a shorter period of time. </p> <h2 id="software">Software</h2> <p><a href="/facts/Libcamera/xtpPJdk5">libcamera</a> is a software library that supports using image signal processors for the capture of pictures. </p> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Color_image_pipeline/ggCgVWLa">Color image pipeline</a></li> <li><a href="/facts/Image_processing/zenalNCf">Image processing</a></li> <li><a href="/facts/Digital_image_processing/zenalNCf">Digital image processing</a></li> <li><a href="/facts/Digital_image_editing/l5LvzVBa">Digital image editing</a></li> <li><a href="/facts/Demosaicing/sbtIMmN6">Demosaicing</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>DIGITAL SIGNAL & IMAGE PROCESSING <a href="http://www.robots.ox.ac.uk/~sjrob/Teaching/B4_SP/b4_sp.pdf" target="_blank">http://www.robots.ox.ac.uk/~sjrob/Teaching/B4_SP/b4_sp.pdf</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Fundamentals of digital image processing <a href="http://homepages.inf.ed.ac.uk/rbf/BOOKS/VERNON/Chap004.pdf" target="_blank">http://homepages.inf.ed.ac.uk/rbf/BOOKS/VERNON/Chap004.pdf</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Merigot, Alain; Petrosino, Alfredo (2008-12-01). "Parallel processing for image and video processing: Issues and challenges". Parallel Computing. 34 (12): 694–699. doi:10.1016/j.parco.2008.09.009. ISSN 0167-8191. <a href="https://www.sciencedirect.com/science/article/abs/pii/S0167819108000999" target="_blank">https://www.sciencedirect.com/science/article/abs/pii/S0167819108000999</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Chang, Lanlan (Mar 2006). "Hybrid color filter array demosaicking for effective artifact suppression" (PDF). Journal of Electronic Imaging. 15 (1): 013003-1 – 013003-17. Bibcode:2006JEI....15a3003C. doi:10.1117/1.2183325. Archived from the original (PDF) on 29 December 2009. <a href="https://web.archive.org/web/20091229063407/http://www3.ntu.edu.sg/home5/CHAN0069/JEI013003.pdf" target="_blank">https://web.archive.org/web/20091229063407/http://www3.ntu.edu.sg/home5/CHAN0069/JEI013003.pdf</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>"Handheld Products". 11 March 2006. Archived from the original on 11 March 2006. Retrieved 14 September 2019. <a href="https://web.archive.org/web/20060311005523/http://www.ati.com/products/handheld.html" target="_blank">https://web.archive.org/web/20060311005523/http://www.ati.com/products/handheld.html</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Inside the Canon Rebel T4i DSLR Archived 2012-09-21 at the Wayback Machine Chipworks <a href="http://www.chipworks.com/blog/recentteardowns/2012/07/06/inside-the-canon-rebel-t4i-dslr/" target="_blank">http://www.chipworks.com/blog/recentteardowns/2012/07/06/inside-the-canon-rebel-t4i-dslr/</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Amadeo, Ron (17 October 2017). "Surprise! The Pixel 2 is hiding a custom Google SoC for image processing". Ars Technica. Retrieved 19 October 2017. <a href="https://arstechnica.com/gadgets/2017/10/the-pixel-2-contains-a-custom-google-soc-the-pixel-visual-core/" target="_blank">https://arstechnica.com/gadgets/2017/10/the-pixel-2-contains-a-custom-google-soc-the-pixel-visual-core/</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>"7.8. Intel Image Processing Unit 6 (IPU6) Input System driver — The Linux Kernel documentation". docs.kernel.org. Retrieved 2024-08-30. <a href="https://docs.kernel.org/admin-guide/media/ipu6-isys.html" target="_blank">https://docs.kernel.org/admin-guide/media/ipu6-isys.html</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>Fujitsu Microelectronics-Leica's Image Processing System Solution For High-End DSLR Archived 2008-10-07 at the Wayback Machine <a href="http://www.fujitsu.com/hk/news/pr/20080926.html" target="_blank">http://www.fujitsu.com/hk/news/pr/20080926.html</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>Milbeaut and EXPEED Archived 2016-05-21 at the Wayback Machine byThom <a href="http://www.dslrbodies.com/newsviews/milbeaut-and-expeed.html" target="_blank">http://www.dslrbodies.com/newsviews/milbeaut-and-expeed.html</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> </ol>