imported>OAbot: Open access bot: url-access updated in citation with #oabot.

2025-05-23T20:11:14Z

Open access bot: url-access updated in citation with #oabot.

New page

{{short description|Specialized digital signal processor used for image processing}}
[[File:Nikon D90 - board 0 - Nikon Expeed EI-149-1769.jpg|thumb|[[EXPEED|Nikon EXPEED]], a [[system on a chip]] including an [[Color image pipeline|image processor]], [[Video_scaler#Video_processor|video processor]], [[digital signal processor]] (DSP) and a [[32-bit]] [[microcontroller]] controlling the chip]]
An '''image processor''', also known as an '''image processing engine''', '''image processing unit''' ('''IPU'''), or '''image signal processor''' ('''ISP'''), is a type of [[media processor]] or specialized [[digital signal processor]] (DSP) used for [[image processing]], in [[digital camera]]s or other devices.<ref>[http://www.robots.ox.ac.uk/~sjrob/Teaching/B4_SP/b4_sp.pdf DIGITAL SIGNAL & IMAGE PROCESSING]</ref><ref>[http://homepages.inf.ed.ac.uk/rbf/BOOKS/VERNON/Chap004.pdf Fundamentals of digital image processing]</ref>
Image processors often employ [[parallel computing]] even with [[Single instruction, multiple data|SIMD]] or [[Multiple instruction, multiple data|MIMD]] technologies to increase speed and efficiency.<ref>{{Cite journal |last1=Merigot |first1=Alain |last2=Petrosino |first2=Alfredo |date=2008-12-01 |title=Parallel processing for image and video processing: Issues and challenges |url=https://www.sciencedirect.com/science/article/abs/pii/S0167819108000999 |journal=Parallel Computing |volume=34 |issue=12 |pages=694–699 |doi=10.1016/j.parco.2008.09.009 |issn=0167-8191|url-access=subscription }}</ref> The [[digital image]] processing engine can perform a range of tasks.
To increase the system integration on [[Embedded system|embedded devices]], often it is a [[system on a chip]] with [[multi-core processor]] architecture.

==Function==
{{Unreferenced section|date=October 2017}}
=== Bayer transformation ===
The [[photodiode]]s employed in an [[image sensor]] are color-blind by nature: they can only record [[Shades of gray|shades of grey]]. To get [[color]] into the picture, they are covered with different color filters: [[red]], [[green]] and [[blue]] ([[RGB]]) according to the pattern designated by the [[Bayer filter]]. As each photodiode records the color information for exactly one [[pixel]] of the image, without an image processor there would be a green pixel next to each red and blue pixel.

This process, however, is quite complex, and involves a number of different operations. Its quality depends largely on the effectiveness of the [[Algorithm|algorithms]] applied to the raw data coming from the sensor. The mathematically manipulated data becomes the recorded photo file.

=== Demosaicing ===
As stated above, the image processor evaluates the color and [[brightness]] data of a given pixel, compares them with the data from neighboring pixels, and then uses a [[demosaicing]] algorithm to produce an appropriate color and brightness value for the pixel.<ref>{{Cite journal |last=Chang |first=Lanlan |date=Mar 2006 |title=Hybrid color filter array demosaicking for effective artifact suppression |url=http://www3.ntu.edu.sg/home5/CHAN0069/JEI013003.pdf |journal=Journal of Electronic Imaging |volume=15 |issue=1 |pages=013003-1 - 013003-17|doi=10.1117/1.2183325 |bibcode=2006JEI....15a3003C |archive-url=https://web.archive.org/web/20091229063407/http://www3.ntu.edu.sg/home5/CHAN0069/JEI013003.pdf |archive-date=29 December 2009 }}</ref> The image processor also assesses the whole picture to guess at the correct distribution of [[Contrast (vision)|contrast]]. By adjusting the [[Gamma correction|gamma]] value (heightening or lowering the contrast range of an image's mid-tones), subtle tonal gradations, such as in [[human skin]] or the blue of the [[sky]], become much more realistic.

=== Noise reduction ===
[[Image noise|Noise]] is a phenomenon found in any [[Electronic circuit|electronic circuitry]]. In [[digital photography]] its effect is often visible as random spots of obviously wrong color in an otherwise smoothly-colored area. Noise increases with temperature and [[Exposure (photography)|exposure]] times. When higher [[ISO 5800:1987#Digital camera ISO speed and exposure index|ISO]] settings are chosen the electronic signal in the image sensor is amplified, which at the same time increases the noise level, leading to a lower [[signal-to-noise ratio]]. The image processor attempts to separate the noise from the image information and to remove it. This can be quite a challenge, as the image may contain areas with fine textures which, if treated as noise, may lose some of their definition.

=== Image sharpening ===
As the color and brightness values for each pixel are [[interpolated]] some [[image sharpening]] is applied to even out any fuzziness that has occurred. To preserve the impression of [[Depth perception|depth]], clarity and fine details, the image processor must sharpen edges and contours. It therefore must [[Edge detection|detect edges]] correctly and reproduce them smoothly and without over-sharpening.

==Models==
Image processor users are using industry standard products, application-specific standard products (ASSP) or even [[application-specific integrated circuit]]s (ASIC) with trade names: Canon's is called [[DIGIC]], Nikon's [[Expeed]], Olympus' TruePic, Panasonic's [[Venus Engine]] and Sony's [[Bionz]]. Some are known to be based on the [[Fujitsu]] [[Milbeaut]], the [[Texas Instruments]] [[OMAP]], [[Panasonic]] [[MN103]], [[Zoran Corporation|Zoran]] Coach, Altek Sunny or [[Sanyo]] image/video processors.

[[ARM architecture]] processors with its [[ARM NEON|NEON SIMD]] [[Media Processing Engine]]s (MPE) are often used in [[mobile phone]]s.

===Processor brand names===
*ATI - [[Imageon]] (graphics co-processor used in many early mobile photos to offer camera image signal processing<ref>{{cite web |title=Handheld Products |url=http://www.ati.com/products/handheld.html |archive-url=https://web.archive.org/web/20060311005523/http://www.ati.com/products/handheld.html |url-status=dead |archive-date=11 March 2006 |access-date=14 September 2019 |date=11 March 2006}}</ref>)
*Canon - [[DIGIC]] (based on Texas Instruments [[OMAP]])<ref>[http://www.chipworks.com/blog/recentteardowns/2012/07/06/inside-the-canon-rebel-t4i-dslr/ Inside the Canon Rebel T4i DSLR] {{webarchive|url=https://web.archive.org/web/20120921024804/http://www.chipworks.com/blog/recentteardowns/2012/07/06/inside-the-canon-rebel-t4i-dslr/ |date=2012-09-21 }} Chipworks</ref>
*Casio - EXILIM engine
*Epson - EDiART
*Fujifilm - EXR III or X Processor Pro
*Google - [[Pixel Visual Core]]<ref>{{cite news|last1=Amadeo|first1=Ron|title=Surprise! The Pixel 2 is hiding a custom Google SoC for image processing|url=https://arstechnica.com/gadgets/2017/10/the-pixel-2-contains-a-custom-google-soc-the-pixel-visual-core/|access-date=19 October 2017|publisher=Ars Technica|date=17 October 2017}}</ref>
*HTC - ImageSense
*Intel - IPU<ref>{{Cite web |title=7.8. Intel Image Processing Unit 6 (IPU6) Input System driver — The Linux Kernel documentation |url=https://docs.kernel.org/admin-guide/media/ipu6-isys.html |access-date=2024-08-30 |website=docs.kernel.org}}</ref>
*MediaTek - Imagiq
*Minolta / Konica Minolta - [[SUPHEED]] with CxProcess
*Leica - MAESTRO (based on Fujitsu [[Milbeaut]])<ref>[http://www.fujitsu.com/hk/news/pr/20080926.html Fujitsu Microelectronics-Leica's Image Processing System Solution For High-End DSLR] {{Webarchive|url=https://web.archive.org/web/20081007190028/http://www.fujitsu.com/hk/news/pr/20080926.html |date=208-10-04 }}</ref>
*Nikon - [[Expeed]] (based on Fujitsu [[Milbeaut]])<ref>[http://www.dslrbodies.com/newsviews/milbeaut-and-expeed.html Milbeaut and EXPEED] {{Webarchive|url=https://web.archive.org/web/20160521154952/http://www.dslrbodies.com/newsviews/milbeaut-and-expeed.html |date=2016-05-21 }} byThom</ref>
*Olympus - TruePic (based on Panasonic [[MN103]]/MN103S)
*OPPO - MariSilicon X
*Panasonic - [[Venus Engine]] (based on Panasonic [[MN103]]/MN103S)
*Pentax - PRIME (Pentax Real IMage Engine) (newer variants based on Fujitsu [[Milbeaut]])
*Qualcomm - [[Qualcomm Spectra]] (based on [[Qualcomm Snapdragon]])
*Ricoh - GR engine (GR digital), Smooth Imaging Engine
*Samsung - DRIMe (based on [[Samsung Electronics|Samsung]] [[Exynos]])
*Sanyo - Platinum engine
*Sigma - True
*Sharp - ProPix
*Socionext - [http://socionextus.com/products/image-processors-milbeaut/ Milbeaut] Family of ISPs - SC2000 (M-10V), SC2002 (M-11S)
*Sony - [[Bionz]]
*THine - THP series [https://www.thine.co.jp/en/products/camera_processor/Image-Signal-Processor/] with compatible SDK Kit for developing firmware [https://www.thinesolutions.com/camera-development-kit-cdk]
*UNISOC - Vivimagic

=== Speed ===
With the ever-higher pixel count in image sensors, the image processor's speed becomes more critical: [[Photographer|photographers]] don't want to wait for the camera's image processor to complete its job before they can carry on shooting - they don't even want to notice some processing is going on inside the camera. Therefore, image processors must be optimised to cope with more data in the same or even a shorter period of time.

== Software ==
[[libcamera]] is a software library that supports using image signal processors for the capture of pictures.

== See also ==
*[[Color image pipeline]]
*[[Image processing]]
*[[Digital image processing]]
*[[Digital image editing]]
*[[Demosaicing]]

==References==
{{reflist}}

{{System on a chip}}

[[Category:Digital signal processors]]
[[Category:Image processors| ]]
[[Category:Photography equipment]]

Image processor - Revision history

imported>OAbot: Open access bot: url-access updated in citation with #oabot.