CN108513112B - Image processing method based on color filter array - Google Patents
Image processing method based on color filter array Download PDFInfo
- Publication number
- CN108513112B CN108513112B CN201810260123.7A CN201810260123A CN108513112B CN 108513112 B CN108513112 B CN 108513112B CN 201810260123 A CN201810260123 A CN 201810260123A CN 108513112 B CN108513112 B CN 108513112B
- Authority
- CN
- China
- Prior art keywords
- unit
- pixel
- color filter
- information
- units
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/77—Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Color Television Image Signal Generators (AREA)
Abstract
The invention discloses a color filter array and an image processing method, wherein the color filter array comprises a plurality of color filter arrays with M rows and N columns and a plurality of transparent filter arrays for separating adjacent color filter arrays, wherein M is more than or equal to 3, N is more than or equal to 3, each transparent filter array comprises a plurality of transparent filter units, each color filter array comprises at least one color filter unit with a first spectral response, at least one color filter unit with a second spectral response, at least one color filter unit with a third spectral response and a plurality of transparent filter units, and each transparent filter unit is arranged between the color filter units, so that at least one transparent filter unit is arranged between two color filter units in any direction.
Description
Technical Field
The invention relates to the technical field of image imaging, in particular to a color filter array and an image processing method based on the color filter array.
Background
Digital color images typically use the three primary colors of red, green and blue to represent color, and in a typical image sensor, a Color Filter Array (CFA) is typically used to filter light in order to acquire a color image. Typically, each pixel on the image sensor covers a filter such that each pixel can only receive one of a single color red, a single color green, and a single color blue. The image obtained after CFA filtering is a Raw image, and the Raw image can be a full-color image only after being subjected to a color interpolation process.
There are many types of CFAs, and the most commonly used is a Bayer-type CFA, which is characterized by a green component of 50% and a red and blue component of 25% each. However, since all pixels receive only monochromatic light, the light incident amount of each pixel is reduced to one third, and thus the light sensitivity of each pixel is reduced. Processing images in this way may degrade the resolution of the image, make transitions between different colors in the image unnatural, and may cause slight color anomalies.
Disclosure of Invention
In order to overcome the defects of the prior art, an objective of the present invention is to provide a color filter array, which is beneficial to increase the light-entering amount of an image sensor, thereby improving the resolution of an image.
The second objective of the present invention is to provide an image processing method based on a color filter array, which is beneficial to obtaining an image with high resolution and good color rendition capability.
One of the purposes of the invention is realized by adopting the following technical scheme:
a color filter array is characterized by comprising a plurality of color filter arrays with M rows and N columns and a plurality of transparent filter arrays arranged between the color filter arrays, wherein M is larger than or equal to 3, N is larger than or equal to 3, the adjacent color filter arrays are separated by the transparent filter arrays, each transparent filter array comprises a plurality of transparent filter units, each color filter array comprises at least one color filter unit with a first spectral response, at least one color filter unit with a second spectral response, at least one color filter unit with a third spectral response and a plurality of transparent filter units arranged between the color filter units, so that at least one transparent filter unit is arranged between two color filter units of the color filter array in any direction.
Further, each color filter unit in each color filter array is located at the vertex of the color filter array.
Furthermore, the color filter array is a 3-row 3-column square array, the transparent filter array between the color filter arrays which are adjacent to each other is one row or one column, and the color filter array comprises one color filter unit with the first spectral response, one color filter unit with the second spectral response, two color filter units with the third spectral response and five transparent filter units.
The color filter array is a square array with 4 rows and 4 columns, the transparent filter array between the adjacent color filter arrays is two rows or two columns, and the color filter array comprises one color filter unit with first spectral response, one color filter unit with second spectral response, two color filter units with third spectral response and twelve transparent filter units.
Further, the color filter unit with the first spectral response is a red filter unit, and only red light is allowed to pass through; the color filter unit with the second spectral response is a blue filter unit and only allows blue light to pass through; the color filter unit with the third spectral response is a green filter unit and only allows green light to pass through.
The second purpose of the invention is realized by adopting the following technical scheme:
a method of image processing based on said color filter array, said color filter array being positioned over a pixel array, said pixel array comprising a plurality of pixel cells, said method comprising the steps of:
acquiring initial monochromatic information of the pixel unit corresponding to the color filter unit, and acquiring initial brightness information of the pixel unit corresponding to the transparent filter unit;
dividing the pixel array into a plurality of imaging units, wherein the imaging units at least comprise three pixel units corresponding to color filter units of three different colors, and the pixel unit corresponding to the color filter unit belongs to at least one imaging unit;
calculating color information of the imaging unit, wherein the color information of the imaging unit comprises first monochromatic information, second monochromatic information and third monochromatic information, each monochromatic information is an average value of initial monochromatic information of all pixel units with corresponding colors in the imaging unit, and the color information of the imaging unit is recorded into each pixel unit contained in the imaging unit;
calculating output color information of the pixel unit, wherein when one or more pieces of color information are counted in the pixel unit, the output color information of the pixel unit is an average value of all pieces of color information recorded in the pixel unit, and when no color information is counted in the pixel unit, each piece of single color information of the output color information of the pixel unit is an average value of initial single color information of a plurality of pixel units with corresponding colors nearest to the pixel unit;
calculating output brightness information of the pixel units, wherein the output brightness information of the pixel units corresponding to the transparent filtering units is initial brightness information of the pixel units, and the output brightness information of the pixel units corresponding to the color filtering units is an average value of the brightness information of a plurality of pixel units at the periphery of the pixel units, and the initial brightness information is obtained;
and outputting a color image according to the output color information and the output brightness information of the pixel unit.
Further, adjacent imaging units are partially overlapped.
Further, the imaging unit is a minimum rectangular array or a minimum triangular array including three of the pixel units corresponding to the color filter units of three different colors.
Further, the output brightness information of the pixel unit corresponding to the color filter unit is an average value of the brightness information of four pixel units corresponding to the transparent filter unit.
The output luminance information of the pixel unit corresponding to the color filter unit is an average value of the luminance information of the eight pixel units corresponding to the transparent filter unit around the pixel unit.
Further, the color filter array is a 3-row and 3-column square array, the transparent filter array between the adjacent color filter arrays is one row or one column, the color filter array includes one color filter unit with the first spectral response, one color filter unit with the second spectral response, two color filter units with the third spectral response, and a plurality of transparent filter units, the four color filter units are respectively located at four vertices of the color filter array, the pixel unit corresponding to the color filter unit with the first spectral response is designated as a first pixel unit, the pixel unit corresponding to the color filter unit with the second spectral response is designated as a second pixel unit, and the pixel unit corresponding to the color filter unit with the third spectral response is designated as a third pixel unit, each imaging unit includes one of the first pixel unit, one of the second pixel unit, and one of the third pixel units, The first pixel unit, the second pixel unit and the two third pixel units are respectively positioned at four vertexes of the imaging units, one column is shared among the imaging units adjacent in the row direction, and one row is shared among the imaging units adjacent in the column direction.
Further, the color filter array is a 3-row and 3-column square array, the transparent filter array between the adjacent color filter arrays is one row or one column, the color filter array includes one color filter unit with the first spectral response, one color filter unit with the second spectral response, two color filter units with the third spectral response, and a plurality of transparent filter units, the four color filter units are respectively located at four vertices of the color filter array, the pixel unit corresponding to the color filter unit with the first spectral response is designated as a first pixel unit, the pixel unit corresponding to the color filter unit with the second spectral response is designated as a second pixel unit, and the pixel unit corresponding to the color filter unit with the third spectral response is designated as a third pixel unit, each imaging unit includes one of the first pixel unit, one of the second pixel unit, and one of the third pixel units, A minimum triangular array of the second pixel unit and the third pixel unit, wherein the first pixel unit, the second pixel unit and the third pixel unit are respectively located at three vertexes of the imaging unit.
Furthermore, the color filter units with the first spectral response and the color filter units with the second spectral response are not in the same row or in the same column, a plurality of pixel units on the diagonal line or a column are shared between the imaging units adjacent in the row direction, and a plurality of pixel units on the diagonal line or a row is shared between the imaging units adjacent in the column direction.
Furthermore, the color filter units with the first spectral response and the color filter units with the second spectral response are in the same row, one column is shared among the imaging units adjacent in the row direction, and one row is shared among the imaging units adjacent in the column direction.
Compared with the prior art, the invention has the beneficial effects that:
(1) according to the color filter array, the transparent filtering units are added, and the color filtering units and the transparent filtering units are reasonably arranged, so that the light inlet quantity of the image sensor is increased, and an image with high resolution and good color reduction capability can be obtained in the later image processing;
(2) in the color filter array, each color filter unit is surrounded by a plurality of transparent filter units, so that the light sensitivity of each pixel is improved;
(3) the image processing method provided by the invention has the advantages of high image resolution and good color reduction capability.
Drawings
FIG. 1A is a partial schematic diagram of a preferred embodiment of a color filter array according to the present invention, showing different rows and different columns of red filter elements and blue filter elements;
FIG. 1B is a partial schematic diagram of a preferred embodiment of a color filter array according to the present invention, showing the red filter elements and the blue filter elements in the same row;
FIG. 2A is a partial schematic view of another preferred embodiment of a color filter array according to the present invention, showing different rows and different columns of red filter elements and blue filter elements;
FIG. 2B is a partial schematic view of another preferred embodiment of a color filter array according to the present invention, showing the red filter elements and the blue filter elements in the same row;
FIG. 3A is a schematic diagram of a pixel array corresponding to the color filter array shown in FIG. 1A, in which a rectangular wire frame shows the division of the imaging unit;
3B, 3C, 3D, 3E show the imaging element of the pixel array of FIG. 3A alone;
FIG. 3F is intermediate color information for each pixel cell of the imaging unit shown in FIG. 3B;
FIG. 3G is the output color information of each pixel cell of the imaging unit shown in FIG. 3B;
FIG. 3H shows two rows and two columns of the array with the extreme edges removed when computing the output luminance information;
FIG. 3I shows the output luminance information when the luminance of each pixel cell is calculated in a first manner;
FIG. 3J shows output luminance information when the luminance of each pixel cell is calculated in a second manner;
FIG. 4A is a schematic diagram of a pixel array corresponding to the color filter array shown in FIG. 1A, wherein a triangular wire frame shows the division of the imaging elements;
4B, 4C, 4D, 4E, 4F, 4G, 4H, 4I show the imaging element of the pixel array of FIG. 4A alone;
FIG. 4J is intermediate color information for each pixel cell of the imaging unit shown in FIG. 4B;
FIG. 4K is the output color information of the individual pixel cells of the imaging unit shown in FIG. 4B;
FIG. 5A is a schematic diagram of a pixel array corresponding to the color filter array shown in FIG. 1B, wherein a triangular wire frame shows the division of the imaging elements;
5B, 5C, 5D, 5E, 5F, 5G, 5H, 5I show the imaging element of the pixel array of FIG. 5A alone;
FIG. 5J is intermediate color information for each pixel cell of the imaging unit shown in FIG. 5B;
fig. 5K is output color information of each pixel unit of the image forming unit shown in fig. 5B.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and the detailed description, and it should be noted that any combination of the embodiments or technical features described below can be used to form a new embodiment without conflict.
The invention provides a color filter array, which comprises a plurality of color filter arrays which are tiled, arranged in M rows and N columns, and a plurality of transparent filter arrays which separate the adjacent color filter arrays, wherein M is more than or equal to 3, and N is more than or equal to 3.
Each transparent filter array comprises a plurality of transparent filter units, each color filter array comprises at least one color filter unit with a first spectral response, at least one color filter unit with a second spectral response, at least one color filter unit with a third spectral response and a plurality of transparent filter units, and each transparent filter unit is arranged between the color filter units, so that at least one transparent filter unit is arranged between the two color filter units in any direction.
In other words, any two of the color filter cells in the color filter array are not adjacent in the row direction, the column direction, and the diagonal direction. The color filter array is provided with more than half of transparent filter units, so that the image sensor has good light incoming quantity, and the image resolution can be improved. In addition, the color filter array comprises three color filter units with different spectral responses, so that the image sensor can receive color information, and an image with high resolution and good color restoration capability can be obtained through later-stage image processing.
In a preferred embodiment, the first spectrally responsive color filter element is a red filter element (R), the second spectrally responsive color filter element is a blue filter element (B), and the third spectrally responsive color filter element is a green filter element (G). The red filter unit only allows red light of the monochromatic light to pass through, and the pixel unit corresponding to the red filter unit only receives red information; the blue light filtering unit only allows the blue light of the monochromatic light to pass through, and the pixel unit corresponding to the blue light filtering unit only receives blue information; the green filter unit only allows green light of the monochromatic light to pass through, and the pixel unit corresponding to the green filter unit only receives green information. The transparent filter unit allows full color light to pass through.
[ example 1 ]
As shown in fig. 1A and 1B, the color filter array includes a plurality of 3 rows and 3 columns of color filter arrays 1 arranged in a tiled manner and a transparent filter array 2 separating adjacent color filter arrays 1. Each color filter array comprises a red filter unit (R), a blue filter unit (B), two green filter units (G) and 5 transparent filter units (W), the four color filter units are respectively located at four vertexes of the color filter array, and a transparent filter unit (W) is arranged between the two color filter units in the row direction, the column direction and the diagonal direction of the color filter array. The transparent filter arrays 2 between adjacent color filter units 1 are in a row or a column, that is, the transparent filter arrays 2 are in a row or a column.
It should be noted that the red filter unit (R) and the blue filter unit (B) can be in different rows and different columns, as shown in fig. 1A; or in the same row, as in fig. 1B, or in the same column.
[ example 2]
As shown in fig. 2A and 2B, the color filter array includes a plurality of color filter arrays 1 'arranged in a tiled manner in 4 rows and 4 columns, and a transparent filter array 2' separating each adjacent color filter array 1 ', each color filter array 1' includes a red filter unit (R), a blue filter unit (B), two green filter units (G), and 12 transparent filter units (W), the four color filter units are respectively located at four vertexes of the color filter array 1 ', and two transparent filter units (W) are arranged between two color filter units in the row direction, the column direction, and the diagonal direction of the color filter array 1'. The transparent filtering units 2 ' between the adjacent color filtering units 1 ' are in two rows or two columns, that is, the transparent filtering array 2 ' is two rows or two columns of transparent filtering units.
It should be noted that the red filter unit (R) and the blue filter unit (B) can be in different rows and different columns, as shown in fig. 2A; or in the same row as shown in fig. 2B.
Examples 1 and 2 are not exhaustive lists of the color filter array of the present invention, and various modifications are possible if the above-described limitations are satisfied, and all fall within the scope of the present invention.
The invention also provides an image processing method based on the color filter array, the color filter array is positioned on a pixel array, the pixel array comprises a plurality of pixel units, and the image processing method outputs a color image according to the color and brightness information received by the pixel units. The image processing method comprises the following steps:
acquiring initial monochromatic information of the pixel unit corresponding to the color filter unit, and acquiring initial brightness information of the pixel unit corresponding to the transparent filter unit;
dividing the pixel array into a plurality of imaging units, wherein the imaging units at least comprise three pixel units corresponding to the color filter units with three different colors, and the pixel unit corresponding to the color filter unit belongs to at least one imaging unit;
calculating color information of the imaging unit, wherein the color information of the imaging unit comprises first monochromatic information, second monochromatic information and third monochromatic information, each monochromatic information is an average value of initial monochromatic information of all pixel units with corresponding colors in the imaging unit, and the color information of the imaging unit is recorded into each pixel unit contained in the imaging unit;
calculating output color information of the pixel unit, wherein when one or more pieces of color information are counted in the pixel unit, the output color information of the pixel unit is an average value of all pieces of color information recorded in the pixel unit, and when no color information is counted in the pixel unit, each piece of single color information of the output color information of the pixel unit is an average value of initial single color information of a plurality of pixel units with corresponding colors nearest to the pixel unit;
calculating output brightness information of the pixel units, wherein the output brightness information of the pixel units corresponding to the transparent filtering units is initial brightness information of the pixel units, and the output brightness information of the pixel units corresponding to the color filtering units is an average value of the brightness information of a plurality of pixel units at the periphery of the pixel units, and the initial brightness information is obtained;
and outputting a color image according to the output color information and the output brightness information of the pixel unit.
Preferably, each imaging unit includes the same number of pixel units, and each imaging unit is a rectangular array or a triangular array with the same number of rows and columns.
Increasing the number of imaging units allows more pixel units to be included in the imaging units. One preferred way to add the imaging unit is: partial overlapping between the imaging units is allowed, that is, one or more pixel units can be shared between the imaging units, so that as many pixel units as possible are included in the imaging units.
In addition, in order to improve the accuracy of color reproduction as much as possible, the image forming unit should be as small as possible. Preferably, the imaging unit is a minimum rectangular array or a minimum triangular array including three of the pixel units corresponding to the color filter units of three different colors.
To further describe the image processing method, a pixel cell corresponding to the color filter cell with the first spectral response is denoted as a first pixel cell, a pixel cell corresponding to the color filter cell with the second spectral response is denoted as a second pixel cell, a pixel cell corresponding to the color filter cell with the third spectral response is denoted as a third pixel cell, and a pixel cell corresponding to the transparent filter cell is denoted as a luminance pixel cell. The steps of the image processing method may also be described as:
acquiring initial monochromatic information of the first pixel unit, the second pixel unit and the third pixel unit, and acquiring initial brightness information of the brightness pixel unit;
dividing each pixel unit into a plurality of imaging units, wherein each imaging unit comprises at least one first pixel unit, at least one second pixel unit and at least one third pixel unit, and each first pixel unit, each second pixel unit and each third pixel unit belong to at least one imaging unit;
calculating color information of each imaging unit, wherein the color information of the imaging unit comprises first monochromatic information, second monochromatic information and third monochromatic information, the first monochromatic information is an average value of initial monochromatic information of all the first pixel units in the imaging unit, the second monochromatic information is an average value of initial monochromatic information of all the second pixel units in the imaging unit, the third monochromatic information is an average value of initial monochromatic information of all the third pixel units in the imaging unit, and the color information of the imaging unit is recorded into each pixel unit contained in the imaging unit and is recorded as middle color information of the pixel unit;
calculating output color information of each pixel unit, wherein when one or more pieces of color information are counted in the pixel unit, the output color information of the pixel unit is an average value of all middle color information recorded in the pixel unit, when no middle color information is counted in the pixel unit, the first single color information of the pixel unit is an average value of initial single color information of a plurality of first pixel units nearest to the pixel unit, the second single color information of the pixel unit is an average value of initial single color information of a plurality of second pixel units nearest to the pixel unit, and the third single color information of the pixel unit is an average value of initial single color information of a plurality of third pixel units nearest to the pixel unit;
calculating output brightness information of each pixel unit, wherein the initial brightness information of the brightness pixel unit is the output brightness information of the brightness pixel unit, and the output brightness information of the first pixel unit, the second pixel unit and the third pixel unit is an average value of the brightness information of a plurality of brightness pixel units around the first pixel unit, the second pixel unit and the third pixel unit;
and outputting a color image according to the output color information and the output brightness information of the pixel unit.
Preferably, the imaging unit is a minimum rectangular array or a minimum triangular array including at least one of the first pixel unit, at least one of the second pixel unit, and at least one of the third pixel unit.
[ example 3 ]
There is provided an image processing method based on the color filter array shown in fig. 1A in embodiment 1, in which the color filter array is positioned over a pixel array including a plurality of pixel units. The image processing method comprises the following steps:
s01, marking a pixel unit corresponding to each red filter unit (R) as a first pixel unit Rab, a pixel unit corresponding to each blue filter unit (B) as a second pixel unit Bab, a pixel unit corresponding to each green filter unit (G) as a third pixel unit Gab, and a pixel unit corresponding to each transparent filter unit (W) as a brightness pixel unit Wab, wherein a and B are respectively the number of rows and columns of the pixel unit, as shown in FIG. 3A, acquiring initial monochromatic information of the first pixel unit, the second pixel unit and the third pixel unit, acquiring initial brightness information of the brightness pixel unit, marking initial red light information of R11 as R11, initial green light information of G13 as G13, initial blue light information of B33 as B33, initial brightness information of W12 as Y12, and expressing monochromatic initial information or brightness information of other pixel units in the same way, are not listed one by one here;
s02, dividing each pixel unit into a plurality of imaging units with the same size, wherein each imaging unit is a 3 x 3 square matrix comprising a first pixel unit, a second pixel unit, two third pixel units and four brightness pixel units, the first pixel unit, the second pixel unit and the two third pixel units are respectively positioned at four vertexes of the imaging unit, and two adjacent imaging units share one column or one row; the imaging unit part is divided as shown in fig. 3A, 3B, 3C, 3D, 3E respectively and individually display the imaging units in each thick-line frame, two adjacent imaging units share one column in the row direction, and two adjacent imaging units share one row in the column direction;
s03, calculating each compositionThe color information of the imaging unit comprises first monochromatic information, second monochromatic information and third monochromatic information, wherein the first monochromatic information is an average value of initial red light information of all first pixel units in the imaging unit, the second monochromatic information is an average value of initial blue light information of all second pixel units in the imaging unit, and the third monochromatic information is an average value of initial green light information of all third pixel units in the imaging unit; taking the imaging unit shown in fig. 3B as an example, the way of calculating the color information of the imaging unit is explained: in the imaging unit shown in FIG. 3B, the received initial red light information R11 of the first imaging unit R11, the initial blue light information B33 received by the second imaging unit B33, and the average (G13+ G31)/2 of the initial green light information received by the two third imaging units G13 and G31 are taken to form a complete RGB color information
Then, converting the RGB color information into YUV information, taking the color information UV in the YUV information, and recording the color information of the imaging unit shown in FIG. 3B as UV 11; similarly, the color information of the imaging unit shown in fig. 3C is UV12, the color information of the imaging unit shown in fig. 3D is UV21, and the color information of the imaging unit shown in fig. 3E is UV 22; recording color information of an imaging unit into each pixel unit contained in the imaging unit, recording the color information as intermediate color information of the pixel unit, recording a plurality of intermediate color information into a part of pixel units, and displaying the intermediate color information recorded into the pixel unit of fig. 3B in fig. 3F, wherein the intermediate color information of other pixel units is not listed one by one; the conversion formula for converting RGB information into YUV information is as follows:
Y＝0.299*R+0.587*G+0.114*B；
U＝-0.169*R-0.331*G+0.5*B+128；
V＝0.5*R-0.419*G-0.081*B+128；
s04, calculating output color information of each pixel unit: the output color information of the pixel unit is the average value of all the intermediate color information recorded in the pixel unit, fig. 3G shows the color information output by each pixel unit in fig. 3B, and the output color information of other pixel units are not listed one by one in the same way;
s05, calculating output brightness information of each pixel unit: the initial brightness information Y of the brightness pixel unit is the output brightness information of the brightness pixel unit, and the output brightness information of the first pixel unit, the second pixel unit and the third pixel unit is the average value of the initial brightness information of four or eight brightness pixel units around the first pixel unit, the second pixel unit and the third pixel unit; taking fig. 3H as an example, when calculating the luminance information, the pixels in two rows and two columns at the edge are discarded and not output, that is, the luminance of the pixel unit in the black frame shown in fig. 3H is calculated;
the output brightness information of the first, second and third pixel units is the average value of the initial brightness information of the four brightness pixel units, namely the upper, lower, left and right brightness pixel units, and fig. 3I is the output brightness information of each pixel unit;
alternatively, the output luminance information of the first, second and third pixel units is an average value of the initial luminance information of eight luminance pixel units surrounding the first, second and third pixel units, and fig. 3J is the output luminance information of each pixel unit;
and finally, converting the output color information and the output brightness information of each pixel unit into RGB information and outputting an image.
[ example 4 ]
There is provided an image processing method based on the color filter array shown in fig. 1A in embodiment 1, in which the color filter array is positioned over a pixel array including a plurality of pixel units. The image processing method comprises the following steps:
s01, marking a pixel unit corresponding to each red filter unit (R) as a first pixel unit Rab, a pixel unit corresponding to each blue filter unit (B) as a second pixel unit Bab, a pixel unit corresponding to each green filter unit (G) as a third pixel unit Gab, and a pixel unit corresponding to each transparent filter unit (W) as a brightness pixel unit Wab, wherein a and B are respectively the number of rows and columns of the pixel unit, as shown in FIG. 3A, acquiring initial monochromatic information of the first pixel unit, the second pixel unit and the third pixel unit, acquiring initial brightness information of the brightness pixel unit, marking initial red light information of R11 as R11, initial green light information of G13 as G13, initial blue light information of B33 as B33, initial brightness information of W12 as Y12, and expressing monochromatic initial information or brightness information of other pixel units in the same way, are not listed one by one here;
s02, dividing each pixel unit into a plurality of imaging units with the same size, wherein each imaging unit is a triangular matrix comprising a first pixel unit, a second pixel unit, a third pixel unit and three brightness pixel units, the first pixel unit, the second pixel unit and the third pixel unit are respectively positioned at three vertexes of the imaging unit, a plurality of pixel units on a diagonal line or a column are shared among the imaging units adjacent in a row direction, and a plurality of pixel units on the diagonal line or a row is shared among the imaging units adjacent in the column direction; division of imaging unit portion as shown in fig. 4A, 4B, 4C, 4D, 4E, 4F, 4G, 4H, 4I each show an imaging unit within each wire frame individually;
s03, calculating color information of each imaging unit, wherein the color information of the imaging unit comprises first monochromatic information, second monochromatic information and third monochromatic information, the first monochromatic information is an average value of initial red light information of all first pixel units in the imaging unit, the second monochromatic information is an average value of initial blue light information of all second pixel units in the imaging unit, and the third monochromatic information is an average value of initial green light information of all third pixel units in the imaging unit; taking the imaging unit shown in fig. 4B as an example, the manner of calculating the color information of the imaging unit will be described: in the imaging unit shown in fig. 4B, the received initial red light information R11 of the first imaging unit R11, the initial blue light information B33 received by the second imaging unit B33, and the initial green light information G31 received by the third imaging unit G31 are taken to form a complete RGB color information (R11G 31B 33), and then the RGB color information is converted into YUV information, the color information UV is taken, and the color information of the imaging unit shown in fig. 4B is recorded as UV 11'. Similarly, let us say that the color information of the imaging unit shown in fig. 4C is UV12 ', let us say that the color information of the imaging unit shown in fig. 4D is UV13 ', let us say that the color information of the imaging unit shown in fig. 4E is UV14 ', let us say that the color information of the imaging unit shown in fig. 4F is UV21 ', let us say that the color information of the imaging unit shown in fig. 4G is UV22 ', let us say that the color information of the imaging unit shown in fig. 4H is UV23 ', and let us say that the color information of the imaging unit shown in fig. 4I is UV24 '; recording color information of an imaging unit into each pixel unit contained in the imaging unit, recording the color information as intermediate color information of the pixel unit, recording a plurality of pieces of intermediate color information into a part of the pixel units, and displaying the intermediate color information recorded into the pixel unit of fig. 4B in fig. 4J, wherein the intermediate color information of other pixel units is not listed one by one;
s04, calculating output color information of each pixel unit: the output color information of the pixel unit is the average value of all the intermediate color information recorded in the pixel unit, fig. 4K shows the output color information of the pixel unit of fig. 4B, and the output color information of other pixel units are the same and are not listed one by one;
s05, calculating output brightness information of each pixel unit, wherein the initial brightness information Y of the brightness pixel unit is the output brightness information of the brightness pixel unit, and the output brightness information of the first pixel unit, the second pixel unit and the third pixel unit is the average value of the initial brightness information of four or eight brightness pixel units around the first pixel unit, the second pixel unit and the third pixel unit; the brightness information is calculated as in example 3, and is not described in detail herein;
and finally, converting the output color information and the output brightness information of each pixel unit into RGB information and outputting an image.
[ example 5 ]
There is provided an image processing method based on the color filter array shown in fig. 1B in embodiment 1, in which the color filter array is positioned over a pixel array including a plurality of pixel units. The image processing method comprises the following steps:
s01, marking a pixel unit corresponding to each red filter unit (R) as a first pixel unit Rab, a pixel unit corresponding to each blue filter unit (B) as a second pixel unit Bab, a pixel unit corresponding to each green filter unit (G) as a third pixel unit Gab, and a pixel unit corresponding to each transparent filter unit (W) as a brightness pixel unit Wab, wherein a and B are respectively the number of rows and columns of the pixel unit, as shown in FIG. 5A, acquiring initial color information of the first pixel unit, the second pixel unit and the third pixel unit, acquiring initial brightness information of the brightness pixel unit, marking initial red light information of R11 as R11, initial green light information of G13 as G13, initial blue light information of B33 as B33, initial brightness information of W12 as Y12, and expressing monochrome initial information or brightness information of other pixel units in the same way, are not listed one by one here;
s02, dividing each pixel unit into a plurality of imaging units with the same size, wherein each imaging unit is a triangular matrix comprising a first pixel unit, a second pixel unit, a third pixel unit and three brightness pixel units, the first pixel unit, the second pixel unit and the third pixel unit are respectively positioned at three vertexes of the imaging unit, a column is shared among the imaging units adjacent in the row direction, and a row is shared among the imaging units adjacent in the column direction; division of imaging unit portion as shown in fig. 5A, 5B, 5C, 5D, 5E, 5F, 5G, 5H, 5I each show an imaging unit within each wire frame individually;
s03, calculating color information of each imaging unit, wherein the color information of the imaging unit comprises first monochromatic information, second monochromatic information and third monochromatic information, the first monochromatic information is an average value of initial red light information of all first pixel units in the imaging unit, the second monochromatic information is an average value of initial blue light information of all second pixel units in the imaging unit, and the third monochromatic information is an average value of initial green light information of all third pixel units in the imaging unit; taking the imaging unit shown in fig. 5B as an example, the manner of calculating the color information of the imaging unit will be described: in the imaging unit shown in fig. 5B, the red monochromatic information R11 received by the first imaging unit R11, the blue monochromatic information B13 received by the second imaging unit B13, and the green monochromatic information G31 received by the third imaging unit G31 are taken to form a complete RGB color information (R11G 31B 12), and then the RGB color information is converted into YUV information, and the color information UV therein is taken, and the color information of the imaging unit shown in fig. 5B is recorded as UV11 ". Similarly, let us say that the color information of the imaging unit shown in fig. 5C is UV12 ", let us say that the color information of the imaging unit shown in fig. 5D is UV 13", let us say that the color information of the imaging unit shown in fig. 5E is UV14 ", let us say that the color information of the imaging unit shown in fig. 5F is UV 21", let us say that the color information of the imaging unit shown in fig. 5G is UV22 ", let us say that the color information of the imaging unit shown in fig. 5H is UV 23", and let us say that the color information of the imaging unit shown in fig. 5I is UV24 "; recording color information of an imaging unit into each pixel unit contained in the imaging unit, recording the color information as intermediate color information of the pixel unit, recording a plurality of intermediate color information into a part of pixel units, and displaying the intermediate color information recorded into the pixel unit of fig. 5B in fig. 5J, wherein the intermediate color information of other pixel units is not listed one by one;
s04, calculating output color information of each pixel unit: when one or more pieces of color information are recorded in the pixel unit, the output color information of the pixel unit is an average value of all pieces of intermediate color information recorded in the pixel unit, fig. 5K shows the output color information of the pixel unit of fig. 5B, and the output color information of other pixel units are similarly arranged and are not listed one by one; as shown in fig. 5A, a partial luminance pixel unit does not belong to any imaging unit, when no intermediate color information is counted in the pixel unit, the output red light information of the pixel unit is an average value of the initial red light information of two first pixel units nearest to the pixel unit, the output blue light information is an average value of the initial blue light information of two second pixel units nearest to the pixel unit, and the output green monochromatic information is an average value of the initial green light information of two third pixel units nearest to the pixel unit; taking the color information of W32 as an example, two first pixel units nearest to the luminance pixel unit W32 are R11 and R51, the output red light information of the luminance pixel unit W32 is (R11+ R51)/2, two second pixel units nearest to the luminance pixel unit W32 are B13 and B53, the output blue light information of the luminance pixel unit W32 is (B13+ B53)/2, two third pixel units nearest to the luminance pixel unit W32 are G31 and G33, the output green light information of the luminance pixel unit W32 is (G31+ G33)/2, and the output RGB color information of the luminance pixel unit W32 is [ (R11+ R51)/2(B13+ B53)/2 (G31+ G33)/2 ];
s05, calculating output brightness information of each pixel unit, wherein the initial brightness information Y of the brightness pixel unit is the output brightness information of the brightness pixel unit, and the output brightness information of the first pixel unit, the second pixel unit and the third pixel unit is the average value of the initial brightness information of four or eight brightness pixel units around the first pixel unit, the second pixel unit and the third pixel unit; the luminance information is calculated as in example 3 and will not be described in detail here.
And finally, converting the output color information and the output brightness information of each pixel unit into RGB information and outputting an image.
The above embodiments are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited thereby, and any insubstantial changes and substitutions made by those skilled in the art based on the present invention are within the protection scope of the present invention.
Claims (9)
1. An image processing method based on a color filter array is characterized in that the color filter array comprises a plurality of color filter arrays with M rows and N columns and a plurality of transparent filter arrays arranged between the color filter arrays, wherein M is more than or equal to 3, N is more than or equal to 3, the adjacent color filter arrays are separated by the transparent filter arrays, each transparent filter array comprises a plurality of transparent filter units, each color filter array comprises at least one color filter unit with a first spectral response, at least one color filter unit with a second spectral response, at least one color filter unit with a third spectral response and a plurality of transparent filter units arranged between the color filter units, so that at least one transparent filter unit is arranged between two color filter units in any direction of the color filter array, the color filter array is positioned over a pixel array, the pixel array including a plurality of pixel cells, the method comprising the steps of:
acquiring initial monochromatic information of the pixel unit corresponding to the color filter unit, and acquiring initial brightness information of the pixel unit corresponding to the transparent filter unit;
dividing the pixel array into a plurality of imaging units, wherein the imaging units at least comprise three pixel units corresponding to color filter units of three different colors, and the pixel unit corresponding to the color filter unit belongs to at least one imaging unit;
calculating color information of the imaging unit, wherein the color information of the imaging unit comprises first monochromatic information, second monochromatic information and third monochromatic information, each monochromatic information is an average value of initial monochromatic information of all pixel units with corresponding colors in the imaging unit, and the color information of the imaging unit is recorded into each pixel unit contained in the imaging unit;
calculating output color information of the pixel unit, wherein when one or more pieces of color information are counted in the pixel unit, the output color information of the pixel unit is an average value of all pieces of color information recorded in the pixel unit, and when no color information is counted in the pixel unit, each piece of single color information of the output color information of the pixel unit is an average value of initial single color information of a plurality of pixel units with corresponding colors nearest to the pixel unit;
calculating output brightness information of the pixel units, wherein the output brightness information of the pixel units corresponding to the transparent filtering units is initial brightness information of the pixel units, and the output brightness information of the pixel units corresponding to the color filtering units is an average value of the brightness information of a plurality of pixel units at the periphery of the pixel units, and the initial brightness information is obtained;
and outputting a color image according to the output color information and the output brightness information of the pixel unit.
2. The image processing method according to claim 1, wherein adjacent ones of the imaging units are partially overlapped.
3. The image processing method of claim 1, wherein each of said imaging elements is a smallest rectangular array or a smallest triangular array including three of said pixel elements corresponding to three color filter elements of different colors.
4. The image processing method of claim 1, wherein the output luminance information of the pixel unit corresponding to the color filter unit is an average value of luminance information of four pixel units corresponding to the transparent filter unit, up, down, left, and right.
5. The image processing method according to claim 1, wherein the output luminance information of the pixel unit corresponding to the color filter unit is an average value of luminance information of eight pixel units corresponding to the transparent filter unit around the pixel unit.
6. The image processing method according to claim 1, wherein the color filter array is a 3-row 3-column square array, the transparent filter array between each adjacent color filter array is one row or one column, the color filter array includes one color filter unit with the first spectral response, one color filter unit with the second spectral response, two color filter units with the third spectral response, and a plurality of transparent filter units, the four color filter units are respectively located at four vertexes of the color filter array, a pixel unit corresponding to the color filter unit with the first spectral response is designated as a first pixel unit, a pixel unit corresponding to the color filter unit with the second spectral response is designated as a second pixel unit, and a pixel unit corresponding to the color filter unit with the third spectral response is designated as a third pixel unit, each imaging unit is a minimum rectangular array including one first pixel unit, one second pixel unit and two third pixel units, the first pixel unit, the second pixel unit and the two third pixel units are respectively located at four vertexes of the imaging unit, one column is shared among the imaging units adjacent in the row direction, and one row is shared among the imaging units adjacent in the column direction.
7. The image processing method according to claim 1, wherein the color filter array is a 3-row 3-column square array, the transparent filter array between each adjacent color filter array is one row or one column, the color filter array includes one color filter unit with the first spectral response, one color filter unit with the second spectral response, two color filter units with the third spectral response, and a plurality of transparent filter units, the four color filter units are respectively located at four vertexes of the color filter array, a pixel unit corresponding to the color filter unit with the first spectral response is designated as a first pixel unit, a pixel unit corresponding to the color filter unit with the second spectral response is designated as a second pixel unit, and a pixel unit corresponding to the color filter unit with the third spectral response is designated as a third pixel unit, each imaging unit is a minimum triangular array including one first pixel unit, one second pixel unit and one third pixel unit, and the first pixel unit, the second pixel unit and the third pixel unit are respectively located at three vertexes of the imaging unit.
8. The image processing method according to claim 7, wherein the color filter elements of the first spectral response and the color filter elements of the second spectral response are not in the same row and not in the same column, and share a plurality of pixel elements on a diagonal or share a column between the imaging elements adjacent in a row direction, and share a plurality of pixel elements on a diagonal or share a row between the imaging elements adjacent in a column direction.
9. The image processing method of claim 7, wherein said first spectrally responsive color filter elements are in the same row as said second spectrally responsive color filter elements, sharing a column between said imaging elements adjacent in the row direction, sharing a row between said imaging elements adjacent in the column direction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810260123.7A CN108513112B (en) | 2018-03-27 | 2018-03-27 | Image processing method based on color filter array |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810260123.7A CN108513112B (en) | 2018-03-27 | 2018-03-27 | Image processing method based on color filter array |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108513112A CN108513112A (en) | 2018-09-07 |
| CN108513112B true CN108513112B (en) | 2020-12-08 |
Family
ID=63378828
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810260123.7A Active CN108513112B (en) | 2018-03-27 | 2018-03-27 | Image processing method based on color filter array |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108513112B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109905681B (en) * | 2019-02-01 | 2021-07-16 | 华为技术有限公司 | Image sensor, method of acquiring image data therefrom, and image pickup apparatus |
| WO2020237591A1 (en) * | 2019-05-30 | 2020-12-03 | 深圳市大疆创新科技有限公司 | Color filter array, image sensor, and photographing device |
| CN117880653A (en) * | 2021-12-22 | 2024-04-12 | 荣耀终端有限公司 | Multispectral sensor and electronic device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150731A (en) * | 2006-09-22 | 2008-03-26 | 罗罡 | Color filtering array for digital imaging and its imaging method |
| CN102957917A (en) * | 2011-08-30 | 2013-03-06 | 比亚迪股份有限公司 | Pixel array, camera and array-based color processing method |
| CN105430361A (en) * | 2015-12-18 | 2016-03-23 | 广东欧珀移动通信有限公司 | Imaging method, image sensor, imaging device and electronic device |
| CN106067935A (en) * | 2015-04-15 | 2016-11-02 | 佳能株式会社 | Image pick-up device, image picking system and signal processing method |
| WO2017034076A1 (en) * | 2015-08-27 | 2017-03-02 | (주)픽셀플러스 | Image sensor, digital image processing apparatus using same, and image signal processing method therefor |
-
2018
- 2018-03-27 CN CN201810260123.7A patent/CN108513112B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150731A (en) * | 2006-09-22 | 2008-03-26 | 罗罡 | Color filtering array for digital imaging and its imaging method |
| CN102957917A (en) * | 2011-08-30 | 2013-03-06 | 比亚迪股份有限公司 | Pixel array, camera and array-based color processing method |
| CN106067935A (en) * | 2015-04-15 | 2016-11-02 | 佳能株式会社 | Image pick-up device, image picking system and signal processing method |
| WO2017034076A1 (en) * | 2015-08-27 | 2017-03-02 | (주)픽셀플러스 | Image sensor, digital image processing apparatus using same, and image signal processing method therefor |
| CN105430361A (en) * | 2015-12-18 | 2016-03-23 | 广东欧珀移动通信有限公司 | Imaging method, image sensor, imaging device and electronic device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108513112A (en) | 2018-09-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR100982685B1 (en) | System and method for a high performance color filter mosaic array | |
| US7400332B2 (en) | Hexagonal color pixel structure with white pixels | |
| US8704922B2 (en) | Mosaic image processing method | |
| KR101012537B1 (en) | Solid-state image sensor | |
| US8922683B2 (en) | Color imaging element and imaging apparatus | |
| US11258992B2 (en) | Image processing method and filter array including wideband filter elements and narrowband filter elements | |
| US9431444B2 (en) | Single-plate color imaging element including color filters arranged on pixels | |
| US9159758B2 (en) | Color imaging element and imaging device | |
| US8643742B2 (en) | Crosstalk filter in a digital image processing pipeline | |
| CN104412581B (en) | Color image sensor and camera head | |
| US9036061B2 (en) | Color imaging apparatus | |
| US9324749B2 (en) | Color imaging element and imaging device | |
| CN108513112B (en) | Image processing method based on color filter array | |
| CN104412580A (en) | Colour imaging element and imaging device | |
| US9188480B2 (en) | Color filter array and image sensor | |
| CN105826335A (en) | Solid-state image pickup device | |
| JP4119566B2 (en) | Color imaging device and color imaging device | |
| CN113141488B (en) | Color restoration method of small-size pixel image sensor | |
| JP4119565B2 (en) | Color imaging device and color imaging device | |
| Lai et al. | Color Interpolation with Full Resolution for Hybrid RGB-IR CMOS Sensor | |
| CN104025575A (en) | Imaging device, control method for imaging device, and control program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |