KR101332689B1 - Image sensor and method for converting output pattern of image sensor - Google Patents

Abstract

An image sensor and a method of converting an output pattern of the image sensor are disclosed. The image sensor may include: a pixel array unit configured to output a plurality of pixels including a panchromatic pixel and a color pixel in a two-dimensional array to output image information of an RGBP pattern; And a pattern converter configured to convert the panchromatic pixel of the RGBP pattern into a pixel having a green (G) color value and output image information of an RGB pattern that is a Bayer pattern. According to the present invention, image processing in the conventional image signal processor and the rear end processor may be facilitated by converting the image information of the RGBP pattern into image information of the RGB pattern, which is a conventional Bayer pattern.

Description

Image sensor and method for converting output pattern of image sensor}

The present invention relates to an image sensor and a method of converting an output pattern of the image sensor.

Digital devices with imaging functions, such as video cameras, digital still cameras, and the like, generally include an image sensor having an array of color filters.

The image sensor includes a plurality of pixels (eg, color pixels having a color value of any one of red (R), green (G) and blue (B)) arranged in a two-dimensional array, on top of the pixels. And generate a color image by forming a color filter array (CFA). CFA patterns of the type commonly used in image sensors are disclosed in US Pat. No. 3,971,065 as Bayer patterns. The Bayer CFA pattern provides each pixel with a color light response that is predominantly sensitive to one of three designated portions of the visible spectrum (eg, red (R), green (G), and blue (B)).

Since an image captured using an image sensor with a Bayer CFA pattern has only one color at each pixel, the missing color values at each pixel are derived from the color values of adjacent pixels to produce a full color image. Interpolated. Interpolation is described in detail in US Pat. No. 5,652,621 and the like, which is obvious to those skilled in the art.

FIG. 1 illustrates a general RGB pattern, and FIG. 2 illustrates a process in which color values of adjacent pixels are used for interpolation processing.

As shown in FIG. 1, the RGB pattern output from the general image sensor is colored in the form of blue (B), green (G), blue (B), and green (G) in the first row and the third row. The values are alternated, and the color values are alternated in the form of green (G), red (R), green (G), and red (R) in the second row and the fourth row.

Further, referring to FIG. 2 in which missing color values at each pixel are interpolated from color values of adjacent pixels to produce a full color image, the G component of the centrally located R pixel is Can be calculated as Those skilled in the art will be able to easily understand the interpolation process for other color values with reference to Equation 1 below.

In addition, U.S. Patent No. 8,139,130 discloses improved CFA patterns including patterns that provide panchromatic light response to some pixels. The panchromatic optical response has a broader spectral sensitivity than the spectral sensitivitys represented in the selected set of color optical responses, and may for example have a high sensitivity over substantially the entire visible spectrum.

3 illustrates a general RGBP pattern. That is, depending on how the CFA pattern is configured, the RGBP pattern may vary from the illustrated patterns A to E and the like.

Image sensors composed of improved CFA patterns exhibit greater light sensitivity and are therefore suitable for applications involving low scene lighting, short exposure times, small apertures, or other restrictions on the amount of light reaching the image sensor. It is very suitable to use.

However, in the case of an RGBP sensor including a panchromatic pixel, which is a pixel for receiving visible and infrared light, a color conversion (Bayer to RGB color conversion) is performed in an image signal processor (ISP) that processes image information of an existing RGB pattern. ), The color of the image is distorted, which requires a separate image signal processor suitable for RGBP pattern processing.

The present invention converts the image information of the RGBP pattern into image information of the RGB pattern, which is a conventional Bayer pattern, and outputs the image pattern of the image sensor and the image sensor to facilitate image processing in the conventional image signal processor and the rear end processor. It is to provide a conversion method.

The present invention is to provide an image sensor and an output pattern conversion method of the image sensor to ensure the general purpose of the RGBP sensor by making it possible to use a general image signal processor for the RGB sensor.

Other objects of the present invention will become readily apparent from the following description.

According to an aspect of the present invention, there is provided an image sensor, comprising: a pixel array unit configured to output a plurality of pixels including a panchromatic pixel and a color pixel in a two-dimensional array to output image information of an RGBP pattern; And a pattern converting unit configured to convert the panchromatic pixel of the RGBP pattern into a pixel having a green (G) color value and output image information of an RGB pattern that is a Bayer pattern.

The image information of the RGBP pattern includes a first column arranged so that color values of green (G) and red (R) alternate in a row direction, and color values of blue (B) and panchromatic (P) rows. And a second column that is alternately arranged in the) direction, and the first column and the second column may be alternately arranged in a column direction.

The pattern converter may convert the panchromatic pixel G (P) to be converted into a pixel having a green color value using Equation G (P) = (G1 + G2 + G3 + G4) / 4. Can be. Here, the G1 to G4 may be green (G) color values of four pixels closest to the panchromatic pixel to be converted.

According to another aspect of the invention, in the output pattern conversion method of the image sensor, the pixel array unit comprising a plurality of pixels including a panchromatic pixel and color pixels in a two-dimensional array outputting the image information of the RGBP pattern ; And a pattern conversion unit converting the panchromatic pixel of the RGBP pattern into a pixel having a green (G) color value and outputting image information of an RGB pattern that is a Bayer pattern. to provide.

The image information of the RGBP pattern includes a first column arranged so that color values of green (G) and red (R) alternate in a row direction, and color values of blue (B) and panchromatic (P) rows. And a second column that is alternately arranged in the) direction, and the first column and the second column may be alternately arranged in a column direction.

The pattern converter may convert the panchromatic pixel G (P) to be converted into a pixel having a green color value using Equation G (P) = (G1 + G2 + G3 + G4) / 4. Can be. Here, the G1 to G4 may be green (G) color values of four pixels closest to the panchromatic pixel to be converted.

Other aspects, features, and advantages will become apparent from the following drawings, claims, and detailed description of the invention.

According to the exemplary embodiment of the present invention, the image information of the RGBP pattern is converted into image information of the RGB pattern, which is a conventional Bayer pattern, and output.

In addition, the general purpose of the RGBP sensor can be secured by allowing the user to use an image signal processor for a general RGB sensor.

1 shows a general RGB pattern.
2 illustrates a process in which color values of adjacent pixels are used for interpolation processing.
3 is a diagram illustrating a general RGBP pattern.
4 is a view schematically showing the configuration of an RGBP sensor according to an embodiment of the present invention.
5 illustrates a method of interpolation processing for panchromatic pixels according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, the terms "comprises" or "having" and the like refer to the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

Also, the terms " part, "" unit," " module, "and the like, which are described in the specification, refer to a unit for processing at least one function or operation and may be implemented by hardware or software or a combination of hardware and software .

In the following description of the present invention with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of the reference numerals, and redundant explanations thereof will be omitted. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

FIG. 4 is a diagram schematically illustrating a configuration of an RGBP sensor according to an embodiment of the present invention, and FIG. 5 is a diagram illustrating an interpolation processing method for panchromatic pixels according to an embodiment of the present invention.

Referring to FIG. 4, the RGBP sensor 410 converts the pixel array unit 430 for outputting image information of the RGBP pattern and the image information of the RGBP pattern into image information of the RGB pattern of the Bayer pattern (RGBP to Bayer RGB color conversion) and outputs a pattern conversion unit 440.

An image sensor, such as an RGBP sensor 410, an RGB sensor, includes a plurality of pixels arranged in a two dimensional array, and is configured to generate a color image by forming a color filter array (CFA) over the pixels.

The RGB sensor, for example, forms one of red (R), green (G), and blue (B) filters on the pixels, respectively, to block light other than the corresponding light. However, since this causes a decrease in the use efficiency of light, the RGBP sensor 420 is an image sensor proposed to enable a clearer image output even in a low light environment by increasing the use efficiency of light. That is, the RGBP sensor 410 includes a plurality of panchromatic pixels that receive all inputs from visible light to infrared light because no filter is formed, and the use of the pixels can increase the efficiency of light use. There is this.

However, when the image information of the RGBP pattern output from the RGBP sensor 410 is processed by the image signal processor 420 which processes the image information of the conventional RGB pattern, the color of the image is distorted. The unit 440 converts the image information of the RGBP pattern into the image information of the RGB pattern of the Bayer pattern and provides the image information to the image signal processor 420.

The image information converted into the Bayer pattern by the pattern converter 440 is processed by a conventional image signal processor (ISP) 420 which processes image information of the RGB pattern. In addition to the image signal processor 420 for receiving and processing image information from an image sensor by a digital device having an imaging function, a multimedia processor for performing processing such as encoding data processed by the image signal processor 420. Naturally, it may be further included, and their configuration and function will be apparent to those skilled in the art, and description thereof will be omitted.

FIG. 5 illustrates a method in which the pattern converter 440 converts image information of an RGBP pattern into image information of an RGB pattern.

5, the first column in which the color values of green (G) and red (R) are alternately arranged in the row direction, and the color values of the blue (B) and panchromatic (P) alternate in the row direction. A case in which a second column is arranged, is arranged such that the first column and the second column are alternately arranged in a column direction. However, in the image information of the RGBP pattern, the first column arranged so that the color values of blue (B) and green (G) alternate in the row direction, and the color values of the panchromatic (P) and red (R) rows. Of course, it may be various, such as including a second row arranged alternately in the) direction.

The pattern converter 440 converts a color value of a panchromatic pixel among the pixels of the RGBP pattern into a green color by converting the color value of the RGB pattern, which is a general Bayer pattern, into image information of the RGB pattern. Output

Here, G1 to G4 represent green (G) color values of four pixels respectively adjacent to the diagonal direction of the panchromatic pixel to be converted.

The color values of the other pixels are used as they are because they satisfy the RGB pattern of the Bayer pattern.

The image signal processor 420 receives RGB image information of the Bayer pattern converted by the pattern converter 440, and determines a RGB component of each pixel by combining or correcting color values of each pixel. Color interpolation processing is performed.

As described above, in the present embodiment, the RGBP sensor 410 converts the image information of the RGBP pattern into image information of the RGB pattern and outputs the image information of the RGB pattern. Since it can be used as it is there is an advantage that can ensure the versatility of the RGBP sensor 410.

Of course, the above-described output pattern conversion method of the image sensor may be performed by an automated procedure in a time-series order by a program embedded in or installed in the image sensor. The codes and code segments that make up the program can be easily deduced by a computer programmer in the field. In addition, the program is stored in a computer readable medium readable by the digital processing apparatus, and is read and executed by the digital processing apparatus to implement the method. The information storage medium includes a magnetic recording medium, an optical recording medium, and a carrier wave medium.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the following claims And changes may be made without departing from the spirit and scope of the invention.

410 RGBP sensor
420: image signal processor
430: pixel array unit
440: pattern conversion unit

Claims (6)

In the image sensor,
A pixel array unit configured to output an image information of an RGBP pattern by forming a plurality of pixels including a panchromatic pixel and a color pixel in a two-dimensional array; And
Bayer pattern by converting the panchromatic pixel of the RGBP pattern into a pixel having a green (G) color value by using pixels having four green (G) color values disposed to be closest to the periphery of the panchromatic pixel to be converted. Bayer pattern) Image sensor including a pattern conversion unit for outputting the image information of the RGB pattern.
The method of claim 1,
Image information of the RGBP pattern,
A first column in which color values of green (G) and red (R) are alternately arranged in a row direction, and a first column in which color values of blue (B) and panchromatic (P) are alternately arranged in a row direction Include two columns,
A first column in which color values of blue (B) and green (G) are alternately arranged in a row direction, and a first column arranged so that color values of panchromatic (P) and red (R) alternate in a row direction Contains two columns,
And the first column and the second column are alternately arranged in a column direction.
3. The method of claim 2,
The pattern converting unit converts a panchromatic pixel (G (P)) to be converted into a pixel having a green (G) color value by using the following equation.

Here, the G1 to G4 are green (G) color values of four pixels closest to the panchromatic pixel to be converted.
In the output pattern conversion method of the image sensor,
Outputting image information of an RGBP pattern by a pixel array unit including a two-dimensional array of a plurality of pixels including a panchromatic pixel and a color pixel; And
The pattern conversion unit converts the panchromatic pixel of the RGBP pattern into a pixel having a green (G) color value by using pixels having four green (G) color values disposed to be closest to the periphery of the panchromatic pixel to be converted. Output image conversion method of the image sensor comprising the step of outputting the image information of the RGB pattern that is the Bayer pattern.
5. The method of claim 4,
Image information of the RGBP pattern,
A first column in which color values of green (G) and red (R) are alternately arranged in a row direction, and a first column in which color values of blue (B) and panchromatic (P) are alternately arranged in a row direction Include two columns,
A first column in which color values of blue (B) and green (G) are alternately arranged in a row direction, and a first column arranged so that color values of panchromatic (P) and red (R) alternate in a row direction Contains two columns,
And the first column and the second column are alternately arranged in a column direction.
The method of claim 5,
And the pattern conversion unit converts a panchromatic pixel (G (P)) to be converted into a pixel having a green (G) color value by using the following equation.

Here, the G1 to G4 are green (G) color values of four pixels closest to the panchromatic pixel to be converted.

Images