KR102326165B1 - Image processing method and display device using the same - Google Patents

Abstract

In the image processing method according to the present invention, primary reconstruction data of a second resolution higher than the first resolution is generated in blocks of a first size from image data of a first resolution by using a first filter, and the second filter is applied. Secondary reconstruction data is generated in units of a local area having a second size smaller than the first size from the primary reconstruction data of the block using the second reconstruction data, and image data input to the display device is compensated in units of local areas by using the second reconstruction data. Accordingly, display defects inherent in the display device may be compensated using the low-resolution compensation data.

Description

Image processing method and display device using the same

The present invention relates to an image processing method and a display device using the same.

As the information society progresses, the demand for a display device for displaying an image is diversifying, and in recent years, various A flat panel display is widely used.

When manufacturing a flat panel display device, much effort is made to reduce the difference in image quality between products and to make the image quality uniform for each pixel within the product, but it is not easy to make the luminance or color uniform for each product and for each pixel, The luminance of a specific location may be high or low, or a specific color may appear more prominent. In addition, during the manufacturing process of the display device, a stain or scratch may occur at a specific location on the screen, which appears as a display defect of the display device.

In consideration of this, the process of obtaining information related to display defects for each product and obtaining compensation data to compensate for it is performed after product manufacturing and before shipment. It goes through a camera compensation process that obtains compensation data based on the image captured by the camera.

The display device stores the compensation data in a memory, performs a compensation operation that adds or subtracts compensation data to or from the image data input from the host, and displays the compensated image data on the screen, so that a display defect unique to the display device is eliminated. Removed images can be displayed.

However, as the resolution of the display device increases, the size of compensation data to be stored in the memory increases, resulting in increased memory capacity and increased product cost.

In addition, the camera compensation process to obtain high-resolution compensation data must be performed for each product, but there is a problem in that the compensation data cannot be obtained properly because the camera compensation process takes a lot of time or moire occurs on the screen shot by the camera. .

In addition, when the compensation data is obtained in a low quality or a low resolution, the compensation data does not properly reflect a display defect unique to the corresponding product, so compensation performance is deteriorated and the screen quality is deteriorated.

The present invention has taken this situation into account, and an object of the present invention is to provide a method and an apparatus for compensating for a display defect inherent in a display device by using compensation data having a small capacity.

Another object of the present invention is to provide a method and apparatus for compensating for a display defect inherent in a display device without increasing resources of the display device.

Another object of the present invention is to provide a method of quickly obtaining compensation data for compensating for a display defect inherent in a display device with a small amount of data.

The image processing method according to an embodiment of the present invention comprises the steps of generating primary reconstructed data of a second resolution higher than the first resolution in units of blocks of a first size from image data of a first resolution by using a first filter; ; generating secondary restoration data in units of a local area having a second size smaller than the first size from primary restoration data of a block by using a second filter; compensating for image data input to a display device in units of a local area using the secondary reconstruction data; and displaying the compensated image data.

In one embodiment, a block may be a set of pixels with a size of n x n (where n is an even natural number), and the local area may be one pixel line included in the block.

In an embodiment, the first filter and the second filter may be implemented as an artificial neural network including a plurality of input nodes, two or more hidden layers including a plurality of nodes, and a plurality of output nodes, and the nodes are connected to each other.

In an embodiment, the artificial neural network may include parameters learned using image data of a first resolution and image data of a second resolution photographed for a plurality of display devices.

In an embodiment, the image data of the first resolution may be stored in the corresponding display device by down-sampling an image obtained by photographing a screen on which a predetermined image is displayed on the corresponding display device in a defocused state.

In an embodiment, the generating of the secondary reconstructed data may include: a first step of extracting data in units of a local area from the primary reconstructed data in units of blocks; a second step of predicting a detailed profile of the extracted local region data using a second filter; and a third step of generating secondary reconstructed data by combining the predicted detailed profile and the extracted local area data, wherein the detailed profile is formed between the first reconstructed data of the local area and the actual data of the corresponding local area. It may correspond to an error value.

In an embodiment, when classifying detailed profiles of a plurality of local regions into a plurality of classes, the second filter outputs a probability value for each class, but a first probability value for a class to which input local region data belongs may be output as a high value, and a probability value for another class may be output as a value lower than the first probability value.

In an embodiment, the third step is to obtain a first value by multiplying the representative data of the corresponding class and the probability value for the corresponding class for each of the plurality of classes and adding the plurality of multiplication results, and adding a predetermined value to the extracted local area data. A second value may be obtained by multiplying a parameter of , and the second reconstructed data for the corresponding local area may be generated by adding the first value and the second value.

An image processing apparatus according to another embodiment of the present invention includes: a memory for storing image data of a first resolution; A first filter is built in, and for each local area of input image data, a block including the corresponding local area is found from the image data, and a second resolution higher than the first resolution is obtained from the image data of the block using the first filter. a global feature restoration unit for generating primary restoration data; The second filter is built-in, the data of the corresponding local area is extracted from the primary reconstruction data of the block, and the second reconstruction data in which the detailed profile of the corresponding local area is reconstructed from the data of the extracted local area using the second filter. a local profile restoration unit for generating; and a compensator for outputting image data compensated by adding or subtracting second reconstructed data to data of a corresponding local area from among input image data, wherein the detailed profile includes the first reconstructed data of the local area and the corresponding local area. It is characterized in that it corresponds to an error value between the actual data of the region.

In an embodiment, the memory may store image data of a first resolution obtained by down-sampling an image obtained by photographing a screen displaying a predetermined image on a corresponding display device in a defocused state.

In an embodiment, the local profile restoration unit includes: a local profile extraction unit for extracting data of a corresponding local area from the primary restoration data of a block; a detailed profile prediction unit for predicting a detailed profile for the local region extracted using the second filter; and a local profile calculator for generating secondary reconstruction data by combining the predicted detailed profile and extracted local region data, wherein the second filter classifies detailed profiles of a plurality of local regions into a plurality of classes When outputting the probability value for each class, the first probability value for the class to which the data of the input local area belongs is output as a high value, and the probability value for the other class is set to a value lower than the first probability value can be printed out.

A display device according to another embodiment of the present invention includes: a display panel including pixels in which data lines and gate lines intersect and are formed in a matrix form; an image processing unit for generating compensation data and compensating and outputting the input image data using the compensation data; a timing controller for generating control signals for displaying image data output by the image processing unit; a data driving circuit configured to convert image data input from the timing controller into data voltages under the control of the timing controller and output the converted image data to data lines; and a gate driving circuit for generating scan pulses synchronized with the data voltage under the control of the timing controller and sequentially supplying them to the gate lines, wherein the image processing unit includes a memory for storing image data of a first resolution ; A first filter is built in, and for each local area of input image data, a block including the corresponding local area is found from the image data, and a second resolution higher than the first resolution is obtained from the image data of the block using the first filter. a global feature restoration unit for generating primary restoration data; The second filter is built-in, the data of the corresponding local area is extracted from the primary reconstruction data of the block, and the second reconstruction data in which the detailed profile of the corresponding local area is reconstructed from the data of the extracted local area using the second filter. a local profile restoration unit for generating; and a compensator for outputting image data compensated by adding or subtracting second reconstructed data to data of a corresponding local area from among input image data, wherein the detailed profile includes the first reconstructed data of the local area and the corresponding local area. It is characterized in that it corresponds to an error value between the actual data of the region.

Accordingly, it is possible to improve compensation performance for compensating for a display defect inherent in a display device while using compensation data having a low resolution or a low capacity.

In addition, it is possible to effectively compensate the screen characteristics of the high-resolution display device without increasing the memory capacity for storing compensation data for compensating for display defects inherent in the display device.

In addition, it is possible to reduce the time required for the camera compensation process for obtaining compensation data in the process of shipping the display device.

1 is a diagram illustrating a camera compensation process in which a screen is photographed multiple times while moving a lit pixel dot and synthesized to obtain high-resolution compensation data;
2 is a diagram illustrating that moiré occurs in compensation data when high-resolution compensation data is obtained by focusing on the screen in the camera compensation process;
3 shows two images with different profiles in the high-frequency region;
4 shows an example in which the profile and compensation data of the image are displayed on the image when the image is compensated with compensation data that does not exactly match the profile of the image;
5 is a diagram illustrating a process of obtaining high-resolution compensation data from defocused image data taken at low resolution according to an embodiment of the present invention;
6 is a block diagram of a display device to which the present invention is applied;
7 is a block diagram illustrating the configuration of an image processing unit according to the present invention;
8 is a block diagram illustrating the configuration of a local profile restoration unit according to the present invention;
9 is a diagram illustrating a global feature learning process for obtaining a parameter of a filter applied to a global feature restoration unit according to an embodiment of the present invention;
10 is a diagram illustrating a local profile learning process for obtaining a parameter of a filter applied to a local profile restoration unit according to an embodiment of the present invention;
11 shows in detail the local profile extraction process in the local profile learning process of FIG.
12 is a diagram illustrating a process of extracting a local profile class using a detailed profile obtained in the process of extracting a local profile of FIG. 11;
13 is a detailed view of a process of learning a local profile class in the local profile learning process of FIG.
14 is a diagram illustrating a process of obtaining a plurality of local profile data from compensation screen data captured by a camera for a plurality of panels;
15 is a diagram illustrating the classification of the local profile data obtained in the process of FIG. 14 for each pixel position;
16 is a diagram illustrating a process of classifying each pixel position using a distribution characteristic of its value;
17 and 18 show the results of classification into a predetermined number of classes for each pixel position,
19 shows another example of classifying local profile data according to the distribution characteristics of the values by using two pixel positions as one unit;
20 is a comparison of compensation data images obtained according to a plurality of methods,
21 is a comparison of local profile data obtained according to a plurality of methods.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Like reference numerals refer to substantially identical elements throughout. In the following description, if it is determined that a detailed description of a known function or configuration related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

During the manufacturing process of the display device, stains or scratches may occur on the surface of the screen, and the luminance or color changes depending on the location, which deteriorates the quality of the image displayed on the screen. As described above, in order to solve the display defects inherent in the display device, a camera compensation process is performed before product shipment.

That is, a predetermined image, for example, gray of medium luminance is displayed on the assembled display device, and it is captured by a camera to detect display defects unique to the screen, that is, stains, scratches, luminance differences, color abnormalities, etc., and compensate them. Compensation data that can be used may be generated and stored in the memory of the display device.

1 shows a camera compensation process of photographing a screen a plurality of times while moving a lit pixel dot and synthesizing them to obtain high-resolution compensation data, and FIG. It shows that moiré occurs in the compensation data when

1 is a block diagram in which a predetermined number of pixels of a display device are divided into one block in order to obtain high-resolution compensation data, and only pixels in corresponding positions of each block are sequentially lit and the screen is photographed with a camera. It is called a shift dot pattern shooting method in which a screen is taken with a camera as many as the number of pixels included in the camera, an image is synthesized, and compensation data for compensating for a display defect of the screen is generated based on this.

In Fig. 1, the pixels of the panel are divided into 4x4 blocks, and first, only the pixels at the (1, 1) position of each block are turned on and the other pixels are turned off. Shooting in a pattern, lighting only the pixels at the (1, 2) position of each block, and displaying the screen with the other pixels turned off (4, 4) It is possible to obtain high-resolution compensation data by synthesizing 16 pattern images by shooting 16 times up to the pattern.

The shift dot pattern shooting method of FIG. 1 has the advantage of obtaining high-resolution compensation data by photographing the screen at high resolution, but as the size of the block increases, the number of times of shooting increases and the time required to generate compensation data increases. .

In FIG. 2, the screen is captured at a high resolution by accurately focusing on the screen, and compensation data is generated using this. There is a problem that cannot be reflected.

In the shift dot pattern shooting method of Fig. 1, reducing the size of the block can reduce the shooting time, but reducing the size of the block, for example, reducing the block to a 2x2 size, and shooting a screen at a high resolution as shown in Fig. 2, causes moiré. Therefore, there is a limit in reducing the size of the block.

On the other hand, in order to prevent moiré, the screen can be photographed in a defocused state without focusing on the screen. You can only get low-resolution data.

That is, since small specks or speckles with sharp features are crushed and photographed due to defocus, it is difficult to accurately compensate for these specks with a screen image taken in a defocused state, and the compensation performance is poor.

3 shows two images having different profiles in the high-frequency region, and FIG. 4 is an example in which the profile and compensation data of the corresponding image are displayed on the image when the corresponding image is compensated with compensation data that does not exactly match the profile of the image. will show

In FIG. 3 , the first normal image (Normal Image #1) and the second normal image (Normal Image #2) do not match the characteristics or defects of the high-frequency region, that is, the high-frequency profiles (Profile #1, Profile #2). Since the patterns are similar to each other, it is difficult for a viewer to recognize a difference between the first normal image and the second normal image.

4, when the first compensation data (Compensation Data for Profile #1) capable of compensating the high-frequency profile (Profile #1) for the first general image is synthesized with the first general image, defects in the high-frequency region The high-frequency region defect is not displayed in the first normal image because it is canceled by the first compensation data.

In the figure below of FIG. 4 , the high frequency profile (Profile #1) of the first general image similar to the high frequency profile (Profile #2) of the second general image is used as compensation data for the second general image and synthesized with the second general image Although a result is shown, unlike the comparison of normal images in FIG. 3 , if accurate compensation data is not used for an image to be compensated, not only high-frequency defects of the corresponding image but also compensation data may be displayed in the compensation result.

As such, unlike a general image, the compensation data must include a specific profile as well as an overall pattern to offset display defects on the screen. That is, it is impossible to properly compensate for display defects of the screen by simply using the compensation data obtained by photographing the screen at a low resolution in a defocused state.

In order to obtain correct compensation data from image data taken at a low resolution, it is necessary to restore a specific profile that accurately expresses screen defects from the captured image data.

In the present invention, high-resolution compensation data with low loss is obtained from image data taken at low resolution, the first high-resolution reconstructed data is obtained by learning the characteristics of each block, and the profile is learned by dividing it into local area units. It is possible to obtain the differential high-resolution compensation data and combine them to obtain high-resolution compensation data.

5 is a diagram illustrating a process of obtaining high-resolution compensation data from defocused image data captured at low resolution according to an embodiment of the present invention.

The image data taken at low resolution is divided into blocks of a predetermined size, the characteristics of each block image data are learned (Global Feature Training), the global characteristics of the block are restored (Global Feature Recovery), the first high-resolution restored data is obtained, and the restored By learning the local profile, that is, the profile of each local area included in the block using global features (Local Profile Training), the second high-resolution restoration data is obtained (Local Profile Recovery), and high-resolution compensation is based on this. data can be obtained.

The learning process in FIG. 5 , that is, the global feature learning process and the local profile learning process, is a process of obtaining a filter that outputs a high-resolution image by taking a low-resolution image as an input as a result of performing it on a plurality of display devices. In addition, the restoration process in FIG. 5 , that is, the global feature restoration process and the local profile restoration process may be performed to obtain compensation data for compensating an image input from each display device.

For each of a plurality of display devices of the same model, an image is taken in a defocused state while a predetermined luminance and color are displayed on the screen to obtain low-resolution image data, and a first filter using the plurality of low-resolution image data; For example, a first artificial neural network (ANN: Artificial Neural Network) is trained, a low-resolution block is input, and a high-resolution block (first-order high-resolution reconstructed data) is obtained as an output, the parameter of the first filter is obtained (global feature learning). ).

In addition, a plurality of high-resolution blocks are obtained from a plurality of low-resolution image data using a first filter, and a second filter, for example, a second filter, 2 The artificial neural network is trained to obtain the parameters of the second filter that takes the primary local data as input and outputs the secondary local data reflecting the detailed profile (local profile learning).

The secondary local data is high-resolution data in which the detailed profile of the corresponding local area is reflected, and high-resolution compensation data of the corresponding display device can be obtained by combining the secondary local data.

That is, in the state that the parameters for obtaining high-resolution compensation data by learning the first filter and the second filter using a plurality of low-resolution image data are obtained, other display devices of the same model (display devices not used for learning) are displayed. By applying the low-resolution image data captured in the defocused state to the first filter and the second filter, high-resolution compensation data in which global features in block units and detailed profiles in local area units are reflected can be obtained.

Here, the local area is an area included in a block and has a smaller size than the block, and may be set to one line (a group of pixels included in one row) among a plurality of lines constituting the block, which is a line unit. This is because, in a display device that writes data to a raw panel, compensating for input image data line by line is advantageous for data processing speed or memory management.

When the display device is shipped, it downsamples the low-resolution image data captured on the screen in a defocused state, stores it in the memory of the display device as a small capacity low-resolution compensation data, and applies the parameters acquired in the learning process to the first filter and a second filter can also be added to the compensation circuit as hardware.

The display device obtains high-resolution compensation data from low-resolution image data stored in a memory for each local region, ie, a line, in a local region unit by using a first filter and a second filter for each local region, that is, a line of the input image, Display defects inherent in the display device can be offset by combining the input image data and displaying it on the screen.

6 is a block diagram of a display device to which the present invention is applied. A display device according to an exemplary embodiment may include a display panel 10 , a timing controller 11 , a data driving circuit 12 , a gate driving circuit 13 , and an image processing unit 20 .

The display panel 10 includes pixels in which data lines 14 and gate lines 15 cross each other and are formed in a matrix form. A thin film transistor (TFT) is formed at the intersection of the data lines and the gate lines of the display panel 10 . The display panel 10 includes a liquid crystal display (LCD), a field emission display (FED), a plasma display panel (PDP), an inorganic electroluminescent element, and an organic light emitting diode element. It may be implemented as a flat panel display device such as an electroluminescence device (EL) including an organic light emitting diode (OLED) and an electrophoresis display device (EPD). When the display panel 10 is implemented as a display panel of a liquid crystal display device, a backlight unit is required. The backlight unit may be implemented as a direct type backlight unit or an edge type backlight unit.

The image processing unit 20 restores high-resolution compensation data from low-resolution image data, and outputs image data RGB′ in which display defects are canceled by adding or subtracting the restored compensation data to input image data RGB. The image processing unit 20 may be disposed in front of the timing controller 11 or mounted inside the timing controller 11 . A detailed description of the image processing unit 20 will be described next in conjunction with FIGS. 7 and 8 .

The timing controller 11 supplies the image data RGB′ output from the image processing unit 20 to the data driving circuit 12 . The timing controller 11 inputs timing signals such as a vertical synchronization signal Vsync, a horizontal synchronization signal Hsync, a data enable signal DE, and a dot clock CLK from a host system (not shown). It receives and generates control signals for controlling operation timings of the data driving circuit 12 and the gate driving circuit 13 . The control signals include a gate timing control signal GCS for controlling an operation timing of the gate driving circuit 13 and a data timing control signal DCS for controlling an operation timing of the data driving circuit 12 .

The data driving circuit 12 converts the image data RGB' into a data voltage under the control of the timing controller 11 and outputs the converted image data RGB' to the data lines 14 . The gate driving circuit 13 sequentially supplies scan pulses synchronized with the data voltage to the gate lines 15 under the control of the timing controller 11 .

The gate driving circuit 13 sequentially supplies a scan pulse synchronized with the data voltage to the gate lines 15 of the display panel 10 under the control of the timing controller 11 . The gate driving circuit 13 may be composed of a plurality of gate driving integrated circuits each including a shift register, a level shifter for converting an output signal of the shift register to a swing width suitable for driving a TFT of a pixel, an output buffer, and the like. . Alternatively, the gate driving circuit 13 may be directly formed on the lower substrate of the display panel 10 using a GIP (Gate Drive IC in Panel) method. In the case of the GIP method, the level shifter may be mounted on a printed circuit board (PCB), and the shift register may be formed on a lower substrate of the display panel 10 .

7 is a block diagram illustrating the configuration of an image processing unit according to the present invention.

The image processing unit 20 receives image data RGB to be displayed on the panel from the host, restores high-resolution compensation data from low-resolution compensation data, and compensates for image data RGB using the restored high-resolution compensation data. Thus, the image data RGB' from which the display defect is removed is output.

To this end, the image processing unit 20 uses the memory 210 for storing the low-resolution compensation data and the first filter to which the first parameter (Parameter #1) is employed, from the low-resolution compensation data to the corresponding block in block units. The global feature restoration unit 220 that generates primary high-resolution restoration data (Global Image) by restoring the global characteristics of The local profile restoration unit 230 that generates high-resolution compensation data for each local region by combining the predicted detailed profile and data of the corresponding local region, and the high-resolution compensation data of the local region generated by the local profile restoration unit The compensator 240 may be configured to compensate the input image data RGB of a corresponding local area using the compensator 240 to output the compensated image data RGB'.

In the memory 210 , a low-resolution compensation image or low-resolution compensation data may be stored by downsampling an image taken in a defocused state to store it in a small capacity. Since the first filter built into the global feature restoration unit 220 may input image data in block units and output image data in units of the same block as an output, the global feature restoration unit 220 receives data from the memory 210 . The process of converting the read low-resolution compensation data into high-resolution compensation data through interpolation, etc. can be performed first, which is simply interpolated, which is different from restoration using a high-dimensional filter, so that specific features cannot be restored. .

The global feature restoration unit 220 uses a first filter such as an artificial neural network to which the first parameter is employed to restore the global feature of the corresponding block in block units to generate high-resolution primary restored data, the first filter calculates It may be implemented in a hardware manner in consideration of speed or power consumption.

The compensator 240 may output compensated image data by adding or subtracting compensation data that the local profile restoration unit 230 restores and outputs for a local region corresponding to each local region data of the input image data. The local area may be one line included in a block of a predetermined size.

The global feature restoration unit 220 and the local profile restoration unit 230 search for a block corresponding to the local region in the low-resolution compensation image for each local region of the input image data whenever image data is input, and First, the first high-resolution reconstructed data is generated for the block by restoring the global features of the block from the low-resolution compensated image data using a filter, and the data of the corresponding local area is extracted from the first reconstructed data and using the second filter. By reconstructing the detailed profile of the local area, high-resolution second reconstructed data for the corresponding local area may be generated. These operations of the global feature restoration unit 220 , the local profile restoration unit 230 , and the compensator 240 are performed in real time in units of local regions according to the input speed of image data.

8 is a block diagram illustrating the configuration of a local profile restoration unit according to the present invention.

The local profile restoration unit 230 receives the primary high-resolution restoration data (Global Image) restored in units of blocks from the global feature restoration unit 220 and outputs high-resolution compensation data restored in units of local regions, extracting the local profile. It may be configured to include a unit 231 , a detailed profile prediction unit 232 , and a local profile calculation unit 233 .

The local profile extracting unit 231 extracts data in units of a local area, for example, in units of lines, from the primary restoration data in units of blocks.

The detailed profile prediction unit 232 predicts the detailed profile of the data of the local area extracted by the local profile extraction unit 231 by using the second filter to which the second parameter (Parameter #2) is employed. The prediction value for the detailed profile is a probability value for finely reconstructing the local area data in the local area unit among high-resolution reconstructed data of the block unit coarsely reconstructed through global feature restoration in the block unit, and the actual data of the local area When categorizing detailed profiles, which are difference values between and the reconstructed data of a corresponding local area, into a predetermined number of classes, it may be a probability value or a weight for each class.

The local profile calculator 233 may obtain compensation data for the corresponding local area by adding the sum of values obtained by multiplying the representative data of each class and the probability value of the corresponding class to the extracted data of the corresponding local area. Probability value for the i-th class predicted by HR1, representative data of the i-th class as Ci, and the detailed profile predicting unit 232 as the extracted data (primary high-resolution restored data) of the local region extracted by the local profile extraction unit 231 is Pi, and the final high-resolution compensation data (secondary high-resolution reconstructed data) for the local area is HR2, HR2 = b*HR1 + (C1*P2 + C2*P2 + + Cn*Pn), where b is a parameter reflecting the characteristics of the panel, and n is the number of classes.

9 is a diagram illustrating a global feature learning process for obtaining a parameter of a filter applied to a global feature restoration unit according to an embodiment of the present invention.

The first filter built into the global feature restoration unit 220 of the image processing unit 20 removes a screen for displaying a predetermined image for a plurality of display devices having the same model as the corresponding display device before shipping the corresponding display device. A parameter is learned using a low-resolution compensation image obtained by shooting in a defocused state, and a parameter for outputting a high-resolution image is obtained from the low-resolution compensation image.

The output of the first filter such as an artificial neural network is obtained by taking the low-resolution compensation image obtained by photographing the screen in a defocused state as an input, and the output is compared with the high-resolution compensation image photographed by the shift dot pattern shooting method to obtain the error value, and the error value is obtained. The first filter may be trained by feeding back a value to update the parameters of the first filter.

First, a predetermined image, for example, gray of medium luminance, is displayed on a screen for each of a plurality of display devices of the same model, and the camera is photographed in a defocused state to obtain a defocused image. The resolution of the image taken by the camera may be high, but since it is taken in a defocused state, the high-frequency component is crushed and does not reflect the detailed characteristics of the screen.

A defocused image projected in a defocused state is divided into a plurality of blocks having a predetermined pixel size, for example, 8x8 or 16x16, and a filter is trained for each block.

When the data of the first block is input to the first filter (Global Feature Training Filter in FIG. 9) whose parameters are set to initial values, the first filter outputs a high-resolution image (Recovered Global Image (A)) with global features restored. do.

In addition, high-resolution image data is obtained by shooting the screen of the corresponding display device in a shift dot pattern shooting method, and data of a block corresponding to the first block that has passed through the first filter is extracted (Shift Dot Image (B)), 1 Obtain the error value, which is the difference from the reconstructed high-resolution image output by the filter (Error Calculation (BA)). The parameter of the first filter is updated by reflecting the obtained error value (Parameter Update).

The next block is input to the first filter whose parameters have been updated, and the output is compared with the corresponding block in the high-resolution image data taken by the shift dot pattern shooting method to obtain the error value, and the parameter of the first filter is calculated by reflecting the obtained error value. update again.

This filter learning process is repeated for all blocks of the defocused image captured by one display device, and this filter learning process is repeated for a plurality of other display devices of the same model, but there is little change in the error value and the Repeat until (Error Saturated?) to obtain the first parameter (Parameter #1) for the first filter.

The first filter obtained through learning for a plurality of display devices of the same model in this way restores the low-resolution compensation image photographed on the display device in block units, even for a display device that is not used for learning with the same model, in block units, A primary reconstructed image close to the actual high-resolution compensation image of the display device (a high-resolution compensation image photographed by a shift dot pattern imaging method) may be output.

At this time, the first filter implemented as an artificial neural network includes two or more hidden layers each including a plurality of nodes, and uses as many as the number of pixels included in the block as input nodes and output nodes, so that the nodes are connected to each other. can be A feature image is extracted through a convolution operation of data and parameters of a block input from the first filter, and as the layer becomes deeper, the output data includes higher-resolution features, so that the output data of the first layer The resolution of the output data of the second layer is higher than the resolution of the output data of the third layer, and the resolution of the output data of the third layer is higher. The more the parameters are updated, the better the high-resolution global features are extracted.

However, since the first filter restores low-resolution image data into high-resolution image data in blocks of 8x8 or 16x16 size, only the overall characteristics of the block, that is, only the global characteristics, are roughly reflected, and detailed characteristics are specifically reflected. There are limits to what you can do. Accordingly, it is necessary to additionally learn a detailed profile in units of local regions constituting the block.

10 is a diagram illustrating a local profile learning process for obtaining a parameter of a filter applied to a local profile restoration unit according to an embodiment of the present invention.

The process of learning a local profile is a local profile extraction process (Local Profile Extraction), the process of extracting a predetermined number of classes by classifying a detailed profile that is a difference value between the data of the corresponding local area and the actual data of the corresponding local area for each of the extracted data of the plurality of local areas (Local Profile Class Extraction) and It may be configured to include a local profile class training process for learning a second filter for outputting information indicating a class corresponding to the data of the corresponding local area using the extracted data of the local area.

The local profile learning process of FIG. 10 is also performed for a plurality of display devices of the same model. A first filter obtained in the global feature learning process by dividing a low-resolution compensation image photographed in a defocused state in each display device into blocks, and A global feature is reconstructed in units of blocks divided using The second parameter is embedded in the image processing unit of each display device.

Each step of the local profile learning process included in FIG. 10 will be described in detail with reference to FIGS. 11 to 13 .

FIG. 11 specifically illustrates a local profile extraction process in the local profile learning process of FIG. 10 .

The image data of the block extracted by dividing the low-resolution defocused image into blocks is passed through a first filter learned as a first parameter to restore global features to obtain primary restored high-resolution image data (Recovered Global Image). Data of a predetermined local area, for example, data of one line, is extracted from the primary reconstructed image data (Extracted Local Profile (A)).

Image data (Shift Dot Image) of a corresponding block is obtained from a high-resolution compensation image photographed by a shift dot pattern imaging method, and data of a corresponding local area (Extracted Local Profile (B)) is extracted therefrom.

A difference value between the data (A) of the local area extracted from the first reconstructed block image data and the actual data (B) of the corresponding local area is obtained as a detailed profile (Detail Profile (B-A)).

That is, the detailed profile is divided into local area units among block data composed of the difference between the high resolution compensation image restored by the first filter and the actual high resolution compensation image. When the local area is set as one line included in the block, 8 Alternatively, 16 consecutive pixel values (difference values) may be a detailed profile.

12 is a diagram illustrating a process of extracting a local profile class using a detailed profile obtained in the process of extracting a local profile of FIG. 11 .

For each local area, that is, for each of a plurality of local areas included in a plurality of blocks included in a low-resolution compensation image captured by a plurality of display devices of the same model, the detail value that is the difference value between the reconstructed compensation data and the actual compensation data Obtain a profile (Detail Profile (BA)), classify them by grouping them with similar types (Similar Profile Classification), create a predetermined number of classes using similar groups as a class, and extract a representative profile for each class. (Representative Profile Extraction for Each Class).

In FIG. 12 , for example, detailed profiles are classified into 36 classes. For each class, for example, an average value of detailed profiles bundled with the corresponding class may be used as a representative value.

A specific method of classifying the detailed profile into a predetermined number of classes will be described in detail next with reference to FIGS. 14 to 19 .

FIG. 13 is a detailed diagram illustrating a process of learning a local profile class in the local profile learning process of FIG. 10 , and includes parameters of the second filter built in the local profile restoration unit 230 of the image processing unit 20 of the display device. It is in the process of retrieval.

The local profile class learning of FIG. 13 is performed in units of local regions.

Through the first filter, global features are restored in block units from the low-resolution compensation image to obtain block-sized primary restored data, and from this, local profile data for learning the secondary filter is extracted (Local Profile Data Extracted from Recovered). Global Image).

When the extracted local region data is input to a second filter (Local Profile Training Filter), for example, an artificial neural network filter, the second filter determines the class to which the data of the local region belongs, that is, the data of the local region is It is scheduled to output a probability value indicating the degree to which the belonging class corresponds to each of the plurality of classes obtained in FIG. 12 .

In a state in which the parameters of the second filter are sufficiently updated, when, for example, data of a local region belonging to the first class among 36 classes is input to the second filter, the value corresponding to the first class is set to 1 in the second filter. It is possible to output 36 probability values that are close and the values corresponding to the remaining classes are close to 0.

When the data of the first local region is input to the second filter in which the parameter is set as an initial value, the second filter predicts the class corresponding to the input data of the local region, and a probability value for each of the number of classes classified in FIG. 12 . (Probability values for each Class (A))

In the high-resolution image data obtained by photographing the screen of the display device by the shift dot pattern imaging method, the class to which the real data corresponding to the local area belongs, that is, the difference between the real data and the primary reconstructed data for the local area, that is, The class to which the detailed profile belongs is extracted and expressed as an actual probability value (Corresponding Class (B)). For example, if the difference value between the actual data of the current local area and the primary reconstructed data, that is, the detailed profile of the data of the current local area belongs to the first class, the actual probability value (B) of the local area is It can be expressed as (1, 0, 0, , 0). Similarly, if the detailed profile of the current local area data belongs to the second class, the actual probability value (B) of the local area can be expressed as (0, 1, 0, , 0).

An error value that is the difference between the probability value (A) output by the second filter and the actual probability value (B) is calculated (Error Calculation (BA)), and the parameters of the second filter are updated by reflecting the obtained error value (Parameter Update) ).

The data of the next local region is input to the second filter to obtain the predicted probability value for predicting the class to which the data of the corresponding local region belongs, and the error value is obtained by comparing the actual probability value and the predicted probability value of the corresponding local region. Reflecting this, the parameter of the second filter is updated again.

This filter learning process is repeated for data of all local regions included in all blocks in which global features are reconstructed for a defocused image captured by one display device, and this filter learning process is also performed on a plurality of other display devices of the same model. The process is repeated, but until there is little change in the error value and is saturated (Error Saturated?), a second parameter (Parameter #2) for the second filter is obtained.

The second filter obtained through learning for a plurality of display devices of the same model in this way is applied to the data of each local area of the low-resolution compensation image photographed on the display device, even for a display device that is not used for learning with the same model. It is possible to obtain a second filter that outputs a difference value between the primary reconstruction data of the corresponding local region and the actual data, that is, a probability value for predicting a class to which the detailed profile belongs.

The probability value output by the second filter corresponds to the weights of classes necessary to reconstruct the actual data of the local area from the primary reconstructed data of the local area. That is, the difference value between the primary reconstruction data of the local region and the actual data is obtained by weighting and summing the representative value of each class with the probability value output by the second filter.

When the second filter accurately predicts the class to which the reconstructed data of the local area being processed belongs, the second filter has a probability value of a class to which the reconstructed data of the local area belongs is close to 1 and a probability value of another class is close to 0 to output The class to which the data of the local area belongs is the clustering of the difference value between the reconstructed data and the actual data of the local area, and it is considered that the representative profile of the class sufficiently reflects the difference value between the reconstructed data and the actual data of the local area. Therefore, if the representative profile of the corresponding class is weighted with a value of almost 1 and added to the reconstructed data of the local area, the actual data of the local area can be approximated from the reconstructed data of the local area.

The second parameter (Parameter #2) obtained in this way is applied to the second filter implemented in a hardware manner and built-in in the detailed profile prediction unit 232 of FIG. Compensation data can be restored in real time in units of lines with a limited length.

Meanwhile, a process of classifying data of a local area unit into a detailed profile will be described with reference to FIGS. 14 to 19 .

14 is a diagram illustrating a process of obtaining a plurality of local profile data from compensation screen data captured by a camera for a plurality of panels.

For a plurality of display devices of the same model, a camera captures a screen in a defocused state or captures a screen using a shift dot pattern method to obtain a low-resolution compensation image and a high-resolution compensation image. Each of the low-resolution compensation images is divided into blocks of a predetermined size, for example, 8x8, and each block is divided and processed into a local area of a predetermined size. .

15 is a diagram illustrating the classification of the local profile data obtained in the process of FIG. 14 by pixel position, and each local profile data extracted in 14 is sorted based on 8 pixel positions from 1 to 8, and for each pixel position. Separately, the data values at the corresponding positions are displayed on a vertical line.

Since there are N panels in FIG. 14 and M pieces of local profile data for each panel, NxM pieces of local profile data are classified into a predetermined number of partial classes.

16 shows a process of classifying each pixel position using the distribution characteristic of the value. Since one pixel value exists at each pixel position in the local profile data, the values of each pixel position are on a one-dimensional straight line. However, it is expressed in two dimensions to make it easier to understand the process of classifying classes and determining the representative values of the classes.

When data is distributed as shown in Fig. 16(a), the user arbitrarily selects two classes, and as shown in (b), the center (large circle) of the first class and the second class is arbitrarily selected as many as the number of classes. can be set.

The primary classification is based on the distance between the data, and the distance between the random centers of the first class and the second class is obtained, respectively, and the class is first classified by the central value located at a close distance as shown in (c).

As in (d), the central value is calculated based on the data of the first classified class and set as a new central value (square).

As in (e), secondary classification is performed based on the distance, and the distance between two new centroid values and each data is obtained, and the class is reclassified into a class of centroid values located close to the corresponding data.

As shown in (f), based on the secondary classification, the central value of data belonging to the same class is calculated and set as a new central value.

As in (g) and (h), tertiary classification is performed based on the distance between data, and if the classification result is the same as before, the classification operation is terminated. Repeat.

17 and 18 show the results of classification into a predetermined number of classes for each pixel position. The data at the first position has 4 partial classes, the data at the second position has 3 partial classes, and the data at the 8th position is It is classified into three subclasses, and the average value for each subclass is obtained as a representative value.

A partial class is combined for each pixel position, a full class is obtained based on this, and a representative profile of each class is generated using the representative value of each partial class.

14 to 18 , for each pixel position of a line, a class is set by classifying two or more based on the data value of the corresponding position, and the entire class is classified by a combination of the classes set at each pixel position.

19 shows another example of classifying local profile data according to the distribution characteristic of two pixel positions as a unit, and the horizontal axis is the i-th pixel position of data values of two neighboring pixel positions. The data can be classified by dividing the data into four equal areas for convenience and assigning one subclass to each area. have. Alternatively, partial classes may be classified by the method described with reference to FIG. 16 .

Eight pixel positions may be classified by grouping each two pixel positions into one, some may be classified as a subclass for each pixel position, and some may be classified by grouping each two pixel positions.

Data values of consecutive three pixel positions may be displayed in a three-dimensional space and classified into eight classes.

20 is a comparison of compensation data images obtained according to a plurality of methods, and FIG. 21 is a comparison of local profile data obtained according to a plurality of methods.

In FIG. 20, (a) is a defocused image photographed in a defocused state, (b) is an image in which global features are reconstructed from the defocused image using a first filter, and (c) is a defocused image using the first filter. to restore the global features and use the second filter to restore the local profile, (d) is an image taken by the shift dot pattern method.

The image in which the global features are restored is closer to the real image than the defocus image, and the image in which the local profile is restored is closer to the real image than the image in which the global features are restored.

In FIG. 21 , the image in which the global features and the local profile are restored closely follows the image captured by the shift dot pattern method.

Accordingly, since the display device stores only the low-resolution compensation image, it is possible to reduce the use of the non-volatile memory for storing the high-resolution compensation data. In addition, since the high-resolution compensation data is restored and output line by line from the low-resolution compensation image in real time for a portion of one line of the panel, the capacity of the volatile memory required to compensate the input image data can be reduced. have.

In addition, since compensation data close to the actual high-resolution compensation data is restored even with a low-resolution compensation image, display defects unique to the corresponding display device can be properly healed to improve display quality.

In addition, it is possible to reduce the manufacturing cost by reducing the time required to obtain camera compensation data before shipping the display device.

Those skilled in the art from the above description will be able to see that various changes and modifications are possible without departing from the technical spirit of the present invention. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

10: display panel 11: timing controller
12: data driving circuit 13: gate driving circuit
14: data line 15: gate line
20: image processing unit 210: memory
220: Glover feature restoration unit 230: local profile restoration unit
231: local profile extraction unit 232: local profile prediction unit
233: local profile calculation unit 240: compensation unit

Claims (13)

generating primary reconstruction data of a second resolution higher than the first resolution in block units of a first size from image data of a first resolution by using a first filter;
generating secondary restored data in units of a local area having a second size smaller than the first size from the primary restored data of the block by using a second filter;
compensating for the image data in units of the local area by adding or subtracting the secondary reconstruction data to data of a corresponding local area among image data input to a display device; and
Displaying the compensated image data,
The generating of the secondary restoration data comprises:
a first step of extracting data in units of the local area from the primary reconstruction data in units of blocks;
a second step of predicting a detailed profile of the extracted local region data using the second filter; and
A third step of generating the secondary reconstruction data by combining the predicted detailed profile and the extracted local area data,
The detailed profile corresponds to an error value between the primary reconstruction data of the local area and actual data of the corresponding local area. According to claim 1,
The block is a set of pixels with a size of nxn (where n is an even natural number), and the local area is one pixel line included in the block. According to claim 1,
wherein the first filter and the second filter are implemented as an artificial neural network including a plurality of input nodes, two or more hidden layers including a plurality of nodes, and a plurality of output nodes, and the nodes are connected to each other. Way. 4. The method of claim 3,
and the artificial neural network includes parameters learned by using image data of a first resolution and image data of a second resolution photographed for a plurality of display devices. According to claim 1,
The image data of the first resolution is an image obtained by photographing a screen displaying a predetermined image on a corresponding display device in a defocused state, down-sampled, and stored in the corresponding display device. delete According to claim 1,
When classifying detailed profiles of a plurality of local regions into a plurality of classes, the second filter outputs a probability value for each class, and sets the first probability value for a class to which input local region data belongs to a high value. and outputting a probability value for another class as a value lower than the first probability value. 8. The method of claim 7,
In the third step, for each of the plurality of classes, the representative data of the corresponding class and the probability value for the corresponding class are multiplied, and a first value is obtained by adding the plurality of multiplication results, and a predetermined value is obtained from the extracted local area data. An image processing method, characterized in that a second value is obtained by multiplying a parameter, and the second value is generated by adding the first value and the second value to generate secondary reconstructed data for a corresponding local area.
a memory for storing image data of a first resolution;
A first filter is built in, and for each local area of input image data, a block including a corresponding local area is found from the image data, and a higher resolution than the first resolution is obtained from the image data of the block using the first filter. a global feature restoration unit for generating primary restoration data of a second resolution;
A second filter is built in, data of a corresponding local area is extracted from the primary reconstruction data of the block, and a detailed profile of the corresponding local area is restored from the extracted local area data using the second filter. a local profile restoration unit for generating restoration data; and
and a compensator for outputting image data compensated by adding or subtracting the secondary reconstruction data to data of a corresponding local area among the input image data,
The local profile restoration unit,
a local profile extractor for extracting data of a corresponding local area from the primary reconstruction data of the block;
a detailed profile prediction unit for predicting the detailed profile for the extracted local region using the second filter; and
and a local profile calculator for generating the secondary reconstruction data by combining the predicted detailed profile and the extracted local area data,
The detailed profile corresponds to an error value between the primary reconstruction data of the local area and actual data of the corresponding local area. 10. The method of claim 9,
The first filter and the second filter are implemented as an artificial neural network including a plurality of input nodes, two or more hidden layers including a plurality of nodes, and a plurality of output nodes, wherein the nodes are connected to each other,
and the artificial neural network includes parameters learned by using image data of a first resolution and image data of a second resolution photographed for a plurality of display devices. 10. The method of claim 9,
and the memory stores image data of a first resolution obtained by down-sampling an image obtained by photographing a screen on which a predetermined image is displayed on a corresponding display device in a defocused state. 10. The method of claim 9,
The second filter, when classifying detailed profiles of a plurality of local regions into a plurality of classes, outputs a probability value for each class, and sets a first probability value for a class to which input local region data belongs to a high value. and outputting a probability value for another class as a value lower than the first probability value. a display panel including pixels in which data lines and gate lines cross and are formed in a matrix;
an image processing unit for generating compensation data and compensating and outputting the input image data using the compensation data;
a timing controller for generating control signals for displaying image data output by the image processing unit;
a data driving circuit for converting image data input from the timing controller into data voltages under the control of the timing controller and outputting them to data lines; and
and a gate driving circuit for generating a scan pulse synchronized with the data voltage under the control of the timing controller and sequentially supplying it to the gate lines;
The image processing unit,
a memory for storing image data of a first resolution;
A first filter is built in, and for each local area of input image data, a block including a corresponding local area is found from the image data, and a higher resolution than the first resolution is obtained from the image data of the block using the first filter. a global feature restoration unit for generating primary restoration data of a second resolution;
A second filter is built in, data of a corresponding local area is extracted from the primary reconstruction data of the block, and a detailed profile of the corresponding local area is restored from the extracted local area data using the second filter. a local profile restoration unit for generating restoration data; and
and a compensator for outputting image data compensated by adding or subtracting the restored secondary restored data to data of a corresponding local area among the input image data,
The local profile restoration unit,
a local profile extractor for extracting data of a corresponding local area from the primary reconstruction data of the block;
a detailed profile prediction unit for predicting the detailed profile for the extracted local region using the second filter; and
and a local profile calculator for generating the secondary reconstruction data by combining the predicted detailed profile and the extracted local area data,
The detailed profile corresponds to an error value between the primary reconstruction data of the local area and actual data of the corresponding local area.

Images