JP6791299B2 - Color swatches, color swatch creation device and color swatch creation method, and image processing system using color swatches - Google Patents

Description

The present invention relates to a color swatch, a color swatch making apparatus and a color swatch making method, and an image processing system using the color swatch.

A color swatch as a color chart imitates the appearance of a color in an object or the like, and a conventional color swatch is composed of a single color patch in which a uniform color area is printed in a rectangular shape (for example,). See Patent Document 1).

Originally, such a color sample is used as a sample for coloring such as painting the color of a printed matter or an object, or for checking or comparing the color reproducibility of a printed matter or an object after coloring.

For an actually colored object, the color reproducibility is checked in the real space. However, the actual color of an object is, for example, the lighting conditions such as direct illumination or indirect illumination, the material of the object, the uneven shape of the surface of the object, or the position of illumination-object-checker's viewpoint. It has a spatial distribution due to relationships and is not always uniform.

In particular, in the case of a photograph in which a printed matter captures an object or a scene containing an object in the real space, even if the object is uniformly colored, the color of the object is spatially uneven due to shooting conditions and the like. Is usually the case.

Therefore, it is difficult to reproduce the impression received from the appearance of a color having a spatial distribution, which is not uniform depending on the spatial distribution condition, with a single color patch such as a conventional color sample.

For these reasons, we are currently proposing a color sample, a color sample creating device, a color sample creating method, and an image processing system using the color sample. In the color sample, all the pixels included in the predetermined area of the original drawing (original image) are rearranged based on the lightness, saturation, and hue angle, and each pixel after the rearrangement is placed in the predetermined area in the color sample. It is placed in the corresponding area. In this way, the color sample can approximately reproduce the impression received from the appearance of colors in an actual environment.

However, in the color sample, pixels of similar colors may be arranged in a dispersed form in the pixel group in the region. When the pixels of the similar color are arranged in a dispersed manner, the pixels of the similar color are easily perceived by the observer, and the graininess is increased. The higher the granularity, the easier it is to give the observer a grainy feeling. Since the graininess (graininess) is perceived by the observer as noise, it is desirable to suppress the graininess.

The present invention has been made to solve the above-mentioned problems, and it is possible to approximately reproduce the impression received from the appearance of colors in an actual environment, and to create a color sample and a color sample with less granularity. An object of the present invention is to provide an apparatus, a method for creating a color sample, and an image processing system using the color sample.

The color swatch according to the present invention does not necessarily match the first pixel group generated from the first pixel group consisting of the first pixel number constituting the first region in the first image . A first color sample including a second region in which a second pixel group consisting of two pixels is arranged, and the pixels constituting the first pixel group are sorted in descending order of chromaticity . If the number of the first pixel and the number of the second pixel do not match after rearranging, the number of the second pixel is obtained by performing linear interpolation for each pixel with the brightness, saturation, and hue angle. The number of pixels is adjusted so that the number of pixels is the same, and for each pixel after the first rearrangement and the adjustment of the number of pixels as necessary, the row in the second region is assumed. When pixels of the same lightness are consecutive for each row, a second sort is performed to sort them in order of saturation, and after the second sort, the lightness and saturation are the same for each row. When the pixels are continuous, they are rearranged in the order of chromaticity angle, and the pixels are arranged in the second region.

According to the present invention, an impression received from the appearance of a color in an actual environment can be approximately reproduced, and a color swatch, a color swatch making device, a color swatch making method, and a color swatch with less granularity are used. An image processing system can be provided.

It is a block diagram which shows the outline of the image processing system which concerns on embodiment of this invention. It is a block diagram which shows the structural example of the image processing apparatus in the image processing system which concerns on embodiment of this invention. It is a block diagram which shows the functional structure example of the image processing system which concerns on embodiment of this invention. It is a figure which shows the example of the case information used for the modification processing in the image processing system which concerns on embodiment of this invention. It is a figure which shows the modification processing by the standard type of the image in the image processing system which concerns on embodiment of this invention. It is a figure which shows the modification processing by the transfer type of the image in the image processing system which concerns on embodiment of this invention. It is a figure explaining a color sample example. It is a figure which shows the 1st image example used for making a color sample. It is a figure explaining the example of the 1st pixel group copied to RAM 23. It is a figure which shows the state example of the 1st pixel group after performing the 1st rearrangement. It is a figure which shows the state example of the 1st pixel group after performing the 2nd rearrangement. It is a figure explaining the example of the method of adding a pixel to the 1st pixel group after performing the 2nd rearrangement. It is a figure explaining the example of the 2nd pixel group immediately before performing the 3rd rearrangement. It is a figure explaining the example of the 2nd pixel group after performing the 3rd rearrangement. It is a figure which explains the example of the 2nd pixel group after performing the 3rd rearrangement more broadly. It is a figure explaining the example of the 2nd pixel group after performing the 4th rearrangement. It is a figure which shows the created color sample example. It is a flowchart which shows the procedure of making the color sample image which concerns on 1st Example of this invention. It is a figure explaining the method of limiting the movable range of a pixel. It is a flowchart which shows the procedure of making the color sample image which concerns on 2nd Example of this invention.

Hereinafter, an embodiment according to the present invention will be described with reference to the drawings.

FIG. 1 shows a configuration example of an image processing system according to an embodiment of the present invention.

As shown in FIG. 1, the image processing system 100 includes, for example, an image processing device 2 and a server device 3. In the image processing system 100, the image processing device 2 and the server device 3 are connected to each other via the network 4.

The image processing device 2 is an information processing device capable of performing correction processing on an input image (pre-correction image) to be corrected such as a photograph by using a color sample (second image) described later for color appearance. Is. The information processing device is, for example, a personal computer (PC), a tablet terminal, or the like.

Further, the image processing device 2 inputs an original image (hereinafter referred to as "first image" because it corresponds to the first image) which is an image used for creating a color sample (second image). And a color sample can be created from the input first image. The color sample creating device according to the present embodiment is realized by being mounted on the image processing device 2.

The server device 3 is an information processing device capable of storing a first image and a color sample created by using the first image as image data, respectively. Although only one server device 3 is shown here, a plurality of server devices 3 may exist. That is, the number of server devices 3 is not particularly limited. The number of image processing devices 2 is not limited to one.

The network 4 is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or a group of networks including two or more of them. The network 4 may be a cable that directly connects the image processing device 2 and the server device 3. The image processing device 2 and the server device 3 may be connected wirelessly.

In the image processing system 100 having the above configuration, the image processing device 2 can receive various data including images (image data) from the server device 3 via the network 4, and transmits various data to the server device 3. can do. In the present embodiment, it is assumed that the server device 3 functions as a database server device that supplies various data required by the image processing device 2 and stores various data generated by the image processing device 2.

FIG. 2 is a block diagram showing a configuration example of the image processing device 2. The configuration example shown in FIG. 2 is an example, and the configuration of the image processing device 2 is not limited to the configuration example shown in FIG.

As shown in FIG. 2, the image processing device 2 has a CPU 21, a flash memory 22, a RAM 23, a display device 24, an input device 25, two or more I / Fs (Interfaces) 26, 27, and 21 to 27 of each unit. A bus 28 for connecting is provided.

Here, the flash memory 22 is described as including a storage such as an SSD (Solid State Drive). Various programs and various data are stored in the flash memory 22. The program includes a program that enables the creation of color samples (hereinafter referred to as "color sample creation program") 22a, and an image to be modified (hereinafter referred to as "pre-correction image") with reference to the color sample. A program (hereinafter referred to as "image modification program") 22b that enables the generation of a modified image that is an image before modification after modification is performed is included.

The RAM 23 is mainly used as a work for the CPU 21 to execute various programs. Various programs stored in the flash memory 22 are read into the RAM 23 and executed.

The display device 24 is, for example, a liquid crystal display device. The display device 24 displays various data under the control of the CPU 21.

The input device 25 is a group of input devices including, for example, a keyboard and a pointing device. Various instructions and various data inputs by the user can be performed via operations on the input device 25.

The I / F 26 enables communication via the network 4. As a result, the image processing device 2 communicates with the server device 3 via the I / F 26.

The I / F 27 is, for example, a connection device that enables connection with an external device. The external device is, for example, a medium drive device for driving a recording medium such as a scanner or an optical disk, a hard disk device, or a storage such as a semiconductor memory (for example, a USB (Universal Serial Bus) memory).

The I / Fs 26 and 27 can be used for inputting and outputting various data including images. However, here, for convenience of explanation, only the server device 3 is assumed as an input destination and an output destination of various data including images.

Similar to the image processing device 2, the configuration of the server device 3 is not limited. For the sake of convenience, it is assumed that the server device 3 is also a configuration example shown in FIG. 2, and the reference numerals of the respective parts are the same as those of the image processing device 2.

FIG. 3 is a block diagram showing a functional configuration example of the image processing system according to the embodiment of the present invention. Here, the image processing device 2 and the server device 3 are divided into, and examples of their functional configurations are shown.

As shown in FIG. 3, the image processing device 2 is equipped with a color sample creating device 210 and an image modifying device 230.

The color sample creating device 210 is a color sample creating device according to the present embodiment, which can create a color sample by referring to the first image. The color sample creating device 210 is realized by the CPU 21 executing the color sample creating program 22a stored in the flash memory 22.

On the other hand, the image retouching device 230 is a device capable of retouching an uncorrected image with reference to a color sample and creating a retouched image obtained by the retouching. The image correction device 230 is realized by the CPU 21 executing the image correction program 22b stored in the flash memory 22.

First, the image retouching device 230 will be described in detail. As shown in FIG. 3, the image retouching device 230 includes a data acquisition unit 231, an image input unit 232, a case selection unit 233, a retouching unit 234, and a data output unit 235 as functional configurations.

The data acquisition unit 231 is a function of acquiring necessary data from the server device 3 via the network 4. Data is acquired from the server device 3 under the control of the CPU 21 that executes the image modification program 22b stored in the flash memory 22. From this, the data acquisition unit 231 is realized by, for example, a CPU 21, a flash memory 22, a RAM 23, an I / F 26, and a bus 28.

Various data acquired from the server device 3 include a color sample, an image before modification, case information, and the like.

The case information is information for designating the modification contents for the image before modification, and includes reference information at the time of modification and case identification information. The reference information at the time of modification is information for designating the modification method, and the case identification information is information representing a case to be referred to when performing modification.

FIG. 4 is a diagram illustrating a configuration example of case information.

Examples of modifying an image before modification with reference to a color sample are roughly classified into a standard type that refers to two color samples and a transfer type that refers to one color sample. FIG. 4 (a) shows standard case information, and FIG. 4 (b) shows transfer type case information.

The case identification information includes type information, case ID (Identifier) information, image A information, image A color tone information, image B information, image B color tone information, impression expression information, selected area shape information, and selected area information. ..

The type information is information indicating the type of the case, that is, whether the case is a standard type or a transcription type. As a result, as the content of the information, the "standard type" is shown in FIG. 4 (a) and the "transfer type" is shown in FIG. 4 (b).

The case ID information is information that represents a case. As the content of the information, "0001" is shown in FIG. 4A and "0001-2" is shown in FIG. 4B.

The image A information is information representing a color sample (hereinafter, also referred to as “image A”) that should be referred to as a case image before modification. As the content of the information, in FIG. 4A, “img0001.jpg” representing the file name is shown. Since the image A is not referred to in the transfer type, “NULL” indicating that the information does not exist is shown in FIG. 4 (b).

The color tone information of the image A is information representing the color tone of the entire image A. In the present embodiment, it is assumed that each pixel has data of three elements of the color system of L * (Elster), a * (Aster), and b * (Beaster). For this reason, as the content of the information, "(50, 12, 15)" representing the average Lab value is shown in FIG. 4 (a). The type of data representing the pixels is not particularly limited.

The image B information is information representing a color sample (hereinafter, also referred to as “image B”) that should be referred to as a modified case image. As the content of the information, "img0002.jpg" is shown in FIGS. 4 (a) and 4 (b), respectively. In order to distinguish the image B between the standard type and the transfer type, the image B in the transfer type is also referred to as a "reference image".

The color tone information of the image B is information representing the color tone of the entire image B. As the content of the information, "(34, 11, 14)" is shown in FIGS. 4 (a) and 4 (b).

The impression expression information is information representing the impression received by the observer from the images A and B or the image B. As the content of the information, "dark and smooth" is described in FIGS. 4 (a) and 4 (b), respectively.

The selected area shape information is information representing the shape of the image A and B, or the area (second area) to be referred to in the image B. As the content of the information, "circle" is shown in FIGS. 4 (a) and 4 (b), respectively.

The selected area information is information representing an area to be referred to in the images A and B or the image B. As the content of the information, "center (50, 50), radius 50" is shown in FIGS. 4 (a) and 4 (b), respectively.

The selected area shape information and the selected area information whose contents are described in FIGS. 4 (a) and 4 (b) are based on the assumption of a color sample as shown in FIG. 7. There are a plurality of rows in which pixels are arranged in the selection area (second area), and the pixel group (second pixel group) arranged in the selection area is divided into rows.

The reference information at the time of modification includes image processing operator information and image processing parameter information.

The image processing operator information is information indicating an operation method for modification. As the content of the information, "tone curve (L tone curve)" is shown in FIG. 4A, and "transfer operator" is shown in FIG. 4B.

The tone curve specifies the operation content for modifying the entire image. The image parameter information is information used to perform an operation specified by the image processing operator information. For this reason, in the standard type, control point information used for creating a tone curve is registered as image parameter information.

The transfer operator specifies an operation for modifying the unmodified image toward the image B (reference image) as a target. For this reason, in the transfer type, the image parameter information is "NULL (see image B)".

The configuration example of the case information shown in FIG. 4 is only one example, and the configuration of the case information is not particularly limited. The method of describing case information is also not limited. The case information may be described using, for example, XML (Extensible Markup Language).

Here, it is assumed that the server device 3 stores a plurality of case information as shown in FIG. 4A or FIG. 4B, and also stores a plurality of unmodified images.

In the above assumption, the image input unit 232 is a function of acquiring a designated unmodified image from the server device 3 according to a user's instruction. The user's instruction is given by operating the input device 25, and the unmodified image is acquired from the server device 3. For this reason, the image input unit 232 is realized by, for example, a CPU 21, a flash memory 22, a RAM 23, an input device 25, an I / F 26, and a bus 28.

The case selection unit 233 is a function of presenting, for example, case information that can be acquired from the server device 3 to the user, and causing the user to select desired case information. The acquisition of data from the server device 3 by the data acquisition unit 231 is controlled by the case selection unit 233. The case selection unit 233 is realized by, for example, a CPU 21, a flash memory 22, a RAM 23, a display device 24, and a bus 28.

The modification unit 234 is a function of referring to the case information selected by the user and performing modification on the designated pre-modification image. The modification unit 234 is realized by, for example, a CPU 21, a flash memory 22, a RAM 23, and a bus 28.

The modification of the pre-modification image performed by the modification unit 234 will be specifically described with reference to FIGS. 5 and 6.

FIG. 5 is a diagram illustrating an example of a method of modifying an image before modification using standard case information.

In the modification using the standard case information, the image A (FIG. 5 (a)) and the image B (FIG. 5 (b)) specified in the case information are used. The tone curve (FIG. 5C) specified by the image processing operator information is created to correct the gradation of the image A toward the image B. The unmodified image (FIG. 5 (d); hereinafter referred to as “input image”) is converted using the created tone curve to create a modified image (FIG. 5 (e)). In the example shown in FIG. 5, the skin of a person existing as an object in the unmodified image (for example, a photograph), the clothes of the person, the sky and mountains behind the person, and the like are modified.

FIG. 6 is a diagram illustrating an example of a method for modifying an image before modification using transfer-type case information.

In the modification using the transfer type case information, only the image B (FIG. 6A) specified in the case information is used. In this transfer type, a pre-modification image (FIG. 6 (b); hereinafter referred to as “input image”) is modified using a known method such as color transfer to create a modified image (FIG. 6 (c)). .. Also in the example shown in FIG. 6, the skin of a person existing as an object in the unmodified image (for example, a photograph), the clothes of the person, the sky and mountains behind the person, and the like are modified.

The data output unit 235 shown in FIG. 3 is a function capable of outputting a modified image. In addition to the server device 3, there is a display device 24 shown in FIG. 2 as a specific output destination of the modified image. For this reason, the data output unit 235 is realized by, for example, a CPU 21, a flash memory 22, a RAM 23, a display device 24, an I / F 26, and a bus 28.

Here, a functional configuration example of the server device 3 connected to the image processing device 2 will be specifically described.

As shown in FIG. 3, the server device 3 includes a control unit 31, a communication unit 32, a first storage unit 33, and a second storage unit 34 as functional configurations.

The communication unit 32 enables communication via the network 4. Communication via the network 4 is enabled by the CPU 21 executing a network OS (Operating System) stored in the flash memory 22 or an application program for providing a service. For this reason, the communication unit 32 is realized by, for example, a CPU 21, a flash memory 22, a RAM 23, an I / F 26, and a bus 28.

The first storage unit 33 is used for storing, for example, an unmodified image, a first image, case information, a modified image, and the like. The first storage unit 33 corresponds to, for example, a flash memory 22 and a RAM 23.

The second storage unit 34 is used for storing, for example, an image A or a color sample used as an image B. The second storage unit 34 also corresponds to, for example, a flash memory 22 and a RAM 23.

The control unit 31 controls the entire server device 3. This control is realized by the CPU 21 executing a network OS (Operating System) stored in the flash memory 22 or an application program for providing a service. For this reason, the control unit 31 is realized by, for example, a CPU 21, a flash memory 22, a RAM 23, and a bus 28.

The image processing system 100 as the present embodiment is realized by the image correction device 230 and the server device 3 of the image processing device 2.

Assuming that the image retouching device 230 inputs the pre-retouched image from the server device 3 and retouches the pre-retouched image, the image processing system 100 stores the pre-retouched image to be retouched by the server device 3. By doing so, the image before the modification has been acquired. The acquired unmodified image is stored in the first storage unit 33. For this reason, the image acquisition unit of the image processing system 100 is realized by, for example, a control unit 31, a communication unit 32, and a first storage unit 33.

The color sample used as the image A or the image B is stored in the second storage unit 34 and transmitted to the image processing device 2 as needed. From this, the image storage means of the image processing system 100 includes at least a second storage unit 34.

The image retouching device 230 performs retouching on the image before retouching. The image modification device 230 acquires the image before modification and various data necessary for the modification from the server device 3 in order to modify the image before modification. For this reason, the retouching means of the image processing system 100 includes, for example, the control unit 31, the communication unit 32 and the first storage unit 33 of the server device 3, the data acquisition unit 231 and the image input unit 232 of the image retouching device 230. And realized by the retouching unit 234.

The image retouching device 230 receives various necessary data including a pre-retouch image and a color sample (image A and image B, or image B) from the server device 3, and retouches the received pre-retouch image. The image correction device 230 stores the data received from the server device 3, even temporarily. For this reason, the image correction device 230 itself also functions as one image processing system 100. This means that the image processing system 100 can be realized by using only one information processing device.

When the image correction device 230 is regarded as one image processing system, the image acquisition means corresponds to, for example, the image input unit 232. The image storage unit corresponds to, for example, a data acquisition unit 231 and a modification unit. The retouching means is realized by, for example, a data acquisition unit 231, a case selection unit 233, and a retouching unit 234.

Next, the color sample creating device 210 will be described in detail. As shown in FIG. 3, the color sample creating device 210 includes a data acquisition unit 211, an arrangement determination unit 212, a color sample creation unit 213, and a registration unit 214 as functional configurations.

The data acquisition unit 211 is a function of acquiring necessary data including the first image from the server device 3 via the network 4. Necessary data is acquired from the server device 3 under the control of the CPU 21 that executes the color sample creation program 22a stored in the flash memory 22. The first image acquired from the server device 3 is designated by the user who visually recognizes the information displayed on the display device 24 by operating the input device 25. From this, the data acquisition unit 211 is a function corresponding to the image acquisition means, and is realized by, for example, a CPU 21, a flash memory 22, a RAM 23, a display device 24, an input device 25, an I / F 26, and a bus 28.

The arrangement determination unit 212 is a function of referring to the first image designated by the user and determining a pixel group (second pixel group) to be arranged in the second region of the color sample. The arrangement determination unit 212 is realized by, for example, a CPU 21, a flash memory 22, a RAM 23, and a bus 28.

The arrangement determination unit 212 includes a first rearrangement unit 221, a second rearrangement unit 222, and a third rearrangement unit 223. The first sorting unit 221 and the second sorting unit 222, and the third sorting unit 223 are the first sorting means, the second sorting means, and the third sorting means, respectively. Corresponds to.

In the first image, a first area as shown in FIG. 8 is preset or set by the user. This first region is a region referred to when creating a second pixel group to be arranged in the second region of the color swatch.

The arrangement determination unit 212 copies the pixel group (first pixel group) constituting the first region to the RAM 23.

FIG. 9 is a diagram illustrating a first pixel group copied to the RAM 23. In FIG. 9, the RAM 23 is referred to as a “memory”.

Each of the rectangular frames shown in FIG. 9 represents a pixel. As a result, in FIG. 9, the first pixel group copied to the RAM 23 is represented by a row of pixels.

The first sorting unit 221 performs the first sorting of the copied first pixel group based on the brightness (L * value here based on the above assumption). Due to this rearrangement, the first pixel group shown in FIG. 9 is changed as shown in FIG. In the present embodiment, the first rearrangement based on the lightness is performed by rearranging the pixels in descending order of lightness. The first sorting may be performed in the reverse order.

In FIG. 9, the lightness is represented by the color depth. The lighter the color, the higher the brightness.

In the first region, there may be a plurality of pixels having the same brightness. In that case, as shown in FIG. 10, in the first pixel group after the first rearrangement, there are two or more pixels having the same brightness and one or more consecutive points. .. In the present embodiment, in that case, the first rearrangement unit 221 sets the saturation (= √ (a * × a * + b * × b *)), which is an attribute based on human color vision different from the lightness. The second sort based on the basis (corresponding to the "fourth sort" described in the claims) is performed.

In the present embodiment, the second sorting based on the saturation is performed in the form of sorting in descending order of saturation. The target for the second rearrangement is only a plurality of consecutive pixels having the same brightness. By performing such a second rearrangement, the first pixel group is changed from the state shown in FIG. 10 to the state shown in FIG. If a plurality of pixels having the same brightness do not exist, the second rearrangement is not performed.

Even if the first rearrangement and the second rearrangement are performed, the number of pixels of the first pixel group does not change. When the number of pixels in the first pixel group is smaller than the number of pixels in the second pixel group, it is necessary to add the missing pixels to the first pixel group. In the present embodiment, the pixel is added (generated) by using linear interpolation. This pixel addition is also performed by the first rearrangement unit 221. By adding this pixel, the first pixel group after the second rearrangement is changed from the state shown in FIG. 11 to the state shown in FIG. 12, for example, so as to have the number of pixels of the second pixel group. become.

The addition of pixels is the same as the nth pixel from the beginning of the second pixel group is the Nth pixel from the beginning of the first pixel group after the first rearrangement or the second rearrangement. It is performed by linear interpolation so that it becomes a color value. When the number of pixels of the first pixel group is N1 and the number of pixels of the second pixel group is N2, N corresponding to n is obtained by, for example, N = N1 × (n / N2).

When N is an integer, the pixel value V, which is the data of the pixel to be added as the nth pixel, is the pixel value V of the Nth pixel in the first pixel group N after sorting. Interpolate. When N is not an integer, the pixel value V uses the pixel value Va of the a-th pixel, which is the largest integer that does not exceed N, and the pixel value Vb of the b-th pixel, which is the smallest integer that does not fall below N. V = Va + (Vb-Va) × (NA) ... (1)
To be calculated by.

Each pixel on which linear interpolation is performed as described above has data of three elements of the color system of L * (Elster), a * (Aster), and b * (Beaster). Therefore, the calculation of the pixel value V is performed for all of the L * value which is the brightness and the a * value and the b * value which are the chromatic exponents.

In this way, in the present embodiment, when the number of pixels in the first pixel group is smaller than the number of pixels in the second pixel group, the required number of pixels is increased (added). Therefore, it is possible to create a color sample in which all the pixels are arranged in the second region without being limited by the number of pixels of the first pixel group. As the pixel increasing method, a method other than linear interpolation may be used. The number of pixels to be increased may be any number as long as the number of pixels in the first pixel group is equal to or greater than the number of pixels in the second pixel group.

The above interpolation can be applied even when the number of pixels in the first pixel group is larger than the number of pixels in the second pixel group. In this case, since there are pixels that cannot be arranged in the second region, it is not always necessary to perform interpolation.

As described above, the first rearrangement unit 221 generates a first pixel group having the same number of pixels as the number of pixels of the second pixel group. This first pixel group is output as a second pixel group for performing subsequent operations.

The second rearrangement unit 222 rearranges the pixels based on the hue angle with respect to the second pixel group output by the first rearrangement unit 221 (the "claims". Corresponds to "second sort").

This third rearrangement is performed on a line-by-line basis, assuming a row in which the pixels are arranged in the second region in order to consider the positional relationship of the pixels actually arranged in the second region.

FIG. 13 is a diagram illustrating an example of a second pixel group immediately before performing the third rearrangement.

Each numerical value of "8", "14", and "16" shown in FIG. 13 conveniently represents, for example, a pixel number assigned to each pixel constituting the second pixel group. A frame (pixel) in which any numerical value is written in the vicinity is a pixel whose color is similar or whose brightness, saturation, and hue angle are all similar, that is, each pixel within a predetermined allowable range. It is a pixel in which the value or the value calculated from each pixel value matches. Here, in order to facilitate understanding, the color of the frame (pixel) in which any numerical value is written in the vicinity is made different from the color of the other frame (pixel).

In the second pixel group generated by the first rearrangement unit 221, the pixels are rearranged according to the lightness (in some cases, the saturation). However, when the second pixel group is simply arranged in the second region, as shown in FIG. 13, there is a possibility that pixels having similar hue angles are arranged apart from each other.

Pixels so spaced apart are relatively likely to appear in a color that is significantly different from the surrounding pixels. For example, if the colors of the pixels arranged apart from each other are red (or green) and the colors of the peripheral pixels are green (or red), the hue angle is 180 degrees or an angle close to 180 degrees. For this reason, when pixels having similar hue angles are arranged apart from each other, it is easy to give the observer a grainy feeling. Therefore, in the present embodiment, the third rearrangement based on the hue angle is performed so as to further suppress giving a grainy feeling to the observer.

FIG. 14 is a diagram illustrating an example of a second pixel group after performing the third rearrangement.

As shown in FIG. 14, by performing the third rearrangement, the pixels arranged at the position where the pixel number is 8 in FIG. 13 are moved to the position where the pixel number is 7, and the pixel number is 7. It becomes a pixel. Similarly, in FIG. 13, each pixel arranged at the positions where the pixel numbers are 14 and 16 moves to the positions where the pixel numbers are 16 and 17, respectively, and becomes the pixels whose pixel numbers are 16 and 17. In this way, pixels with close hue angles are gathered in the vicinity, in other words, within a narrow range, so that the observer is not given a grainy feeling, or even if it is given a grainy feeling, the degree is further suppressed. become. Therefore, the created color sample also has less noise and feels more natural.

By the third rearrangement, it is possible to collect pixels having similar hue angles in a plurality of adjacent rows. However, pixels whose colors are similar, or whose lightness, saturation, and hue angle are all similar (pixels in which each pixel value within an allowable range or a value calculated from each pixel value matches. Notated as "approximate pixels") may be on distant rows. The possibility is particularly high when the ratio of the number of pixels for each hue angle in the pixel group of each row is not within a narrow range.

FIG. 15 is a diagram for explaining a second pixel group example after the third rearrangement in a wider range. In FIG. 15, as in FIGS. 13 and 14, the approximate pixel of interest is represented by having a different color from the other pixels. As a result, in FIG. 15, the approximate pixel with the pixel number 43 is arranged at a position away from the group including the three approximate pixels with the pixel numbers 7, 16 and 17.

Even if the approximate pixels are arranged at distant positions in the overlapping direction of the rows, it is easy to give the observer a grainy feeling for the same reason as described above. From this, in the present embodiment, the second rearrangement after the third rearrangement is performed, and the second rearrangement is performed, and the pixels are rearranged based on the color. (Corresponding to the "third rearrangement") is performed to further suppress giving a grainy feeling to the observer. As a result, the color swatches created by performing this fourth rearrangement will feel more natural.

The third rearrangement unit 223 performs the fourth rearrangement on the second pixel group after the third rearrangement. This rearrangement is performed in a form in which pixels arranged at distant positions in the direction in which the rows overlap are densely packed.

FIG. 16 is a diagram illustrating an example of a second pixel group after performing the fourth rearrangement.

As shown in FIG. 16, by performing the fourth rearrangement, the approximate pixel arranged at the position where the pixel number is 43 in FIG. 15 is moved to the position where the pixel number is 29, and the pixel number is 29. It becomes a pixel of. In FIG. 15, the pixel arranged at the position where the pixel number is 29 moves to the position where the pixel number is 43, and becomes the pixel whose pixel number is 43. By such replacement, more pixels having similar colors are arranged in a narrow range, so that a color sample that does not give a graininess (noise) can be created.

By performing the fourth rearrangement, the arrangement of all the pixels in the second region is determined. The creation unit 213 creates a color sample by arranging pixels in an area other than the second area. From this, the creating unit 213 corresponds to the image creating means. The creation unit 213 is realized by, for example, a CPU 21, a flash memory 22, a RAM 23, and a bus 28.

FIG. 17 is a diagram showing a created color sample example.

As described above, the first rearrangement based on the lightness is performed by rearranging the pixels in descending order of lightness. As a result, as shown in FIG. 17, the pixel with high brightness is arranged in the upper part of the second region. This is because in general space, objects that are light sources such as the sun or electric lamps are often located above objects that are not light sources, and in many images the upper part is bright. .. In other words, by arranging the pixels in the second region according to the features common to many images, a color sample that gives a more natural impression is created.

The first pixel group, organized by sorting based on lightness (and saturation), focuses on the hue angle and color, and is placed in an undesired position in the first pixel group due to human visual characteristics. Pixels are rearranged. In the rearrangement, a pixel whose placement position is not desirable is moved, and one or more pixels are also moved according to the movement (FIG. 14), or the pixel is replaced with the pixel to which the pixel is moved (FIG. 16). In the second region of the color swatch created by such rearrangement, a second pixel group in which the pixels are arranged so as to be organized and feel extremely natural in terms of human visual characteristics is formed. It will be arranged more reliably.

If the pixels to be arranged in an area other than the second area are predetermined, the creating unit 213 can be omitted. In that case, the third rearrangement unit 223 or the arrangement determination unit 212 including the third rearrangement unit 223 creates the color sample.

The registration unit 214 is a function of transmitting a color sample created by the creation unit 213 and registering (storing) it in the server device 3. When the color sample is registered according to the user's instruction, the registration unit 214 is realized by, for example, a CPU 21, a flash memory 22, a RAM 23, a display device 24, an input device 25, an I / F 26, and a bus 28.

The image retouching device 230 mounted on the image processing device 2 retouches the pre-retouching image using the color samples created as described above as images A and B or as a reference image (image B). The created color swatch more closely reproduces the impression of the subject's color when viewed in an actual environment. Therefore, the image retouching device 230 creates a retouched image in which the color of the object on the pre-retouched image is retouched to the actual color of the impression that the observer who directly sees the object receives, or a color close to the actual color. be able to.

An example of creating a color sample using the first image will be specifically described below.

(First Example)
FIG. 18 is a flowchart showing a procedure for creating a color sample image according to the first embodiment. The creation procedure shown in the flowchart in FIG. 18 is realized by the CPU 21 executing the color sample creation program 22a. In FIG. 18, for convenience, in order to determine the arrangement of pixels in the second region included in the color sample to be created by the CPU 21 after the user specifies the first image and instructs the creation of the color sample. Only the control procedure to be executed is shown in. As a result, the arrangement determination unit 212 is realized by the CPU 21 executing the control procedure shown in FIG. In FIG. 18, RAM 23 is referred to as “memory”.

First, the CPU 21 copies the pixels existing in the first area in the first image received from the server device 3 to the area secured in the RAM 23 (hereinafter referred to as "first area") (step S1). ).

Next, the CPU 21 performs the first rearrangement on the copied pixels, that is, the first pixel group (step S2). By this first rearrangement, the first pixel group copied as shown in FIG. 9 is changed as shown in FIG.

Next, the CPU 21 confirms a portion of the first pixel group after the first rearrangement in which pixels of equal brightness are continuous, and when that portion is confirmed, the pixel at that location is confirmed. Is rearranged in order of saturation (step S3). By this second rearrangement, the first pixel group as shown in FIG. 10 is changed as shown in FIG.

After that, the CPU 21 secures an area (hereinafter, referred to as “second area”) for storing the second pixel group arranged in the second area in the RAM 23 (step S4).

Next, when the number of pixels in the first pixel group is smaller than the number of pixels in the second pixel group, the CPU 21 uses linear interpolation to add the insufficient pixels as shown in FIG. The pixel value of the added pixel is determined (step S5). As a result, the second pixel group as shown in FIG. 13 is stored in the second area.

The steps S2 to S5 are procedures related to the first sorting unit 221. When the CPU 21 realizes the function of the first sorting unit 221, the CPU 21 executes the procedure of steps S2 to S5.

Next, the CPU 21 performs a third rearrangement in which the pixels are rearranged in the order of hue angles in units of rows existing in the second region (step S6). As a result, the second pixel group as shown in FIG. 13 is changed as shown in FIG. 14 or 15. This step S6 is a procedure relating to the second rearrangement unit 222.

Next, the CPU 21 performs a fourth rearrangement for arranging pixels of approximate colors dispersed in the direction in which the rows overlap with respect to the second pixel group so as to be closer to each other (step S7). As a result, the second pixel group as shown in FIG. 15 is changed as shown in FIG. After that, the control procedure shown in FIG. 18 ends. This step S7 is a procedure relating to the third rearrangement unit 223.

The fourth rearrangement in step S7 may be performed as needed, as described above. As an index used to automatically determine the necessity of performing the fourth sorting, for example, graininess can be mentioned.

To calculate the granularity, the second region is divided into the L *, a *, and b * power spectra obtained by FFT (Fast Fourier Transform) for each L *, a *, and b * value of the pixel. Techniques such as weighting according to the spatial frequency characteristics of color vision (contrast sensitivity characteristics (CSF)) and taking linear sums can be used (eg, K. Kagitani, H. Makoto, AND S. Imagawa, "Image noise evolution method". for color hardcopy ", in Spec. IS & T NIP12, 1996, pp. 173-76.). The determination as to whether or not to perform the fourth sorting may be performed depending on whether or not the calculated granularity value is equal to or greater than a predetermined value. As the predetermined value, for example, if the granularity is calculated by the method given in the example, it may be 0.1.

The user may be allowed to select whether or not to perform the fourth sorting. The selection by the user may be made visually by displaying a color sample in which the second pixel group after the third rearrangement is arranged in the second region on the display device 24.

The fourth sort itself may be performed by a self-organizing map (SOM: Self-Organizing Maps). As a method of self-organizing, an algorithm proposed by T. Kohonen is known (for example, T. Kohonen, "Self-Organizing Maps", Springer, 2000, etc.).

The self-organizing map for the fourth sorting may be performed by, for example, the following procedure.

First, as step 1, a color sample in which the second pixel group after the third rearrangement is arranged in the second area is created in the area secured in the RAM 23 (hereinafter, “third area”). Hereinafter, the created color sample is referred to as a "third image", and the second region in the third image and the second pixel group arranged in the second region are each "third region". It is expressed as "third pixel group".

Next, as step 2, the color difference between the pixel number of the second region (second pixel group) and the pixel i (hereinafter referred to as "pixel i") is the smallest, and the pixel i is within the third region. The pixel I having the closest positional relationship with is searched for. The positional relationship here is a case where the second region and the third region are evaluated assuming that they are the same space.

Next, as step 3, the values of L *, a *, and b * of the pixel I and each pixel J in the vicinity thereof centered on the pixel I are changed in the third region.

The above steps 2 and 3 are carried out for all the pixels i in the second region, and this is repeated for a predetermined maximum number of repetitions t.

The values L (t), a (t), and b (t) of L *, a *, and b * after being changed t times, the values L (t + 1), a (t + 1), and the values to be set next to b (t), and b (t + 1) may be calculated as follows, for example.
L (t + 1) = L (t) + h (t) · dL (J, i) ... (2)
a (t + 1) = a (t) + h (t) · da (J, i) ... (3)
b (t + 1) = b (t) + h (t) · db (J, i) ... (4)
However, dL (J, i), da (J, i), and db (J, i) are the L * value and a * value between the pixel i in the second region and the pixel J in the third region, respectively. , B * value difference, h (t) is a function of the distance between pixel I and pixel J.

h (t) is, for example, the following function.
h (t) = α (t) · exp (d2 / (2 · (σ (t) 2)) ... (5)
Here, α (t) is a monotonous decrease function with respect to the number of repetitions t, d is the distance between the pixel I and the pixel J, and σ (t) is a constant at each number of repetitions. α (t) is a function such as α (t) = 1-t / (T + 1).

The difference dL (J, i), da (J, i), and db (J, i) are, for example, the CIE1976 color difference equation represented by the Euclidean distance between two points in the CIE (1976) L * a * b * space. The value calculated by can be used. A value calculated by another color difference formula (CIE1994 color difference formula, etc.) may be used.

In Kohonen's self-organizing map, it is usual that a pixel group having the same number of pixels as the second pixel group is first secured separately, and the initial value of each secured pixel is assigned by a random number. However, the first to third sorts are all means for organizing the pixel group. For this reason, in the fourth rearrangement by the self-organizing map, the above-mentioned procedure is adopted so that the organized state of the second pixel group can be more maintained.

(Second Example)
In the first embodiment, the fourth rearrangement is performed with the entire second region as the movable range of the pixels. In the second embodiment, the movable range of the pixels is limited.

FIG. 19 is a diagram illustrating a method of limiting the movable range of pixels.

In the second embodiment, as shown in FIG. 19, the second region is virtually divided into a plurality of subregions, and the movement of the pixels is restricted to the subregions in which the pixels exist. As a result, the fourth rearrangement is performed for each partial area.

When the second region is divided into a plurality of partial regions, the number of pixels in each partial region is reduced, so that the search for the destination of the pixel to be moved can be performed at high speed. Even if the fourth sorting is performed in all the partial areas, the processing can be performed at a higher speed than in the case where the fourth sorting is performed for the entire second area.

The reason why the second region is divided into a plurality of subregions is that when the third rearrangement is performed, even if the pixels of similar colors are dispersed, those pixels are present in a relatively narrow region. This is because there are many things to do. From this, even if the fourth rearrangement is performed for each partial region, the pixels to be moved can be appropriately moved.

When the second region is divided into a plurality of subregions, the necessity of the fourth reordering is determined for each subregion from the viewpoint of further speeding up the processing, and each subregion is individually determined according to the determination result. It is desirable to do it. The automatic determination of whether or not to perform the fourth sort for the second region is, for example, calculating the granularity for each partial region and statistic such as the average value or the maximum value of the calculated granularity. Is further calculated, and the calculated statistic may be used. The judgment may be made by the user.

It is conceivable that the second area is divided according to the setting by the user. Further, even if the degree of color change of the entire second region is confirmed and the size or shape of the partial region is automatically set according to the confirmation result, the second region is divided. good.

FIG. 20 is a flowchart showing a procedure for creating a color sample image according to the second embodiment. The creation procedure shown in the flowchart in FIG. 20 is also realized by the CPU 21 executing the color sample creation program 22a. Again, FIG. 20 shows only the control procedure that the CPU 21 executes to determine the placement of pixels in the second region after the user has specified a first image and instructed to create a color swatch. There is. In FIG. 20, the RAM 23 is referred to as a “memory”.

In FIG. 20, the procedures having the same or almost the same contents as those in the first embodiment are designated by the same reference numerals as those in FIG. As a result, in the description of FIG. 20, the procedure with a reference numeral which does not exist in FIG. 18 will be focused on.

In the second embodiment, after executing the procedure of step S6, the process proceeds to step S11. In step S11, the CPU 21 virtually divides the second area into a plurality of partial areas, for example, as shown in FIG.

Next, the CPU 21 calculates the granularity for each partial region and extracts the maximum granularity (maximum granularity) among the calculated granularity (step S12).

Next, the CPU 21 determines whether or not the maximum granularity is equal to or less than a predetermined value (step S13). When the maximum granularity is equal to or less than a predetermined value, the determination in step S13 is yes, and the control procedure shown in FIG. 20 ends here.

On the other hand, when the maximum granularity exceeds a predetermined value, the determination in step S13 is no, and the process proceeds to step S14.

In step S14, the CPU 21 uses the self-organizing map to perform the fourth rearrangement in units of partial regions. The fourth rearrangement may be performed in units of partial regions, for example, by comparing a predetermined value different from the predetermined value referred to in step S13 with the granularity of each partial region. After performing the fourth rearrangement, the control procedure shown in FIG. 20 ends.

As described above, according to the present embodiment, in the color swatch, all the pixels included in the predetermined region of the original drawing (first image) are abstractly organized and rearranged in the second region. As a result, the impression received from the appearance of colors in an actual environment can be reproduced more approximately.

That is, the color sample creating device 210 mounted on the image processing device 2 rearranges the pixel groups arranged in the first region in the first image which is the original drawing based on the brightness. Assuming the sorting of 1 and the rows in the second area, at least the second sorting of sorting based on the hue angle and the third sorting of sorting based on the color are performed for each row, and the sorting is performed. As a result, the obtained pixel group is arranged in the second region.

As a result, even if the image before modification is a photograph, the color sample creating device 210 assumes an actual photograph and organizes the pixels in an abstract manner, and at the same time, the position which is not desirable in the organization due to human visual characteristics. It is possible to create a color sample to be created having a pixel group in which the pixels arranged in the above are arranged at the optimum positions in the second region (selected region). Therefore, the impression of the color swatch can be very close to that of the object existing on the unmodified image when viewed in an actual environment.

Therefore, when the unmodified image (input image) such as a photograph is modified using such a color sample, it becomes possible to compare the shapes according to the visual characteristics of a person. As a result, the modification to the pre-modification image can be performed so as to more closely reproduce the impression received from the appearance of the color in the actual environment.

Further, by using such a color sample, the image processing system 100 enables the user to more intuitively select a case. As a result, the convenience of operation is improved, and the user can obtain the desired modified image more easily and quickly.

Although a photograph is taken as an example of the image before modification, the image before modification is not limited to the photograph. The type of object on the unmodified image is also not particularly limited.

Further, the color sample does not have to be rectangular. The shape of the second region is also not limited to the circle.

As described above, the present invention has the effect of being able to more approximately reproduce the impression received from the appearance of colors in an actual environment, and has a color sample creating device, a color sample creating method, and image processing using a color sample. Useful for the entire system.

2 Image processing device 3 Server device 21 CPU
22 Flash memory 22a Color sample creation program 22b Image modification program 24 Display device 25 Input device 26, 27 I / F
31 Control unit 32 Communication unit 33 First storage unit 34 Second storage unit 100 Image processing system 210 Color sample creation device 211 Data acquisition unit 212 Arrangement determination unit 213 Creation unit 214 Registration unit 221 First rearrangement unit 222 2 Sorting unit 223 3rd sorting unit 230 Image retouching device 231 Data acquisition unit 232 Image input unit 233 Case selection unit 234 Retouching unit 235 Data output unit

Japanese Unexamined Patent Publication No. 2001-0745556

Claims (1)

A second pixel group consisting of a second pixel group generated from a first pixel group consisting of the first pixel number constituting the first region in the first image and not necessarily matching the first pixel group. A color swatch that includes a second region in which the pixels of
Each pixel constituting the first pixel group is rearranged in descending order of brightness, and the first rearrangement is performed.
When the number of the first pixel and the number of the second pixel do not match, the number of pixels is the same as the number of the second pixel by performing linear interpolation on each pixel with brightness, saturation, and hue angle. Adjust the number of pixels so that
When the rows in the second region are assumed for each pixel after the first rearrangement and the adjustment of the number of pixels as necessary, and the pixels having the same brightness are consecutive for each row, Perform a second sort to sort these in order of saturation,
Further, after the second rearrangement, when pixels having the same brightness and saturation are consecutive for each row, a third rearrangement is performed in which these are rearranged in the order of hue angle, and within the second region. A color swatch characterized by being placed in.