JP2010081057A - Image processing apparatus and method, and colorimetry system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To precisely correct image data of a photographed object even if photographing conditions change during photographing. <P>SOLUTION: An image processing apparatus 3 is provided, which is provided with: a calibration data setting unit 123 which establishes color indicator image data in a temperature near to a temperature of the photographed object upon photographing as the calibration data, out of temperatures between a temperature of an image capturing apparatus 1 in the time of photographing the color indicator before imaging the photographed object and a temperature of the image capturing apparatus 1 upon photographing the color indicator after imaging the photographed object, based on the temperature of the image capturing apparatus 1 when the image capturing apparatus 1 photographs the photographed objects, the temperature of the image capturing apparatus 1 upon imaging the color indicator before and after imaging the photographed object, and image data of the color indicator acquired by imaging the color indicator before and after imaging the photographed object; and an image correction unit 71 which corrects the image data of the photographed object acquired by imaging the photographed object using the calibration data established in the calibration data setting unit 123. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and a color measurement system.

  Conventionally, calibration conditions are stored in advance in association with printing conditions such as the device temperature, and at the time of printing, the temperature closeness at the time of printing and at the time of calibration is determined, and calibration with high approximation is performed. A calibration device that calibrates print image data using data is known (see, for example, Patent Document 1).

JP 2007-221703 A

  However, in a conventional apparatus that acquires calibration data in advance, it is not possible to calibrate print image data considering fluctuations in shooting conditions during shooting and fluctuations in the state of the imaging apparatus, and correction accuracy is high. There is an inconvenience of lowering.

  The present invention has been made in view of the above-described circumstances, and an image processing apparatus, an image processing method, and a measurement method that can accurately correct image data of a subject even when shooting conditions change during shooting. It aims to provide a color system.

In order to solve the above problems, the present invention employs the following means.
The present invention relates to the temperature of the imaging device at the time of photographing the subject by the imaging device, the temperature of the imaging device at the time of photographing the color chart before and after photographing the subject, and before and after photographing the subject. Based on the color chart image data acquired by photographing the color chart, the imaging at the time of photographing the color chart after photographing the subject from the temperature of the imaging device at the time of photographing the color chart before photographing the subject. A calibration data setting unit for setting color chart image data at a temperature closer to the temperature at the time of photographing the subject as calibration data among temperatures up to the temperature of the apparatus, and calibration data set in the calibration data setting unit And an image correction unit that corrects image data of the subject acquired by photographing the subject. To provide an image processing apparatus comprising.

  According to the present invention, when the subject and color chart image data acquired by the imaging device and the temperature of the imaging device at the time of acquisition of these image data are input, the calibration data setting unit captures the subject. The color chart image data at a temperature close to this temperature is set as calibration data, and the image data of the subject is corrected by the image correction unit using the set calibration data. Since the color chart image data is acquired both before and after the shooting of the subject, if the temperature at the time of shooting changes, the shooting conditions and the state of the imaging device change according to the temperature change. It can be estimated that the calibration data suitable for correcting the image data of the subject also changes between the two color chart image data acquired before and after the photographing of the subject. Therefore, among the temperatures from the temperature of the imaging device at the time of photographing the color chart before photographing the subject to the temperature of the imaging device at the time of photographing the color chart after photographing the subject, it depends on the temperature at the time of photographing the subject. By setting color chart image data at a close temperature as calibration data, it is possible to obtain image data of a subject that is accurately corrected regardless of variations in shooting conditions during shooting.

In the above-described invention, the calibration data setting unit uses the color chart image data acquired before and after photographing the subject as the color chart image data obtained before and after photographing the subject. It is also possible to generate color chart image data at the temperature at which the subject is photographed by interpolation based on the temperature, and set this as calibration data.
In this way, optimum calibration data can be obtained even when shooting conditions or the like change during shooting, and the image data of the subject can be corrected with high accuracy.

Further, in the above invention, the calibration data setting unit calculates the temperature of the imaging device at the time of photographing the subject and the temperature of the imaging device at the time of photographing the color chart before and after photographing the subject. In contrast, color chart image data corresponding to a temperature closer to the temperature of the imaging device at the time of photographing the subject may be set as calibration data.
In this way, calibration data suitable for correcting the image data of the subject can be set and corrected with high accuracy by simple processing.

  Further, in the above invention, the calibration data setting unit, when the temperature of the imaging device at the time of photographing the color chart before and after photographing the subject fluctuates beyond a predetermined threshold, Color chart image data at an intermediate temperature may be generated, and color chart image data corresponding to a temperature closer to the temperature of the imaging device at the time of photographing the subject may be set as calibration data.

  In this way, if the temperature greatly fluctuates beyond a predetermined threshold before and after photographing the subject, a large difference also occurs in the color chart image data acquired before and after that, so calibration is performed. By generating intermediate color chart image data in the data setting unit, the difference between the color chart image data can be reduced, and the calibration data can be set more finely according to the shooting conditions of the subject.

  Further, the present invention provides the temperature of the imaging device when the subject is photographed by the imaging device, the temperature of the imaging device before photographing the subject and the color chart when photographing the subject, and before and after photographing the subject. Based on the color chart image data acquired by photographing the color chart later, color chart data at a temperature close to the temperature at the time of photographing the subject is set as calibration data, and the set calibration data is used. An image processing method for correcting image data of a subject acquired by photographing the subject is provided.

  In addition, the present invention provides an imaging device including an imaging unit that captures a subject and a color chart, and a temperature measurement unit that measures the temperature of the imaging unit at the time of shooting, and processes image data acquired by the imaging device. A color measurement system comprising the image processing apparatus according to any one of claims 1 to 4 is provided.

  According to the present invention, there is an effect that image data of a subject can be accurately corrected even when shooting conditions or the like change during shooting.

Hereinafter, an image processing apparatus and a color measurement system according to an embodiment of the present invention will be described with reference to the drawings.
The color measurement system according to this embodiment is a dental color measurement system, and as shown in FIGS. 1 and 4, an imaging device 1, a cradle 2, a dental color measurement device (image processing device) 3, and a monitor 4. And an input device 5.

As illustrated in FIG. 1, the imaging device 1 includes a light source 10, an imaging unit 20, an imaging control unit 30, a display unit 40, and an operation unit 50.
The light source 10 is disposed in the vicinity of the tip of the photographing apparatus 1 and emits illumination light having four or more different wavelength bands for illuminating the subject.

  For example, the light source 10 includes seven light sources 10a to 10g that emit light in different wavelength bands. Each of the light sources 10a to 10g includes four LEDs. As shown in FIG. 2, the center wavelengths of the light sources 10a to 10g are about 450 nm for the light source 10a (λ1), about 465 nm for the light source 10b (λ2), about 505 nm for the light source 10c (λ3), and about 505 nm for the light source 10d (λ4). The light source 10e (λ5) is about 575 nm, the light source 10f (λ6) is about 605 nm, and the light source 10g (λ7) is about 630 nm. The emission spectrum information of the LED is stored in the LED memory 11 and used in the dental color measuring device 3 described later.

  For example, the LEDs of the light sources 10a to 10g are divided into a plurality of groups. Each group is disposed at a position facing the imaging unit 20. The arrangement is not particularly limited, and for example, four LEDs may be arranged in the order of shorter wavelengths, or may be arranged in reverse order or at random. In addition to the arrangement in which the LEDs are divided into a plurality of groups, all the LEDs may be arranged so as to form one ring surrounding the imaging unit 20. In addition, as long as the LED arrangement does not hinder imaging by the imaging unit 20 described later, an appropriate arrangement such as a linear arrangement, a ring arrangement, a cross arrangement, a rectangular arrangement, or a random arrangement is adopted. It is possible. The light emitting element of the light source 10 is not limited to an LED, and for example, a semiconductor laser such as an LD (laser diode) or other light emitting elements can be used.

  In the imaging apparatus 1, an illumination optical system for irradiating illumination light from the light source 10 onto the subject surface substantially uniformly is provided on the subject side of the light source 10. Further, in the vicinity of the light source 10, a temperature sensor (temperature measurement unit) 13 for detecting the temperature of the LED is provided.

  The imaging unit 20 includes an imaging lens 21, an RGB imaging device 22, a signal processing unit 23, and an A / D converter 24. The imaging lens 21 forms a subject image irradiated with illumination light from the light source 10. The RGB imaging device 22 captures the subject image formed by the imaging lens 21 and outputs an image signal. The RGB imaging element 22 is constituted by, for example, a CCD, and the sensor sensitivity is such that it substantially covers the visible light range. This CCD may be a monochrome type or a color type. The RGB image sensor 22 is not limited to the CCD, and a CMOS type or other various image sensors can be widely used.

  The signal processing unit 23 performs gain correction, offset correction, and the like on the analog signal output from the RGB imaging element 22. The A / D converter 24 converts the analog signal output from the signal processing unit 23 into a digital signal. A focus lever 25 for adjusting the focus is connected to the imaging lens 21. The focus lever 25 is for adjusting the focus manually. Further, a position detection unit 26 for detecting the position of the focus lever 25 is provided.

  The imaging control unit 30 includes a CPU 31, an LED driver 32, a data I / F 33, a communication I / F controller 34, an image memory 35, and an operation unit I / F 36. Each of these units is connected to the local bus 37 and is configured to be able to exchange data via the local bus 37.

The CPU 31 controls the imaging unit 20, records the subject spectral image acquired and processed by the imaging unit 20 in the image memory 35 via the local bus 37, and outputs it to the LCD controller 41 described later.
The LED driver 32 controls light emission of various LEDs included in the light source 10.
The data I / F 33 is an interface for receiving the contents of the LED memory 11 provided in the light source 10 and the information of the temperature sensor 13.

The communication I / F controller 34 is connected to a communication I / F contact 61 serving as a connection point with the outside, and has a function for realizing communication compliant with USB 2.0, for example.
The image memory 35 temporarily records image data captured by the imaging unit 20. In the present embodiment, the image memory 35 has a memory capacity capable of recording at least seven spectral images and one RGB color image.
The operation unit I / F 36 is connected to various operation buttons provided in the operation unit 50 to be described later, and functions as an interface for transferring an input instruction input via the operation unit 50 to the CPU 31 via the local bus 37.

The display unit 40 includes an LCD controller 41 and an LCD (liquid crystal display) 42.
The LCD controller 41 causes the LCD 42 to display an image based on the image signal transferred from the CPU 31, for example, an image currently captured by the imaging unit 20 or an image that has been captured.

  At this time, if necessary, the image pattern stored in the overlay memory 43 may be superimposed on the image acquired from the CPU 31 and displayed on the LCD 42. Here, the image pattern recorded in the overlay memory 43 is, for example, a horizontal line that captures the entire tooth horizontally, a cross line that intersects this, an imaging mode, an identification number of the tooth to be imaged, or the like. .

  The operation unit 50 includes various operation switches and operation buttons for a user to input an instruction to start a spectral image shooting operation and to input an instruction to start and end a moving image shooting operation. Specifically, a shooting mode changeover switch 51, a shutter button 52, a viewer control button 53, and the like are provided. The shooting mode switch 51 is a switch for switching between normal RGB shooting and multiband shooting. The viewer control button 53 is a switch for performing operations such as changing the image displayed on the LCD 42.

  Moreover, the imaging device 1 is equipped with a lithium battery 60 as a storage battery. The lithium battery 60 is used to supply power to each unit included in the imaging device 1 and is connected to a charging contact 62 that is a contact for charging. In addition, the imaging apparatus 1 is provided with a battery LED 63 for notifying the charging state of the lithium battery 60, a power LED 64 for notifying the state of the camera, and an alarm buzzer 65 for notifying danger during photographing.

The battery LED 63 includes, for example, three LEDs of red, yellow, and green, notifies that the charging capacity of the lithium battery 60 is sufficient by lighting in green, and in other words, indicates that the battery capacity is low by lighting in yellow. The fact that charging is necessary is notified, and the fact that the battery capacity is extremely low by lighting in red, in other words, the fact that urgent charging is required is notified.
The power LED 64 includes, for example, two LEDs of red and green, and notifies that the preparation for shooting is completed by lighting in green, and that the preparation for shooting (initial warming, etc.) is in progress by blinking green. Then, it is notified that the battery is being charged by lighting in red.
The alarm buzzer 65 notifies that the captured image data is invalid by warning.

  The cradle 2 that supports the imaging device 1 described above includes a color chart 100 for calibrating the imaging unit 20, a micro switch 101 for confirming whether the imaging device 1 is mounted at a normal position, The power supply 102 for turning on / off the power, the power lamp 103 which is turned on / off in conjunction with the on / off of the power switch 102, and the light on / off in conjunction with the microswitch 101, the imaging apparatus 1 A mounting lamp 104 for notifying whether or not the camera is mounted at the normal position is provided.

  For example, when the imaging apparatus 1 is mounted at a normal position, the mounting lamp 104 is lit green, and when it is not mounted, the mounting lamp 104 is lit red. In addition, the cradle 2 is provided with a power connection connector 105 to which an AC adapter 106 is connected. Then, when the charging capacity of the lithium battery 60 included in the imaging device 1 is reduced and the yellow or red of the battery LED 63 is lit, charging of the lithium battery 60 starts when the imaging device 1 is mounted on the cradle 2. It is configured to be.

  In the imaging device 1 of the colorimetric system configured in this way, illumination light of four or more wavelength bands (illumination light of four or more primary colors) is sequentially irradiated onto the subject, and the same number as the number of wavelength bands of the illumination light. In addition to multiband imaging that captures a subject spectral image as a still image, it is also possible to capture RGB images. As one of the methods for capturing RGB images, as in a normal digital camera, there is no illumination light of the seven primary colors, and an RGB color CCD having an object illuminated by natural light or room light is used. Imaging is performed. In addition, one or more illumination lights from the seven primary colors can be selected as RGB three-color illumination lights, and these can be sequentially irradiated to capture a frame-sequential still image. Yes.

Among these imaging modes, RGB imaging is used when imaging a wide area, such as when imaging the entire patient's face or when imaging the entire jaw. On the other hand, multiband imaging is used when measuring the color of one or two teeth of a patient accurately, that is, when measuring the color of teeth.
Hereinafter, a color measurement process of teeth by multiband imaging will be described.

[Multiband shooting]
First, color chart image data is acquired prior to photographing of a subject by the imaging apparatus 1.
The color chart image data is acquired by photographing the color chart 100 by the imaging device 1 installed in the cradle 2 in the same procedure as multiband photographing described later. Thereby, the multiband image data of the color chart 100 is recorded as the first color chart image data in the image memory 35 together with the temperature data detected by the temperature sensor 13 when the color chart image data is acquired.

  Next, the imaging device 1 is picked up from the cradle 2 by a doctor, and a contact cap is attached to an attachment port (not shown) provided on the projection port side of the housing of the imaging device 1. The contact cap is formed in a substantially cylindrical shape from a flexible material.

  Subsequently, when the photographing mode is set to the “colorimetry mode” by the doctor, the subject image is displayed on the LCD 42 as a moving image. While confirming the image displayed on the LCD 42, the doctor positions the patient's tooth to be measured at an appropriate position in the imaging range, and performs focus adjustment using the focus lever 25. At this time, since the contact cap is formed in a shape that guides the tooth to be measured to an appropriate photographing position, positioning can be easily performed.

  When the doctor presses the shutter button 52 in a state where the positioning and focus adjustment have been performed, a signal to that effect is transferred to the CPU 31 via the operation unit I / F 36, and multiband imaging is executed under the control of the CPU 31. Is done. When multiband shooting is executed, the temperature at each multiband shooting is detected by the temperature sensor 13.

  In multi-band imaging, the LED driver 32 sequentially drives the light sources 10a to 10g, so that LED irradiation light having different wavelength bands is sequentially irradiated onto the subject. The reflected light of the subject is imaged on the surface of the RGB imaging element 22 of the imaging unit 20 and is captured as an RGB image. The captured RGB image is transferred to the signal processing unit 23.

  The signal processing unit 23 performs predetermined image processing on the input RGB image signal and selects predetermined one color image data corresponding to the wavelength bands of the light sources 10a to 10g from the RGB image signal. Specifically, the signal processing unit 23 selects B image data from the image signals corresponding to the light sources 10a and 10b, selects G image data from the image signals corresponding to the light sources 10c to 10e, and selects the light sources 10f, 10f, R image data is selected from the image signal corresponding to 10g. Thus, the signal processing unit 23 selects image data having a wavelength that approximately matches the center wavelength of the irradiation light.

  The image data selected by the signal processing unit 23 is transferred to the A / D converter 24 and recorded in the image memory 35 via the CPU 31. At this time, temperature data at the time of photographing transferred from the temperature sensor 13 via the data I / F 33 and the local bus 37 is also recorded in the image memory 35 together with the image data.

As a result, an image of a color selected from the RGB image corresponding to the center wavelength of the LED is recorded in the image memory 35 as multiband image data together with the temperature data. That is, the multiband image data is data obtained by capturing an image of a color corresponding to the center wavelength of each primary color included in the illumination light from the light source 10 for each of a plurality of primary colors. When photographing, the CPU 31 controls the irradiation time of the LED, the irradiation intensity, the electronic shutter speed of the image sensor, etc. so that the photographing of each wavelength is properly exposed, and the temperature change during the photographing is severe. In this case, the alarm buzzer 65 sounds and issues a warning.
Note that an image of a tooth may be further taken without irradiating illumination light from the LED, and recorded in the image memory 35 as external light image data.

Subsequently, when the imaging is finished and the imaging device 1 is placed on the cradle 2 by the doctor, the color chart image data is acquired again.
As a result, the multiband image data of the color chart 100 is recorded as the second color chart image data in the image memory 35 together with the temperature data at the time of obtaining the color chart image data.

  Subsequently, the color chart 100 is shot in a state where the LED is not lit at all (under dark), and this image data is recorded in the image memory 35 as dark current image data. Note that this dark current image data may be obtained by performing image capturing a plurality of times and averaging the image data.

  Subsequently, signal correction using the external light image data and dark current image data recorded in the image memory 35 is performed on each of the multiband image data and the color chart image data. The signal correction for the multiband image data is performed, for example, by subtracting the signal value of the external light image data for each pixel from the image data of the multiband image data, thereby removing the influence of external light during shooting. be able to. Similarly, signal correction for color chart image data is performed, for example, by subtracting the signal value of dark current image data for each pixel from the image data of color chart image data, and the darkness of the CCD, which changes with temperature. Current noise removal can be performed.

FIG. 3 shows an example of the signal correction result for the color chart image data. In FIG. 3, the vertical axis indicates the sensor signal value, the horizontal axis indicates the input light intensity, the solid line indicates the original signal before correction, and the broken line indicates the signal after signal correction.
The multiband image data and the color chart image data after the signal correction is performed in this manner are associated with the temperature data detected by the temperature sensor 13 at the time of photographing each image data, and then the local bus 37, The data is transmitted to the dental color measuring device 3 via the communication I / F controller 34 and the communication / I / F contact 61 and recorded in the multiband image memory 110 in the dental color measuring device 3 shown in FIG. The temperature data associated with the image data is transmitted to the dental colorimetric device 3 via the data IF 33, the local bus 37, the communication I / F controller 34, and the communication / I / F contact 61.
The association between the image data and the temperature data can be performed by attaching an identifier such as a serial number, for example.

  Note that the multiband image data and dark current image data of the color chart 100 are not recorded in the image memory 35 in the image pickup apparatus 1, but the local bus 37, the communication I / F controller 34, and the communication / I / F contact point. A configuration may be adopted in which the data is directly transmitted to the dental colorimetric device 3 via 61 and recorded in the multiband image memory 110 in the dental colorimetric device 3. In this case, the signal correction described above is performed in the dental color measuring device 3.

[Dental color measuring device]
As shown in FIG. 4, the dental colorimetric device 3 according to the present embodiment is composed of, for example, a personal computer or the like, and multiband image data output via the communication I / F contact 61 of the imaging device 1. Receives color chart image data and temperature data and applies various processing to this multiband image data to create image data with advanced color reproduction of the subject's teeth and suitable for the patient's teeth The selected shade guide number is selected and the information is displayed on the monitor 4.

  The dental color measuring device 3 is connected to the input device 5. In the input device 5, the user inputs a shade guide number and treatment support information. The shade guide number and the treatment support information input via the input device 5 are stored in the color sample information storage unit via the color sample selection unit 80. The dental colorimetric device 3 selects a suitable one for the patient's teeth from the shade guide number and treatment support information input by the user via the input device 5 and displays the selected on the monitor 4.

  Treatment support information includes various information necessary for a doctor to fill a cavity formed in a patient's tooth with a dental filler, such as cavity depth, position, and class information. Information, the number of dental filler layers, the presence or absence of layers, and information on dental fillers such as the material and thickness of each layer.

  The dental color measurement device 3 includes an RGB image memory 111 that records RGB color image data input from the imaging device 1, a multiband image memory 110 that stores multiband image data and color chart image data, and temperature data. And a temperature memory 121 for storage. The dental colorimetric device 3 receives the image data in the multiband image memory 110, and identifies the color chart image data and the multiband image data of the subject by the image identification unit 122. A calibration image generation unit (calibration data setting unit) 123 that generates calibration image data based on the identified color chart image data and temperature data stored in the temperature memory 121, and an image identification unit 122 A chromaticity calculating unit that calculates chromaticity of the multiband image data based on the identified multiband image data of the tooth and the calibration image data generated by the calibration image generating unit 123;

  The dental colorimetric device 3 is suitable for the color of the patient's teeth based on the color sample information storage unit 114 storing the shade guide information and the shade guide information stored in the color sample information storage unit 114. A color sample selection unit 80 for selecting a shade guide number, an image display control unit 115 for controlling the display content of the monitor 4, a color image creation processing unit 112 for creating color image data based on signals from the chromaticity calculation unit 70, And an RGB image memory 111 and an image filing unit 113 for storing RGB color image data sent from the color image creation processing unit 112 to the image display control unit 115.

The calibration image generation unit 123 generates calibration image data by interpolating the first and second color chart image data acquired before and after the photographing of the teeth with temperature. That is, when two color chart image data obtained by photographing a plurality of times of tooth imaging are t _s1 and t _s2 and the color chart image data is g _s1 and g _s2 , Calibration image data c for correcting the multiband image data g _n of the teeth photographed at t _n is calculated by the following equation.
c = (| t _s2 −t _n | × g _s1 + | t _s1 −t _n | × g _s2 ) / | t _s1 −t _s2 |

The chromaticity calculation unit 70 includes a spectrum estimation calculation unit (image correction unit) 71, an observation spectrum calculation unit 72, and a chromaticity value calculation unit 73.
As shown in FIG. 5, the spectrum estimation calculation unit 71 includes a conversion table creation unit 711, a conversion table 712, an input γ correction unit 713, a pixel interpolation calculation unit 714, an in-plane unevenness correction unit 715, a matrix calculation unit 716, and A spectrum estimation matrix creation unit 717 is provided. The input γ correction unit 713 and the pixel interpolation calculation unit 714 are individually provided for each of the multiband image data and the calibration image data. The input γ correction unit 713a and the pixel interpolation calculation are performed for the multiband image data. A unit 714a is provided, and an input γ correction unit 713b and a pixel interpolation calculation unit 714b are provided for the calibration image data.

  In the spectrum estimation calculation unit 71, first, the multiband image data and the calibration image data are transferred to the separately provided input γ correction units 713a and 713b, and after each input γ correction is performed, each pixel Image interpolation calculation processing is performed by the interpolation calculation units 714a and 714b. The signals after these processes are transferred to the in-plane unevenness correction unit 715, where the in-plane unevenness correction processing (calibration) of the multiband image data using the calibration image data is performed. Thereafter, the multiband image data is transferred to the matrix calculation unit 716, and the spectral reflectance is calculated using the matrix created by the spectrum estimation matrix creation unit 717.

Hereinafter, each image processing performed in each unit will be specifically described.
First, a conversion table 712 is created by the conversion table creation unit 711 as a pre-stage of input γ correction. Specifically, the conversion table creation unit 711 has data in which the input light intensity is associated with the sensor signal value, and creates the conversion table 712 based on these data. The conversion table 712 is created from the relationship between the input light intensity and the output signal value. For example, as shown by the solid line in FIG. 6A, the input light intensity and the sensor signal value are approximately proportional to each other. Created to be

  Each of the input γ correction units 713a and 713b performs input γ correction on the multiband image data and the calibration image data by referring to the conversion table 712. This conversion table is created so that an input light intensity D corresponding to the sensor value A at the present time is obtained, and an output sensor value B corresponding to the input light intensity D is output. As a result, the conversion table shown in FIG. It becomes like this. When the input γ correction is performed on the multiband image data and the calibration image data in this way, the corrected image data is transferred to the pixel interpolation calculation units 714a and 714b, respectively.

  In the pixel interpolation calculation units 714a and 714b, pixel interpolation calculation is performed by applying a low-pass filter for pixel interpolation to each of the multiband image data and calibration image data after the input γ correction. FIG. 7 shows an example of a low-pass filter applied to the R signal and the B signal. FIG. 8 shows a low-pass filter applied to the G signal. By multiplying each multiband image data by such a low-pass filter for pixel interpolation, for example, an image of 144 pixels × 144 pixels is changed to an image of 288 pixels × 288 pixels.

The image data gk (x, y) on which the image interpolation calculation has been performed is transferred to the in-plane unevenness correction unit 715.
The in-plane unevenness correction unit 715 corrects the luminance at the center of the screen of the multiband image data using the following equation (1).

In the above equation (1), c _k (x, y) is calibration image data, g _k (x, y) is multiband image data after input γ correction, and (x ₀ , y ₀ ) is the center pixel position. , Δ (= 5) is the area average size, and g ′ _k (x, y) is the image data after in-plane unevenness correction (where k = 1,..., N (number of bands)).
The in-plane unevenness correction is performed on each image data of the multiband image data.

The multiband image data g ′ _k (x, y) after the in-plane unevenness correction is transferred to the matrix calculation unit 716. The matrix calculation unit 716 performs a spectrum (spectral reflectance in this embodiment) estimation process using the multiband image data g ′ _k (x, y) from the in-plane unevenness correction unit 715. In this spectrum estimation process, spectral reflectance is estimated at 1 nm intervals in a wavelength band from 380 nm to 780 nm. That is, in this embodiment, a 401-dimensional spectral reflectance is estimated.

In general, a heavy and expensive spectroscopic measuring instrument is used to obtain the spectral reflectance for each wavelength. However, in this embodiment, since the subject is limited to teeth, the subject has a certain amount. By utilizing the feature, the 401-dimensional spectral reflectance is estimated with a small number of bands.
Specifically, a 401-dimensional spectrum signal is calculated by performing a matrix operation using the multiband image data g ′ _k (x, y) and the spectrum estimation matrix Mspe.

  The spectrum estimation matrix Mspe is created by the spectrum estimation matrix creation unit 717 based on the spectral sensitivity data of the camera, the LED spectrum data, and the statistical data of the subject (tooth). The creation of the spectrum estimation matrix is not particularly limited, and a well-known method can be used. For example, one example is described in detail in S. K. Park and F. O. Huck “Estimation of spectral reflectance curves from multispectrum image data”, Applied Optics, 16, pp 3107-3114 (1977).

  Note that the spectral sensitivity data of the camera, the LED spectrum data, the subject (tooth) statistical data, and the like are stored in advance in the image filing unit 113 shown in FIG. Further, when the spectral sensitivity of the camera changes depending on the position of the sensor, spectral sensitivity data may be acquired according to the position, or the center position may be appropriately corrected and used. .

The spectrum of the spectral reflectance calculated by the spectrum estimation calculation unit 71 is input to the observation spectrum calculation unit 72.
The observation spectrum calculation unit 72 multiplies the tooth spectrum calculated by the spectrum estimation calculation unit 71 by the spectrum S (λ) of the illumination light used for observation, so that the subject under the illumination light used for observation is obtained. Ask for the spectrum. The spectrum S (λ) of the illumination light is a spectrum of illumination light from a light source used when observing the color of the tooth, such as a D65 or D55 light source or a fluorescent light source, and is stored in a color reproduction characteristic storage unit (not shown). Stored in advance.

The chromaticity value calculation unit 73 calculates chromaticity values L ^* , a ^* , and b ^* from the spectrum G (x, y, λ) of the subject under illumination light used for observation. Specifically, the chromaticity value calculation unit 73 averages the chromaticity values L ^* , a ^* , and b ^* for each tooth part, and outputs them to the image display control unit 115 and the color sample selection unit 80.

  On the other hand, the spectrum G (x, y, λ) of the subject under illumination light used for observation is sent to the color image creation processing unit 112 and the image filing unit 113. The color image creation processing unit 112 creates color image data RGB2 (x, y) to be displayed on the monitor 4 and outputs it to the image display control unit 115. Note that the color image creation processing unit 112 may perform image processing such as contour enhancement.

  In the color sample information storage unit 114, for example, for each manufacturer that manufactures a shade guide in which color samples are arranged in a row, the captured image data of the shade guide manufactured by the manufacturer is associated with the shade guide number. In addition to being stored, a spectral spectrum (specifically, spectral reflectance spectrum) of a predetermined area of the shade guide and a shade guide image with gingiva are stored. Further, the color sample information storage unit 114 stores colorimetric information (for example, RGB value, L * a * b * value, CMYK value, XYZ value, Yuv value, Lv) obtained based on the captured image data of the shade guide. * C * h value) is stored for each pixel of the captured image data.

The color sample selection unit 80 includes the average chromaticity values L ^* , a ^* , b ^* transferred from the chromaticity calculation unit 70, the RGB color image data input from the color image creation processing unit 112, and the input device 5. A set of color sample information relating to the shade guide is created by associating the shade guide number and the treatment support information input from, and is stored in the color sample information storage unit 114.

In addition, the color sample selection unit 80 selects a shade guide number and treatment support information corresponding to the average chromaticity value output from the chromaticity value calculation unit 73 of the chromaticity calculation unit 70 from the color sample information storage unit 114.
The selected treatment support information is output to the image display control unit 115 together with the RGB color image of the shade guide.
The image display control unit 115 displays information input from each unit on the monitor 4.

  As described above, according to the color measurement system according to the present embodiment, when calculating the spectrum of the spectral reflectance in the spectrum estimation calculation unit 71, correction is performed only by the color chart image data acquired before photographing. Instead of using the calibration image data obtained by interpolating the two color chart image data acquired before and after the photographing with the temperature data, the change in the photographing condition etc. from the photographing of the color chart image data to the photographing of the teeth Regardless, multiband image data can be corrected with high accuracy. Accordingly, there is an advantage that high-precision color reproduction can be realized by a simple method without depending on photographing conditions such as apparatus temperature characteristics.

  In the present embodiment, the color chart image data acquired before and after the imaging of the tooth is interpolated with the temperature data to generate the color chart image data at the temperature at the time of imaging of the tooth, and this is used as the calibration image. However, instead of this, the similarity between the temperature at the time of acquiring the multiband image data of the tooth and the temperature at the time of acquiring the color chart image data is determined, and the closest one of the color chart image data before and after photographing is determined as the multiband image. Data calibration image data may be employed. As a result, multiband image data can be corrected more simply and quickly without performing an interpolation calculation.

  In addition, color chart image data at an intermediate temperature of these color chart image data is generated from the color chart image data acquired before and after the photographing of the teeth, and the color chart image data of the three color chart image data is obtained by the temperature approximation. Either of them may be adopted as calibration image data. By doing in this way, even when the temperature difference when the color chart image data is photographed before and after the photographing of the tooth is large, the temperature difference is reduced by the color chart image data created at the intermediate temperature, and the closer temperature The calibration image data corresponding to the above can be obtained, and the multiband image data can be corrected with higher accuracy.

  In this case, the generation of color chart image data at an intermediate temperature is performed when the difference in temperature data at the time of obtaining the color chart image data before and after the photographing of the tooth greatly fluctuates beyond a predetermined threshold. Also good. By doing so, since it is not necessary to generate intermediate color chart image data when the temperature difference is small, it is possible to easily perform calibration from two color chart image data without performing calculations. When the temperature difference is large, calibration image data close to the imaging conditions at the time of imaging of the teeth can be generated, and the multiband image data can be corrected with high accuracy.

  Here, the intermediate temperature may be one or plural. By increasing the number, correction can be made more finely, but the number of calculations increases. Therefore, the accuracy of correction and the calculation amount should be compared and appropriately determined.

In this embodiment, the information on the shade guide having the average chromaticity value close to the color of the patient's tooth is selected based on the average chromaticity value. However, the above example is an example, and other colorimetry is performed. It is good also as selecting based on information. For example, instead of the average chromaticity value, the treatment support information may be selected based on a spectrum (in this embodiment, spectral reflectance) obtained by the chromaticity calculation unit 70.
In this case, the color sample information storage unit 114 stores the spectral reflectance acquired by the spectrum estimation calculation unit 71 (see FIG. 4) instead of the average chromaticity value.
The selection of the treatment support information is performed based on, for example, a spectrum determination value Jvalue based on the following equation (2) and the spectrum determination value Jvalue.

In the above equation (2), Jvalue is a spectrum determination value, C is a normalization coefficient, n is a statistical number (the number of λ used for calculation), λ is a wavelength, and f ₁ (λ) is a spectral sensitivity of a patient's tooth. The spectral value, f ₂ (λ) is a spectral sensitivity spectral value in the information of the shade guide, and E (λ) is a determination sensitivity correction value. In the present embodiment, weighting relating to spectral sensitivity according to λ is performed by E (λ).

In this way, the spectral reflectance of each shade guide is substituted into f ₂ (λ) in the above equation (2), and the respective spectrum determination values Jvalue are calculated. And it is good also as selecting the information of a shade guide when the smallest spectrum judgment value Jvalue is shown. Further, the number of shade guides to be selected is not limited to one, and it is also possible to select all information having a preset number or difference within a certain range.

In this embodiment, the average chromaticity value of each part is used as the colorimetric information. However, the chromaticity value at the center position of the part may be adopted as the colorimetric information instead of the average. The color space for obtaining colorimetric information is not limited to the L ^* a ^* b ^* color space. For example, luminance, saturation, and hue in the VCH space may be used, and X, Y, and Z values in the XYZ space may be used. It is good also as using.

The shade guide may be further classified into anterior teeth, molar teeth, canines, etc., and may be classified according to the age of children, adults, elderly people, and the like. This is also because the layer structure, particularly the layer thickness, differs depending on these.
In the present embodiment, the chromaticity value of each part of the shade guide is acquired by photographing the shade guide with the imaging device 1, but the method for acquiring the chromaticity value of the shade guide is limited to this example. Not. For example, the spectroscope may measure the chromaticity value of the shade guide for each part, and the average chromaticity value at each part may be transferred to the color sample selection unit 80.

1 is a block diagram illustrating a schematic configuration of an imaging device and a cradle of a color measurement system according to an embodiment of the present invention. It is a figure which shows the spectrum of the light source shown in FIG. It is a figure explaining signal correction. 1 is a block diagram illustrating a schematic configuration of a dental color measurement device of a color measurement system according to an embodiment of the present invention. It is a block diagram which shows the spectrum estimation calculating part with which the dental colorimetry apparatus of FIG. 4 is equipped. It is a figure explaining input gamma correction. It is a figure which shows an example of the low pass filter applied to R signal and B signal in pixel interpolation calculation. It is a figure which shows an example of the low pass filter applied to G signal in pixel interpolation calculation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 3 Dental colorimetric device (image processing device)
13 Temperature sensor (temperature measurement part)
71 Spectrum estimation calculation unit (image correction unit)
100 color chart 123 calibration image generation unit (calibration data setting unit)

Claims (6)

The temperature of the imaging device at the time of photographing the subject by the imaging device, the temperature of the imaging device at the time of photographing the color chart before and after photographing the subject, and the color chart before and after photographing the subject Based on the color chart image data acquired by shooting, from the temperature of the imaging device when shooting the color chart before shooting the subject to the temperature of the imaging device when shooting the color chart after shooting the subject A calibration data setting unit for setting color chart image data at a temperature closer to the temperature at the time of shooting of the subject as calibration data,
An image processing apparatus comprising: an image correction unit that corrects image data of a subject acquired by photographing the subject using calibration data set in the calibration data setting unit.   The calibration data setting unit interpolates the color chart image data acquired before and after photographing the subject according to the temperature of the imaging device when photographing the color chart before and after photographing the subject. The image processing apparatus according to claim 1, wherein color chart image data at a temperature at which the subject is photographed is generated and set as calibration data.   The calibration data setting unit compares the temperature of the imaging device at the time of photographing the subject with the temperature of the imaging device at the time of photographing the color chart before and after photographing the subject, and shoots the subject. The image processing apparatus according to claim 1, wherein any color chart image data corresponding to a temperature closer to the temperature of the imaging apparatus at the time is set as calibration data.   The calibration data setting unit is configured to output a color chart image at an intermediate temperature when the temperature of the imaging device fluctuates beyond a predetermined threshold when photographing the color chart before and after photographing the subject. The image processing apparatus according to claim 1, wherein data is generated, and color chart image data corresponding to a temperature closer to the temperature of the imaging apparatus at the time of photographing the subject is set as calibration data. The temperature of the imaging device at the time of photographing the subject by the imaging device, the temperature of the imaging device at the time of photographing the color chart before and after photographing the subject, and the color chart before and after photographing the subject Based on the color chart image data acquired by shooting, the color chart image data at a temperature close to the temperature at the time of shooting the subject is set as calibration data,
An image processing method for correcting image data of a subject acquired by photographing the subject using set calibration data. An imaging device comprising: an imaging unit that captures a subject and a color chart; and a temperature measurement unit that measures the temperature of the imaging unit during shooting;
A colorimetry system comprising: the image processing apparatus according to claim 1, which processes image data acquired by the imaging apparatus.

Images