JP2009033385A - Image processor, image processing method, image processing program and imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform development processing with high definition and to attain the acceleration of the development processing applied in accordance with an instruction of a user. <P>SOLUTION: A normal exposure/WB adjustment processing part 101 and an interpolation, etc., image processing part 102 of a stage 1 perform one or more development processing with a large processing load about Raw image data, generate an intermediate development image and temporarily store the intermediate development image in a RAM 123. Consequently, a user exposure/WB adjustment processing part 103 and a final development processing part 104 of a stage 2 can repeatedly perform one or more development processing with a small processing load of the intermediate development image of the RAM 123. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an apparatus, a method, and a program for processing so-called raw image data that is undeveloped image data.

  In recent years, Raw development, in which Raw image data in which signals from an image sensor are recorded as they are, is developed while a user performs development adjustment by a Raw development device such as a personal computer, has attracted attention in recent years. In the raw development processing, the user can obtain a result image corresponding to the development parameters by designating various development parameters to the raw development device.

  However, in Raw development, the user repeats the development parameter tracking operation. That is, as shown in FIG. 11, the raw image data of the target image recorded and held in the raw image file Rd-f is read (step S1), developed and displayed (step S2), and the user Make sure you can check. Then, a development parameter change instruction from the user is received (step S3).

  Then, it is determined whether there is a change instruction (step S4). If there is a change instruction, the development parameter is changed according to the change instruction received in step S3 (step S6), and the processing from step S2 is performed. repeat. In this way, the user changes the parameters little by little and repeats the development process, and as a result, confirms the image, and further changes the parameters based on the result, thereby repeating the operation of the user. Make an image that suits your taste.

  However, since the calculation load of the Raw development processing itself is generally large, the calculation load of the entire repetitive work for driving the development parameters described above becomes very large, and it takes time and effort to obtain the target image. This is a burden on the user. For this reason, there is an increasing demand for high-speed Raw development processing.

  As one of the methods for speeding up the raw development processing, Japanese Patent Application Laid-Open No. H10-228707 proposes an intermediate developed image. In this method, for example, as shown in FIG. 12, when the development processing includes three steps of Step 1 to Step 3, Stage 1 including Step 1 and Step 2 is performed on Raw image data Rd. In the middle of processing, the result image is held as the intermediate developed image Cd, and after stage 2 consisting of step 3 is executed and the result image Dd is formed (even after development is completed), the intermediate developed image Cd is held.

  Then, when the next user-specified development parameter is not different from the parameter necessary for generating the intermediate developed image, that is, when it is not necessary to execute the process 1 and the process 2 of the stage 1, the intermediate development is performed. By using the image Cd, only the process 3 of the stage 2 is executed while changing the parameters so that the development can be repeatedly performed, thereby increasing the speed of the development process. As a result, high performance can be provided when the user changes the development parameters for the development processing after the intermediate development image.

The above-mentioned Patent Document 1 is as follows.
JP 2005-341551 A

  By the way, the user adjusts various development parameters depending on the situation, but exposure adjustment (exposure correction) and white balance adjustment (white balance correction) are generally considered to be frequently adjusted. Since exposure adjustment and white balance adjustment (hereinafter referred to as exposure / WB adjustment) are processes directly performed on Raw image data (Raw signal), luminance adjustment after a result image such as an RGB image is obtained. The meaning differs from color adjustment. Exposure / WB adjustment is very important in Raw development.

  As described above, the exposure / WB adjustment is often performed on the raw image data. Therefore, in the development flow in a normal image pickup apparatus, the exposure / WB adjustment is often arranged at the front stage. For example, as shown in FIG. 13, the exposure / WB adjustment process is positioned before the stage 1. That is, in the case of the example shown in FIG. 13, the raw image data is first subjected to exposure / WB adjustment processing for the raw image data, and then subjected to interpolation processing to form an intermediate developed image Cd. Then, stage 2 processing consisting of other development processing is performed.

  Thus, the reason why the exposure / WB adjustment processing is positioned in the preceding stage of the development flow is because appropriate exposure and white balance are necessary in various development processing. For example, in an interpolation process or the like, if the process is advanced in order to obtain a high image quality, there may be a case where high quality cannot be maintained for raw image data in which appropriate exposure and white balance are not maintained.

  In addition, if appropriate exposure adjustment is not performed in the previous stage, the quantization noise and the like accumulated during various development processes may be doubled. For these reasons, in the development flow (Raw development processing flow), exposure adjustment and white balance adjustment are often arranged in the preceding stage of the development flow.

  However, in the RAW development apparatus using the intermediate development image, if the development parameter for the development processing in the previous stage of the Raw development processing flow is changed, the intermediate development image cannot be used, and all development processing needs to be performed again. . For example, as shown in FIG. 13, in stage 1, the exposure adjustment parameter exp = 20, the three primary color white balance parameters wbR = 10, wbG = 30, wbB = 40, and other parameters b = 20, c = 30. Is used to form an intermediate developed image Cd, and in stage 2 another parameter d = 50 is used to form a result image Dd.

  Here, as shown in FIG. 14, the user who has confirmed the result image Dd changes the exposure adjustment parameter exp to the value “15”, and among the white balance parameters, the red signal white balance parameter wbR to the value “20”. I want to do it. In this case, as is clear from FIGS. 13 and 14, the exposure / WB adjustment process is located in the preceding stage of the stage 1, and therefore the intermediate developed image Cd cannot be used. It becomes processing. Therefore, the exposure / WB adjustment cannot benefit from the high speed due to the intermediate developed image.

  As described above, the exposure / WB adjustment is the most basic adjustment item (correction item) in the Raw development processing, and is frequently used by the user. The fact that these two processes are not targeted for speeding up the intermediate developed image is less advantageous in view of the cost (memory space and the like) for using the intermediate developed image.

  In other words, for example, as shown in FIG. 15, if the exposure / WB adjustment processing position is brought to the position after the intermediate developed image is obtained (after the development flow) in the development flow, the development in the preceding stage is performed. The processing result will be adversely affected. That is, when the exposure / WB adjustment process is shifted to the stage 2 as shown in FIG. 15, the interpolation process of the stage 1 must be performed using the raw image data before the exposure / WB adjustment. May affect

  In this way, even if the configuration using the intermediate developed image is adopted for speeding up the Raw development processing, the parameters for exposure adjustment and white balance adjustment positioned before the intermediate developed image are generated. In the case of changing, the intermediate developed image cannot be used, and the benefits of speeding up the processing using the intermediate developed image cannot be obtained.

  In view of the above, an object of the present invention is to enable high-quality development processing and to realize high-speed development processing performed in accordance with a user instruction.

In order to solve the above problem, an image processing apparatus according to claim 1 is provided.
First development processing means for receiving raw image data that is undeveloped image data and performing one or more development processes with a large processing load on the raw image data to generate intermediate developed image data When,
Storage means for storing and holding the intermediate developed image data generated in the first development processing means;
One or more development processes with a small processing load are performed on the intermediate developed image data from the first development processing unit or the intermediate developed image data stored and held in the storage unit. And a second development processing means for generating image data resulting from the development process.

  According to the image processing apparatus of the first aspect of the present invention, the first development processing means performs one or more development processes with a large processing load on the raw image data, and generates intermediate development image data. The The intermediate developed image data generated by the first development processing unit is stored and held in the storage unit. Then, the intermediate development image data from the first development processing unit or the intermediate development image data stored and held in the storage holding unit is processed by the second development processing unit by the one or more low processing loads. Development is performed, and a result image obtained as a result of a series of development processes is generated.

  Thus, the intermediate developed image data is continuously held in the storage unit. First, a result image obtained as a result of a series of development processing is obtained through the first development processing means and the second development processing means, and after confirming the result image, the intermediate image stored and held in the storage means Using the developed image data, only one or more development processes with a small processing load are repeatedly performed through the second development processing means so that a result image having a desired image quality can be obtained. Therefore, it is possible to perform development processing with high quality and to realize high-speed development processing.

The invention according to claim 2 is the image processing apparatus according to claim 1,
Comprising a receiving means for receiving an instruction input relating to development processing from a user;
The one or more development processes performed in the first development processing unit include one or both of an exposure adjustment process and a white balance adjustment process,
When one or both of an instruction input related to exposure adjustment processing and an instruction input related to white balance adjustment processing are received through the receiving means,
The second development processing unit passes the reception unit for the intermediate development image data from the first development processing unit or the intermediate development image data stored in the storage unit. One or both of the exposure adjustment process and the white balance adjustment process are executed in accordance with the received instruction input.

  According to the image processing apparatus of the second aspect of the present invention, high-quality development processing is performed by performing one or both of exposure adjustment processing and white balance adjustment processing by the first development processing means. When the instruction input for one or both of the exposure adjustment process and the white balance adjustment process is received through the reception unit, the second development processing unit receives the instruction from the first development processing unit. One or both of exposure adjustment processing and white balance adjustment processing according to the instruction input received through the receiving unit is performed on the intermediate developed image data or the intermediate developed image data stored in the storage unit. Is called.

  Thus, when an instruction input for exposure adjustment processing or white balance adjustment processing is received through the reception unit, the second development processing unit does not function the first development processing unit and the exposure adjustment processing or white balance is performed. The balance adjustment process can be performed repeatedly. Therefore, even when the parameters of the exposure adjustment process and the white balance adjustment process are changed by the user through the receiving unit, the exposure adjustment process and the white balance adjustment process can be performed using the intermediate developed image data. Development processing can be performed with high quality, and high-speed development processing can be realized.

An image processing apparatus according to a third aspect of the present invention is the image processing apparatus according to the second aspect,
When one or both of the exposure adjustment process and the white balance adjustment process are executed by the first development processing unit, the first development processing unit includes an adjustment value supplied together with the raw image data, and the raw value. Performing an adjustment process using one or more of an adjustment value obtained by detecting image data and a predetermined fixed adjustment value;
When one or both of exposure adjustment processing and white balance adjustment processing is executed by the second development processing means, the second development processing means responds to the adjustment value used in the first development processing means. Then, after canceling the same adjustment process as the adjustment process to be performed by the second development processing unit performed by the first processing unit, the adjustment process is performed according to the instruction input received through the reception unit. It is characterized by performing.

  According to the image processing apparatus of the third aspect of the invention, in the first development processing means, (1) an adjustment value set in, for example, an imaging apparatus supplied together with the raw image data, (2 The exposure adjustment process and the white balance adjustment process are performed using so-called predetermined adjustment values such as an adjustment value obtained by detecting the raw image data, and (3) a predetermined fixed adjustment value. .

  On the other hand, since the adjustment value used in the first development processing unit is known in the second development processing unit, the exposure adjustment processing and white balance adjustment processing performed in the first development processing unit are canceled. The exposure adjustment process and the white balance adjustment process are performed in accordance with the instruction input received through the receiving unit.

  Accordingly, since one or both of the exposure adjustment process and the white balance adjustment process is performed in the first development processing unit, the development process can be performed with high quality, and the user can perform the exposure adjustment process and the white balance adjustment process. When the parameters of the balance adjustment process are changed, the intermediate development image data is used by the second development processing unit. First, the exposure adjustment process performed by the first development processing unit or the white adjustment process is performed. Since the exposure adjustment process and the white balance adjustment process are performed according to the user's instruction after the balance adjustment process is canceled, the development process according to the user's instruction can be performed quickly and appropriately. .

  According to the present invention, it is possible to perform development processing with high quality, and it is possible to realize high-speed development processing performed according to a user instruction. Therefore, the so-called development parameter tracking operation can be performed quickly, and the time burden on the user for the development processing for the raw image data can be greatly reduced.

  Hereinafter, an embodiment of an apparatus, a method, and a program according to the present invention will be described with reference to the drawings.

[First Embodiment]
First, an image processing apparatus 100 to which an embodiment of an apparatus, method, and program according to the present invention is applied will be described. The image processing apparatus 100 according to the first embodiment includes, for example, a personal computer, a game machine, a television receiver, a home digital recorder called PVR (Personal Video Recorder), a mobile phone terminal, and a PDA (Personal Digital). It is realized in various modes such as a portable information terminal called Assistants). The image processing apparatus 100 according to the first embodiment to be described below will be described by taking as an example a case where the image processing apparatus 100 is realized in an aspect such as a personal computer, a television receiver, or a PVR, and is used at home. .

[Configuration Example of Image Processing Device 100]
FIG. 1 is a block diagram for explaining an image processing apparatus 100 according to the first embodiment. As shown in FIG. 1, an image processing apparatus 100 according to the first embodiment includes a normal exposure / WB adjustment processing unit 101, an interpolation image processing unit 102, a user exposure / WB adjustment processing unit 103, a final development processing unit. 104, display controller 105, display unit 106, hard disk drive (hereinafter abbreviated as HDD) 107, operation unit 108, media drive 111, external interface (hereinafter abbreviated as external I / F) 112, communication interface ( Hereinafter, it is abbreviated as a communication I / F.) 114, a transmission / reception antenna 115, and a control unit 120 are provided.

  The control unit 120 controls each unit of the image processing apparatus 100 according to the first embodiment. As illustrated in FIG. 1, the control unit 120 includes a CPU (Central Processing Unit) 121, a ROM (Read Only Memory) 122, A microcomputer in which a RAM (Random Access Memory) 123 and an EEPROM (Electrically Erasable and Programmable ROM) 124 are connected through a CPU bus 125.

  Here, the CPU 121 reads and executes a program stored in the ROM 122, which will be described later, and forms a control signal to be supplied to each part and supplies it to each part, or receives a signal from each part and processes it. It is the main subject of control. As described above, the ROM 122 stores programs executed by the CPU 121 and various data necessary for processing in advance.

  The RAM 123 is used as a so-called work area for temporarily storing intermediate results of various processes. The EEPROM 124 is a so-called non-volatile memory, and is retained even when the power of the image processing apparatus 100 according to the first embodiment is turned off, such as various setting parameters and a program newly provided for function addition. Various information to be stored is stored and held.

  The image processing apparatus 100 according to the first embodiment can receive raw image data obtained by photographing with an imaging device such as a digital camera and can develop the raw image data. .

  Raw image data is simply digitized subject image signals, which are electrical signals obtained from an imaging device such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensor of an imaging device, It means image data in an undeveloped state before development processing.

  The development process performs a series of image signal processes such as an exposure adjustment process, a white balance adjustment process, an interpolation process (demosaic process) on the raw image data, and R (red) which is a three primary color signal for each pixel. ), G (green), and B (blue) three-channel frame image data is formed, and finally image data that can reproduce an image visible to the user is formed.

  Therefore, the image processing apparatus 100 according to the first embodiment includes a media drive 111, an external I / F 112, an input / output terminal 113, a communication I / F 114, and a transmission / reception antenna 115 as a raw image data receiving unit. .

  The media drive 111 can be attached to and detached from a removable medium, and can read data from and write data to the removable medium attached thereto. Therefore, by attaching the removable medium on which the raw image data is recorded by the imaging device to the media drive 111, the control unit 120 of the image processing apparatus 100 according to the first embodiment is recorded on the removable medium. It is possible to read out and use the raw image data.

  There are various types of removable media such as a memory card, an optical disk, a magneto-optical disk, and a magnetic disk. The media drive 111 of the image processing apparatus 100 according to the first embodiment is, for example, for a memory card. Is.

  The external I / F 112 and the input / output terminal 113 are digital interfaces such as a USB (Universal Serial Bus) standard and an IEEE (Institute of Electrical and Electronics Engineers, Inc.) 1394 standard, and are externally equipped with the same digital I / F. The equipment can be touched.

  Therefore, by connecting an external device such as an imaging device that stores and holds Raw image data through the external I / F 112 and the input / output terminal 113, the control unit 120 of the image processing apparatus 100 according to the first embodiment is connected. However, the raw image data stored and held in the external device can be provided and used.

  The communication I / F 114 and the transmission / reception antenna 115 are configured to transmit / receive information to / from an external device through a wide area network such as the Internet, for example. Therefore, the control unit 120 has a communication function and performs communication processing with an external device such as an imaging device or a mobile terminal holding Raw image data through the communication I / F 114 and the transmission / reception antenna 115. Raw image data can be acquired and used.

  As described above, the image processing apparatus 100 according to the first embodiment includes a raw image through the media drive 111 as the raw image data receiving unit, the external I / F 112 and the input / output terminal 113, the communication I / F 114, and the transmission / reception antenna 115. Data can be acquired, and the acquired raw image data can be developed by the control of the control unit 120.

  In the image processing apparatus 100 according to the first embodiment, the development processing for the raw image data is performed in the normal exposure / WB adjustment processing unit 101, the image processing unit 102 such as interpolation, and the user exposure / WB adjustment processing unit 103. The final development processing unit 104 can perform this.

  In the image processing apparatus 100 according to the first embodiment, the Law image data acquired through the Law image data receiving unit is first supplied to the normal exposure / WB adjustment processing unit 101, where an adjustment value determined in advance is supplied. Are used to perform exposure adjustment processing and white balance adjustment processing, and the adjusted raw image data is supplied to the interpolating image processing unit 102 through the control unit 120.

  The image processing unit 102 such as the interpolation receives the raw image data subjected to the normal exposure adjustment process and the white balance adjustment process, and uses the adjacent pixel data for each pixel data for the three RGB channels (three primary colors). ) Intermediate developed image data (frame data) having a light / dark level. The intermediate developed image data formed here is temporarily stored in the RAM 123 of the control unit 120 and supplied to the user exposure / WB adjustment processing unit 103.

  The user exposure / WB adjustment processing unit 103 executes an exposure adjustment process and a white balance adjustment process in accordance with the parameter designated by the user. For this reason, the operation unit 108 is connected to the control unit 120 of the image processing apparatus 100 of the first embodiment.

  The operation unit 108 is composed of a keyboard including a numeric key group, an alphabet key group, a function key group, and a pointing device such as a so-called mouse. The operation unit 108 receives an operation input from a user and converts it into an electric signal, which is converted into a control unit. 120 can be supplied.

  Therefore, when developing the raw image data, when the user wants to change the parameters of the exposure adjustment process and the white balance adjustment process, the user inputs an instruction for changing the parameters through the operation unit 108. Specifically, parameters for exposure adjustment processing and white balance adjustment processing are input numerically, and the state of adjustment values for exposure adjustment processing and white balance adjustment processing displayed on the display unit 106 described later are displayed. The displayed display is changed using an arrow key or a mouse.

  The instruction input for changing the parameters input in this way is supplied to the control unit 120 through the operation unit 108. The control unit 120 forms parameters for exposure adjustment processing and white balance adjustment processing in response to an instruction input from the user supplied to the control unit 120, and supplies the parameters to the user exposure / WB adjustment processing unit 103.

  As a result, the user exposure / WB adjustment processing unit 103 is instructed by the user for the intermediate developed image data from the image processing unit 102 such as interpolation or for the intermediate developed image data temporarily stored in the RAM 123. Depending on the parameters, exposure adjustment processing and white balance adjustment processing are performed.

  In this case, for example, in the case of the first development process, the user exposure / WB adjustment processing unit 103 performs exposure adjustment processing and white balance adjustment processing on the intermediate developed image from the interpolated image processing unit 102, and from the user In the second and subsequent development processes performed by changing the parameters according to the instruction, the exposure adjustment process and the white balance adjustment are performed on the intermediate developed image data stored and held in the RAM 123 as will be described later. Process.

  In this way, the image data subjected to the exposure adjustment process and the white balance adjustment process according to the user instruction in the user exposure / WB adjustment processing unit 103 is supplied to the final development processing unit 104 through the control unit 120. Here, for example, necessary development processing such as color space conversion processing and nonlinear spatial processing such as gamma correction is performed to form result image data.

  Result image data formed in the final development processing unit 104 is supplied to the display controller 105 through the control unit 120. The display controller 105 forms a video signal (image signal) to be supplied to the display unit 106 and supplies the video signal to the display unit 106.

  The display unit 106 may be provided as a part of the image processing apparatus 100 according to the first embodiment, or may be configured as a monitor receiver separate from the image processing apparatus 100. A relatively large display screen is provided, and a video (image) corresponding to a video signal from the display controller 105 is displayed on its own display screen.

  As described above, the image processing apparatus 100 according to the first embodiment receives the raw image data, develops the raw image data, forms result image data, and displays an image corresponding to the display image data. 106 can be displayed on the display screen 106.

  In the image processing apparatus 100 according to the first embodiment, the supplied Raw image data, generated intermediate development image data, and result image data obtained as a result of a series of development processes are stored in the HDD 107. By recording and holding in the hard disk, it can be read and used at any time.

  Here, the HDD 107 has a built-in hard disk having a relatively large storage capacity, and can write data into the hard disk and read data from the hard disk.

  Therefore, if the raw image data is stored in the hard disk of the HDD 107, it is possible to repeatedly perform development processing using the raw image data. Further, if the intermediate developed image data is stored in the hard disk of the HDD 107, a development process from the middle using the intermediate developed image data can be performed at any time to develop a desired image. Of course, if the result image data is stored in the hard disk of the HDD 107, the result image can be quickly displayed and used, or provided to another device and used in the other device.

  As described above, the image processing apparatus 100 according to the first embodiment can perform development processing on raw image data supplied from the outside, but simply performs development processing automatically. Rather, the user can perform development processing according to his / her preference by changing parameters for exposure adjustment processing and white balance adjustment processing.

  Since the exposure adjustment process and the white balance adjustment process that should normally be performed are performed in the normal white balance adjustment processing unit 101 as described above, the raw image data that has not been subjected to the exposure adjustment process or the white balance adjustment process. Therefore, high-quality development processing can be performed.

  Furthermore, even if the user changes the parameters for exposure adjustment processing and white balance adjustment processing, it is not necessary to start development processing from the beginning, and only the necessary development processing is performed using the intermediate developed image. Therefore, the development processing can be performed quickly without taking time for the development processing, and the raw image data can be developed into an image having a target hue by the user.

  In FIG. 1, each of the normal exposure / WB adjustment processing unit 101, the interpolating image processing unit 102, the user exposure / WB adjustment processing unit 103, and the final development processing unit 104, which are indicated by double-line blocks, is shown in FIG. As shown in FIG. 5, it can be configured as an independent circuit part, or can be realized as a function of the control unit 120 by software (program) executed in the control unit 120.

[Details of development processing]
FIG. 2 is a block diagram for explaining the details of the development processing performed in the image processing apparatus 100 of this embodiment, and shows how each unit functions according to the stage of the development processing. FIG.

  In the image processing apparatus 100 according to the first embodiment, a series of development processing is separated into two groups (processing groups), that is, a processing group performed in stage 1 (previous stage) and a processing group performed in stage 2 (rear stage). Like to do. In each of the stage 1 and the stage 2, exposure / WB adjustment processing can be performed.

  Specifically, as will be described in detail below, the storage 1 performs normal exposure / WB adjustment processing performed in a general imaging device, thereby performing high-quality development processing and intermediate developed image data. In stage 2, the intermediate / development image data generated in stage 1 is subjected to exposure / WB adjustment processing according to the user's instruction, so that the result matches the user's intention and preference. As a result of forming an image, image data can be obtained. Hereinafter, the development processing in the image processing apparatus 100 of the first embodiment will be described in detail.

  As shown in FIG. 2, in the image processing apparatus 100 according to the first embodiment, the stage 1 includes processing performed by a normal exposure / WB adjustment processing unit 101 and an image processing unit 102 such as an interpolation. 2 includes processing performed by the user exposure / WB adjustment processing unit 103 and the final development processing unit 104.

  As described above, the raw image data Rd received through the raw image data receiving unit including the media drive 111, the external I / F 112, the input / output terminal 113, the communication I / F 114, and the transmission / reception antenna 115 is the normal image of the stage 1. It is supplied to the exposure / WB adjustment processing unit 101.

  As described above, the normal exposure / WB adjustment processing unit 101 of the stage 1 performs a so-called normal exposure / WB adjustment process that is performed in a general imaging apparatus. The processing performed by the normal exposure / WB adjustment processing unit 101 is always performed at the same value without being affected by the user exposure / WB adjustment parameter.

  The exposure / WB adjustment parameter (adjustment value) used here is calculated by (1) what is previously recorded in the RAW image file by the imaging device, or (2) self-detection by the image processing device 100 which is a RAW developing device. Or (3) a fixed value that is set in advance so that an appropriate effect can be expected.

  Of course, any of the adjustment values (1) to (3) described above can be used, and the adjustment value for exposure adjustment processing is recorded in the raw image file in advance by the imaging device, and the raw image data. At the same time, the one supplied to the image processing apparatus 100 according to the first embodiment is used, and the adjustment value for white balance adjustment processing is obtained by performing detection processing in the image processing apparatus 100. In addition, an appropriate value can be selected and used from the adjustment values (1) to (3) described above.

  FIG. 3 is a diagram for explaining a specific example of the exposure adjustment process and the white balance adjustment process performed in the normal exposure / WB adjustment processing unit 101 of the stage 1. As shown in the equations (1), (2), and (3) in FIG. 3, the normal exposure adjustment processing is performed on the normal exposure adjustment gain value ANormal which is a fixed value and each Raw pixel of the original raw image data. By multiplying the values Rraw, Graw, and Braw, the values R′raw, G′raw, and B′raw of the Raw pixels after the normal exposure adjustment are obtained.

  Then, as shown in the equations (4), (5), and (6) in FIG. 3, the normal white balance adjustment processing is performed in the normal white balance adjustment gain values WBnormal, WBnormal, By multiplying WBbnormal and the corresponding Raw pixel values R′raw, G′raw and B′raw after normal exposure adjustment, the Raw pixel values R ″ raw and G ″ raw after normal white balance adjustment are obtained. , B ″ raw.

  That is, as described above, the normal exposure adjustment gain value ANormal which is a fixed value used here and the normal white balance adjustment gain values WBnormal, WBnormal, and WBbnormal corresponding to each color signal which are fixed values are as described above. Recorded in a Raw image file in advance, (2) calculated by the image processing apparatus 100, which is a Raw developing device, by self-detection, or (3) set in advance so that a certain effect can be expected. Fixed value etc.

  Although FIG. 3 shows an example in which the exposure adjustment is performed first and the WB adjustment is performed later, the present invention is not limited to this. The calculation order may be reversed, such that the WB adjustment is performed first and the exposure adjustment is performed later, and the exposure coefficient (normal exposure adjustment gain value) and the WB coefficient (normal WB adjustment gain value) are multiplied in advance. May be calculated at once.

  In this way, the normal exposure / WB adjustment processing unit 101 of the stage 1 shown in FIG. 2 performs normal exposure / WB adjustment processing using a predetermined adjustment value, and develops the image with high quality. It is guaranteed to be able to do. In FIG. 2, for the sake of simplicity, the normal exposure adjustment gain value, which is a fixed value, is “exp”, and the normal white balance adjustment gain value is “wbR, wbG, wbB”. The usage parameters are shown below the WB adjustment processing unit 101.

  Then, as shown in FIG. 2, the RAW image data subjected to exposure adjustment and white balance adjustment in the normal exposure / WB adjustment processing unit 101 is supplied to an image processing unit 102 such as interpolation, where interpolation processing (demosaic processing) or Image processing is performed using predetermined parameters (adjustment values) b and c, and intermediate developed image data is formed, which is stored and held in the RAM 123.

  As described above, in stage 1, exposure / WB adjustment processing is performed without being affected by the user exposure / WB adjustment parameters, and predetermined image processing such as interpolation processing is performed to obtain intermediate developed image data. Is generated.

  The intermediate developed image data generated by the processing of stage 1 in this way is supplied to the user exposure / WB adjustment processing unit 103 of stage 2. Then, in stage 2, by performing exposure / WB adjustment at the development parameters specified by the user and performing other necessary development processing, final result image data for forming an image to be presented to the user is generated, Output this.

  FIG. 2 shows the case where the intermediate developed image data is supplied to the user exposure / WB adjustment processing unit 103 of the stage 2 via the RAM 123, but in the first development processing, the intermediate developed image data is stored. In addition to writing in the RAM 123, it can also be directly supplied to the user exposure / WB adjustment processing unit 103 to perform exposure / WB adjustment with development parameters specified by the user.

  The adjustment value (gain value) used for adjustment in the user exposure / WB adjustment processing unit 103 of the stage 2 is (adjustment value specified by the user / adjustment value used in the stage 1). In this way, by using the adjustment value specified by the user as the numerator and the adjustment value used in stage 1 as the denominator, the exposure / WB adjustment performed in stage 1 can be canceled and the adjustment parameter of the user can be reflected. I can do it.

  FIG. 4 is a diagram for explaining a specific example of exposure adjustment processing and white balance adjustment processing performed in the normal exposure / WB adjustment processing unit 103 of the stage 2. As shown in the equations (1), (2), and (3) in FIG. 4, the user exposure adjustment processing includes (user exposure adjustment gain value AEuser / normal exposure adjustment gain value ANormal) and Raw of the intermediate developed image. By multiplying pixel values Rbuffer, Gbuffer, and Bbuffer, Raw pixel values R′buffer, G′buffer, and B′buffer after user exposure adjustment are obtained.

  Then, as shown in the equations (4), (5), and (6) in FIG. 4, the user white balance adjustment process is performed by the user white balance adjustment gain value / normal white balance adjustment gain value, that is, WBruser / WBrnormal. , WBguser / WBnormal, WBbuser / WBbnormal, and the corresponding Raw pixel value R′buffer, G′buffer, B′buffer after user exposure adjustment, and the Raw pixel after user white balance adjustment. The values R ″ buffer, G ″ buffer, and B ″ buffer are obtained.

  In this way, the user exposure / WB adjustment processing unit 103 of the stage 2 shown in FIG. 2 performs the exposure / WB adjustment processing using the adjustment value instructed by the user, and the result that matches the purpose of the user Image data can be formed. In FIG. 2, for ease of description, the user exposure adjustment gain value designated by the user is “Uexp”, and the user white balance adjustment gain values designated by the user are “UwbR, UwbG, UwbB ”is shown as a use parameter below the user exposure / WB adjustment processing unit 103.

  As for the intermediate developed image data, each of the user white balance adjustment gain values UwbR, UwbG, and UwbB has a value “as shown in FIG. 2 as a parameter of the intermediate developed image data stored and held in the RAM 123. 1 ”.

  This is done by canceling the normal exposure / WB adjustment processing performed by the normal exposure / WB adjustment processing unit 101 of the stage 1 as can be seen from the equations (1) to (6) shown in FIG. This is because it is possible to generate result image data reflecting only the user exposure / WB adjustment processing and present a result image corresponding to this to the user.

  Then, the intermediate developed image data subjected to the exposure / WB adjustment processing in the user exposure / WB adjustment processing unit 103 is supplied to the final development processing unit 104, where necessary development processing is performed to form an image to be presented to the user. As a result, image data Dd is formed and output. In the case of the example shown in FIG. 2, the final development processing unit 104 of the stage 2 performs a predetermined development process using the use parameter d, and generates result image data Dd.

  In this way, the result image data Dd is generated, and a result image corresponding to this is displayed on the display unit 106. The user confirms this, and the user changes the parameters of the exposure / WB adjustment processing through the operation unit 108. To do. In this case, since the intermediate developed image data has already been generated in the RAM 123, the control unit 120 controls the user exposure / WB adjustment processing unit 103, and the user is applied to the intermediate developed image data stored in the RAM 123. The exposure / WB adjustment process is performed with the parameters corresponding to the instruction input from.

  As a result, the user exposure / WB adjustment process according to the user's instruction input is performed without repeating the development process from the stage 1, and the result image data is generated through the final development processing unit 104, and according to the result image data The result image can be provided to the user.

  As described above, in the image processing apparatus 100 of the first embodiment, the stage 1 performs the normal exposure / WB adjustment process using the predetermined adjustment value, and the stage 2 uses the adjustment value input by the user. By having a configuration in which the exposure / WB adjustment can be performed twice, such as performing the user exposure / WB adjustment processing, the entire Raw development processing time through the stage 1 and the stage 2 becomes slightly longer.

  However, when the user performs exposure / WB adjustment, an intermediate developed image can be used, and only the processing of stage 2 can be performed without repeating the processing of stage 1. The processing time of the second and subsequent development processes performed by changing the parameters of the exposure / WB adjustment process can be greatly shortened, and the same processing result as before can be obtained. This is very advantageous in the Raw development adjustment flow in which exposure / WB adjustment is frequently performed.

  FIG. 5 is a diagram for explaining a difference in processing time between the conventional RAW development processing and the RAW development processing in the image processing apparatus 100 according to the first embodiment. For example, when the exposure / WB adjustment processing is performed only in the stage 1 as shown in FIG. 13, the developed result image is confirmed, and the parameters for exposure / WB adjustment are changed. As shown in FIG. 5A, unless the development process is always performed from the stage 1, the development process according to the changed parameters for exposure / WB adjustment cannot be performed.

  Therefore, as shown in FIG. 13, in the case where the exposure / WB adjustment processing is performed only on stage 1, if the parameters for exposure / WB adjustment are changed, as shown in FIG. The process with 2 is repeated.

  On the other hand, in the case of the image processing apparatus 100 according to the first embodiment shown in FIG. 2, the stage 1 performs the normal exposure / WB adjustment process, and the stage 2 performs the user exposure / WB adjustment process. I have to. For this reason, as compared with the case of the conventional development processing shown in FIG. 13, the time for the user exposure / WB adjustment processing in the stage 2 increases.

  However, in the case of the image processing apparatus 100 according to the first embodiment, when the developed result image is confirmed and the parameter for exposure / WB adjustment is changed, as shown in FIG. 5B, Since the intermediate development image data generated in the RAM 123 may be used to perform processing from the stage 2, the parameters for exposure / WB adjustment are changed after the intermediate development image data is generated. Therefore, when the development process is performed again, there is no need to perform the stage 1 process.

  Therefore, even if the user changes the parameters for exposure / WB adjustment, the time required for the re-development process can be greatly reduced, and the load on the image processing apparatus 100 can be reduced. In addition, since the normal exposure / WB adjustment processing is also performed in the stage 1, high-quality development processing is realized without deteriorating the development quality of the resulting image.

[Development process performed in stage 1 and stage 2]
In the image processing apparatus 100 according to the first embodiment, in the stage 1 shown in FIG. 2, it is conceivable to perform various development processes in addition to exposure / WB adjustment, but the processing load is as large as possible (high) ) It is desirable to arrange development processing. As a matter of course, the Raw image interpolation processing may be performed by performing at least one of noise reduction processing, aberration / distortion correction processing, and camera shake / subject blur correction processing. By arranging these processes on the stage 1, the result is reflected in the intermediate developed image, and when the user performs other development adjustments including exposure / WB adjustment, the development process with a heavy processing load is not performed again. Will also be good.

  The noise reduction process includes various processes for reducing noise mixed in an image. The aberration / distortion correction process also includes various processes for reducing the influence of lens aberration and distortion. Further, the camera shake / subject blur correction process includes various processes for reducing image blur caused by vibration of the camera and image blur caused by movement of the subject itself.

  Noise reduction processing, aberration / distortion correction processing, and camera shake / subject blur correction processing are examples of development processing with a large processing load, and other development processing can be performed on the stage 1. In stage 1, it is not always necessary to perform a plurality of development processes with a large processing load, and at least the normal exposure / WB adjustment process may be performed.

  On the other hand, in the stage 2, it is desirable to arrange a developing process with a processing load as low as possible in addition to the user exposure / WB adjustment, contrary to the case of the stage 1. For example, it is conceivable to perform at least one of color space conversion processing, nonlinear spatialization processing, picture making processing, and sharpness adjustment processing.

  Here, the color space conversion processing means conversion of standards for color expression, and mutual changes between standards such as sRGB standard, sYCC standard, bg-sRGB standard established by IEC (International Electrotechnical Commission) are performed. It is a process to make. The color space standard given here as an example is an example, and the present invention is not limited to this, and various color space standards can be changed.

  The non-linear spatialization process includes various non-linear image conversion processes such as a gamma correction process, an edge (contour) enhancement process, and a smoothing process. Also, the picture making process includes various processes performed to form a feeling received from the entire image, such as emphasizing the skin color, intentionally making the tone of a flashy feeling, or making the image a hard feeling. Sharpness adjustment processing means adjustment of the frequency characteristics of the image.For example, the image is clarified by finely adjusting a fine part, or vice versa. It is processing such as blurring.

  Note that the color space conversion process, the non-linear spatialization process, the picture making process, and the sharpness adjustment process are examples of development processes with a small processing load, and other development processes can be performed in the stage 2. In stage 2, it is not always necessary to perform a plurality of development processes with a large processing load, and at least the user exposure / WB adjustment process may be performed.

[Summary of Development Processing in Image Processing Apparatus 100]
Next, the development processing performed in the image processing apparatus 100 of the first embodiment described above will be summarized with reference to the flowchart of FIG. FIG. 6 is a flowchart for explaining the developing process performed in the image processing apparatus 100 according to the first embodiment. The process shown in FIG. 6 is a process that is mainly executed by the control unit 120 when the start of the development process is instructed by the user through the operation unit 108, for example.

  When execution of development processing is instructed, the control unit 120 acquires raw image data to be developed and supplies it to the normal exposure / WB adjustment processing unit 101 (step S101). The processing of step S101 is acquired through the media drive 111 when the raw image data exists in the removable medium, and through the input / output terminal 113 including the external I / F 112 when the raw image data exists in the external device. Alternatively, the information is acquired by communicating with an external device through the communication I / F 114 and the transmission / reception antenna 115. If the raw image data is already stored in the HDD 107 of the own device, the raw image data is acquired by reading the target raw image data from the HDD 107.

  Then, the normal exposure / WB adjustment processing unit 101 performs normal exposure / WB adjustment processing on the raw image data supplied to the normal exposure / WB adjustment processing unit 101 according to the control of the control unit 120 using a predetermined adjustment value. The processed raw image data is supplied to the interpolating image processing unit 102 (step S102). The adjustment process performed in step S102 is a normal exposure / WB adjustment process performed according to the formula shown in FIG.

  Next, under the control of the control unit 120, the image processing unit 102 such as interpolation performs predetermined image processing such as interpolation processing on the raw image data after the normal exposure / WB adjustment processing supplied to the own device. To generate intermediate developed image data, which is supplied to the user exposure / WB adjustment processing unit 103 and recorded in the RAM 123 (step S103). As described above, the processing in step S102 and step S103 is processing performed by the normal exposure / WB adjustment processing unit 101 and the interpolating image processing unit 102 in stage 1.

  Thereafter, the user exposure / WB adjustment processing unit 103 performs exposure / WB adjustment using the adjustment value instructed by the user for the generated intermediate developed image data in accordance with the control of the control unit 120. The process is executed, and the intermediate development image data after the process is supplied to the final development processing unit 104 (step S104). The adjustment process performed in step S104 is a user exposure / WB adjustment process performed according to the formula shown in FIG.

  In the first development process, the adjustment value for the exposure / WB adjustment process is not yet instructed by the user, so in step S104, as described with reference to FIG. Only the process of canceling the normal exposure / WB adjustment process performed in the WB adjustment processing unit 101 is performed. As will be described later, when the user inputs an instruction to change the adjustment value of the exposure / WB adjustment process, the user exposure / WB adjustment process is performed according to the instruction input from the user. It will be.

  Then, the final development processing unit 104 performs necessary development processing on the intermediate development image data subjected to the exposure / WB adjustment processing from the user exposure / WB adjustment processing unit 103 to form an image to be provided to the user. Result image data is formed and displayed on the display unit 106 through the display controller 105 (step S105). As described above, the processing in step S104 and step S105 is processing performed by the user exposure / WB adjustment processing unit 103 and the final development processing unit 104 in stage 2.

  Thereafter, the control unit 120 receives an instruction input such as an instruction to change the adjustment value of the exposure / WB adjustment process from the user through the operation unit 108 (step S106). Then, the control unit 120 determines whether or not an instruction to change the adjustment value of the exposure / WB adjustment process has been input (step S107).

  If it is determined in the determination process of step S107 that an adjustment value change instruction for the exposure / WB adjustment process has been input, the process from step S104, that is, the process from stage 2 is repeated, and the adjustment value change instruction from the user is repeated. The user exposure / WB adjustment process according to the input is performed, the development process is performed again, the result image after the development process is generated, the result image according to this is displayed, and the process provided to the user repeat.

  In the determination process of step S107, when it is determined that an adjustment value change instruction for the exposure / WB adjustment process has not been input and an instruction to end the development process has been issued, the process shown in FIG. 6 ends.

  After the development processing shown in FIG. 6 is completed, the finally obtained result image data is recorded and retained in the HDD 107, or the original raw image data and intermediate developed image data are stored in the HDD 107. It can also be recorded and held. That is, only the data selected by the user among the result image data, the raw image data, and the intermediate developed image data can be recorded and held in the HDD 107.

  In addition, by forming a program for executing the processing of the flowchart shown in FIG. 6 and mounting the program on an information processing apparatus such as a personal computer, the information processing apparatus may be an image processing apparatus that performs raw image data development processing. it can.

[Cooperation between image processing apparatus 100 and imaging apparatus]
Next, a specific example in the case where the image processing apparatus 100 according to the first embodiment and the imaging apparatus perform development processing in cooperation with each other will be described. FIG. 7 is a diagram for describing a specific example in the case where the image processing apparatus 100 and the imaging apparatus 200 perform development processing in cooperation.

  As described with reference to FIG. 1, the image processing apparatus 100 according to the first embodiment includes a communication I / F 104 and a transmission / reception antenna 105. For example, communication is performed through a wide area network such as a telephone communication network or the Internet. Has been able to do. On the other hand, it is assumed that there is an imaging apparatus 200 that can capture an image of a subject and can store the image as Raw image data and has a communication function, but cannot perform the development function of the Raw image data due to heavy processing.

  In such a case, as shown in FIG. 7, the first image processing apparatus 100 and the imaging apparatus 200 are connected through a wide area network 500. In FIG. 7, reference numeral 300 denotes a base station of a mobile phone network. Therefore, the image processing apparatus 100 and the imaging apparatus 200 are connected to each other via a wide area network 500 and a base station 300 so that data can be transmitted and received between them.

  Then, when it is necessary to develop the raw image data obtained by photographing the subject at the place where the user is going, the image capturing apparatus 200 uses the communication function of the own device to perform the image transmission through the base station 300 and the wide area network 500. The processing apparatus 100 is accessed and a communication line is connected to the image processing apparatus 100. Then, the imaging device 200 transmits Raw image data to the image processing device 100 as shown in (1-1).

  As shown in (2-1), the image processing apparatus 100 receives the raw image data from the imaging apparatus 200, and first performs the first development process as described with reference to FIGS. Then, as shown in (2-2), a result image obtained as a result of the development processing is returned to the imaging apparatus 200.

  As shown in (1-2), the imaging apparatus 200 receives the result image from the image processing apparatus 100 and displays it on the display element of the own apparatus so that the user can confirm it. Then, the imaging apparatus 200 receives an adjustment value change instruction input for exposure / WB adjustment through its operation unit, and adjusts for exposure / WB adjustment from the user as shown in (1-3). A parameter corresponding to the value change instruction input is transmitted to the image processing apparatus 100.

  As shown in (2-3), the image processing apparatus 100 receives only the parameter from the imaging apparatus 200, and in response to this parameter, the user exposure / WB adjustment processing unit 103 and the final development processing unit 104 of the stage 2 receive the parameter. The developed image is processed to generate a result image, which is transmitted to the imaging apparatus 200 as shown in (2-4).

  As shown in (1-4), the imaging apparatus 200 receives the result image from the image processing apparatus 100 and displays it on the display element of the own apparatus so that the user can confirm it. Then, the imaging apparatus 200 repeats the processes (1-3) and (1-4), and sets the parameters according to the adjustment value change instruction input for exposure / WB adjustment from the user. 100, and as a result, the image data is provided and displayed.

  One image processing apparatus 100 also repeats the processes shown in (2-3) and (2-4) according to the transmission information from the imaging apparatus, and is an intermediate generated from the raw image data provided first. The developed image is subjected to user exposure / WB adjustment processing and final development processing according to the parameters from the imaging device 200 to generate new result image data, and this processing is returned to the imaging device 200 repeatedly. To.

  As a result, the user of the imaging apparatus 200 uses his / her raw image using the raw image data development processing function of the image processing apparatus 100 even though the imaging apparatus 200 itself does not have the raw image data development processing function. Data development processing can be performed to check the developed image.

  Therefore, even when an inexpensive imaging device that does not have a RAW image data development processing function is configured, the RAW image data development function of the image processing device 100 connected through a wide area network is used, anytime, anywhere. Raw image data of a photographed subject can be developed and used.

  As a result, the RAW image data obtained by photographing the subject in the field can be appropriately developed according to the photographer's intention, and can be used quickly.

  As described above, the image processing apparatus 100 according to the first embodiment can perform development adjustment on the RAW image data obtained by photographing after photographing.

  Further, when the imaging apparatus 200 has a function as an imaging apparatus such as a mobile phone terminal or a PDA, but does not have a function for developing raw image data (the processing capability of the user terminal is insufficient). In other cases, the raw development processing, the raw image data and the intermediate development image data are all stored on the image processing apparatus 100 on the wide area network, and only the development result is transmitted to the user terminal. Can be realized. In this case, the image processing device can be realized as a personal device installed in each individual home, or a general-purpose device provided on a network as a so-called server device that processes Raw image data of an unspecified number of users. Can also be realized.

[Generation and retention of multiple types of intermediate development images]
In the case of the image processing apparatus 100 according to the first embodiment described above, the development process when the user changes the adjustment value of the exposure / WB adjustment process has been described. In this case, as described above, since the user exposure / WB adjustment processing is performed on the intermediate developed image data, the intermediate developed image data itself is not changed.

  However, there may be a case where the parameters of the development processing with a heavy processing load executed in stage 1 are changed. The image processing apparatus 100 according to the first embodiment can change the development parameters of the stage 1 through the operation unit 108. In this case, not the process from stage 2 but the process from stage 1 can be performed using the changed parameters as usual. In this case, the intermediate developed image data generated this time and the previously developed intermediate developed image data are different in the intermediate image data itself if the parameters used are different.

  Therefore, in such a case, the control unit 120 overwrites and saves the new intermediate image data generated this time on the old intermediate developed image data generated last time from the same raw image data, or the memory 120 If there is room in the space, it is possible to retain both the new intermediate developed image data and the old intermediate developed image data without overwriting.

  In addition, a plurality of parameters can be selected and used in the stage 1. FIG. 8 is a diagram for explaining a case where a plurality of intermediate developed image data can be generated by selecting and using a plurality of parameters in stage 1. As shown in FIG. 8, when a plurality of different parameters such as parameter 1, parameter 2, parameter 3,... Can be used in stage 1 for one raw image data Rd, the plurality of parameters are used. Development processing is performed using different parameters.

  If the parameters used in stage 1 are different, the generated intermediate developed images are also different as shown by intermediate developed images C1, C2, C3,... In FIG. To keep in memory. In the stage 2, by using any of the intermediate developed images C1, C2, C3,..., The development processing can be continued, so that different result images Dd1,. It is also possible to obtain Dd2, Dd3,.

  That is, when development processing is performed using a plurality of different parameters in stage 1, each of the plurality of different intermediate development images obtained as a result can be stored and held, and the intermediate development image used in stage 2 can be stored by the user. Can be selected through the operation unit 108, more flexible raw image data development processing can be realized.

  In addition, by making it possible to store and hold a plurality of different intermediate developed images, it is not necessary to repeat the development process in the stage 1 according to the parameters already used. There is no need to repeat the development processing used.

  In the case of the example shown in FIG. 8, the parameters are not changed in the stage 2 processing, but the development processing parameters in the stage 2 can of course be changed as described above. Therefore, in the case of this example, in stage 2, the final developed image is obtained by performing development processing using the target parameter for the user on the intermediate developed image targeted by the user. Can do.

  In the image processing apparatus 100 according to the first embodiment described above, the exposure / WB adjustment process, that is, the exposure adjustment process and the white balance adjustment process are performed in both the stage 1 and the stage 2. However, it is not limited to this. For example, if the image quality of the result image is satisfactory, at least one of the exposure adjustment process and the white balance adjustment process can be executed in the stage 1 and the stage 2.

  As described above, even when only one of the exposure adjustment process and the white balance adjustment process is performed in each of the stage 1 and the stage 2, the development process after the user changes the parameter is performed in the stage. By performing the processing from step 2, it is possible to shorten the time related to the development processing.

  In the image processing apparatus 100 according to the first embodiment described above, the exposure / WB adjustment process performed in the stage 1 is canceled in the development process of the stage 2, but the present invention is not limited to this. . Since the intermediate developed image data that has been subjected to the normal exposure / WB adjustment processing performed in stage 1 on the raw image data is generated, the user is further subjected to the exposure / WB adjustment processing. May be.

  In the image processing apparatus 100 according to the first embodiment described above, the Raw image data, the intermediate developed image data, and the result image data can be recorded and held in the HDD 107. However, the user can perform exposure / WB. When the adjustment process is performed, the development process using the same parameter is repeatedly performed by storing and holding the parameter used in stage 2 in association with the image data stored and held in the HDD 107. It can also be used as a reference for parameter adjustment in subsequent development processing.

[Second Embodiment]
Next, a camera system (imaging apparatus) 600 to which an embodiment of the apparatus, method, and program according to the present invention is applied will be described. The camera system 600 according to the second embodiment can be realized as, for example, a digital camera alone or as a camera function mounted on a mobile phone terminal, a PDA, or the like. In the following, in order to simplify the description, a case where the present invention is applied to a camera system configured as a digital still camera will be described as an example.

  FIG. 9 is a block diagram for explaining a camera system 600 according to the second embodiment. As shown in FIG. 9, the camera system 600 according to the second embodiment is roughly divided into a lens unit 601, an image sensor unit 602, an I / F unit LSI 603, a signal processing LSI 604, and a display treatment LSI 605. The signal processing LSI 4 includes a nonvolatile memory 607, a general-purpose DRAM 608, a recording medium 609, and an operation unit 610.

  Here, the non-volatile memory 607 stores data that should be retained even when the power of the camera system 600 of the second embodiment is turned off, for example, various parameters that are used in a fixed manner, processing results, and additional functions. A program or the like is stored and held. The DRAM 608 is a so-called volatile memory, and is mainly used as a work area, such as temporarily storing intermediate results of various processes.

  The recording medium 609 is a main recording medium in the camera system 600 of the second embodiment. For example, a built-in semiconductor memory, a memory card using the semiconductor memory and removable, a small hard disk Etc. In the camera system 600 of the second embodiment, the recording medium 609 is, for example, a removable memory card. The operation unit 610 includes a shutter button switch (hereinafter referred to as a shutter button), a telephoto adjustment key, and other various adjustment keys and function keys.

  As shown in FIG. 9, the signal processing LSI 604 includes a sensor I / F unit 401, a detection processing unit 402, an image processing unit 403, a resolution conversion processing unit 404, a compression processing unit 405, a display processing unit 406, and a memory controller 407. 408, an expansion processing unit 409, a recording / reproduction processing unit 410, a CPU 411, and a synchronization signal generation unit 412.

  As shown in FIG. 9, the units constituting the signal processing LSI 604 are connected through a common bus, and data exchange between the processing units is basically performed through the common bus. However, data may be directly exchanged between processing units without going through a bus. When data is transferred by directly connecting the processing units without going through the bus, it can be regarded as one processing unit.

  The CPU 411 provided in the signal processing LSI 604 controls each unit in the signal processing LSI 604. The signal processing LSI 604 is also provided with a ROM (not shown) in which various processing paths and data necessary for processing are recorded in advance, and the CPU 411 reads out and executes a necessary program from the ROM. A control signal to be supplied to the control unit can be generated and supplied to each unit, or a signal from each unit can be received and processing corresponding to this can be performed. The synchronization signal generation unit 412 generates various timing signals such as a frame start reference signal VD, a line start reference signal HD, and the like, and supplies the timing signals to necessary units in the signal processing LSI 604.

  As described above, a plurality of processing units exist for each function in each LSI constituting the camera system 600. The configuration shown in FIG. 9 is an example thereof, and of course, the processing unit in each LSI shown in FIG. 9 may belong to another LSI depending on the design concept. The number of processing units constituting each LSI and the functions of each LSI vary depending on the purpose.

  In the camera system 600 according to the second embodiment, image data obtained by photographing can be developed as before, and the data can be compressed and recorded on a predetermined recording medium. Further, the raw image data before the development processing can be recorded on a predetermined recording medium, and the raw image data can be subjected to the development processing according to a user instruction.

  First, a normal photographing process performed in the camera system 600 of the second embodiment, that is, a case where image data obtained by photographing is developed, and the compressed data is recorded on a predetermined recording medium. A series of processes will be described.

  An image of the subject imaged on the imaging surface of the image sensor of the image sensor unit 602 through the lens unit 601 is converted into an electric signal (analog image signal) by the image sensor and supplied to the I / F unit LSI 603. The I / F unit LSI 603 converts the analog image signal supplied thereto into a digital signal to form data (raw image data) in a format to be supplied to the signal processing LSI 604, and supplies this to the signal processing LSI 604. To do.

  Raw image data from the I / F unit LSI 603 is received by the sensor I / F unit 401 of the signal processing LSI 604 and supplied to the detection processing unit 402. Based on the raw image data supplied thereto, the detection processing unit 402 generates various parameters for image processing to be performed later, such as parameters for exposure adjustment processing and parameters for white balance adjustment processing. The generated parameters and the raw image data are supplied to the image processing unit 403.

  In this case, the image processing unit 403 performs exposure / WB adjustment processing, interpolation processing, and other development processing on the raw image data from the detection processing unit 402 in accordance with various parameters from the detection processing unit 402. Finally, YC data (color difference data) is generated. Note that the image processing unit 403 can perform edge enhancement processing, so-called aperture control processing, noise reduction processing, and the like on YC data as necessary.

  In the camera system 600 according to the second embodiment, the YC data generated by the image processing unit 403 is temporarily stored in the DRAM 608 from the image processing unit 403 through the memory controller 408.

  Thus, the YC data (image data) once stored in the DRAM 608 is read from the DRAM 608 as necessary, processed in each processing unit, and written back to the DRAM 608 again. It is used as a work area so that image data of a target format can be formed.

  In the camera system 600 according to the second embodiment, when the YC data formed in the image processing unit 403 is temporarily stored in the DRAM 608, the resolution conversion processing unit 404 reads it and performs image scaling processing. Then, display image data is formed, and the display image data is temporarily stored in the DRAM 608 through the memory controller 408.

  When the display image data is formed in the DRAM 608 in this way, the memory controller 408 reads the display image data from the DRAM 608 under the control of the CPU 411 and supplies the display image data to the display processing unit 406. The display processing unit 406 forms display image data in a format to be supplied to the display processing LSI 605 from the display image data supplied thereto, and supplies this to the display processing LSI 605.

  The display processing LSI 605 interprets the image data from the display processing unit 406 of the signal processing LSI 604 according to the format, forms an image signal to be supplied to the display device 606 such as a liquid crystal panel, and supplies this to the display device 606. . The display device 606 includes a display element such as a liquid crystal panel as described above. The display device 606 receives an image signal from the display processing LSI 605 and supplies it to its own display element. An image corresponding to the signal can be displayed on the display screen of its own display element.

  When the user confirms the image displayed on the display screen of the display device 606 in this way and the user presses the shutter button of the operation unit 610, the CPU 411 controls the memory controller 408 to display the display formed in the DRAM 608. The image data is read out and supplied to the compression processing unit 405. The compression processing unit 405 forms image data for recording by subjecting the supplied image data to data compression processing according to a predetermined image compression method such as a JPEG (Joint Photographic Experts Group) method, and the DRAM 608 Temporarily store.

  The recording image data formed in the DRAM 608 is read by the memory controller 408 controlled by the CPU 411 and supplied to the recording / playback processing unit 410. The recording / playback processing unit 410 records the recording image data supplied thereto on the recording medium 609.

  In this way, an image of the subject can be photographed while being displayed on the display screen of the display device 606 for confirmation, and the developed compressed image data of the photographed subject image can be recorded on the recording medium 609. I have to.

  In the camera system 600 according to the second embodiment, the RAW image data is also recorded on the recording medium 609 at the time of shooting, and the recorded RAW image data is developed according to the parameter designated by the user. Processing is performed so that image data after development that suits the user's preference can also be recorded on the recording medium 609.

  Here, the raw image data recording process will be described. In the camera system 600 according to the second embodiment, the detection processing unit 402 temporarily stores raw image data to be detected in the DRAM 608 through the memory controller 408. That is, raw image data of image data that has been subjected to image processing and displayed on the display screen of the display device 6 is stored and held in the DRAM 608.

  As described above, when the shutter button of the operation unit 610 is pressed, the display image data formed in the DRAM 608 is compressed to generate recording image data as described above. In parallel with the process of recording this on the recording medium 609, the CPU 411 controls the memory controller 408 to read out raw image data stored in the DRAM 608 and stores it on the recording medium 609 through the recording / playback processing unit 410. It is possible to record.

  Accordingly, the camera system 600 according to the second embodiment records not only JPEG-compressed still image data but also Raw image data corresponding to the JPEG-compressed still image data on the recording medium 609. Will be able to.

  The JPEG-compressed still image data recorded on the recording medium 609 can be reproduced in accordance with a reproduction instruction input from a user received through the operation unit 610. That is, when playback of still image data recorded on the recording medium 609 is instructed through the operation unit 610, the CPU 411 controls the recording / playback processing unit 410 to record the compressed data from the recording medium 609 onto the recording medium 609. The still image data instructed to be reproduced is read out and supplied to the expansion processing unit 409.

  The decompression processing unit 409 decompresses the JPEG-compressed still image data supplied thereto to restore the original still image data before data compression, thereby forming display image data. Is supplied to the display processing unit 409. As described above, the display processing unit 409 forms display image data in a format to be supplied to the display processing LSI 605 from the display image data supplied thereto, and supplies the display image data to the display processing LSI 605.

  As described above, the display processing LSI 605 interprets the image data from the display processing unit 406 of the signal processing LSI 604 according to the format, forms an image signal to be supplied to the display device 606, and outputs this to the display device 606. Supply. As a result, the display device 606 displays an image corresponding to the supplied image signal on the display screen of its own display element.

  In the camera system 600 according to the second embodiment, when raw image data development processing is executed through the operation unit 610, selection of raw image data stored in the recording medium 609 is selected. Will be able to do. This selection of raw image data can be performed in various ways.

  For example, shooting date / time information and a recording sequence number are attached to Raw image data, specified by the shooting date / time and recording sequence number, or JPEG-compressed still image recorded and held in the recording medium 609. A thumbnail image of an image is displayed on the display screen of the display device 606, and a corresponding raw image data can be selected by selecting a target still image from the thumbnails. Is done.

  The raw image data selected in this way is read from the recording medium 609 by the CPU 411 controlling the recording / playback processing unit 410, and is temporarily stored in the DRAM 608 in the camera system 600 of the second embodiment. Remembered. The CPU 411 controls the memory controller 408 to read the temporarily stored raw image data, and supplies the raw image data to the image processing unit 403 so as to perform development processing.

  That is, as described above, the image processing unit 403 performs exposure / WB adjustment processing, interpolation processing, and other development processing on the RAW image data supplied thereto, and finally YC data (Color difference data) is generated. The parameters for development processing used in this case are used by storing and holding the parameters generated in the detection processing unit 402 in association with the raw image data at the time of shooting the raw image data. can do.

  Of course, when developing the raw image data, for example, a development processing parameter prepared in advance in a ROM or a non-volatile memory 607 (not shown) may be used. The image processing unit 403 of the camera system 600 according to the second embodiment also performs the development process between stage 1 and stage 2 as in the case of the image processing apparatus 100 according to the first embodiment. Stage development processing is performed.

  Then, as described in detail below, in stage 1, normal exposure / WB adjustment processing and interpolation processing using predetermined parameters are performed to generate intermediate developed image data, which is temporarily stored in DRAM 608. Remember. In stage 2, the normal exposure / WB adjustment process performed in stage 1 is canceled for the intermediate developed image data formed by the process in stage 1, and then input from the user input through operation unit 610. In addition to executing user exposure / WB adjustment processing according to instruction input, other necessary development processing is performed to form result image data (image data for display), which is temporarily stored in DRAM 608 through memory controller 408 To do.

  In this way, the raw image data stored and held in the recording medium 609 is formed by performing exposure / WB adjustment processing and other development processing in accordance with an instruction input from the user using the function of the image processing unit 403. The image data is read by the memory controller 408 according to the control of the CPU 411 and supplied to the display device 606 through the display processing unit 406 and the display processing LSI, and the image is displayed on the display screen of the display device 606. .

  Then, an image displayed on the display screen of the display device 606 is confirmed, and the parameters of the user exposure / WB adjustment process executed in the stage 2 can be changed. Thus, by repeatedly using the intermediate developed image data formed in the DRAM 608 and changing the parameters of the user exposure / WB adjustment processing repeatedly, the parameters of the user exposure / WB adjustment processing are driven, the user's desired color tone, An image having a texture can be formed.

[Configuration Example of Image Processing Unit 403]
FIG. 10 is a block diagram for explaining a configuration example of the image processing unit 403 of the camera system 600 according to the second embodiment. As shown in FIG. 10, the image processing unit 403 is roughly composed of two processing units, stage 1 and stage 2. As shown in FIG. 10, the stage 1 includes a normal exposure / WB adjustment processing unit 4031 and an image processing unit 4032 such as an interpolation. The stage 2 includes a user exposure / WB adjustment processing unit 4033 as shown in FIG. And a final development processing unit 4034.

  The stage 1 and stage 2 of the image processing unit 403 shown in FIG. 10 are the same as the stage 1 and stage 2 that perform the development processing of the image processing apparatus 100 according to the first embodiment described with reference to FIG. It is configured. That is, the normal exposure / WB adjustment processing unit 4031 of the stage 1 of the image processing unit 403 shown in FIG. 10 performs a predetermined parameter according to the equation shown in FIG. 3 on the Raw image data Rd supplied thereto. Is used to perform exposure adjustment processing and white balance adjustment processing.

  In the camera system 600 of the second embodiment, the parameters used in the normal exposure / WB adjustment processing unit 4031 of the stage 1 are the detection processing unit at the time of recording the raw image data as described above. The parameters for exposure / WB adjustment generated in 402 and recorded and held together with the raw image data are used.

  That is, in FIG. 10, the exposure adjustment parameter exp, which is the use parameter used in the normal exposure / WB adjustment processing unit 4031, and the three primary color white balance parameters wbR, wbG, and wbB are detected by the detection processing unit 402. Is generated when

  Then, the raw image data subjected to the normal exposure / WB adjustment processing in the normal exposure / WB adjustment processing unit 4031 is supplied to the image processing unit 4032 such as an interpolation. As described above, the image processing unit 4032 such as the interpolation generates intermediate developed image data by performing so-called demosaic processing or other necessary processing, and temporarily stores it in the DRAM 608. In the case of the example shown in FIG. 10, processing using the use parameters b and c is performed.

  As a result, as described in the DRAM 608 portion in FIG. 10, the intermediate developed image data temporarily stored in the DRAM 608 is the exposure adjustment parameter exp = 20, the three primary color white balance parameters wbR = 10, wbG = 30, wbB. = 40, and other parameters b = 20 and c = 30 are used and formed by the processing of stage 1. At this stage, since the user does not instruct the parameters for the exposure / WB adjustment processing, the user exposure adjustment parameter Uexp = 1, the three primary color white balance parameters UwbR = 1, UwbG = 1, and UwbB = 1. ing.

  Note that the processing performed by the image processing unit 4032 such as interpolation is processing with a relatively large processing load in addition to demosaic processing, such as noise reduction processing, aberration / distortion correction processing, camera shake / subject shake correction processing, and the like. One or more of them are allowed to do.

  Then, the stage 2 user exposure / WB adjustment processing unit 4033 of the image processing unit 403 shown in FIG. 10 operates on the intermediate development image data temporarily stored in the DRAM 608 according to the formula shown in FIG. The exposure adjustment process and the white balance adjustment process are performed using the parameters for the exposure / WB adjustment process in accordance with the instruction input from the user received through.

  That is, in the user exposure / WB adjustment processing unit 4033, as shown in FIG. 4, first, the exposure / WB adjustment processing performed in the normal exposure / WB adjustment processing unit 4031 of the stage 1 is canceled, and then the operation is performed. The exposure / WB adjustment process is executed with parameters according to the instruction input regarding the exposure / WB adjustment process received from the user through the unit 610.

  The image data processed by the user exposure / WB adjustment processing unit 4033 is supplied to the final development processing unit 4034, where final development processing is performed to generate result image data Dd that is image data for display. Is output. As described above, the result image data Dd is temporarily stored in the DRAM 608 as display image data, read out through the memory controller 408 under the control of the CPU 411, and is displayed in the display device 606 through the display processing unit 406 and the display processing LSI 605. Supplied. Thus, an image corresponding to the display image data (result image data) temporarily stored in the DRAM 608 is displayed on the display screen of the display device.

  Note that in the final development processing unit 4934 of the stage 2, one or more of, for example, color space conversion processing, nonlinear spatialization processing, picture making processing, and sharpness adjustment processing, which are processing with a relatively small processing load, are performed. To be done.

  As described above, also in the camera system 600 of the second embodiment, the stage 1 performs the normal exposure / WB adjustment process, and the stage 2 performs the user exposure / WB adjustment process, so that the user can Even if the parameter of the exposure / WB adjustment process is changed, the process is not repeated from the process of stage 1, and the exposure / WB adjustment process performed in stage 1 using the intermediate developed image data generated by the process of stage 1 It is possible to perform exposure / WB adjustment processing according to the user's instruction input after canceling.

  That is, also in the camera system 600 of the second embodiment, the image processing unit 403 performs development processing similar to the development processing in the image processing apparatus 100 of the first embodiment described with reference to FIG. Has been done.

  Also in the camera system 600 according to the second embodiment, since the normal exposure / WB adjustment process is performed in the stage 1 so that an appropriate development process can be performed, the exposure / WB adjustment process is performed. It is also possible to obtain a properly processed result image by appropriately performing development processing without performing demosaic processing or the like on raw image data that has not been processed.

  In the camera system 600 of the second embodiment, the exposure / WB adjustment process, that is, the exposure adjustment process and the white balance adjustment process are performed in both the stage 1 and the stage 2. This is not a limitation. For example, if the image quality of the result image is satisfactory, at least one of the exposure adjustment process and the white balance adjustment process can be executed in the stage 1 and the stage 2.

  As described above, even when only one of the exposure adjustment process and the white balance adjustment process is performed in each of the stage 1 and the stage 2, the development process after the user changes the parameter is performed in the stage. By performing the processing from step 2, it is possible to shorten the time related to the development processing.

  In the camera system 600 of the second embodiment described above, the exposure / WB adjustment process performed in the stage 1 is canceled in the development process of the stage 2, but the present invention is not limited to this. Since the intermediate developed image data that has been subjected to the normal exposure / WB adjustment processing performed in stage 1 on the raw image data is generated, the user is further subjected to the exposure / WB adjustment processing. May be.

  In addition, the camera system 600 according to the second embodiment can of course record and hold the intermediate developed image data and the result image data obtained by performing the developing process on the recording medium 609. When the user performs exposure / WB adjustment processing, the parameters used in stage 2 are stored and held in association with the image data stored and held in the recording medium 609. The development process using parameters can be repeated, or it can be used as a reference for parameter adjustment in subsequent development processes.

  In addition, since the camera system 600 according to the second embodiment includes the nonvolatile memory 607, the nonvolatile memory 607 stores the parameters used for the development processing and uses them repeatedly. It can also be.

  Also in the camera system 600 of the second embodiment, the function of the image processing unit 403 can be realized by software (program) executed by the CPU 411. In addition to the image processing unit 403, functions of each unit such as the detection processing unit 402, the resolution conversion processing unit 404, the compression processing unit 405, and the expansion processing unit 409 are also realized as software (programs) executed by the CPU 411. Can do.

  In the camera system 600 according to the second embodiment described above, Raw image data is also stored in the recording medium 609 together with normal JPEG image data at the time of shooting, and later development of the Raw image data is performed. Although processing is performed, the present invention is not limited to this. At the time of shooting, the shot image can be confirmed, and if necessary, the raw image data can be developed. That is, it is of course possible to perform the development processing of the raw image data at each photographing time point every time an image is photographed.

[Summary of embodiment]
In the image processing apparatus 100 according to the first embodiment and the camera system 600 according to the second embodiment described above, the processing load is as much as possible when developing the raw image that is the original signal image data of the image sensor. The development processing group with a large size is arranged in the previous stage (stage 1) and the development processing group with a small processing load is arranged in the subsequent stage (stage 2), and the intermediate developed image that has undergone only the development processing in the previous stage is retained even after the development processing is completed. I keep trying. Thus, by using the held intermediate developed image, it is possible to omit the first-stage processing and to repeat only the second-stage processing.

  When the exposure adjustment and white balance adjustment can be performed as the processing of the development processing group, the user performs the exposure / white balance adjustment while performing the exposure / white balance adjustment in the preceding process. The exposure / white balance can be adjusted by the stage 2 using the intermediate developed image. In this case, by using the intermediate developed image, the first-stage processing is omitted, and the second-stage exposure / WB adjustment process is repeatedly performed by the user, so that the user-exposure / WB-adjustment process is quickly performed. Will be able to.

  The exposure / white balance adjustment performed in the preceding development processing group (stage 1) may be a value stored by the imaging device that captured the raw image data to be processed, a value detected by the development processing device, or a reasonable effect. In the exposure / white balance adjustment performed in the subsequent development processing group (stage 2) using a fixed adjustment value desired by the user, the processing performed in stage 1 is canceled to reflect the user-specified exposure / white balance adjustment. Thereby, it is possible to cancel the exposure / WB adjustment process in the stage 1 and appropriately perform the exposure / WB adjustment process intended by the user.

  Further, as the development processing with a relatively large processing load executed in the stage 1, for example, one or more of noise reduction, aberration / distortion correction, interpolation processing, and camera shake / subject blur correction can be performed. .

  In addition, as development processing with a relatively small processing load executed in stage 2, for example, one of user exposure adjustment, user white balance adjustment, color space conversion processing, nonlinear spatialization processing, picture making processing, and sharpness adjustment. This can be done.

  Then, when the user performs exposure / white balance adjustment, development of user exposure adjustment / user white balance adjustment arranged on stage 2 is performed without using image processing of exposure / white balance adjustment arranged on stage 1. By using the processing apparatus, the exposure by the user can be performed by repeatedly performing the processing of the stage 2 using the intermediate developed image that has been subjected to the development processing having a heavy processing load without performing the development processing having a heavy processing load of the stage 1. Even when the / WB adjustment processing is adjusted, rapid development processing can be performed.

  Also, when adjusting development parameters for development processing arranged on stage 2, such as user exposure adjustment, user white balance adjustment, parameters for picture creation and sharpness, development time can be shortened by starting development processing from the intermediate development image It can be carried out.

  Further, when the user adjusts the development parameters for the development processing arranged on the stage 1, the processing result image of the stage 1 generated by the development processing, that is, the intermediate development image is overwritten on the previous intermediate development image or newly By continuing to hold, the processing of the stage 1 can be prevented from being repeated, and the development processing efficiency can be improved.

  As a result, the time burden on the user in the exposure / WB adjustment work is greatly reduced. Since the intermediate developed image can be used for the most frequently performed processing, the intermediate developed image can be effectively used, and a new memory usage and calculation load are not required.

  In addition, when the program for executing the processing of the flowchart shown in FIG. 6 that can be executed in the image processing apparatus 100 of the first embodiment and the camera system 600 of the second embodiment is formed, this is executed. It can be recorded and distributed on a recording medium such as a CD (Compact Disc), distributed through a wide area network such as the Internet, or distributed through a broadcasting medium such as digital television broadcasting. As a result, an information processing apparatus that does not have a developing function can be easily changed to an image processing apparatus that has a developing function.

1 is a block diagram for explaining an image processing apparatus 100 to which an embodiment of the present invention is applied. FIG. FIG. 2 is a block diagram for explaining details of development processing performed in the image processing apparatus 100 of FIG. 1. It is a figure for demonstrating the specific example of the exposure adjustment process and the white balance adjustment process which are performed in the normal exposure / WB adjustment process part 101 of the stage 1 shown in FIG. It is a figure for demonstrating the specific example of the exposure adjustment process and white balance adjustment process which are performed in the normal exposure / WB adjustment process part 103 of the stage 2 shown in FIG. 7 is a diagram for explaining a difference in processing time between a conventional RAW development process and a RAW development process in the image processing apparatus 100. FIG. 3 is a flowchart for explaining development processing performed in the image processing apparatus 100 shown in FIG. 1. FIG. 10 is a diagram for describing a specific example in the case where the image processing apparatus 100 and the imaging apparatus 200 perform development processing in cooperation. It is a figure for demonstrating the case where a some intermediate | middle development image data can be produced | generated by selecting and using a several parameter in stage 1. FIG. It is a block diagram for explaining a camera system 600 to which an embodiment of the present invention is applied. It is a block diagram for demonstrating the structural example of the image process part 403 of the camera system 600 shown in FIG. It is a flowchart for demonstrating an example of the conventional Raw image development processing. It is a block diagram for demonstrating an example of the conventional Raw image development processing. It is a block diagram for demonstrating an example of the conventional Raw image development processing. It is a block diagram for demonstrating the problem in the conventional Raw image development processing. It is a block diagram for demonstrating the problem in the conventional Raw image development processing.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus, 101 ... Normal exposure / WB adjustment process part, 102 ... Image processing part, such as interpolation, 103 ... User exposure / WB adjustment process part, 104 ... Final development process part, 105 ... Display controller, 106 ... Display part 107, hard disk drive, 108, operation unit, 111, media drive, 112, external interface, 114, communication interface, 115, transmission / reception antenna, 120, control unit 120, 121, CPU, 122, ROM, 123, RAM, 124 ... EEPROM, 125 ... CPU bus, 600 ... camera system, 601 ... lens section, 602 ... imaging element section, 603 ... I / F section LSI, 604 ... signal processing LSI, 401 ... sensor I / F section, 402 ... detection processing 403 ... Image processing unit 404 ... Resolution conversion processing unit 405 ... Pressure Processing unit, 406 ... Display processing unit, 407, 408 ... Memory controller, 409 ... Decompression processing unit, 410 ... Recording / playback processing unit, 411 ... CPU, 412 ... Synchronization signal generation unit, 605 ... Display processing LSI, 606 ... Display device 607: Non-volatile memory, 608: DRAM, 609: Recording medium, 610: Operation unit

Claims (11)

First development processing means for receiving intermediate supply of raw image data that is undeveloped image data, and performing one or more development processes with a large processing load on the raw image data to generate intermediate developed image data When,
Storage means for storing and holding the intermediate developed image data generated in the first development processing means;
One or more development processes with a small processing load are performed on the intermediate developed image data from the first development processing unit or on the intermediate developed image data stored and held in the storage unit. An image processing apparatus comprising: a second development processing unit that generates image data resulting from the development process. The image processing apparatus according to claim 1,
Comprising a receiving means for receiving an instruction input relating to development processing from a user;
The one or more development processes performed in the first development processing unit include one or both of an exposure adjustment process and a white balance adjustment process,
When one or both of an instruction input related to exposure adjustment processing and an instruction input related to white balance adjustment processing are received through the receiving means,
The second development processing unit passes the reception unit for the intermediate development image data from the first development processing unit or the intermediate development image data stored in the storage unit. An image processing apparatus that executes one or both of an exposure adjustment process and a white balance adjustment process in accordance with a received instruction input. The image processing apparatus according to claim 2,
When one or both of the exposure adjustment process and the white balance adjustment process are executed by the first development processing unit, the first development processing unit includes an adjustment value supplied together with the raw image data, and the raw value. Performing an adjustment process using one or more of an adjustment value obtained by detecting image data and a predetermined fixed adjustment value;
When one or both of exposure adjustment processing and white balance adjustment processing is executed by the second development processing means, the second development processing means responds to the adjustment value used in the first development processing means. Then, after canceling the same adjustment process as the adjustment process to be performed by the second development processing unit performed by the first processing unit, the adjustment process is performed according to the instruction input received through the reception unit. An image processing apparatus characterized by executing The image processing apparatus according to claim 2,
The processing executed in the first development processing means includes, in addition to one or both of the exposure adjustment processing and white balance processing, noise reduction processing, aberration / distortion correction processing, interpolation processing, camera shake / subject blur correction. An image processing apparatus comprising one or more of processes. The image processing apparatus according to claim 2,
The processing executed by the second development processing means includes, in addition to one or both of the exposure adjustment processing and the white balance processing, color space conversion processing, nonlinear spatialization processing, picture making processing, and sharpness adjustment processing. An image processing apparatus comprising one or more of the above. The image processing apparatus according to claim 3,
When one or both of an instruction input related to exposure adjustment processing and an instruction input related to white balance adjustment processing are received through the receiving means,
Without the first development processing means repeating the development processing,
The second development processing means passes through the receiving means for the intermediate development image data from the first development processing means or for the intermediate development image data stored in the storage means. An image processing apparatus that executes one or both of an exposure adjustment process and a white balance adjustment process in accordance with a received instruction input. The image processing apparatus according to claim 5,
When an instruction input relating to processing executed by the second development processing unit is received through the reception unit, the second processing unit adds the intermediate developed image data from the first development processing unit to the intermediate developed image data. Alternatively, the image processing apparatus executes processing corresponding to the instruction input received through the receiving unit with respect to the intermediate developed image data stored and held in the storage unit. The image processing apparatus according to claim 4,
When an instruction input relating to processing executed by the first development processing unit is received through the reception unit, the first development processing unit performs a process according to the instruction input received through the reception unit. Image processing characterized in that new intermediate developed image data is generated by executing and overwritten on the previously generated intermediate processed image data stored in the storage means or additionally recorded in the storage means apparatus. A first development processing step for generating an intermediate developed image by performing one or more development processes with a large processing load on Raw image data that is undeveloped image data;
A recording step of recording the intermediate developed image data generated in the first development processing step in a predetermined memory;
One or more development processes with a small processing load are applied to the intermediate developed image data generated in the first development processing step or to the intermediate developed image data stored and held in the predetermined memory. And a second development processing step for generating image data resulting from the development processing. A program to be executed by a computer mounted on an image processing apparatus that develops raw image data that is undeveloped image data,
A first development processing step for generating intermediate development image data by performing one or more development processes with a large processing load on the raw image data;
A recording step of recording the intermediate developed image data generated in the first development processing step in a predetermined storage unit;
One or more development processes with a small processing load are applied to the intermediate developed image data generated in the first development processing step or to the intermediate developed image data stored and held in the storage means. And an image processing program for causing the computer of the image processing apparatus to execute a second development processing step of generating image data obtained as a result of the development processing. An image sensor that captures a subject image;
First development processing means for generating intermediate developed image data by performing one or more development processes with a large processing load on raw image data that is undeveloped image data captured through the image sensor;
Storage means for storing and holding the intermediate developed image data generated in the first development processing means;
One or more development processes with a small processing load are performed on the intermediate developed image data from the first development processing unit or on the intermediate developed image data stored and held in the storage unit. An image pickup apparatus comprising: a second development processing unit that generates image data resulting from the development process.

Images