JP6780762B2 - Electronics - Google Patents

Description

The present invention relates to electronic devices.

An imaging device equipped with an image sensor capable of setting imaging conditions for each screen area is known (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2006-49361

The conventional technique has a problem that it is difficult to set imaging conditions for each region.

The electronic device according to one aspect of the invention uses a display unit that displays a plurality of images of a subject imaged in the first image pickup region of the image sensor, and an image selected from the plurality of images displayed on the display unit. A control unit that controls the image pickup device so that the image pickup condition of the first image pickup region is different from the image pickup condition of the second image pickup region of the image pickup element is provided.

It is a block diagram which illustrates the structure of the camera by one Embodiment. It is sectional drawing of the laminated type image sensor. It is a figure explaining the pixel arrangement and the unit area of an image pickup chip. It is a figure explaining the circuit in a unit area. It is a block diagram which shows the functional structure of the image sensor corresponding to the circuit of FIG. It is a figure which shows typically the image of the subject which is imaged on the image sensor of a camera. It is a figure which illustrates the setting screen of the imaging condition. It is a figure which illustrates the setting screen of the imaging condition. It is a figure which illustrates the display screen which displays the candidate image side by side. It is a figure which illustrates the display screen which displays the candidate image side by side. It is a figure which illustrates the display screen which displays the candidate image side by side. It is a figure which illustrates the display screen which displays the candidate image side by side. It is a figure which illustrates the display screen which displays the candidate image side by side. It is a flowchart explaining the flow of the process of setting the imaging condition for each region and imaging. It is a figure which illustrates the display screen by the modification 1. FIG. It is a figure which illustrates the display screen by the modification 1. FIG. It is a figure which illustrates the display screen by the modification 2. It is a figure which illustrates the display screen by the modification 2. It is a block diagram which illustrates the structure of the imaging system by the modification 5. It is a figure explaining the supply of a program to a mobile device.

<Camera description>
FIG. 1 is a block diagram illustrating the configuration of an electronic device (for example, camera 1) according to an embodiment. In FIG. 1, the camera 1 includes an imaging optical system 31, an imaging unit 32, an image processing unit 33, a work memory 34, a control unit 35, a liquid crystal monitor 36, an operating member 37, and a non-volatile memory 38. , A recording unit 39, and a light emitting device 40.

The imaging optical system 31 guides the light flux from the field of view to the imaging unit 32. The image pickup unit 32 includes the image pickup element 100 and the drive unit 32a, and photoelectrically converts the image of the subject imaged by the image pickup optical system 31. The image pickup unit 32 can take an image under the same conditions over the entire area of the image pickup surface of the image pickup element 100, or can take an image under different conditions for each region of the image pickup surface of the image pickup element 100. The details of the imaging unit 32 will be described later. The drive unit 32a generates a drive signal necessary for causing the image sensor 100 to perform storage control. An imaging instruction such as a charge accumulation time for the imaging unit 32 is transmitted from the control unit 35 to the driving unit 32a.

The image processing unit 33 cooperates with the work memory 34 to perform image processing on the image data captured by the image capturing unit 32. Image processing includes, for example, contour enhancement processing, gamma correction, white balance adjustment, display luminance adjustment, saturation adjustment, and the like. As will be described later, the image processing unit 33 may apply the same parameters or the like to the entire area of the image to perform image processing, or apply different parameters or the like to each area of the image to perform image processing. it can.

The work memory 34 is a memory for temporarily recording image data before and after image processing. The control unit 35 is composed of, for example, a CPU, and controls the entire operation by the camera 1. For example, a predetermined exposure calculation is performed based on the image signal acquired by the image pickup unit 32, and the charge accumulation time (exposure time) of the image pickup element 100 required for proper exposure, the aperture value of the image pickup optical system 31, the ISO sensitivity, and the like are determined. The exposure condition is determined and instructed to the drive unit 32a. Further, the image processing conditions for adjusting the saturation, contrast, sharpness, etc. are determined according to the imaging scene mode set in the camera 1 and the type of the detected subject element, and are instructed to the image processing unit 33. The detection of the subject element will be described later. Furthermore, when the light emitting device 40 is set to allow light emission, the light emitting amount of the light emitting device 40 required for proper exposure is determined and instructed to the light emitting device 40.

The control unit 35 includes an object detection unit 35a, a region division unit 35b, a setting unit 35c, an imaging condition setting unit 35d, an imaging control unit 35e, an AF calculation unit 35f, and a display control unit 35g. .. These are realized by software when the control unit 35 executes a program stored in the non-volatile memory 38, but these may be configured by an ASIC or the like.

The object detection unit 35a performs a known object recognition process, and from the image acquired by the image pickup unit 32, the object detection unit 35a includes a person (human face), an animal such as a dog or a cat (animal face), a plant, a bicycle, or an automobile. Detects subject elements such as vehicles such as trains, structures, stationary objects, landscapes such as mountains and clouds, and specific predetermined objects.

The area division unit 35b divides (divides) the image pickup screen by the image pickup unit 32 into a plurality of areas including the subject element detected as described above. The setting unit 35c ranks a plurality of areas divided by the area dividing unit 35b. How to rank them will be described later.

The imaging condition setting unit 35d sets imaging conditions for a plurality of regions divided by the region dividing unit 35b. The imaging conditions include the above exposure conditions (charge accumulation time, gain, ISO sensitivity, frame rate, etc.) and the above image processing conditions (for example, white balance adjustment parameters, gamma correction curve, display brightness adjustment parameters, saturation adjustment parameters, etc.). ) And. As for the imaging conditions, the same imaging conditions can be set for all of the plurality of regions, or different imaging conditions can be set for the plurality of regions.

The image pickup control unit 35e controls the image pickup unit 32 (image sensor 100) and the image processing unit 33 by applying the image pickup conditions set for each of the above regions by the image pickup condition setting unit 35d. As a result, it is possible for the image pickup unit 32 to perform imaging under different exposure conditions for each of a plurality of regions divided by the region division unit 35b, and for the image processing unit 33 to perform image capture under different exposure conditions. It is possible to perform image processing under different image processing conditions for each of a plurality of regions divided by 35b.

The AF calculation unit 35f performs an automatic focus adjustment (autofocus: AF) calculation for focusing on a corresponding subject at a predetermined position (called a focus point) on the imaging screen. The AF calculation unit 35f sends a drive signal for moving the focus lens of the imaging optical system 31 to the focusing position based on the AF calculation result.
The AF method may be either a contrast detection method or a phase difference detection method.

The display control unit 35g controls the display of the image to be displayed on the liquid crystal monitor 36. The display control unit 35g may, for example, display one image on the display surface of the liquid crystal monitor 36, or display a plurality of images side by side on the display surface of the liquid crystal monitor 36. In addition, the display control unit 35g scrolls and displays the image being displayed on the display surface of the liquid crystal monitor 36 in response to the user's operation.

The liquid crystal monitor 36 reproduces and displays an image processed by the image processing unit 33 and an image read by the recording unit 39. The liquid crystal monitor 36 also displays an operation menu screen, a setting screen for setting imaging conditions, and the like.

The operation member 37 is composed of various operation members such as a release button and a menu button. The operation member 37 sends an operation signal corresponding to each operation to the control unit 35. The operation member 37 also includes a touch operation member provided on the display surface of the liquid crystal monitor 36.

The non-volatile memory 38 records a program or the like executed by the control unit 35. The recording unit 39 records image data or the like on a recording medium composed of a memory card or the like (not shown) in response to an instruction from the control unit 35. Further, the recording unit 39 reads out the image data recorded on the recording medium in response to an instruction from the control unit 35.

The light emitting device 40 is a photographing auxiliary light source that illuminates the subject. For example, when the light emission permission is set, the light emitting device 40 emits light with the amount of light instructed by the control unit 35 in response to the light emission instruction from the control unit 35.

<Explanation of stacked image sensor>
The stacked image sensor 100 provided in the camera 1 described above will be described. The stacked image sensor 100 is described in International Publication WO13 / 164915, which was previously filed and published by the applicant of the present application. FIG. 2 is a cross-sectional view of the stacked image sensor 100. The image sensor 100 includes a back-illuminated image pickup chip 113 that outputs a pixel signal corresponding to incident light, a signal processing chip 111 that processes the pixel signal, and a memory chip 112 that records the pixel signal. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are laminated and electrically connected to each other by a conductive bump 109 such as Cu.

As shown in the figure, the incident light is mainly incident in the Z-axis plus direction indicated by the white arrow. In the present embodiment, in the image pickup chip 113, the surface on the side where the incident light is incident is referred to as a back surface (imaging surface). Further, as shown in the coordinate axes, the left direction of the paper surface orthogonal to the Z axis is defined as the X-axis plus direction, and the Z-axis and the front direction of the paper surface orthogonal to the X-axis are defined as the Y-axis plus direction. In some subsequent figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.

An example of the image pickup chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is arranged on the back surface side of the wiring layer 108. The PD layer 106 has a plurality of PDs (photodiodes) 104 arranged two-dimensionally and accumulating charges according to incident light, and a transistor 105 provided corresponding to the PD 104.

A color filter 102 is provided on the incident side of the incident light in the PD layer 106 via the passivation film 103. The color filter 102 has a plurality of types that transmit different wavelength regions from each other, and has a specific arrangement corresponding to each of the PD 104. The arrangement of the color filters 102 will be described later. A set of a color filter 102, a PD 104, and a transistor 105 forms one pixel.

A microlens 101 is provided on the incident side of the incident light in the color filter 102 corresponding to each pixel. The microlens 101 collects incident light toward the corresponding PD 104.

The wiring layer 108 has a wiring 107 that transmits a pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may have multiple layers, and may be provided with passive elements and active elements.

A plurality of bumps 109 are arranged on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the facing surfaces of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are aligned by pressurization or the like. The bumps 109 are joined together and electrically connected.

Similarly, a plurality of bumps 109 are arranged on the surfaces of the signal processing chip 111 and the memory chip 112 facing each other. These bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized, so that the aligned bumps 109 are joined to each other and electrically connected.

The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be adopted. Further, for example, about one bump 109 may be provided for one block described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. Further, in the peripheral region other than the pixel region in which the pixels are arranged, a bump larger than the bump 109 corresponding to the pixel region may be provided together.

The signal processing chip 111 has TSVs (Through Silicon Vias) 110 that connect circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in the peripheral region. Further, the TSV 110 may also be provided in the peripheral area of the image pickup chip 113 and the memory chip 112.

FIG. 3 is a diagram for explaining the pixel arrangement and the unit area 131 of the image pickup chip 113. In particular, the state in which the imaging chip 113 is observed from the back surface (imaging surface) side is shown. For example, 20 million or more pixels are arranged in a matrix in the pixel area. In the example of FIG. 3, 16 pixels of adjacent 4 pixels × 4 pixels form one unit region 131. The grid lines in the figure show the concept that adjacent pixels are grouped to form a unit region 131. The number of pixels forming the unit region 131 is not limited to this, and may be about 1000, for example, 32 pixels × 64 pixels, more or less.

As shown in the partially enlarged view of the pixel region, the unit region 131 of FIG. 3 includes four so-called Bayer arrays including four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R in the vertical and horizontal directions. The green pixels Gb and Gr are pixels having a green filter as the color filter 102, and receive light in the green wavelength band among the incident light. Similarly, the blue pixel B is a pixel having a blue filter as a color filter 102 and receives light in the blue wavelength band, and the red pixel R is a pixel having a red filter as the color filter 102 and has a red wavelength band. Receives light.

In the present embodiment, a plurality of blocks are defined so that each block includes at least one unit area 131, and each block can control the pixels included in each block with different control parameters. That is, it is possible to acquire an imaging signal having different imaging conditions between the pixel group included in a certain block and the pixel group included in another block. Examples of control parameters are frame rate, gain, thinning rate, number of rows or columns to be added to add pixel signals, charge accumulation time or number of times, digitization bits (word length), and the like. The image sensor 100 can freely thin out not only the row direction (X-axis direction of the image pickup chip 113) but also the column direction (Y-axis direction of the image pickup chip 113). Further, the control parameter may be a parameter in image processing after acquiring an image signal from a pixel.

FIG. 4 is a diagram illustrating a circuit in the unit region 131. In the example of FIG. 4, one unit region 131 is formed by 9 pixels of 3 adjacent pixels × 3 pixels. As described above, the number of pixels included in the unit region 131 is not limited to this, and may be less or more. The two-dimensional positions of the unit region 131 are indicated by reference numerals A to I.

The pixel reset transistor included in the unit region 131 is configured to be individually on / off for each pixel. In FIG. 4, a reset wiring 300 for turning on / off the reset transistor of the pixel A is provided, and a reset wiring 310 for turning on / off the reset transistor of the pixel B is provided separately from the reset wiring 300. Similarly, the reset wiring 320 for turning on / off the reset transistor of the pixel C is provided separately from the reset wirings 300 and 310. Dedicated reset wiring for turning on / off each reset transistor is also provided for the other pixels D to I.

The transfer transistors of the pixels included in the unit area 131 are also configured to be individually on / off for each pixel. In FIG. 4, a transfer wiring 302 for turning on / off the transfer transistor of pixel A, a transfer wiring 312 for turning on / off the transfer transistor of pixel B, and a transfer wiring 322 for turning on / off the transfer transistor of pixel C are separately provided. Dedicated transfer wiring for turning on / off each transfer transistor is also provided from the other pixel D to the pixel I.

Further, the pixel selection transistor included in the unit region 131 is also configured to be individually on / off for each pixel. In FIG. 4, the selection wiring 306 for turning on / off the selection transistor of pixel A, the selection wiring 316 for turning on / off the selection transistor of pixel B, and the selection wiring 326 for turning on / off the selection transistor of pixel C are separately provided. Dedicated selection wiring for turning on / off each selection transistor is also provided for the other pixels D to I.

The power supply wiring 304 is commonly connected by pixels A to pixels A included in the unit area 131. Similarly, the output wiring 308 is commonly connected by pixels A to pixels A included in the unit area 131. Further, the power supply wiring 304 is commonly connected between the plurality of unit areas, but the output wiring 308 is individually provided for each unit area 131. The load current source 309 supplies current to the output wiring 308. The load current source 309 may be provided on the imaging chip 113 side or the signal processing chip 111 side.

By turning on / off the reset transistor and the transfer transistor of the unit region 131 individually, charge accumulation including charge accumulation start time, accumulation end time, and transfer timing is controlled for pixels A to I included in the unit region 131. can do. Further, by turning on and off the selection transistors of the unit region 131 individually, the pixel signal of the pixel I can be output from each pixel A via the common output wiring 308.

Here, a so-called rolling shutter method is known in which charge accumulation is controlled in a regular order with respect to rows and columns for pixels A to I included in the unit region 131. If a pixel is selected for each row by the rolling shutter method and then a column is specified, pixel signals are output in the order of "ABCDEFGHI" in the example of FIG.

By configuring the circuit with the unit region 131 as a reference in this way, the charge accumulation time can be controlled for each unit region 131. In other words, pixel signals with different frame rates can be output between the unit regions 131. Further, in the imaging chip 113, the unit region 131 included in the other blocks is rested while the unit region 131 included in some blocks is charged (imaging), so that the unit region 131 included in the other blocks is rested in the predetermined block of the imaging chip 113. Only the image can be taken and the pixel signal can be output. Further, it is also possible to switch the block for accumulating charges (imaging) between frames (the target block for accumulating control), sequentially perform imaging with different blocks of the imaging chip 113, and output a pixel signal.

FIG. 5 is a block diagram showing a functional configuration of the image sensor 100 corresponding to the circuit illustrated in FIG. The analog multiplexer 411 sequentially selects nine PD 104s forming the unit area 131, and outputs each pixel signal to the output wiring 308 provided corresponding to the unit area 131. The multiplexer 411 is formed on the imaging chip 113 together with the PD 104.

The pixel signal output via the multiplexer 411 is CDS and A / by a signal processing circuit 412 formed on the signal processing chip 111 that performs correlated double sampling (CDS) analog / digital (A / D) conversion. D conversion is performed. The A / D converted pixel signal is passed to the demultiplexer 413 and stored in the pixel memory 414 corresponding to each pixel. The demultiplexer 413 and the pixel memory 414 are formed on the memory chip 112.

The arithmetic circuit 415 processes the pixel signal stored in the pixel memory 414 and hands it over to the image processing unit in the subsequent stage. The arithmetic circuit 415 may be provided on the signal processing chip 111 or the memory chip 112. In addition, although FIG. 5 shows the connection for one unit area 131, in reality, these exist for each unit area 131 and operate in parallel. However, the arithmetic circuit 415 does not have to exist for each unit area 131. For example, even if one arithmetic circuit 415 processes sequentially while referring to the value of the pixel memory 414 corresponding to each unit area 131 in order. Good.

As described above, the output wiring 308 is provided corresponding to each of the unit regions 131. Since the image sensor 100 has an image pickup chip 113, a signal processing chip 111, and a memory chip 112 stacked on top of each other, by using an electrical connection between the chips using bumps 109 for these output wirings 308, each chip is placed in the plane direction. Wiring can be routed without making it large.

<Image capture condition setting for each area>
In the present embodiment, the imaging conditions can be set for each of a plurality of blocks in the image sensor 100 (imaging chip 113). The control unit 35 makes the region divided by the region division unit 35b correspond to the block, and causes the control unit 35 to perform imaging under the imaging conditions set for each region.

The user sets imaging conditions for a desired region while checking the live view image displayed on the display surface of the liquid crystal monitor 36, for example. The live view image is a monitor image that is repeatedly captured at a predetermined frame rate (for example, 60 fps).

When the user sets (changes) the imaging conditions for a certain area, the control unit 35 causes the image processing unit 33 to perform image processing on the area of the live view image. The image processing performed here is an image processing for confirming the effect of changing the imaging conditions. For example, when a change is made to increase the brightness, image processing is performed to increase the display brightness displayed on the liquid crystal monitor 36. Further, for example, when the light emitting device 40 is changed to emit light, image processing is performed on a predetermined subject element (for example, a person) to increase the display brightness displayed on the liquid crystal monitor 36. Furthermore, for example, when a change for increasing the saturation is made, image processing for increasing the saturation is performed on a predetermined subject element (for example, a flower) to be displayed on the liquid crystal monitor 36.
In this way, the user can confirm the effect of changing the imaging conditions on the image-processed live view image before the main imaging. The imaging condition setting for each such region will be described in detail below.
It should be noted that the method is not limited to the method of performing image processing and displaying on the liquid crystal monitor 36 in order to confirm the effect of changing the image pickup condition operated on a certain area, and the change of the image pickup condition operated on a certain area is the same. A method may be used in which a live view image of the set (changed) imaging conditions is displayed on the liquid crystal monitor 36 by reflecting the image on the image sensor corresponding to the region.

FIG. 6 is a diagram schematically showing an image of a subject imaged on the image sensor 100 of the camera 1. The camera 1 obtains a live view image by photoelectrically converting the subject image before the imaging instruction is given.

The control unit 35 first sets the same imaging conditions for the entire area of the imaging chip 113 (that is, the entire area of the imaging surface). Of the imaging conditions, the exposure condition is determined based on the exposure condition calculated so as to obtain an appropriate exposure according to the photometric value of the subject brightness, or the exposure condition manually set by the user. Of the imaging conditions, the image processing conditions are determined to be image processing conditions prepared in advance so as to obtain standard image quality. Hereinafter, the imaging conditions determined in this way will be referred to as reference conditions. Further, the image acquired by applying the reference condition is called a reference image.

An example of determining the reference condition will be described. The control unit 35 is set to the "program auto" mode, the "shutter speed priority auto" mode, the "aperture priority auto" mode, and the "manual" mode according to the operation signal from the exposure mode changeover switch constituting the operation member 37. Select one of them. Then, the control unit 35 calculates the above exposure conditions (charge accumulation time, gain, ISO sensitivity, frame rate, etc.) by performing a predetermined exposure calculation according to the selected exposure mode.

Specifically, in the "program auto" mode, the control unit 35 determines, for example, the aperture value, shutter speed (charge accumulation time), gain, and the like so that the average brightness in the screen becomes appropriate. In the "shutter speed priority auto" mode, the control unit 35 uses the shutter speed (charge accumulation time) set by the user to set the aperture value, gain, etc. so that the average brightness in the screen becomes appropriate, for example. decide. In the "aperture priority auto" mode, the control unit 35 determines the shutter speed (charge accumulation time), gain, etc. using the aperture value set by the user, for example, so that the average brightness in the screen becomes appropriate. To do. In the "manual" mode, the control unit 35 calculates the deviation from the proper exposure based on the aperture value and the shutter speed (charge accumulation time) set by the user.

In FIG. 6, an image including a person 61a, an automobile 62a, a bag 63a, a mountain 64a, and clouds 65a and 66a is formed on the imaging surface of the imaging chip 113. The person 61a is holding the bag 63a with both hands. The car 62a is parked to the right rear of the person 61a.

<Area division>
Based on the live view image, the control unit 35 divides the image acquired as the live view image into a plurality of regions as follows. First, the object detection unit 35a detects the subject element from the live view image. A known subject recognition technique is used to detect the subject element. In the example of FIG. 6, the object detection unit 35a detects the person 61a, the automobile 62a, the bag 63a, the mountain 64a, the cloud 65a, and the cloud 66a as subject elements.

Next, the area division unit 35b divides the image acquired as the live view image into the area including the subject element based on the detection means. In this example, the area including the person 61a is the first area 61, the area including the automobile 62a is the second area 62, the area including the bag 63a is the third area 63, and the area including the mountain 64a is the fourth area 64. The region including the cloud 65a will be referred to as the fifth region 65, and the region including the cloud 66a will be referred to as the sixth region 66.

The setting unit 35c ranks a plurality of areas divided by the area dividing unit 35b. When the setting unit 35c detects the following areas (1) to (4) among the areas divided by the area dividing unit 35b, the setting unit 35c gives priority to the imaging condition over the other areas divided by the area dividing unit 35b. Set as an area to be changed.
(1) Area in which a specific subject is detected according to the imaging scene mode The specific subject differs depending on the set imaging scene mode of the camera 1. For example, when the imaging scene mode setting of the camera 1 is set to the portrait mode, the setting unit 35c sets the region including the automobile 62a as the second region 62 with respect to the first region 61 including the person 61a. The region including the bag 63a is the third region 63, the region including the mountain 64a is the fourth region 64, the region including the cloud 65a is the fifth region 65, and the region including the cloud 66a is the imaging condition more than the sixth region 66. Is set as the area to be changed preferentially. In the case of the portrait mode, it is based on the idea that there is a high possibility that the imaging conditions of the first region 61 including the person 61a are set (changed).

For example, when the imaging scene mode setting of the camera 1 is set to the close-up mode and the object detection unit 35a detects a flower as a subject element, the setting unit 35c sets the area including the flower. , Set as an area where the imaging conditions are changed preferentially over other areas. In the close-up mode, it is based on the idea that there is a high possibility of setting (changing) the imaging conditions of the area including flowers.

For example, when the image pickup scene mode setting of the camera 1 is set to the landscape mode and the mountain is detected as a subject element by the object detection unit 35a, the setting unit 35c sets the area including the mountain. It is set as an area where the imaging conditions are changed preferentially over other areas. In the landscape mode, it is based on the idea that there is a high possibility of setting (changing) the imaging conditions for the area including mountains.

For example, when the image pickup scene mode setting of the camera 1 is set to the beach mode and the sea is detected as a subject element by the object detection unit 35a, the setting unit 35c sets the area including the sea. It is set as an area where the imaging conditions are changed preferentially over other areas. In the case of beach mode, it is based on the idea that there is a high possibility of setting (changing) the imaging conditions of the area including the sea.

For example, when the imaging scene mode setting of the camera 1 is set to the sunset mode and the object detection unit 35a detects the red sky as a subject element, the setting unit 35c sets the area including the red sky. Therefore, it is set as an area in which the imaging conditions are preferentially changed over other areas. In the case of the sunset mode, it is based on the idea that there is a high possibility of setting (changing) the imaging conditions of the area including the red sky.

For example, when the imaging scene mode setting of the camera 1 is set to the cooking mode, and the object detection unit 35a detects the food on the plate as the subject element, the setting unit 35c sets the plate and the food. The region including the above is set as an region in which the imaging conditions are preferentially changed over the other regions. In the cooking mode, it is based on the idea that there is a high possibility of setting (changing) the imaging conditions of the area including the dishes served on the plate.

The setting of the imaging scene mode of the camera 1 may be set by the user operating the operation member 37, or may be set by the control unit 37 based on the subject element detected from the screen of the live view image. It may be.

(2) Area including a subject element in focus When an AF operation for focusing on a subject corresponding to the focus point is performed, the setting unit 35c sets the subject element in focus (that is, the focus point). The region including the subject element corresponding to) is determined as an region in which the imaging condition is preferentially changed over the region including the subject element that is out of focus.
If there is no in-focus subject element, the area including the subject element with the smallest amount of focus shift (that is, the subject element closest from the position of the subject element to the in-focus position) is captured. The condition may be determined as an area to be changed preferentially.

(3) The area setting unit 35c including the subject element having the minimum brightness or the maximum brightness selects the subject element having the darkest brightness or the subject element having the brightest brightness among the plurality of subject elements detected by the object detection unit 35a. The region to be included is determined as a region in which the imaging conditions are preferentially changed over the other regions. It is based on the idea that there is a high possibility that imaging conditions will be set (changed) for areas that include too dark subject elements such as blackouts and too bright subject elements such as overexposure.

(4) The area setting unit 35c including the subject element having a high ratio of the specific color component is the above-mentioned green pixel Gb, Gr, or blue pixel B among the plurality of subject elements detected by the object detection unit 35a. Alternatively, a region including a subject element in which the ratio of the color component of the red pixel R is higher than the ratio of the other color components is determined as a region in which the imaging conditions are preferentially changed over the other regions. For example, an area containing subject elements such as sky blue, sea blue, mountain green, and sunset red is based on the idea that there is a high possibility of setting (changing) imaging conditions.

(5) The area including the subject element closest to the camera 1 The setting unit 35c sets the imaging condition preferentially over the distant subject element with respect to the area including the subject element closest to the camera 1. decide. For example, the control unit 35 acquires the distance information from the camera 1 to the divided subject element, and determines the area including the subject element close to the camera 1. Further, when the distance information cannot be obtained, the control unit 35 may determine the area including the subject element having the largest proportion of the screen as the subject element closest to the camera 1.

When the setting unit 35c detects the following areas (6) to (10) among the areas divided by the area dividing unit 35b, the order is lower than the above-mentioned areas (1) to (5). Therefore, it is determined as an area for changing the imaging conditions.

(6) Area including a subject element having the smallest amount of focus shift When an AF operation for focusing on a subject corresponding to the focus point is performed, a plurality of setting units 35c are detected by the object detection unit 35a. Of the subject elements in the above, the region including the subject element having the smallest amount of focus shift among the regions other than the region including the subject element in focus is set as the region in which the imaging conditions are preferentially changed over the other regions. Set. In general, the smaller the amount of focus shift, the higher the sharpness of the outline of the subject element. Therefore, it can be said that the subject element having a high degree of contour sharpness has a smaller amount of focus shift. The setting unit 35c sets a region including a subject element having a high contour sharpness as a region in which the imaging condition is preferentially changed over a region having a low contour sharpness.

(Example 1)
For example, in FIG. 6, it is assumed that the person 61a is in focus because the AF calculation unit performs the AF operation on the first region 61. The setting unit 35c is the subject element having the smallest amount of defocus among the automobile 62a, the bag 63a, the mountain 64a, the cloud 65a, and the cloud 66a (that is, the position of the person 61a in focus). The third region 63 including the bag 63a) located at a close position is set as an region in which the imaging conditions are preferentially changed next to the first region 61 including the person 61a. It is based on the idea that the imaging conditions of the third region 63 including the bag 63a are likely to be set (changed) next to the first region 61 including the person 61a.

(Example 2)
For example, in FIG. 6, it is assumed that the AF calculation unit is in focus between the person 61a and the automobile 62a. The setting unit 35c moves the subject element (that is, the position closest to the in-focus position) having the smallest amount of defocus among the automobile 62a, the bag 63a, the mountain 64a, the cloud 65a, and the cloud 66a. The second region 62 including a certain automobile 62a) is set as an region in which the imaging conditions are preferentially changed next to the first region 61 including the person 61a. It is based on the idea that the imaging conditions of the second region 62 including the automobile 62a are likely to be set (changed) next to the first region 61 including the person 61a.

(7) The area setting unit 35c including the subject element having a small distance difference sets the order based on the difference in the distance from the camera 1 to the subject element. For example, in FIG. 6, it is assumed that the person 61a is in focus because the AF operation is performed on the first region 61. In this case, the setting unit 35c has a distance from the camera 1 of the automobile 62a, the bag 63a, the mountain 64a, the cloud 65a, and the cloud 66a close to the distance from the camera 1 to the person 61a (distance difference). The area containing the subject element (smaller) is set as an area in which the imaging conditions are preferentially changed.

(8) Region Containing Subject Elements with Large Luminance Difference For example, in FIG. 6, it is assumed that the person 61a is in focus due to the AF operation performed on the first region 61. The setting unit 35c sets a region including a subject element having the largest brightness difference between the automobile 62a, the bag 63a, the mountain 64a, the cloud 65a, the cloud 65a, and the cloud 66a. It is set as a region in which the imaging conditions are preferentially changed next to the first region 61 including 61a. For example, when the brightness difference of the mountain 64a from the person 61a is the maximum, the setting unit 35c gives priority to the imaging condition in the fourth region 64 including the mountain 64a after the first region 61 including the person 61a. Set as the area to be changed. The region including the subject element having a large brightness difference with the person 61a is based on the idea that the imaging conditions are likely to be set (changed).

(9) Area containing a large subject element When the magnitude relation of the difference in the amount of focus shift between the regions including the subject element is poor and the magnitude relation of the brightness difference between the regions is also poor, the setting unit 35c may be used for another subject. An area containing a subject element larger than the element, in other words, an area occupying a larger area in the screen is set as an area for preferentially changing the imaging conditions.

(10) Area including a subject element near the center of the screen When the difference in the amount of focus shift between the areas including the subject element is poor and the difference in brightness between the areas is also poor, the setting unit 35c is used. , The area including the subject element closer to the center of the screen is set as the area for preferentially changing the imaging conditions.

In the present embodiment, any one of the above (6) to (10) may be selected in advance by the user operating the operation member 37. Further, a plurality of the above (6) to (10) may be combined.

By the above procedure, the setting unit 35c ranks the second region 63 to the sixth region 66. For example, the second region 62 including the automobile 62a, which is a large subject element, is ranked second, the third region 63 including the bag 63a is ranked third, the fourth region 64 including the mountain 64a is ranked fourth, and the cloud 65a is included. The fifth region 65 is the fifth place, and the sixth region 66 including the cloud 66a is the sixth place.

The setting unit 35c treats the regions that cannot be ranked even after the processes according to (6) to (10) above as having the same rank.

<Setting of imaging conditions for the area>
When the screen is divided into a plurality of areas by the area dividing unit 35b, the control unit 35 causes the liquid crystal monitor 36 to display a setting screen as illustrated in FIG. 7. In FIG. 7, the live view image 60a is displayed, and the icon 71 and the icon 72 are displayed on the live view image 60a. The live view image 60a is a reference image acquired by applying the above-mentioned reference conditions.

The icon 71 is an operation icon for selecting "brightness" as an example of setting items of imaging conditions. The icon 72 is an operation icon for selecting “saturation” as an example of setting items for imaging conditions. The operation icon for selecting the setting item of the imaging condition may be added as appropriate in addition to those illustrated in FIG. Further, the setting items of the imaging conditions may be selected and determined by the photographer, but may be determined and determined by the control unit 35 of the camera 1 from the detected subject image.

In FIG. 7, of the 1st to 6th regions, the region to be displayed with the outline emphasized (thick display, bright display, display with different colors, display with broken line, blinking display, etc.) is set (changed) for imaging conditions. ) Indicates the target area. In the example of FIG. 7, since the live view image 60a in which the outline of the first region 61 is emphasized is displayed, the first region 61 is the target for setting (changing) the imaging conditions. By emphasizing the outline of the first region 61, it is possible to clearly indicate to the user the region for which the imaging condition is set (changed). In the setting screen of FIG. 7, the first target for setting (changing) the imaging condition is the region in which the order given by the setting unit 35c is the first.

For example, in the camera 1 capable of touch operation, when the icon 71 is tapped by the user, the control unit 35 sets (changes) the highlighted area (first area 61). Select brightness as a setting item for imaging conditions.
In the following description, the camera 1 will be described on the premise of a touch operation, but the setting items of the imaging conditions may be selected by operating the buttons and the like constituting the operation member 37.

<When changing the brightness>
In the case of the setting screen illustrated in FIG. 7, the control unit 35 that selects the brightness as the setting item of the imaging condition determines the brightness of the first region 61 that is the target of setting (changing) the imaging condition in the live view image. The imaging conditions corresponding to the first region 61 are determined so as to be higher (or lower) than the brightness of the reference image. For regions (regions other than the first region 61) that are not subject to setting (changing) of imaging conditions, the set imaging conditions are maintained so as to maintain the brightness of the reference image.
Of the regions that are not subject to the setting (change) of the imaging conditions, in the vicinity of the boundary with the first region 61, the balance of the entire image is taken into consideration so as to blur the discontinuity of the imaging conditions in the vicinity of the boundary. The imaging conditions may be changed.

When increasing the brightness of the first region 61, the imaging condition setting unit 35d may, for example, make the charge accumulation time longer than the charge accumulation time of the reference condition among the imaging conditions of the first region 61, or increase the gain. The imaging condition for increasing the brightness is determined by increasing the gain as compared with the gain of the reference condition.

Further, when the brightness of the first region 61 is lowered, the imaging condition setting unit 35d may shorten the charge accumulation time of the imaging conditions of the first region 61, for example, as compared with the charge accumulation time of the reference condition, or gain. Is lower than the gain of the above reference condition, so that the imaging condition for lowering the brightness is determined.

When determining the imaging conditions for increasing or decreasing the brightness, the imaging condition setting unit 35d determines the imaging conditions in multiple stages (for example, eight stages) so that the brightness is gradually different from the brightness obtained under the reference conditions. To determine. However, the upper limit when the brightness is increased is a value at which the high-luminance region in the first region 61 of the live view image is not saturated (so-called whiteout does not occur). Further, the lower limit when the brightness is lowered is set to a value at which the low brightness region in the first region 61 of the live view image is not too dark (so-called black crushing does not occur).

The image pickup condition setting unit 35d determines the multi-step imaging conditions for increasing or decreasing the brightness in the first region 61 as described above, and sets the multi-step imaging conditions determined for the brightness in the image processing unit 33. The reason for setting the image processing unit 33 is to obtain a multi-stage candidate image, which will be described later, by performing image processing on the live view image.

<When changing the saturation>
In the case of the setting screen illustrated in FIG. 7, the control unit 35 that selects the saturation as the setting item of the imaging condition has the saturation of the first region 61 that is the target of setting (changing) the imaging condition in the live view image. The imaging condition corresponding to the first region 61 is determined so as to increase (or decrease) the saturation of the reference image. For regions that are not subject to setting (changing) of imaging conditions (regions other than the first region 61), the set imaging conditions are maintained so as to maintain the saturation of the reference image.
Of the regions that are not subject to the setting (change) of the imaging conditions, in the vicinity of the boundary with the first region 61, the balance of the entire image is taken into consideration so as to blur the discontinuity of the imaging conditions in the vicinity of the boundary. The imaging conditions may be changed.

When increasing the saturation of the first region 61, the imaging condition setting unit 35d receives a signal read from, for example, the green pixel Gb, Gr, or the blue pixel B, or the red pixel R among the imaging conditions of the first region 61. By increasing the value of in relative to the signal values of other color components, the imaging conditions for making a specific color vivid are determined.

When reducing the saturation of the first region 61, the imaging condition setting unit 35d is read from, for example, green pixels Gb, Gr, blue pixels B, and red pixels R among the imaging conditions of the first region 61. By making the value of the signal of the color component that protrudes from the other color components of the signal relatively low, the imaging condition for suppressing the vividness of a specific color is determined.

When determining the imaging conditions for increasing or decreasing the saturation, the imaging condition setting unit 35d performs multi-step (for example, eight-step) imaging so that the saturation is gradually different from the saturation obtained under the reference condition. Determine the conditions. However, the upper limit when increasing the saturation is a value at which the high saturation (protruding color component) signal in the first region 61 of the live view image is not saturated. Further, the lower limit when the saturation is lowered is a value that does not suppress the vividness until the color in the first region 61 of the live view image becomes achromatic.

The image pickup condition setting unit 35d determines the multi-step imaging conditions for increasing or decreasing the saturation for the first region 61 as described above, and sets the multi-step imaging conditions determined for the saturation in the image processing unit 33. To do. The reason for setting the image processing unit 33 is to obtain a multi-stage candidate image, which will be described later, by performing image processing on the live view image.

For example, when the imaging scene mode setting of the camera 1 is set to the portrait mode, the imaging condition setting unit 35d reduces the saturation of the person with respect to the first region 61 including the person 61a. The imaging conditions corresponding to one region 61 are determined. Further, the imaging condition setting unit 35d increases the saturation of the mountain with respect to the fourth area 64 in the area including the mountain 64a as the background (for example, the value of the signal read from the green pixels Gr and Gb is used as another value. The imaging conditions corresponding to the fourth region 64 are determined so as to be relatively higher than the signal values of the color components). In the case of portrait mode, it is based on the idea that there is a high possibility that the background will be vivid and the person 61a will be complemented.

For example, when the imaging scene mode setting of the camera 1 is set to the cooking mode and the object detecting unit 35a detects the food placed on the plate as the subject element, the imaging condition setting unit 35d sets the plate. And the imaging conditions are determined so as to increase the saturation for the area including the dish.

For example, when the imaging scene mode setting of the camera 1 is set to the autumn leaves mode and the object detection unit 35a detects the red autumn leaves as the subject element, the imaging condition setting unit 35d sets the region including the red autumn leaves. On the other hand, the imaging condition is determined so as to increase the saturation (for example, the value of the signal read from the red pixel R is relatively increased as compared with the value of the signal of another color component). The same applies when the imaging scene mode setting of the camera 1 is the sunset mode.

For example, when the image pickup scene mode setting of the camera 1 is set to the beach mode and the sea is detected as a subject element by the object detection unit 35a, the image pickup condition setting unit 35d is an area including the sea and the sky. On the other hand, the imaging conditions are determined so as to increase the saturation (for example, the value of the signal read from the blue pixel B is relatively increased as compared with the value of the signal of another color component).

<For other live view images>
FIG. 8 is a diagram illustrating a setting screen of the liquid crystal monitor 36 that displays a live view image 60b different from the live view image 60a. For example, when the imaging scene mode setting of the camera 1 is set to the close-up mode and the object detection unit 35a detects a red flower as a subject element, the setting unit 35c refers to the area including the red flower. Make the order first. As a result, the outline of the first region 61b is highlighted and displayed as a target region for setting (changing) the imaging conditions. As described above, in the close-up mode, it is based on the idea that there is a high possibility that the imaging conditions of the region including flowers are set (changed) with priority over other regions.
The icon 71 and the icon 72 in FIG. 8 are the same as in the case of the setting screen illustrated in FIG. 7, and thus the description thereof will be omitted.

In the case of the setting screen illustrated in FIG. 8, the control unit 35 that selects saturation as the setting item of the imaging condition relates to the first region 61b that is the target of setting (changing) the imaging condition in the live view image. , The imaging condition is determined so as to increase the saturation (for example, the value of the signal read from the red pixel R is relatively increased as compared with the value of the signal of another color component). By increasing the value of the signal read from the red pixel R relative to the value of the signal of another color component, the imaging conditions for making the color of the red flower vivid are determined.

As illustrated above, the imaging condition setting unit 35d receives signals read from the green pixels Gb, Gr, the blue pixel B, and the red pixel R for each color of the subject element in the area divided by the area dividing unit 35b. By changing the signal value of the color component that is more prominent than the other color components, the imaging conditions in which the saturation is changed are determined.

<Display of candidate images>
The image processing unit 33 is included in the first region 61 of the reference image acquired by applying the reference conditions when the multi-step imaging conditions for the above-mentioned brightness are set by the imaging control unit 35e. A plurality of images (8 images in the case of the above 8 steps) in which the brightness of the image is different in multiple steps are generated. In the present embodiment, the plurality of images thus generated are referred to as candidate images. The plurality of candidate images generated by the image processing unit 33 are temporarily recorded in the work memory 34.

Further, in the image processing unit 33, when the multi-step imaging conditions for the saturation described above are set by the imaging condition setting unit 35d, the saturation of the image in the first region 61b of the reference image is changed. A plurality of images (8 images in the case of the above 8 stages) different in multiple stages are generated. These plurality of images are also called candidate images. The plurality of candidate images generated by the image processing unit 33 are temporarily recorded in the work memory 34.

When the plurality of candidate images described above are recorded in the work memory 34, the display control unit 35g displays the candidate images side by side on the display surface of the liquid crystal monitor 36 as shown in FIG. 9 to 12 are diagrams illustrating a screen of the liquid crystal monitor 36, which displays candidate images side by side.

When the above-mentioned candidate images having different brightnesses in multiple stages are recorded in the work memory 34, the display control unit 35g causes the display surface of the liquid crystal monitor 36 to display the candidate images having these brightnesses side by side. Further, when the above-mentioned candidate images having different saturations in multiple stages are recorded in the work memory 34, the display control unit 35g arranges the candidate images for these saturations on the display surface of the liquid crystal monitor 36. Display it.

In FIG. 9, four candidate images (candidate images 60-1 to 60-4) out of the eight candidate images are displayed. The brightness (or saturation) of the candidate image 60-2 is one step higher than that of the candidate image 60-1. The brightness (or saturation) of the candidate image 60-3 is one step higher than that of the candidate image 60-2. The brightness (or saturation) of the candidate image 60-4 is one step higher than that of the candidate image 60-3.

The left icon 73 is displayed at the left position of the candidate image 60-1, and the right icon 74 is displayed at the right position of the candidate image 60-4. For example, when the user taps the right icon 74 with the finger 80 on the display screen of FIG. 9, the display control unit 35g displays four candidate images (candidate images) out of the eight candidate images as illustrated in FIG. 60-2 to 60-5) of candidate images are displayed. According to FIG. 10, the candidate image 60-1 displayed in FIG. 9 is no longer displayed, and the candidate image 60-5 that was not displayed in FIG. 9 is newly displayed. The brightness (or saturation) of the candidate image 60-5 is one step higher than that of the candidate image 60-4.

Although not shown, for example, when the user taps the left icon 73 with the finger 80 on the display screen of FIG. 10, the display control unit 35g is 4 out of 8 candidate images as illustrated in FIG. Two candidate images (candidate images 60-1 to 60-4) are displayed. That is, the candidate image 60-5 displayed in FIG. 10 is no longer displayed, and the candidate image 60-1 not displayed in FIG. 10 is newly displayed.

FIG. 11 is a diagram illustrating a screen of a liquid crystal monitor 36 for displaying four candidate images (candidate images 60-3 to 60-6) out of eight candidate images. The brightness of the candidate image 60-6 is one step higher than that of the candidate image 60-5.

For example, when the user slides the display screen from top to bottom in the direction of the arrow with the finger 80 on the display screen of FIG. 11, the display control unit 35g has eight candidate images as illustrated in FIG. Of these, four candidate images (candidate images 60-1 to 60-4) are displayed. According to FIG. 12, the candidate images 60-4 and 60-5 displayed in FIG. 11 are no longer displayed, and the candidate images 60-1 and 60-2 that were not displayed in FIG. 11 are newly replaced. Is displayed.

In this way, the user can confirm a plurality of candidate images. Further, the user can display and confirm other candidate images that are not displayed on the liquid crystal monitor 36 by tapping the left icon 73 or the right icon 74 or sliding with the finger 80.

FIG. 13 is a diagram illustrating a screen of a liquid crystal monitor 36 that displays candidate images having different saturations in multiple stages side by side. In FIG. 13, four candidate images (candidate images 160-1 to 160-4) out of eight candidate images are displayed. The saturation of the candidate image 160-2 is one step higher than that of the candidate image 160-1. The saturation of the candidate image 160-3 is one step higher than that of the candidate image 160-2. The saturation of the candidate image 160-4 is one step higher than that of the candidate image 160-3. The left icon 73 and the right icon 74 are the same as those illustrated in FIGS. 9 and 10, and thus the description thereof will be omitted.

<Selection operation>
The user selects an arbitrary candidate image from a plurality of candidate images. For example, when the user double-tap the position where any candidate image is displayed among the plurality of candidate images on the screen illustrated in FIGS. 9 to 12 with the finger 80, the control unit 35 is then instructed to take an image. When this is performed, the imaging conditions to be applied to the first region 61 are determined as follows, and the process returns to the state of displaying the setting screen as illustrated in FIG. 7 on the liquid crystal monitor 36.

That is, the control unit 35 (imaging control unit 35e) sets the imaging conditions applied to the first region 61 of the candidate image that has been double-tapped to the imaging unit 32 or the image processing unit 33. The reason for setting the image pickup unit 32 is that the brightness can be changed by changing the exposure condition of the image pickup device 100 at the time of the next imaging. Further, the reason for setting the image processing unit 33 is that the saturation can be changed by changing the image processing conditions of the image processing unit 33 at the time of the next imaging.

Specifically, when one of the plurality of candidate images for brightness is double-tapped, the imaging condition setting unit 35d first sets the exposure condition for obtaining the brightness corresponding to the double-tapped candidate image. It is determined as an imaging condition (exposure condition) to be applied to the region 61, and the imaging control unit 35e sets the condition in the imaging unit 32.

Further, when one of the plurality of candidate images for saturation is double-tapped on the screen illustrated in FIG. 13, the imaging condition setting unit 35d obtains the saturation corresponding to the double-tapped candidate image. The image processing condition for this purpose is determined as an image processing condition (image processing condition) to be applied to the first region 161, and the imaging control unit 35e sets the condition in the image processing unit 33.

The user can perform an operation of selecting an arbitrary candidate image from a plurality of candidate images related to brightness and then an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation. Further, the user can perform an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation and then an operation of selecting an arbitrary candidate image from a plurality of candidate images related to brightness.

Further, the user selects an arbitrary candidate image from a plurality of candidate images in terms of brightness or saturation even in an area other than the first area 61, that is, an area in which the order given by the setting unit 35c is second or lower. It can be performed. When the user taps an area other than the first area 61 with a finger while the setting screen illustrated in FIG. 7 is displayed on the liquid crystal monitor 36, the control unit 35 replaces the first area 61 with the control unit 35. The tapped area is highlighted and displayed. The imaging condition setting unit 35d sets (changes) the imaging condition in the highlighted area. Subsequent setting operations are the same as in the case of the first region 61 described above.

It should be noted that, without the user tapping an area other than the first area 61, the control unit 35 sets (changes) the imaging conditions in order for the areas in the second or lower rank given by the setting unit 35c. It may be switched as a target area.

In the present embodiment, an example of highlighting the outline of the region (first region 61 in the above example) for which the imaging condition is set (changed) has been described, but instead of highlighting the contour, the contrast of the entire target region is displayed. It may be raised and displayed, or the entire target area may be blinked. Further, the target area may be surrounded by a frame. The display of the frame surrounding the target area may be a double frame or a single frame, and the display mode such as the line type of the surrounding frame may be changed as appropriate. Further, the control unit 35 may display an arrow or the like in the vicinity of the target area to indicate the area to be set as the imaging condition. The control unit 35 may display the area other than the target area for which the imaging condition is set (changed) to be dark, or may display the area other than the target area with a low contrast.

After the imaging conditions for the regions described above are set, when the release button (not shown) constituting the operating member 37 or the display instructing the start of imaging is operated, the control unit 35 controls each of the divided regions. Imaging is performed under the imaging conditions set for the above, image processed image data is generated, and the image data is recorded on a recording medium composed of a memory card (not shown) or the like.

<Explanation of flowchart>
FIG. 14 is a flowchart illustrating a flow of processing in which imaging conditions are set for each region and imaging is performed. When the main switch of the camera 1 is turned on, the control unit 35 activates a program that executes the process shown in FIG. In step S10, the control unit 35 causes the liquid crystal monitor 36 to start the live view display, and proceeds to step S20.

Specifically, the control unit 35 instructs the imaging unit 32 to start acquiring the live view image, and sequentially displays the acquired live view image on the liquid crystal monitor 36. As described above, at this time, the same imaging conditions are set for the entire area of the imaging chip 113, that is, the entire screen.
When the AF operation is set during the live view display, the control unit 35 (AF calculation unit 35f) controls the AF operation for focusing on the subject element corresponding to the predetermined focus point. If the AF operation is not set during the live view display, the control unit 35 (AF calculation unit 35f) performs the AF operation when the AF operation is later instructed.

In step S20, the control unit 35 (object detection unit 35a) detects the subject element from the live view image and proceeds to step S30. In step S30, the control unit 35 (area dividing unit 35b) divides the screen of the live view image into an area including the subject element, and proceeds to step S40.

In step S40, the control unit 35 determines the above-mentioned reference condition based on the setting state of the camera 1 or the state of the subject. That is, the same imaging conditions are set for the entire screen, and the process proceeds to step S50.

In step S50, the control unit 35 acquires a live view image (reference image) to which the reference condition is applied, and proceeds to step S60. In step S60, the control unit 35 selects an area to be set (changed) for imaging conditions from the areas divided by the area dividing unit 35b. For example, the control unit 35 selects in order from the region in which the order given by the setting unit 35c is the first.

In step S70, the control unit 35 determines whether or not an operation for displaying the candidate image has been performed. As described above, when the icon 71 or 72 of FIG. 7 (or FIG. 8) is tapped by the user, the control unit 35 positively determines step S70 and proceeds to step S80, and taps both the icons 71 and 72. If not, step S70 is negatively determined and the process proceeds to step S120.

In step S80, the control unit 35 determines the imaging conditions to be set (changed). That is, the control unit 35 determines the setting item corresponding to the icon 71 or 72 tapped by the user, and proceeds to step S90.

In step S90, the control unit 35 (imaging condition setting unit 35d) acquires a plurality of candidate images for the setting items determined in step S80 and proceeds to step S100. In step S100, the control unit 35 displays a plurality of candidate images on the display surface of the liquid crystal monitor 36 and proceeds to step S110.

In step S110, when the above selection operation (for example, double tapping) is performed by the user, the control unit 35 corresponds to the double-tapped candidate image for the area to be set (changed) in the imaging condition. The image pickup condition is determined, the condition is set in the image pickup unit 32 or the image processing unit 33, and the process proceeds to step S120.
The display transition of the liquid crystal monitor 36 and the setting of the imaging conditions according to the user operation in step S110 are as described above.

In step S120, the control unit 35 determines the presence or absence of an imaging instruction. When the release button (not shown) constituting the operation member 37 or the display instructing the imaging is operated, the control unit 35 positively determines step S120 and proceeds to step S130. If the image pickup instruction is not given, the control unit 35 negatively determines step S120 and returns to step S70.

In step S130, the control unit 35 performs a predetermined main imaging process. That is, the image pickup control unit 35e controls the image pickup unit 32 so as to take an image under the image pickup conditions set for each of the above regions, and the image processing unit 33 performs predetermined image processing on the image data obtained by the image pickup. Give. Then, the recording unit 39 records the image data on a recording medium (not shown).

In step S140, the control unit 35 determines whether or not the end operation has been performed. When the end operation is performed, the control unit 35 positively determines step S140 and ends the process according to FIG. If the end operation is not performed, the control unit 35 negatively determines step S140 and returns to step S20. When returning to step S20, the control unit 35 repeats the above-described process.

According to the above-described embodiment, the following effects can be obtained.
(1) The camera 1 includes an image pickup unit 32 that captures a subject image by an image sensor 100 configured to be able to set imaging conditions for a plurality of regions on the imaging surface, and parameters for changing the imaging conditions for each region. It includes a control unit 35 that determines based on a subject element detected from the captured live view image. As a result, the camera 1 automatically determines the parameters for changing the imaging conditions for each region, so that the user can easily set the imaging conditions for each region.

(2) The control unit 35 determines the parameters based on the subject element and the set imaging mode. As a result, appropriate parameters are automatically determined according to the type of subject element (for example, person, mountain, flower, etc.) and the imaging mode (for example, portrait mode, landscape mode, etc.), so that the user can use it. It becomes easier to set the imaging conditions for each area.

(3) The camera 1 generates a candidate image in which the imaging conditions are changed by using the parameters determined by the control unit 35 in at least one region among the plurality of regions with respect to the captured live view image. A processing unit 33 is provided. As a result, the user can confirm the effect of changing the imaging conditions for the region by checking the candidate image in which the live view image is image-processed before the main imaging.

(4) The control unit 35 determines a plurality of parameters, and the image processing unit 33 steps the imaging conditions for the captured live view image using a plurality of parameters determined by the control unit 35 that are different from each other. Generate multiple modified candidate images. As a result, the user can compare the effect of changing the imaging conditions with a plurality of candidate images having different imaging conditions in stages.

(5) The imaging condition is brightness or saturation, and the image processing unit 33 generates a plurality of candidate images whose brightness or saturation is changed stepwise. This allows the user to compare the effect of changing the brightness (or saturation) with a plurality of candidate images having stepwise differences in brightness (or saturation).

(6) The image processing unit 33 generates the captured live view image as a reference image before changing the imaging conditions, and also generates a candidate image after changing the imaging conditions. This allows the user to compare the effect of changing the imaging conditions for the area with that before changing the imaging conditions.

(7) The image processing unit 33 generates a reference image, an image having higher brightness than the reference image, and / or an image having lower brightness than the reference image. This allows the user to compare the effect of changing the brightness on the area with that before changing the brightness.

(8) The image processing unit 33 generates a reference image, an image having a high saturation with respect to the reference image, and / or an image having a low saturation with respect to the reference image. This allows the user to compare the effect of changing the saturation on the area with that before changing the saturation.

(9) Since the liquid crystal monitor 36 for displaying the image generated by the image processing unit 33 is provided, the user can visually compare the effect of changing the imaging conditions with respect to the area. Become.

(10) When the operation of selecting the candidate image displayed on the liquid crystal monitor 36 is performed, the candidate image is generated for the region of the imaging surface of the imaging unit 32 corresponding to the region where the candidate image is generated. A control unit 35 for setting imaging conditions corresponding to the above parameters is provided. This makes it easier for the user to set the imaging conditions for each area.

(11) Of the plurality of regions, the display control unit 35g is provided so that the region for displaying the image generated by changing the imaging conditions by the image processing unit 33 is displayed on the liquid crystal monitor 36 in a mode different from that of the other regions. As a result, it is possible to clearly indicate to the user the area to be set (changed).

The following modifications are also within the scope of the present invention, and one or more of the modifications can be combined with the above-described embodiment.

(Modification example 1)
In the above description, as illustrated in FIGS. 9 and 13, an example in which the display control unit 35g displays a plurality of candidate images side by side on the display surface of the liquid crystal monitor 36 has been described. Instead, a reference image is displayed on the display surface of the liquid crystal monitor 36, and only the image in the region (first region 61 in the above example) of the reference image to be set (changed) in the imaging condition is displayed. , It may be displayed by replacing the image of the corresponding area in the other candidate image.

15 and 16 are diagrams illustrating a display screen according to the first modification. In the first modification, when the plurality of candidate images described above are stored in the work memory 34, the display control unit 35g displays a reference image (reference image) under the reference conditions on the display surface of the liquid crystal monitor 36 as shown in FIG. 60-1) is displayed. At this time, the image in the first region 61 is the reference image 61-1.
Here, the point that the outline of the first region 61, which is the target of setting (changing) the imaging conditions, is highlighted is the same as in the case of the above embodiment.

In FIG. 15, the left icon 73 is displayed at the left side position of the first area 61, and the right icon 74 is displayed at the right side position of the first area 61. For example, when the user taps the right icon 74 with the finger 80 on the display screen of FIG. 15, the display control unit 35g selects only the image in the first region 61 as the candidate image 61-as illustrated in FIG. The composite image replaced with 2 is displayed. The background other than the first region 61 remains the reference image 60-1.

The candidate image 61-2 is an image whose brightness (or saturation) is one step higher than that of the candidate image 61-1. In this way, the display control unit 35g reads out the necessary candidate images 60-2 from the eight candidate images temporarily stored in the work memory 34 in response to the tap operation of the right icon 74, and the read candidates. Only the candidate image 61-2 in the first region 61 of the images 60-2 is extracted and used for the composite display illustrated in FIG.

Although not shown, for example, when the user taps the left icon 73 with the finger 80 on the display screen of FIG. 16, the display control unit 35g is an image in the first region 61 as illustrated in FIG. Is replaced with the reference image 61-1 to display a composite image. The background other than the first region 61 remains the reference image 60-1.

According to the first modification, the user can confirm the candidate image for the first region 61 that is the target of setting (changing) the imaging condition. Further, the user can display and confirm another candidate image in the first area 61 by tapping the left icon 73 or the right icon 74.

In the first modification, when the user double-taps any position on the screen with the finger 80 while an arbitrary candidate image is displayed, the control unit 35 (imaging condition setting unit 35d) moves at that time. The imaging conditions applied to the candidate images 61-n (n is a natural number of 1 to 8) displayed in the first region 61 are applied to the first region 61 when an imaging instruction is next given. It is determined as the imaging condition to be performed, and the process returns to the state where the setting screen illustrated in FIG. 7 is displayed on the liquid crystal monitor 36.

It should be noted that the user can perform an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation and then an operation of selecting an arbitrary candidate image among a plurality of candidate images related to saturation. The point that the user can select an arbitrary candidate image from a plurality of candidate images related to brightness after performing an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation is described above. It is the same as the said embodiment.

Further, the user can set (change) the imaging conditions regarding the brightness or the saturation even in the area other than the first area 61, that is, the area in which the order given by the setting unit 35c is the second or lower. , The same as the above-described embodiment.

(Modification 2)
17 and 18 are diagrams illustrating a display screen according to the second modification. In the second modification, when the plurality of candidate images described above are stored in the work memory 34, the display control unit 35g displays a reference image of the reference condition on the left side of the display surface of the liquid crystal monitor 36 as shown in FIG. The reference image 60-1) is displayed, and the candidate image 60-2 is displayed on the right side of the display surface.
Here, the point that the outline of the first region 61, which is the target of setting (changing) the imaging conditions, is highlighted is the same as in the case of the above embodiment.

In FIG. 17, the upper icon 75 is displayed at the upper position of the candidate image 60-2 on the right side, and the lower icon 76 is displayed at the lower position of the candidate image 60-2. For example, when the user taps the lower icon 76 with the finger 80 on the display screen of FIG. 17, the display control unit 35g replaces the candidate image 60-2 on the right side with the candidate image 60 as illustrated in FIG. Display -3. The display of the reference image 60-1 on the left side remains unchanged.

The candidate image 60-3 is an image whose brightness (or saturation) is one step higher than that of the candidate image 60-2. In this way, the display control unit 35g reads out the necessary candidate images 60-3 in response to the tap operation of the lower icon 76 from the eight candidate images temporarily stored in the work memory 34, and shows FIG. Used for the illustrated display.

Although not shown, for example, when the user taps the upper icon 75 with the finger 80 on the display screen of FIG. 18, the display control unit 35g will perform the candidate image 60-3 on the right side as illustrated in FIG. The candidate image 60-2 is displayed instead of. The display of the reference image 60-1 on the left side remains unchanged.

According to the second modification, the user can confirm the candidate image together with the reference image. Further, the user can display and confirm another candidate image that is not displayed on the liquid crystal monitor 36 by tapping the upper icon 75 or the lower icon 76.

In the second modification, when the user double-taps any position on the screen with the finger 80 while an arbitrary candidate image is displayed, the control unit 35 (imaging condition setting unit 35d) moves at that time. The imaging conditions applied to the candidate images 60-n (n is a natural number of 1 to 8) displayed on the right side of the display surface are applied to the first region 61 when an imaging instruction is next given. It is determined as the imaging condition to be performed, and the process returns to the state where the setting screen illustrated in FIG. 7 is displayed on the liquid crystal monitor 36.

It should be noted that the user can perform an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation and then an operation of selecting an arbitrary candidate image among a plurality of candidate images related to saturation. The point that the user can select an arbitrary candidate image from a plurality of candidate images related to brightness after performing an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation is described above. It is the same as the said embodiment.

Further, the user can set (change) the imaging conditions regarding the brightness or the saturation even in the area other than the first area 61, that is, the area in which the order given by the setting unit 35c is the second or lower. , The same as the above-described embodiment.

(Modification 3)
In the above-described embodiment, one of "program auto" mode, "shutter speed priority auto" mode, "aperture priority auto" mode, and "manual" mode is selected and selected as an example of determining the reference condition. Appropriate exposure is determined by performing a predetermined exposure calculation according to the exposure mode, and image processing conditions prepared in advance so as to obtain standard image quality are determined, and these determined exposure conditions and image processing conditions are determined. An example of using the reference condition has been described. Instead of this, the reference conditions may be determined as follows.

<Example of determining reference conditions for brightness>
In the third modification, the method of determining the appropriate exposure is changed depending on the imaging scene mode setting of the camera 1 and the subject element detected by the object detection unit 35a, and this is set as a reference condition (exposure condition). For example, when the imaging scene mode setting of the camera 1 is set to the portrait mode and a person is detected as a subject element by the object detection unit 35a, the control unit 35 determines the area including the person. The exposure condition is determined so that the brightness of the person is appropriate, and the reference condition is used.

For example, when the imaging scene mode setting of the camera 1 is set to the night view portrait mode and a person is detected as a subject element by the object detection unit 35a, the control unit 35 determines the area including the person. In addition to brightening the brightness of the person, the exposure conditions are determined so as to be appropriate for the area including the night view, which is the background, in consideration of the brightness of the night view, and used as the reference condition. Further, when the light emitting device 40 emits light, the appropriate exposure is determined so as to balance the brightness of both the person and the background, and the required light emitting amount of the light emitting device 40 is calculated.

For example, when the imaging scene mode setting of the camera 1 is set to the landscape mode and the shooting distance is farther than a predetermined distance (almost infinity), the control unit 35 controls the color component (green pixel) of the subject element. Based on the ratio of Gb, Gr, blue pixel B, red pixel R) and the brightness of the subject element, the exposure condition is set so that the brightness of the mountain is appropriate for the region including the subject element (for example, the mountain). Determine and use as a reference condition.

<Example of determining standard conditions for saturation>
In the third modification, an image processing condition prepared separately from the standard image quality is set as a reference condition (image processing condition) according to the imaging scene mode setting of the camera 1 and the subject element detected by the object detection unit 35a.

For example, the control unit 35 protrudes from the signals read from the green pixels Gb, Gr, the blue pixel B, and the red pixel R in the area divided by the area dividing unit 35b as compared with other color components. When it has a color component, a condition for obtaining a plain image (flat) without image creation is determined for that region, and the condition determined in this way is used as a reference condition.

For example, when the imaging scene mode setting of the camera 1 is set to the close-up mode and the object detection unit 35a detects a red flower as a subject element, the setting unit 35c is set to the area including the red flower. On the other hand, the condition for lowering the exposure and lowering the saturation of the red component is determined, and the condition determined in this way is used as the reference condition.

(Modification example 4)
In the above description, the camera 1 has been described as an example, but it may be configured by a high-performance mobile phone 250 having a camera function such as a smartphone or a mobile device such as a tablet terminal.

(Modification 5)
In the above-described embodiment, the camera 1 in which the imaging unit 32 and the control unit 35 are configured as a single electronic device has been described as an example. Instead, for example, an imaging system 1B may be configured in which the imaging unit 32 and the control unit 35 are separately provided and the control unit 35 controls the imaging unit 32 via communication.
Hereinafter, an example in which the imaging device 1001 provided with the imaging unit 32 is controlled by the control device 1002 provided with the control unit 35 will be described.

FIG. 19 is a block diagram illustrating the configuration of the imaging system 1B according to the modified example 5. In FIG. 19, the image pickup system 1B is composed of an image pickup device 1001 and a display device 1002. The image pickup apparatus 1001 includes a first communication section 1003 in addition to the image pickup optical system 31 and the image pickup section 32 described in the above embodiment. Further, the display device 1002 is added to the image processing unit 33, the work memory 34, the control unit 35, the liquid crystal monitor 36, the operation member 37, the non-volatile memory 38, the recording unit 39, and the light emitting device 40 described in the above embodiment. A second communication unit 1004 is provided.

The first communication unit 1003 and the second communication unit 1004 can perform bidirectional data communication by, for example, a well-known wireless communication technology or optical communication technology.
The image pickup device 1001 and the display device 1002 may be connected by wire with a wired cable, and the first communication unit 1003 and the second communication unit 1004 may perform bidirectional data communication.

In the image pickup system 1B, the control unit 35 controls the image pickup unit 32 by performing data communication via the second communication unit 1004 and the first communication unit 1003. For example, by transmitting and receiving predetermined control data between the image pickup device 1001 and the display device 1002, the display device 1002 divides the screen into a plurality of regions or is divided based on the image as described above. The regions are ordered, different imaging conditions are set for each divided region, and the imaging signals captured in each region are read out.

According to the fifth modification, the live view image acquired on the image pickup device 1001 side and transmitted to the display device 1002 is displayed on the liquid crystal monitor 36 of the display device 1002, so that the user can move to a position away from the image pickup device 1001. Remote operation can be performed from a certain display device 1002.
The display device 1002 can be configured by, for example, a high-performance mobile phone 250 such as a smartphone. Further, the image pickup device 1001 can be configured by an electronic device including the above-mentioned stacked image pickup device.
The control unit 35 of the display device 1002 includes an object detection unit 35a, an area division unit 35b, a setting unit 35c, an imaging condition setting unit 35d, an imaging control unit 35e, an AF calculation unit 35f, and a display control unit 35g. Although the example of providing the above is described, the object detection unit 35a, the area division unit 35b, the setting unit 35c, the imaging condition setting unit 35d, the imaging control unit 35e, the AF calculation unit 35f, and a part of the display control unit 35g are imaged. It may be provided in the device 1001.

(Modification 6)
The supply of the program to the mobile device such as the camera 1, the high-performance mobile phone 250, or the tablet terminal described above is performed by infrared communication or short-range wireless communication from the personal computer 205 storing the program, for example, as illustrated in FIG. It can be sent to mobile devices.

The program may be supplied to the personal computer 205 by setting a recording medium 204 such as a CD-ROM in which the program is stored in the personal computer 205, or by using a communication line 201 such as a network to the personal computer 205. It may be loaded. When passing through the communication line 201, the program is stored in the storage device 203 or the like of the server 202 connected to the communication line.

Further, the program can be directly transmitted to the mobile device via the access point (not shown) of the wireless LAN connected to the communication line 201. Further, a recording medium 204B such as a memory card in which the program is stored may be set in the mobile device. As described above, the program can be supplied as various forms of computer program products such as those provided via a recording medium or a communication line.

Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other aspects conceivable within the scope of the technical idea of the present invention are also included within the scope of the present invention.

1 ... Camera 1B ... Imaging system 32 ... Imaging unit 35 ... Control unit 35a ... Object detection unit 35b ... Area division unit 35c ... Setting unit 35d ... Imaging condition setting unit 35e ... Imaging control unit 35g ... Display control unit 36 ... LCD monitor 40 ... Light emitting device 100 ... Image sensor 1001 ... Image device 1002 ... Display device

Claims (10)

A display unit that displays a plurality of images of the subject captured in the first image pickup region of the image sensor, and a display unit.
Using an image selected from the plurality of images displayed on the display unit, the imaging condition of the first imaging region is different from the imaging condition of the second imaging region of the image sensor. A control unit that controls the image sensor and
Electronic equipment equipped with. In the electronic device according to claim 1,
The control unit is an electronic device that determines imaging conditions in the first imaging region based on a subject imaged in the first imaging region and an imaging mode set in the image sensor. In the electronic device according to claim 1 or 2.
An electronic device including an image processing unit that generates a candidate image in which the imaging conditions of the first imaging region changed, which are determined by the control unit. In the electronic device according to claim 3,
The image processing unit is an electronic device that generates a plurality of candidate images in which the imaging conditions of the first imaging region are changed stepwise. In the electronic device according to claim 3 or 4.
The imaging condition is at least one of brightness and saturation.
The image processing unit is an electronic device that generates a plurality of candidate images whose brightness or saturation is changed stepwise. In the electronic device according to any one of claims 3 to 5.
The image processing unit is an electronic device that generates an image before changing the imaging conditions of the first imaging region as a reference image and also generates the candidate image after changing the imaging conditions of the first imaging region. In the electronic device according to claim 6,
The image processing unit is an electronic device that generates an image having a brightness different from that of the reference image. In the electronic device according to claim 6 or 7.
The image processing unit is an electronic device that generates an image having different saturation from the reference image and the reference image. In the electronic device according to any one of claims 1 to 8.
The first imaging region is an electronic device in which a photoelectric conversion unit that converts light into electric charges is arranged. In the electronic device according to claim 9.
The first imaging region is an electronic device in which a plurality of the photoelectric conversion units are arranged in a first direction and a second direction intersecting the first direction.

Images