JP2020057878A - Electronic equipment and setting program - Google Patents

Abstract

To set an appropriate imaging condition to an imaging region where different subject images coexist.SOLUTION: Electronic equipment comprises: an imaging element that can set a different imaging condition for each of a plurality of imaging regions for imaging a subject; and a setting unit that sets a first imaging condition to a first imaging region where a first subject image exists among the plurality of imaging regions, sets a second imaging condition different from the first imaging condition to a second imaging region where a second subject image exists among the plurality of imaging regions, and sets a third imaging condition for a third imaging region where the first subject image and the second subject image exist among the plurality of imaging regions on the basis of at least one of the first imaging condition and the second imaging condition.SELECTED DRAWING: Figure 10

Description

  The present invention relates to an electronic device and a setting program.

  2. Description of the Related Art There is known an imaging apparatus equipped with an imaging element capable of setting different imaging conditions for each area of a screen (see Patent Document 1). However, when processing image data generated in regions having different imaging conditions, the imaging conditions are not considered.

JP 2006-197192 A

  An electronic device disclosed in the present application includes an imaging element that can set different imaging conditions for each of a plurality of imaging regions for imaging a subject, and a first imaging region in which a first subject image exists among the plurality of imaging regions. A first imaging condition is set, and a second imaging condition different from the first imaging condition is set in a second imaging region where a second subject image is present among the plurality of imaging regions. A setting unit that sets a third imaging condition of a third imaging region in which the first subject image and the second subject image exist based on at least one of the first imaging condition and the second imaging condition. .

  An electronic device disclosed in the present application is configured to set an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject, and to set a first imaging condition in a first imaging region of the plurality of imaging regions. Setting a second imaging condition in a second imaging area of the plurality of imaging areas, and setting a second imaging condition in the third imaging area sandwiched between the first imaging area and the second imaging area in the plurality of imaging areas; A setting unit for setting an imaging condition between the first imaging condition and the second imaging condition.

  An electronic device disclosed in the present application includes an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject, and a first imaging device in the imaging region where a first subject image exists among the plurality of imaging regions. An imaging condition is set, and in the imaging region where a first subject image and a second subject image different from the first subject image are present in the plurality of imaging regions, the first subject image is different from the first imaging condition, and A setting unit for setting a second imaging condition under which an image and an image signal of the second subject image are not saturated.

  An electronic device disclosed in the present application is configured to set an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject, and to set a first imaging condition in a first imaging region of the plurality of imaging regions. A second imaging condition different from the first imaging condition is set in a second imaging region of the plurality of imaging regions, and the second imaging condition is sandwiched between the first imaging region and the second imaging region in the plurality of imaging regions. Setting for setting a third imaging condition in which a pixel signal having a magnitude between a pixel signal output under the first imaging condition and a pixel signal output under the second imaging condition is output to a third imaging region. And a part.

  An electronic device disclosed in the present application includes an imaging element that can set different imaging conditions for each of a plurality of imaging regions for imaging a subject, and a first imaging region in which a first subject image exists among the plurality of imaging regions. A first imaging condition is set, a second imaging condition different from the first imaging condition is set in a second imaging region where a second subject image is present among the plurality of imaging regions, and the second imaging condition is set in the plurality of imaging regions. In the third imaging region sandwiched between the first imaging region and the second imaging region, the magnitude of the magnitude between the pixel signal output under the first imaging condition and the pixel signal output under the second imaging condition is set. A setting unit for setting a third imaging condition for outputting a pixel signal.

  The setting program according to one aspect of the present invention disclosed in the present application is a setting program that causes a processor to execute setting control of an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject, Causing the processor to set a first imaging condition in a first imaging area where a first subject image is present among the plurality of imaging areas, and in a second imaging area where a second subject image is present among the plurality of imaging areas, A second imaging condition different from the first imaging condition is set, and a third imaging condition of a third imaging region where the first subject image and the second subject image are present among the plurality of imaging regions is set to the first imaging condition. The setting is made based on at least one of a condition and the second imaging condition.

FIG. 1 is a cross-sectional view of the stacked image sensor. FIG. 2 is a diagram illustrating a pixel array of the imaging chip. FIG. 3 is a circuit diagram of the imaging chip. FIG. 4 is a block diagram illustrating a configuration example of an image sensor. FIG. 5 is an explanatory diagram illustrating an example of a block configuration of an electronic device. FIG. 6 is an explanatory diagram illustrating a configuration example of an image file generated by imaging of an electronic device. FIG. 7 is an explanatory diagram illustrating a relationship between an imaging surface and a subject image. FIG. 8 is an explanatory diagram showing a specific configuration example of the image file. FIG. 9 is an explanatory diagram illustrating a detailed block configuration example of an electronic device. FIG. 10 is an explanatory diagram illustrating a first setting example of the third imaging condition. FIG. 11 is an explanatory diagram showing a setting example 2 of the third imaging condition. FIG. 12 is an explanatory diagram illustrating a third setting example of the third imaging condition. FIG. 13 is an explanatory diagram showing the degree of influence on the coexistence area. FIG. 14 is an explanatory diagram illustrating an example of a setting unit of an undefined block. FIG. 15 is an explanatory diagram showing the boundary length of the coexistence area. FIG. 16 is an explanatory diagram showing reduction of the influence of halo. FIG. 17 is a graph showing an example of halo occurrence prediction using the γ function. FIG. 18 is a flowchart illustrating an example 1 of an imaging condition setting process procedure in the electronic device. FIG. 19 is a flowchart illustrating a detailed processing procedure example 1 of the area priority setting processing (step S1808) illustrated in FIG. FIG. 20 is a flowchart illustrating a detailed processing procedure example 2 of the area priority setting processing (step S1808) illustrated in FIG. FIG. 21 is a flowchart illustrating a detailed processing procedure example 2-1 of the area priority setting processing (step S1808) illustrated in FIG. FIG. 22 is a flowchart illustrating a detailed processing procedure example 2-2 of the area priority setting processing (step S1808) illustrated in FIG. FIG. 23 is a flowchart illustrating a detailed processing procedure example 1 of the halo countermeasure processing (step S2202) illustrated in FIG. FIG. 24 is a flowchart illustrating a detailed processing procedure example 2-3 of the area priority setting processing (step S1808) illustrated in FIG. FIG. 25 is a flowchart illustrating a detailed processing procedure example 1 of the complexity priority setting processing (step S1809) illustrated in FIG. FIG. 26 is a flowchart illustrating a detailed processing procedure example 2 of the complexity priority setting processing (step S1809) illustrated in FIG. FIG. 27 is a flowchart illustrating a second example of the setting procedure of the imaging condition in the electronic device. FIG. 28 is a flowchart illustrating a detailed processing procedure example 2 of the halo countermeasure processing (step S2202) illustrated in FIG. FIG. 29 is a flowchart illustrating a detailed processing example of the halo countermeasure process.

<Configuration example of imaging device>
The imaging device mounted on the electronic device described in the embodiments of the present specification is a stacked imaging device. In the stacked imaging device, different imaging conditions can be set for each of a plurality of different imaging regions. First, a description will be given of a structure of a stacked type imaging device in which different imaging conditions can be set for each of a plurality of different imaging regions. This stacked-type image sensor is described in Japanese Patent Application No. 2012-139026 filed earlier by the present applicant. The electronic device is, for example, an imaging device such as a digital camera or a digital video camera.

  FIG. 1 is a cross-sectional view of the stacked image sensor 100. The stacked image sensor (hereinafter simply referred to as “image sensor”) 100 processes a pixel signal, and a back-illuminated image sensor (hereinafter simply referred to as “image sensor”) 113 that outputs a pixel signal corresponding to incident light. A signal processing chip 111 and a memory chip 112 for storing pixel signals are provided. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by a conductive bump 109 such as Cu.

  Note that, as shown in FIG. 1, the incident light mainly enters in the positive Z-axis direction indicated by the white arrow. In the present embodiment, the surface of the imaging chip 113 on which incident light is incident is referred to as a back surface. Further, as shown by the coordinate axis 120, the left direction of the paper orthogonal to the Z axis is defined as the plus direction of the X axis, and the near side of the paper orthogonal to the Z axis and the X axis is defined as the plus direction of the Y axis. In some of the subsequent drawings, the coordinate axes 120 are displayed on the basis of the coordinate axes 120 in FIG. 1 so that the directions of the respective drawings can be understood.

  One example of the imaging chip 113 is a back-illuminated MOS (Metal Oxide Semiconductor) image sensor. The PD (photodiode) layer 106 is disposed on the back side of the wiring layer 108. The PD layer 106 includes a plurality of PDs 104 that are arranged two-dimensionally and accumulate charges according to incident light, and a transistor 105 provided corresponding to the PDs 104.

  The color filter 102 is provided on the incident side of the incident light in the PD layer 106 via the passivation film 103. The color filters 102 have a plurality of types that transmit different wavelength regions, and have a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filters 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  On the incident side of the incident light in the color filter 102, a micro lens 101 is provided corresponding to each pixel. The micro lens 101 collects incident light toward the corresponding PD 104.

  The wiring layer 108 has a wiring 107 for transmitting the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be a multilayer, and may be provided with a passive element and an active element.

  A plurality of bumps 109 are arranged on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surface of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are aligned by applying pressure or the like. The bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are arranged on surfaces of the signal processing chip 111 and the memory chip 112 that face each other. These bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized or the like, so that the aligned bumps 109 are joined and electrically connected.

  Note that the bonding between the bumps 109 is not limited to Cu bump bonding by solid-phase diffusion, and micro-bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one block described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. In a peripheral region other than the pixel region where the pixels are arranged, a bump larger than the bump 109 corresponding to the pixel region may be additionally provided.

  The signal processing chip 111 has a TSV (through-silicon via) 110 that connects circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in a peripheral area. Further, the TSV 110 may be provided in a peripheral area of the imaging chip 113 and the memory chip 112.

  FIG. 2 is a diagram illustrating a pixel array of the imaging chip 113. In particular, this shows a state in which the imaging chip 113 is observed from the back side. (A) is a plan view schematically showing an imaging surface 200 which is a back surface of the imaging chip 113, and (b) is an enlarged plan view of a partial area 200a of the imaging surface 200. As shown in (b), a large number of pixels 201 are two-dimensionally arranged on the imaging surface 200.

  Each of the pixels 201 has a color filter (not shown). The color filters are composed of three types of red (R), green (G), and blue (B), and the notations “R”, “G”, and “B” in (b) are color filters of the pixel 201. Represents the type. As shown in (b), pixels 201 having such color filters are arranged on an imaging surface 200 of the imaging device 100 according to a so-called Bayer arrangement.

  The pixel 201 having the red filter photoelectrically converts light in the red wavelength band of the incident light and outputs a light receiving signal (photoelectric conversion signal). Similarly, the pixel 201 having the green filter photoelectrically converts light in the green wavelength band of the incident light and outputs a light receiving signal. The pixel 201 having the blue filter photoelectrically converts light in the blue wavelength band of the incident light to output a light receiving signal.

  The image sensor 100 is configured to be individually controllable for each block 202 including a total of four pixels 201 of 2 × 2 pixels adjacent to each other. For example, when charge accumulation is started at the same time for two different blocks 202, one block 202 reads out the charge, that is, reads the light-receiving signal 1/30 seconds after the start of charge accumulation, and the other block 202 The charge can be read out 1/15 second after the start of charge accumulation. In other words, the imaging element 100 can set a different exposure time (charge accumulation time, so-called shutter speed) for each block 202 in one image pickup.

  The image sensor 100 can make the amplification factor of the image signal (so-called ISO sensitivity) different for each block 202 in addition to the above-described exposure time. The imaging device 100 can change the timing of starting the charge accumulation and the timing of reading the light receiving signal for each block 202. That is, the image sensor 100 can change the frame rate at the time of capturing a moving image for each block 202.

  In summary, the imaging device 100 is configured to be able to vary imaging conditions such as an exposure time, an amplification factor, and a frame rate for each block 202. For example, if a reading line (not shown) for reading an image signal from a photoelectric conversion unit (not shown) included in the pixel 201 is provided for each block 202 and the image signal can be read independently for each block 202, The exposure time (shutter speed) can be made different for each block 202.

  In addition, an amplifier circuit (not shown) for amplifying an imaging signal generated by the photoelectrically converted charge is provided independently for each block 202, and the amplification factor of the amplifier circuit can be controlled independently for each amplifier circuit. , The signal amplification rate (ISO sensitivity) can be made different for each block 202.

  The imaging conditions that can be made different for each block 202 include, in addition to the above-described imaging conditions, a frame rate, a gain, a resolution (thinning rate), the number of rows or columns to be added with pixel signals, and the accumulation of charges. The time, the number of accumulations, the number of bits for digitization, and the like. Further, the control parameter may be a parameter in image processing after acquiring an image signal from a pixel.

  In addition, for the imaging conditions, for example, a liquid crystal panel having sections that can be controlled independently for each block 202 (one section corresponds to one block 202) is provided in the image sensor 100 and used as a dark filter that can be turned on and off. Then, the brightness (aperture value) can be controlled for each block 202.

  Note that the number of the pixels 201 configuring the block 202 does not have to be the above-described 2 × 2 four pixels. The block 202 only needs to have at least two or more pixels 201, and conversely, may have more than four pixels 201.

  FIG. 3 is a circuit diagram of the imaging chip 113. In FIG. 3, a rectangle typically surrounded by a dotted line represents a circuit corresponding to one pixel 201. A rectangle surrounded by a dashed line corresponds to one block 202 (202-1 to 202-4). Note that at least part of each transistor described below corresponds to the transistor 105 in FIG.

  As described above, the reset transistor 303 of the pixel 201 is turned on / off for each block 202. Further, the transfer transistor 302 of the pixel 201 is also turned on / off for each block 202. In the example shown in FIG. 3, a reset wiring 300-1 for turning on / off four reset transistors 303 corresponding to the upper left block 202-1 is provided, and four transfer transistors corresponding to the same block 202-1 are provided. A TX wiring 307-1 for supplying a transfer pulse to 302 is also provided.

  Similarly, a reset wiring 300-3 for turning on / off the four reset transistors 303 corresponding to the lower left block 202-3 is provided separately from the reset wiring 300-1. Further, a TX wiring 307-3 for supplying a transfer pulse to the four transfer transistors 302 corresponding to the same block 202-3 is provided separately from the TX wiring 307-1.

  Similarly, the reset wiring 300-2 and the TX wiring 307-2, and the reset wiring 300-4 and the TX wiring 307-4 are provided in the upper right block 202-2 and the lower right block 202-4, respectively. Have been.

  The 16 PDs 104 corresponding to each pixel 201 are connected to the corresponding transfer transistor 302. A transfer pulse is supplied to the gate of each transfer transistor 302 via the TX wiring of each block 202. The drain of each transfer transistor 302 is connected to the source of the corresponding reset transistor 303, and the so-called floating diffusion FD between the drain of the transfer transistor 302 and the source of the reset transistor 303 is connected to the gate of the corresponding amplifying transistor 304. You.

  The drain of each reset transistor 303 is commonly connected to a Vdd wiring 310 to which a power supply voltage is supplied. A reset pulse is supplied to the gate of each reset transistor 303 via the reset wiring of each block 202.

  The drain of each amplification transistor 304 is commonly connected to a Vdd wiring 310 to which a power supply voltage is supplied. The source of each amplification transistor 304 is connected to the drain of the corresponding selection transistor 305. The gate of each selection transistor 305 is connected to a decoder wiring 308 to which a selection pulse is supplied. The decoder wiring 308 is provided independently for each of the 16 selection transistors 305.

  The source of each selection transistor 305 is connected to a common output wiring 309. The load current source 311 supplies a current to the output wiring 309. That is, the output wiring 309 for the selection transistor 305 is formed by a source follower. Note that the load current source 311 may be provided on the imaging chip 113 side or on the signal processing chip 111 side.

  Here, a flow from the start of charge accumulation to pixel output after the end of accumulation will be described. When a reset pulse is applied to the reset transistor 303 through the reset wiring of each of the blocks 202 and at the same time, a transfer pulse is applied to the transfer transistor 302 through the TX wiring of each of the blocks 202 (202-1 to 202-4), At each 202, the potentials of the PD 104 and the floating diffusion FD are reset.

  When the application of the transfer pulse is released, each PD 104 converts the received incident light into a charge and stores the charge. Thereafter, when the transfer pulse is applied again in a state where the reset pulse is not applied, the accumulated charges are transferred to the floating diffusion FD, and the potential of the floating diffusion FD changes from the reset potential to the signal potential after the charge accumulation. .

  Then, when a selection pulse is applied to the selection transistor 305 through the decoder wiring 308, a change in the signal potential of the floating diffusion FD is transmitted to the output wiring 309 via the amplification transistor 304 and the selection transistor 305. Accordingly, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

  As described above, the reset wiring and the TX wiring are common to the four pixels forming the block 202. That is, the reset pulse and the transfer pulse are simultaneously applied to four pixels in the same block 202, respectively. Therefore, all the pixels 201 forming a certain block 202 start charge accumulation at the same timing and end charge accumulation at the same timing. However, a pixel signal corresponding to the stored charge is selectively output from the output wiring 309 by sequentially applying a selection pulse to each selection transistor 305.

  Thus, the charge accumulation start timing can be controlled for each block 202. In other words, different blocks 202 can be imaged at different timings.

  FIG. 4 is a block diagram illustrating a configuration example of the image sensor 100. The analog multiplexer 411 sequentially selects the 16 PDs 104 forming the block 202 and outputs each pixel signal to the output wiring 309 provided corresponding to the block 202. The multiplexer 411 is formed on the imaging chip 113 together with the PD 104.

  The pixel signals output via the multiplexer 411 are converted into CDS and A / D signals by a signal processing circuit 412 formed on the signal processing chip 111 for performing correlated double sampling (CDS) and analog / digital (A / D) conversion. D conversion is performed. The A / D-converted pixel signal is delivered to a demultiplexer 413 and stored in a pixel memory 414 corresponding to each pixel. The demultiplexer 413 and the pixel memory 414 are formed on the memory chip 112.

  The arithmetic circuit 415 processes the pixel signal stored in the pixel memory 414 and delivers it to the subsequent image processing unit. The arithmetic circuit 415 may be provided in the signal processing chip 111 or may be provided in the memory chip 112. Note that FIG. 4 shows connections for the four blocks 202, but these actually exist for each of the four blocks 202 and operate in parallel.

  However, the arithmetic circuit 415 may not be provided for each of the four blocks 202. For example, one arithmetic circuit 415 sequentially performs processing while sequentially referring to the value of the pixel memory 414 corresponding to each of the four blocks 202. You may.

  As described above, the output wiring 309 is provided for each of the blocks 202. Since the image pickup device 100 has the image pickup chip 113, the signal processing chip 111, and the memory chip 112 stacked, by using the electrical connection between the chips using the bump 109 for the output wiring 309, each chip can be placed in the plane direction. The wiring can be routed without increasing the size.

<Example of block configuration of electronic device>
FIG. 5 is an explanatory diagram illustrating an example of a block configuration of an electronic device. The details of the block configuration example will be described later (FIG. 9). The electronic device 500 is, for example, a lens-integrated camera. The electronic device 500 includes an imaging optical system 501, the imaging device 100, the control unit 502, a liquid crystal monitor 503, a memory card 504, an operation unit 505, a DRAM 506, a flash memory 507, and a recording unit 508. . The control unit 502 includes a detection unit that detects camera shake and subject shake as described later.

  The imaging optical system 501 includes a plurality of lenses, and forms a subject image on the imaging surface 200 of the imaging device 100. In FIG. 5, the imaging optical system 501 is illustrated as a single lens for convenience.

  The image sensor 100 is, for example, an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device). The image sensor 100 images a subject image formed by the imaging optical system 501 and outputs an image signal. The control unit 502 is an electronic circuit that controls each unit of the electronic device 500, and includes a processor that executes a program, peripheral circuits such as an image processing circuit, and various sensors such as an acceleration sensor.

  A predetermined control program is previously written in the flash memory 507 which is a nonvolatile storage medium. The processor of the control unit 502 controls each unit by reading and executing the control program from the flash memory 507. This control program uses the DRAM 506, which is a volatile storage medium, as a work area.

  The liquid crystal monitor 503 is a display device using a liquid crystal panel. The control unit 502 causes the image sensor 100 to repeatedly capture a subject image at predetermined intervals (for example, 1/60 second). Then, various image processing is performed on the image pickup signal output from the image sensor 100 to create a so-called through image, and the image is displayed on the liquid crystal monitor 503. On the liquid crystal monitor 503, for example, a setting screen for setting an imaging condition is displayed in addition to the through image.

  The control unit 502 creates an image file, which will be described later, based on the imaging signal output from the imaging element 100, and records the image file on a memory card 504, which is a portable recording medium. The operation unit 505 has various operation members such as push buttons, and outputs an operation signal to the control unit 502 in response to operation of the operation members.

  The recording unit 508 is configured by, for example, a microphone, converts the environmental sound into an audio signal, and inputs the audio signal to the control unit 502. The control unit 502 does not record the moving image file on the memory card 504 which is a portable recording medium, but uses a recording medium (not shown) such as an SSD (Solid State Drive) or a hard disk built in the electronic device 500. May be recorded.

<Example of image file configuration>
FIG. 6 is an explanatory diagram illustrating a configuration example of an image file generated by imaging of the electronic device 500. The image file 600 is generated by an image processing circuit or an image processing program in the control unit 502 and stored in the memory card 504, the DRAM 506, or the flash memory 507. The image file 600 includes two blocks, a header section 601 and a data section 602. The header section 601 is a block located at the head of the image file 600. The header section 601 stores a file basic information area 611, a mask area 612, and an imaging information area 613 in the order described above.

  In the file basic information area 611, for example, the size and offset of each part (the header part 601, the data part 602, the mask area 612, the imaging information area 613, and the like) in the image file 600 are recorded. In the mask area 612, imaging condition information, mask information, and the like, which will be described later, are recorded. In the imaging information area 613, information on imaging such as the model name of the electronic device 500 and information on the imaging optical system 501 (for example, information on optical characteristics such as aberration) is recorded. The data section 602 is a block located after the header section 601 and records image information.

<Relationship between imaging surface 200 and subject image>
FIG. 7 is an explanatory diagram illustrating the relationship between the imaging surface 200 and the subject image. (A) shows an example of imaging a subject image on an imaging surface 200 (imaging range) of the imaging device 100. (B) shows mask information set at the time of imaging in (a). (C) shows an example of imaging a subject image on the imaging surface 200 (imaging range) of the imaging device 100 after a lapse of a predetermined time from (a). (D) shows the mask information set at the time of (c) imaging.

  In (a), the control unit 502 captures an image of a subject. The imaging in (a) may be, for example, imaging performed for creating a live view image (a so-called through image). The control unit 502 performs a predetermined image analysis process on the subject image obtained by the imaging in (a). The image analysis processing detects a main subject by, for example, a known subject detection technique (a technique of calculating a feature amount and detecting a range where a predetermined subject exists). In the first embodiment, a background other than the main subject is set as the background.

  A block 202 in which only the detected main subject image (main subject image MI) is shown is a main subject block 202. A set of main subject blocks 202 becomes a main subject area 702. A block 202 in which only the background image is shown is a background block 202. A set of background blocks 202 is a background area 701. A block 202 in which both the main subject image MI and the background image are shown is referred to as an undefined block 202. A set of undefined blocks 202 is referred to as a coexistence area 703.

  In (a), specifically, for example, the main subject area 702 is set as a shape along the outer shape of the main subject image MI. The control unit 502 sets different imaging conditions for each block 202 in the main subject area 702 and each block 202 in the background area 701. For example, a faster shutter speed is set in each of the former blocks 202 than in each of the latter blocks 202. In this way, in the imaging of (c), which is performed after the imaging of (a), image blurring is less likely to occur in the main subject area 702.

  When the main subject area 702 is in a backlight state due to the influence of a light source such as the sun existing in the background area 701, the control unit 502 gives the former block 202 a relatively high ISO sensitivity. Or set a slow shutter speed. Further, the control unit 502 sets a relatively low ISO sensitivity or a high shutter speed in each of the latter blocks 202. In this way, it is possible to prevent the main subject area 702 in the backlight state from being blackened and the background area 701 having a large amount of light from being overexposed in the imaging of FIG. The setting for the block 202 configuring the coexistence area 703 will be described later.

  Note that the image analysis process may be a process different from the process of detecting the background region 701 and the main subject region 702 described above. For example, a process of detecting a portion where the brightness is equal to or more than a certain value (a portion that is too bright) or a portion where the brightness is less than a certain value (a portion that is too dark) in the entire imaging surface 200 may be used. When the image analysis processing is such processing, the control unit 502 sets the shutter so that the exposure value (Ev value) of the block 202 included in the former area is lower than that of the block 202 included in the other area. Set the speed and ISO sensitivity.

  Further, the control unit 502 sets the shutter speed and the ISO sensitivity so that the exposure value (Ev value) of the block 202 included in the latter area is higher than that of the block 202 included in the other area. By doing so, the dynamic range of the image obtained by the imaging of (c) can be wider than the original dynamic range of the image sensor 100.

  FIG. 7B illustrates an example of the mask information 704 corresponding to the imaging surface 200 illustrated in FIG. “1” is stored at the position of the block 202 belonging to the background area 701, “2” is stored at the position of the block 202 belonging to the main subject area 702, and “0” is stored at the position of the block 202 belonging to the coexistence area 703. Have been.

  However, the position of the block 202 belonging to the coexistence area 703 is “2” when the control unit 502 determines that the block 202 belongs to the main subject area 702, and “1” when it determines that it belongs to the background area 701. May be updated. The control unit 502 may store the value “0” before the update and the value “1” or “2” after the update.

  The control unit 502 performs an image analysis process on the image data of the first frame, and detects a background area 701, a main subject area 702, and a coexistence area 703. As a result, the frame obtained by the imaging in (a) is divided into a background area 701, a main subject area 702, and a coexistence area 703, as shown in (b). The control unit 502 sets different imaging conditions for each of the blocks 202 in the background area 701 and each of the blocks 202 in the main subject area 702, performs imaging in (c), and creates image data. An example of the mask information 704 at this time is shown in FIG.

  In the mask information 704 of (b) corresponding to the imaging result of (a) and the mask information 704 of (d) corresponding to the imaging result in (c), imaging is performed at different times (the time difference is different). Therefore, for example, when the subject is moving or when the user moves the electronic device 500, these two pieces of mask information 704 have different contents. In other words, the mask information 704 is dynamic information that changes over time. Therefore, in a certain block 202, different imaging conditions are set for each frame.

<Specific example of image file>
FIG. 8 is an explanatory diagram illustrating a specific configuration example of the image file 600. In the mask area 612, identification information 801, imaging condition information 802, and mask information 704 are recorded in the order described above.

  The identification information 801 indicates that the image file 600 has been created by the multi-imaging condition still image imaging function. The multi-image capturing condition still image capturing function is a function of capturing a still image with the image sensor 100 in which a plurality of different image capturing conditions (for example, a long exposure time and a short exposure time) are set. For example, in backlight, the main subject is set to long seconds, and the background is set to short seconds. Long seconds and short seconds are set values that are relatively determined.

  The imaging condition information 802 is information indicating what purpose (purpose, role) the block 202 has. For example, as described above, when dividing the imaging plane 200 (FIG. 7A) into the background area 701, the main subject area 702, and the coexistence area 703, each block 202 keeps the coexistence area 703 undefined. If there is, it belongs to either the background area 701, the main subject area 702, or the coexistence area 703. On the other hand, if the coexistence area 703 is determined to be one of the background area 701 and the main subject area 702, each block 202 belongs to the background area 701 or the main subject area 702. It is.

  In other words, when creating the image file 600, the imaging condition information 802 includes, for example, “take a still image of the main subject area at 1/60 [sec] (long seconds)” and “set the background area at 1/1000 [ [sec]], and information indicating a unique number assigned to each of these imaging conditions. For example, the number “1” indicates an imaging condition “to capture a still image of the background area in 1/1000 [sec]”. The number “2” indicates an imaging condition “to capture a still image of the main subject area in 1/60 [sec]”. The number “0” indicates the imaging condition “the exposure time for capturing a still image of the coexistence area is indefinite”.

  Here, for the sake of simplicity, “1” and “2” are assigned as unique numbers assigned to each imaging condition, but three or more types of exposure times may be assigned. Therefore, for example, numbers corresponding to the number of stages indicating the exposure time may be assigned.

  The mask information 704 is information indicating the use (purpose, role) of each block 202. As described with reference to FIG. 7, the mask information 704 is “information that represents the number assigned to the imaging condition information 802 in the form of a two-dimensional map according to the position of the block 202”. That is, when the blocks 202 arranged in a two-dimensional manner are specified by the two-dimensional coordinates (x, y) using two integers x and y, the use of the block 202 existing at the position of (x, y) is determined by the mask information. This is represented by an imaging number existing at the position (x, y) 704.

  The imaging number “1” is a number assigned to the block 202 where the background image exists, the imaging number “2” is a number assigned to the block 202 where the main subject image exists, and the imaging number “0” is This is a number assigned to the block 202 where the background image and the main subject image exist. For example, when the imaging number “2” is included in the position of the coordinate (3, 5) in the mask information 704, the block 202 located at the coordinate (3, 5) includes “the main subject area is 1/60 [ [sec]] is given. In other words, it can be seen that the block 202 located at the coordinates (3, 5) belongs to the main subject area 702.

  The data section 602 includes image information 811, a Tv value map 812, an Sv value map 813, a Bv value map 814, and Av value information 815.

  The image information 811 is information in which an image pickup signal output from the image pickup device 100 by the image pickup in FIG. 7B is recorded in a form before various image processing is performed, and is so-called RAW image data.

  The Tv value map 812 is information expressing a Tv value representing a shutter speed set for each block 202 in the form of a two-dimensional map according to the position of the block 202. For example, the shutter speed set in the block 202 located at the coordinates (x, y) can be determined by checking the Tv value stored at the coordinates (x, y) of the Tv value map 812.

  The Sv value map 813 is information that expresses the Sv value representing the ISO sensitivity set for each block 202 in the form of a two-dimensional map, like the Tv value map 812.

  The Bv value map 814 calculates the subject brightness measured for each block 202 at the time of imaging in FIG. 7B, that is, the Bv value representing the brightness of the subject light incident on each block 202 in the same manner as the Tv value map 812. Is information expressed in the form of a two-dimensional map.

  The Av value information 815 is information indicating the aperture value at the time of imaging in FIG. 7B. The Av value is different from the Tv value, the Sv value, and the Bv value, and is not a value existing for each block 202. Therefore, unlike the Tv value, the Sv value, and the Bv value, the Av value stores only a single value and does not become information in which a plurality of values are mapped two-dimensionally.

  As described above, the control unit 502 performs image capturing using the multi-image capturing condition still image capturing function, so that the image information 811 generated by the image sensor 100 capable of setting different image capturing conditions for each block 202 and the image information 811 generated for each block 202 The image file 600 associated with the data (imaging condition information 802, mask information 704, Tv value map 812, Sv value map 813, Bv value map 814, etc.) and the Av value information 815 is stored in the memory card 504. Record.

<Detailed block configuration example of electronic device>
FIG. 9 is an explanatory diagram illustrating a detailed block configuration example of an electronic device. The electronic device 500 includes the imaging device 100, a control unit 502, a display unit 901, and a recording unit 902. The display unit 901 specifically corresponds to, for example, the liquid crystal monitor 503 or the EVF (Electronic View Finder) shown in FIG. The recording unit 902 specifically corresponds to, for example, the memory card 504, the DRAM 506, and the flash memory 507 shown in FIG.

  As described above, the imaging element 100 can set different imaging conditions for each of a plurality of imaging regions for imaging a subject, and has the imaging chip 113 and the signal processing chip 111. The imaging region is a block 202 existing on the imaging surface 200. The signal processing chip 111 includes a setting circuit 911. The setting circuit 911 sets the imaging conditions for each block 202 according to the setting information from the control unit 502.

  The control unit 502 includes an image processing unit 912 and a setting unit 913. The image processing unit 912 is realized by an image processing program executed by an image processing circuit or a processor. The image processing unit 912 performs the image analysis processing as described with reference to FIG. More specifically, for example, the image processing unit 912 performs subject detection, and discriminates the subject image from the background image, and performs region discrimination for specifying the background region 701, the main subject region 702, and the coexistence region 703.

  The setting unit 913 sets the first imaging condition in the first imaging region where the first subject image is present among the plurality of imaging regions, and sets the first imaging condition in the second imaging region where the second subject image is present among the plurality of imaging regions. (2) The imaging condition is set, and the third imaging condition of the third imaging region in which the first subject image and the second subject image are present among the plurality of imaging regions is based on at least one of the first imaging condition and the second imaging condition. To set.

  Specifically, for example, the setting unit 913 sets the exposure time as the first imaging condition for the block 202 (the value of the mask information 704 is “1”) in which the background image exists in the group of blocks constituting the imaging surface 200. : Set a short second (for example, 1/60 second). The block 202 in which the value of the mask information 704 is “1” is referred to as “background block 202”.

  In addition, the setting unit 913 sets the exposure time: long seconds (second value) as the second imaging condition for the block 202 (the value of the mask information 704 is “2”) in which the main subject image MI exists in the block group forming the imaging surface 200. For example, 1/1000 second) is set. The block 202 in which the value of the mask information 704 is “2” is referred to as “main subject block 202”.

  In addition, the setting unit 913 sets the background of the third imaging condition of the block 202 (the value of the mask information 704 is “0”) in which the main subject image MI and the background image are present among the blocks forming the imaging surface 200. Based on the luminance value of the block 202 and the luminance value of the main subject block 202, the following [1] and [2] are set. The block 202 in which the value of the mask information 704 is “0” is referred to as “undefined block 202”.

[1. One of background block 202 (short second) and main subject block 202 (long second)]
The setting unit 913 sets the third imaging condition of the indefinite block 202 as one of the first imaging condition of the background block 202 and the second imaging condition of the main subject block 202. For example, when the background block 202 is overexposed and the main subject block 202 is not overexposed, when the second imaging condition is set for the undefined block 202, the image data of the background image portion in the undefined block 202 is overexposed. The setting unit 913 sets the third imaging condition of the indefinite block 202 to the first imaging condition (short second) of the background block 202.

  In addition, when the first imaging condition is set in the undefined block 202 and the main subject portion in the undefined block 202 is lost in black, the setting unit 913 sets the third imaging condition of the undefined block 202 to the third imaging condition of the main subject block 202. 2. Set the imaging conditions (long seconds). In addition, no matter which of the first imaging condition and the second imaging condition is set in the indefinite block 202, if there is no overexposure or underexposure in the indefinite block, the main subject is given priority. , The third imaging condition of the indefinite block 202 is set to the second imaging condition (long time) of the main subject block 202.

  Here, the overexposed case is a case where the luminance value of the indefinite block 202 is equal to or more than the first luminance value. The first luminance value is a luminance value at which a pixel group (a group of photoelectric conversion units) forming the undefined block 202 becomes saturated. Further, the case where the black area is lost is a case where the luminance value of the undefined block 202 is equal to or less than a second luminance value lower than the first luminance value. The second luminance value is a state in which the luminance of a pixel group (a set of photoelectric conversion units) forming the undefined block 202 is insufficient. Specifically, this is the case when the luminance value is 0. Note that the second luminance value may not be 0, and a value larger than 0 may be set as the second luminance value.

  It is not necessary that the background image portions of all the undefined blocks 202 be overexposed. For example, when the background image portion of some of the undefined blocks 202 is overexposed, the setting unit 913 sets the third imaging condition of the undefined block 202 to the first imaging condition of the background block 202 (short-time ) May be set. Similarly, it is not necessary that the main subject image MI portions of all the indefinite blocks 202 be blackened. For example, when the main subject image MI portion of some of the indefinite blocks 202 is blackened, the setting unit 913 sets the third imaging condition of the indefinite block 202 to the second imaging condition of the main subject block 202. (Long).

[2. Between the background block 202 (short second) and the main subject block 202 (long second)]
The setting unit 913 sets the third imaging condition of the indefinite block 202 as an imaging condition between the first imaging condition of the background block 202 and the second imaging condition of the main subject block 202. For example, when the exposure time as the first imaging condition of the background block 202 is 1/1000 second (short second) and the exposure time as the second imaging condition of the main subject block 202 is 1/60 second (long second). There are three steps of exposure time (1/125 seconds, 1/250 seconds, 1/500 seconds) between the first imaging condition and the second imaging condition.

  Therefore, the setting unit 913 sets the third imaging condition to any one of these exposure times. The setting unit 913 may set the third imaging condition to an intermediate exposure time: 1/250 second between the first imaging condition and the second imaging condition. Further, when the setting for suppressing the brightness of the background is set, the setting unit 913 may set the third imaging condition to an exposure time near the first imaging condition: 1/500 second. Further, when a setting is made such that the main subject is prioritized, the setting unit 913 may set the third imaging condition to an exposure time near the second imaging condition: 1/125 second.

  After that, the setting unit 913 transmits setting information as a setting result to the setting circuit 911 in the signal processing chip 111. The setting information includes a coordinate value specifying the block 202 and a value indicating an imaging condition set for each block 202. The value indicating the imaging condition is, for example, a number indicating the exposure time.

  Upon receiving the setting information, the setting circuit 911 in the signal processing chip 111 sets the charge accumulation time from the start to the end of charge accumulation, that is, the exposure time, in units of the block 202 according to the setting information. Thereby, the image sensor 100 can acquire the image data of the next frame according to the setting information.

  The image processing unit 912 in the control unit 502 has an adjustment unit 921. The adjustment unit 921 adjusts the signal level of the image data output from the indefinite block 202 so as to amplify or reduce the signal level according to the setting information from the setting unit 913 (so-called digital gain adjustment). Specifically, for example, the adjusting unit 921 performs gain adjustment as in [3] and [4] below.

[3. Gain adjustment with the above setting [1]]
When the setting unit 913 sets the third imaging condition of the indefinite block 202 to the first imaging condition (short-time) by the setting unit 913, the adjustment unit 921 replaces the image data of the main subject in the image data from the indefinite block 202 with the second imaging condition. The image becomes darker than image data from the main subject block 202 for which the imaging condition (long seconds) is set. As a result, the boundary between the main subject block 202 and the indefinite block 202 becomes conspicuous in the main subject area 702. For this reason, the image processing unit 912 specifies the image data regarding the main subject from the image data from the indefinite block 202 by the area determination, and the adjustment unit 921 performs adjustment so that the signal level of the image data regarding the main subject is amplified. . That is, the adjusting unit 921 uniformly increases the luminance value of the pixel group forming the image data for the main subject by a predetermined amount.

  In addition, when the setting unit 913 sets the third imaging condition of the indefinite block 202 to the second imaging condition (long time), the adjustment unit 921 replaces the image data on the background in the image data from the undefined block 202 with the second imaging condition. It becomes brighter than image data from the background block 202 for which one imaging condition (short second) is set. As a result, the boundary between the background block 202 and the undefined block 202 becomes conspicuous in the background area 701. For this reason, the image processing unit 912 specifies the image data regarding the background from the image data from the indefinite block 202 by the area determination, and the adjustment unit 921 adjusts the signal level of the image data regarding the background to decrease. That is, the adjustment unit 921 uniformly reduces the luminance value of the pixel group forming the image data relating to the background by a predetermined amount.

[4. Gain adjustment with the above setting [2]]
When the setting unit 913 sets the third imaging condition of the indefinite block 202 closer to the first imaging condition (short second) (for example, 1/500 second), the adjustment unit 921 outputs the image data from the indefinite block 202. The image data of the main subject is darker than the image data from the main subject block 202 for which the second imaging condition (long time) is set. As a result, the boundary between the main subject block 202 and the indefinite block 202 becomes conspicuous in the main subject area 702.

  For this reason, the image processing unit 912 specifies the image data of the main subject from the image data from the indefinite block 202 by the area determination, and the adjustment unit 921 performs adjustment so that the signal level of the image data of the main subject is amplified. . That is, the adjusting unit 921 uniformly increases the luminance value of the pixel group forming the image data for the main subject by a predetermined amount. Since the exposure time of the third imaging condition in this case is longer than that of the first imaging condition, the increase amount of the luminance value may be lower than the increase amount in the case of [3].

  When the setting unit 913 sets the third imaging condition of the indefinite block 202 closer to the second imaging condition (long seconds) (for example, 1/125 second), the adjustment unit 921 outputs the image data from the indefinite block 202. Among them, the image data relating to the background becomes brighter than the image data from the background block 202 for which the first imaging condition (short time) is set. As a result, the boundary between the background block 202 and the undefined block 202 becomes conspicuous in the background area 701.

  For this reason, the image processing unit 912 specifies the image data regarding the background from the image data from the indefinite block 202 by the area determination, and the adjustment unit 921 adjusts the signal level of the image data regarding the background to decrease. That is, the adjustment unit 921 uniformly reduces the luminance value of the pixel group forming the image data relating to the background by a predetermined amount. Since the exposure time of the third imaging condition in this case is shorter than that of the second imaging condition, the amount of decrease in the luminance value may be lower than the amount of decrease in the case of [3].

  When the setting unit 913 sets the third imaging condition of the indefinite block 202 to an intermediate value (for example, 1/250 second) between the first imaging condition and the second imaging condition, the above-described first imaging condition (short second) As in the case of the shift and the shift to the second imaging condition (long time), the adjustment unit 921 increases the signal level of the image data of the main subject described above for the indefinite block 202 and decreases the signal level of the image data of the background. May be performed. Further, since the image structure in the undefined block 202 is complicated due to a large number of textures, the area division between the main subject image MI portion and the background image portion may not be appropriately performed in the undefined block 202. The adjustment unit 921 may not execute the gain adjustment for such an indefinite block 202.

  The generation unit 922 in the image processing unit 912 generates the above-described image file 600. The generation unit 922 generates the image file 600, for example, when the subject is imaged by pressing the release button. In the image file 600, the background area 701, the image data from the background area 701, and the first imaging condition (short time) are associated.

  In the image file 600, the main subject area 702, the image data from the main subject area 702, and the second imaging condition (long seconds) are associated. In the image file 600, the coexistence area 703 and the image data from the coexistence area 703 are associated with the third imaging condition.

  Specifically, for example, the type of the subject for each block 202 is recorded in the mask information 704. The imaging conditions set for each block 202 are recorded in the above-described Tv value map 812. Further, the luminance value for each block 202 is recorded in the above-described Bv value map 814.

  When sequentially displaying sequentially generated images (through images) based on the output from the image sensor 100, the image processing unit 912 outputs the thinned image data to the display unit, and outputs the image data to the display unit 901. Display images sequentially. When the release button is pressed, the image processing unit 912 outputs the image data output from all pixels to the display unit 901. The generating unit 922 records the generated image file 600 in the recording unit 902. Note that the prediction unit 923 and the detection unit 924 have the functions of the second embodiment described later, and thus the description thereof is omitted in the first embodiment.

<Example of setting and adjustment of third imaging condition>
10 to 12 are explanatory diagrams showing setting examples 1 to 3 of the third imaging condition. 10A to 12A show the imaging surface 200 before setting the third imaging condition, and FIG. 10B shows the imaging surface 200 after setting the third imaging condition. In addition, the main subject image MI is a central woman, and the background image BI is other than the main subject image MI. Here, an example is described in which the subject is imaged in backlight.

  The grid of the thin lines is the block 202. In particular, a set of the background blocks 202 including only the background image BI is the background area 701, and the first imaging condition (short-time) is set. A set of main subject blocks 202 including only the main subject image MI is a main subject area 702, and a second imaging condition (long seconds) is set. A set of undefined blocks 202 in which the main subject image MI and the background image BI surrounded by a thick line exist is the coexistence area 703, the setting unit 913 sets the third imaging condition, and the adjustment unit 921 adjusts the gain.

  FIG. 10 illustrates a case where the background image BI in the indefinite block 202 is overexposed by setting the same second imaging condition (long time) as that of the main subject area 702 in the indefinite block 202. This is an example in which the same first imaging condition (short seconds) is set. In this case, the luminance value of the main subject image MI portion in the coexistence region 703 is lower than that of the main subject region 702. As a result, the boundary between the main subject block 202 and the indefinite block 202 becomes conspicuous in the main subject area 702. Therefore, the adjustment unit 921 increases the gain of the luminance value of the main subject image MI for each undefined block 202 in the coexistence area 703.

  FIG. 11 illustrates a case where the main subject image MI portion in the indefinite block 202 is blackened by setting the same first imaging condition (short-time) as that of the background area 701 in the indefinite block 202. This is an example in which the same second imaging condition (long time) as that of MI is set. If the background image BI is not overexposed and the main subject image MI is not overexposed, the setting unit 913 gives priority to the main subject. Set conditions (long seconds).

  In these cases, since the coexistence area 703 also has a long time as shown in FIG. 3B, the luminance value of the background image BI in the coexistence area 703 is higher than that of the background area 701. As a result, the boundary between the background block 202 and the undefined block 202 becomes conspicuous in the background area 701. Therefore, the adjustment unit 921 reduces the gain of the luminance value of the background image BI portion for each of the undefined blocks 202 in the coexistence area 703.

  FIG. 12 illustrates a case where the first imaging condition (short-time) and the second imaging condition (long-time) are set in the indefinite block 202 because the background image BI is overexposed and the main subject image MI is overexposed. In addition, this is an example in which the third imaging condition of the indefinite block 202 is set as an imaging condition between the first imaging condition (short seconds) and the second imaging condition (long seconds). In this case, as shown in (B), if the coexistence area 703 is closer to a short time, the luminance value of the main subject image MI portion in the coexistence area 703 becomes lower than that of the main subject area 702. As a result, the boundary between the main subject block 202 and the indefinite block 202 becomes conspicuous in the main subject area 702.

  Therefore, the adjustment unit 921 increases the gain of the luminance value of the main subject image MI for each undefined block 202 in the coexistence area 703. On the other hand, if the coexistence area 703 is closer to a long time, the luminance value of the background image BI in the coexistence area 703 becomes higher than that of the background area 701. As a result, the boundary between the background block 202 and the undefined block 202 becomes conspicuous in the background area 701. Therefore, the adjustment unit 921 reduces the gain of the luminance value of the background image BI portion for each of the undefined blocks 202 in the coexistence area 703.

  In FIGS. 10 to 12, the backlight has been described as an example. However, in the case of normal light, the setting and adjustment unit 921 by the setting unit 913 is performed by setting the first imaging condition to long seconds and the second imaging condition to short seconds. Is performed. In the first embodiment, the exposure time has been described as an example of the imaging condition. However, the imaging condition may be ISO sensitivity or exposure value.

  As described above, according to the first embodiment, the third imaging condition of the third imaging region that is the boundary between the first subject image (background image BI) and the second subject image (main subject image MI) is the first imaging condition. The imaging condition is not related to the condition or the second imaging condition. Therefore, it is possible to display a subject image without a sense of discomfort.

  Next, a second embodiment will be described. In the second embodiment, since the description about the second embodiment will be mainly given, the description of the same content as the first embodiment will be omitted by using the same reference numerals. Also, in the second embodiment, as in the first embodiment, the description will be made by taking an example of shooting at the time of backlight (the first imaging condition is short, and the second imaging condition is long). The first imaging condition is long, and the second imaging condition is short. In the second embodiment, similarly to the first embodiment, the exposure time will be described as an example of the imaging condition. However, the imaging condition may be ISO sensitivity or exposure value.

<Influence on coexistence area 703>
In the second embodiment, at least one of the background region 701 and the main subject region 702 existing within a predetermined range from the coexistence region 703 is set to the third imaging condition of the coexistence region 703 including the indefinite block 202 described in the first embodiment. 7 shows a setting example of setting based on the degree of influence on the imaging region.

  FIG. 13 is an explanatory diagram showing the degree of influence on the coexistence area 703. The background area 701 existing within the predetermined range is a block set in which a predetermined number of background blocks 202 are connected to the undefined block 202. Further, all the background blocks 202 existing on the imaging surface 200 can also be the background area 701 existing within the predetermined range.

  Similarly, the main subject area 702 existing within the predetermined range is a block set in which a predetermined number of main subject blocks 202 are connected to the undefined block 202. In addition, all the main subject blocks 202 existing on the imaging plane 200 can also be the main subject area 702 existing within a predetermined range. The extent of the predetermined range (how much the predetermined number should be) is preset in electronic device 500.

  The degree of influence is an index value indicating the influence of the coexistence area 703 on the image from the background area 701 and the main subject area 702 within a predetermined range. Specifically, for example, the degree of influence is determined by a difference in evaluation value regarding the area between the background image BI portion and the main subject image MI portion in the indefinite block 202 or the background image BI portion and the main subject image in the indefinite block 202. This is the difference between the evaluation value for the complexity of the image and the image MI. The evaluation value relating to the complexity of the image is, for example, an integral value of a frequency component of the image. Details will be described later.

  The evaluation value (hereinafter, area value S1) relating to the area of the background image BI portion in the indefinite block 202 is, for example, the number of pixels of the background image BI portion. The evaluation value (hereinafter, area value S2) regarding the area of the main subject image MI in the indefinite block 202 is, for example, the number of pixels of the main subject image MI. If there are many blocks 202 in which the image having the larger area exists around the indeterminate block 202, the load of image processing can be reduced.

  When S1> S2 for the undefined block 202 in the coexistence area 703, the area of the background image BI in the undefined block 202 is larger than that of the main subject image MI. Therefore, the setting unit 913 sets the third imaging condition of the indefinite block 202 to the first imaging condition (short second) of the background area 701 where the background image BI exists. As a result, the indefinite block 202 and the background block 202 adjacent thereto have the same imaging condition, and the load of image processing is reduced.

  On the other hand, when S2> S1, the area of the main subject image MI portion is larger in the undefined block 202 than in the background image BI portion. Therefore, the setting unit 913 sets the third imaging condition of the indefinite block 202 to the second imaging condition (long time) of the main subject area 702 where the main subject image MI exists. As a result, the indefinite block 202 and the main subject block 202 adjacent to the uncertain block 202 have the same imaging condition, and the load of image processing is reduced.

  The evaluation value (hereinafter, complexity f1) relating to the complexity of the image of the background image BI in the indefinite block 202 is, for example, an integrated value of the frequency component of the background image BI. The evaluation value (hereinafter, complexity f2) relating to the complexity of the image of the main subject image MI in the indefinite block 202 is, for example, an integral value of the frequency component of the main subject image MI. The larger the integration value, in other words, the higher the frequency, the more complicated the structure of the image obtained from the subject image.

  When f1> f2 for the undefined block 202 in the coexistence area 703, the integrated value of the background image BI in the undefined block 202 is larger than that of the main subject image MI. This indicates that the structure of the image of the background image BI is complicated. Therefore, the setting unit 913 sets the third imaging condition of the indefinite block 202 to the first imaging condition (short second) of the background area 701 where the background image BI exists. As a result, the adjustment target of the adjustment unit 921 is the main subject image MI portion having a simpler image structure than the background image BI portion. Therefore, the adjustment unit 921 executes the adjustment process on the simpler image structure. The adjustment process can be facilitated.

  On the other hand, when f2> f1, in the undefined block 202, the integral value of the main subject image MI is larger than that of the background image BI. This indicates that the image structure of the main subject image MI is complicated. Therefore, the setting unit 913 sets the third imaging condition of the indefinite block 202 to the second imaging condition (long time) of the main subject area 702 where the main subject image MI exists. As a result, the adjustment target of the adjustment unit 921 is a background image BI portion having a simpler image structure than the main subject image MI portion. Therefore, the adjustment unit 921 performs an adjustment process on a simpler image structure. The adjustment process can be facilitated.

  As described above, by taking the area into consideration, the target area of the image processing can be reduced. When the image processing is not performed, the boundary between the coexistence region 703 and the background region 701 or the boundary between the coexistence region 703 and the main subject region 702 becomes discontinuous, and the length of the vertical or horizontal boundary line becomes shorter. . As a result, the boundary line becomes less noticeable in the image. In addition, by considering the complexity of the image, adjustment by the adjustment unit 921 becomes easy, and the image quality can be improved.

  Note that, in the above description, the imaging condition with the higher complexity of the image is applied to the third imaging condition. However, the imaging condition with the lower complexity, that is, the imaging condition with the lower frequency component is used. The third imaging condition may be applied. In this case, similarly to the case where the area is taken into consideration, the boundary between different imaging conditions is an imaging region where a subject image having a more complicated image structure exists. For example, when the complexity of the main subject image MI is higher than that of the background image BI, the third imaging condition of the indefinite block 202 is the first imaging condition (long time) of the background image BI. Therefore, a boundary of different imaging conditions exists between the coexistence area 703 and the main subject area 702. However, since the image structure of the main subject region 702 is complicated, even if a boundary line is formed in the main subject region 702, the boundary line is less noticeable in the image.

<Example of setting unit>
FIG. 14 is an explanatory diagram showing an example of a setting unit of the undefined block 202. The setting unit is the number of one block of indefinite blocks 202 that handles the degree of influence shown in FIG. In FIG. 14, a block 202 or a set of blocks indicated by a thick black line is a setting unit. (A) uses one block as a setting unit. (B) uses a block set connected in the horizontal direction and the vertical direction as a setting unit. The setting unit 913 determines a setting unit by performing raster scanning sequentially from left to right in each row from the upper left block 202, for example. Therefore, the blocks 202 overlapping in the horizontal direction and the vertical direction are preferentially included in the horizontal block set.

  The setting unit 913 can set the third imaging condition based on the degree of influence described above in units of the setting. Thereby, the continuity of the background image BI portion and the main subject image MI portion in the indefinite block 202 can be considered. For example, in one undefined block 202, even if the area of the background image BI portion is smaller than the main subject image MI portion, the area of the background image BI portion is small in the setting unit composed of the plurality of undefined blocks 202. It may be larger than the main subject image MI part. Thereby, the third imaging condition of the indefinite block 202 can be set to an appropriate imaging condition.

<Boundary length of coexistence area 703>
FIG. 15 is an explanatory diagram showing the boundary length of the coexistence area 703. 15, a thick black line indicates a boundary between the background region 701 and the coexistence region 703, and a thick white line indicates a boundary between the main subject region 702 and the coexistence region 703. The boundary between the blocks 202 is conspicuous if the imaging conditions of the background block 202 and the undefined block 202 on both sides of the boundary where the region between the thick black line and the thick white line is the coexistence region 703 are different. Similarly, if the imaging conditions of the main subject block 202 and the indefinite block 202 on both sides of the boundary are different, the boundary between the blocks 202 is conspicuous. The length of the conspicuous boundary is preferably shorter.

  Therefore, the setting unit 913 compares the length of the boundary (thick black line) between the background area 701 and the coexistence area 703 with the length of the boundary (thick white line) between the main subject area 702 and the coexistence area 703. Then, the setting unit 913 sets the third imaging condition to the first imaging condition or the second imaging condition so that the imaging conditions are the same on both sides of the longer boundary (so that the imaging conditions are different on both sides of the shorter boundary). Set to. This makes it possible to make the boundary where the imaging conditions differ on both sides less noticeable.

<Halo measures>
FIG. 16 is an explanatory diagram showing reduction of the influence of halo. The halo (phenomenon) is an aberration that occurs as a result of the path of the subject light being distorted when the subject light passes through the lens. For example, in backlight shooting, halo is likely to occur at the boundary between a main subject that is shadowed by backlight and a bright background. (A) shows the main subject image MI in which the halo 1600 has occurred, and (B) and (C) are enlarged views of a certain indefinite block 202a. Although the halo 1600 exists in the background block 202, in the subject detection, up to the outer edge (dashed line) of the halo 1600 including the main subject image MI is detected as the main subject.

  In (B), it is assumed that the third imaging condition of the indefinite block 202a is set to the first imaging condition (short seconds). In this case, the adjustment unit 921 does not adjust the gain of the image data from the background area 701 (excluding the halo 1600). The adjustment unit 921 performs gain-up adjustment on image data from the main subject area 702 (including the halo 1600). In this case, when the halo 1600 is predicted, similarly to the image data from the background region 701, the adjusting unit 921 does not perform gain adjustment on the image data from the halo 1600 region (the gain may be reduced). Therefore, when displaying a subject image, an image in which halo 1600 is suppressed can be displayed.

  In (C), it is assumed that the third imaging condition of the indefinite block 202a is set to the first imaging condition (long seconds). In this case, the adjustment unit 921 performs a gain-down adjustment on the image data from the background area 701 (excluding the halo 1600). The adjustment unit 921 does not perform gain adjustment on image data from the main subject area 702 (including the halo 1600). In this case, when the halo 1600 is predicted, the adjustment unit 921 performs a gain-down adjustment on the image data from the halo 1600 region, similarly to the image data from the background region 701. Therefore, when displaying a subject image, an image in which halo 1600 is suppressed can be displayed.

  The prediction of halo is performed by the prediction unit 923 shown in FIG. Specifically, for example, the prediction unit 923 predicts that a halo will occur if the level difference between the first imaging condition (short seconds) and the second imaging condition (long seconds) is equal to or greater than a predetermined level difference. For example, when the exposure time as the first imaging condition of the background block 202 is 1/1000 second (short second) and the exposure time as the second imaging condition of the main subject block 202 is 1/60 second (long second). There are three steps of exposure time (1/125 seconds, 1/250 seconds, 1/500 seconds) between the first imaging condition and the second imaging condition. If the threshold value of the halo occurrence prediction is two or more steps, there is a difference of three steps, so the prediction unit 923 predicts that a halo will occur. Further, the prediction unit 923 may predict the occurrence of halo based on image data from the image sensor 100.

  FIG. 17 is a graph showing an example of halo occurrence prediction using the γ function. The horizontal axis is the output value of the RAW image data from the image sensor 100, and the vertical axis is the output value after the γ correction in the image processing unit 912. x1 is the output value of image data from the area where the main subject image MI (area other than halo) exists, and y1 is the output value of x1 after γ correction. x2 is the output value of the image data from the area where the outer edge of the main subject image MI (halo 1600) is present, and y2 is the output value of x2 after γ correction.

  The prediction unit 923 predicts that the halo 1600 has occurred when the luminance difference between y1 and y2 is equal to or greater than a predetermined value. Thus, as shown in FIG. 16, the gain adjustment by the adjustment unit 921 is performed.

  When the user captures an image of a subject with the electronic device 500, the user sets imaging conditions based on the degree of influence on the coexistence region 703, sets imaging conditions based on the boundary length of the coexistence region 703, and measures for halo. At least one of the settings is applicable.

  When two or more of these settings are applied, for example, when the setting of the imaging condition based on the degree of influence on the coexistence region 703 and the setting of the imaging condition based on the boundary length of the coexistence region 703 are applied, The electronic device 500 preferentially sets an imaging condition based on the degree of influence on the coexistence area 703.

  When the setting of the imaging condition based on the degree of influence on the coexistence area 703 and the setting for the countermeasure against halo are applied, the electronic device 500 gives priority to the setting of the imaging condition based on the degree of influence on the coexistence area 703. To run. When the setting of the imaging condition based on the degree of influence on the coexistence area 703, the setting of the imaging condition based on the boundary length of the coexistence area 703, and the setting for the halo countermeasure are applied, the electronic device 500 The setting of the imaging condition based on the degree of influence on 703, the setting of the imaging condition based on the boundary length of the coexistence region 703, and the setting for countermeasures against halo are executed in this order.

  When the setting of the imaging condition based on the boundary length of the coexistence region 703 and the setting for the halo countermeasure are applied, the electronic device 500 preferentially sets the imaging condition based on the boundary length of the coexistence region 703. Run.

  However, when the electronic device 500 detects a camera shake or a subject shake by the detection unit 924, the setting of the imaging condition giving priority to the main subject is executed. The setting of the imaging condition giving priority to the main subject is a setting for setting the block 202 including the main subject image MI to the same imaging condition. Specifically, for example, the coexistence region 703 is set to the same imaging condition as the main subject region 702. It is a setting to do.

  Specifically, for example, in FIG. 15, the setting unit 913 sets the coexistence area 703 to the same imaging condition (long time) as the main subject area 702. Regarding camera shake, for example, the detection unit 924 detects a camera shake of a predetermined amount or more using a gyro sensor or an acceleration sensor built in the electronic device 500. For subject blur, for example, the detection unit 924 detects a motion vector indicating the direction and amount of movement of the main subject, and if the amount of movement in the direction in which the main subject moves between consecutive frames is equal to or more than a predetermined amount, It is detected as subject blur.

  As for the setting of the imaging condition based on the degree of influence on the coexistence region 703, one of the area and the complexity of the image is selected. When the area is selected, the setting of the imaging condition based on the degree of influence on the coexistence region 703 can be speeded up. On the other hand, when the complexity of the image is selected, the setting of the imaging condition based on the degree of influence on the coexistence area 703 can be made more accurate.

<Setting procedure example>
FIG. 18 is a flowchart illustrating an example 1 of an imaging condition setting processing procedure in the electronic device 500. The electronic device 500 performs an initial setting according to a user's operation input (step S1801). In the initial setting (step S1801), the degree of influence (area or complexity) on the coexistence area 703, the boundary length of the coexistence area 703, countermeasures against halo, the number of blocks included in the setting unit, and the overexposure and underexposure setting processing are executed. It is set whether or not.

  As shown in FIG. 10, the overexposure / underexposure avoidance setting processing is to set the third imaging condition of the indefinite block 202 to the first imaging condition (short second) or the second imaging condition (long second) by the setting unit 913. This is a process for setting any one. Specifically, the setting unit 913 sets, in the indeterminate block 202, an imaging condition that does not cause overexposure and underexposure among the first imaging condition and the second imaging condition.

  Next, the electronic device 500 starts acquiring a through image from the image sensor 100 (step S1802). In the electronic device 500, the image processing unit 912 performs image analysis on the acquired image data of the through image (step S1803). Specifically, for example, the image processing unit 912 detects the main subject by subject detection, sets the background as a background other than the main subject, and determines the area type (background area 701, main subject area 702, The coexistence area 703) is determined.

  Next, the electronic device 500 determines whether or not the degree of influence (area or complexity of the image) on the coexistence area 703 has been set by the initial setting (step S1801) (step S1804). If the degree of influence on the coexistence area 703 has not been set (step S1804: No), the main object priority mode is set, and the electronic device 500 uses the setting unit 913 to set the white / black loss avoidance setting processing or the main object priority. A setting process is performed (step S1805), and the process proceeds to step S1810. Specifically, for example, when the initial setting (step S1801) is set to execute the overexposure and underexposure setting processing, the overexposure and underexposure setting processing is executed. A main subject priority setting process is executed. In addition, in the case where the overexposure and the overexposure of the black image do not occur regardless of whether the first imaging condition or the second imaging condition is set, the main subject priority setting process may be performed.

  In the main subject priority setting process (step S1805), as shown in FIG. 11, the setting unit 913 sets the third imaging condition of the indefinite block 202 to the same second imaging condition (long second) as the main subject image MI. This is the processing to be performed. In this case, as shown in FIG. 11B, since the coexistence area 703 also has a long time, the luminance value of the background image BI in the coexistence area 703 is higher than that of the background area 701. Therefore, in the electronic device 500, in the gain adjustment process (step S1810), the luminance value of the background image BI portion for each of the undefined blocks 202 in the coexistence area 703 is reduced by the adjustment unit 921.

  If the degree of influence on the coexistence area 703 has been set in step S1804 (step S1804: YES), it is determined whether the detection flag of camera shake or subject detection blur is ON or OFF (step S1806). . If the detection flag is ON (step S1806: ON), it indicates that camera shake or subject detection blur is being detected, and the process proceeds to the main subject priority setting process (step S1805). If the detection flag is OFF (step S1806: OFF), it indicates that no camera shake or object detection blur has been detected, and the flow shifts to step S1807.

  In step S1807, the electronic device 500 determines whether the initial setting (step S1801) of the influence on the coexistence region 703 is an area or image complexity (step S1807). If it is an area (step S1807: area), the electronic device 500 executes an area priority setting process (step S1808), and proceeds to a gain adjustment process (step S1810). Details of the area priority setting processing (step S1808) will be described later with reference to FIG. 19 or FIG.

  On the other hand, in the case of image complexity (step S1807: complexity), the electronic device 500 executes complexity priority setting processing (step S1809), and proceeds to gain adjustment processing (step S1810). Details of the complexity priority setting process (step S1809) will be described later with reference to FIG. 25 or FIG.

  In step S1810, electronic device 500 performs gain adjustment processing by adjustment unit 921 (step S1810). For example, when the third imaging condition of the indefinite block 202 is set to the first imaging condition (short second) of the background block 202, the adjustment unit 921 gains image data from the main subject image MI in the indefinite block 202. Up. When the third imaging condition of the indefinite block 202 is set to the second imaging condition (long seconds) of the main subject block 202, the gain of the image data from the background image BI in the indefinite block 202 is reduced.

  After that, the electronic device 500 detects the camera shake or the subject shake by the detection unit 924 (step S1811). If camera shake or subject blur has been detected (step S1811: YES), the detection flag is set to ON (step S1812), and the flow shifts to step S1814. On the other hand, if no camera shake or subject shake has been detected (step S1811: No), the detection flag is set to OFF, and the flow shifts to step S1814.

  The electronic device 500 determines whether or not the through image acquisition ends (step S1811). If the through image acquisition does not end (step S1814: No), the process returns to step S1803. On the other hand, when ending the live view image acquisition (step S1814: Yes), the electronic device 500 ends the imaging condition setting process. Note that the user can press the release button at any time during the execution of the imaging condition setting process to image the subject. In this case, the image data is generated by the output of all the pixels from the image sensor 100 at the image capturing timing, and the image file 600 is recorded in the recording unit.

<Area priority setting processing (step S1808)>
FIG. 19 is a flowchart illustrating a detailed processing procedure example 1 of the area priority setting processing (step S1808) illustrated in FIG. The area priority setting processing (step S1808) in FIG. 19 is processing for setting the third imaging condition based on the size of the area of the background image BI portion and the main subject image MI portion in the indefinite block 202.

  The electronic device 500 uses the setting unit 913 to determine whether there is an unselected setting unit (step S1901). When there is an unselected setting unit (step S1901: Yes), the electronic device 500 causes the setting unit 913 to select one unselected setting unit (step S1902). Then, the electronic device 500 causes the setting unit 913 to calculate the area value S1 of the background image BI portion and the area value S2 of the main subject image MI portion in the selection setting unit (step S1903).

  Then, the electronic device 500 causes the setting unit 913 to compare the area values S1 and S2 (step S1904). When the area value S1 is larger (Step S1904: Yes), the electronic device 500 sets the third imaging condition of the selection setting unit as the first imaging condition of the background region 701 by the setting unit 913 (Step S1905). ), And return to step S1901.

  If the area value S1 is not larger (step S1904: No), the electronic device 500 sets the third imaging condition of the selection setting unit as the second imaging condition of the main subject region 702 by the setting unit 913. (Step S1906), and returns to Step S1901. If there is no unselected setting unit in step S1901 (step S1901: No), the electronic device 500 ends the area priority setting process (step S1808) and shifts to the gain adjustment process (step S1810).

  FIG. 20 is a flowchart illustrating a detailed processing procedure example 2 of the area priority setting processing (step S1808) illustrated in FIG. The area priority setting processing (step S1808) in FIG. 20 sets the third imaging condition based on the size of the area of the background image BI portion and the main subject image MI portion in the indefinite block 202, and sets the area difference within the allowable range. For example, since there is no difference in the area, the third imaging condition is set based on the boundary length of the coexistence region 703 and the halo countermeasure. The same steps as those in FIG. 19 are denoted by the same step numbers, and description thereof will be omitted.

  The electronic device 500 calculates the area value S1 of the background image BI part and the area value S2 of the main subject image MI part in the selection setting unit by the setting unit 913 (step S1903), and finds the difference between the area values S1 and S2. It is determined whether or not the area difference is within an allowable range (step S2001). If it is within the allowable range (step S2001: Yes), the electronic device 500 causes the setting unit 913 to add 1 to the indefinite count C (step S2002), and proceeds to step S1904. The indefinite count C is the number of times the area difference has become an allowable range. The initial value of the undefined count C is 0. On the other hand, if the area difference is not within the allowable range (step S2001: No), the process moves to step S1904.

  If the area value S1 is larger (Step S1904: Yes), the electronic device 500 temporarily sets the third imaging condition of the selection setting unit as the first imaging condition of the background area 701 by the setting unit 913. (Step S2003), and it returns to step S1901.

  If the area value S1 is not larger (step S1904: No), the electronic device 500 causes the setting unit 913 to tentatively set the third imaging condition of the selection setting unit to the second imaging condition of the main subject area 702. Then (step S2004), the process returns to step S1901. If there is no unselected setting unit in step S1901 (step S1901: No), the setting unit 913 of the electronic device 500 determines whether the ratio of the indefinite count C is greater than or equal to a predetermined ratio (step S2005). For example, assuming that the number of set units is N and the predetermined ratio is 0.4, if C / N is 0.4 or less, the process proceeds to step S2006.

  If the ratio is equal to or greater than the predetermined ratio (step S2005: Yes), the electronic device 500 causes the setting unit 913 to determine the third imaging condition of each setting unit provisionally set in steps S2003 and S2004 (step S2006). That is, as the indeterminate count number C is smaller, the area difference is out of the allowable range when the entire image is viewed, and thus the provisionally determined third imaging condition is reflected. Then, the area priority setting processing (step S1808) is completed, and the processing shifts to the gain adjustment processing (step S1810).

  On the other hand, the case where the ratio of the indefinite count C is not less than the predetermined ratio in step S2005 (step S2005: No) will be described with reference to FIGS.

  FIG. 21 is a flowchart illustrating a detailed processing procedure example 2-1 of the area priority setting processing (step S1808) illustrated in FIG. FIG. 21 shows a process in which, in the initial setting (step S1801), when the setting of the boundary length of the coexistence area 703 is ON, the process branches from step S2005: No in FIG. 20 and shifts to the gain adjustment process (step S1810). is there.

  The electronic device 500 uses the setting unit 913 to calculate the background-side boundary length L1 and the main subject-side boundary length L2 of the coexistence region 703 (step S2101). For example, in FIG. 15, the background-side boundary length L1 is the entire length of the thick black line, and is the total length of the main subject-side boundary length L2. Then, the electronic device 500 causes the setting unit 913 to compare the boundary lengths L1 and L2 (step S1904). When the background-side boundary length L1 is longer (Step S2102: Yes), the electronic device 500 sets the third imaging condition of the coexistence region to the first imaging condition of the main subject region 702 by the setting unit 913 ( (Step S2103), and shifts to gain adjustment processing (step S1810).

  When the background-side boundary length L1 is not larger (step S2102: No), the electronic device 500 sets the third imaging condition of the coexistence area 703 to the second imaging condition of the background area 701 by the setting unit 913. After that (step S1906), the process proceeds to the gain adjustment process (step S1810). In this way, while giving priority to the degree of influence (area) on the coexistence area 703, if the difference in the area does not change, it is possible to set the imaging conditions of the imaging area having the shorter boundary length of the coexistence area 703. .

  FIG. 22 is a flowchart illustrating a detailed processing procedure example 2-2 of the area priority setting processing (step S1808) illustrated in FIG. FIG. 22 shows a case where the setting of the boundary length of the coexistence region 703 and the countermeasure against halo are ON in the initial setting (step S1801), and the process branches from step S2005: No in FIG. This is the processing to be performed. The same processes as those in FIG. 21 are denoted by the same step numbers, and description thereof is omitted.

  After step S2101, the electronic device 500 causes the setting unit 913 to determine whether the difference between the boundary lengths L1 and L2 (boundary length difference) is within an allowable range (step S2201). If it is not within the allowable range (step S2201: No), the process moves to step S2102. On the other hand, when the boundary length is within the allowable range (step S2201: Yes), the electronic device 500 causes the setting unit 913 to execute the halo countermeasure process (step S2202), and shifts to the gain adjustment process (step S1810). .

  FIG. 23 is a flowchart illustrating a detailed processing procedure example 1 of the halo countermeasure processing (step S2202) illustrated in FIG. The electronic device 500 uses the prediction unit 923 to predict the occurrence of halo (step S2301). For example, if the level difference between the first imaging condition and the second imaging condition is equal to or greater than a predetermined level, the prediction unit 923 predicts that halo will occur, and proceeds to step S2302. As shown in FIG. 17, the prediction unit 923 determines that the difference between the γ correction values y1 and y2 of the output value x1 of the image data from inside the main subject image MI and the output value x2 of the image data from the outer edge is predetermined. If the value is equal to or more than the value, it is predicted that halo will occur, and the flow shifts to step S2302.

  If it is predicted that no halo will occur (step S2301: No), the temporary settings in steps S2003 and S2004 are determined (step S2302), and the process proceeds to the gain adjustment processing (step S1810). On the other hand, when it is predicted that halo will occur (Step S2301: Yes), the electronic device 500 sets the imaging condition of the coexistence area 703 as the first imaging condition of the background image by the setting unit 913 (Step S2303). The process proceeds to the gain adjustment process (step S1810). Thus, by matching the coexistence region 703 corresponding to the outer edge of the main subject image MI with the first imaging condition (short seconds) inside the background image BI, the influence of halo can be reduced.

  As described above, when the influence (area) on the coexistence region 703 is prioritized and the difference in the area does not change, the imaging condition of the imaging region having the shorter boundary length of the coexistence region 703 is set, and the boundary condition is set. If there is no difference due to the length difference, the third imaging condition of the coexistence area 703 can be set by the halo countermeasure processing (step S2202).

  FIG. 24 is a flowchart illustrating a detailed processing procedure example 2-3 of the area priority setting processing (step S1808) illustrated in FIG. FIG. 24 is a flowchart of FIG. 24 when the setting of the boundary length of the coexistence area 703 is OFF and the setting for countermeasures against halo is ON in the initial setting (step S1801). This is the process of shifting to the process (step S1810). The same processes as those in FIG. 23 are denoted by the same step numbers, and description thereof will be omitted.

  By the processing shown in FIG. 24, the coexistence area 703 corresponding to the outer edge of the main subject image MI is matched with the first imaging condition (short second) inside the background image BI, so that the influence of halo can be reduced. it can. As described above, when priority is given to the degree of influence (area) on the coexistence area 703 and the difference in the area is not superior, the third imaging condition of the coexistence area 703 is set by the halo countermeasure processing (step S2202). Can be.

<Complexity Priority Setting Process (Step S1809)>
FIG. 25 is a flowchart illustrating a detailed processing procedure example 1 of the complexity priority setting processing (step S1809) illustrated in FIG. The complexity priority setting processing (step S1809) in FIG. 25 is processing for setting the third imaging condition based on the degree of complexity of the image of the background image BI portion and the main subject image MI portion in the undefined block 202.

  The electronic device 500 uses the setting unit 913 to determine whether there is an unselected setting unit (step S2501). When there is an unselected setting unit (step S2501: Yes), the electronic device 500 causes the setting unit 913 to select one unselected setting unit (step S2502). Then, the electronic device 500 uses the setting unit 913 to calculate the complexity f1 of the background image BI portion and the complexity f2 of the main subject image MI portion in the selection setting unit (step S2503).

  Then, the electronic device 500 causes the setting unit 913 to compare the complexity levels f1 and f2 (step S2504). If the complexity f1 is larger (step S2504: YES), the setting unit 913 sets the third imaging condition of the selection setting unit as the first imaging condition of the background area 701 (step S2505). ), And returns to step S2501.

  If the complexity f1 is not higher (step S2504: No), the electronic device 500 sets the third imaging condition of the unit of selection setting as the second imaging condition of the main subject area 702 by the setting unit 913. (Step S2506), and returns to step S2501. If there is no unselected setting unit in step S2501 (step S2501: No), the electronic device 500 ends the area priority setting process (step S1808) and shifts to the gain adjustment process (step S1810).

  FIG. 26 is a flowchart illustrating a detailed processing procedure example 2 of the complexity priority setting processing (step S1809) illustrated in FIG. The complexity priority setting processing (step S1809) in FIG. 26 sets the third imaging condition based on the degree of complexity of the image of the background image BI portion and the main subject image MI portion in the indefinite block 202, and sets a difference in complexity. If is within the allowable range, there is no difference in the complexity of the image, so the third imaging condition is set based on the boundary length of the coexistence area 703 and measures against halo. The same processes as those in FIG. 25 are denoted by the same step numbers, and description thereof is omitted.

  In the electronic device 500, the setting unit 913 calculates the complexity f1 of the background image BI portion and the complexity f2 of the main subject image MI portion in the selection setting unit (step S2503), and calculates the difference between the complexity degrees f1 and f2. It is determined whether or not is within the allowable range (step S2601). If it is within the allowable range (step S2601: Yes), the electronic device 500 causes the setting unit 913 to add 1 to the indefinite count C (step S2602), and proceeds to step S2504. The indefinite count C is the number of times that the difference in complexity has reached an allowable range. The initial value of the undefined count C is 0. On the other hand, if the complexity difference is not within the allowable range (step S2601: No), the process moves to step S2504.

  If the complexity f1 is higher (step S2504: YES), the electronic device 500 uses the setting unit 913 to tentatively set the third imaging condition of the selection setting unit as the first imaging condition of the background area 701. (Step S2603), and it returns to step S2501.

  If the complexity f1 is not greater (step S2504: No), the electronic device 500 causes the setting unit 913 to tentatively set the third imaging condition of the selection setting unit to the second imaging condition of the main subject area 702. Then (step S2604), the process returns to step S2501. If there is no unselected setting unit in step S2501 (step S2501: No), the electronic device 500 uses the setting unit 913 to determine whether the ratio of the indefinite count C is greater than or equal to a predetermined ratio (step S2605). For example, assuming that the number of set units is N and the predetermined ratio is 0.4, if C / N is 0.4 or less, the process proceeds to step S2606.

  If the ratio is equal to or higher than the predetermined ratio (step S2605: Yes), the electronic device 500 causes the setting unit 913 to determine the third imaging condition of each setting unit provisionally set in steps S2603 and S2604 (step S2606). In other words, the smaller the indefinite count number C, the more the difference in complexity is out of the allowable range when viewed over the entire image, so that the provisionally determined third imaging condition is reflected. Then, the complexity priority setting process (step S1809) ends, and the process proceeds to the gain adjustment process (step S1810).

  On the other hand, if the ratio of the indefinite count C is not equal to or lower than the predetermined ratio in step S2605 (step S2605: No), the processing shown in FIGS. 21 to 24 is applied. By applying the processing in FIG. 21, if priority is given to the degree of influence (complexity of the image) on the coexistence area 703 and the difference in complexity is not superior, the boundary length of the coexistence area 703 is shorter. Can be set as the imaging conditions of the imaging region.

  In addition, by applying the processing of FIGS. 22 and 23, while giving priority to the degree of influence (complexity of the image) on the coexistence area 703, if the difference in complexity is not superior, the coexistence area 703 If the imaging condition of the imaging region with the shorter boundary length is set, and the difference in the boundary length does not make a difference, the third imaging condition of the coexistence region 703 can be set by the halo countermeasure process (step S2202).

  Also, by applying the processing shown in FIG. 24, the coexistence area 703 corresponding to the outer edge of the main subject image is matched with the first imaging condition (short seconds) inside the background image, thereby reducing the effect of halo. Can be achieved. As described above, when priority is given to the degree of influence (complexity) on the coexistence area 703 and the difference in complexity is not superior, the third imaging condition of the coexistence area 703 is changed by the halo countermeasure processing (step S2202). Can be set.

  FIG. 27 is a flowchart illustrating a second example of the setting procedure of the imaging condition in the electronic device 500. FIG. 27 is an example of a processing procedure for setting an imaging condition when the setting of the degree of influence on the coexistence area 703 is OFF, unlike FIG. Therefore, in FIG. 27, the area priority setting processing (step S1808) and the complexity priority setting processing (step S1809) are not executed. Also, the same processes as those in FIG. 18 are denoted by the same step numbers, and description thereof will be omitted.

  In FIG. 27, when the detection flag of the camera shake or the subject blur is OFF, the setting process of the imaging condition based on the boundary length of the coexistence area 703 shown in FIG. 21, the boundary length of the coexistence area 703 shown in FIG. Is performed, and the process proceeds to a gain adjustment process (step S1810).

  In the initial setting (step S1801), when the setting of the boundary length of the coexistence area 703 is ON and the setting of the countermeasure against halo is OFF, FIG. 21 is applied to the processing of FIG. When the setting of the boundary length of the coexistence region 703 is ON and the setting of the halo countermeasure is ON, FIG. 22 is applied to the processing of FIG. However, for details of the halo countermeasure process (step S2202) in FIG. 22, FIG. 28 described later is applied. When the setting of the boundary length of the coexistence area 703 is OFF and the setting for countermeasures against halo is ON, FIG. 29 described later is applied to the processing of FIG.

  FIG. 28 is a flowchart illustrating a detailed processing procedure example 2 of the halo countermeasure processing (step S2202) illustrated in FIG. The same processes as those in FIG. 23 are denoted by the same step numbers, and description thereof will be omitted. Since the area priority setting processing (step S1808) and the complexity priority setting processing (step S1809) are not executed in the imaging condition setting processing in FIG. 27, step S2302 in FIG. 23 is unnecessary in FIG. Therefore, when it is predicted that no halo will occur (step S2301: No), the process proceeds to the gain adjustment process (step S1810).

  Thus, the influence of halo can be reduced by matching the coexistence area 703 corresponding to the outer edge of the main subject image with the first imaging condition (short seconds) inside the background image. As described above, when priority is given to the boundary length of the coexistence region 703 and the difference in the boundary length is not determined, the third imaging condition of the coexistence region 703 can be set by the halo countermeasure process (step S2202).

  FIG. 29 is a flowchart illustrating a detailed processing example of the halo countermeasure process. In FIG. 29, similarly to FIG. 28, the area priority setting processing (step S1808) and the complexity priority setting processing (step S1809) are not executed in the imaging condition setting processing of FIG. S2302 becomes unnecessary. Therefore, when it is predicted that no halo will occur (step S2301: No), the process proceeds to the gain adjustment process (step S1810).

  Thus, the influence of halo can be reduced by matching the coexistence area 703 corresponding to the outer edge of the main subject image with the first imaging condition (short seconds) inside the background image. As described above, the third imaging condition of the coexistence area 703 can be set by the measure against halo alone.

  As described above, according to the second embodiment, the third imaging condition of the third imaging region that is the boundary between the first subject image (background image BI) and the second subject image (main subject image MI) is set to The imaging conditions take into account the balance. Therefore, it is possible to display a subject image without a sense of discomfort.

  The present invention is not limited to the above-described contents, and may be arbitrarily combined. Further, other embodiments that can be considered within the scope of the technical idea of the present invention are also included in the scope of the present invention.

Reference Signs List 100 imaging device, 111 signal processing chip, 112 memory chip, 113 imaging chip, 200 imaging surface, 202 block, 500 electronic device, 501 imaging optical system, 502 control unit, 600 image file, 901 display unit, 902 recording unit, 911 Setting circuit, 912 image processing unit, 913 setting unit, 921 adjustment unit, 922 generation unit, 923 prediction unit, 924 detection unit

Claims (29)

An imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject;
A first imaging condition is set in a first imaging region where a first subject image is present among the plurality of imaging regions, and the first imaging condition is set in a second imaging region where a second subject image is present among the plurality of imaging regions. A second imaging condition different from the imaging condition is set, and a third imaging condition of a third imaging region where the first subject image and the second subject image are present among the plurality of imaging regions is set to the first imaging condition and the third imaging condition. A setting unit for setting based on at least one of the second imaging conditions;
Electronic equipment having The electronic device according to claim 1,
The electronic device, wherein the setting unit sets the third imaging condition of the third imaging region based on a luminance value based on an output from the third imaging region. The electronic device according to claim 1 or 2,
The electronic device, wherein the setting unit sets the third imaging condition of the third imaging region to one of the first imaging condition and the second imaging condition. The electronic device according to claim 2, wherein
The electronic device, wherein the setting unit sets the third imaging condition to the first imaging condition when a luminance value of the first subject image in the third imaging region is equal to or greater than a first luminance value. The electronic device according to claim 4,
The electronic device, wherein the first luminance value is a luminance value indicating a saturated state in which overexposure occurs in a first subject image in the third imaging region based on the first imaging condition. The electronic device according to claim 4, wherein:
The setting unit sets the third imaging condition to the first imaging condition when a luminance value of the second subject image in the third imaging region is not less than or equal to a predetermined second luminance value lower than the first luminance value. Set, electronic equipment. The electronic device according to claim 6,
The electronic device according to claim 1, wherein the second luminance value is a luminance value indicating a state in which the second subject image in the third imaging region is blackened based on the third imaging condition. The electronic device according to claim 4, wherein:
An electronic device, comprising: an adjustment unit configured to adjust so as to amplify a signal level of image data from an area where the second subject image exists in the third imaging area. The electronic device according to claim 4, wherein:
The setting unit sets the third imaging condition to the first imaging condition when a luminance value of the second subject image in the third imaging region is equal to or less than a predetermined second luminance value lower than the first luminance value. And an electronic device configured to be set based on the second imaging condition. The electronic device according to claim 9,
The electronic device, wherein the setting unit sets the third imaging condition as an imaging condition between the first imaging condition and the second imaging condition. The electronic device according to claim 10,
An electronic device, comprising: an adjustment unit configured to adjust a signal level of image data from the third imaging region to increase or decrease based on a third imaging condition set by the setting unit. The electronic device according to claim 11,
When the third imaging condition is closer to the first imaging condition than the second imaging condition, the adjustment unit is configured to output a signal of image data from a second existence region of the second subject image in the third imaging region. An electronic device that adjusts to amplify levels. The electronic device according to claim 11,
When the third imaging condition is closer to the second imaging condition than the first imaging condition, the adjustment unit is configured to output a signal of image data from a first existence region of the first subject image in the third imaging region. An electronic device that adjusts to reduce the level. The electronic device according to claim 2, wherein
The electronic device, wherein the setting unit sets the third imaging condition to the second imaging condition when a luminance value of the first subject image in the third imaging region is not equal to or greater than a first luminance value. The electronic device according to claim 14,
The electronic device, wherein the first luminance value is a luminance value indicating a saturated state in which the first subject image in the third imaging region is overexposed based on the first imaging condition. The electronic device according to claim 14, wherein:
The setting unit sets the third imaging condition to the second imaging condition when a luminance value of the second subject image in the third imaging region is equal to or less than a predetermined second luminance value lower than the first luminance value. Set to the electronic equipment. The electronic device according to claim 16,
The electronic device, wherein the second luminance value is a luminance value that indicates a state in which the second subject image in the third imaging region is underexposed based on the second imaging condition. The electronic device according to any one of claims 14 to 17, wherein
An electronic apparatus, comprising: an adjustment unit that adjusts the signal level of image data from the first existence area of the first subject image in the third imaging area so as to decrease the signal level. The electronic device according to claim 14, wherein:
The setting unit sets the third imaging condition to the first imaging condition and the first imaging condition when a luminance value of the second subject image in the third imaging region is not equal to or less than a predetermined second luminance value lower than the first luminance value. An electronic device that is set based on the second imaging condition. The electronic device according to claim 19,
The electronic device, wherein the setting unit sets the third imaging condition as an imaging condition of an imaging region in which a subject image of a main subject exists among the first imaging region and the second imaging region. The electronic device according to claim 20, wherein
An electronic device, comprising: an adjustment unit configured to adjust a signal level of image data from the third imaging region to increase or decrease based on a third imaging condition set by the setting unit. The electronic device according to claim 21,
When the third imaging condition is the first imaging condition, the adjustment unit adjusts to amplify a signal level of image data from a region where a main subject image exists in the third imaging region. . The electronic device according to claim 21,
When the third imaging condition is the second imaging condition, the adjustment unit adjusts the signal level of the image data from the non-existent region of the main subject image in the third imaging region to reduce the signal level. Electronics. The electronic device according to any one of claims 1 to 23,
Associating the first imaging region with image data from the first imaging region and the first imaging condition, associating the second imaging region with image data from the second imaging region and the second imaging condition, An electronic apparatus having a generation unit that generates an image file by associating the third imaging region with image data from the third imaging region and the third imaging condition, and records the generated image file in a recording unit . An imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject;
A first imaging condition is set in a first imaging region of the plurality of imaging regions, a second imaging condition is set in a second imaging region of the plurality of imaging regions, and the first imaging condition is set in the plurality of imaging regions. An electronic apparatus comprising: a setting unit configured to set an imaging condition between the first imaging condition and the second imaging condition in a third imaging region sandwiched between the imaging region and the second imaging region. An imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject;
Setting a first imaging condition in an imaging area in which a first subject image exists among the plurality of imaging areas;
In the plurality of imaging regions, an imaging region in which a first object image and a second object image different from the first object image are present differs from the first imaging condition, and the first object image and the second object are different. An electronic device comprising: a setting unit that sets a second imaging condition under which an image signal of an image is not saturated. An imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject;
Setting a first imaging condition in a first imaging region of the plurality of imaging regions;
Setting a second imaging condition different from the first imaging condition in a second imaging region of the plurality of imaging regions;
A pixel signal output according to the first imaging condition and a pixel output according to the second imaging condition in a third imaging region sandwiched between the first imaging region and the second imaging region among the plurality of imaging regions. A setting unit for setting a third imaging condition for outputting a pixel signal having a magnitude between the first and second signals. An imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject;
Setting a first imaging condition in a first imaging area where a first subject image is present among the plurality of imaging areas;
Setting a second imaging condition different from the first imaging condition in a second imaging region where a second subject image is present among the plurality of imaging regions;
A pixel signal output according to the first imaging condition and a pixel output according to the second imaging condition in a third imaging region sandwiched between the first imaging region and the second imaging region among the plurality of imaging regions. A setting unit for setting a third imaging condition for outputting a pixel signal having a magnitude between the first and second signals. A setting program for causing a processor to perform setting control of an image sensor capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject,
To the processor,
A first imaging condition is set in a first imaging region where a first subject image is present among the plurality of imaging regions,
A second imaging condition different from the first imaging condition is set in a second imaging region where a second subject image is present among the plurality of imaging regions,
A third imaging condition of a third imaging region where the first subject image and the second subject image are present among the plurality of imaging regions is set based on at least one of the first imaging condition and the second imaging condition. Let
Configuration program.