JP2024079153A - Image Processing Device - Google Patents

Abstract

To provide a technique capable of facilitating performing focus adjustment, without impeding other works such as exposure confirmation.SOLUTION: An image processing device includes acquisition means for acquiring an image captured by an imaging device, detection means for detecting a manual focus operation with respect to the imaging device, processing means for performing image processing for improving the gradation characteristic of a part of the gradation range of the image, with respect to the captured image, and control means for performing control so as to display the image based on the captured image on display means and for performing control so as to display the image after the image processing on the display means, when the manual focus operation is detected.SELECTED DRAWING: Figure 5

Description

The present invention relates to an image processing device.

An increasing number of cameras are capable of capturing images of a subject and obtaining images with a wide dynamic range (HDR images) known as High Dynamic Range (HDR). Typical HDR image formats include Perceptual Quantization (PQ) and Hybrid Log Gamma (HLG). To display HDR images, a display (monitor) with high peak luminance (upper limit of display luminance) is required. In the case of a display with low peak luminance, such as a small display attached to the camera body, the image signal (dynamic range) of the HDR image is compressed, and a low-contrast image is displayed on the display.

To increase the compatibility between digital and film, an increasing number of cameras are capable of generating images (Log images) using a conversion characteristic called Log gamma (a conversion characteristic in which the output value increases logarithmically with respect to an increase in the input value). Cineon Log is known as a representative Log gamma. In Log images, a wide dynamic range corresponds to a narrow tonal range, and unless grading processing is performed, Log images are images with low contrast.

When displaying an HDR image, image processing is performed so that the output luminance of the display increases linearly with an increase in the input luminance of the camera. The same is true when displaying a Log image. This type of image processing is disclosed in Patent Document 1. Patent Document 1 also discloses image processing that allocates more luminance to a portion of the dynamic range before image processing.

JP 2015-29225 A

As mentioned above, HDR and Log images may be displayed with low contrast. Therefore, in a shooting mode in which an HDR image is recorded as the captured image (HDR shooting mode), the contrast of the displayed live view (LV) image is low, making it difficult for the user (cameraman) to adjust the focus (focus). The same is true in a shooting mode in which a Log image is recorded as the captured image (Log shooting mode).

By performing image processing that allocates more brightness to a portion of the dynamic range before image processing (image processing described in Patent Document 1), the user can view a high-contrast LV image and easily adjust the focus.

However, when more brightness is allocated to a portion of the dynamic range before image processing, less brightness is allocated to the remaining portion of the dynamic range. As a result, the LV image becomes distorted (e.g., whiteout and blackout) in the area corresponding to this remaining portion, preventing the user from checking the exposure in detail.

An object of the present invention is to provide a technique for making it easier to adjust focus without interfering with other tasks such as checking exposure.

The first aspect of the present invention is an image processing device characterized by having an acquisition means for acquiring an image captured by an imaging device, a detection means for detecting a manual focus operation on the imaging device, a processing means for performing image processing on the captured image to enhance the gradation of a part of the gradation range of the image, and a control means for controlling a display means to display an image based on the captured image, and a control means for controlling the display means to display the image after the image processing when the manual focus operation is detected.

A second aspect of the present invention is an image processing method comprising the steps of: acquiring an image captured by an imaging device; detecting a manual focus operation on the imaging device; performing image processing on the captured image to enhance the gradation of a portion of the gradation range; and controlling a display means to display an image based on the captured image, wherein when the manual focus operation is detected, the image after the image processing is controlled to be displayed on the display means.

A third aspect of the present invention is a program for causing each of the means of the image processing device to function. A fourth aspect of the present invention is a computer-readable storage medium that stores a program for causing a computer to function as each of the means of the image processing device.

The present invention makes it easier to adjust focus without interfering with other tasks such as checking exposure.

FIG. 1 is a block diagram showing an example of a configuration of an imaging device. FIG. 1 is a diagram illustrating an example of dynamic range compression of an HDR image. FIG. 4 is a diagram illustrating an example of input/output characteristics of an image. FIG. 13 is a diagram showing an example of an image display. 4 is a flowchart showing a process according to the first embodiment. 10 is a flowchart showing a process according to a second embodiment.

Below, an embodiment of the present invention will be described. Note that, although an example in which the present invention is applied to an imaging device will be described, devices to which the present invention can be applied are not limited to imaging devices. For example, the present invention can also be applied to information processing devices such as smartphones, tablets, and personal computers (PCs) connected to an imaging device (capable of communicating with the imaging device). The present invention can also be applied to a display (monitor, display device) connected to an imaging device. The present invention can also be applied to devices separate from the imaging device, and can be applied to various image processing devices.

Fig. 1 is a block diagram showing the configuration of an imaging device 100 according to this embodiment. The imaging device 100 is an electronic device with a camera function (photographing function), such as a camera such as a digital camera or a digital video camera. The imaging device 100 may also be a smartphone, tablet, or PC equipped with a camera.

The optical system 101 is an imaging optical system equipped with a lens group, a shutter, and an aperture. The lens group includes a correction lens that performs various corrections such as image stabilization, and a focus lens. The optical system 101 forms an image of light from a subject (subject light) on the imaging unit 102 (the image sensor of the imaging unit 102) in accordance with a control signal from a central processing unit (CPU) 104.

The imaging unit 102 has imaging elements such as a Charge Coupled Device (CCD) image sensor and a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor. The imaging elements of the imaging unit 102 convert the light that has passed through the optical system 101 and formed an image into an image signal by photoelectric conversion. The imaging unit 102 also has an analog-to-digital (A/D) converter. The A/D converter of the imaging unit 102 converts the image signal (analog signal) obtained by photoelectric conversion of the imaging element into image data (digital signal) by A/D conversion.

The CPU 103 controls each part of the imaging device 100 by expanding the programs stored in the secondary memory unit 105 into the primary memory unit 104 and executing them.

The primary storage unit 104 is a volatile storage unit, such as a Random Access Memory (RAM), and temporarily stores various data. The primary storage unit 104 is used as a work memory for the CPU 103. The data (information) stored in the primary storage unit 104 is used by the image processing unit 109 or stored in the storage medium 106.

The secondary storage unit 105 is a non-volatile storage unit such as an Electrically Erasable Programmable Read-Only Memory (EEPROM). The secondary storage unit 105 stores a program (firmware) for controlling the imaging device 100 and various setting information, which are used by the CPU 103.

The storage medium 106 stores various data. For example, the storage medium 106 stores captured images (captured image data) stored in the primary storage unit 104. The captured images are images captured by the imaging unit 102 and subjected to various image processing. The storage medium 106 may be a storage medium built into the imaging device 100, or may be a storage medium that is detachable from the imaging device 100, such as a semiconductor memory card. The data recorded in the storage medium 106 can be read by an external device such as a PC (personal computer) by attaching the storage medium 106 to the external device.

The display unit 107 displays various images. For example, the display unit 107 displays a live view (LV) image captured by the imaging unit 102. An LV image is an image that represents a subject in almost real time. The display unit 107 may also display captured images stored in the storage medium 106, or GUI (Graphical User Interface) images for interactive operation.

The operation unit 108 is a group of input devices that accepts operations by a user (user operation) and outputs an operation signal corresponding to the operation to the CPU 103, and includes, for example, a button, a lever, and a touch panel. The operation unit 108 may be capable of accepting user operations using voice and user operations using line of sight. Some of the input devices in the group of input devices may be external devices. In that case, the operation unit 108 receives an operation signal output from the external device in response to a user operation on the external device, and outputs the operation signal to the CPU 103. One of multiple shooting modes is set in the imaging device 100. For example, the user can select one of multiple shooting modes by user operation, and the CPU 103 sets the shooting mode selected by the user.

The image processing unit 109 acquires an image captured by the imaging unit 102, and performs various image processing, including development processing, on the image. The image processing unit 109 performs image processing according to the shooting mode, for example, luminance conversion processing and color conversion processing according to the shooting mode. The image processing performed on the image captured by the imaging unit 102 may be realized by hardware or software.

First Embodiment
A first embodiment of the present invention will be described. In the first embodiment, a shooting mode (High Dynamic Range (HDR) shooting mode) for recording an HDR image as a shooting image can be set, or a shooting mode (Log shooting mode) for recording a Log image can be set. The HDR image is an image having a wide dynamic range called HDR, for example, an image in the Perceptual Quantization (PQ) method or the Hybrid Log Gamma (HLG) method. The Log image is an image generated using a conversion characteristic called Log gamma (a conversion characteristic in which the output value increases logarithmically with respect to an increase in the input value), for example, an image generated using Cineon Log.

When the dynamic range of the display unit 107 is narrower than the dynamic range of the HDR image, in conventional technology, the dynamic range of the HDR image is compressed and a low-contrast image is displayed on the display unit 107, as shown in Figure 2.

In Log images, a wide dynamic range corresponds to a narrow tonal range, and without grading processing, Log images are low-contrast images.

In this way, HDR images and Log images may be displayed with low contrast. Therefore, in conventional HDR and Log shooting modes, the contrast of the displayed LV image is low, and the user (photographer) cannot easily adjust the focus (align the focus).

Here, it is assumed that the image processing unit 109 performs image processing on the captured image to increase the contrast (gradation) of a portion of the dynamic range (tone range) of the image. For example, the image processing unit 109 uses brightness conversion processing (gamma conversion processing) to convert an image with the input/output characteristics shown in FIG. 3(A) (linear characteristics in which the output luminance of the display unit 107 increases linearly with an increase in the input luminance of the imaging unit 102) into an image with the input/output characteristics shown in FIG. 3(B). Similarly, the image processing unit 109 also performs color conversion processing (color gamut conversion processing). Image processing (image processing group) that includes at least one of brightness conversion processing and color conversion processing is referred to as assist processing, and display of the image after assist processing is referred to as assist display.

The image processing unit 109 may perform the luminance conversion process and the color conversion process separately, or may perform one image process including the luminance conversion process and the color conversion process. These image processes may be arithmetic processes using a function that indicates the correspondence between input values and output values, or may be processes using a Look Up Table (LUT) that indicates the correspondence between input values and output values.

In addition, the input/output characteristics of the image before the assist process (before the image processing) are not limited to the linear characteristics as shown in FIG. 3A. The input/output characteristics of the image after the assist process (after the image processing) may or may not be fixed characteristics. Since the user rarely adjusts the focus in the bright or dark areas, the input/output characteristics in FIG. 3B are set to enhance the contrast of the intermediate brightness part. The CPU 103 may determine the input/output characteristics of the image after the assist process based on the image before the assist process. For example, the CPU 103 determines the input/output characteristics in which the brighter the shooting scene is, the more brightness (after conversion) is assigned to the part with high brightness (before conversion), and the darker the shooting scene is, the more brightness (after conversion) is assigned to the part with low brightness (before conversion). When a part of the image before the assist process is displayed on the display unit 107, the CPU 103 may determine the input/output characteristics of the image after the assist process based on that part.

When performing an assist display, the CPU 103 may switch the image displayed on the display unit 107 from the image before the assist processing to the image after the assist processing, or may maintain the display of the image before the assist processing while also displaying the image after the assist processing.

With the assist display, the HDR image (or Log image) that was displayed with low contrast as in Figure 4 (A) is now displayed with high contrast as in Figure 4 (B), making it easier to adjust the focus.

However, while assisted display increases the contrast (gradation) of part of the dynamic range (tone range) of the image, it reduces the contrast (gradation) of the remaining part. As a result, in the area corresponding to this remaining part, the LV image becomes distorted (e.g., whiteout and blackout), and the user is unable to check the exposure in detail.

Therefore, in the first embodiment, the assist display is automatically displayed only while the user is performing focus adjustment. This makes it easier to perform focus adjustment without interfering with other tasks such as checking exposure.

Figure 5 is a flowchart showing the processing of the imaging device 100. The processing of Figure 5 is performed repeatedly in the shooting standby state.

In step S501, the CPU 103 determines whether the HDR shooting mode or the Log shooting mode is set. If the CPU 103 determines that the HDR shooting mode or the Log shooting mode is set, the process proceeds to step S502, and if the CPU 103 determines that another shooting mode is set, the process in FIG. 5 ends.

In step S502, the CPU 103 determines whether the assist display setting is disabled. If the CPU 103 determines that the assist display setting is disabled, the process proceeds to step S503, and if the CPU 103 determines that the assist display setting is enabled, the process of FIG. 5 ends. The assist display setting is a setting for whether or not the image after assist processing is displayed. The assist display setting of "disabled" is a setting that does not display the image after assist processing, and the assist display setting of "enabled" is a setting that displays the image after assist processing. In step S502, if the assist display is disabled, the image before assist processing is displayed on the display unit 107, and if the assist display is enabled, the image after assist processing is displayed on the display unit 107.

In step S503, the CPU 103 determines whether or not the user has performed a focus operation (manual focus operation) on the imaging device 100. A focus operation is an operation for adjusting focus, such as a user operation such as turning a focus ring or pressing a focus button in manual focus mode. If the CPU 103 determines that a focus operation has been performed (if a focus operation has been detected), the process proceeds to step S504, and if it determines that a focus operation has not been performed, the process of FIG. 5 ends.

In step S504, the CPU 103 controls the image processing unit 109 to perform assist processing, and controls the display unit 107 to perform assist display. As a result, the image after the assist processing is displayed on the display unit 107. For example, the CPU 103 switches the image displayed on the display unit 107 from the image before the assist processing to the image after the assist processing. In the first embodiment, the process of step S504 is performed when the HDR shooting mode or the Log shooting mode is set (YES in step S501, when the captured image is a predetermined type of image (HDR image or Log image)). The process of step S504 is not performed when another shooting mode is set (NO in step S501, when the captured image is not a predetermined type of image).

In step S505, the CPU 103 determines whether a predetermined time (e.g., 2 seconds) has elapsed since the last focus operation (the control in step S504 was performed). The CPU 103 waits for the predetermined time to elapse, and when the predetermined time has elapsed, the process proceeds to step S506.

In step S506, the CPU 103 stops (cancels) the assist processing of the image processing unit 109 and stops (cancels) the assist display on the display unit 107. This makes the image after the assist processing invisible. For example, the CPU 103 returns the image displayed on the display unit 107 from the image after the assist processing to the image before the assist processing.

As described above, according to the first embodiment, the assist display is automatically displayed during the period when the user is performing focus adjustment, allowing the user to easily perform focus adjustment. Furthermore, the assist display is not displayed during the period when the user is not performing focus adjustment, allowing the user to easily perform other tasks such as checking the exposure.

Second Embodiment
A second embodiment of the present invention will be described. In the second embodiment, while the user is performing focus adjustment, not only the assist display but also the enlarged display by digital zoom is automatically performed. This makes it easier to perform focus adjustment.

Figure 6 is a flowchart showing the processing of the imaging device 100 according to the second embodiment. The processing of Figure 6 is performed repeatedly in the shooting standby state.

Steps S601 to S603 are the same as those in the first embodiment (steps S501 to S503 in FIG. 5). If it is determined in step S603 that a focus operation has been performed, the process proceeds to step S604, and if it is determined that a focus operation has not been performed, the process in FIG. 6 ends.

In step S604, the CPU 103 controls the image processing unit 109 to enlarge the image displayed on the display unit 107 by digital zoom (enlarged display). If the subject is displayed small, it is difficult to check whether the focus is correct. For example, if the display unit 107 is a small display, the subject is often displayed small, and it is often difficult to check whether the focus is correct. By enlarging the display, it becomes easier to adjust the focus.

In step S605, similar to step S504, the CPU 103 controls the image processing unit 109 to perform assist processing, and controls the display unit 107 to perform assist display. In the second embodiment, the CPU 103 switches the image displayed on the display unit 107 from the image before the assist processing to the image after the assist processing. As a result, the image after the assist processing is enlarged and displayed on the display unit 107 (enlarged assist display).

Note that the process of step S605 may be performed before step S604. Furthermore, the CPU 103 may display the image before the assist process on the display unit 107 at a normal magnification (1x, life size), and may also enlarge and display the image after the assist process on the display unit 107.

In step S606, similar to step S504, the CPU 103 determines whether a predetermined time (e.g., 2 seconds) has elapsed since the last time the focus operation was performed (the time when the enlarged assist display was performed). The CPU 103 waits for the predetermined time to elapse, and when the predetermined time has elapsed, the process proceeds to step S607.

In step S607, the CPU 103 stops (cancels) the digital zoom processing and assist processing of the image processing unit 109, and controls the display unit 107 to stop (cancel) the enlarged assist display. In the second embodiment, the CPU 103 returns the image displayed on the display unit 107 from the enlarged image (enlarged image) after the assist processing to the normal magnification image (image at normal magnification) before the assist processing. This makes the image after the assist processing non-display.

As described above, according to the second embodiment, during the period when the user is performing focus adjustment, not only the assist display but also the enlarged display using digital zoom is automatically performed, allowing the user to perform focus adjustment more easily. Furthermore, during the period when the user is not performing focus adjustment, neither the assist display nor the enlarged display is performed, allowing the user to easily perform other tasks such as checking the exposure.

The CPU 103 displays the assist display when zooming for focus adjustment (when zooming in response to a focus operation), but does not display the assist display when zooming other than for focus adjustment. For example, zooming in response to a zoom operation is performed not for focus adjustment but for recording an enlarged image. Therefore, the CPU 103 does not display the assist display when zooming in response to a zoom operation. Even when zooming in response to a zoom operation, if a focus operation is performed, the CPU 103 displays the assist display.

The above embodiment (including modified examples) is merely an example, and the present invention also includes configurations obtained by appropriately modifying or changing the configuration of the above embodiment within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately combining the configurations of the above embodiments. For example, the above embodiment may be combined with image processing (focus peaking) that emphasizes the contours of areas in focus.

<Other embodiments>
The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

The disclosure of the present embodiment includes the following configuration, method, program, and medium.
(Configuration 1)
An acquisition means for acquiring an image captured by an imaging device;
a detection unit that detects a manual focus operation on the imaging device;
a processing means for performing image processing on the captured image to enhance a gradation of a part of the gradation range of the image;
and a control means for controlling a display means to display an image based on the captured image, and when the manual focus operation is detected, a control means for controlling the display means to display the image after image processing.
(Configuration 2)
2. The image processing apparatus according to claim 1, further comprising the display means.
(Configuration 3)
3. The image processing device according to configuration 1 or 2, wherein the image processing device is provided in the imaging device.
(Configuration 4)
The control means
If the captured image is not a predetermined type of image, even if the manual focus operation is detected, the image after the image processing is not displayed on the display means;
The image processing device according to any one of configurations 1 to 3, characterized in that when the captured image is an image of the predetermined type and the manual focus operation is detected, the image after the image processing is controlled to be displayed on the display means.
(Configuration 5)
5. The image processing device according to configuration 4, wherein the predetermined type of image is an HDR image or a Log image.
(Configuration 6)
6. The image processing device according to any one of configurations 1 to 5, wherein the image processing includes at least one of a luminance conversion process and a color conversion process.
(Configuration 7)
7. The image processing device according to any one of configurations 1 to 6, further comprising a setting means for setting whether or not the image after the image processing is to be displayed.
(Configuration 8)
The image processing device according to configuration 7, wherein the control means controls the display means to display the image after the image processing when the manual focus operation is detected in a state in which the image after the image processing is set not to be displayed.
(Configuration 9)
The image processing device according to any one of configurations 1 to 8, wherein the control means controls the display means to enlarge the image after the image processing by digital zoom when the manual focus operation is detected.
(Configuration 10)
The image processing device according to any one of configurations 1 to 9, wherein the control means controls the display means to display the image after the image processing, and then controls the display means to hide the image after the image processing after a predetermined time has elapsed.
(Method)
acquiring an image captured by an imaging device;
detecting a manual focus operation on the imaging device;
performing image processing on the captured image to enhance the gradation of a portion of a gradation range;
and a step of controlling so that an image based on the captured image is displayed on a display means, and when the manual focus operation is detected, a step of controlling so that the image after image processing is displayed on the display means.
(program)
A program for causing a computer to function as each of the means of the image processing device according to any one of configurations 1 to 10.
(Medium)
A computer-readable storage medium storing a program for causing a computer to function as each of the means of the image processing device according to any one of configurations 1 to 10.

100: Imaging device 102: Imaging section 103: CPU
107: Display unit 108: Operation unit 109: Image processing unit

Claims (13)

An acquisition means for acquiring an image captured by an imaging device;
a detection unit that detects a manual focus operation on the imaging device;
a processing means for performing image processing on the captured image to enhance a gradation of a part of the gradation range of the image;
and a control means for controlling a display means to display an image based on the captured image, and when the manual focus operation is detected, a control means for controlling the display means to display the image after image processing. 2. The image processing apparatus according to claim 1, further comprising the display means. The image processing device according to claim 1 , wherein the image processing device is provided in the imaging device. The control means
If the captured image is not a predetermined type of image, even if the manual focus operation is detected, the image after the image processing is not displayed on the display means;
2. The image processing apparatus according to claim 1, wherein when the captured image is an image of the predetermined type, if the manual focus operation is detected, the image after the image processing is displayed on the display means. The image processing device according to claim 4 , wherein the predetermined type of image is an HDR image or a Log image. 2. The image processing apparatus according to claim 1, wherein the image processing includes at least one of a luminance conversion process and a color conversion process. 2. The image processing apparatus according to claim 1, further comprising a setting unit for setting whether or not the image after the image processing is to be displayed. 8. The image processing device according to claim 7, wherein the control means controls the display means to display the image after the image processing when the manual focus operation is detected in a state in which the image after the image processing is set not to be displayed. 2 . The image processing apparatus according to claim 1 , wherein the control means performs control so that, when the manual focus operation is detected, the processed image is enlarged by digital zoom and displayed on the display means. 2. The image processing device according to claim 1, wherein the control means controls the display means to display the image after the image processing, and then controls the display means to stop displaying the image after the image processing when a predetermined time has elapsed. acquiring an image captured by an imaging device;
detecting a manual focus operation on the imaging device;
performing image processing on the captured image to enhance the gradation of a portion of a gradation range;
and a step of controlling so that an image based on the captured image is displayed on a display means, and when the manual focus operation is detected, a step of controlling so that the image after image processing is displayed on the display means. A program for causing a computer to function as each of the means of the image processing device according to any one of claims 1 to 10. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the image processing device according to any one of claims 1 to 10.

Images