JP2018207326A - Camera and method for taking moving image - Google Patents

Abstract

To provide a camera capable of nestling a subject of interest in a frame.SOLUTION: A viewfinder 120 displays an image corresponding to a viewfinder region positioned substantially in the center of an imaging element 110 in a followup mode. An image processing part 130 generates an image to be stored, by extracting a crop region including a target subject out of temporary images outputted from the imaging element 110, in the followup mode.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a camera.

  When shooting a specific subject with a video camera or still camera, it is necessary to keep the subject within the frame while looking at the viewfinder. However, when shooting a moving subject, the subject may protrude outside the frame. Alternatively, even when the subject is stationary, the subject may protrude out of the frame due to slight camera shake when photographing with a telephoto lens.

  In situations where it is difficult to re-shoot, such as a children's athletic meet or a presentation, if the subject moves out of the frame, it cannot be recovered.

  The present invention has been made in view of such a problem, and one of exemplary purposes of an aspect thereof is to provide a camera capable of placing a subject of interest in a frame.

  One embodiment of the present invention relates to a camera. The camera includes an image sensor, a finder that displays an image in a range narrower than an angle of view corresponding to the effective imaging area of the image sensor in the tracking mode, and a target subject among raw images corresponding to the effective imaging area in the tracking mode. And an image processing unit that generates a crop image to be extracted and stored.

  In this specification, a finder refers to a device for confirming an image captured by a user during shooting, and includes a liquid crystal panel in addition to an electronic finder and an optical finder. According to this aspect, even when the target subject is out of the viewfinder, as long as it falls within the effective imaging area, it can be recorded automatically following the target subject inside the camera, preventing shooting failures. it can. Since the finder displays an image of the finder area with a narrow angle of view instead of the entire image sensor, the user must pay attention to fit the target subject in the finder with a narrow angle of view. The possibility of being accommodated in the image sensor can be increased.

  The angle of view of the viewfinder area and the angle of view of the crop area may be substantially equal. In this case, when the target subject is included in the finder area, an image that is visible in the finder can be recorded as in a normal camera.

  The image processing unit may set the subject designated by the user as the target subject immediately before the start of shooting or after the start of shooting.

  The image processing unit may automatically determine the subject included in the predetermined portion as the target subject immediately before the start of shooting or during a predetermined period after the start of shooting.

  The camera may be able to select the normal mode in addition to the follow mode. In the normal mode, an image displayed on the viewfinder may be recorded.

  The viewfinder may be an electronic viewfinder. In the normal mode, the finder may display an image of the effective imaging area, and the image processing unit may store the image of the effective imaging area.

  In the normal mode, the finder may display an image in the finder area displayed by the finder in the follow-up mode, and the image processing unit may store the image in the finder area.

  The image in the effective imaging area may be discarded. Thereby, the capacity of the recording medium for storing data can be reduced. Alternatively, the recording time or number of recordings that can be recorded on a recording medium having a predetermined capacity can be increased.

  Note that any combination of the above-described constituent elements and the constituent elements and expressions of the present invention replaced with each other among methods, apparatuses, systems, and the like are also effective as an aspect of the present invention. Further, the description of this item (means for solving the problem) does not explain all the essential features of the present invention, and therefore a sub-combination of these described features can also be the present invention. .

  According to an aspect of the present invention, a camera that can fit a subject in a frame can be provided.

It is a figure which shows the camera which concerns on embodiment. It is a figure explaining operation | movement of the camera of FIG. It is a figure explaining an example of the setting of a target subject. It is a figure explaining another example of the setting of a target subject. It is a figure explaining another example of the setting of a target subject. It is a figure which shows the image process part which concerns on one Example. It is a figure which shows the image process part which concerns on one Example. It is a flowchart of the image processing of the image processing part of FIG.

  The present invention will be described below based on preferred embodiments with reference to the drawings. The same or equivalent components, members, and processes shown in the drawings are denoted by the same reference numerals, and repeated descriptions are omitted as appropriate. The embodiments do not limit the invention but are exemplifications, and all features and combinations thereof described in the embodiments are not necessarily essential to the invention.

  The vertical and horizontal axes of the waveform diagrams and time charts referred to in this specification are enlarged or reduced as appropriate for easy understanding, and each waveform shown is also simplified for easy understanding. Or exaggerated or emphasized.

  FIG. 1 is a diagram illustrating a camera according to an embodiment. In the present embodiment, the camera is a video camera that captures moving images. The camera 100 includes an image sensor 110, a viewfinder 120, an image processing unit 130, and a recording medium 140.

  The imaging element 110 has an effective imaging area having a resolution of, for example, 4K, that is, about 4000 pixels in the horizontal direction and 2000 pixels in the vertical direction (specifically, 4096 × 2160 or 3840 × 2160). The camera 100 supports a shooting mode (referred to as a follow-up mode) different from that of a normal video camera.

  The finder 120 displays an image in a range narrower than the angle of view corresponding to the effective image pickup area of the image sensor 110 in the follow-up mode. The finder 120 is a liquid crystal panel or an electronic view finder, and may display an image corresponding to the finder region 202 located substantially in the center of the image sensor 110.

  In the follow-up mode, the image processing unit 130 extracts a cropped region including a target subject from a 4K resolution raw image (raw image) output from the image sensor 110, and extracts a cropped image (hereinafter, cropped image). A moving image obtained by compression is recorded on the recording medium 140. The recording medium 140 may be built in the camera 100 or may be removable like an SD card.

  The above is the configuration of the camera 100. Next, the operation will be described. FIG. 2 is a diagram for explaining the operation of the camera 100 of FIG. In the upper left of FIG. 2, a raw image 200 of the effective imaging area before cropping output from the imaging device 110 is shown. A finder area 202 is fixedly set at the center of the raw image 200. As shown in the lower left of FIG. 2, an image 210 of the finder area 202 is displayed on the finder 120. The image processing unit 130 moves the crop area 206 so as to include the target subject 204 while tracking the target subject 204. The cropped image 220 in the crop area 206 is compressed into a moving image and recorded on the recording medium 140.

  The above is the operation of the camera 100. According to the camera 100, even when the target subject is out of the viewfinder 120, as long as it falls within the effective imaging area of the image sensor 110, the camera 100 automatically tracks and records the target subject. Can prevent shooting failures.

  In addition, since the image of the finder area with a narrower angle of view is displayed on the finder 120 instead of the entire image sensor 110, the user is careful to place the target subject in the finder 120 with a narrowed angle of view. Therefore, it is possible to increase the possibility that the target subject will fit in the image sensor 110.

  As a comparative technique, it is assumed that the same processing is performed while displaying an image having an angle of view corresponding to the entire image sensor 110 on the finder 120. In this comparison technique, the user pays attention to place the target subject in the viewfinder 120, that is, the image sensor 110. However, if the target subject protrudes from the viewfinder 120, the target subject cannot be relieved because it protrudes from the image sensor 110. In this respect, the camera 100 according to the embodiment is easier to fit the subject in the frame than the comparative technique.

  It is preferable to discard the image in the effective imaging area. Thereby, the capacity of the recording medium for storing data can be reduced. Alternatively, the recording time or the number of recordings that can be recorded on a recording medium having a predetermined capacity can be increased. If the capacity of the recording medium is sufficiently large and the hardware processing speed is sufficient, a moving image can be generated based on the raw image corresponding to the effective imaging area and recorded together with the moving image based on the cropped image. Good. Whether or not to record a moving image based on the raw image may be set by the user.

  It is preferable that the angle of view of the finder area 202 and the angle of view of the crop area 206 are substantially equal. In the present embodiment, it is preferable that the size of the finder area 202 and the size of the crop area 206 are substantially equal, and may be, for example, 2K, that is, a resolution of 1920 × 1080 pixels (Full-HD). As a result, when the finder area 202 and the crop area 206 match, the image displayed in the finder area 202 can be matched with the recorded image (view angle).

  In the embodiment, the case where the sizes of the finder area 202 and the crop area 206 coincide with each other has been described, but one may be slightly larger than the other.

  Although autofocus is not particularly described here, basically, it is sufficient to keep focusing on the target subject 204.

  FIG. 3 is a diagram illustrating an example of setting a target subject. 3 is a touch panel. When one point of the finder 120 is touched with the intention of setting the target subject before or after the start of shooting, the subject moved to the touched point is set as the target subject.

  For example, if “target subject setting” is selected from a menu screen (not shown), the target subject can be set by touch. At this time, a message 122 that prompts a touch may be displayed on the finder 120.

  Alternatively, a touch input intended to set a target subject may be prepared. For example, touching one point of the finder 120 twice in succession may be a touch input intended for target setting. Alternatively, touching and holding one point of the finder 120 for a long time may be a touch input intended for target setting.

  FIG. 4 is a diagram illustrating another example of setting a target subject. The image processing unit 130 recognizes the shape of a person or an object displayed on the viewfinder 120 and displays at least one marker 124 indicating a target subject candidate. When a plurality of markers 124 are displayed, one of them is highlighted, and the user operates the hardware key 150 to move the highlighted marker 124 to select a target subject.

  FIG. 5 is a diagram illustrating still another example of setting a target subject. The image processing unit 130 may automatically determine a subject included in a predetermined portion (preferably a central portion) of the finder 120 as a target subject immediately before the start of shooting or during a predetermined period after the start of shooting.

  The camera 100 can select the normal mode in addition to the follow mode. In the normal mode, the finder 120 displays the entire 4K image output from the image sensor 110, and the image processing unit 130 compresses the entire 4K image output from the image sensor 110 and stores it in the recording medium 140. . Note that when compressing a moving image, the resolution may be reduced to 2K and stored.

FIG. 6 is a diagram illustrating the image processing unit 130A according to an embodiment. The image processing unit 130 </ b> A is hardware that supports the resolution (4K) of the image sensor 110. The image processing unit 130A receives the data _{S 1} of the raw image 200 generated at a frame rate of the imaging device 110 (e.g. 60 Hz), is stored in a memory (frame buffer) 132. The finder display unit 134 extracts the finder image 210 in the finder area 202 from the raw image data S ₁ and displays it on the finder 120. Tracking processing unit 136, a target object 204 extracts from the data _{S 1,} and set the crop region 206 to include it, every frame the crop image 220 of crop region 206, generates. The encoder 138 compresses the sequence of the cropped image 220 to generate moving image data S ₂ and writes it to the recording medium 140. The moving image compression method is not particularly limited, and the frame rate of the compressed moving image may be different from the frame rate of the image sensor 110.

FIG. 7 is a diagram illustrating the image processing unit 130B according to an embodiment. The image processing unit 130B is configured based on an architecture that does not support the resolution (4K) of the image sensor 110 but supports a smaller resolution (2K). The image processing unit 130B receives the data _{S 1} of the raw image 200 generated at a frame rate of the imaging device 110 (e.g. 60 Hz), is stored in the memory 132. The memory 132 supports 4K resolution.

  In this embodiment, the finder display unit 134, the tracking processing unit 136, and the encoder 138 perform processing by dividing the 4K raw image 200 into subframes 201 corresponding to 2K. Therefore, it is desirable that the finder display unit 134, the tracking processing unit 136, and the encoder 138 operate at a speed four times the frame rate.

  The finder display unit 134 sequentially refers to the four subframes 201_1 to 201_4, extracts images that overlap with the finder region 202 from each of the four subframes 201_1 to 201_4, and combines them to finder the finder. An image 210 is generated.

The tracking processing unit 136 extracts the target subject 204 by sequentially referring to the four subframes 201_1 to 201_4. Then set the subframe included target subject 204 in the crop region 206 to generate each frame image data S ₂ indicating the crop image 220 of crop region 206.

If the target subject 204 exists over a plurality of subframes, the crop area 206 also extends over a plurality of subframes. In this case, an image that overlaps the crop area 206 is extracted from each of the four subframes 201_1 to 201_4, and the crop image 220 may be generated by combining them. Encoder 138, a sequence of crop image 220 generated for each frame in the moving picture compression generates moving image data S _2, writing to the recording medium 140. There is no particular limitation on the video compression method.

  FIG. 8 is a flowchart of image processing of the image processing unit 130B of FIG. Note that this flowchart does not limit the order of the processes, and the processes can be interchanged as long as the processes are not hindered.

  When shooting starts, the raw image 200 is output from the image sensor 110 and written in the memory 132 (S100). The variable i is initialized (S102), and the target object is detected, the crop area is detected, and the finder area is extracted for the i-th subframe (S104). If the variable i has not reached the total number N of subframes (N in S106), the variable i is incremented (S108), and the process returns to S104. When the process S108 is completed for all subframes (Y in S106), a finder image is generated (S110) and a crop image is generated (S112). The cropped image is then encoded. When the processing for one frame is completed, the process returns to step S100, and the processing for the next frame is started.

  The present invention has been described based on the embodiments. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. is there. Hereinafter, such modifications will be described.

(Modification 1)
In the embodiment, the video camera that mainly captures moving images has been described. This video camera may have a still image shooting function, and may support a follow-up mode when shooting a still image.

(Modification 2)
The application range of the present invention is not limited to a video camera, and may be a still camera that mainly captures still images. Or a smart phone and a tablet terminal provided with an imaging | photography function may be sufficient.

(Modification 3)
In the image processing unit 130B of FIG. 7, the raw image 200 does not necessarily need to be written in the memory, and the image sensor 110 may be configured to output every subframe at a quadruple frame rate.

(Modification 4)
In the embodiment, the raw image is 4K and the cropped image is 2K, but the combination of resolutions is not limited thereto. For example, an intermediate resolution such as 4K for a raw image and 2560 × 1440 for a cropped image may be used. The raw image may be 8K and the cropped image may be 4K or 2560 × 1440, or the raw image may be 2K and the cropped image may be half HD or lower resolution.

(Modification 5)
In the embodiment, in the normal mode, an image corresponding to the finder area may be displayed on the finder 120, and the image processing unit 130 may set the image corresponding to the finder area as an image to be saved.

(Modification 6)
In the embodiment, a camera including an electronic viewfinder has been described. However, the present invention is not limited thereto, and an optical viewfinder may be used.

  Although the present invention has been described using specific terms based on the embodiments, the embodiments only illustrate the principles and applications of the present invention, and the embodiments are defined in the claims. Many variations and modifications of the arrangement are permitted without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 100 ... Camera, 110 ... Image sensor, 120 ... Finder, 130 ... Image processing part, 132 ... Memory, 134 ... Finder display part, 136 ... Tracking processing part, 138 ... Encoder, 140 ... Recording medium, 150 ... Hardware key, 200 ... Raw image, 202 ... Viewfinder area, 204 ... Target subject, 206 ... Crop area.

Claims (10)

An image sensor;
In the follow-up mode, a finder that displays an image in a range narrower than the angle of view corresponding to the effective imaging area of the imaging device;
In the follow-up mode, an image processing unit that extracts a crop area including a target subject from a raw image corresponding to the effective imaging area of the image sensor and generates a crop image to be stored;
A camera comprising:   The camera according to claim 1, wherein an angle of view of the viewfinder and an angle of view of the crop area are substantially equal.   3. The camera according to claim 1, wherein the image processing unit sets a subject designated by a user as the target subject immediately before or after the start of photographing.   3. The camera according to claim 1, wherein the image processing unit automatically determines a subject included in a predetermined portion as the target subject immediately before the start of shooting or in a predetermined period after the start of shooting. .   5. The camera according to claim 1, wherein a normal mode can be selected in addition to the follow-up mode, and an image displayed on the viewfinder is recorded in the normal mode.   6. The finder is an electronic finder, and in the normal mode, the finder displays an image of the effective imaging area, and the image processing unit stores the image of the effective imaging area. Camera described in.   The viewfinder is an electronic viewfinder, and in the normal mode, the viewfinder displays an image of the viewfinder area displayed by the viewfinder in the follow-up mode, and the image processing unit stores the image of the viewfinder area. The camera according to claim 5, wherein   The camera according to claim 1, wherein the raw image is discarded.   The camera according to any one of claims 1 to 7, wherein a moving image is photographed. A method of shooting a video,
Photographing with an image sensor;
Displaying a range corresponding to a predetermined portion of the image sensor on the viewfinder during shooting;
Cropping and storing a crop area including a target subject in the temporary image output from the image sensor; and
A photographing method characterized by comprising:

Images