JP5084542B2 - Automatic tracking imaging device, automatic tracking imaging method, and automatic tracking imaging program - Google Patents

Description

  The present invention relates to an automatic tracking imaging device, an automatic tracking imaging method, and an automatic tracking imaging program for tracking and shooting a target subject specified in an image.

  Automatic tracking shooting using a video camera, surveillance camera, or the like mounted on a helicopter is performed in order to track and continuously shoot target objects such as damaged buildings and incident vehicles. A conventional automatic tracking imaging device displays a moving image being shot by a video camera on a display, causes a user to specify a target subject using a rectangular frame or the like, and detects the movement of the target subject by image processing or the like. Then, the pan head is controlled at any time so that the moving target subject is maintained within the shooting range of the video camera, and the direction of the optical axis of the video camera attached to the pan head is adjusted (for example, Patent Documents). 1 or Patent Document 2).

Japanese Patent Laid-Open No. 5-6434 JP-A-8-9227

  Since the conventional automatic tracking imaging apparatus is configured as described above, the user has to specify the target subject while viewing the moving image input from the video camera. Therefore, when the target subject position in the image changes drastically, there is a problem that it is difficult to specify the target subject.

  The present invention has been made to solve the above-described problems, and presents a single still image extracted from a moving image input from a video camera when a user designates a target subject. An object is to obtain an automatic tracking imaging apparatus, an automatic tracking imaging method, and an automatic tracking imaging program that allow a user to easily specify a target subject using only an image.

An automatic tracking imaging apparatus according to the present invention includes an imaging unit that continuously acquires images, a display unit that displays images acquired by the imaging unit, and outputs an image acquired by the imaging unit to the display unit to perform a pause operation. When the image is received, the image displayed on the display unit is paused, the designation of the target subject existing in the paused image is received, and the image newly acquired by the imaging unit during the pause is recorded on the recording unit A pause type target subject designating unit to be recorded in the target, a target subject search unit for continuously searching for a target subject position from images continuously recorded in the recording unit during the pause, and a target subject search unit. An imaging range calculation unit that calculates the imaging range of the imaging unit so that the searched target subject position becomes a predetermined position of the image, and an imaging direction that controls the direction of the imaging unit based on the imaging range calculated by the imaging range calculation unit With change section In which was to so that.

According to the present invention, when an imaging unit that continuously acquires images, a display unit that displays images acquired by the imaging unit, and an image acquired by the imaging unit are output to the display unit and subjected to a pause operation Temporarily pauses the image displayed on the display unit, receives the designation of the target subject existing in the paused image, and records the newly acquired image in the imaging unit during the pause on the recording unit A target subject search unit that continuously searches for a target subject position from a stop-type target subject designation unit, an image continuously recorded in the recording unit during the pause, and a target subject searched by the target subject search unit An imaging range calculation unit that calculates the imaging range of the imaging unit so that the position of the image becomes a predetermined position of the image, and an imaging direction change unit that controls the direction of the imaging unit based on the imaging range calculated by the imaging range calculation unit I prepared Allows a user using only still image obtaining automatic tracking imaging apparatus can easily specify a target object.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing the overall configuration of the automatic tracking imaging apparatus according to Embodiment 1 of the present invention. An automatic tracking imaging apparatus shown in FIG. 1 includes a video camera (imaging unit) 1 that captures a target subject, a camera orientation changing device (imaging direction changing unit) 2 that changes the orientation of the video camera 1 according to a control signal, and a video camera 1. A continuously input image is displayed on a display (display unit) 7 with a touch panel, and an image memory (recording) is newly captured while the user designates a target subject via the display 7 with a touch panel. Part), a pause type target subject specifying unit 3 to be recorded, a target subject searching unit 4 for searching for a target subject from an image recorded in the image memory 6 or an image inputted from the video camera 1, and a target subject searching unit 4 The direction of the video camera 1 is calculated so that the searched target object does not deviate from the shooting range of the video camera 1, and a control signal is output. The camera orientation calculation unit (imaging range calculation unit) 5, the image memory 6 for recording the image output from the pause target target designation unit 3, and the image captured by the video camera 1 are displayed and the input of the user is accepted. A display 7 with a touch panel is provided.

  FIG. 2 is an explanatory diagram showing an outline of automatic tracking of the automatic tracking imaging apparatus according to Embodiment 1 of the present invention. In the present embodiment, as shown in FIG. 2A, an automatic tracking imaging apparatus will be described using an example in which a video camera 1 mounted on a helicopter 11 automatically tracks a passenger car 12 traveling on a road. .

  The video camera 1 shown in FIG. 1 and FIG. 2 (a) captures the surroundings and outputs an image. The camera orientation changing device 2 is composed of a pan head or the like capable of rotating the pan angle and the tilt angle, and changes the orientation of the video camera 1 based on a control signal input from the camera orientation computing unit 5. Change the shooting range.

  The pause type target subject designating unit 3 displays an image taken by the video camera 1 on the display 7 with a touch panel as a video frame. When a user inputs a pause operation by operating a pause button displayed on the display 7 with a touch panel, the pause type target subject designating unit 3 detects the input and displays the display 7 with the touch panel. At the same time, the displayed image is recorded in the image memory 6. Further, as will be described later, the pause-type target subject designating unit 3 causes the image memory 6 to record a region image based on the target subject input information input from the display 7 with the touch panel.

  The display 7 with a touch panel displays the image output from the pause type target subject designating unit 3 as a video frame. FIG. 2B shows a still image displayed on the display 7 with a touch panel. The pause target target designating unit 3 sets the coordinates of the X axis and the Y axis in the still image, and sets the origin (0, 0) at the center of the still image. The user designates the passenger car 12 as the target subject with a rectangular frame or the like from the paused video frame F0 displayed on the display 7 with the touch panel (FIG. 2 (c)).

  Information input by the user is output from the display 7 with a touch panel to the pause target target designating unit 3, and the pause target target designating unit 3 identifies the region image R0 based on the rectangular frame (FIG. 2C). ). The specified area image R0 is recorded by the image memory 6. Note that while the user designates the target subject, an image output by the video camera 1 every predetermined time is not displayed on the display 7 with the touch panel but is only recorded in the image memory 6. Therefore, the user can easily specify the target subject using the still image. During a period when the user is not performing an input operation, the pause target object designating unit 3 sequentially displays images taken by the video camera 1 on the display 7 with a touch panel.

  The target subject searching unit 4 performs a template matching on a similar region image that is the region best matched with the region image specified by the pause type target subject specifying unit 3 from images recorded in the image memory 6 at predetermined time intervals. Search and calculate the center coordinates. FIG. 2D shows a video frame F1 taken after the video frame F0 used by the user to designate a region image. The target subject searching unit 4 searches the similar region image R1 that best matches the region image from the video frame F1, and calculates the center coordinates (dx, dy) (FIG. 2 (e)).

  The camera direction calculation unit 5 calculates a pan angle and a tilt angle so that the calculated center coordinates (dx, dy) become the origin (0, 0), and outputs the pan angle and the tilt angle to the camera direction changing device 2 as a control signal (see FIG. 2 (f)). The camera direction changing device 2 changes the direction of the optical axis of the video camera 1 based on the control signal, and changes the shooting range so that the target subject does not deviate from the shooting range of the video camera 1.

  Next, the operation of the automatic tracking imaging apparatus will be described with reference to FIGS. In the following, it is assumed that shooting by the video camera 1 is performed every 100 ms, and that the similar area R1 search of the target subject searching unit 4 requires 50 ms. In addition, after the user designates a target subject, an image newly taken by the video camera 1 while searching using the image recorded in the image memory 6 is recorded in the image memory 6.

  3 (FIGS. 3-1 and 3-2) is an explanatory diagram showing a state immediately after the start of a user input operation of the automatic tracking imaging apparatus according to the first embodiment of the present invention. In FIG. 3, the triangle below the time axis indicates the current time, and the numerical value on the time axis indicates the time when the image fi is acquired. FIG. 4 is a flowchart showing the operation of the automatic tracking imaging apparatus according to Embodiment 1 of the present invention. As shown in FIG. 3A, the images fi taken by the video camera 1 are sequentially displayed on the display 7 with the touch panel via the pause type target subject designating unit 3 (step ST1). In step ST2, when the user presses a pause button (not shown) displayed on the display 7 with a touch panel, the pause type target subject specifying unit 3 inputs a pause operation via the display 7 with the touch panel. Detect.

  In step ST3, the pause type target subject designating unit 3 pauses the video frame F0 displayed on the display 7 with the touch panel when the input of the pause operation is detected. While the user performs an input operation to designate the target subject, the video frame F0 displayed on the display 7 with the touch panel is displayed as it is when the input operation is started, and an image f0 that is the video frame F0 is displayed. It is recorded in the image memory 6. The user designates the target subject as a rectangular frame from the video frame F0 displayed in a paused state on the display 7 with the touch panel. The target object specified by the rectangular frame input via the display with the touch panel is received as the area image R0 by the pause target object specifying unit 3.

  Images f1, f2, f3,..., Fi, which are taken every 100 ms by the video camera 1 while the user designates the target subject, are sequentially recorded in the image memory 6 by the pause type target subject designation unit 3. The When the time required for rectangular input by the user is 475 ms, images f1, f2, f3, and f4 captured during 475 ms are recorded in the image memory 6 as shown in FIG. When the pause-type target subject designating unit 3 detects that the user's input operation has been completed (step ST4 “Yes”), the automatic tracking imaging device first searches for the target subject from the image fi recorded in the image memory 6.

  In step ST5, the target subject search unit 4 uses the image f1 as the video frame F1, and searches the video frame F1 for a similar region image R1 that best matches the region image R0 recorded in the image memory 6 (FIG. 3 (c). )). In the subsequent step ST6, the pause type target subject designating unit 3 updates the area image R0 recorded in the image memory 6 to the content of the searched similar area image R1, and adds 1 to the variable i.

  Since the incremented fi, that is, the image f2 exists in the image memory 6 (step ST7 “Yes”), the process returns to step ST5. The target subject search unit 4 uses the image f2 as the video frame F1, and searches the video frame F1 for a similar region image R1 that best matches the updated region image R0 in the image memory 6 (FIG. 3 (d)). In subsequent step ST6, the pause type target subject designating unit 3 updates the region image R0 to the content of the similar region image R1, and adds 1 to the variable i.

  Since the incremented image f3 exists in the image memory 6, the process returns to step ST5 as described above, and the target subject search unit 4 sets the image f3 as the video frame F1, and updates the region image R0 in the image memory 6. The similar region image R1 that best matches is searched from the video frame F1 (FIG. 3E). Further, since the time exceeds 600 ms during the process of step ST5, the newly photographed image f6 is recorded in the image memory 6 by the pause type target subject designating unit 3. In step ST6, the pause target object designating unit 3 updates the area image R0 to the contents of the similar area image R1, and adds 1 to the variable i.

  Similarly to the above, as shown in FIG. 3 (f) to FIG. 3 (j), the images f4, f5, f6, f7, and f8 become video frames F1, respectively, and a similar region image R1 is searched from each video frame F1. Is done. In step ST7 after the similar region image R1 is searched from the video frame F1 based on the image f8, the next image f9 does not exist in the image memory 6 (step ST7 “No”), and the process proceeds to step ST8.

  After step ST8, from the process of searching for the similar area image R1 using the image fi already recorded in the image memory 6, the similar area image R1 is searched in real time using the image captured by the video camera 1, and automatically The process switches to tracking shooting. In step ST8, the target subject searching unit 4 calculates the center coordinates (dx, dy) of the region image R0 recorded and updated in the image memory 6, and sets the image f9 to the video frame F0. The area image R0 at this time is the content of the similar area image R1 searched from the image f8.

  In step ST9, the camera orientation calculation unit 5 changes the orientation of the video camera 1 so that the center coordinates (dx, dy) of the region image R0 are at the origin (0, 0). The angle is calculated and output as a control signal, and the camera orientation changing device 2 controls the orientation of the video camera 1 based on the control signal.

  After the process of step ST9 is completed, the target subject searching unit 4 acquires a video frame F1 100 ms after the video frame F0 is captured from the video camera 1 (step ST10). In subsequent step ST11, the target subject search unit 4 searches the acquired video frame F1 for a similar region image R1 that best matches the region image R0 recorded in the image memory 6. In step ST12, the video frame F1 is changed to the video frame F0, and the searched similar area image R1 is changed to the area image R0, and the image memory 6 is updated. Next, in step ST13, the center coordinates (dx, dy) of the updated region image R0 are calculated, and the process returns to step ST9 again. As described above, after step ST8, automatic tracking shooting is performed by the iterative processing of the target subject searching unit 4 and the camera direction changing device 5.

  As described above, according to the first embodiment, the video camera 1 that continuously acquires images and the touch panel that displays the images acquired by the video camera 1 and accepts designation of a target subject by a user input operation are provided. When the image acquired by the display 7 and the video camera 1 is output to the display 7 with a touch panel and a user's pause operation is received, the image displayed on the display 7 with the touch panel is paused and the paused image is displayed. From the image recorded in the image memory 6 and the pause type target object designating unit 3 which receives the designation of the target object existing in the image memory 6 and records in the image memory 6 an image newly taken by the video camera 1 during the pause. The target subject search unit 4 for searching for the position of the target subject, and the position of the target subject searched by the target subject search unit 4 are A camera orientation calculator 5 that outputs a control signal for adjusting the orientation of the video camera 1 so as to be the center of the image, and a camera orientation that controls the orientation of the video camera 1 based on the control signal input from the camera orientation calculator 5 The change device 2 is provided. Therefore, it is possible to obtain an automatic tracking imaging device that allows a user to easily specify a target subject using only a still image.

  Further, according to the first embodiment, the target subject search unit 4 is configured to search and detect the position of the target subject from the image captured by the video camera 1 when no image exists in the image memory 6. For this reason, after searching for the position of the target subject in the image captured during the user's input operation, the process automatically searches for the position of the target subject from the image captured by the video camera 1 and continues automatic tracking shooting. Can do.

  The pause type target subject designating unit 3, the target subject searching unit 4, and the camera direction calculation unit 5 described above may be configured by various electronic circuits as hardware, but these are configured by software, that is, temporarily stopped. It may be an automatic tracking photographing program composed of a type target subject designation step, a target subject search step, and a camera direction calculation step.

  In Embodiment 1 described above, the orientation of the video camera 1 is not changed while the target subject search is being performed on the image recorded in the image memory 6, but the orientation of the video camera 1 is changed. It may be a configuration. Specifically, between step ST6 and step ST7 shown in FIG. 4, the target subject search unit 4 calculates the center coordinates (dx, dy) using the searched region image R0, and the camera direction calculation unit 5 Controls the camera orientation changing device 2 to add processing for adjusting the orientation of the video camera 1 so that the center coordinates (dx, dy) are at the origin (0, 0). In the case of this configuration, there is an effect of preventing a target subject having a high moving speed from deviating from the shooting range of the video camera 1 while searching for an image recorded in the image memory 6.

  In the first embodiment, when the target subject searching unit 4 searches for the similar region image R1, the template matching is performed using the region image R0 as a template and the video frame F1 as a search image. A configuration using other algorithms such as extracting feature amounts from an image and searching for similar regions based on the information may be used.

  In the first embodiment, the camera orientation changing unit 2 that can rotate the pan angle and the tilt angle of the video camera 1 is used for the camera orientation calculation unit 5 to correct the position of the target subject in the image. However, a configuration using a camera orientation changing device having a rotation axis other than the pan angle and tilt angle, for example, a roll angle, a pitch angle, or the like may be used.

It is a block diagram which shows the whole structure of the automatic tracking imaging device which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the outline | summary of the automatic tracking of the automatic tracking imaging device which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the state immediately after the user input operation start of the automatic tracking imaging device which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the state immediately after the user input operation start of the automatic tracking imaging device which concerns on Embodiment 1 of this invention. It is a flowchart which shows operation | movement of the automatic tracking imaging device which concerns on Embodiment 1 of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Video camera (imaging part), 2 Camera direction change apparatus (imaging range calculation part), 3 Pause type target subject designation part, 4 Target object search part, 5 Camera direction calculation part (imaging direction conversion part), 6 Image memory (Recording unit), 7 Display with touch panel (display unit), 11 Helicopter, 12 Passenger car, f0, f1, f2..., Fi image, F0, F1 video frame, R0 region image, R1 similar region image.

Claims (4)

An imaging unit for continuously acquiring images;
A display unit for displaying an image acquired by the imaging unit;
When the image acquired by the imaging unit is output to the display unit and subjected to a pause operation, the image displayed on the display unit is paused and the target subject existing in the paused image is displayed. A pause type target subject designating unit that receives a designation and causes the recording unit to record an image newly acquired by the imaging unit during the pause;
A target subject search unit that continuously searches for a position of the target subject from images continuously recorded in the recording unit during the pause;
An imaging range calculation unit that calculates an imaging range of the imaging unit such that the position of the target subject searched by the target subject search unit is a predetermined position of the image;
An automatic tracking imaging apparatus comprising: an imaging direction changing unit that controls a direction of the imaging unit based on an imaging range calculated by the imaging range calculation unit.   The target subject search unit searches for the position of the target subject from the image acquired by the imaging unit when an image newly acquired by the imaging unit during pause is not present in the recording unit. Automatic tracking shooting device. An imaging step for continuously acquiring images;
A display step for displaying the image acquired in the imaging step;
When the image acquired in the imaging step is output to the display step and subjected to a pause operation, the image displayed in the display step is paused and the target subject existing in the paused image is displayed. A pause type target subject designation step for receiving a designation and causing the recording unit to record an image newly acquired in the imaging step during the pause;
A target subject search step for continuously searching for the position of the target subject from images continuously recorded in the recording unit during the pause;
An imaging range calculating step for calculating the imaging range of the imaging step so that the position of the target subject searched in the target subject searching step is a predetermined position of the image;
An automatic tracking imaging method comprising: an imaging direction changing step for controlling the direction of the imaging step based on the imaging range calculated in the imaging range calculation step. The pause-type target subject designating unit outputs an image acquired by the imaging unit that continuously acquires images to the display unit, and when receiving a pause operation, pauses the image displayed by the display unit, And a pause type target subject designation step of receiving a designation of a target subject existing in the paused image and causing the recording unit to record an image newly acquired by the imaging unit during the pause,
A target subject search unit for continuously searching for the position of the target subject from images continuously recorded in the recording unit during the pause;
Automatic tracking for causing the computer to execute an imaging range calculation step in which the imaging range calculation unit calculates the imaging range of the imaging unit so that the position of the target subject searched in the target subject search step becomes a predetermined position of the image Shooting program.

Images