JP2005037273A - Video image shake detecting device, method and program thereof, and camera video image selecting device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video image shake detecting device, its method and program, for detecting a shake of a video image produced when a camera is shook by an external factor such as earthquake or the like, from photographed camera video images of an object to be photographed. <P>SOLUTION: The video image shake detecting device 10 divides each image forming a video image into blocks of specific size. extracts a movement vector by a movement vector extracting means 11, from a coded video image formed by coding the video image on the basis of a moving vector between blocks at each time interval, and detects the start of a shake with a shake start detecting section 12a by the size of the movement vector; and a shake stop detecting section 12b totals moving vectors from the start of the shake up to the stop of the shake by a movement vector totaling section 12b1. If the total of the movement vectors is smaller than a predetermined threshold, it is determined that the shake has been generated due to an earthquake or the like. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a video shake detection apparatus, a method thereof, and a program therefor for detecting a shake of a video obtained by photographing an imaging target, and further comprising the video shake detection apparatus, wherein an earthquake occurs at a place where a camera is installed from a plurality of camera videos. The present invention relates to a camera video selection device that selects a camera video when it occurs.

Currently, robot cameras that shoot desired images by remote control remote control are installed in broadcasting stations throughout the country, major ports, airports, etc., and remote control remote control from each broadcast station that directly controls the robot camera Makes it possible to shoot on-site images during disasters such as earthquakes.
Conventionally, for example, when a broadcasting station, which is a key station, uses a video taken by robot cameras installed throughout the country as a video for broadcasting, the robot camera images transmitted from broadcasting stations throughout the country are monitored, The optimal video was selected by visual confirmation and used as a broadcast video.

  Here, with reference to FIG. 6, a conventional robot camera monitoring system that generates a broadcast video from a video shot by a robot camera will be described. FIG. 6 is a schematic configuration diagram of a conventional robot camera monitoring system.

In FIG. 6, the robot camera monitoring system 100 </ b> B connects a broadcasting station (central broadcasting station) 4 </ b> B that performs national broadcasting and a broadcasting station (local broadcasting station) 4 </ b> C that controls the robot camera 7 through a communication line 6. . Further, in the area under the jurisdiction of the local broadcasting station 4C, a plurality of robot cameras (hereinafter simply referred to as cameras) 7 (7 ₁ , 7 ₂ ,..., 7 _n ) and a plurality of base stations 8 (8 ₁ , 8 ₂ ,..., 8 _n ) are installed, and the video (camera video) captured by the camera 7 passes through the base station 8 and the FPU (Field Pickup Unit: wireless transceiver) of the local broadcasting station 4C. 5 is transmitted.

The local broadcasting station 4C switches and receives the video captured by the camera 7 with the FPU 5. That is, the local broadcasting station 4C secures any of the cameras 7 ₁ , 7 ₂ ,..., 7 _{n with} the FPU 5 and receives the secured video of the camera 7. Further, when receiving video from another camera 7, the other camera 7 is secured (switched) by the FPU 5. A plurality of local broadcasting stations 4C exist in various parts of the country, but are omitted in FIG.

The central broadcasting station 4B receives images (camera images) taken by the camera 7 necessary for broadcasting from the local broadcasting stations 4C throughout the country via the communication line 6, and broadcasts them as broadcast images.
Here, the operation of the robot camera monitoring system 100B when an earthquake occurs in the area under the jurisdiction of the local broadcasting station 4C will be described. First, when an earthquake occurs in the area under the jurisdiction of the local broadcasting station 4C, the person in charge arranges broadcasting lines for the local broadcasting station 4C and the central broadcasting station 4B.

Then, the person in charge of the central broadcasting station 4B requests the local broadcasting station 4C to investigate the situation regarding the earthquake and connect the camera 7. Upon receiving this request, the local broadcasting station 4C sequentially switches all the cameras 7 ₁ , 7 ₂ ,..., 7 _n managed by the local station by the FPU 5 and grasps the situation of the earthquake. Then, the broadcasting line (communication line 6) of the local broadcasting station 4C and the central broadcasting station 4B is connected, and the optimum video (camera video) at the present time is transmitted to the central broadcasting station 4B. Then, the central broadcasting station 4B generates a broadcast video from the received camera video and broadcasts it.

A digital transmission system using an FPU, which is a radio transmission device for transmitting television broadcast program material, has been standardized by the Association of Radio Industries and Businesses (ARIB) (non-patent literature). 1).
Radio Industry Association (ARIB) Standard “Portable OFDM Digital Wireless Transmission System for Transmission of Television Broadcasting Program Materials” (ARIB STD-B33) formulated on March 28, 2002

  However, in the conventional robot camera monitoring system described above, it is necessary to manually select and broadcast the optimal camera while confirming (viewing) the images sent from the cameras installed in each region one by one. As a result, a time lag for judging the local situation occurred, and it was not possible to provide camera images quickly.

  In addition, initial response is most important in emergency and disaster reports, but in conventional robot camera monitoring systems, it is necessary to grasp the situation in each region when information is not organized. Multiple camera images will be arranged at the same time. Therefore, the central broadcasting station may connect the cameras of a plurality of local broadcasting stations at almost the same time. As a result, there is a problem in that information transmitted through a communication line connecting the local broadcasting station and the central broadcasting station is congested and video cannot be transmitted quickly.

  Furthermore, even if the epicenter is known, the shaking in the area close to the epicenter is not always large, and the shaking may be large in the area far from the epicenter. For this reason, when the central broadcasting station acquires camera video from a local broadcasting station that has jurisdiction over the camera installed in the area close to the epicenter, video with little shaking or no recorded phenomenon of shaking itself In some cases, appropriate camera images could not be provided.

  The present invention has been made in view of the above-described problems, and is a video shake detection device that detects a video shake with respect to a shooting target from a camera video, a method thereof, a program thereof, and each region. It is an object of the present invention to provide a camera image selection device that can determine the magnitude of an earthquake based on the shaking of a camera image of a camera and select an optimal camera image.

  The present invention was devised to achieve the above object. First, the video shake detection device according to claim 1 divides a time-series image constituting a video into blocks of a specific size. A video shake detection device for detecting a shake of the video with respect to a subject to be photographed from an encoded video obtained by encoding the video based on a motion vector between the blocks for each time, comprising a motion vector extraction means, and a shake detection Means.

  According to such a configuration, the video shake detection apparatus extracts motion vectors from the encoded video every time by the motion vector extraction means. This motion vector indicates a moving direction and an amount of movement for each block between blocks obtained by dividing an image (picture: frame or field) constituting a video into a specific size. For example, in an encoding method such as MPEG (Moving Picture Experts Group), video is compressed and encoded using motion vectors in units of macroblocks.

  As described above, in a coded video obtained by coding a video with a motion vector, a motion vector is generated in the entire image when the camera shakes due to an external factor such as an earthquake. Therefore, the video shake detection device can detect the video shake with respect to the object to be photographed by determining the magnitude of the motion vector included in the encoded video by the shake detection means.

  The video shake detection apparatus according to claim 2 is the video shake detection apparatus according to claim 1, wherein the shake detection means includes a shake start detection means and a shake stop detection means. And

  According to such a configuration, the image shake detection device compares the magnitude of the motion vector extracted by the motion vector extraction means with the predetermined magnitude by the shake start detection means, and the magnitude of the motion vector is predetermined. If it is larger than the size, it is determined that the image has started to shake. For example, if the image of the subject is captured by the camera and the size of the motion vector is greater than or equal to a certain size for the entire image, it is not the motion of the subject itself but the image is shaken by the movement of the camera. Can be determined. In this case, the magnitude of the motion vector may be determined for all blocks (macroblocks) of the entire image, or the magnitude of the motion vector may be determined for a plurality of predetermined blocks. Good.

  Then, the video shake detection device detects the start of video shake by the shake stop detection means, compares the magnitude of the motion vector extracted by the motion vector extraction means with a predetermined magnitude, and determines the magnitude of the motion vector. Is smaller than the predetermined size, it is determined that the shaking of the video has stopped. In other words, after detecting the start of video fluctuation, the motion vector size is larger than a certain size for all blocks (macroblocks) of the entire image or for a plurality of predetermined blocks. It can be determined that the shaking of the video has stopped.

  Furthermore, the video shake detection device according to claim 3 is the video shake detection device according to claim 2, further comprising motion vector accumulating means, wherein the motion stop detection means accumulates motion vectors accumulated by the motion vector accumulating means. , And whether or not the motion vector extracted by the motion vector extraction means is smaller than a predetermined size, thereby detecting the stop of the shaking of the video. .

According to such a configuration, the video shake detection device accumulates the motion vectors extracted by the motion vector extraction unit every time after the motion vector totaling unit detects the start of video shake by the shake start detection unit. In general, when the image is shaken in one direction due to an external factor such as an earthquake, the shake in the opposite direction occurs due to the shake back. That is, in the case of a shake such as an earthquake, when the motion vectors are accumulated from the start to the stop of the shake, the magnitude converges to almost “0”. For example, when pitching or rolling occurs due to an earthquake, vertical motion vectors and horizontal motion vectors are generated, and the respective motion vectors converge with time.
Thereby, the shaking stop detection means can determine whether the shaking is a shaking generated based on an external factor such as an earthquake when the stopping of the shaking of the image is detected.

  According to a fourth aspect of the present invention, there is provided a video shake detection apparatus according to the second or fourth aspect, further comprising a maximum shake measurement unit.

  According to such a configuration, the video shake detection apparatus uses the maximum shake value measurement unit to obtain the maximum shake value that maximizes the magnitude of the motion vector extracted by the motion vector extraction unit between the start and stop of video shake. Measure. Since the maximum value of the shaking indicates the magnitude of the shaking such as an earthquake, for example, it was taken at a place where the seismic intensity of the earthquake is large from among the images taken with multiple cameras installed in each area. It becomes possible to identify the camera.

  The video shake detection method according to claim 5 further divides a time-series image constituting a video into blocks having a specific size and encodes the video based on a motion vector between the blocks for each time. A video shake detection method for detecting a shake of the video with respect to an imaging target from the encoded video, comprising a motion vector extraction step, and further includes a shake start detection step, a motion vector accumulation step, and a shake stop detection step. It is characterized by including.

  According to this procedure, in the video shake detection method, the motion vector is extracted from the encoded video every time in the motion vector extraction step. This motion vector indicates a moving direction and an amount of movement for each block between blocks obtained by dividing an image (picture: frame or field) constituting a video into a specific size.

Then, the image shake detection method compares the magnitude of the motion vector extracted in the motion vector extraction step with a predetermined magnitude in the shake start detection step, and the magnitude of the motion vector is larger than the predetermined magnitude. Then, it is determined that the shaking of the video has started. In the video shake detection method, after detecting the video shake, the motion vectors extracted in the motion vector extraction step are accumulated every time in the motion vector accumulation step.
Then, in the shaking stop detection step, the magnitude of the motion vector accumulated in the motion vector accumulation step and the magnitude of the motion vector extracted in the motion vector extraction step are respectively compared with a predetermined magnitude, and a predetermined magnitude is obtained. If it is smaller than this, it is determined that the shaking has stopped.

  The video shake detection program according to claim 6 divides a time-series image constituting a video into blocks of a specific size, and encodes the video based on a motion vector between the blocks for each time. In order to detect the fluctuation of the video with respect to the subject to be photographed from the encoded video, the computer is caused to function as a motion vector extracting means, a shake start detecting means, a motion vector accumulating means, and a shake stop detecting means.

  According to this configuration, the video shake detection program extracts the motion vector from the encoded video every time by the motion vector extraction unit. This motion vector indicates a moving direction and an amount of movement for each block between blocks obtained by dividing an image (picture: frame or field) constituting a video into a specific size.

  The video shake detection program compares the magnitude of the motion vector extracted by the motion vector extraction means with the predetermined magnitude by the shake start detection means, and the magnitude of the motion vector is larger than the predetermined magnitude. Then, it is determined that the shaking of the video has started. The video shake detection program accumulates the motion vectors extracted by the motion vector extraction means every time after the motion vector accumulation means detects the start of the video shake by the shake start detection means.

  The video shake detection program compares the magnitude of the motion vector accumulated by the motion vector accumulation means and the magnitude of the motion vector extracted by the motion vector extraction means by the shake stop detection means, respectively, with a predetermined magnitude. If it is smaller than the predetermined size, it is determined that the shaking has stopped.

  Furthermore, the camera video selection device according to claim 7 divides a time-series image constituting a video shot by a camera installed in each region into blocks of a specific size, and between the blocks for each time A camera video selection device for selecting and outputting a video of a camera installed in an area where an earthquake has occurred from an encoded video obtained by encoding the video based on a motion vector, the camera video receiving means, and a video The apparatus includes a shake detection unit, a storage unit, and a video selection unit.

  According to such a configuration, the camera video selection device receives the encoded video of the video shot by the camera installed in each region by the camera video reception unit via the communication line such as a network, and stores it in the storage unit. Accumulate sequentially. Further, the camera image selection device detects the image shake with respect to the object to be photographed based on the motion vector included in the encoded image received by the camera image receiving means by the image shake detecting means, and uses the information on the occurrence of the shake. Certain shaking information is stored in the storage means in the storage means. As a result, when a video shake occurs in the video (encoded video) shot by each camera, the shake information is stored in association with each camera.

  Then, the camera video selection device specifies the camera that has captured the video that has been shaken based on the shake information from the storage means by the video selection means, and uses the video that has been shaken as a playback video. It becomes possible.

  According to the first aspect of the present invention, the fluctuation of the video with respect to the shooting target is detected from the video (encoded video) encoded by the motion vector generated in block units of a specific size such as MPEG. be able to. For example, it is possible to specify a video shot at the time of an earthquake from a video (encoded video) shot by a camera installed in each region by the shaking of the video.

  According to the second aspect of the present invention, it is possible to specify the start and stop of the image shake based on the magnitude of the motion vector. As a result, it is possible to specify the video while the shaking is occurring in the video.

  According to the invention described in claim 3, claim 5 or claim 6, it is possible to detect the shaking of the image based on an external factor such as an earthquake. That is, even when the camera is controlled by operations such as panning and tilting, it is possible to distinguish a change in motion vector associated with the control from an earthquake shake. Thereby, for example, when monitoring an image taken by a camera installed in each region, it is possible to reliably detect the shaking of the image caused by the earthquake.

  According to the fourth aspect of the present invention, the magnitude of the moment when the magnitude of the motion vector is maximum can be measured as the maximum shake value between the start and stop of the video shake. Thereby, for example, when monitoring a video taken by a camera installed in each region, it is possible to specify a camera video in a region having a large seismic intensity.

  According to the invention described in claim 7, camera video (encoded video) captured by a camera installed in each region, and shaking information indicating information when a shaking such as an earthquake occurs in the video is provided. Based on the stored and shake information, it is possible to identify the camera in the area where the shake such as an earthquake has occurred and its video. Furthermore, it is possible to read from the storage means the video at the moment when the video shakes. As a result, it is possible to quickly search for an image at the time of a disaster such as an earthquake without inquiring about the state of the disaster in each region. In addition, even if information on the epicenter, seismic intensity, etc. announced by the Japan Meteorological Agency etc. does not match the shaking in the area where the camera is installed, it is possible to reliably detect earthquakes in the area where the camera is installed. It is possible to quickly search for images used for emergency reports and disaster reports where initial response is important, and use them as broadcast images.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
Here, we will first explain the relationship between the shaking of the image relative to the subject to be photographed by the camera and the encoded image resulting from encoding the image, and then detecting the shaking of the image. An image shake detection device and a camera image selection device that selects an image in which an earthquake has occurred from images taken by a camera installed in each region using the image shake detection device will be sequentially described. .

[About shaking the image]
First, with reference to FIG.4 and FIG.5, the relationship between the fluctuation | variation of the image | video with respect to the to-be-photographed object (photographing object) and the encoding image | video which encoded the image | video is demonstrated. The encoded video is obtained by dividing a time-series image constituting the video into blocks of a specific size and encoding the video based on a motion vector between the blocks for each time. For example, the stream data is encoded by an encoding method such as MPEG.

  FIG. 4 is a schematic diagram in which the fluctuation of the video is expressed by the motion of the motion vector of the block (macroblock) for each image constituting the video. FIG. 4A shows a state in which an image constituting an image is divided into blocks B of a specific size (for example, a macro block of 16 × 16 pixels in MPEG). Here, the image is shaken. It shows a steady state that is not. In the figure, a circle indicates a state where no motion vector is generated (no motion).

FIG. 4B shows a state in which a motion vector V ₁ in which the subject has moved in the upper right direction has occurred. FIG. 4C shows a state in which a motion vector V ₂ in which the subject has moved in the lower left direction has occurred. FIG. 4D shows a state in which a motion vector V ₃ in which the subject has moved to the upper right has occurred.

Here, when the camera that is photographing the subject is shaken due to an earthquake or the like, the uniform motion vector V is applied to each block B of the image as shown in FIG. 4B from the steady state of FIG. ₁ (here, the motion vector in the upper right direction) occurs. Further, with the passage of time, a motion vector V ₂ (here, a motion vector in the lower left direction) shown in FIG. 4C is generated as a swing back with respect to the swing of FIG. In addition, as a swing back with respect to the swing of FIG. 4C, the motion vector V ₃ (here, the motion vector in the upper right direction) shown in FIG. 4D is generated. This movement is repeated sequentially, and the shaking converges to return to the steady state of FIG.
In this way, when the camera shakes, the motion vector of each block B that is uniformly generated converges while generating a motion vector in the reverse direction as time passes.

  Here, with reference to FIG. 5, focusing on only one block (macro block), a change in motion vector from the start to the end of shaking will be described. FIG. 5 is a conceptual diagram conceptually showing the motion of a motion vector in one macro block from the start to the end of video shaking. FIG. 5 two-dimensionally shows how the motion vector from the start (occurrence) to the stop of the video shake changes with time t.

  As shown in FIG. 5, the encoded video obtained by encoding the video in which the camera shakes due to an earthquake or the like is accompanied by a motion vector in the reverse direction (here, the motion vector in the right direction and the left motion vector). Direction motion vector) is generated and the shaking is settled, the magnitude of the motion vector converges to “0”. Here, reverse direction motion vectors are alternately generated, but this depends on the video sampling time, and after a plurality of one direction motion vectors are generated, a plurality of reverse direction motion vectors are generated. In some cases.

  In this way, in a shake due to an earthquake or the like, not only a motion vector in one direction is generated, but a motion vector in the reverse direction is generated, so the sum of motion vectors generated from the start to the stop of the shake is The size is as close to “0” as possible. In this way, by measuring whether or not the sum of the motion vectors has approached “0” after the motion vector has been generated, is the fluctuation caused by an external factor such as an earthquake? I can recognize.

[Image shake detection device: Configuration]
Next, with reference to FIG. 1, the configuration of the image shake detection apparatus according to the present invention will be described. FIG. 1 is a block diagram illustrating a configuration of a video shake detection apparatus. The video shake detection device 1 detects video shake with respect to a subject (photographing target) from an encoded video obtained by encoding video based on a motion vector between macroblocks for each time such as MPEG. Is. Here, the video shake detection device 10 is configured to include a motion vector extraction means 11 and a shake detection means 12.

  The motion vector extracting means 11 analyzes encoded video (stream data) encoded by MPEG or the like, and extracts a motion vector from the encoded video in units of macroblocks for each time. The extracted motion vector is notified to the shaking start detection unit 12a and the shaking stop detection unit 12b of the shaking detection unit 12. The motion vector extracting means 11 may extract motion vectors of all macroblocks constituting each frame (or field) in units of frames (or fields) of the input encoded video. It is also possible to extract a motion vector of a specific macroblock predetermined in (or field).

In MPEG, a stream of encoded video has a hierarchical structure, and a sequence (Sequence) layer, a GOP (Group of Picture) layer, a slice (Slice) layer, and a macro block (Macro Block) in order from the upper layer. It is a layer and a block layer. In addition, since the motion vector data is described in the macroblock layer, the motion vector extraction unit 11 can acquire the motion vector in units of macroblocks from the macroblock layer.
Here, the motion vector extraction means 11 acquires the time code (hour, minute, second, picture) included in the GOP layer header (GOP header) and notifies the motion detection means 12 simultaneously with the motion vector. And

  The shake detection unit 12 detects a video shake with respect to the subject based on the motion vector extracted by the motion vector extraction unit 11. Here, the shake detection means 12 includes a shake start detection unit 12a and a shake stop detection unit 12b.

  The shake start detection unit (swing start detection means) 12a detects the start of video shake based on the motion vector extracted by the motion vector extraction means 11. The shake start detection unit 12a detects the start of the shake by determining whether or not the magnitude of the motion vector is larger than a predetermined size (threshold: shake start detection threshold). When detecting the start of shaking, the shaking start detecting unit 12a notifies the shaking stop detecting unit 12b of a detection start notification including the time code at which the shaking is detected.

  In the case of a shake such as an earthquake, the entire image is accompanied by a uniform motion. Therefore, the shake start detection unit 12a determines all macroblocks in the image (picture: frame or field) or predetermined in the image. If the direction and size of the macroblocks at a plurality of positions are within a predetermined range and the size is larger than a predetermined threshold value, it is determined that a shake has occurred (started).

  The shaking stop detection unit (swing stop detection unit) 12b detects the stop of shaking of the video based on the motion vector extracted by the motion vector extraction unit 11 after receiving the detection start notification from the shaking start detection unit 12a. To do. After detecting the shaking, the shaking stop detecting unit 12b detects the shaking stop by determining whether or not the magnitude of the motion vector is smaller than a predetermined size (threshold: shaking stop detection threshold).

The shaking stop detection unit 12b detects a shaking stop of the video, a shaking start time code indicating a time when the shaking has occurred, a shaking stop time code indicating a time when the shaking has stopped, and a shaking start (occurrence) ) To the stop, and the maximum value indicating the maximum value of the motion vector is output to the outside as vibration information.
Here, the shake stop detecting unit 12b includes the motion vector accumulating unit 12b1 and the maximum shake value measuring unit 12b2, thereby stopping the shaking of the image based on external factors such as an earthquake, and from the start to the stop of the shaking of the image. The value with the maximum magnitude of shaking is detected.

The motion vector accumulating unit (motion vector accumulating unit) 12b1 individually accumulates the motion vectors of the macroblocks extracted by the motion vector extracting unit 11 after receiving the detection start notification from the shake start detecting unit 12a. To do. The motion vector accumulation unit 12b1 accumulates the accumulation results in an accumulation unit (not shown). Then, the shaking stop detection unit 12b refers to the accumulated result when the shaking stop is detected according to the magnitude of the motion vector, and the magnitude of the accumulated motion vector (cumulative motion vector) is a predetermined value ( If it is smaller than (shake stop detection threshold) (close to “0”), it is determined that the shake of the video is based on an external factor such as an earthquake.
As a result, the shaking stop detection unit 12b can recognize the shaking back of the video (see FIG. 5), so that uniform motion vectors generated by internal factors such as camera pan and tilt are omitted from the shaking target. be able to.

  The maximum shake value measuring unit (maximum shake measuring means) 12b2 measures the maximum shake value at which the size of the motion vector of the macroblock extracted by the motion vector extracting means 11 is maximized. Note that the maximum shake value measuring unit 12b2 compares the magnitudes of the motion vectors of the respective macroblocks input after the detection start notification is notified from the shake start detecting unit 12a, and the magnitude of the maximum motion vector. Is the maximum value of shaking.

  Note that in the case of shaking such as an earthquake, the entire image is accompanied by a uniform motion, so that the shaking stop detection unit 12b can determine all macroblocks in the image (picture: frame or field) or predetermined in the image. In addition, the motion vectors are accumulated for the macroblocks at a plurality of positions, and the stop of shaking is detected. In this way, video fluctuations caused by external factors such as earthquakes can be reliably detected by detecting video fluctuations based on the motion vectors of multiple macroblocks in an image (picture: frame or field). can do.

As described above, the configuration of the video shake detection apparatus 10 has been described as an embodiment, but the present invention is not limited to this. Here, the image shake detection apparatus 10 outputs the shake information (the shake start time code, the shake stop time code, and the shake maximum value) when the shake is stopped, but only the start and stop of the shake are detected. If so, the maximum swing measurement unit 12b2 may be removed from the configuration.
Further, if the start and stop of video shake are detected in real time, the shake start detection unit 12a may output a shake start time code as shake information to the outside when the shake is detected. . In this case, it is also possible to notify only the occurrence of the shaking start and stop events without including the time code.

  Note that the video shake detection device 10 can be realized by causing a general computer to execute a program and operating an arithmetic device or a storage device in the computer. This program (video shake detection program) can be distributed via a communication line, or can be written and distributed on a recording medium such as a CD-ROM.

[Image shake detection device: Operation]
Next, referring to FIG. 2 (refer to FIG. 1 as appropriate), the operation of the video shake detection apparatus according to the present invention will be described. FIG. 2 is a flowchart showing the operation of the image shake detection apparatus.

<Motion vector extraction step>
First, the video shake detection apparatus 10 analyzes the input encoded video by the motion vector extraction unit 11 and extracts a motion vector in an image (picture: frame or field). Here, first, the motion vector extraction unit 11 extracts a time code from the GOP header of the GOP layer (step S1), and extracts a motion vector of a macroblock in a picture (image) included in the GOP layer (step S1). S2).

<Swing start detection step>
Then, in the video shake detection device 10, the magnitude of the motion vector extracted by the motion vector extraction means 11 by the shake start detection unit 12 a of the shake detection means 12 is a predetermined magnitude (threshold: shake start detection threshold). It is determined whether it is larger (step S3). If the magnitude of the motion vector is larger than the shake start detection threshold (Yes in step S3), the shake start detection unit 12a notifies the shake stop detection unit 12b of a detection start notification, and the following shake stop detection operation is performed. Proceed to (after step S4). On the other hand, if the magnitude of the motion vector is equal to or smaller than the shake start detection threshold (No in step S3), it is determined that no shake has occurred in the image, and the process proceeds to step S13 to analyze the encoded video of the next image. Continue.

  Here, the shaking stop detection unit 12b that has received the notification of the detection start notification adds the motion vectors in the image extracted by the motion vector extraction unit 11 by the motion vector totaling unit 12b1, and serves as an initial value of the total motion vector. In step S4, the maximum value of the motion vector in the image is set as the initial value of the maximum shake value by the maximum shake measurement unit 12b2.

  Then, the image shake detection apparatus 10 extracts the time code from the GOP header of the GOP layer indicating the next image in time series by the motion vector extraction unit 11 (step S6), and the picture (image) included in the GOP layer The motion vector of the macroblock is extracted (step S7).

<Motion vector accumulation step>
Here, the motion vector accumulation unit 12b1 of the shaking stop detection unit 12b adds the motion vectors in the image, and adds the motion vectors accumulated up to the previous image, thereby accumulating the motion vectors (step S1). S8). Further, when the magnitude of the motion vector in the image is larger than the maximum shake value up to the previous image, the maximum shake value measurement unit 12b2 of the shake stop detection unit 12b updates the magnitude as the maximum shake value ( Step S9).

<Swing stop detection step>
Then, the video shake detection apparatus 10 determines whether the magnitude of the motion vector in the image is smaller than a predetermined size (threshold: shake stop detection threshold) by the shake stop detection unit 12b (step S10). . If the magnitude of the motion vector is smaller than the shake stop detection threshold (Yes in step S10), the shake stop detection unit 12b determines that the shake has stopped, and proceeds to step S11. On the other hand, if the magnitude of the motion vector is equal to or greater than the shake stop detection threshold (No in step S10), it is determined that the shake has not stopped in the image, and the process returns to step S6 and the encoded image to be input next. The GOP layer is analyzed.

  Further, the video shake detection apparatus 10 determines whether the magnitude of the accumulated motion vector accumulated in step S8 is smaller than the shake stop detection threshold by the shake stop detection unit 12b (step S11). If the cumulative motion vector is smaller than the shake stop detection threshold (Yes in step S11), the shake stop detection unit 12b determines that the shake causing the shake return has stopped, and The time code is set as a shaking stop time code, and the shaking start time code, which is the time code at which the shaking starts, and the shaking maximum value are output together as shaking information (step S12). On the other hand, when the magnitude of the cumulative motion vector is equal to or greater than the shake stop detection threshold (No in step S11), the shake stop detection unit 12b determines that the shake is not a shake like an earthquake that causes a shake back, and proceeds to step S13. move on.

Then, the video shake detection apparatus 10 determines whether or not the next image exists in the input encoded video (step S13). If the next image exists (Yes in step S13), the process returns to step S1. And continue to detect image shaking. On the other hand, when the next image does not exist (No in step S13), the video shake detection apparatus 10 ends the detection of the video shake.
With the above operation, the video shake detection apparatus 10 can detect video shake caused by an external factor such as an earthquake from a video (encoded video) obtained by photographing a subject.

[Camera image selection device]
Next, referring to FIG. 3, a camera image selection device according to the present invention will be described. FIG. 3 is a block diagram showing a configuration of a robot camera monitoring system including a camera image selection device. This robot camera monitoring system 100 is installed in major ports, airports, etc. throughout the country with a robot camera that shoots a desired image by remote control by remote control. It will be broadcast as. Here, the robot camera monitoring system 100 is configured by connecting cameras (robot cameras) 1 and broadcast stations 4 installed throughout the country via a network 3.

The cameras 1 (1 ₁ ,... 1 _n ) are robot cameras that are installed in broadcasting stations throughout the country, main ports, airports, and the like and shoot images of the site. Images captured by the camera 1 are transmitted to the IP encoders 2 (2 ₁ ,... 2 _n ), respectively.

The IP encoder 2 (2 ₁ ,... 2 _n ) performs encoding such as MPEG on the video captured by the camera 1. The encoded video (stream data) encoded here is encrypted based on IPsec (IP Security Protocol) by an encryption device (not shown) and transmitted to the broadcasting station 4.

The network 3 is a virtual private network such as an IP-VPN (Internet Protocol-Virtual Private Network), for example. Accordingly, the camera 1 and the broadcasting station 4 can be connected to each other like a dedicated line, and safe communication can be performed.
The broadcasting station 4 includes a camera video selection device 20 and selects and broadcasts a camera video necessary for broadcasting from the camera video shot by the cameras 1 installed throughout the country.

  The camera image selection device 20 selects an image of a camera installed in an area where an earthquake has occurred from camera images taken by the camera 1. The video selected here is broadcast as a broadcast video. Here, the video selected by the camera video selection device 20 is a news program or the like shot in the studio 30, and is combined with the video of the broadcast camera C that shot the announcer M as necessary to produce a broadcast video. Broadcast.

Hereinafter, the camera image selection device 20 will be described in detail.
Here, the camera video selection device 20 includes a camera video reception means 21, a video shake detection means (video shake detection device) 10, a storage means 22, a video selection means 23, and a broadcast video output means 24. Is configured.

  The camera video receiving means 21 receives video (encoded video) taken by the cameras 1 arranged throughout the country via the network (IP-VPN) 3. The camera image receiving means 21 accumulates the camera image (encoded image) 22a in various places in the accumulation means 22 together with the identification information (identification number) for identifying the camera and the time code by constant connection.

  The image shake detecting means (image shake detecting device) 10 detects the image shake caused by the earthquake based on the motion vector described in the camera image (encoded image) received by the camera image receiving means 21, and the shake is detected. The shaking information 22b including the shaking start time code when the shaking is started, the shaking stop time code when the shaking is stopped, and the maximum value (the shaking maximum value) of the motion vector from the start to the stop of the shaking The information is stored in the storage unit 22 together with identification information (identification number) to be specified. The video shake detection means (video shake detection device) 10 is the same as the video shake detection device described with reference to FIG.

  The storage unit 22 stores the video of the camera 1 (camera video 22a) received by the camera video reception unit 21 and the shake information 22b detected by the video shake detection unit 10, and is a storage medium such as a hard disk. In this storage means 22, for example, camera images from all over the country are stored for one day. It should be noted that videos exceeding the number of days are deleted.

  The video selection unit 23 selects a video in which shaking due to an earthquake has occurred from the camera video 22 a based on the shaking information 22 b stored in the storage unit 22. For example, since the shaking start time code at which the shaking has started is recorded together with the identification information of the camera 1 in the shaking information 22b, the camera 1 in which the shaking has occurred can be specified. In addition, the start time of the video where the shaking has occurred can be specified by the time code.

  The video selection unit 23 displays a plurality of videos when an earthquake occurs on a display unit (not shown), and an operator (switcher or the like) selects a video to be used as a broadcast video. In addition, since the shake maximum value is recorded for each camera 1 in the shake information 22b, when displaying a video that is a candidate for broadcast video on a display unit (not shown), the camera in the descending order of the maximum shake value. By displaying this video, the operator can quickly select an optimal video as a video used for broadcasting. The identification information of the camera 1 selected here and the time (time code) when the earthquake occurred are notified to the broadcast video output means 24.

The broadcast video output means 24 reads the camera video 22a corresponding to the camera identification number and time selected by the video selection means 23 from the storage means 22 and outputs it as a broadcast video. The broadcast video output here is broadcast.
By configuring the camera video selection device 20 in this way, the robot camera monitoring system 100 can quickly search for and output as a video for broadcasting a video taken by a camera installed in the area where the earthquake occurred. Is possible. At this time, since the shake is detected from the camera image, it is possible to accurately select the image of the camera installed in the place where the shake is large irrespective of the epicenter.

It is a block diagram which shows the structure of the video shake detection apparatus which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the image | video shake detection apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structure of the robot camera monitoring system containing the camera image | video selection apparatus which concerns on embodiment of this invention. It is the schematic diagram which expressed the fluctuation of a picture with the motion of the motion vector of a block (macroblock) for every picture which constitutes a picture. It is the conceptual diagram which showed notionally the motion of the motion vector in one macroblock from generation | occurrence | production to a stop of a video | video. It is a schematic block diagram of the conventional robot camera monitoring system.

Explanation of symbols

10 Image shake detection device (image shake detection means)
11 motion vector extraction means 12 shake detection means 12a shake start detection unit (swing start detection means)
12b Shake stop detector (swing stop detector)
12b1 motion vector accumulation unit (motion vector accumulation means)
12b2 Maximum shaking value measuring unit (Maximum shaking measuring means)
20 Camera image selection device 21 Camera image receiving means 22 Storage means 23 Image selecting means

Claims (7)

A time-series image constituting a video is divided into blocks of a specific size, and the video is shaken with respect to a shooting target from an encoded video obtained by encoding the video based on a motion vector between the blocks every time. An image shake detection device for detecting,
Motion vector extraction means for extracting the motion vector from the encoded video every time;
Based on the magnitude of the motion vector extracted by the motion vector extraction means, a shake detection means for detecting the shake of the video,
An image shake detection apparatus comprising: The shake detecting means compares the magnitude of the motion vector extracted by the motion vector extracting means with a predetermined magnitude, thereby detecting the start of shaking of the video,
After detecting the start of shaking of the video by the shaking start detecting means, the stop of shaking of the video is detected by comparing the magnitude of the motion vector extracted by the motion vector extracting means with a predetermined magnitude. Shaking stop detection means;
The image shake detection apparatus according to claim 1, further comprising: After detecting the start of shaking of the video by the shaking start detecting means, comprising motion vector totaling means for totaling the motion vectors extracted by the motion vector extracting means every time,
The shaking stop detection means compares the magnitude of the motion vector accumulated by the motion vector accumulation means and the magnitude of the motion vector extracted by the motion vector extraction means with a predetermined magnitude, respectively. The video shake detection apparatus according to claim 2, wherein stoppage of video shake is detected.   It is provided with a shake maximum value measuring means for measuring a shake maximum value at which the magnitude of the motion vector extracted by the motion vector extracting means is maximum between the start and stop of the shake of the video. The image shake detection apparatus according to claim 2 or 3. A time-series image constituting a video is divided into blocks of a specific size, and the video is shaken with respect to a shooting target from an encoded video obtained by encoding the video based on a motion vector between the blocks every time. An image shake detection method for detecting,
A motion vector extracting step of extracting the motion vector from the encoded video every time;
A shake start detection step for detecting the start of the shake by comparing the magnitude of the motion vector extracted in this motion vector extraction step with a predetermined magnitude;
After detecting the start of the shaking in this shaking start detection step, a motion vector accumulation step for accumulating the motion vectors every time,
The shaking that detects the stop of the shaking by comparing the magnitude of the motion vector accumulated in the motion vector accumulation step and the magnitude of the motion vector extracted in the motion vector extraction step with a predetermined magnitude, respectively. A stop detection step;
An image shake detection method comprising: A time-series image constituting a video is divided into blocks of a specific size, and the video is shaken with respect to a shooting target from an encoded video obtained by encoding the video based on a motion vector between the blocks every time. Computer to detect,
Motion vector extraction means for extracting the motion vector from the encoded video every time;
A shake start detecting means for detecting the start of the shake by comparing the magnitude of the motion vector extracted by the motion vector extracting means with a predetermined magnitude;
A motion vector accumulating means for accumulating the motion vectors every time after the start of the shaking is detected by the shaking start detecting means;
The shaking that detects the stop of the shaking by comparing the magnitude of the motion vector accumulated by the motion vector accumulating means and the magnitude of the motion vector extracted by the motion vector extracting means with a predetermined magnitude, respectively. Stop detection means,
An image shake detection program characterized by functioning as A coded video obtained by dividing a time-series image constituting a video shot by a camera installed in each area into blocks of a specific size and encoding the video based on a motion vector between the blocks for each time. From the camera image selection device that selects and outputs the image of the camera installed in the area where the earthquake occurred,
Camera video receiving means for receiving the encoded video for each camera via a communication line;
Based on the motion vector included in the encoded video received by the camera video receiving means, video shake detection means for detecting the shake of the video with respect to the shooting target;
Storage means for storing for each camera the encoded video received by the camera video receiving means and the shake information indicating the shake of the video detected by the video shake detection means;
Based on the shake information stored in the storage means, video selection means for selecting an encoded video that is a video of the camera in which the shake has occurred,
A camera image selection device comprising:

Images