JP2006081053A - System for supporting trace of mobile body - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for supporting trace of a mobile body at a low cost by avoiding highly dense arrangement of cameras. <P>SOLUTION: The system for supporting trace of the mobile body for supporting a user using a plurality of cameras to trace the mobile body includes: a plurality of cameras for respectively photographing individual photographing areas; a display means 34 for displaying photographed images of two or more cameras among a plurality of the cameras; a camera selection means 32 whereby a user can select a first camera for photographing the mobile body; a connection relation storage means 38 for storing a connection relation among the photographing areas of the cameras; and a display photographed video image determining means 50 that displays photographed images of one or more second cameras for photographing the photographing areas in a connection relation to the photographing area of the first camera on the display means 34 together with the photographed video image of the first camera by referring to the storage contents of the connection relation storage means 38 when the first camera is selected by way of the camera selection means 32. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a moving body tracking support system that supports a user who tracks a moving body that moves in a predetermined monitoring space using a plurality of cameras.

As a moving body tracking system that uses a plurality of cameras and moves in a predetermined monitoring space, for example, for tracking a moving body such as a car or a person, there is, for example, the one of Non-Patent Document 1. This moving body tracking system has a plurality of cameras that shoot each of a plurality of predetermined spaces in a predetermined monitoring space, and first, the characteristics of the moving body that is the tracking target shown in the captured video of the camera (Feature amount) is extracted by image processing. When a moving body appears in the captured image of the camera (when it is projected), it is automatically recognized by image processing that the moving body is a tracking target moving body based on the feature amount. By automatically displaying the video in which the tracking target moving body is recognized on the display device, the user can track the tracking target moving body by viewing the displayed video.
Kenji Kitamura and two others, “Distributed Video Surveillance System” measurement technology, Vol. 25, no. 7, pp. 11-15, 1997

  However, in the above-described moving body tracking system, the image processing for recognizing the moving body shown in the video based on the feature amount as the tracking target moving body requires a large amount of calculation. An expensive computing device was required. In addition, in order to perform image processing, it is necessary to overlap the space taken by the camera, and therefore, it is necessary to arrange many cameras at high density, and as a result, the moving body tracking system is expensive. It was.

  Therefore, the present invention is a system that supports user tracking that enables interactive processing with a user, that does not require an expensive computing device and that does not require dense camera arrangement. The purpose is to provide.

The mobile tracking support system of the present invention is a mobile tracking support system that supports a user who tracks a mobile using a plurality of cameras.
Multiple cameras that shoot each individual shooting area,
Display means for displaying video images taken by two or more of a plurality of cameras;
Camera selection means for enabling the user to select a first camera for photographing the moving body;
A connection relationship storage means for storing the connection relationship between the shooting areas of each camera;
When the first camera is selected via the camera selection means, the connection relation storage means is referred to, and one or more second images for photographing the photographing area connected to the photographing area of the first camera. And a display photographed video determining means for displaying the photographed video of the first camera together with the photographed video of the first camera on the display means.

  In the present application, the connection relationship is defined as follows. There are imaging areas A and B, and the imaging areas A and B have a connection relationship. In a tracking area having a plurality of imaging areas, a moving object that has passed the imaging area A passes through another imaging area. This is a case where the user can move to the shooting area B without doing.

  According to the present invention, it is possible to provide a moving body tracking support system that can track a moving body in cooperation with a user without the need for an expensive computing device and the need for densely arranging cameras. it can.

Embodiment 1 FIG.
FIG. 1 shows a configuration diagram of a mobile tracking support system according to Embodiment 1 of the present invention. The moving body tracking support system 10 supports a user (monitorer) who monitors a predetermined monitoring area, for example, a road or a passage. However, the moving body tracking support system of the present invention is not limited to monitoring a moving body, and can also be used to observe the state of movement of a moving body (such as a movement path and a state of movement) For example, it can be used for observation of microorganisms that transform while moving.

  The moving body tracking support system 10 includes a plurality of cameras 12-1 to 12-M (M is an integer of 2 or more) that captures each of a plurality of predetermined areas within a predetermined monitoring area, and a plurality of networks via the network 14. Mobile body tracking support server 16 connected to the cameras 12-1 to 12-M.

  The plurality of cameras 12-1 to 12 -M convert a captured video into video data (not shown), for example, convert video into MPEG-2 / 4 (Motion Picture Expert Group Phase 2/4) data. The converted video data is always transmitted to the network 14. Further, information on the time (year / month / day and time) when the video of the video data was taken is added to the transmitted video data.

  Each of the plurality of cameras 12-1 to 12-M is arranged at a position where a predetermined area within a predetermined monitoring area can be photographed. Hereinafter, the predetermined area is referred to as an imaging area. The imaging areas captured by the plurality of cameras 12-1 to 12 -M are classified into cases where they have a connection relationship with each other and cases where they do not have a connection as described below.

  The connection relationship will be described with an example. FIG. 2 is an example of a monitoring area having a plurality of imaging areas for understanding the connection relationship. The monitoring area 20 in the figure has a one-lane road 24 having a median strip 22. In the monitoring area 20, each of the plurality of cameras C1 to C7 images the corresponding imaging areas A1 to A7 on the road 24 (for example, the camera C1 images the imaging area A1). The connection relationship will be described by taking the shooting area A2 shot by the camera C2 as an example. The shooting areas A1 and A5 are connected to the shooting area A2. The shooting areas A3 and A4 are adjacent to the shooting area A2, but they are not connected to the shooting area A2. That is, the moving bodies (cars) that have passed through the shooting area A2 may move next in the shooting areas A1 and A5. On the other hand, the moving body that has passed through the imaging area A2 does not move to the imaging area A3 or A4 on the opposite lane through the central separation band 22 without passing through another imaging area.

  For confirmation, for example, a tracking target moving body that has passed through the imaging area A6 (hereinafter, the tracking target moving body is referred to as a “tracked object”) goes out of the monitoring area 20 and is monitored again. When entering the area 20 and moving to the shooting area A7, it is assumed that the shooting areas A6 and A7 have no connection relationship.

  For further confirmation, the plurality of cameras 12-1 to 12-M are not necessarily arranged in the monitoring area, and the imaging areas of the plurality of cameras 12-1 to 12-M are in the monitoring area. Only. For example, a camera arranged outside the monitoring area may photograph a certain space in the monitoring area.

  In this embodiment, an area having a plurality of imaging areas is a monitoring area to be monitored, but in a broad sense, a tracking area in which a user tracks a moving object. Is valid.

  The connection relationship information indicating the connection relationship of the imaging areas captured by the plurality of cameras 12-1 to 12-M is stored in the connection relationship storage unit of the moving body tracking support server described later.

  FIG. 3 is a diagram showing a schematic configuration of the moving object tracking support server 16 according to the present embodiment. The mobile body tracking support server 16 generally includes a video storage unit 30 that stores video data transmitted from a plurality of cameras 12-1 to 12-M, an input unit 32 for inputting a command by a user, and a captured video of the camera. And a control unit 36 for controlling the entire mobile tracking support server 16.

  In addition, the mobile body tracking support server 16 includes information used when supporting tracking of a tracked object by a user, a connection relationship storage unit 38, a camera information storage unit 40, a monitoring area information storage unit 42, And a tracking result information recording unit 44 in which tracking result information is recorded.

  The video storage unit 30 receives and stores (records) video data from each of the plurality of cameras 12-1 to 12 -M transmitted via the network 14. The video data is recorded by, for example, endless recording (a recording method that erases the oldest video data when new video data is recorded). The video storage unit 30 sends the recorded video data to the control unit 36 based on a read command from the control unit 36. The video data sent to the control unit 36 is not limited to those recorded in the past. Real-time video data from each of the plurality of cameras 12-1 to 12 -M is recorded in the video storage unit 30 and may be transmitted via the video storage unit 30. The user instructs the control unit 36 via the input unit 32 whether the captured video displayed on the display unit 34 is real-time or past.

  The input unit 32 is connected to the control unit 36, and controls a user's selection display command (other commands will be described later) for displaying the captured image of the camera selected by the user on the display screen of the display unit 34. This is input to the unit 36 (in other words, the input unit functions as a camera selection unit that allows the user to select a camera that captures the tracked object), and the input unit is, for example, a keyboard or a mouse. Moreover, you may comprise with a touchscreen.

  The display unit 34 is connected to the control unit 36 and is, for example, a liquid crystal display.

  The control unit 36 includes a display control unit 46 that controls display on the display unit 34 and a camera selection history creation unit 48 that creates a camera selection history.

  The display control unit 46 includes a display shooting image determination unit 50 that determines a shooting image to be displayed on the display unit 34, and a tracking support information display control unit 52 that displays information for supporting tracking of the user's tracked object on the display unit 34. Have.

  Hereinafter, details of the display control unit 46, the camera selection history creation unit 48, the display shooting video determination unit 50, and the tracking support information display control unit 52 in the control unit 36 will be described with reference to the display screen of the display unit 34.

  FIG. 4 is a diagram showing a display screen of the display unit 34. Based on a selection display instruction via the user input unit 32, the displayed captured video determination unit 50 is transmitted from the captured video (a plurality of cameras 12-1 to 12 -M) of the camera (first camera) selected by the user. Real-time video data or a video data video captured in the past recorded in the video storage unit 30) 60 is displayed on the display screen 62 of the display unit 34. In parallel with this, the display photographed video determination unit 50 refers to the connection relationship storage unit 38 and includes one or more cameras (second sensors) having a shooting region connected to the shooting region of the first camera. The captured image 64 of the camera) is displayed on the display screen 62 of the display unit 34. Note that the number of captured images 64 of the second camera displayed on the display screen 62 is determined by the number of captured areas (number of second cameras) connected to the captured area of the first camera. At this time, the shooting time of the shooting image of the first camera and the shooting image of the second camera at the same shooting time are displayed. The shooting time is also displayed on the shooting time counter 66 of the display screen 62 of the display unit 34.

  The tracked object disappears from the captured image 60 of the first camera by the user (becomes a non-imaged state), and one of the captured images 64 of the second camera is moved to the tracked moving object. When the second camera that captures the confirmed captured image is selected via the input unit 32, the display captured image determination unit 50 is displayed. Determines the selected second camera as the new first camera, and regards the captured video as that of the new first camera, and the captured video display position of the first camera on the display screen 62 of the display unit 34. To display. Subsequently, referring to the connection relationship storage unit 38, a camera that captures an imaging area connected to the imaging area of the new first camera is set as a new second camera, and the imaging of the new second camera is performed. The video is displayed together with the new video captured by the first camera at the video captured video display position of the second camera on the display screen 62 of the display unit 34.

  In this way, the display photographed video determination unit 50 selects a new first camera from the photographed video of the second camera by the user, and creates a new photographed video of the second camera based on the new first camera. Is repeatedly displayed on the display screen 62 of the display unit 34, that is, by switching the camera based on the user's instruction, the user's tracking of the tracked object is supported.

  The tracking support information display control unit 52 shows a map 70 of a predetermined monitoring area as one of various pieces of information (hereinafter referred to as tracking support information) for supporting the tracking of the tracked object of the user. (See FIG. 4). The surveillance area map 70 shows the positions of the imaging areas (or the arrangement positions of the plurality of cameras 12-1 to 12-M) taken by the plurality of cameras 12-1 to 12-M. Such map information is included in the monitoring area information, and the monitoring area information is stored in the monitoring area information storage unit 42. The monitoring area information is not only mere map information, but also numerical information such as distances between a plurality of imaging areas, or road traffic regulation information (such as speed limit and one-way traffic) when the monitoring area is a road. Etc. are included.

  Further, the tracking support information display control unit 52 can display an image of the tracked object (tracked object image) 72 determined by the user on the display screen 62 of the display unit 34 as one of the tracking support information. It is. As a result, the user shoots the photographic video 64 in which the tracked object is shown by referring to the tracked object image 72 without making a mistake even if a plurality of moving objects are shown in the photographic video. A second camera can be selected. Such a tracked object image 72 is referred to as a tracking target reference image 72.

  Further, the tracking support information display control unit 52 may first appear in the captured image 64 of the second camera displayed on the display screen 62 of the display unit 34 as one of the tracking support information. The position is configured to have a mark 74 (for example, an arrow).

  When a plurality of captured images 64 of the second camera are displayed on the display screen 62 of the display unit 34, the tracked object disappears from the captured images 60 of the first camera, and one of the captured images 64 of the plurality of second cameras is displayed. When the tracked object appears, the tracked object may disappear from one of the captured images 64 of the plurality of second cameras without being confirmed by the user. This is because the user cannot see all of the captured images 64 because there are a plurality of captured images 64 of the second camera, and from where the tracked object appears in the captured images 64 of the second camera. This is because there is not.

  If the tracked object disappeared from the captured image 60 of the first camera has a mark 74 at a position where it can appear first in the captured images 64 of the plurality of second cameras, the user captures images of the plurality of second cameras. It is only necessary to see the vicinity of the mark 74 in the video 64, and the possibility of missing the tracked object is reduced. Hereinafter, such a mark 74 is referred to as an appearance location specifying mark 74.

  The appearance location specifying mark 74 differs depending on the position of the second camera and the shooting direction. This will be described with reference to FIG. FIG. 5 shows a schematic model of a straight road 76 having a plurality of cameras C10 to C12 and photographing areas A10 to A12 photographed by the plurality of cameras C10 to C12. Fig.5 (a) is the figure which looked at the straight road 76 from the side surface, and FIG.5 (b) is the figure which looked at the straight road 76 from upper direction. The plurality of cameras C <b> 10 to C <b> 12 are disposed at substantially equal distances from the straight road 76 above the center line 78 in the width direction of the straight road 76. The cameras C10 and C12 are arranged at substantially the same position with respect to the respective shooting areas A10 and A12 (the shooting directions 80-10 and 80-12 are substantially the same), and the shooting direction 80-11 of the camera C11 is the same. The imaging directions 80-10 and 80-12 of the cameras C10 and C12 are substantially symmetrical with respect to the extending direction of the center line 78 in the width direction.

  Here, when the camera C10 is the second camera, the appearance location specifying mark 74 differs depending on whether the first camera is C11 or C12. When the first camera is C11, the moving body M1 that has passed through the shooting area A11 of the camera C11 appears from above the shot video of the second camera and is reflected in the shot video. On the other hand, when the first camera is C12, the moving body M2 that has passed through the shooting area A12 of the camera C12 appears from below the shot video of the second camera and is reflected in the shot video.

  Thus, the appearance location specifying mark 74 is determined by the arrangement position and the shooting direction of the second camera together with the first camera. The tracking support information display control unit 52 refers to the connection relationship storage unit 38 and also refers to the camera information storage unit 40 that stores information related to the camera placement position and shooting direction, and the shooting area of the first camera. An appearance location specifying mark 74 is attached to a position where the tracked object in the track can appear in the captured image 64 of the second camera when it moves into the imaging region of the second camera.

  In addition, the tracking support information display control unit 52 refers to the monitoring area information storage unit 42 as one of the tracking support information, and is selected from the captured video 60 of the first camera among the plurality of second cameras. The tracking object disappears, and a mark 82 (for example, a frame-shaped mark) is added to the captured image 64 of the second camera that captures the imaging region where the tracked object is likely to move next. It may be configured.

  For specific description, FIG. 2 is used. In the road 24 shown in the figure, it is assumed that the moving body (car) travels in the arrow direction (traveling direction) 26. In this case, if attention is paid to the shooting area A2, the shooting areas A1 and A5 are connected to the shooting area A2. The moving object that has disappeared from the imaging area A2 is most likely to move next in the imaging area A5 ahead of the traveling direction 26. The movement to the photographing area A1 is a case where the moving body runs backward, and the possibility is low.

  In this way, with reference to the monitoring area storage unit 42, in the plurality of second cameras, the tracked object may disappear from the captured image 60 of the first camera, and the tracked object may move next. By attaching a mark 82 to the shot image 64 of the second camera that is shooting a high shooting area, the user puts weight on the second shot image 64 with the mark 82 and places a plurality of second cameras. Can be seen. Hereinafter, such a mark 82 is referred to as an appearance candidate mark 82.

  When the monitoring area is, for example, a road, the speed of the tracked body (moving body) is limited. From this speed limit (included in the monitoring area information) and the distance (included in the monitoring area information) between the imaging area of the first camera and each of the imaging areas of the plurality of second cameras, The appearance time until the tracked object disappears from the captured image of the first camera and the tracked object appears in the captured image of the second camera can be predicted. Using this predicted appearance time, the appearance candidate mark is attached only to, for example, a captured image of the second camera in which the tracked object may appear first in the captured images of the plurality of second cameras. May be. Further, every time when the tracked object disappears from the video captured by the first camera and the time elapses, the video captured by the second camera to which the appearance candidate mark is attached may be changed. This is because an expected appearance time elapses until the tracked object appears in the captured image of the second camera, to which the tracked object appears first, among the captured images of the plurality of second cameras with the appearance candidate mark attached. After that, it indicates that the tracked object has not moved to the shooting area of the second camera, and means that an appearance candidate mark is attached to the shot image of the second camera where the tracked object can appear. .

  The display control unit 46, for example, moves the playback speed adjustment button 84 displayed on the display screen 62 of FIG. 4 by the user based on the user's command such as change of playback speed or rewind playback. Is clicked via the button, the playback speed of the shot video (the shot video of the first camera and the second camera) is increased or delayed and displayed on the display screen 62 of the display unit 34, or in rewind playback. It is configured to display.

  Further, the display control unit 46 is configured to display the captured images of the plurality of cameras 12-1 to 12 -M on the display screen 62 of the display unit 34. For example, when there are nine cameras, as shown in FIG. 6, the captured images 85-1 to 85-9 of the nine cameras are displayed on the display screen 62 of the display unit 34. As described above, in order to select the first camera from the second cameras as described above, first, the first camera (first first camera) must be determined first (first first camera). Therefore, the second camera is not determined unless the first camera is determined.) For this purpose, the captured images of the plurality of cameras 12-1 to 12-M are displayed on the display screen 62 of the display unit 34, and the first first is shown to the user. This is to select a camera.

  Returning to FIG. 3, the camera selection history creation unit 48 creates and records the selection history of the first camera as camera selection history information when tracking the tracked object. The imaging region through which the tracked object has passed is known from the selection history of the first camera, and as a result, the trajectory of the tracked object in the predetermined monitoring region can be identified. The camera selection history information created by the camera selection history creation unit 48 is added to the tracking result information stored in the tracking result information storage unit 44.

  The mobile body tracking support server 16 is configured to end tracking based on an instruction to end tracking via the user input unit 32. For example, the tracking displayed on the display screen 62 of FIG. When the completion button 86 is clicked via the mouse, the display of the captured image of the second camera by the display captured image determination unit 50 is terminated.

  Note that buttons and display images that are not described in the display screen 62 of the display unit 34 illustrated in FIG. 4 will be described in an embodiment described later.

  Next, a tracking support process performed by the mobile body tracking support server 16 while performing a dialog process with the user will be described along a processing flow shown as an example in FIG.

  First, in step S <b> 10, the control unit 36 confirms the input of the first camera selection command via the user input unit 32. When the input of the first camera selection command is confirmed, the process proceeds to step S20. As a first camera selection command by the user in Step 10, the user causes the display unit 34 to display the captured images of the cameras 12-1 to 12-M on the display screen 62 as shown in FIG. Then, a selection command for selecting a camera that refers to the captured image as the first camera is performed. Instead, there is also a selection command for selecting a camera that is shooting a certain area as the first camera, for example, when the user is notified that the suspicious moving body is moving in a certain area. Until the input of the first camera selection command is confirmed, the control unit 36 waits for execution of the tracking support process.

  In step S20, the displayed captured video determination unit 50 displays the captured video 60 of the first camera selected by the user in step S10 on the display screen 62 of the display unit 34, and refers to the connection relationship storage unit 38. The second camera is determined, and the captured video 64 is displayed on the display screen 62 of the display unit 34.

  In step S30, the tracking support information display control unit 52 refers to the camera information storage unit 40 and the monitoring area information storage unit 42 based on the second camera determined in step S20, and displays the tracking support information display unit. 34 on the display screen 62.

  In step S <b> 40, the control unit 36 inputs a new first camera selection command that the user has selected through the input unit 32 by selecting a new first camera from the captured images 64 of the plurality of second cameras. Confirm. When the input of a new first camera selection command is confirmed, the process proceeds to step S50. The control unit 36 repeats step S40 until the input of the first camera selection command is confirmed.

  In step S 50, the camera selection history creation unit 48 creates new first camera selection history information, and the created new first camera selection history information is stored in the tracking result information storage unit 44. Added to the tracking result information.

  In step S60, the display shooting video determination unit 50 displays the shooting video of the new first camera selected by the user in step S40 on the display screen 62 of the display unit 34, and refers to the connection relationship storage unit 38. Then, a new second camera is determined, and the captured video is displayed on the display screen 62 of the display unit 34.

  In step S70, the tracking support information display control unit 52 refers to the camera information storage unit 40 and the monitoring area information storage unit 42 based on the new second camera determined in step S60, and provides tracking support information ( For example, an arrow 74 or a frame-shaped mark 82) is displayed on the display screen 62 of the display unit 34.

  In step S <b> 80, the control unit 36 confirms the input of the tracking completion command via the user input unit 32. If the input is confirmed, the control unit 36 completes the tracking. If the input is not confirmed, the process returns to step S40 to continue the tracking support.

  According to the present embodiment, a mobile object tracking support system that tracks a mobile object in cooperation with a user (in support of the user), which does not require an expensive computing device and does not require a dense arrangement of cameras. Can be provided. In addition, since the burden on the user who confirms the captured image of the camera on which the tracking target moving body is displayed is reduced, the user can easily track the tracking target moving body.

  Hereinafter, the flow of support for tracking a user's moving object performed by the moving object tracking support server 16 will be described with reference to an example.

  The moving body tracking support server 16 described here is configured by incorporating dedicated software into a computer (personal computer), and displays a captured image of two or more cameras 12-1 to 12-M. Is displayed on a display screen of a display (hereinafter referred to as a display 34).

  In addition, the moving body tracking support server 16 displays a plurality of command buttons to which commands relating to tracking of a moving body instructed by the user are assigned on the display screen of the display 34 and a pointer that moves in conjunction with the mouse (mouse A pointer) is displayed on the display screen of the display 34. The input unit 32 includes a mouse, command buttons, and a mouse pointer. In the moving body tracking support server 16, the user moves the mouse pointer onto the command button via the mouse on the display screen of the display 34, and the decision button (not shown) provided on the mouse is clicked there. A command relating to tracking of the moving object assigned to the command button is executed. The command assigned to the command button will be described later. The input unit 32 also includes a keyboard for inputting sentences and numbers.

  In the embodiment described here, each of the plurality of cameras A to H captures a predetermined shooting area on the road and uses the past shot video (video data) stored in the video storage unit 30 to A truck (tracked object) whose luggage has been dropped on the road in the shooting area A is tracked. A display screen of the display 34 displayed according to the flow of the user's moving body tracking support performed by the moving body tracking support server 16 is shown in the figure.

  First, when activated by the user, the moving body tracking support server 16 is captured by the plurality of cameras 12-1 to 12 -M, or is captured and stored in the video storage server 16. Display past video. Whether to display the video to be displayed, that is, the current video or the past video, is determined, for example, by inputting a time when the user wants to view to the shooting time counter 108 using the keyboard. The shooting time counter 108 indicates the shooting time of the displayed shooting image, and allows the display control unit 46 to find the portion of the shooting image at the desired shooting time by inputting the shooting time desired by the user. It is.

  When the user instructs the moving body tracking support server 16 via the input unit 32 to display the past shot video of the camera A, the past shot video of the camera A becomes the shot video of the first camera on the display 34. Displayed on the display screen.

  In FIG. 8A, the display shooting video determination unit 50 displays the shooting video of the camera A at the shooting time desired by the user, and the second camera B, A display screen 100 of the display 34 on which captured images of C, D, and E are displayed is shown.

  In order to distinguish between the photographed image 110 of the first camera and the photographed image 112 of the second camera, the display photographed image determining unit 48 uses the second camera to capture the photographed image 110 of the camera A, which is the first camera. It is displayed larger than the captured image 112 of a certain camera B, C, D, E. In the upper left corner of the display screen 100, the tracking support information display control unit 52 displays a monitoring area map 114 as monitoring area information, and the map 114 shows the positions of the plurality of cameras A to H. Has been. In addition, on the display screen 100, the captured video 112 of the second cameras B, C, D, and E is given an appearance location identification mark 116 by the tracking support information display control unit 52.

  Also, the display screen 100 shows a plurality of command buttons 120, 122, 124, 126, 128, and 130 that are clicked by a mouse pointer 118 that moves on the display screen 100 in conjunction with the mouse of the input device 44. Yes. Hereinafter, for the sake of omission, an operation in which the user moves the mouse pointer 118 over the button with the mouse and clicks the operation button by clicking the determination button of the mouse is simply expressed by the user clicking the command button. .

  The playback speed adjustment button 120 is a button that is clicked when the playback speed of the displayed captured video is adjusted. A plurality of captured images (captured images of the first camera and the second camera) displayed on the display screen 100 are synchronously reproduced by the display control unit 46, and the reproduction speed is adjusted synchronously. When the playback speed adjustment button 120 is clicked, the display control unit 50 performs fast-forward playback, slow playback, reverse playback, frame-by-frame playback, and still playback of the captured video. The user tracks the track shown in the captured image at a playback speed that is easy to track.

  When the user decides to track the track shown in the captured image 110 of the camera A of the first camera, the user presses the appearance button 124. When the appearance button 124 is pressed, the camera selection history creation unit 48 creates camera selection history information related to the track to which the information of the camera A and the information of the shooting time when the appearance button 124 is clicked are added.

  Next, the user displays an image of the track that is the tracking target as tracking target reference information. FIG. 8B is a display screen 100 in which the track image 132 is displayed in the tracking target reference information display area 134. First, the user stops the reproduction of the captured video using the reproduction speed adjustment button 120. The image of the track is captured (captured) from the captured image 110 of the camera A of the first camera whose reproduction is stopped. Capturing is performed by pressing the record button 122, enclosing a rectangular frame 138 with the mouse pointer 118 on the image of the track shown in the captured image of the first camera, and pressing the determination button of the mouse. The captured image of the track is displayed in the tracking target reference information display area 134 as a track image 132. Thereafter, the user tracks the track while referring to the image 132 of the track.

  As shown in FIG. 9C, when the track image displayed on the captured image 110 of the camera A of the first camera disappears from the captured image 110 by the tracking support information display control unit 52, An appearance candidate mark 140 is attached to the captured image 112 of the second camera that is highly likely to appear. The appearance candidate mark 140 is, for example, a frame that surrounds the captured image 112 of the second camera. Here, the appearance candidate mark 140 surrounds the captured images 112 of the cameras D and E.

  FIG. 9D shows the display screen 100 after the track image disappears from the captured image 110 of the first camera. A camera-like image is displayed on the camera E, which is the second camera, but at this stage, the user cannot confirm that the truck-like image is the track to be tracked.

  FIG. 10E shows a display screen 100 when an image captured by the camera E of the second camera is enlarged. The enlarged display of the photographed image 112 of the second camera is executed by moving the mouse pointer 118 over the photographed image to be magnified and pressing the enter button on the mouse. The user temporarily replaces the captured image of the first camera with the captured image 112L of the camera E of the second camera which is enlarged and displays the track displayed on the camera E of the second camera. Confirm that the track is the track target.

  As shown in FIG. 10 (f), after confirmation, when the user presses the appearance button 124, the display shooting video determination unit 50 determines that the camera E is a new first camera. A new video image 110 ′ of the first camera is obtained. In addition, the display shooting video determination unit 50 determines that the cameras A, D, G, and F that are shooting the shooting areas connected to the shooting area of the camera E are new second cameras, and the cameras A and D , G, F shot images become the new second camera shot images 112 ′. Further, the camera selection history creation unit 48 adds information on the shooting time when the camera E and the appearance button 124 are clicked to the camera selection history information regarding the track.

  As described above, the user selects a new first camera from the captured image of the second camera, and further adds a new one from the captured image of the second camera newly displayed based on the new first camera. Repeat the selection of the first camera, i.e., track the track by switching cameras.

  The user clicks the track completion button 128 to complete track tracking. When the tracking completion button 128 is clicked, a window 142 showing tracking result information about the track is displayed as shown in FIG. The window 142 displays the time when the shot video showing the track was shot, the file name of the tracking result information regarding the track, the user's comment, the camera selection history information, the track image 132, and the like. In addition, a save confirmation button 144 is displayed to ask the user whether to save the tracking result information regarding the track as a file or whether to save the track according to the contents of the information 142. When saving the tracking result information regarding the track, the user clicks a “Yes” button of the save confirmation button 144. When changing the file name displayed in the window 142 showing the tracking result information regarding the track or adding a user comment, the user changes the file name or adds a comment using a keyboard, for example. The tracking result information regarding the track is stored in the additional result information recording unit 44.

  When the user browses the stored tracking result information, the user clicks the tracking history playback button 130 (see FIG. 8A). When the tracking history reproduction button 130 is clicked, the display control unit 46 displays a list of tracking history information 146 relating to a plurality of stored moving objects, as shown in FIG. When the mouse pointer 118 is moved over the tracking history information to be browsed and the determination button of the mouse is pressed, the tracking history information is displayed.

  The tracking history information is displayed on the display screen 100 as shown in FIG. Detailed information 148 added to the tracking history information is displayed at the top of the display screen 100. In addition, information 150 indicating the switching history of the cameras along the time series is displayed. In addition, a tracking image 152 is displayed in which captured images captured by a plurality of cameras during tracking (captured images of a plurality of cameras selected as the first camera) are continuously displayed.

  Further, when browsing the tracking history information, new command buttons (skip button 154, mode button 156, pass button 158) are displayed, and it becomes possible to execute “skip playback” and “mode playback” of tracking video 152. .

  “Skip playback” is for skipping and playing back the shot video during the shooting time when the skip button 150 is kept pressed in the tracking video 152, for example.

  The “mode playback” is a mode in which the user once views the tracking video 152 and plays back a trimmed (deleted) portion of the tracking video 152 where a moving object to be tracked is not shown. For example, while the tracking video 152 is being played back, the tracked object disappeared from the video captured by a certain first camera (one of the video captured by the first camera included in the tracking video 152 and selected by the first camera). When the pass button 158 is clicked by the user who has confirmed that, the reproduction of the captured image of a certain first camera is finished, and the captured image of the next first camera is reproduced. At this time, the tracking image 152 is changed from the portion of the captured image of the first camera that was reproduced when the pass button 158 was clicked to the first portion of the captured image of the first camera (the first first image). The image is cropped up to the last part of the camera image. By repeating this trimming process, a portion of the tracked video 152 where the tracked object is not projected is trimmed. After the trimmed tracking video 152 is saved as a new file (or before saving), when the mode button 156 is clicked, the trimmed tracking video 152 is reproduced. The tracked object continues to be reflected in the trimmed tracking image 152, and the user can clearly see the movement of the tracked object.

  For example, when the tracked object disappears from the first camera X, the portion in which the tracked object is not reflected in the tracking image 152 occurs before the tracked object appears in the second camera Y. This is because it may take time because the shooting area of the camera X and the shooting area of the camera Y are separated, and during this time, the camera X is shooting a state in which the tracked object does not exist. Until the tracked object appears in the camera Y and the user clicks the appearance button 124, information on the camera X that captures the state where the tracked object does not exist remains in the camera selection history.

  In the first embodiment, the track after the load is dropped is tracked, but the traveling of the track before the load is dropped can be traced in the reverse direction. Is possible by reverse playback. In addition, on the display screen, the video image captured by the first camera and the video image captured by the second camera are displayed in different sizes. However, the present invention is not limited to this. For example, a high resolution XGA or SXGA In the case of a display, the captured image of the first camera and the captured image of the second camera may be displayed in the same size. Furthermore, each captured image may be arranged on the display screen so that the positions of the first camera and the second camera can be intuitively understood. For example, a map of the monitoring area may be shown on the background screen, and captured images of a camera that is capturing a plurality of imaging areas may be displayed at a plurality of imaging area positions on the map. In addition, on the display screen, the layout of the display such as the shooting area and the tracking support information may be freely changed by the user.

Embodiment 2. FIG.
The moving body tracking support system according to the present embodiment is the same as the moving body tracking support system according to the first embodiment, except that the tracked object shown in the captured image of the first camera is tracked in the shooting area of the first camera. To make it easier. When a plurality of moving objects are shown in the captured image of the first camera, the tracked object is marked (for example, the tracked object is colored or a mark of a specific shape is superimposed on the image of the tracked object). Thus, the user only needs to track the marked moving body, and the load of finding the tracked body from the plurality of moving bodies is reduced.

  The moving body tracking support system according to the present embodiment uses a captured image of the first camera in order to track the tracked object reflected in the captured image of the first camera on the captured image of the first camera. The input unit (moving body specifying means) is configured so that the moving body can be specified by the user on the image frame formed, and the movement of the first camera with respect to the moving body specified through the input unit is moved within the imaging region. The configuration is the same as that of the first embodiment except that a moving body tracking unit that tracks using image processing is provided. Therefore, the input unit and the moving body tracking unit will be described.

  FIG. 13 is a diagram showing a schematic configuration of the mobile tracking support server 316 according to the present embodiment. Except for the input unit 388 and the moving body tracking unit 390 in the control unit 336, it is almost the same as that of the first embodiment, and the reference numeral of the first embodiment is denoted by 300, and the description thereof will be omitted as appropriate.

  The input unit 388 is for the user to specify the tracked object in the captured image of the first camera. The tracked object is specified from an image frame of a past captured video (captured video in the video storage unit 330) that has been stopped by the user via the input unit 388, or an image frame (still video) acquired from a real-time video. The The tracking target is specified by, for example, the user indicating the boundary between the portion of the image of the tracking target and the other portion via the input unit 388 on the still image. Information (designation command) on the designated tracked object is input to the moving object tracking unit 390 of the control unit 336.

  The moving body tracking unit 390 extracts a feature amount (for example, a color component or luminance) from the tracked body designated by the user using the input unit 388 by performing image processing, and performs first processing based on the extracted feature amount. The tracked object is specified for each image frame constituting the video captured by the camera. A method for specifying the moving body based on the feature amount of the moving body is described in detail in Non-Patent Document 1, for example.

  The display control unit 346 displays on the display screen of the display unit 334 a photographed image that is marked (with a tracking mark) on the image of the tracked object specified by the moving body tracking unit 390 for each image frame. As a result, the user appears to move both the tracked object and the tracking mark in the captured image of the first camera.

  When another moving body having a feature amount similar to the feature amount of the tracked object is present on the captured image of the first camera, and the tracked object and another moving body are overlapped in the captured image, A tracking mark may be attached to another moving body. In this case, the user notices that the tracking mark of the tracked object is attached to another moving object in the captured image of the first camera, and the tracking image is attached again to the tracked object. Stop. When the tracked object is designated again by the user, the display control unit 346 is configured to erase the tracking mark attached to another moving object.

  The tracking of the tracked object on the captured image of the first camera is not necessarily performed, and the execution of the tracking is arbitrarily determined by the user. Basically, it can be executed if the tracked object is captured in the image captured by the first camera. Hereinafter, the tracking support process on the captured image of the first camera performed by the moving body tracking support server 316 while performing the interactive process with the user will be described along a processing flow shown as an example in FIG.

  First, in step S110, the control unit 336 confirms that playback of the past or real-time captured video is stopped by the user. If it is confirmed that the captured video is stopped, the process proceeds to step S120.

  In step S <b> 120, the control unit 336 confirms whether or not the user has designated the tracked object on the displayed image frame via the input unit 332, where the captured video is stopped from being reproduced. If it cannot be confirmed that the tracked object is specified, the process proceeds to step S130. If it is confirmed that the designation has been made, the process proceeds to step S150, and tracking support processing on the captured image of the first camera is started.

  In step S130, the input of the reproduction command via the user input unit 332 is confirmed. When the input of the reproduction command is confirmed, the process proceeds to step S140. When the input of the reproduction command is not confirmed, the process returns to step S120.

  In step S140, the display control unit 346 plays back the captured video that has been stopped. Next, the process returns to step S110.

  In step S150, the moving body tracking unit 390 extracts a feature amount from the image of the specified tracked body by image processing.

In step S <b> 160, the control unit 336 confirms the input of a reproduction command via the user input unit 332. When the input of the reproduction command is confirmed, the process proceeds to step S170.

  In step S170, the display control unit 346 plays back the captured video that has been stopped.

  In step S180, the moving body tracking processing unit 390 identifies a tracked object for each image frame of the captured video that is being reproduced, based on the feature amount extracted in step S150.

  In step S190, the display control unit 346 performs marking on the tracked object for each image frame of the captured video that is specified and reproduced in step S180.

  In step S <b> 200, the control unit 336 confirms the input of the playback stop command via the user input unit 332. If there is a reproduction stop command input, the process proceeds to step S210. If no playback stop command is input, the process proceeds to step S220.

  In step S <b> 210, the control unit 336 confirms whether or not the user has designated the tracked object on the displayed image frame via the input unit 332, where the captured video is stopped from being reproduced. If it is confirmed that the tracked object is designated, the process returns to step S150. If it is not confirmed that it has been designated, the process returns to step S160. In step S210, the user notices that the tracking mark of the tracked object is attached to another moving object in the captured image of the first camera, and the tracking mark is attached to the tracked object again. Therefore, it is confirmed whether or not the tracked object is redesignated.

  In step S <b> 220, the control unit 336 connects the new first camera (the second camera that is connected to the imaging region of the first camera that is marking and tracking the tracked object) via the user input unit 332. Confirm the input of the selection command of the new first camera selected from (1). When the input is confirmed, it is determined that the tracked object has disappeared from the captured image of the first camera, and the tracking support process on the captured image of the first camera is terminated.

  If a new first camera selection command input is not confirmed, it is determined that there is a tracked object on the captured image of the first camera, and the process returns to step S180 to perform tracking support processing on the captured image of the first camera. To continue.

  According to the present embodiment, when a plurality of moving objects are shown in the captured image of the first camera, the user only needs to track the marked moving object, and find the tracked object from the plurality of moving objects. The load is reduced.

  Hereinafter, the flow of tracking of the tracked object performed on the captured image of the first camera will be described with an example. Here, the example given trackss a person in a building space where a plurality of cameras T to Z are arranged.

  FIG. 15A shows a display screen 200 of the display device when the camera X is shooting a suspicious person to be tracked live. The display control unit 346 displays a captured image 210 of the camera X on the display screen 200. In addition, a space diagram 214 inside the building, which is monitoring area information, is displayed in the upper left corner of the display screen 200 by the tracking support information display control unit 352. The space diagram 214 includes a plurality of cameras TZ. The placement position is shown. In addition, a plurality of command buttons 220 and 260 are displayed on the display screen 200. Since the command button 220 is the same as the playback speed adjustment button 120 of the first embodiment, a description thereof will be omitted. The tracking button 260 is a button that is pressed when tracking the moving body (suspicious person) 261.

  FIG. 15 (b) is switched when the tracking button 260 is pressed by the user on the display screen 200 of FIG. 15 (a) and the playback speed adjustment button 220 is clicked to stop playback. A display screen 200 is shown. The video captured by the camera X is displayed as the video captured by the first camera, and the cameras V and Z that capture the imaging area connected to the camera X imaging area are the video captured by the second camera. It is displayed as.

  In order to specify the suspicious person 261 to which the tracking mark for tracking the suspicious person 261 is given on the captured image of the camera X of the first camera (within the photographing area of the camera X), the user 261 Is specified. The designation is performed by surrounding the image of the suspicious person 261 shown in the image frame of the photographed video 210 being stopped with a mouse pointer 218, for example, by drawing an oval frame 262, and pressing the determination button of the mouse. As a result, the moving body tracking unit 390 extracts the feature amount of the suspicious person 261 by image processing.

  Subsequently, when the user presses the live monitoring button 264, the captured image of the camera X of the first camera displayed on the display screen 200 is switched to the captured image of the camera X in real time (see FIG. 16C). ). A tracking mark 266 is given by the display control unit 346 to the image of the suspicious person 261 shown in the captured image 210 of the camera X of the first camera.

  The tracking of the actual suspicious person 261 while designating the suspicious person 261 on the image frame of the captured video 210 being stopped may be performed by the video data stored in the video storage unit 340. Alternatively, the user or the display control unit 346 fast-forwards the accumulated shooting video at a speed at which the user or the displayability control unit 346 can track the suspicious person 261 until the shooting time of the real-time shooting video is caught up. Tracking may be performed by switching to a captured video.

  Further, when the image of the suspicious person 261 is designated by, for example, drawing an ellipse with the mouse pointer 218, the tracking support information display control unit 352 uses the enclosed image of the suspicious person 261 as the image 268 of the suspicious person. It may be displayed in the tracking target reference information display area 232.

  As shown in FIG. 16D, when it is confirmed that the suspicious person 261 disappears from the captured image 210 of the camera X of the first camera and the suspicious person 261 appears in the camera Z of the second camera, the user Similarly to the first embodiment, the camera Z is selected as the first camera.

  FIG. 17E shows the case where the camera Z is determined as the first camera by the display shooting video determination unit 350 and the cameras X and Y are determined as the second cameras with reference to the connection relationship storage unit 338. The display screen 200 of the display unit 334 is shown.

  When the user tracks the suspicious person 261 in the shooting area of the camera Z, the user designates the suspicious person 261 as described above. The user uses the playback speed adjustment button 220 to stop the captured video before designation. At this time, when designating the suspicious person 261, the image of the suspicious person 261 specified by surrounding the image of the suspicious person 261 with the mouse pointer 218, for example, by drawing an ellipse, is imaged by the tracking support information display control unit 352. For example, it may be specified by surrounding the head portion where the face of the suspicious person 261 can be seen. In this case, an image 270 of the head portion where the face of the suspicious person 261 can be seen is displayed as another image of the suspicious person 261 in the tracking target reference information display area 232.

  When another person 272 having a feature quantity similar to the feature quantity of the suspicious person 261 exists in the shooting area of the camera Z, and the suspicious person 261 and another person 272 are projected on the shooting image 210 of the camera Z, A tracking mark 274 may be attached to another person 272 (see FIG. 17F). In this case, the user designates the suspicious person 261 as described above so that the tracking mark is attached to the suspicious person 261 again. At this time, the display control unit 346 is configured to erase the tracking mark 274 attached to another person 272.

  Although the tracked object is tracked on the captured image of the first camera, the tracked object may be tracked on the captured image of the second camera. In this case, the shooting area is set so that the shooting area of the first camera and the shooting area of the second camera partially overlap. By doing so, the same tracked object appears on the captured image of the first camera and on the captured image of the second camera. At this time, even if the moving body tracking support system is configured such that when the tracked object is marked on the captured image of the first camera, the tracked object is automatically marked also on the captured image of the second camera. Good. In this case, it is useful when the user confirms whether the tracked object is reflected on the second camera.

  The camera may be capable of transmitting the video data in a state where the tracked object shown in the video of the video data is marked. In this case, a feature amount acquisition unit for the camera to acquire a feature amount for specifying the tracked object from the moving body tracking support server, a mobile object specification unit for specifying the tracked object for each image frame based on the feature amount, And an image conversion unit that attaches a tracking mark to the image of the tracked object on the image frame of the video data. (These components are input when the corresponding camera is determined to be the first camera. And moving body tracking means for tracking the movement of the first camera in the imaging region with respect to the tracked object specified via the unit using image processing. In this case, the moving body tracking unit in the control unit of the moving body tracking support server is omitted.

  The mobile tracking support system of the present invention has been described with reference to two embodiments, but the present invention is not limited to this. In addition, it is easy for those skilled in the art to consider various modifications. For example, while tracking one moving body M, there is a case where it is desired to track another moving body M ′ shown in a captured image at a certain shooting time P of a certain camera C in which the moving body M is shown. In this case, after the tracking of the moving body M is completed, the shooting time of the camera C is set so that the shooting video of the shooting time P of a certain camera C can be quickly found so that another moving body M ′ can be tracked immediately. A cueing signal that functions as a “bookmark” is attached to the P video. The user recognizes the portion of the photographed image with the “bookmark” attached as the portion of the photographed image on which another moving body M ′ is reflected, and the tracking of another moving body M ′ can be immediately performed. .

It is the schematic of the mobile body tracking assistance system of this invention. It is a figure for demonstrating the connection relationship between imaging areas. It is the schematic of the mobile body tracking assistance server which concerns on Embodiment 1 of this invention. It is a figure which shows the display screen of a display part. It is a figure explaining an appearance place specific mark. It is a figure which shows the display screen of the display part which displays the picked-up image of a some camera. It is a flowchart of tracking of a mobile body. It is a figure which shows the display screen displayed on a display part in the case of tracking of the mobile body using the mobile body tracking assistance system of FIG. It is a figure which shows another display screen displayed on a display part in the case of tracking of the mobile body using the mobile body tracking assistance system of FIG. It is a figure which shows the further display screen displayed on a display part in the case of tracking of the mobile body using the mobile body tracking assistance system of FIG. It is a figure which shows another display screen displayed on a display part in the case of tracking of the mobile body using the mobile body tracking assistance system of FIG. FIG. 3 is a diagram showing a display screen displayed on a display unit when tracking a mobile object using the mobile object tracking support system of FIG. 1, and tracking history after completion of tracking of the mobile object using the mobile object tracking support system It is a figure which shows the display screen displayed on a display part in reproduction | regeneration mode. It is the schematic of the mobile body tracking assistance server which concerns on Embodiment 2 of this invention. It is a flowchart of the tracking of the moving body performed on the picked-up image of a 1st camera. It is a figure which shows the display screen displayed on a display part in the case of tracking of the moving body using the moving body tracking assistance system which concerns on Embodiment 2 of this invention. It is a figure which shows another display screen displayed on a display part in the case of tracking of the moving body using the moving body tracking assistance system which concerns on Embodiment 2 of this invention. It is a figure which shows the further display screen displayed on a display part in the case of tracking of the moving body using the moving body tracking assistance system which concerns on Embodiment 2 of this invention.

Explanation of symbols

16 moving body tracking support server, 32 input unit (camera selection unit), 34 display unit (display unit), 38 connection relationship storage unit (connection relationship storage unit), 50 display shooting video determination unit (display shooting video determination unit)

Claims (7)

In a moving body tracking support system that supports a user who tracks a moving body using a plurality of cameras,
Multiple cameras that shoot each individual shooting area,
Display means for displaying video images taken by two or more of a plurality of cameras;
Camera selection means for enabling the user to select a first camera for photographing the moving body;
A connection relationship storage means for storing the connection relationship between the shooting areas of each camera;
When the first camera is selected via the camera selection unit, the connection relationship storage unit is referred to, and one or more second ones that shoot a shooting region that is connected to the shooting region of the first camera. A moving body tracking support system comprising: a display photographing image determining means for displaying a photographing image of the first camera together with a photographing image of the first camera on a display means. In a moving body tracking support system that supports a user who tracks a moving body using a plurality of cameras,
Video storage means for storing video data acquired by a plurality of cameras having individual shooting areas;
Display means for displaying shot images of two or more cameras out of a plurality of cameras based on the video data stored in the video storage means;
Camera selection means for enabling the user to select a first camera for photographing the moving body;
A connection relationship storage means for storing the connection relationship between the shooting areas of each camera;
When the first camera is selected via the camera selection unit, the connection relationship storage unit is referred to, and one or more second ones that shoot a shooting region that is connected to the shooting region of the first camera. A moving body tracking support system comprising: a display photographing image determining means for displaying a photographing image of the first camera together with a photographing image of the first camera on a display means.   3. The moving object tracking according to claim 1, wherein the camera selecting means is configured to allow the user to select the first camera with reference to the captured images of the one or more second cameras. Support system.   The mobile body tracking support system according to any one of claims 1 to 3, further comprising tracking support information display control means for causing the display means to display tracking support information for supporting tracking of a user's mobile body. Camera information storage means for storing information on the arrangement position and shooting direction of each camera;
The tracking support information display control means, based on the information stored in the camera information storage means, moves the second camera when the moving body in the shooting area of the first camera moves into the shooting area of the second camera. 5. The moving body tracking support system according to claim 4, wherein a position that can appear in the captured video is displayed on the display means as tracking support information. A moving object designating means for allowing a user to designate a moving object on an image frame forming a video image taken by the first camera;
The mobile object tracking means for tracking the movement within the imaging region of the first camera related to the mobile object specified through the mobile object specifying means using image processing is further provided. The moving body tracking support system described in 1.   The moving body tracking support system according to any one of claims 1 to 6, further comprising camera selection history creation means for creating a history of selection of the first camera.

Images