JP6721915B1 - Video processing apparatus and video processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily select an image to be reproduced. A video processing device includes a data holding unit that holds a plurality of video data in which environment information indicating an imaging environment is recorded for each predetermined frame, an input unit that inputs search information, and the input search. Setting means for setting a range of predetermined environment information based on information; candidate selecting means for selecting, as a display candidate, video data having environment information within the set predetermined range from the plurality of video data; The present invention includes a presentation unit that presents the video data of the selected display candidate, and a display unit that displays the video data selected from the video data of the presented display candidate. [Selection diagram] Figure 1

Description

The present invention relates to a video processing system and a video processing method.

Conventionally, when inspecting road structures including road structures related to the road surface and ancillary structures installed around the road surface, the inspector confirms the image captured by the camera installed in the inspection vehicle. There is.
Patent Document 1 discloses a technique of inserting position information and azimuth information into video data when video data corresponding to the optical video is generated using an image pickup signal corresponding to the optical video.

Publication No. 2009-159279

When confirming the state of an object to be inspected based on the image captured by a camera, for example, if the frame captured under a specific image capturing condition can be taken out and confirmed, the inspection work efficiency will be further improved. To do.
However, the technique described in Patent Document 1 does not consider facilitating the selection of video data to be reproduced from a plurality of video data according to the imaging conditions.
The present invention has been devised in view of the above-mentioned conventional technique, and an object thereof is to provide a video processing system and a video processing method capable of easily selecting a video to be reproduced.

In order to solve the above problems, according to an aspect of a video processing device of the present invention, a data holding unit that holds a plurality of video data in which environmental information indicating an imaging environment is recorded for each predetermined frame, and a search. Input means for inputting information, setting means for setting a range of predetermined environment information based on the input search information, and video having environment information within the predetermined range set from the plurality of video data Candidate selection means for selecting data as a display candidate, presentation means for presenting the video data of the selected display candidate, and display means for displaying the video data selected from the video data of the presented display candidate. An image processing device is provided.

According to one aspect, the environment information includes the time when the image data was captured, the position where the image data was captured, the posture of the vehicle when the image data was captured, and the vehicle when the image data was captured. Vibration, temperature around the vehicle when the image data is captured, sound around the vehicle when the image data is captured, and speed of the vehicle when the image data is captured.

According to the present invention, it is possible to easily select a video to be reproduced.

It is a schematic block diagram which shows an example of the video processing system which concerns on the 1st Embodiment of this invention. It is a schematic block diagram which shows an example of the imaging system which images a video. It is a block diagram which shows the structure of an imaging device. It is an example of composition of a picture file. It is a figure which shows an example of the hardware constitutions of a reproduction control apparatus. It is a flowchart which shows the display process of a video file. It is a figure which shows the example of the screen which selects search information. It is a figure which shows the example of the screen which inputs search information. It is a figure which shows the example of input of search information. It is a figure which shows notionally the setting process of the range of predetermined environmental information, and the selection process of the video file of a display candidate. It is a figure which shows the example of presentation of the video file used as a display candidate. It is a figure which shows the example of presentation of the video file used as a display candidate. It is a figure which shows the example of the screen in the state which selected any of the display candidates. 8 is a flowchart showing a video file display process in the video processing system according to the second embodiment of the present invention. It is a figure which shows the example of the display screen of the selected video. It is a figure which shows the example of the screen which selects search information.

Embodiments of the present invention will be described below with reference to the drawings.
(First embodiment)
FIG. 1 is a schematic configuration diagram showing an example of a video processing system 100 according to this embodiment.
The video processing system includes a video DB (database) 1 in which captured video files (video data) and the like are stored, a display control device 2 for performing processing such as reproduction of the video files, and instructions from a user. An input device 3 for inputting and a display device 4 for displaying an image or the like according to a video file are provided.
The video DB 1 stores a video file 1a such as a road and surrounding scenery imaged under various imaging conditions, image data 1b such as a map, and setting data 1c used for condition setting and display control. ..
The display control device 2 includes an instruction input unit 2a for inputting an instruction from a user via the input device 3, a candidate selection unit 2b for selecting a video file of a reproduction candidate, and an image acquisition for acquiring an image selected by the user. It is provided with a unit 2c and a display control unit 2d that causes the display device 4 to display an image or the like according to the video file acquired by the video acquisition unit 2c.
The input device 3 includes a keyboard 3a, a mouse 3b, and the like for inputting instructions from the user.
The display device 4 is composed of a liquid crystal display device or the like, and displays an image according to control from the display control unit 2 d of the display control device 2.

The image file 1a stored in the image DB 1 is imaged under various image capturing conditions by the image capturing system 300 shown in FIG. 2, for example. The imaging system 300 can be mounted on the vehicle 200, for example. The vehicle 200 is, for example, an inspection vehicle for performing inspections performed every predetermined number of days on various roads including an expressway. The imaging system 300 includes three imaging devices 10a, 10b, and 10c. The three imaging devices 10a, 10b, 10c have the same structure. The imaging devices 10a, 10b, 10c are connected to the synchronization control unit 30. The synchronization control unit 30 performs control for synchronizing the image pickup timing of each of the image pickup devices 10a to 10c in frame units. The imaging system 300 also includes a sensor unit 37 that measures imaging conditions (for example, imaging time, position and orientation or vibration of the vehicle 200, ambient temperature, ambient sound, traveling speed of the vehicle 200, and the like). The output of the sensor unit 37 is supplied to each of the imaging devices 10a to 10c via the synchronization control unit 30.

The sensor unit 37 includes, for example, a time information acquisition unit 37a, a position information acquisition unit 37b, a posture information acquisition unit 37c, a temperature information acquisition unit 37d, a sound information acquisition unit 37e, and a speed information acquisition unit 37f. There is.
The time information acquisition unit 37a includes, for example, a receiver that receives a standard radio wave used in a radio-controlled clock or the like, and the standard time information obtained from the standard radio wave received by the receiver is used as the information indicating the imaging time as a synchronization control unit. The signal is output to the signal processing unit 14 of each of the imaging devices 10a to 10c via 30. Here, as standard radio waves, there are JJY in Japan, WWVB in the United States, MSF in the United Kingdom, DCF77 in Germany, and the like. The time information may be acquired using a time information providing service that provides time information through the Internet or a time signal of broadcasting, or the time information may be acquired using a clock incorporated in the synchronization control unit 30 or a personal computer (not shown). May be obtained.

The position information acquisition unit 37b includes, for example, a GNSS receiver that receives a signal transmitted from a GNSS (Global Navigation Satellite System) satellite. The position information acquisition unit 37b acquires position information indicating the position of the imaging device 10a (position of the vehicle 200) from the received signal, and uses the position information as signal indicating the imaging position of each of the imaging devices 10a to 10c. Output to the unit 14. Note that GNSS is a general term for positioning satellites such as GPS (US), GALILEO (Europe), CLONASS (Russia), and quasi-zenith satellite (Japan).
The posture information acquisition unit 37c is composed of, for example, a triaxial sensor such as a triaxial acceleration sensor or a triaxial gyro sensor. The attitude information acquisition unit 37c acquires the acceleration information and the angular velocity information of the imaging device 10a (vehicle 200), and outputs the acquired information to the signal processing unit 14 of each of the imaging devices 10a to 10c as information indicating the imaging attitude. When the posture information acquisition unit 37c includes a triaxial acceleration sensor, the vibration of the vehicle 200 can be detected by the output of the triaxial acceleration sensor.

The temperature information acquisition unit 37d is composed of, for example, a temperature sensor that detects the temperature around the vehicle 200. Alternatively, a sensor such as a radiation thermometer that detects the temperature of the road surface on which the vehicle 200 is traveling can be used. The temperature information acquisition unit 37d outputs the acquired temperature information to the signal processing unit 14 of each of the imaging devices 10a to 10c.
The sound information acquisition unit 37e is composed of, for example, a noise sensor that detects the level of noise around the vehicle 200. Alternatively, a microphone or the like that picks up the sound around the vehicle 200 may be used. The sound information acquisition unit 37e outputs the acquired sound information to the signal processing unit 14 of each of the imaging devices 10a to 10c.
The speed information acquisition unit 37f is composed of, for example, a speed sensor that detects the traveling speed of the vehicle 200 from the number of rotations of the wheels 200w (axles) of the vehicle 200. The speed information acquisition unit 37f outputs the acquired speed information to the signal processing unit 14 of each of the imaging devices 10a to 10c.
The time information acquisition unit 37a, the position information acquisition unit 37b, and the posture information acquisition unit 37c are not limited to the above configurations as long as the time information, the position information, and the posture information can be respectively acquired. In addition, the time information acquisition unit 37a, the position information acquisition unit 37b, and the attitude information acquisition unit 37c may be built in the imaging device 10a or installed in the vehicle 200. For example, when the posture information acquisition unit 37c is installed at a position where the vehicle body sway is not damped by the suspension of the vehicle 200 (for example, near the wheels), sway caused by a slight change in the road surface can be detected.
Further, the temperature information acquisition unit 37d and the sound information acquisition unit 37e are not limited to the above configurations as long as the temperature information and the sound information can be respectively obtained, and may be built in the imaging device 10a. You may install in the vehicle 200.

The imaging devices 10a, 10b, 10c are attached to the roof of the vehicle 200. The imaging devices 10a, 10b, and 10c are digital video cameras each having an angle of view (viewing angle) of about 60 degrees. The imaging device 10b is installed so as to face the front (the traveling direction) of the vehicle 200 in the longitudinal direction, and images the periphery of the vehicle 200 within a range of 30 degrees to the left and right of the direction (60 degrees in total). The imaging device 10a images the periphery of the vehicle 200 within a range of 60 degrees to the left of the imaging device 10b. The imaging device 10c images the periphery of the vehicle 200 within a range of 60 degrees to the right of the imaging device 10b. Therefore, the imaging devices 10a, 10b, and 10c have an imaging range of 180 degrees in total. Note that, for convenience of illustration, although the imaging devices 10a, 10b, and 10c are illustrated as being arranged side by side in the longitudinal direction of the vehicle 200 in FIG. 1, the imaging devices 10a, 10b, and 10c are illustrated in the vehicle 200. It may be arranged in the width direction. Since the imaging devices 10a, 10b, and 10c have the same structure, the imaging device 10a will be described below.

The image capture device 10a is, for example, a high-definition camera that captures a horizontally long high-definition image with an aspect ratio of 16:9. As a result, it is possible to capture an image having an aspect ratio of 4:3 in a state close to the human visual field. The image capturing device 10a captures an image around the vehicle 200 when the vehicle 200 is traveling on a predetermined inspection target road or when the vehicle 200 is stopped.

Hereinafter, the configurations of the imaging devices 10a, 10b, and 10c will be specifically described by taking the imaging device 10a as an example.
As shown in FIG. 3, the image pickup apparatus 10 a includes a lens 11 that collects light from a subject, an image pickup element 12, a drive circuit 13 that drives the image pickup element 12, control of the drive circuit 13, and the image pickup element 12. A signal processing unit 14 that performs processing on the image pickup signal of 1 and the like, and a recording unit 15 that records the video file from the signal processing unit 14 are provided.
The image pickup device 12 is composed of, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).
The drive circuit 13 drives the image sensor 12 according to the synchronization signal from the signal processing unit 14.
The signal processing unit 14 includes a synchronization signal generation unit 14a that generates a synchronization signal, a signal processing circuit 14b that inputs a signal analog video signal from the image sensor 12, and performs processing such as analog-digital conversion, and a signal processing circuit 14b. An encoder 14c that encodes a digital video signal obtained as a processing result and a control unit 14d that controls the entire processing of the signal processing unit 14 are provided.

A conventional image pickup apparatus is provided with an AFE (Analog Front End) that performs processes such as noise removal and analog-digital conversion on an analog video signal from an image pickup element. In such a conventional image pickup apparatus, a portion corresponding to the signal processing unit 14 of the present embodiment processes the digital video signal processed by AFE. When the AFE is used, the AFE outputs a frame without a video (a so-called black frame) for a frame for which the processing by the AFE is not in time or a frame in which an error occurs. A portion corresponding to the signal processing unit 14 of the conventional image pickup device does not output the digital video signal for the black frame (that is, discards the black frame) and outputs a frame having the next video. As a result, a frame may be missing in the output video signal, and the total time of all frames may be shorter than the actual image capturing time.
Further, in the conventional image pickup device, time information is recorded in the digital video signal output from the portion corresponding to the display device 4 of the present embodiment. Therefore, if there is a missing frame, it becomes difficult to extract a frame at a specific time in the video file 1a stored in the video DB 1 based on such a digital video signal.

On the other hand, in the imaging device 10a (10b, 10c) of the present embodiment, the AFE is not provided, and the signal processing unit 14 side performs processing such as analog-to-digital conversion, and even when a black frame occurs. , Is output as it is (without discarding the black frame) as a digital video signal. Further, in the image pickup apparatus 10a (10b, 10c) of the present embodiment, the time information is recorded in the digital video signals of all the frames including the black frame in the signal processing unit 14. As a result, no frame loss occurs in the imaging device 10a. Therefore, the time when the image is actually captured is equal to the time obtained by adding the times of all the frames.
Therefore, in the present embodiment, the frame at the specific time can be taken out only by counting the number of frames in the video file 1a stored in the video DB 1 based on the digital video signal from the signal processing unit 14. It has become.

The signal processing unit 14 performs predetermined signal processing on the electric signal obtained from the image sensor 12 to generate video data including a plurality of frames (image frames). In the present embodiment, information regarding the imaging environment such as the imaging time, the imaging position, the attitude at the time of imaging (imaging attitude), vibration, ambient temperature, ambient sound, traveling speed of the vehicle 200 (hereinafter referred to as “environmental information”). .) is associated with each frame to generate video data. The method of generating this video data will be described later. The signal processing unit 14 also generates a video file by compressing the video data via an encoder. The compression method here is, for example, H.264. It is based on the standards such as H.264, MPEG4 (Moving Picture Experts Group phase4), and MJPEG (Motion-JPEG). The recording unit 15 includes a magnetic recording medium such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), and a magnetic tape, and a recording medium such as an optical/magneto-optical storage medium such as an optical disk and a magneto-optical disk. The video file generated by the processing unit 14 is recorded.

FIG. 4 shows an example of a video file. As shown in FIG. 4, the video file 40 includes a header section 41 and a video data section 42. The header portion 41 includes header information (H) indicating the structure of the video file 40, such as the file name of the video file 40, the file type, the file size, and the like. The video data section 42 also includes video data in frame units (1F, 2F,... ). The environment information 43 is written as, for example, numerical data between the frames in the video data section 42. The environment information 43 is written just before the corresponding frame, for example. That is, the environment information 43 of the second frame (2F) is written between the first frame (1F) and the second frame (2F). Note that the position where the environment information 43 is written is not limited to the above as long as the environment information 43 is associated with the corresponding frame. The environment information 43 may be written in all the frames of the video data, or may be written at predetermined intervals. The environment information may be written in the frame.

In the present embodiment, the signal processing unit 14 writes the environment information 43 in a form that does not affect the reproduction of the video data. For example, the environment information 43 may be stored in a header (frame header) provided for each frame, or may be stored as metadata for each frame. Further, the environment information 43 may be written by adding a control code that causes the environment information 43 to be skipped during video reproduction. The environment information 43 may be written in such a manner that the environment information 43 is commented out when the video is reproduced. A user of the video processing system 100 (for example, an occupant of the vehicle 200) can appropriately set the method of writing the environment information 43.
In this way, the signal processing unit 14 sequentially generates the video data unit 42 in frame units while writing the environment information 43 related to each frame. Then, when the generation of the video data section 42 is completed, the signal processing section 14 generates the header section 41 corresponding to the video data section 42 and generates the video file 40. The signal processing unit 14 records the generated video file 40 in the recording unit 15. The video file recorded in the recording unit 15 is supplied to the above-described video processing system 100 via a recording medium or a network and stored in the video DB 1.

FIG. 5 is a diagram illustrating an example of a hardware configuration of the display control device 2.
The image processing apparatus 20 includes a CPU 51, a ROM 52, a RAM 53, an external memory 54, an input unit 55, an output unit 56, a communication I/F 57, and a system bus 58.
The CPU 51 centrally controls the operation of the image processing apparatus 20, and controls each component (52 to 57) via the system bus 58.

The ROM 52 is a non-volatile memory that stores control programs and the like necessary for the CPU 51 to execute processing. The program may be stored in the external memory 54 or a removable storage medium (not shown).
The RAM 53 functions as a main memory, a work area, etc. of the CPU 51. That is, the CPU 51 realizes various functional operations by loading necessary programs and the like from the ROM 52 into the RAM 53 when executing the processing and executing the programs and the like.

The external memory 54 previously stores, for example, various data and various information necessary when the CPU 51 performs processing using a program. Further, the external memory 54 stores, for example, various data and various information obtained by the CPU 51 performing processing using a program or the like. The external memory 54 may be an external storage medium such as an SD card or a USB memory. The external memory 54 is a storage medium (storage device) such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), a magnetic recording medium such as a magnetic tape, or an optical/magneto-optical storage medium such as an optical disk or a magneto-optical disk. May be

The input unit 55 corresponds to the input device 3 of FIG. 1 and is configured by a pointing device such as a keyboard and a mouse, and the user of the display control device 2 gives an instruction to the display control device 2 via the input unit 55. You can do it.
The output unit 56 corresponds to the display device 4 and the like in FIG. 1 and is configured by a monitor such as a liquid crystal display (LCD).
The communication I/F 57 is an interface for communicating with an external device (for example, the image pickup devices 10a to 10c). The communication I/F 57 is, for example, a LAN interface.
The system bus 58 communicatively connects the CPU 51, the ROM 52, the RAM 53, the external memory 54, the input unit 55, the output unit 56, and the communication I/F 57.
That is, the CPU 51 functions as the display control device 2 of FIG. 1 by executing the program stored in the ROM 52.

Next, the operation of the video processing system 100 will be described with reference to the flowchart of FIG.
It is assumed that the image file 1a previously imaged by the image capturing system 300 is stored in the image DB 1.
When the processing of FIG. 6 is started, the display control device 2 generates an image for displaying a screen (selection screen) for allowing the user to select the type of environment information to be used as search information in S1, and the display device 4 To display.
Specifically, the candidate selection unit 2b selects, on the display control unit 2d, a selection button 61a for selecting "time" and a "position" in the selection area 61 of the selection screen 60 as shown in FIG. 7, for example. Selection button 61b to select, "posture" selection button 61c, "vibration" selection button 61d, "temperature" selection button 61e, "sound" selection button 61f, A selection button 61g for selecting "speed" is displayed.

The user operates the mouse 3b to move the cursor 65 displayed on the selection screen 60, selects the button corresponding to the desired environment information, and then instructs the enter button 63. When the user's selection is input via the instruction input unit 2a (S2), the candidate selection unit 2b causes the display control unit 2d to display an input screen for inputting search information (S3).
Specifically, for example, when “position” is selected as the type of search information, the candidate selection unit 2b causes the display control unit 2d to display the input screen 70 as shown in FIG. On this input screen 70, a region 71 for displaying a map, a region 72 for displaying buttons 72a to 72j for inputting a user's instruction, a region 73 for displaying environmental information, and regions 74a, 74b for displaying images, 74c is provided. In the area 71 for displaying a map, the candidate selection unit 2b causes the display control unit 2d to display the map corresponding to the position in the initial state.
After displaying such an input screen 70, the candidate selection unit 2b waits for the user to input search information (S4).
When "position" is selected as the type of search information and the input screen 70 shown in FIG. 8 is displayed, the user operates the mouse 3b to instruct the button 72e for instructing the display of the marker ( When clicked, the candidate selection unit 2b causes the display control unit 2d to display the marker 75 in the area 71 of the input screen 70 as shown in FIG. 9, for example. Further, the user operates the mouse 3b to move the marker 75 to a desired position and clicks a certain point on the map, and the clicked point becomes search information regarding the position.
Alternatively, when “attitude” is selected as the type of search information, for example, a screen for inputting the tilt value of the vehicle 200 in the front-rear direction or the left-right direction is displayed, and the user can set the attitude value (for example, “15°”). ) May be input. That is, the value of the input posture becomes the search information. When “vibration” is selected as the type of search information, for example, a screen for inputting a vibration value may be displayed and the user may be prompted to input the vibration value. When “temperature” is selected as the type of search information, for example, a screen for inputting a temperature value may be displayed and the user may be prompted to input the temperature value. When “sound” is selected as the type of search information, for example, a screen for inputting a sound value (noise level) may be displayed and the user may be prompted to input the sound value. Further, when “speed” is selected as the type of search information, for example, a screen for inputting a speed value may be displayed and the user may be prompted to input a speed value.

When the user inputs search information in S4, the candidate selection unit 2b sets a predetermined range of environmental information based on the input search information (S5). Specifically, when the “position” is input as the search information, the candidate selection unit 2b, for example, as shown by the broken line 85 in FIG. The position is set as a range of predetermined environment information. In the figure, the position information 81 to 83 is the position information of each frame of the plurality of video files 1a stored in the video DB 1 plotted on a map.
Alternatively, when the value of “attitude” is input as the search information, for example, a predetermined range from the input value of the inclination value of the front and rear or the left and right of the vehicle 200 is set as the range of the predetermined environmental information. In this case, it may be set as the range of the predetermined environmental information that the state in which the value of the left and right inclination is within the predetermined range continues for a predetermined time (a predetermined number of frames). As a result, a twisted section of the road can be set as a range of predetermined environmental information. When the value of “vibration” is input as the search information, for example, a predetermined range from the input value of the vibration value is set as the predetermined range of environmental information. Further, when the value of “temperature” is input as the search information, for example, a predetermined range from the input value of the temperature value is set as the predetermined range of environmental information. Further, when the value of “sound” is input as the search information, for example, a predetermined range from the input value of the sound value or a predetermined level or more is set as the predetermined environment information range. When the value of "speed" is input as the search information, for example, a predetermined range from the input value of the speed value or a predetermined speed or more from the input value is set as the predetermined environmental information range.

When the range of the predetermined environment information is set in S5, the candidate selection unit 2b searches for candidates (display candidates) of the video file 1a to be displayed via the video acquisition unit 2c (S6).
The search for the candidates of the video file 1a to be displayed is performed by the candidate selection unit 2b by comparing the environment information of each video file 1a stored in the video DB 1 with the set range of the predetermined environment information. Specifically, when a predetermined radius centered on the position of the marker 75 is set as the range of the predetermined environment information, the candidate selection unit 2b determines that the position information in the video file 1a is the marker 75. It is determined whether or not it is within a predetermined distance from the position, and the video file 1a having the position information within the predetermined distance from the position of the marker 75 is selected as a display candidate.
When another predetermined environment information range is set, the candidate selection unit 2b compares the environment information of each video file 1a with the set predetermined environment information range to display candidates. Select.
When the display candidate is selected, the candidate selection unit 2b presents the video data of the display candidate to the user. Specifically, the candidate selection unit 2b instructs the display control unit 2d to display the selected display candidate on the display device 4 (S7). Specifically, when a video file 1a having position information within a predetermined distance from the position of the marker 75 is selected as a display candidate, the candidate selection unit 2b displays a map as shown in FIG. 11, for example. The position information 81, 82 of the video file 1a of the display candidate is displayed by being superimposed on the map displayed in the area 71 for displaying. Note that the position information 83 in FIG. 10 does not fall within the range of the broken line 85 and therefore does not enter the display candidates. Therefore, in the area 71 of FIG. 11, only the position information 81 and 82 of the position information 81 to 83 is displayed.
The candidate selection unit 2b moves the map so that the position of the marker 75 is near the center of the area 71. Further, the position information 81 and 82 in this case is displayed as a line in which the position information of each frame of the video file 1a of each display candidate is plotted.
When there are a plurality of display candidate video files 1a, the display mode (color, line type, etc.) of the position information of each video file 1a may be different in order to improve the visibility. In the example of FIG. 11, the position information 81 and 82 of the two video files 1a are displayed in different line types. Further, instead of displaying the position information 81, 82 of the display candidate video file 1a, for example, as shown in FIG. 12, a window 76 for displaying the display candidate video file 1a in a list format may be displayed. The candidate selection unit 2b causes the display areas 76a and 76b in the window 76 to display environmental information such as time information and position information of the video file 1a of each display candidate. The order of arrangement in the window 76 is, for example, the order in which the position information included in the video file 1a is closer to the position of the marker 75. Alternatively, the time information included in the video file 1a may be in ascending order.
Further, when the display candidate video file 1a is selected according to the range of other predetermined environment information, it may be displayed in a list format similar to that shown in FIG. 12, or similar to FIG. The position information of 1a may be displayed by being superimposed on the map.

After that, the candidate selection unit 2b waits for the user to select any of the display candidate video files 1a (S8).
When the user selects one of the display candidate video files 1a, the candidate selection unit 2b acquires the selected video file 1a via the video acquisition unit 2c and the video file 1a via the display control unit 2d. The image of the frame corresponding to the position information of the position closest to the marker 75 is displayed (S9). Specifically, for example, as shown in FIG. 13, the image files 1a captured by the image capturing devices 10a, 10b, and 10c are synchronized and displayed in the areas 74a, 74b, and 74c for displaying images, respectively. That is, one of the video files 1a simultaneously imaged by the imaging devices 10a, 10b, and 10c (for example, the video file 1a captured by the imaging device 10b) is selected, and the candidate selection unit 2b selects a predetermined one of the video files 1a. While displaying the image of the frame, by using the time information corresponding to the displayed frame in the video file 1a, the image of the frame having the same time information of the other video file 1a captured at the same time (imaging device 10a The image of the frame of the video file 1a captured at 10c) is displayed.
The display control unit 2d also displays the marker 75' at the position of the position information corresponding to the frame. Further, the candidate selection unit 2b causes the display control unit 2d to set the environment information corresponding to the frame of the video file 1a currently displayed in the area 73.
After that, the display control unit 2d controls the display of the video file 1a according to an instruction from the user. For example, when the user instructs the button 72g, the reproduction of the video file 1a is started. Alternatively, when the user instructs the button 72f, the video file 1a is rewound. When the user instructs the button 72h, the video file 1a is fast-forwarded.
The display control unit 2d changes the position of the area 71 according to the reproduction of the video file 1a, and displays the map so that the position on the map corresponding to the position information corresponding to the frame currently being reproduced is in the center of the area 71. To move.
The display control unit 2d continues the above-described processing until the user gives an instruction to end (S10). Alternatively, the process ends when the reproduction of the video file 1a ends.
If the number of display candidates obtained in S6 is one, the process of S9 may be executed without executing the process of S8.

In the present embodiment, as described above, the video file 1a corresponding to the search information input by the user is presented to the user as a candidate to be displayed, and the video file 1a to be reproduced by the user is selected from the presented candidates. Therefore, it is possible to easily select the video to be reproduced.

(Second embodiment)
The video processing system 100 according to the second embodiment of the present invention has the same configuration as that of FIG.
In the first embodiment, a range of predetermined environment information is set based on the input search information, a video file having environment information within the set predetermined range is selected as a display candidate, and the video file of the display candidate is selected. Was presented to the user, and the video file selected from the presented display candidate video files was displayed.
On the other hand, in the second embodiment, a position on the map is input and a video file having position information within a predetermined distance from the position is displayed.
In the image processing system 100 of the present embodiment, as shown in FIG. 14, for example, first, the candidate selection unit 2b displays the same input screen as in FIG. 8 (S11) and waits for the position input from the user (S12). ).
First, when the user operates the mouse 3b to instruct (click) the button 72e for instructing the display of the marker, the candidate selecting unit 2b causes the display control unit 2d to display the area 71 of the input screen 70 as in FIG. The marker 75 is displayed inside. Furthermore, when the user operates the mouse 3b to move the marker 75 to a desired position and clicks a point on the map, the clicked point is input as search information regarding the position.
When the user inputs the position, the candidate selection unit 2b searches for candidates (display candidates) of the video file 1a to be displayed via the video acquisition unit 2c (S13).
Specifically, the candidate selection unit 2b selects only one video file 1a having position information closest to the position of the marker 75 as a display candidate.
After that, the candidate selection unit 2b causes the display control unit 2d to display the display candidate video file 1a (S14).
Specifically, the candidate selection unit 2b causes the display control unit 2d to display an image of a predetermined frame of the video file 1a to be displayed and a map corresponding to the position information of the frame, as shown in FIG. 15, for example. The marker 75' is displayed at the upper position.
The subsequent display control by the display control unit 2d is similar to that of the first embodiment, and the display control unit 2d continues the display control until the user gives an instruction to end (S15).
Note that, in S14, the video file 1a is displayed and the position information of the video data 1a of the display candidate is superimposed and displayed in the same manner as in FIG. 11, and the candidate selection unit 2b operates according to the instruction from the user. The video file 1a to be displayed may be switched.

In this embodiment, the user can input the position on the map and display the video file having the position information closest to the input position, so that the video to be reproduced can be easily selected. At the same time, the input load on the user can be reduced.

(Modification)
In the above embodiment, the case where the image capturing time, the image capturing position, the image capturing attitude, and the like are recorded as the environment information regarding the image capturing environment has been described, but the environment information may include at least the image capturing time. Further, the environment information may include, for example, information regarding speed at the time of image capturing, information regarding sound at the time of image capturing, and the like. Further, the sound (voice data) at the time of image capturing may be included in each frame 1F, 2F, 3F.
Moreover, although the case where the imaging devices 10a to 10c are installed in the vehicle 200 has been described in the above embodiment, the present invention is not limited to this. For example, the imaging devices 10a to 10c may be installed in a moving body (radio control, drone (unmanned aerial vehicle), helicopter, etc.) other than the vehicle, or the imaging device 10 may be fixed to a road shoulder or the like. Furthermore, the imaging devices 10a to 10c can be installed in stores, facilities, and the like. In this case, the video processing system 100 can function as a monitoring system that monitors the inside of a store or facility. Further, the imaging devices 10a to 10c may be carried by a person.

Further, in the above embodiment, the configuration in which the video processing system 100 and the imaging system 300 are separately provided has been described, but the video processing system 100 and the imaging system 300 may be configured as one device or system.
Further, in the above embodiment, the case where the display control device 2 displays the frame searched by the image search processing on the display unit has been described. However, the searched frame is recorded in a recording medium or transferred to the server device. You may do so.
The program executed by the display control device 2 in the above embodiment may be provided by being stored in a computer-readable storage medium in an installable file format or an executable file format. Further, the program executed by the display control device 2 in the above embodiment may be provided via a network such as the Internet.
Although the embodiments of the present invention have been described in detail above, the above-described embodiments are merely examples for implementing the present invention, and the technical scope of the present invention is limited to the above-described embodiments. Not something. The present invention can be modified or changed in various ways without departing from the spirit thereof.

Also, in the first embodiment, the arrangement order may be selected when there are a plurality of display candidate video files 1a. In this case, instead of the selection screen of FIG. 7, a selection screen for selecting a plurality of environment information is displayed as shown in FIG. 16, for example.
The plurality of pieces of environment information are, for example, “first key” and “second key”, and the candidate selection unit 2b inputs each type. Further, the candidate selection unit 2b sets the range of the predetermined environment information based on the "first key" to select the display candidate, and when presenting the display candidate, based on the "second key". The display candidates are presented as the arrangement order of the display candidates.
Further, the image file 1a to be a display candidate may be selected in advance to some extent, and then the process of FIG. 6 may be executed. For example, when the information indicating the prefecture corresponding to the imaging position of the video file 1a is recorded and the prefecture name is input as the search information, only the video file 1a corresponding to the input prefecture name is searched. By doing so, the search time can be shortened.

1... Image DB, 2... Display control device, 2a... Instruction input part, 2b... Candidate selection part, 2c... Image acquisition part, 2d... Display control part, 3... Input device, 4... Display device, 10a, 10b, 10c ... image pickup device, 30... synchronization control unit, 37... sensor unit

Claims (9)

An image processing device used for inspection of highways,
Data holding means for holding at least one image data of the highway , which is image data obtained by imaging while traveling on the highway and in which position information of the highway is recorded for each predetermined frame; ,
Map display means for displaying on the display unit a map image in which the highway and roads other than the highway are drawn;
Input means for accepting a click input on the map image,
Display control means for reproducing and displaying video data of the highway having position information within a predetermined range from a point clicked on the map image on the display unit together with the map image,
An image processing apparatus comprising: The video according to claim 1, wherein the display control means reproduces and displays video data of an expressway having position information closest to the clicked point within the predetermined range on the display unit. Processing equipment. The video processing apparatus according to claim 1 or 2, characterized in <br/> that the region of the point where the previous SL click input within a predetermined distance further comprising setting means for setting as the predetermined range. When there are a plurality of video data of the expressway having position information within the predetermined range, it is further provided with a presenting means for presenting character information or a diagram regarding the plurality of video data to the display unit as a reproduction display candidate. the video processing apparatus according to any one of claims 1 to 3, characterized. It said presenting means, when the video data of the reproduction display candidates have multiple video processing apparatus according to claim 4, characterized in that to present a different display form for each of the diagrams. When the reproduction display of the video data is started, the display control means determines which position of the expressway the frame being reproduced and displayed is based on the position information recorded in the frame of the video data. The video processing device according to claim 1, wherein a marker indicating whether the frame is a frame is displayed on the map image. 7. The video processing device according to claim 1, wherein the map display means moves the map image in accordance with reproduction display of the video data. A video processing method for reproducing a video used for inspection of the highway by using video data of at least one highway in which position information of the highway is recorded for each predetermined frame, wherein the video data is Video data obtained by imaging while traveling on the highway, the video processing method,
Displaying on the display unit a map image in which the highways and roads other than the highways are drawn;
Receiving a click input on the map image,
A step of reproducing and displaying video data of the highway having position information within a predetermined range from a point click-input on the map image on the display unit together with the map image;
An image processing method comprising: The replay display step replays and displays on the display unit video data of an expressway having position information closest to the clicked point within the predetermined range. Video processing method.

Images