JP7254000B2 - Image delivery device and method, and image delivery system - Google Patents

Description

The present invention relates to an image delivery device and method, and an image delivery system.

The drive recorder can photograph and record the state of the exterior or interior of the own vehicle. There is also a technique for generating and displaying a bird's-eye view image of the surroundings of the own vehicle based on the photographed results of a plurality of cameras installed on the own vehicle. All of these are techniques for photographing and imaging the state of the exterior or interior of the own vehicle, and are not techniques for photographing the own vehicle itself.

JP-A-2004-153750 JP 2014-191664 A JP 2007-179198 A

Vehicle owners do not have the opportunity to check the appearance of their vehicle when they are driving it themselves. In order for the owner of the vehicle to check the appearance of the own vehicle while it is running, there is no other way but to have another person drive the own vehicle and check the state of the own vehicle at that time from the outside.

On the other hand, an image (video) taken of a moving vehicle is called a rolling shot, and there is a certain number of car enthusiasts who want to have friends and acquaintances take a rolling shot of their own vehicle for viewing. etc., you can check that. Also, in addition to collecting rolling shots, which is a hobby, if you can check the appearance of your vehicle while it is running, you can check the appearance of your vehicle while driving, such as a failure of the vehicle's lighting device (brake lamp, etc.) or a collapse of the truck's cargo. It is thought that it will also be useful for detecting abnormalities that are difficult for the driver of the vehicle to notice.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image distribution apparatus, method, and image distribution system that enable distribution of an image obtained by photographing the exterior of a vehicle to the vehicle.

The image delivery device according to the present invention is a processing unit that communicates with each on-vehicle device installed in a first vehicle and a second vehicle, and is a second vehicle captured image that is a captured image of a camera installed in the second vehicle. and a communication processing unit capable of receiving the image information of, and using the communication processing unit, acquiring position information of the first vehicle at each time from the in-vehicle device of the first vehicle and from the in-vehicle device of the second vehicle a position information acquisition unit that acquires position information of the second vehicle at each time; and based on the position information of the first vehicle at each time and the position information of the second vehicle at each time, the first vehicle and the second vehicle A target time period setting unit for setting a target time period in which a relative positional relationship between two vehicles satisfies a predetermined condition; and a vehicle image determination unit that determines whether or not the image of the first vehicle is included in the determination image based on external appearance feature information of the first vehicle provided in advance; When it is determined that the image of one vehicle is included, the image information of the distributed image based on the second vehicle captured image generated by the camera installed in the second vehicle in all or part of the target time period and an image distribution unit that transmits the image to an in-vehicle device of the first vehicle using the communication processing unit (first configuration).

In the image delivery device according to the first configuration, the target time period setting unit, based on the position information of the first vehicle at each time and the position information of the second vehicle at each time, the first vehicle and the A configuration (second configuration) may be adopted in which a time period in which the distance between the second vehicles is equal to or less than a predetermined value is set as the target time period.

In the image distribution device according to the first or second configuration, the vehicle image determination unit includes n _A images of the second vehicle in the time series generated by the camera installed in the second vehicle in the target time period. Of the images, n _B second vehicle captured images are referred to as the judgment images (n _A is an integer of 2 or more, and n _B is an integer of 1 or more smaller than n _A ) configuration (third configuration ) may be

In the image delivery device according to any one of the first to third configurations, the vehicle image determination unit includes the determination image, the appearance characteristic information of the first vehicle, and the first vehicle at the time when the determination image was captured. A configuration (fourth configuration) may be employed in which it is determined whether or not the image of the first vehicle is included in the determination image based on the relative positional relationship between the vehicle and the second vehicle.

In the image distribution device according to any one of the first to fourth configurations, the image distribution unit, when it is determined that the image of the first vehicle is included in the determination image, the target time period Referring to a plurality of time-series captured images of the second vehicle generated by the camera installed on the second vehicle in the cut-out time period that is all or part of the above as a plurality of original images, An image based on a part of the original image for each of the original images, based on the relative positional relationship between the first vehicle and the second vehicle and the appearance feature information of the first vehicle, and the image of the first vehicle. and executing a partial image generation process for generating a partial image containing the plurality of original images, and generating a moving image configured by arranging the plurality of partial images generated for the plurality of original images in time series as the distribution image (first 5) may be used.

In the image delivery device according to the fifth configuration, the camera installed on the second vehicle is an omnidirectional camera, the partial image generation processing includes a first partial image generation processing and a second partial image generation processing, and the The one partial image generation process and the second partial image generation process are executed for each of the original images, and the image distribution unit performs the first partial image generation process on the first vehicle and the estimating an existing image region of the first vehicle in the original image based on the relative positional relationship between the second vehicles, setting a cutout image region in the original image based on the position of the estimated region, and setting the cutout image; A first partial image is generated by performing predetermined first image processing including geometric transformation on an image within the region, and in the second partial image generation processing, based on the appearance feature information of the first vehicle. specifying the position of the first vehicle image in the first partial image, correcting the position of the clipped image area based on the specified position, and geometrically transforming the corrected image in the clipped image area; second partial images are generated by performing predetermined second image processing including A configuration (sixth configuration) that is an image may be used.

An image delivery system according to the present invention comprises an image delivery device according to any one of the first to sixth configurations, a first vehicle-mounted device installed in a first vehicle, and a second vehicle-mounted device installed in a second vehicle. a device, wherein the first vehicle-mounted device includes a first vehicle-mounted control unit; a first position information generation unit that sequentially generates position information of the first vehicle; and the image distribution device. a first communication processing unit that communicates with the second vehicle-mounted device, the second vehicle-mounted device includes: a second vehicle-mounted control unit; a second location information generation unit that sequentially generates location information of the second vehicle; a second communication processing unit that communicates with a device, wherein the first vehicle-mounted control unit uses the first communication processing unit to transmit position information of the first vehicle that is sequentially generated to the image distribution device; The second vehicle-mounted control unit sequentially transmits the position information of the second vehicle that is sequentially generated to the image distribution device using the second communication processing unit, and the A camera mounted on a second vehicle is connected to the second vehicle-mounted device or included in a component of the second vehicle-mounted device, generates image information of the second vehicle-captured image at a predetermined frame rate, The in-vehicle control unit responds to a transmission request signal from the image distribution device, and uses the second communication processing unit to transmit the image information of the second captured image of the vehicle in the time period specified by the transmission request signal. The image information of the second vehicle captured image that is transmitted to the image distribution device or sequentially generated by the camera installed in the second vehicle is transmitted to the image at a predetermined transmission timing using the second communication processing unit. This is the configuration (seventh configuration) for transmission to the distribution device.

An image delivery method according to the present invention uses a communication processing unit that communicates with each on-vehicle device installed in a first vehicle and a second vehicle. a communication processing step of receiving image information of an image; and acquiring position information of the first vehicle at each time from an in-vehicle device of the first vehicle using the communication processing unit and from an in-vehicle device of the second vehicle. a position information acquisition step of acquiring position information of the second vehicle at each time; and based on the position information of the first vehicle at each time and the position information of the second vehicle at each time, the first vehicle and the second vehicle a target time zone setting step of setting a target time zone in which a relative positional relationship between two vehicles satisfies a predetermined condition; a vehicle image determination step of determining whether or not the image of the first vehicle is included in the determination image based on external appearance feature information of the first vehicle provided in advance; When it is determined that the image of one vehicle is included, the image information of the distributed image based on the second vehicle captured image generated by the camera installed in the second vehicle in all or part of the target time period , and an image delivery step of transmitting the image to an in-vehicle device of the first vehicle using the communication processing unit (eighth configuration).

According to the present invention, it is possible to provide an image delivery apparatus, method, and image delivery system that enable delivery of an image of the exterior of a vehicle to the vehicle.

1 is a diagram showing the configuration of a drive recorder according to a first embodiment of the present invention together with its peripheral devices; FIG. FIG. 2 is a diagram showing how the drive recorder is mounted on a vehicle; It is a figure which shows a mode that an omnidirectional camera is installed in a vehicle. FIG. 2 is a schematic diagram of the configuration and operation of capturing a surrounding landscape using an omnidirectional camera; 1 is a schematic overall configuration diagram of an image distribution system according to a first embodiment of the present invention; FIG. 1 is a diagram showing the configuration of a drive recorder according to a first embodiment of the present invention together with its peripheral devices; FIG. 1 is a configuration diagram of a server device according to a first embodiment of the present invention; FIG. FIG. 4 is a diagram showing how position information and time information are transmitted from the drive recorder of each vehicle to the server device; FIG. 10 is a diagram showing the flow of operation of the image delivery system according to example EX1_1 belonging to the first embodiment of the present invention; FIG. 10 is a flowchart of an approaching vehicle determination process according to Example EX1_1 belonging to the first embodiment of the present invention; FIG. FIG. 10 is an explanatory diagram of a time zone in which two vehicles approach each other, according to example EX1_1 belonging to the first embodiment of the present invention; FIG. 4 is a diagram showing how a moving image is composed of a plurality of frame images arranged in time series; FIG. 10 is a diagram showing how a plurality of sampled images are extracted in a target time period, according to example EX1_1 belonging to the first embodiment of the present invention; FIG. 10 is an explanatory diagram of target vehicle search processing according to example EX1_1 belonging to the first embodiment of the present invention; FIG. 10 is an explanatory diagram of target vehicle search processing according to example EX1_1 belonging to the first embodiment of the present invention; FIG. 10 is a relational diagram between a target time period and an extraction time period, relating to Example EX1_1 belonging to the first embodiment of the present invention. FIG. 11 is a diagram showing a sequence of original images and two types of partial image sequences arranged in time series in an extraction time period, according to example EX1_1 belonging to the first embodiment of the present invention; FIG. 10 is an explanatory diagram of partial image generation processing according to example EX1_1 belonging to the first embodiment of the present invention; FIG. 10 is a diagram showing how four cameras are installed in a vehicle according to example EX1_3 belonging to the first embodiment of the present invention; FIG. 10 is a diagram showing how four cameras are installed in a vehicle according to example EX1_3 belonging to the first embodiment of the present invention; FIG. 10 is a diagram showing a sequence of frame images acquired by each camera installed in a vehicle, according to Example EX1_3 belonging to the first embodiment of the present invention; FIG. 10 is a diagram showing how server devices are distributed according to Example EX1_6 belonging to the first embodiment of the present invention; FIG. 10 is a diagram showing a flow of transmission and reception of a distribution image by vehicle-to-vehicle communication according to example EX2_1 belonging to the second embodiment of the present invention. FIG. 10 is a diagram showing how communication is performed between four vehicles according to example EX2_1 belonging to the second embodiment of the present invention.

Hereinafter, examples of embodiments of the present invention will be specifically described with reference to the drawings. In each figure referred to, the same parts are denoted by the same reference numerals, and redundant descriptions of the same parts are omitted in principle. In this specification, for simplification of description, by describing symbols or codes that refer to information, signals, physical quantities, or members, etc., the names of information, signals, physical quantities, or members, etc. corresponding to the symbols or codes are It may be omitted or abbreviated. For example, the vehicle image distribution request signal referred to by "610" (see FIG. 9) may be referred to as the vehicle image distribution request signal 610, or may be abbreviated as the request signal 610. but they all refer to the same thing.

<<First Embodiment>>
A first embodiment of the present invention will be described. FIG. 1 shows the configuration of a drive recorder 1 according to the first embodiment of the present invention together with its peripheral devices. As shown in FIG. 2, the drive recorder 1 is mounted on the vehicle CR. The vehicle CR may be any vehicle (such as an automobile) that can travel on a road surface. Focusing on the vehicle CR, front, rear, left, and right are defined as follows. The direction from the driver's seat of the vehicle CR to the steering wheel of the vehicle CR is defined as "forward", and the direction from the steering wheel of the vehicle CR to the driver's seat is defined as "rear". The direction connecting the front and rear is parallel to the straight traveling direction of the vehicle CR, and when the vehicle CR runs on a horizontal road surface, the straight traveling direction is parallel to the horizontal plane. In the vehicle CR, it is assumed that the driver faces forward and sits in the driver's seat, and the direction from the left side to the right side of the driver is defined as "right direction", and the direction from the right side to the left side of the driver is defined as "right direction". Define "to the left". The left-right direction is a direction perpendicular to the straight traveling direction of the vehicle CR and the vertical line. Also, "upward" and "downward" refer to upward and downward directions as seen from the driver, respectively.

A plurality of peripheral devices are connected to the drive recorder 1 . A plurality of peripheral devices here are also installed in the vehicle CR in the same manner as the drive recorder 1 . The multiple peripheral devices include a camera section 21 , an in-vehicle sensor section 22 , a GPS processing section 23 , a clock section 24 and an interface device 30 . Information is exchanged between each peripheral device and the drive recorder 1 through a CAN (Controller Area Network) formed in the vehicle CR.

The camera unit 21 consists of one or more cameras installed on the vehicle CR. Each camera that constitutes the camera unit 21 sequentially captures scenery within its own imaging range (field of view) at a predetermined frame rate, and sequentially transmits camera information including image information (image data) of captured images indicating the imaging results. Send to drive recorder 1. Note that the camera unit 21 may have a microphone. In this case, audio information (audio data) representing sounds outside or inside the vehicle CR is also generated by the camera unit 21, and the generated audio information is also an image. The information is sent from the camera unit 21 to the drive recorder 1 in a form added to the information.

The vehicle-mounted sensor unit 22 includes a plurality of vehicle-mounted sensors installed in the vehicle CR, and detects and acquires vehicle information using each vehicle-mounted sensor. Vehicle information is sequentially acquired at a predetermined cycle, and the acquired vehicle information is sequentially sent to the drive recorder 1 . The vehicle information includes vehicle speed information indicating the speed of the vehicle CR, accelerator information indicating the depression amount of an accelerator pedal provided in the vehicle CR, brake information indicating the depression amount of the brake pedal provided in the vehicle CR, and It includes steering angle information representing the steering angle of the provided steering wheel. The plurality of in-vehicle sensors include an acceleration sensor, and acceleration information output from the acceleration sensor is also included in the vehicle information. The acceleration sensor detects the acceleration of the vehicle CR in directions of three axes or two axes orthogonal to each other, and outputs acceleration information indicating the detection results.

The GPS processing unit 23 detects the position (current location) of the vehicle CR by receiving signals from a plurality of GPS satellites forming a GPS (Global Positioning System), and obtains position information indicating the detected position. In the position information, the position (current location) of the vehicle CR is represented by longitude and latitude on the earth. The GPS processing unit 23 sequentially generates position information in a predetermined cycle and sequentially sends the generated position information to the drive recorder 1 .

The clock unit 24 generates time information indicating the current date and time and sends it to the drive recorder 1 . Time information may be generated or corrected using the reception signal of the GPS processing unit 23 .

The interface device 30 is a man-machine interface between the drive recorder 1 and a passenger (mainly a driver) of the vehicle CR, and includes a display section 31 , a speaker 32 and an operation section 33 . The display unit 31 is a display device configured by a liquid crystal display panel or the like, and can display any image under the control of the in-vehicle control unit 110 . The speaker 32 can output any sound under the control of the in-vehicle controller 110 . The operation unit 33 receives an arbitrary operation from a passenger (mainly a driver) of the vehicle CR. A touch panel may be configured by the display unit 31 and the operation unit 33 . The display unit 31, the speaker 32, and the operation unit 33 may be components of a car navigation system (not shown) that can be installed in the vehicle CR.

[Configuration and function of drive recorder]
The drive recorder 1 includes an in-vehicle control unit 110 , a recording medium 120 and a communication processing unit 130 . The in-vehicle control unit 110 includes a CPU (Central Processing Unit), a ROM (Read only memory), a RAM (Random access memory), and the like. realizes various functions that should be realized. The recording medium 120 is an arbitrary recording medium capable of reading and writing arbitrary information under the control of the in-vehicle control unit 110 . The recording medium 120 may be a recording medium that can be arbitrarily attached to and detached from the drive recorder 1, and is, for example, a memory card typified by an SD card. The communication processing unit 130 is a communication processing block for performing wireless communication with a device other than the drive recorder 1 .

The in-vehicle control unit 110 can perform normal recording control and event recording control on the recording medium 120 . Here, as an example, it is assumed that all recording areas of the recording medium 120 are provided with a normal recording area for normal recording control and an event recording area for event recording control.

Normal recording control will be explained. The in-vehicle control unit 110 can start the normal recording control when the engine of the vehicle CR is started, and can continue executing the normal recording control until the engine of the vehicle CR stops. After starting the normal recording control with the engine start of the vehicle CR as a trigger, even if the engine of the vehicle CR stops, the normal recording control is continued for a certain period of time, and if the engine has stopped beyond the certain period of time. As long as the normal recording control is terminated, the normal recording control may be terminated.

In the normal recording control, the camera information sequentially obtained from the camera section 21 is sequentially recorded in the normal recording area of the recording medium 120 in chronological order. At this time, the non-camera information is also recorded in the normal recording area in a state associated with the camera information. That is, in the normal recording control, the camera information and the non-camera information acquired at a certain time are recorded in the normal recording area while being associated with each other. Non-camera information includes vehicle information, location information and time information. Information consisting of camera information and non-camera information recorded in the normal recording area in normal recording control is referred to as normal information. The normal recording area of the recording medium 120 has a recording capacity sufficient to record normal information for a certain period of time. Overwrite normal information.

While normal recording control is always executed except when the engine is stopped, event recording control is executed only when an event occurs. Establishment of a predetermined event condition corresponds to generation of an event. The in-vehicle control unit 110 determines whether the event condition is established based on camera information or vehicle information, and executes event recording control when the event condition is established. For example, the in-vehicle control unit 110 calculates the absolute value of the acceleration generated in the vehicle CR as a G value based on the acceleration information, and when a G value equal to or greater than a predetermined threshold value is observed, a G event, which is one of the events, occurs. It is determined that the event has occurred (that is, the event condition is satisfied).

When the event condition is satisfied, the in-vehicle control unit 110 sets an event protection period that starts at a timing that is a predetermined time Δt1 before the event condition is satisfied and ends at a predetermined time Δt2 after the event condition is satisfied. Information including camera information and non-camera information in the event protection period is recorded in the event recording area of the recording medium 120 as event information. The in-vehicle control unit 110 may have a buffer memory that temporarily holds camera information and non-camera information for at least a predetermined time Δt1, and may implement event recording control using the buffer memory. The event recording area of the recording medium 120 has a recording capacity enough to record event information for a certain period of time. In event recording control, overwriting of event information in the event recording area is prohibited. Therefore, when a new event occurs in a state where there is no free space in the event recording area, event information about the newly occurring event is not recorded in the event recording area.

[Camera configuration (omnidirectional camera)]
In the first embodiment, it is assumed that the camera section 21 is provided with an omnidirectional camera 21C (see FIG. 3). The omnidirectional camera 21C is a camera capable of simultaneously photographing 360-degree scenery around the omnidirectional camera 21C. Therefore, in the vehicle CR, the omnidirectional camera 21C is installed at the highest position of the vehicle 2, for example, on the roof of the vehicle CR.

FIG. 4(a) is a schematic diagram of the surrounding scenery 500 captured using the omnidirectional camera 21C. The omnidirectional camera 21C captures an omnidirectional landscape 500 . The omnidirectional scenery 500 to be photographed using the omnidirectional camera 21C is the omnidirectional scenery of the vehicle CR. The omnidirectional here means omnidirectional in the horizontal direction. Therefore, the omnidirectional camera 21C has an angle of view of 360° in the horizontal direction, and the omnidirectional scenery 500 photographed by the omnidirectional camera 21C includes the scenery in front of the vehicle CR, the scenery on the right side of the vehicle CR, and the scenery on the right side of the vehicle CR. It will include the scenery on the left side and the scenery behind the vehicle CR. The angle of view of the omnidirectional camera 21C in the vertical direction is 180° or a predetermined value less than 180°, and at least the area immediately below the omnidirectional camera 21C is outside the imaging range of the omnidirectional camera 21C. Such an omnidirectional camera 21C is also called a semi-spherical camera. However, an omnidirectional camera may be used as the omnidirectional camera 21C.

The omnidirectional camera 21C has a curved mirror (corresponding to the shaded area in FIG. 4(a)), collects incident light from all directions based on the scenery 500 by the curved mirror, and arranges it at the focal position below the curved mirror. An image pickup unit (CCD or CMOS image sensor) is used to photograph (i.e., image) the annular scenery 500 reflected on the curved mirror. By this photographing (imaging), an annular photographed image of the scenery 500 is obtained. FIG. 4B shows an annular shot image 510 obtained by shooting the landscape 500 . Image information of an annular shot image 510 generated by the omnidirectional camera 21</b>C is sent to the drive recorder 1 .

The in-vehicle control unit 110 may be able to generate a panorama image 520 in which the ring-shaped photographed image 510 is expanded by performing predetermined development image processing including geometric transformation on the ring-shaped photographed image 510 (FIG. 4(c)). The in-vehicle control unit 110 may not have the function of performing the developed image processing. The panorama image 520 is a planar image obtained by projecting the ring-shaped photographed image 510 onto a predetermined virtual plane. This is a single rectangular image formed by arranging an image of the scenery on the right side, an image of the scenery behind the vehicle CR, and an image of the scenery on the left side of the vehicle CR side by side in the horizontal direction. Any known image processing can be used as the developed image processing for converting the ring-shaped photographed image 510 into a rectangular image such as the panorama image 520 .

Hereinafter, an annular captured image such as the captured image 510 will be referred to as an annular captured image. In the normal recording control or the event recording control, the in-vehicle control unit 110 may record the annular shot image as the image shot by the camera unit 21 in the recording medium 120, or may record the panorama image based on the annular shot image on the camera unit 21. may be recorded on the recording medium 120 as a photographed image. In the following description, it is assumed that an annular photographed image is recorded in the recording medium 120 as an image photographed by the camera unit 21 in normal recording control or event recording control. Regarding an arbitrary image, recording of the image specifically means recording of image information (image data) representing the content of the image.

[Image delivery system]
So far, attention has been paid to one vehicle as the vehicle CR, but the image distribution system according to the present embodiment assumes the existence of n vehicles CR. n is an arbitrary integer of 2 or more. FIG. 5 shows the overall configuration of an image delivery system SS according to this embodiment. The image distribution system SS includes a server device 2 and n in-vehicle devices installed in n vehicles CR. The in-vehicle device includes the drive recorder 1 as a component, and may further include some or all of the peripheral devices of the drive recorder 1 (for example, the camera unit 21 may be considered to be included in the components of the in-vehicle device).

When distinguishing the n vehicles CR belonging to the image distribution system SS, the n vehicles CR are referred to as vehicles CR[1] to CR[n]. Further, the drive recorder 1 mounted on the vehicle CR[i] is particularly referred to as the drive recorder 1[i], and the omnidirectional camera 21C installed on the vehicle CR[i] is particularly referred to as the omnidirectional camera 21C[i]. called. Further, as shown in FIG. 6, a camera unit 21, an in-vehicle sensor unit 22, a GPS processing unit 23, a timer unit 24, an interface device 30, a display unit 31, a speaker 32, and an operation unit 33 installed in the vehicle CR[i] are connected. , in particular referenced 21[i], 22[i], 23[i], 24[i], 30[i], 31[i], 32[i], 33[i] respectively. i is any natural number.

FIG. 7 shows a schematic configuration diagram of the server device 2. As shown in FIG. The server device 2 includes a server control section 210 , a server recording section 220 and a communication processing section 230 .

The server control unit 210 includes a position information acquisition unit 211 , an approaching vehicle determination unit 212 , a target vehicle search unit 213 , an extraction time period setting unit 214 and an image distribution unit 215 . The server control unit 210 is composed of a CPU (Central Processing Unit), a ROM (Read only memory), a RAM (Random access memory), and the like. 211 , the approaching vehicle determination unit 212 , the target vehicle search unit 213 , the clipping time period setting unit 214 and the image distribution unit 215 may be realized.

The server recording unit 220 is a large-capacity recording medium provided in the server device 2 . The communication processing unit 230 is a communication processing block for realizing two-way communication with a device different from the server device 2 . In the image distribution system SS, each drive recorder 1 and the server device 2 are wirelessly connected via a predetermined communication line. Two-way communication of arbitrary signals and information is possible between each drive recorder 1 and server device 2 via a predetermined communication line. The communication line may be a communication line formed via one or more base stations, and is exemplified by a mobile communication line forming a mobile communication network. In the following description, the communication (transmission or reception) of signals and information between the drive recorder 1 and the server device 2 may not be particularly described for the communication processing units 130 and 230. 130 and 230.

The drive recorder 1 of each vehicle CR belonging to the image distribution system SS associates the position information from the GPS processing unit 23 and the time information from the clock unit 24 connected to itself with each other and periodically transmits the information to the server device 2, Those pieces of information received by the communication processing unit 230 are sent to the position information acquisition unit 211 . That is, the position information acquisition unit 211 uses the communication processing unit 230 to execute position information acquisition processing for acquiring position information of each vehicle CR at each time from the drive recorder 1 of each vehicle CR. As shown in FIG. 8, when focusing on the drive recorders 1[1] and 1[2], in the position information acquisition process, the position information acquisition unit 211 uses the communication processing unit 230 to control the drive recorder of the vehicle CR[1]. The position information of vehicle CR[1] at each time is obtained from 1[1], and the position information of vehicle CR[2] at each time is obtained from drive recorder 1[2] of vehicle CR[2]. In other words, in the position information acquisition process, the position information acquisition unit 211 uses the communication processing unit 230 to obtain the latest position information of the vehicle CR[1] from the drive recorder 1[1] of the vehicle CR[1] in real time. Along with the sequential acquisition, the latest position information of the vehicle CR[2] is sequentially acquired in real time from the drive recorder 1[2] of the vehicle CR[2]. The same applies to other vehicles CR.

The position information acquisition process is continuously executed, and each position information acquired by the position information acquisition process is sequentially recorded in the server recording unit 220 while being associated with the corresponding time information. Sufficiently old location information will be discarded from the server storage unit 220, but we ignore the practice of such discarding here. The server control unit 210 (particularly, the adjacent vehicle determination unit 212) collects position information of the vehicle CR[i] at each time accumulated through continuous execution of the position information acquisition process for each vehicle CR[i] belonging to the image distribution system SS. can derive the movement trajectory (in other words, travel history) of the vehicle CR[i]. The locus of movement of the vehicle CR[i] specifies the position of the vehicle CR[i] at any time in the past.

The approaching vehicle determination unit 212 has a function as a target time period setting unit, and the target vehicle search unit 213 has a function as a vehicle image determination unit. The details of the function of are described later.

The first embodiment includes the following examples EX1_1 to EX1_7. The matters described above in the first embodiment are applied to the following examples EX1_1 to EX1_7 unless otherwise stated and inconsistent, and contradict the matters described in the first embodiment in examples EX1_1 to EX1_7. Regarding matters, the description in each embodiment may be given priority. In addition, as long as there is no contradiction, the matter described in any one of the examples EX1_1 to EX1_7 can be applied to any other example (that is, any two or more of the plurality of examples can be applied). It is also possible to combine examples).

[Example EX1_1]
Example EX1_1 will be described. FIG. 9 is a diagram showing the flow of operations of the image distribution system SS according to the embodiment EX1_1. As will be described later in Example EX1_1, in the image delivery system SS, a delivery image based on an image captured by a camera installed in the second vehicle is provided to the in-vehicle device of the first vehicle. The first vehicle in this case is called a target vehicle, and the second vehicle is called a provider vehicle.

In the embodiment EX1_1, the vehicle CR[1] functions as the target vehicle (therefore, hereinafter, the vehicle CR[1] may be referred to as the target vehicle CR[1]), and the driver of the target vehicle CR[1] is It is assumed that the user desires to acquire an image of the exterior of the target vehicle CR[1] photographed from outside the target vehicle CR[1]. In this case, first, the driver of the target vehicle CR[1] inputs a predetermined image distribution request operation for transmitting the above request to the interface device 30[1] installed in the target vehicle CR[1] (Fig. 6 where appropriate). Typically, for example, the driver of the target vehicle CR[ 1 ] inputs a predetermined image distribution request operation corresponding to the above request to the touch panel configured by the display unit 31 and the operation unit 33 . However, the mode of inputting the image distribution request operation is arbitrary, and may be realized by voice operation, line-of-sight operation, or the like.

When an image distribution request operation is input to interface device 30[1], in-vehicle control unit 110[1] transmits host vehicle image distribution request signal 610 to server device 2 in response to the input. The request signal 610 is a signal requesting distribution of an image of the exterior of the target vehicle CR[1] photographed from outside the target vehicle CR[1] to the drive recorder 1[1]. Note that with respect to an arbitrary image, distribution, transmission, and reception of an image specifically mean distribution, transmission, and reception of image information (image data) representing the content of the image.

The request signal 610 includes attention time period information representing the attention time period and appearance feature information. In the image distribution request operation, the driver of the target vehicle CR[1] can specify the attention time zone. If the driver of the target vehicle CR[1] does not specify the attention time zone, the request signal 610 is transmitted from a time a predetermined time (for example, 1 hour or 24 hours) before the transmission time of the request signal 610. The time period up to the time is set as the time period of interest. In this case, the attention time period information may not be included in the request signal 610, and the attention time period may be set on the server device 2 side.

The appearance feature information is information representing the appearance features of the target vehicle CR[1], and includes information representing the vehicle type of the target vehicle CR[1] and the exterior color of the target vehicle CR[1]. The appearance feature information is arbitrary as long as it represents the appearance features of the target vehicle CR[1], and may include three-dimensional shape data of the appearance of the target vehicle CR[1].

When the server device 2 receives the request signal 610 (more specifically, when the communication processing unit 230 receives the request signal 610 from the drive recorder 1[1]), the approaching vehicle determination unit 212 detects the approaching vehicle. A determination process S1 is executed.

In the approaching vehicle determination process S1, the approaching vehicle determination unit 212 determines, among the vehicles CR belonging to the image distribution system SS, the vehicles CR that were in the vicinity of the target vehicle CR[1] during the time period of interest. (in other words, travel history), the vehicle CR that was close to the target vehicle CR[1] during the time period of interest is specified as the provider vehicle, and the target vehicle CR[1] and the provider vehicle are identified. The target time zone and the relative positional relationship are derived in the relationship between the CRs. The target time zone is a time zone in which the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR satisfies a predetermined condition.

FIG. 10 shows a flowchart of the approaching vehicle determination process S1. The approaching vehicle determination process S1 consists of steps S11 to S13. In the approaching vehicle determination process S1, the process of step S11 is first executed.

In step S11, the approaching vehicle determination unit 212 determines the target vehicle CR[1] in the time zone of interest based on the movement trajectory of the target vehicle CR[1] and the movement trajectories of the vehicles CR other than the target vehicle CR[1]. Search for a nearby vehicle CR. For each vehicle CR[i] belonging to the image distribution system SS, the approaching vehicle determination unit 212 determines the position of the vehicle CR[i] from the position information of the vehicle CR[i] at each time accumulated through continuous execution of the position information acquisition process. A movement trajectory (in other words, travel history) can be derived.

In step S12, it is checked whether or not there was a vehicle CR adjacent to the target vehicle CR[1] in the time period of interest. Proceed to step S13. Whether or not the vehicles are close to each other is determined by whether or not the distance between the vehicles is equal to or less than a predetermined distance. A vehicle CR that runs parallel to the target vehicle CR[1] or passes the target vehicle CR[1] in all or part of the time period of interest is determined as a vehicle CR that is close to the target vehicle CR[1]. expected to be When it is determined that there is no vehicle CR in the vicinity of the target vehicle CR[1], the server device 2 responds to the request signal 610 and transmits a predetermined delivery-impossible signal to the target vehicle CR[1]. The data is transmitted to the recorder 1[1], and the processing triggered by the reception of the request signal 610 is completed without performing the processing S3 to S5 described later (see FIG. 9).

In step S13, the approaching vehicle determination unit 212 identifies the vehicle CR that is close to the target vehicle CR[1] in the time period of interest as the provider vehicle CR, and determines the target vehicle CR[1] and the provider vehicle CR in the time period of interest. A time zone in which the distance between CRs is equal to or less than a predetermined distance is derived and set as a target time zone, and the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR at each time within the target time zone is derived.

More specifically, for example, the following may be done. In the approaching vehicle determination process S1, the approaching vehicle determination unit 212 focuses on a vehicle CR[j] other than the target vehicle CR[1] (j is an integer greater than or equal to 2 and less than or equal to n), and Based on the position information of the vehicles CR[1] and CR[j], the distance between the target vehicle CR[1] and the vehicle CR[j] is derived at each time within the time period of interest, and the derived distance is a predetermined distance ( For example, it is determined whether or not there is a time of 50 m or less (step S11).
For vehicle CR[j], if there is a time at which the derived distance is equal to or less than a predetermined distance, vehicle CR[j] is determined to be a vehicle CR that is close to target vehicle CR[1] within the time zone of interest. is specified and set as the provider vehicle. If there is no time at which the derived distance is equal to or less than the predetermined distance for the vehicle CR[j], it is determined that the vehicle CR[j] is not a vehicle CR that is close to the target vehicle CR[1] within the time zone of interest. It is not identified as a provider vehicle with
When the vehicle CR[j] is identified as the provider vehicle, the approaching vehicle determination unit 212 determines the position information of the vehicles CR[1] and CR[j] at each time within the time period of interest. determines and sets a time period in which the distance between the target vehicle CR[1] and the provider vehicle CR[j] is equal to or less than a predetermined distance as a target time period, and determines and sets the target vehicle CR[1] at each time within the target time period. and the relative positional relationship between the provider vehicle CR[j] (step S13). The relative positional relationship between the target vehicle CR[1] and the provider vehicle CR[j] is defined by the orientation of the target vehicle CR[1] as seen from the provider vehicle CR[j], and the distance between the provider vehicles CR[j].

For example, if the movement trajectory of the target vehicle CR[1] in the time period of interest is the trajectory 711 in FIG. 11 and the movement trajectory of the vehicle CR[j] in the time period of interest is the trajectory 712 in FIG. is set as the target time period.

Note that the approaching vehicle determination process S1 does not have to be started after receiving the request signal 610 . That is, for example, the approaching vehicle determination unit 212 regards any vehicle CR belonging to the image distribution system SS as a target vehicle, and then determines the position information of each vehicle CR at each time accumulated through continuous execution of the position information acquisition process. Based on this, the presence or absence of another vehicle CR close to the target vehicle CR and the time zone in which the other vehicle CR was close to the target vehicle CR may be constantly determined. In this case, when the request signal 610 is received by the server device 2, the determination results that have already been made are used to identify the provider vehicle and derive the target time period and the relative positional relationship.

Further, when a plurality of provider vehicles are set, the following processing may be performed for each of the set provider vehicles. Assuming that the vehicle CR[2] is focused on, the details of the processing shown below will be described.

Referring to FIG. 9 again, when the vehicle CR[2] is set as the provider vehicle in the adjacent vehicle determination process S1, the server device 2 (server control unit 210) controls the drive recorder 1[ of the provider vehicle CR[2]. 2], a sampling image transmission request signal 620 is transmitted. This request signal 620 is a signal requesting transmission of the sampling image, and includes target time period information indicating a target time period.

When the request signal 620 is received by the drive recorder 1[2] of the provider vehicle CR[2], the image sampling process S2 is executed by the in-vehicle control unit 110[2] of the drive recorder 1[2]. In the providing source vehicle CR[2], the omnidirectional camera 21C[2] sequentially captures the 360° scenery around the vehicle CR[2] at a predetermined frame rate, and image information (image data ) are sequentially recorded on the recording medium 120[2]. Therefore, at the time the request signal 620 is received, the past images taken by the omnidirectional camera 21C[2] are recorded in the form of moving images in the recording medium 120[2].

As shown in FIG. 12, a moving image consists of a plurality of still images arranged in time series at intervals of a predetermined frame period corresponding to the reciprocal of the frame rate, and the still images are called frame images. A still image obtained by photographing at a certain time of interest is referred to as a frame image of the time of interest. A frame image at each time obtained by photographing by the omnidirectional camera 21C[2] is an annular photographed image as shown in FIG. 4(a).

In the image sampling process S2, the in-vehicle control unit 110[2] extracts one or more images at one or more times belonging to the target time period from the captured images recorded in the recording medium 120[2] in the form of moving images. Extract (capture) the frame image as a sampled image. At this time, if the number of frame images belonging to the target time period is _nA , it is preferable to extract _nB frame images out of the _nA frame images as sampling images. Here, _nA is an integer of 2 or more, and _nB is an integer of 1 or more that is smaller than _nA . The in-vehicle control unit 110 [ 2 ] transmits a response signal 630 including image information of each sampling image to the server device 2 . In the response signal 630, the image information of each sampling image is added with related information such as shooting time information indicating the shooting time of each sampling image.

In Example EX1_1, it is assumed that the number of sampled images (that is, the value of _nB above) is 2 or more. Further, here, as shown in FIG. 13, the target time period is assumed to be the time period from time t _A1 to time t _A7 for concrete explanation, and times t _A1 , t _A2 , t _A3 , t _A4 , t Assume that _A5 , _tA6 and _tA7 are each set to the extraction time of the sampled image. Assuming that the variable i is an arbitrary integer, time t _Ai+1 is later than t _Ai and the interval between times t _A and t _Ai+1 is constant. Therefore, for example, when the target time zone is a one-minute time zone from 13:00:00 to 13:01:00, times t _A1 , t _A2 , t _A3 , t _A4 , t _A5 , t _A6 , t _A7 are respectively 13:00:00, 13:00:10, 13:00:20, 13:00:30, 13:00:40, 13:00:50, 13 It is hour 01:00. Although the sampling image extraction interval in this numerical example is 10 seconds, the sampling image extraction interval can be modified in various ways. However, the sampling image extraction interval is assumed to be longer than the frame cycle (for example, 1/30 second).

In _{the image} sampling process _S2 , the in _{- vehicle} control _unit 110[2 _] performs frame The images are extracted as sampled images SMPL _A1 , SMPL _A2 , SMPL _A3 , SMPL _A4 , SMPL _A5 , SMPL _A6 , SMPL _A7 respectively.

There is a possibility that the target vehicle is photographed in the provider vehicle specified through the approaching vehicle determination process S1. However, at the stage immediately after execution of the approaching vehicle determination process S1, it is unclear whether effective photography is actually performed in the provider vehicle. On the other hand, the image information of moving images is large-capacity data, and transmitting all the image information of moving images to the server device 2 at the stage immediately after execution of the approaching vehicle determination processing S1 is difficult for the server device 2 and the communication line. It's a heavy load and can be wasteful. According to the image sampling processing S2 described above, useless information transmission is suppressed, and the load thereof is reduced.

When the response signal 630 is received by the server device 2 (more specifically, when the response signal 630 from the drive recorder 1 [2] is received by the communication processing unit 230), the target vehicle search unit 213 A search process S3 is executed (see FIGS. 7 and 9).

In the target vehicle search process S3, the target vehicle search unit 213 determines whether the image of the target vehicle CR[1] is included in the sampled image based on the sampling image and the appearance feature information of the target vehicle CR[1]. determine whether If there are a plurality of sampled images, the target vehicle search process S3 is executed for each sampled image. Information included in the request signal 610 can be used as the appearance feature information of the target vehicle CR[1] (see FIG. 9).

With reference to FIG. 14, the target vehicle search process S3 executed for one sampled image 720, which is one of the sampled images SMPL _A1 to SMPL _A7 , will be described. A sampling image 720 provided from the drive recorder 1[2] of the provider vehicle CR[2] to the server device 2 is an annular photographed image by the omnidirectional camera 21C[2]. In the target vehicle search processing S3, first, a developed image processing unit (not shown) provided in the server control unit 210 performs the above-described developed image processing including geometric transformation on the sampled image 720, which is an annular photographed image. , a sampled panorama image 722 obtained by expanding the sampled image 720 is generated. It may be understood that the developed image processing section is provided in the target vehicle search section 213 . The sampling panorama image 722 is a rectangular plane image obtained by projecting the sampling image 720, which is an annulus photographed image, onto a predetermined virtual plane.

The target vehicle search unit 213 detects the image features of the target vehicle CR[1] within the search area set in the entire image area of the sampling panoramic image 722 by image recognition based on the appearance feature information of the target vehicle CR[1]. Search for images that have Then, when an image having the image characteristics of the target vehicle CR[1] is found within the search area, it is determined that the image of the target vehicle CR[1] is included in the sampling image 720, and If no image having the image characteristics of the target vehicle CR[1] is found in , it is determined that the sampled image 720 does not include the image of the target vehicle CR[1].

The search area described above may coincide with the entire image area of the sampled panoramic image 722 . However, it takes a long time to find a specific vehicle from all angles of view of the omnidirectional camera by image recognition, and the processing load is heavy. Also, search errors can easily occur.

In consideration of this, in the target vehicle search process S3, the target vehicle search unit 213 extracts the sampling image, the appearance feature information of the target vehicle CR[1], the target vehicle CR[1] at the shooting time of the sampling image, and the provider It is preferable to determine whether or not the image of the target vehicle CR[1] is included in the sampled image based on the relative positional relationship between the vehicles CR[2]. In other words, regarding the sampling image 720, after the sampling panoramic image 722 is generated as described above, in the entire image area of the sampling panoramic image 722, the target vehicle CR[1] and the provider vehicle CR A search area 724 based on the relative positional relationship between [2] is set (see FIG. 15), and image recognition based on the appearance feature information of the target vehicle CR[1] is performed within the search area 724 in the sampling panoramic image 722. Search for images with image features of vehicle CR[1]. Then, when an image having the image characteristics of the target vehicle CR[1] is found in the search area 724, it is determined that the image of the target vehicle CR[1] is included in the sampling image 720, and the search area If no image having the image characteristics of the target vehicle CR[1] is found in 724, it can be determined that the sampled image 720 does not include the image of the target vehicle CR[1].

Search area 724 is smaller than the full image area of sampled panoramic image 722 . Where in the sampling panorama image 722 the image of the target vehicle CR[1] can exist based on the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR[2] at the shooting time of the sampling image 720 can be estimated, and based on the estimation result, the search area 724 can be set based on the area where the image of the target vehicle CR[1] can exist.

By narrowing down the search area by referring to the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR[2], it is possible to reduce the processing time and load related to the search, and it is also expected to suppress search errors. be done.

At the stage of step 13 in FIG. 10, the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR[j] at each time within the target time zone is derived, and the derived result is used as the target vehicle It can also be used in search processing S3.

Further, it is assumed that the imaging parameter information indicating the correspondence relationship between the direction viewed from the vehicle CR[2] and each pixel in the image captured by the omnidirectional camera 21C[2] is already known to the server device 2, and the imaging parameter information may be included in response signal 630, for example. From the photographing parameter information, it is assumed that each pixel in the photographed image of the omnidirectional camera 21C[2] is a pixel representing an image of the landscape in which direction as viewed from the vehicle CR[2]. do. For example, if the pixels 730N and 730S in the sampled image 720 shown in FIG. 15 are the pixels imaging the scenery located to the north and south respectively when viewed from the vehicle CR[2], the photographing parameter specifying that fact Information is known to the server device 2 . Pixels 730N and 730S in sampled image 720 correspond to pixels 730N' and 730S' in sampled panoramic image 722, respectively. In this case, if the azimuth of the location of the target vehicle CR[1] seen from the provider vehicle CR[2] at the shooting time of the sampling image 720 is north (north is the relative positional relationship between them ), the image region that contains pixel 730N′ but does not contain pixel 730S′ will be set as search region 724. FIG.

Here, the method of searching for the image of the target vehicle CR[1] on the sampling panorama image 722, which is a rectangular planar image, has been described. image may be searched for.

As described above, when the sampled images SMPL _A1 to SMPL _A7 are extracted in the image sampling process S2 (see FIG. 13), the target vehicle search process S3 is executed for each of the sampled images SMPL _A1 to SMPL _A7 . be done.

After the target vehicle search process S3, the cutout time period setting unit 214 executes a cutout time period setting process S4 for setting the cutout time period based on the result of the target vehicle search process S3 (see FIGS. 7 and 9). In the extraction time zone setting process S4, the time zone including the photographing time of the sampling image determined to include the image of the target vehicle CR[1] is set as the extraction time zone among the sampled images SMPL _A1 to SMPL _A7 . do. As shown in FIG. 16(a), the entire target time period may be set as the extraction time period, or as shown in FIG. 16(b) or (c), a part of the target time period may be set as the extraction time period. Sometimes it is done.

For example, when it is determined that the image of the target vehicle CR[1] is included in all of the sampled images SMPL _A1 to SMPL _A7 , the target time period itself is set as the extraction time period.
Further, for example, it is determined that each of the sampling images SMPL _A1 to SMPL _A3 includes an image of the target vehicle CR[1], and all of the sampling images SMPL _A4 to SMPL _A7 include an image of the target vehicle CR[1]. When it is determined that the image is not included, the time period from time t _A1 to time t _A3 is set as the extraction time period.
Further, for example, when it is determined that the image of the target vehicle CR[1] is included only in the sampling image SMPL _A4 among the sampling images SMPL _A1 to SMPL _A7 , the image of the target vehicle CR[ ₁ ] A time period with a length (eg, 10 seconds) is set as the cut-out time period.

Further, for example, it is determined that the image of the target vehicle CR[1] is included in each of the sampling images SMPL _A1 , SMPL _A2 , SMPL _A5 , and SMPL _A6 , and the sampling images SMPL _A3 , SMPL _A4 , and SMPL _A7 include When it is determined that none of the images includes the image of the target vehicle CR[1], the time period from time t _A1 to time t _A6 may be set as the extraction time period, or from time t _A1 . The time period up to time t _A2 or the time period from time t _A5 to time t _A6 may be set as the extraction time period.

If it is determined that the image of the target vehicle CR[1] is not included in any of the sampled images SMPL _A1 to SMPL _A7 , the server device 2 sends a predetermined delivery-impossible signal to the target vehicle CR[1]. is transmitted to the drive recorder 1[1], and the processing triggered by the reception of the request signal 610 is finished without performing the image distribution processing S5 described later.

After setting the clipping time period, the server control unit 210 (for example, the target vehicle searching unit 213, the clipping time setting unit 214, or the image distribution unit 215) distributes the image transmission request signal 640 is transmitted. This request signal 640 is a signal for requesting transmission of an image that is the source of a distribution image, which will be described later, and includes cut-out time period information indicating a cut-out time period.

When the request signal 640 is received by the drive recorder 1 [2] of the provider vehicle CR [2], the in-vehicle control unit 110 [2] of the drive recorder 1 [2] responds to the request of the request signal 640 with a response signal. 650 is sent to the server device 2 . Among the past images captured by the omnidirectional camera 21C[2] recorded in the recording medium 120[2], the response signal 650 includes the image information of the image captured during the clipping time period. For example, if the clipping time period is from 13:00:00 to 13:00:20 for a total of 20 seconds, the 20 seconds from 13:00:00 to 13:00:20 The response signal 650 includes the image information of the image captured by the omnidirectional camera 21C[2]. At this time, frame thinning may be performed as appropriate. In the response signal 650, related information such as shooting time information indicating the shooting time of each shot image is added to the image information.

When the response signal 650 is received by the server device 2 (more specifically, when the response signal 650 from the drive recorder 1 [2] is received by the communication processing unit 230), the image delivery unit 215 performs image delivery processing. S5 is executed (see FIGS. 7 and 9).

In the image distribution process S5, the image distribution unit 215 generates a distribution image based on the image information included in the response signal 650, and uses the communication processing unit 230 to transmit the image distribution signal 660 including the image information of the distribution image to the target vehicle. Send to drive recorder 1[1] of CR[1]. In the drive recorder 1[1] that has received the image distribution signal 660, the in-vehicle control unit 110[1] can record the image information of the distribution image on the recording medium 120[1] based on the image distribution signal 660. , the distribution image can also be displayed on the display unit 31[1].

The captured image included in the response signal 650, which is the source of the distributed image, is the captured image in the clipping time zone among the past captured images by the omnidirectional camera 21C[2] recorded in the recording medium 120[2]. Since it is a captured image in the moving image format, as shown in FIG. 17, it consists of a bundle of a plurality of frame images arranged in time series within the clipping time zone. This bundle forms a moving image 750 . Each frame image that constitutes the moving image 750 is an annular photographed image as a still image generated by photographing by the omnidirectional camera 21C[2]. is called an original image. In the clipping time zone, a plurality of original images are arranged in time series at intervals of the frame period of the omnidirectional camera 21C[2]. Further, here, for the sake of specificity of explanation, it is assumed that the plurality of original images forming the moving image 750 are composed of p original images, and when the p original images need to be distinguished from each other, p original images The original images are referred to as original images I _A [1] to I _A [p] (p is an integer equal to or greater than 2). For any integer i, the original image I _A [i+1] is the image taken after the original image I _A [i].

The image distribution processing S5 includes partial image generation processing (see FIG. 9). In the partial image generation process, for each original image, based on the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR[2] at the shooting time of each original image and the appearance feature information of the target vehicle CR[1] Then, a partial image (corresponding to I _B [i] and I _C [i], which will be described later), which is based on a part of the original image and includes the image of the target vehicle CR[1], is generated. The image distribution unit 215 can generate, as a distribution image, a moving image configured by arranging the partial images generated for the original images I _A [1] to I _A [p] in time series. See below for details).

As a result, it is possible to provide the driver of the target vehicle CR[1] with the delivery image including the image of the target vehicle CR[1], which is a photograph of the appearance of the target vehicle CR[1].

The partial image generation processing will be described with reference to FIG. As partial images that can be generated for the original image I _A [i], there are a partial image I _B [i] and a partial image I _C [i]. The partial image generation process is executed for each original image, and a partial image is generated for each original image. Referring to FIG. ₁₈ , one partial image I _B [ i] and one partial image I _C [i] will be described.

The partial image generation processing includes first partial image generation processing and second partial image generation processing. Since the partial image generation processing is executed for each original image, the first partial image generation processing and the second partial image generation processing are also executed for each original image.

In the first partial image generation process, the image distribution unit 215 generates the original image I _A [i] based on the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR[j] at the shooting time of the original image I A [i]. Estimate the presence image area of the target vehicle CR[1] (that is, the image area in which the image of the target vehicle CR[1] exists) within _A [i]. The image area estimated here is called an estimated area 810 . In the first partial image generation process, the image distribution unit 215 sets a clipped image area 815 in the original image I _A [i] based on the position of the estimated area 810 in the original image I _A [i]. A partial image _I B [i] (first partial image) is obtained by performing predetermined first developed image processing (first image processing) including geometric transformation on the image in the clipped image area 815 at _A [i]. to generate The contents of the developed image processing are as described above. At the stage of step 13 in FIG. 10, the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR[j] at each time within the target time zone is derived, and the image distribution unit 215 The derived results are available.

Partial image I _B [i] is a rectangular planar image obtained by projecting an image in clipped image area 815, which is a part of original image I _A [i], onto a predetermined virtual plane. The shape of the clipped image area 815 is set so that the partial image I _B [i] generated through the projection has a rectangular shape.

A clipped image area 815 is an image area that includes the estimated area 810 . Image information of the center position 812 of the estimation area 810 in the original image I _A [i] appears at a predetermined target position 812′ in the partial image I _B [i] (that is, the center position of the original image I _A [i] A cropped image area 815 is set in the original image I _A [i] such that the pixels of 812 are associated with the pixels of the target position 812′ of the partial image I _B [i] by the first development image processing. The target position 812' here is assumed to be the center position of the partial image I _B [i]. However, the target position 812' may be shifted from the central position of the partial image I _B [i].

If there is no error in the estimation in the first partial image generation process, the image information of the target vehicle CR[1] will appear at the target position 812' in the partial image I _B [i]. However, due to the error in the estimation, the image information of the target vehicle CR[1] often does not appear at the target position 812′ in the partial image I _B [i].

Therefore, after generating the partial image I _B [i], in the second partial image generation process, the image distribution unit 215 generates a target image in the partial image I _B [i] based on the appearance feature information of the target vehicle CR[1]. The image of the vehicle CR[1] is searched to specify the position (center position or center of gravity position) of the image of the target vehicle CR[1] in the partial image I _B [i], and the specified position and the target position 812′ are identified. Derive the position difference of Then, based on the derived positional difference, the position of the clipped image area 815 in the original image I _A [i] is corrected. The cropped image area 815 after correction is represented by a cropped image area 815C. In the second partial image generation processing, the image distribution unit 215 performs predetermined second developed image processing (second image processing) including geometric transformation on the image within the clipped image region 815C in the original image I _A [i]. to generate a partial image I _C [i] (second partial image).

The partial image I _C [i] is a rectangular plane image obtained by projecting the image in the clipped image region 815C, which is a part of the original image I _A [i], onto a predetermined virtual plane. The shape of the clipped image area 815C is set so that the partial image I _C [i] generated through the projection has a rectangular shape.

Image information of the center or center of gravity position of the image of the target vehicle CR[1] in the original image I _A [i] appears at a predetermined target position 812′ in the partial image I _C [i] (that is, the original image I so that the pixel at the center or center of gravity position of the image of the target vehicle CR[1] in _A [i] is associated with the pixel at the target position 812′ of the partial image I _C [i] by the second developed image processing), A cropped image area 815C is set in the original image I _A [i] (that is, the above correction is performed so that such a cropped image area 815C is set).

Image distribution unit 215 generates moving image 770 in which partial images I _C [1] to I _C [p] generated for original images I _A [1] to I _A [p] are arranged in time series. It can be generated as a delivery image (see FIG. 17), and in this case, the image information of the moving image 770 as the delivery image is included in the image delivery signal 660 (see FIG. 9) through the communication processing unit 230. ] is sent to the drive recorder 1 [1].

As a result, it is possible to provide the driver of the target vehicle CR[1] with a good-looking distribution image in which the image of the target vehicle CR[1] is arranged at an appropriate position.

Alternatively, the image distribution unit 215 arranges partial images I _B [1] to I _B [p] generated for the original images I _A [1] to I _A [p] in time series to form a moving image. 760 may be generated as a distribution image (see FIG. 17). In this case, execution of processing for generating partial image I _C [i] is unnecessary, and image information of moving image 760 as a distribution image is included in image distribution signal 660 (see FIG. 9) through communication processing unit 230. is sent to the drive recorder 1[1] of the target vehicle CR[1] in the form of However, in terms of appearance, it is preferable to distribute the moving image 770 .

Alternatively, the image distribution unit 215 may generate a moving image 750 configured by arranging the original images I _A [1] to I _A [p] in time series as a distribution image (see FIG. 17). In this case, execution of processing for generating partial images I _B [i] and I _C [i] is unnecessary, and image information of moving image 750 as a distribution image is transmitted through communication processing unit 230 to image distribution signal 660 ( 9) is transmitted to the drive recorder 1[1] of the target vehicle CR[1]. In this case, the processing for obtaining the moving image 760 or 770 from the moving image 750 may be performed by the drive recorder 1[1] or a computer (not shown).

Here, the partial image I _C [i] is generated through the generation of the partial image I _B [i] _. Searching for the existing position of the image of the target vehicle CR[1] on the original image I _A [1] based on the relative positional relationship between the provider vehicles CR[j] and the external shape information of the target vehicle CR[1]. , the partial image I _C [i] may be generated at once from the original image I _A [1] without generating the partial image I _B [i].

As described above, according to the present embodiment, it is possible to provide the driver of the target vehicle CR[1] with a delivery image including an image of the target vehicle CR[1], which is an image of the appearance of the target vehicle CR[1]. can.
Images (videos) taken of a moving vehicle are called rolling shots, and there is a certain number of car enthusiasts who want to have friends and acquaintances take rolling shots of their own vehicle for viewing. , can be confirmed. According to this embodiment, such needs can be met.
In addition to collecting rolling shots, which are highly hobbyistic, being able to check the appearance of your own vehicle while it is running can also be useful in situations such as when a vehicle lighting system (brake lamp, etc.) breaks down or when a truck collapses. It is also useful for detecting abnormalities that are difficult for drivers to notice, and can also be used to notify drivers of warnings.

[Example EX1_2]
Example EX1_2 will be described. In Example EX1_1, the number of sampled images (that is, the value of _nB described above) was considered to be two or more, but the number of sampled images (that is, the value of _nB described above) may be one. In Example EX1_2, "n _B =1".

In this case, in the image sampling process S2, the in-vehicle control unit 110[2] causes the omnidirectional camera 21C[2] to capture an image at an arbitrary time within the target time period (for example, a time exactly in the middle of the target time period). An image (sampled image SMPL _A4 in the example of FIG. 13) is extracted as one sampled image, and a response signal 630 including image information of the extracted one sampled image is transmitted to the server device 2 . In Example EX1_2, it is assumed below that the image information of the sampling image SMPL _A4 is transmitted to the server device 2 .

The target vehicle search process S3 is performed on the sampled image SMPL _A4 , and only when it is determined that the image of the target vehicle CR[1] is included in the sampled image SMPL _A4 , the above-described extraction time period setting process is performed. S4 and image distribution processing S5 (see FIG. 9) are executed. In the extraction time zone setting process S4, a time zone having a predetermined length of time (for example, 10 seconds) centered on the photographing time t _A4 (see FIG. 13) of the sampling image SMPL _A4 is set as the extraction time zone. The contents of processing after the extraction time zone setting processing S4 are the same as those of the embodiment EX1_1.

Note that if it is determined that the image of the target vehicle CR[1] is included in the sampled image SMPL _A4 , the image distribution process S5 may be executed without executing the extraction time period setting process S4. good. In this case, the transmission/reception of the signals 640 and 650 in FIG. 9 becomes unnecessary, and in the image distribution processing S5, the sampling image SMPL _A4 , which is one still image, is regarded as the original image I _A [1] in FIG. Then, partial images I _B [1] and I _C [1] can be generated from the original image I _A [1] as the sampling image SMPL _A4 . Then, the partial image I _B [1] or I _C [1] based on the sampling image SMPL _A4 may be set as the distribution image. In this case, the image information of the partial image I _B [1] or I _C [1], which is one still image, is sent to the drive recorder 1 [1] of the target vehicle CR [1] as the image information of the distribution image. become. Alternatively, the sampled image SMPL _A4 itself may be set as the distribution image. In this case, the image information of the sampling image SMPL _A4 , which is one still image, is sent to the drive recorder 1[1] of the target vehicle CR[1] as the image information of the distribution image.

[Example EX1_3]
Example EX1_3 will be described. The camera mounted on each vehicle CR may not be the omnidirectional camera 21C. In Example EX1_3, as shown in FIG. 19, it is assumed that each vehicle CR is equipped with a front camera 21F, a rear camera 21B, a right camera 21R, and a left camera 21L. A front camera 21F, a rear camera 21B, a right camera 21R, and a left camera 21L are components of the camera section 21 in FIG. In each vehicle CR, the cameras 21F, 21B, 21R, and 21L sequentially photograph the scenery within their own photographing range at a predetermined frame rate, and generate image information (image data) of photographed images showing photographing results. Sequentially send to drive recorder 1. In each vehicle CR, the image information of the images captured by the cameras 21F, 21B, 21R, and 21L are recorded in the recording medium 120 in the drive recorder 1 by the above-described normal recording control or event recording control.

As shown in FIG. 20, the front camera 21F, the rear camera 21B, the right camera 21R, and the left camera 21L, which are installed with respect to the vehicle CR[i], are denoted by reference numerals "21F[i]" and "21B[i]", respectively. , “21R[i]” and “21L[i]”. The front camera 21F[i], the rear camera 21B[i], the right camera 21R[i], and the left camera 21L[i] have imaging ranges in front, rear, right, and left of the vehicle CR[i], respectively. The front, rear, right, and left landscapes of the vehicle CR[i] are photographed.

Even under the assumption of the embodiment EX1_3, the operation shown in FIG. 9 can be realized. However, the camera installed on each vehicle CR is not the omnidirectional camera 21C, but the cameras 21F, 21B, 21R and 21L, so that the following modifications are made (matters not particularly described in embodiment EX1_3 may be the same as Example EX1_1). It should be noted that the vehicle CR[2] is set as the provider vehicle also in the example EX1_3.

When the server apparatus 2 transmits the sampling image transmission request signal 620 to the drive recorder 1[2] of the vehicle CR[2] through the approaching vehicle determination process S1, the request signal 620 includes camera direction information in addition to target time period information. Include. The camera direction information is information based on the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR[2] at each time point within the target time period. 1] and the provider vehicle CR [2].

At the time of receiving the request signal 620, the past captured images by the cameras 21F[2], 21B[2], 21R[2] and 21L[2] are recorded in the form of moving images in the recording medium 120[2]. ing. A moving image consists of a plurality of still images arranged in time series at intervals of a predetermined frame period corresponding to the reciprocal of the frame rate, and the still images are called frame images as described above. As shown in FIG. 21, for an arbitrary integer i, the frame image obtained by the front camera 21F[2] at time _tBi is referred to by reference symbol "FL _Fi ", and the rear camera 21B at time _tBi The frame image obtained by photographing [2] is referenced by reference numeral “FL _Bi ”, and the frame image obtained by photographing by the right camera 21R [2] at time t _Bi is referenced by reference numeral “FL _Ri ”. The frame image obtained by shooting with the left camera 21L[2] at time t _Bi is referred to by the code "FL _Li ". Assuming that the variable i is an arbitrary integer, time t _Bi+1 is later than t _Bi , and the interval between times t _Bi and t _Bi+1 is constant.

In the image sampling process S2, the in-vehicle control unit 110[2] extracts one or more images at one or more times belonging to the target time period from the captured images recorded in the recording medium 120[2] in the form of moving images. Extract (capture) the frame image as a sampled image. At this time, based on the camera direction information, the target vehicle CR[1] was included in any of the shooting ranges of the cameras 21F[2], 21B[2], 21R[2] and 21L[2] in the target time period. is estimated, and a camera that is estimated to have included the target vehicle CR[1] within the shooting range is set as a sampling target camera. Then, a sampling image is extracted from the shot images obtained by shooting with the sampling target camera.

For example, when the time t _B1 belongs to the target time zone and the time t _B1 is set to the extraction time of the sampling image, the target vehicle CR[1] is the providing source vehicle CR at the time t _B1 based on the camera direction information. [2], the camera to be sampled at time t _B1 is the front camera 21F[2], and the image captured by the camera to be sampled at time t _B1 (that is, the frame image FL _F1 ) is extracted as the sampled image at time _tB1 .
Alternatively, for example, when the time t _B1 belongs to the target time zone and the time t _B1 is set to the extraction time of the sampling image, the target vehicle CR[1] is the providing source vehicle at the time t _B1 based on the camera direction information. If it is found to be located behind CR[2], the camera to be sampled at time t _B1 is the rear camera 21B[2], and the image captured by the camera to be sampled at time t _B1 (that is, the frame image FL _B1 ) is extracted as the sampled image at time t _B1 .

Since the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR[2] can change within the target time period, when extracting a plurality of sampled images, the sampling target camera changes between the plurality of sampled images. I can.

For example, when the times t _B1 and t _B3 belong to the target time zone and the times t _B1 and t _B3 are set to the sampling image extraction times, the target vehicle CR is detected at the time t _B1 based on the camera direction information. [1] is located in front of the provider vehicle CR[2] and the target vehicle CR[1] is located to the right of the provider vehicle CR[2] at time _tB3 . , the camera to be sampled at time t _B1 is the front camera 21F[2], and the image captured by the camera to be sampled at time t _B1 (that is, the frame image FL _F1 ) is extracted as the sampled image at time t _B1 , The camera to be sampled at time _tB3 is the right camera 21R[2], and the image captured by the camera to be sampled at time _tB3 (that is, the frame image FL _R3 ) is extracted as the sampled image at time _tB3 . be. The sampling image extraction interval is arbitrary, but is longer than the frame cycle (for example, 1/30 second) of each camera.

Hereinafter, in Example EX1_3, it is assumed that the front camera 21F[2] is always set as the sampling target camera throughout the target time period unless otherwise specified.

When the server device 2 receives the response signal 630 including the image information of the sampled image, the target vehicle search process S3 is executed by the target vehicle search unit 213 (see FIGS. 7 and 9). The contents of the target vehicle search process S3 may be as described in the embodiment EX1_1, and the target vehicle search unit 213 searches the target vehicle in the sampling image based on the sampling image and the appearance feature information of the target vehicle CR[1]. It is sufficient to determine whether or not the image of the vehicle CR[1] is included. If there are a plurality of sampled images, the target vehicle search process S3 is executed for each sampled image.

The cut-out time period setting unit 214 executes a cut-out time period setting process S4 for setting the cut-out time period based on the result of the target vehicle search process S3. , is transmitted from the server device 2 to the drive recorder 1[2] of the provider vehicle CR[2] (see FIGS. 7 and 9). The extraction time period setting process S4 and the image transmission request signal 640 for distribution may also be as described in the embodiment EX1_1.

When the request signal 640 is received by the drive recorder 1 [2] of the provider vehicle CR [2], the in-vehicle control unit 110 [2] of the drive recorder 1 [2] responds to the request of the request signal 640 with a response signal. 650 is sent to the server device 2 . The response signal 650 includes the image information of the image captured in the cut-out time period among the past images captured by the sampling target camera (here, the front camera 21F[2] is assumed) recorded in the recording medium 120[2]. . For example, if the clipping time period is from 13:00:00 to 13:00:20 for a total of 20 seconds, the 20 seconds from 13:00:00 to 13:00:20 The response signal 650 includes the image information of the image captured by the sampling target camera (the front camera 21F[2] is assumed here). At this time, frame thinning may be performed as appropriate. In the response signal 650, related information such as shooting time information indicating the shooting time of each shot image is added to the image information of each shot image.

When the response signal 650 is received by the server device 2 (more specifically, when the response signal 650 from the drive recorder 1 [2] is received by the communication processing unit 230), the image delivery unit 215 performs image delivery processing. S5 is executed (see FIGS. 7 and 9).

In the image distribution process S5, the image distribution unit 215 generates a distribution image based on the image information included in the response signal 650, and uses the communication processing unit 230 to transmit the image distribution signal 660 including the image information of the distribution image to the target vehicle. Send to drive recorder 1[1] of CR[1]. In the drive recorder 1[1] that has received the image distribution signal 660, the in-vehicle control unit 110[1] can record the image information of the distribution image on the recording medium 120[1] based on the image distribution signal 660. , the distribution image can also be displayed on the display unit 31[1].

In Example EX1_3, the captured image itself included in the response signal 650 may be used as the distribution image. That is, for example, if the cut-out time period is from 13:00:00 to 13:00:20 for a total of 20 seconds, the response signal 650 includes The image captured by the sampling target camera (assumed to be the front camera 21F[2] here) for 20 seconds up to 00 minutes and 20 seconds may be used as the delivery image.

Alternatively, in the image distribution process S5, image processing based on the appearance feature information of the target vehicle CR[1] is performed on the image provided by the provider vehicle CR[2], and the image after the image processing is distributed. It may be generated as an image. That is, for example, if the extraction time zone is a time zone for a total of 20 seconds from 13:00:00 to 13:00:20, 20 seconds from 13:00:00 to 13:00:20 Image processing based on the appearance feature information of the target vehicle CR[1] is performed on the captured image of the sampling target camera (assumed to be the front camera 21F[2] here) for a second, and the image after the image processing is processed by 20 You may generate|occur|produce as a delivery image (delivery image of a moving image format) for seconds. In this case, for example, image processing may be performed such that the image of the target vehicle CR[1] is positioned at the center of each frame image in the distributed image.

Assuming that the front camera 21F[2] is always set as the camera to be sampled, the operation after step S3 related to the embodiment EX1_3 has been described. In order to include the image of the target vehicle CR[1] in each frame image in the distribution image, the relative The server apparatus 2 and the drive recorder 1[2] may operate based on the positional relationship. For example, based on the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR[2] at each time within the cutout time period, the target vehicle CR[1] is the provider vehicle CR in the first half of the cutout time period. If it is found that the target vehicle CR[1] is located in front of [2] and is located on the right side of the provider vehicle CR[2] in the second half section of the cut-out time period, then during the first half section frame images (including, for example, FL _F1 ) captured by the front camera 21F[2] at each time in the second half of the period above, and frames captured by the right camera 21R[2] at each time in the latter half of the period. Images (including, for example, FL _R3 ) are transmitted from the drive recorder 1[2] to the server device 2, and the server device 2 generates a distribution image from a set of received frame images.

Also, in order to transform Example EX1_1 into Example EX1_2, the number of sampled images can be set to 1 in Example EX1_3 as well. You may make it set to a delivery image (in this case, a delivery image becomes a still image).

[Example EX1_4]
Example EX1_4 will be described. In the example of FIG. 9, the request signal 610 includes the appearance characteristic information of the target vehicle CR[1]. It is optional as long as it is before the execution of S3. The appearance feature information of the target vehicle CR[1] may be provided to the server apparatus 2 from a device (not shown) other than the drive recorder 1[1]. In any case, as long as the appearance feature information of the target vehicle CR[1] is provided to the server device 2 before the target vehicle search process S3 is executed, the timing and method of providing the appearance feature information are arbitrary. good

[Example EX1_5]
Example EX1_5 will be described. As described above, the omnidirectional camera 21C generates an annular captured image (annular captured image). The central portion of the ring-shot image becomes a hollow area with no image information. However, the hollow area may not exist in the captured image generated by the omnidirectional camera 21C. In this case, the image information of the scenery right above the vehicle CR exists in the hollow area in the annular shot image. In summary, the captured image generated by the omnidirectional camera 21C may be a circular captured image, and one form of the circular captured image is the annular captured image.

[Example EX1_6]
Example EX1_6 will be described. The server device 2 may be composed of a plurality of server devices that are separated from each other but interconnected by wires or wirelessly. As a result, processing on the server device side can be distributed. For example, when focusing on the target vehicle CR[1] and the provider vehicle CR[2], as shown in FIG. 22, the server device 2 may be configured by server devices 2a, 2b and 2c.

In this case, bidirectional communication of arbitrary signals is performed between the target vehicle CR[1] and the server device 2a, and bidirectional communication of arbitrary signals is performed between the provider vehicle CR[2] and the server device 2b, In addition, by performing two-way communication of arbitrary signals between the server device 2c and the server devices 2a and 2b, the server devices 2a, 2b and 2c as a whole may realize the functions of the server device 2 described above. From the point of view of edge computing, among the server devices 2a, 2b, and 2c, the server device 2a is located closest to the target vehicle CR[1], and the server device 2b is located closest to the provider vehicle CR[2]. It should be placed in the closest position to the

[Example EX1_7]
Example EX1_7 will be described.

In each of the above-described embodiments, when the drive recorder 1[i] (more specifically, for example, the vehicle-mounted control unit 110[i]) receives a transmission request signal (620 or 640) from the server device 2, the transmission of images captured by the cameras (21C[i], 21F[i], 21B[i], 21R[i], 21L[i]) in the time period (target time period, extraction time period) specified by the request signal The image information is transmitted to the server device 2 using the communication processing unit 130[i].

However, the drive recorder 1[i] (more specifically, the in-vehicle control unit 110[i]) uses cameras (21C[i], 21F[i], 21B[i], 21R[i], 21L[i]) is sent to the server at a predetermined transmission timing using the communication processing unit 130[i] regardless of whether or not a transmission request signal is received. An automatic upload method of transmitting to the device 2 may be employed.

The transmission in the automatic upload method may be a so-called real-time transmission. That is, when the camera installed in the vehicle CR[i] is the omnidirectional camera 21C[i] and real-time transmission is performed, the image information of the captured images sequentially generated by the omnidirectional camera 21C[i] is , is sequentially transmitted to the server apparatus 2 substantially simultaneously with generation (in this case, the transmission timing substantially coincides with the generation timing of the image information by the omnidirectional camera 21C[i]). Alternatively, the image information of the captured images sequentially generated by the cameras installed in the vehicle CR[i] is recorded in the recording medium 120[i] for a certain period of time, and then recorded in the recording medium 120[i]. The received image information may be transmitted to the server device 2 at an arbitrary transmission timing.

The automatic upload scheme eliminates the need to send and receive signals 620, 630, 640 and 650 of FIG. 9, simplifying the processing sequence. With the future development of communication networks, the automatic upload method may become a more rational method.

<<Second Embodiment>>
A second embodiment of the present invention will be described. The second embodiment and third and fourth embodiments to be described later are embodiments based on the first embodiment. The description of the embodiments also applies to the second to fourth embodiments. In interpreting the description of the second embodiment, the description of the second embodiment may be prioritized for items that contradict between the first and second embodiments (the same applies to third and fourth embodiments described later). . As long as there is no contradiction, arbitrary plural embodiments may be combined among the first to fourth embodiments.

The second embodiment includes the following examples EX2_1 to EX2_4. As long as there is no contradiction, the matter described in any one of Examples EX2_1 to EX2_4 can also be applied to any other example (that is, any two or more examples among a plurality of examples) can also be combined).

[Example EX2_1]
Example EX2_1 will be described. When the distance between the target vehicle and the provider vehicle is relatively short and wireless communication is possible between the in-vehicle devices of the target vehicle and the provider vehicle (here, between the drive recorder 1), the camera installed in the provider vehicle The delivery of the captured image to the target vehicle may be executed without going through the server device 2 .

It is assumed that the target vehicle and the provider vehicle are the vehicles CR[1] and CR[2], respectively, and this will be explained. In-vehicle control unit 110[1] of drive recorder 1[1] installed in target vehicle CR[1] and in-vehicle control unit 110[2] of drive recorder 1[2] installed in provider vehicle CR[2] tries to establish a communication line by wireless connection between the drive recorders 1[1] and 1[2] using the communication processing units 130[1] and 130[2]. A state in which direct wireless communication between the drive recorders 1[1] and 1[2] is possible using the communication processing units 130[1] and 130[2] is referred to as a vehicle-to-vehicle communication enabled state. In the embodiment EX2_1, hereinafter, it is assumed that the vehicle-to-vehicle communication enabled state is maintained unless otherwise specified. Also, in Example EX2_1, an omnidirectional camera 21C is installed in each vehicle CR.

When the image distribution request operation described above is input to the interface device 30[1], as shown in FIG. to drive recorder 1 [2]. The request signal 860 distributes to the drive recorder 1 [1] an image that is based on the captured image of the camera installed in the provider vehicle CR [2] and that includes the image information of the target vehicle CR [1]. is a signal requesting that The request signal 860 includes the appearance feature information, which is information representing the appearance features of the target vehicle CR[1], and the position information of the target vehicle CR[1] at the current time.

When the drive recorder 1[2] receives the request signal 860, the in-vehicle control unit 110[2] detects that the image of the target vehicle CR[1] is included in the image captured by the omnidirectional camera 21C[2]. A target vehicle search process S3' for determining whether or not there is is executed. The contents of the target vehicle search processing S3′ are the same as the above-described target vehicle search processing S3 (see FIG. 9). ] at the current time. That is, in the target vehicle search process S3′, the annular shot image obtained by the latest shot of the omnidirectional camera 21C[2] is regarded as the sampling image 720 in FIG. Based on the appearance feature information of the target vehicle CR[1], the same processing as the above-described target vehicle search processing S3 may be performed (in this case, the target vehicle search unit 213 in FIG. 7 is incorporated in the vehicle control unit 110[2] You can think that it is done). Based on the location information of the target vehicle CR[1] included in the request signal 860 and the latest location information of the provider vehicle CR[2] obtained from the GPS processing unit 23[2], the in-vehicle control unit 110[2] , the relative positional relationship between the current target vehicle CR[1] and the provider vehicle CR[2] can be recognized, and the target vehicle search process S3' can be performed using the relative positional relationship.

In the target vehicle search process S3′, if it is determined that the image of the target vehicle CR[1] is not included in the image captured by the omnidirectional camera 21C[2] (for example, the image captured at the current time), the in-vehicle control unit 110[2] transmits a predetermined delivery-impossible signal to the drive recorder 1[1] of the target vehicle CR[1]. finish the process.

In the target vehicle search process S3′, when it is determined that the image of the target vehicle CR[1] is included in the image captured by the omnidirectional camera 21C[2] (for example, the image captured at the current time), the in-vehicle control unit 110[2] performs an image delivery process S5'. The content of the image distribution processing S5′ may be the same as the above-described image distribution processing S5 (see FIG. 9). In this case, it can be considered that the original image I _A [i] in FIG. 17 is an annulus photographed image obtained by the latest photographing by the omnidirectional camera 21C[2].

More specific description will be given. In the vehicle-to-vehicle communication enabled state, the in-vehicle control unit 110[1] in the target vehicle CR[1] sequentially transmits the latest position information of the target vehicle CR[1] to the drive recorder 1[2] after transmitting the signal 860. It is assumed that it has been transmitted.

Assuming that the original image I _A [i] is the current-time ring-shot image obtained by the omnidirectional camera 21C[2] in one frame period (see FIG. 17), in the image distribution process S5′ In the included partial image generation processing, the original image I _A [ i] _and including the image of the target vehicle _CR [1]. The method of generating the partial image I _B [i] or I _C [i] based on the original image I _A [i] is as described above. Although the partial image I _B [i] or I _C [i] is generated as the distribution image in the image distribution processing S5′, the original image I _A [i] may be the distribution image.

Each time the omnidirectional camera 21C[2] obtains an annular image, the in-vehicle control unit 110[2] treats it as an original image _IA [i] and obtains a partial image _IB [i] or _IC [i] can be generated.

In the image distribution process S5′, the in-vehicle control unit 110 [2] converts the image information of the original image I _A [i], the partial image I _B [i], or the partial image I _C [i] as the image information of the distribution image. , can be transmitted to the drive recorder 1[1] of the target vehicle CR[1] in the form included in the image distribution signal 870 in real time.

Therefore, assuming that the original images I _A [1], I _A [2], _{I A [} 3], . , the in-vehicle control unit 110[2] can generate a moving image 750, 760 or 770 shown in FIG. 17 as a distribution image. The image information of the moving image 750, 760 or 770 as the distribution image is included in the image distribution signal 870 from the drive recorder 1[2] and transmitted to the drive recorder 1[1] of the target vehicle CR[1]. In the drive recorder 1[1] that has received the image distribution signal 870, the in-vehicle control unit 110[1] can record the image information of the distribution image on the recording medium 120[1] based on the image distribution signal 870. , the distribution image can also be displayed on the display unit 31[1].

For example, when receiving a predetermined distribution stop request signal from the drive recorder 1[1], the in-vehicle control unit 110[2] stops transmission of the distribution image. Further, when the image of the target vehicle CR[1] is not included in the captured image of the omnidirectional camera 21C[2] for a certain period of time or longer, the transmission of the distribution image may be stopped.

[Example EX2_2]
Example EX2_2 will be described. In the embodiment EX2_1, it is assumed that each vehicle CR is equipped with an omnidirectional camera 21C, but the cameras 21F, 21B, 21R and 21L shown in FIG. The method shown in Example EX2_1 can be implemented even when the above-mentioned Example EX2_1 is combined.

That is, when the cameras 21F, 21B, 21R, and 21L are installed in each vehicle CR, when the request signal 860 is received by the drive recorder 1 [2] in the vehicle-to-vehicle communication enabled state, the drive recorder 1 [2] The in-vehicle control unit 110 [2] of
Based on the relative positional relationship between the target vehicle CR[1] and the provider vehicle CR[2] at the current time, the camera 21F[2 ], 21B[2], 21R[2] and 21L[2] as target cameras,
determining whether or not the image of the target vehicle CR[1] is included in the current captured image of the target camera based on the appearance feature information of the target vehicle CR[1];
If the image of the target vehicle CR[1] is included in the current image captured by the target camera, the image itself captured by the target camera or an image obtained by performing predetermined image processing on the image captured by the target camera. , can be transmitted to the drive recorder 1[1] of the target vehicle CR[1] in real time as a distribution image.

[Example EX2_3]
Example EX2_3 will be described. In the examples EX2_1 and EX2_2, the communication between the drive recorders 1[1] and 1[2] installed in the target vehicle CR[1] and the provider vehicle CR[2] is performed by another one as shown in FIG. Via one or more other drive recorders (e.g. 1[3] and 1[4]) installed in one or more vehicles CR (e.g. CR[3] and CR[4]) can be According to this, even if the distance between the target vehicle CR[1] and the provider vehicle CR[2] is large and the direct communication between the drive recorders 1[1] and 1[2] is impossible, the embodiment EX2_1 And it becomes possible to send and receive a delivery image as shown in EX2_2.

[Example EX2_4]
Example EX2_4 will be described. In a situation where the planned travel route of each vehicle CR is set by the navigation device (not shown) set in each vehicle CR, when the vehicles CR[1] and CR[2] are the target vehicle and the provider vehicle, If so, you can do the following.

That is, the in-vehicle control units 110[1] and 110[2] of the drive recorders 1[1] and 1[2] respectively transmit the planned travel routes of the vehicles CR[1] and CR[2] to the server device 2. do. Based on the scheduled travel routes of vehicles CR[1] and CR[2], the server control unit 210 estimates the time zone in which vehicle-to-vehicle communication is possible for the drive recorders 1[1] and 1[2], Necessary signals are transmitted to the drive recorders 1[1] and 1[2] so that delivery images are exchanged as shown in Examples EX2_1 and EX2_2 in the estimated time zone. This enables the target vehicle CR[1] to acquire a photographed image of its own vehicle in real time.

<<Third Embodiment>>
A third embodiment of the present invention will be described. The technology shown in the first and second embodiments uses a camera that captures the exterior of the target vehicle from outside the target vehicle (hereinafter referred to as the provider camera J1 for convenience), and the imaging result of the provider camera J1 is used. In the first and second embodiments, the provider camera J1 is installed in the vehicle, but the provider camera J1 is not installed in the vehicle. good.

For example, the provider camera J1 may be a fixed-point camera installed on a fixed object such as a building. In this case, for example, a device (hereinafter referred to as a fixed point device) having the same or similar configuration as the in-vehicle device provided in the provider vehicle CR[2] is installed on the fixed object, and the fixed point device The operation of the in-vehicle device provided in the provider vehicle CR[2] (that is, the operation of the drive recorder 1[2] shown in the first or second embodiment) may be performed. However, since the position information of the fixed object is unchanged, the fixed point device does not need the GPS processing unit 23, and the position information representing the position (latitude and longitude) of the fixed point device is constant position information.

<<Fourth Embodiment>>
A fourth embodiment of the present invention will be described. The techniques shown in the first and second embodiments use a camera (hereinafter referred to as a provider camera J2 for convenience) that captures the appearance of an object from outside a certain target. This is a technique of providing a distribution image based on the result to a provision destination device. In the first and second embodiments, the target object is a vehicle, but the target object may be other than the vehicle.

The target object is any movable object, and may be, for example, a human being or an animal such as a pet. In this case, the target possesses a device (hereinafter referred to as a target device) having an equivalent or similar configuration to the vehicle-mounted device provided in the target vehicle CR[1], or the target device has a target device. It is attached.

The provider camera J2 may be a camera installed on the provider vehicle CR[2], or may be a fixed-point camera installed on a fixed object such as a building. When the provider camera J2 is a fixed-point camera, a fixed-point device having the same or similar configuration as the in-vehicle device provided in the provider vehicle CR[2] is installed on the fixed object, and the fixed-point device: The operation of the in-vehicle device provided in the provider vehicle CR[2] (that is, the operation of the drive recorder 1[2] shown in the first or second embodiment) may be performed. However, since the position information of the fixed object is unchanged, the fixed point device does not need the GPS processing unit 23, and the position information representing the position (latitude and longitude) of the fixed point device is constant position information.

The provision destination device may be a target device possessed by the target or attached to the target, or may be another device. For example, when the method shown in this embodiment is applied to a watching service, the provision destination device may be, for example, a terminal device (such as a smart phone) possessed by the person watching over the object.

The technology of this embodiment can contribute to watching services for children, the elderly, and the like. Also, if public buses and taxis are included as a type of vehicle CR and each vehicle CR is provided with a provider camera J2, opportunities for photographing the target object can be increased. When the target object is an animal such as a pet, it can be used for behavior management and nuisance prevention of the pet.

<<Consideration of the present invention>>
Consider the invention embodied in some of the above-described embodiments (particularly, for example, the first embodiment).

An image delivery device according to one aspect of the present invention (see FIGS. 7 and 9)
A processing unit that communicates with each in-vehicle device installed in a first vehicle (CR[1]) and a second vehicle (CR[2]), and is an image captured by a camera installed in the second vehicle (21C[2 ], 21F[2], 21B[2], 21R[2] or 21L[2]);
Using the communication processing unit, the position information of the first vehicle at each time is obtained from the in-vehicle device of the first vehicle, and the position information of the second vehicle at each time is obtained from the in-vehicle device of the second vehicle. a position information acquisition unit (211) to
Based on the positional information of the first vehicle at each time and the positional information of the second vehicle at each time, a target time period in which the relative positional relationship between the first vehicle and the second vehicle satisfies a predetermined condition is set. a target time zone setting unit (212);
One or more of the second vehicle captured images in the target time period are referred to as judgment images (sampling images), and the A vehicle image determination unit (213) that determines whether or not the image of the first vehicle is included;
When it is determined that the image of the first vehicle is included in the determination image, the captured image of the second vehicle generated by the camera installed in the second vehicle in all or part of the target time zone and an image distribution unit (215) for transmitting image information of a distribution image based on the communication processing unit to an in-vehicle device of the first vehicle using the communication processing unit.

As a result, it is possible to provide the distribution image expected to include the image of the first vehicle to the in-vehicle device of the first vehicle (for example, the may be provided to the driver).

Although the server device 2 functions as an image delivery device in the above-described several embodiments, the image delivery device may be any device as long as it can realize the above functions.
The sampling images described above are examples of determination images.

In the above-described several embodiments, the drive recorder is shown as an example of the on-vehicle device, or the combination of the drive recorder and the peripheral device is considered as the on-vehicle device. However, the in-vehicle device in the present invention may be any device that is not classified as a drive recorder. For example, a vehicle surroundings monitoring system or a navigation device may function as an in-vehicle device according to the present invention.
The camera installed in the second vehicle may be provided outside the in-vehicle device of the second vehicle and connected to the in-vehicle device of the second vehicle, or may be included as a component of the in-vehicle device of the second vehicle. It can be anything.

In the above-described image delivery device, the target time period setting unit (212) sets the first vehicle and the second vehicle based on the position information of the first vehicle at each time and the position information of the second vehicle at each time. A time period in which the distance between vehicles is equal to or less than a predetermined value may be set as the target time period.

As a result, it is possible to set, as the target time period, the time period in which the exterior of the first vehicle is likely to be captured by the camera installed in the second vehicle. As a result, it is more likely that the delivery image expected to include the image of the first vehicle can be provided to the in-vehicle device of the first vehicle.

The embodiments of the present invention can be appropriately modified in various ways within the scope of the technical idea indicated in the scope of claims. The above embodiments are merely examples of the embodiments of the present invention, and the meanings of the terms of the present invention and each constituent element are not limited to those described in the above embodiments. The specific numerical values given in the above description are merely examples and can of course be changed to various numerical values.

1, 1[i] Drive recorder 2 Server device 21, 21[i] Camera unit 21C, 21C[i] Omnidirectional camera 21F, 21F[i] Front camera 21B, 21B[i] Rear camera 21R, 21R[i] Right camera 21L, 21L[i] Left camera 210 Server control unit 211 Position information acquisition unit 212 Approaching vehicle determination unit 213 Target vehicle search unit 214 Cutout time period setting unit 215 Image distribution unit CR, CR[i] Vehicle SS Image distribution system

Claims (10)

A processing unit that communicates with each in-vehicle device installed in a first vehicle and a second vehicle, and is a communication process capable of receiving image information of a second vehicle captured image that is a captured image of a camera installed in the second vehicle. Department and
Using the communication processing unit, the position information of the first vehicle at each time is obtained from the in-vehicle device of the first vehicle, and the position information of the second vehicle at each time is obtained from the in-vehicle device of the second vehicle. a location information acquisition unit that
Based on the positional information of the first vehicle at each time and the positional information of the second vehicle at each time, a target time period in which the relative positional relationship between the first vehicle and the second vehicle satisfies a predetermined condition is set. a target time zone setting unit;
One or more of the second vehicle captured images in the target time period are referred to as determination images, and the image of the first vehicle is included in the determination images based on the determination images and external feature information of the first vehicle provided in advance. A vehicle image determination unit that determines whether an image is included;
When it is determined that the image of the first vehicle is included in the determination image, the captured image of the second vehicle generated by the camera installed in the second vehicle in all or part of the target time zone an image distribution unit that transmits image information of a distribution image based on the communication processing unit to an in-vehicle device of the first vehicle using the communication processing unit
, image delivery device. The target time period setting unit sets the distance between the first vehicle and the second vehicle to a predetermined value or less based on the position information of the first vehicle at each time and the position information of the second vehicle at each time. Set the time period as the target time period
, The image distribution device according to claim 1. The vehicle image determination unit selects n _B second vehicle captured images out of n _A second vehicle captured images in time series generated by the camera installed in the second vehicle in the target time period. Referenced as a judgment image (n _A is an integer of 2 or more, and n _B is an integer of 1 or more smaller than n _A )
3. The image distribution device according to claim 1 or 2. The vehicle image determination unit performs the determination based on the determination image, the appearance feature information of the first vehicle, and the relative positional relationship between the first vehicle and the second vehicle at the time when the determination image was captured. Determining whether an image of the first vehicle is included in the image
4. The image distribution device according to claim 1. When it is determined that the image of the first vehicle is included in the determination image, the image distribution unit is configured to set the camera mounted on the second vehicle in the clipping time period that is all or part of the target time period. A plurality of time-series captured images of the second vehicle generated by is referred to as a plurality of original images, and the relative positional relationship between the first vehicle and the second vehicle at the time of capturing each original image and the first vehicle executing partial image generation processing for generating a partial image based on a part of the original image and including an image of the first vehicle for each of the original images, based on the appearance feature information of the plurality of Generating, as the distribution image, a moving image configured by arranging a plurality of partial images generated for the original image in time series.
The image delivery device according to any one of claims 1 to 4 . The camera installed on the second vehicle is an omnidirectional camera,
the partial image generation processing includes a first partial image generation processing and a second partial image generation processing;
the first partial image generation process and the second partial image generation process are executed for each original image,
The image distribution unit
in the first partial image generation process, estimating an existing image area of the first vehicle in the original image based on the relative positional relationship between the first vehicle and the second vehicle at the time when the original image was captured; A first partial image is generated by setting a clipped image area in the original image based on the position of the estimated area and performing predetermined first image processing including geometric transformation on the image in the clipped image area. death,
In the second partial image generation process, the position of the image of the first vehicle in the first partial image is specified based on the appearance feature information of the first vehicle, and the position of the clipped image area is specified based on the specified position. generating a second partial image by performing predetermined second image processing including geometric transformation on the corrected image within the clipped image area after correction;
The distribution image is a moving image configured by arranging a plurality of second partial images generated for the plurality of original images in time series.
6. The image delivery device according to claim 5. an image delivery device according to any one of claims 1 to 6;
a first in-vehicle device installed in a first vehicle;
and a second in-vehicle device installed in a second vehicle, wherein
The first in-vehicle device includes a first in-vehicle control unit, a first position information generation unit that sequentially generates position information of the first vehicle, and a first communication processing unit that communicates with the image distribution device. ,
The second in-vehicle device includes a second in-vehicle control unit, a second position information generation unit that sequentially generates position information of the second vehicle, and a second communication processing unit that communicates with the image distribution device. ,
The first in-vehicle control unit sequentially transmits the sequentially generated position information of the first vehicle to the image distribution device using the first communication processing unit,
The second in-vehicle control unit sequentially transmits the sequentially generated position information of the second vehicle to the image distribution device using the second communication processing unit,
a camera installed on the second vehicle, connected to the second vehicle-mounted device or included in a component of the second vehicle-mounted device, generating image information of the second vehicle-captured image at a predetermined frame rate;
The second on-vehicle control unit responds to a transmission request signal from the image distribution device, and transmits the image information of the second captured image of the vehicle in the time period specified by the transmission request signal to the second communication processing unit. or
Image information of the second vehicle captured image sequentially generated by the camera installed in the second vehicle is transmitted to the image distribution device at a predetermined transmission timing using the second communication processing unit.
, image distribution system. Communication processing of receiving image information of a second vehicle captured image, which is a captured image of a camera installed in the second vehicle, using a communication processing unit that communicates with each in-vehicle device installed in the first vehicle and the second vehicle. a step;
Using the communication processing unit, the position information of the first vehicle at each time is obtained from the in-vehicle device of the first vehicle, and the position information of the second vehicle at each time is obtained from the in-vehicle device of the second vehicle. a location information acquisition step for
Based on the positional information of the first vehicle at each time and the positional information of the second vehicle at each time, a target time period in which the relative positional relationship between the first vehicle and the second vehicle satisfies a predetermined condition is set. a target time zone setting step;
One or more of the second vehicle captured images in the target time period are referred to as determination images, and the image of the first vehicle is included in the determination images based on the determination images and external feature information of the first vehicle provided in advance. A vehicle image determination step for determining whether an image is included;
When it is determined that the image of the first vehicle is included in the determination image, the captured image of the second vehicle generated by the camera installed in the second vehicle in all or part of the target time zone and an image distribution step of transmitting image information of a distribution image based on the communication processing unit to an in-vehicle device of the first vehicle using the communication processing unit.
, image delivery method. It has a control unit that acquires image information of a first vehicle that is a target vehicle for image information acquisition, and a second vehicle that is an image captured by a camera installed in the second vehicle from an in-vehicle device of a second vehicle other than the first vehicle. An image information acquisition device capable of receiving image information of a captured image of a vehicle,
The control unit
A position information acquisition process of acquiring position information of the first vehicle at each time from the in-vehicle device of the first vehicle and acquiring position information of the second vehicle at each time from the in-vehicle device of the second vehicle;
Based on the positional information of the first vehicle at each time and the positional information of the second vehicle at each time, a target time period in which the relative positional relationship between the first vehicle and the second vehicle satisfies a predetermined condition is set. Target time zone setting processing;
One or more of the second vehicle captured images in the target time period are referred to as determination images, and the image of the first vehicle is included in the determination images based on the determination images and external feature information of the first vehicle provided in advance. A vehicle image determination process for determining whether an image is included;
a clipping time zone setting process of setting all or part of the target time zone as a clipping time zone when it is determined that the image of the first vehicle is included in the determination image;
an image information acquisition process of acquiring image information of the captured image of the second vehicle in the clipped time period from an in-vehicle device of the second vehicle;
, an image information acquisition device. An image information acquisition method executed by a control unit that acquires image information of a first vehicle that is a target vehicle for image information acquisition, wherein the control unit acquires image information from an in-vehicle device of a second vehicle other than the first vehicle. provided in an image information acquisition device capable of receiving image information of a second vehicle captured image, which is a captured image of a camera installed in two vehicles,
a step of acquiring location information of the first vehicle at each time from an in-vehicle device of the first vehicle;
a step of acquiring location information of the second vehicle at each time from an in-vehicle device of the second vehicle;
Based on the positional information of the first vehicle at each time and the positional information of the second vehicle at each time, a target time period in which the relative positional relationship between the first vehicle and the second vehicle satisfies a predetermined condition is set. a step;
One or more of the second vehicle captured images in the target time period are referred to as determination images, and the image of the first vehicle is included in the determination images based on the determination images and external feature information of the first vehicle provided in advance. determining whether an image is included;
setting all or part of the target time period as an extraction time period when it is determined that the image of the first vehicle is included in the determination image;
and obtaining image information of the captured image of the second vehicle in the clipped time period from an in-vehicle device of the second vehicle.
, image information acquisition method.

Images