JP7433794B2 - Own ship surrounding information storage device and own ship surrounding information display device - Google Patents

Description

The present disclosure relates to a technique for detecting targets around a ship.

A technique for detecting targets around one's own ship is disclosed in Patent Document 1 and the like. In Patent Document 1, a superimposed image of a radar image and a visual landscape is displayed to support vessel maneuvering judgment in a congested sea area.

Japanese Patent Application Publication No. 2010-003274

In Patent Document 1, small ships, wooden ships, people who have fallen overboard, marine creatures, etc. can be detected using visual scenery instead of radar images. However, in Patent Document 1, it is not possible to reduce the visual burden on many sailors, and it is not possible to realize a marine drive recorder.

Therefore, in order to solve the above problems, the present disclosure reduces the visual burden on many sailors when detecting small ships, wooden ships, people who have fallen overboard, marine creatures, etc. that cannot be detected using radar images. At the same time, the purpose is to realize a drive recorder for ships.

In order to solve the above problem, a same time stamp is attached to a radar image and a camera image around the own ship at the same time, and the radar image, the camera image, and the same time stamp are stored in association with each other.

Specifically, the present disclosure includes: an image acquisition unit that acquires a radar image and a camera image around the own ship at the same time; a time adding unit that adds a same time stamp to the radar image and the camera image; An own ship surrounding information storage device comprising: an image storage unit that stores a radar image, the camera image, and the same time stamp in association with each other.

According to this configuration, it is possible to detect AIS-equipped ships, AIS-hidden ships, buoys, aquaculture rafts, etc. located at medium and short distances from the own ship using radar images. On the other hand, using camera images, it is possible to detect AIS-equipped vessels, AIS-hidden vessels, non-AIS-equipped vessels, buoys, aquaculture rafts, people who have fallen overboard, marine life, etc. located within a short distance from the own vessel. By combining the radar image and the camera image, the disadvantages of one can be compensated for by the advantages of the other, and the burden of visual inspection by many sailors can be reduced. Furthermore, by adding the same time stamp to the radar image and the camera image, falsification can be prevented, and a drive recorder for a ship can be realized. Note that AIS concealed vessels include warships and pirate ships, and vessels not equipped with AIS include small vessels and wooden vessels.

Further, in the present disclosure, the time assigning unit synchronizes a time at which the same time stamp is assigned to the radar image and the camera image with a time at which the radar image and the camera image are generated by the own ship. This is an own ship surrounding information storage device that is characterized by:

According to this configuration, the time stamps added to the radar images and camera images are synchronized with the generation time of the radar images and camera images on the own ship. The accuracy of drive recorders can be improved.

Further, in the present disclosure, the image acquisition unit further acquires AIS (Automatic Identification System) information at the same time around the own ship, and the time adding unit further adds the same time stamp to the AIS information. The image storage unit further stores the AIS information and the same time stamp in association with each other.

According to this configuration, AIS-equipped ships located at long distances, intermediate distances, and short distances from one's own ship can be detected using AIS information. By combining AIS information, the weaknesses of one can be compensated for by the strengths of the other, and the burden of visual inspection by many sailors can be reduced. Furthermore, by adding a same time stamp to the AIS information, it is possible to prevent tampering and realize a marine drive recorder. Furthermore, by using AIS information transmitted by other ships, it is possible to guarantee the target position accuracy of radar images and camera images generated by the own ship.

Further, in the present disclosure, the time assigning unit (1) includes a time when the AIS information is acquired by the own ship, and a traveling direction of the other ship measured based on a time change of the radar image and the camera image. and the time when the AIS information was generated on the other ship based on the traveling speed and the deviation between the position of the other ship measured based on the radar image, the camera image, and the AIS information. (2) As a time stamp to be added to the AIS information, set the time when the AIS information was generated on the other ship, which is approximately the same time but earlier than the same time stamp. This is an own ship peripheral information storage device characterized by the following.

According to this configuration, in order to synchronize the time stamp given to AIS information with the generation time of AIS information on another ship, which is a slightly earlier time than the same time stamp, the accuracy of the time stamp given to AIS information is As a result, the accuracy of the marine drive recorder can be improved.

Further, the present disclosure provides an area around own ship, characterized in that the image acquisition unit acquires the camera image from a horizontally rotating camera attached to a horizontally rotating antenna of a radar from which the radar image is acquired. It is an information storage device.

With this configuration, the camera is installed at a high position on the own ship, similar to the radar, so the blind spot of the own ship is reduced, and it can detect small ships, wooden ships, people falling overboard, marine life, etc. in the vicinity of the own ship. can do.

The present disclosure also provides an image reading unit that reads out the radar image and the camera image stored in association with the same time stamp from the own ship surrounding information storage device described above; The own ship surrounding information display device is characterized by comprising: an image display unit that displays the radar image and the camera image around the own ship at the same time in a superimposed manner or in a composite manner.

According to this configuration, the radar image and the camera image can be displayed in a superimposed manner or in a composite manner, and the visual burden on many sailors can be reduced.

In addition, the present disclosure provides that, in a superimposed display or a composite display of the radar image and the camera image around the own ship at the same time, one of the radar image and the camera image is detected, or both of the radar image and the camera image are detected. The own ship surrounding information display device further comprises an object indicator separate section for identifying the type of target based on the object.

According to this configuration, it is possible to identify the type of target (AIS-equipped vessel, AIS concealed vessel, non-AIS-equipped vessel, buoy, aquaculture raft, people falling overboard, marine life, etc.), especially small vessels, wooden vessels, etc. , people who have fallen overboard, marine life, etc. can be detected.

In order to solve the above problem, a same time stamp is attached to a camera image and AIS information around the own ship at the same time, and the camera image, AIS information, and the same time stamp are stored in association with each other.

Specifically, the present disclosure provides an image acquisition unit that acquires a camera image and AIS (Automatic Identification System) information around the own ship at the same time, and a time stamp that adds a same time stamp to the camera image and the AIS information. A own ship surrounding information storage device comprising: a giving section; and an image storage section that stores the camera image, the AIS information, and the same time stamp in association with each other.

According to this configuration, using camera images, it is possible to detect AIS-equipped ships, AIS-hidden ships, non-AIS-equipped ships, buoys, aquaculture rafts, people who have fallen overboard, marine life, etc. that are located a short distance from the own ship. . On the other hand, AIS information can be used to detect AIS-equipped ships located at long distances, intermediate distances, and short distances from one's own ship. By combining camera images and AIS information, the weaknesses of one can be compensated for by the strengths of the other, and the burden of visual inspection by many sailors can be reduced. Furthermore, by adding the same time stamp to the camera image and AIS information, it is possible to prevent falsification and realize a marine drive recorder. Furthermore, by using AIS information transmitted by other ships, it is possible to guarantee the target position accuracy of camera images (which may include radar images) generated by the own ship. Note that AIS concealed vessels include warships and pirate ships, and vessels not equipped with AIS include small vessels and wooden vessels.

Further, the present disclosure is characterized in that the time adding unit synchronizes the time at which the same time stamp is added to the camera image with the time at which the camera image is generated by the own ship. It is a storage device.

According to this configuration, since the time stamp added to the camera image is synchronized with the time when the camera image was generated on the own ship, the accuracy of the time stamp added to the camera image and the accuracy of the marine drive recorder can be improved. .

Further, in the present disclosure, the time assigning unit (1) determines the time when the AIS information was acquired by the own ship, and the traveling direction and traveling speed of the other ship measured based on the time change of the camera image. (2) measuring the time when the AIS information was generated on the other ship based on the deviation between the position of the other ship measured based on the camera image and the AIS information; The area around one's own ship, characterized in that the time stamp added to the information is set to a time when the AIS information was generated on the other ship, which is approximately the same time as the same time stamp, but an earlier time. It is an information storage device.

According to this configuration, in order to synchronize the time stamp given to AIS information with the generation time of AIS information on another ship, which is a slightly earlier time than the same time stamp, the accuracy of the time stamp given to AIS information is As a result, the accuracy of the marine drive recorder can be improved.

Further, in the present disclosure, the image acquisition unit further acquires a radar image around the own ship at the same time, and the time adding unit further adds the same time stamp to the radar image, The storage unit is an own ship surrounding information storage device that further stores the radar image and the same time stamp in association with each other.

According to this configuration, it is possible to detect AIS-equipped ships, AIS-hidden ships, buoys, aquaculture rafts, etc. located at medium and short distances from the own ship using radar images. By combining radar images, the weaknesses of one can be compensated for by the strengths of the other, and the burden of visual inspection by many sailors can be reduced. Furthermore, by adding a same-time stamp to the radar image, tampering can be prevented, and a drive recorder for a ship can be realized.

Further, in the present disclosure, the image acquisition unit acquires the camera image from a horizontally rotating camera attached to a horizontally rotating antenna of a radar from which the radar image is acquired. It is an information storage device.

With this configuration, the camera is installed at a high position on the own ship, similar to the radar, so the blind spot of the own ship is reduced, and it can detect small ships, wooden ships, people falling overboard, marine life, etc. in the vicinity of the own ship. can do.

The present disclosure also provides an image reading unit that reads out the camera image and the AIS information stored in association with the same time stamp from the own ship surrounding information storage device described above; The own ship surroundings information display device is characterized by comprising: an image display section that displays the camera image and the AIS information of the own ship's surroundings at the same time in a superimposed manner or in a composite manner.

According to this configuration, the camera image and the AIS information can be displayed in a superimposed manner or in a composite manner, and the visual burden on many sailors can be reduced.

Further, the present disclosure provides that in a superimposed display or composite display of the camera image and the AIS information around the own ship at the same time, one or both of the camera image and the AIS information is detected. The own ship surrounding information display device further comprises an object indicator separate section for identifying the type of target based on the object.

According to this configuration, it is possible to identify the type of target (AIS-equipped vessel, AIS concealed vessel, non-AIS-equipped vessel, buoy, aquaculture raft, people falling overboard, marine life, etc.), especially small vessels, wooden vessels, etc. , people who have fallen overboard, marine life, etc. can be detected.

As described above, the present disclosure reduces the visual burden on many sailors when detecting small ships, wooden ships, people who have fallen overboard, marine life, etc. that cannot be detected using radar images, and also reduces the visual burden on shipboard drives. A recorder can be realized.

1 is a diagram illustrating an overview of an own ship surrounding information display system according to the present disclosure. FIG. 1 is a diagram showing the configuration of a own ship surrounding information display system according to the present disclosure. FIG. 6 is a diagram illustrating a procedure for own ship surrounding information storage/display processing according to the present disclosure. FIG. 7 is a diagram illustrating the contents of own ship surrounding information storage processing according to the present disclosure. FIG. 7 is a diagram illustrating the contents of own ship surrounding information display processing according to the present disclosure. FIG. 2 is a diagram illustrating an overview of processing for each own ship peripheral object marker according to the present disclosure. FIG. 7 is a diagram illustrating the contents of processing for each own ship peripheral object marker according to the present disclosure. FIG. 7 is a diagram illustrating the effects of processing for each own ship peripheral object marker according to the present disclosure. FIG. 3 is a diagram illustrating high precision of time stamps of radar images and camera images according to the present disclosure. FIG. 3 is a diagram illustrating high precision of a time stamp of AIS information according to the present disclosure. FIG. 2 is a diagram illustrating a specific example of the own ship surrounding information display system of the present disclosure.

Embodiments of the present disclosure will be described with reference to the accompanying drawings. The embodiments described below are examples of implementation of the present disclosure, and the present disclosure is not limited to the following embodiments.

FIG. 1 shows an overview of the own ship surrounding information display system according to the present disclosure. A own ship surrounding information display system D is installed in the wheelhouse of the own ship S. A horizontally rotating antenna of the radar R is installed at a high position on the own ship S. A plurality of cameras C are installed on the horizontally rotating antenna of the radar R.

Since the multiple cameras C are installed at a high position on the own ship S, the blind spot of the own ship S is reduced, and the targets T near the own ship S (in particular, small ships, wooden ships, people falling overboard, marine life, etc.) .) can be detected. If the plurality of cameras C are visible light cameras, targets T around the own ship S can be detected during the daytime. If the plurality of cameras C are infrared cameras, it is possible to detect targets T around own ship S at night, and to detect living things around own ship S (especially people who have fallen overboard, marine life, etc.) by measuring temperature. can also be detected.

FIG. 2 shows the configuration of the own ship surrounding information display system according to the present disclosure. The own-ship surrounding information display system D includes an own-ship surrounding information storage device 1 and an own-ship surrounding information display device 2. The own ship surrounding information storage device 1 includes an image acquisition section 11, a time assignment section 12, and an image storage section 13. The own ship surrounding information display device 2 includes an image reading section 21, an image display section 22, and an object marker classification section 23.

FIG. 3 shows the procedure of own ship surrounding information storage/display processing according to the present disclosure. FIG. 4 shows the contents of own ship surrounding information storage processing according to the present disclosure. FIG. 5 shows the content of own ship surrounding information display processing according to the present disclosure.

First, the own ship surrounding information storage device 1 will be explained. The image acquisition unit 11 acquires radar images, camera images, and AIS information around the own ship S at the same time (step S1). The time adding unit 12 adds a same time stamp to radar images, camera images, and AIS information around the own ship S at the same time (step S2). The image storage unit 13 stores radar images, camera images, and AIS information around the own ship S at the same time, as well as the same time stamp in association with each other (step S3).

The upper part of FIG. 4 shows stored data M1 at time T1. The stored data M1 at time T1 includes a radar image R1, a camera image C1, and AIS information A1 at time T1. The radar image R1 at time T1 includes image information at time T1 and a time stamp. The camera image C1 at time T1 includes image information at time T1, an imaging direction (bow direction, starboard direction, stern direction, port direction, etc.) and a time stamp. The AIS information A1 at time T1 includes a plurality of other ship specific information, a plurality of other ship position information, and a time stamp at time T1.

The lower part of FIG. 4 shows stored data Mn at time Tn. The stored data Mn at time Tn includes a radar image Rn, a camera image Cn, and AIS information An at time Tn. The radar image Rn at time Tn includes image information at time Tn and a time stamp. The camera image Cn at time Tn includes image information at time Tn, an imaging direction (bow direction, starboard direction, stern direction, port direction, etc.) and a time stamp. The AIS information An at time Tn includes a plurality of other ship specific information, a plurality of other ship position information, and a time stamp at time Tn.

The time stamp is provided by a GPS (Global Positioning System) clock, a clock of own ship S, a time certification authority, or the like. In this way, by adding the same time stamp to the radar image, camera image, and AIS information, falsification can be prevented, and a marine drive recorder can be realized.

Next, the own ship surrounding information display device 2 will be explained. The image reading unit 21 reads radar images, camera images, and AIS information around the own ship S at the same time, which are stored in association with the same time stamp, from the own ship surrounding information storage device 1 (step S4). The image display unit 22 superimposes or displays the radar image, camera image, and AIS information around the own ship S at the same time, which are stored in association with the same time stamp (step S5). The object label classification section 23 will be described later.

The upper part of FIG. 5 shows a superimposed display of a radar image, a camera image, and AIS information around the own ship S at time T1. The superimposed screen DS uses the PPI (Plan Position Indicator) screen of the radar R to show target positions P2, P3, P4, and P5 detected in the radar image as circles, and the target positions detected in the camera image. P4, P5, and P6 are indicated by triangle marks, and target positions P1, P2, and P4 detected by AIS information are indicated by square marks.

The lower part of FIG. 5 shows a composite display of the radar image, camera image, and AIS information around the own ship S at time T1. The superimposed screen DS uses the PPI screen of the radar R to show target positions P2, P3, P4, and P5 detected in the radar image as circles, and the target positions P1, P2, and P4 detected in the AIS information. is indicated by a square mark. Around view screens DA1, DA2, DA3, and DA4 show camera images in the bow direction, starboard direction, stern direction, and port direction around the PPI screen of Radar R, and target object positions P4 and P5 detected in the camera images. , P6 are indicated by triangles.

The target positions P4, P5, and P6 detected in the camera images are the distance and direction from each camera C to a predetermined position of own ship S (bow, starboard, stern, port side, etc. of own ship S), and each from camera C to target positions P4, P5, and P6. In this way, by combining radar images, camera images, and AIS information, the weaknesses of one can be compensated for by the strengths of the other (see Figures 6 to 8), reducing the visual burden on many sailors. can do.

FIG. 6 shows an overview of the processing for each own ship peripheral object marker according to the present disclosure. FIG. 7 shows the contents of the processing for each own ship peripheral object marker according to the present disclosure. FIG. 8 shows the effects of the process according to the own ship's surrounding object mark according to the present disclosure.

The object marker separate unit 23 displays one, two, or all of the radar image, camera image, and AIS information in a superimposed display or composite display of the radar image, camera image, and AIS information around the own ship S at the same time. The type of target T is identified based on whether it is detected.

FIG. 6 shows a policy for identifying targets T around own ship S. Using the radar image, it is possible to detect AIS-equipped ships, AIS-hidden ships, buoys, aquaculture rafts, and the like located at relatively intermediate and short distances from own ship S. Using camera images, it is possible to detect AIS-equipped vessels, AIS-hidden vessels, non-AIS-equipped vessels, buoys, aquaculture rafts, people who have fallen overboard, marine creatures, etc. located at a relatively short distance from own ship S. Using AIS information, it is possible to detect AIS-equipped ships located at relatively long, medium, and short distances from own ship S. Note that AIS concealed vessels include warships and pirate ships, and vessels not equipped with AIS include small vessels and wooden vessels.

In order to simplify the explanation, in FIGS. 7 to 10, (1) radar images are used to detect AIS-equipped ships and AIS-hidden ships that are located at relatively intermediate and short distances from own ship S; , (2) detect AIS-equipped vessels, AIS hidden vessels, and non-AIS-equipped vessels that are located relatively short distances from own ship S using camera images, and (3) detect own ship S using AIS information. It is assumed that AIS-equipped ships located at relatively long, medium, and short distances from S are detected.

The upper part of FIG. 7 shows the results of identifying targets T around own ship S at time T1. The superimposed screen DS uses the PPI screen of the radar R to show target positions P2, P3, P4, and P5 detected in the radar image as circles, and shows target positions P4, P5, and P6 detected in the camera image as circles. are indicated by triangle marks, and target positions P1, P2, and P4 detected by AIS information are indicated by square marks.

The target position P1 is located at a relatively long distance from the own ship S, is detected only by AIS information, and is identified as the position of an AIS-equipped ship. The target position P2 is located at a relatively medium distance from the own ship S, is detected only by the radar image and AIS information, and is identified as the position of the AIS-equipped ship. The target position P3 is located at a relatively intermediate distance from the own ship S, is detected only by the radar image, and is identified as the position of the AIS hidden ship. The target position P4 is located at a relatively short distance from the own ship S, is detected by all the radar images, camera images, and AIS information, and is identified as the position of the AIS-equipped ship. The target position P5 is located at a relatively short distance from the own ship S, is detected only by radar images and camera images, and is identified as the position of the AIS hidden ship. The target position P6 is located at a relatively short distance from the own ship S, is detected only by camera images, and is identified as the position of a ship not equipped with AIS.

The lower part of FIG. 7 shows the results of identifying targets T around own ship S at time Tn. The superimposed screen DS uses the PPI screen of the radar R to show the target positions P2', P3', P4', and P5' detected in the radar image as circles, and the target position P6 detected in the camera image. ' is shown by a triangle mark, and target positions P1', P2', and P4' detected by AIS information are shown by square marks.

Target position P1' is located at a relatively long distance from own ship S, is detected only by AIS information, and is identified as the position of an AIS-equipped ship. The target position P2' is located at a relatively medium distance from the own ship S, is detected only by the radar image and AIS information, and is identified as the position of the AIS-equipped ship. The target position P3' is located at a relatively intermediate distance from the own ship S, is detected only by the radar image, and is identified as the position of the AIS hidden ship. Target position P4' is located at a relatively intermediate distance from own ship S, is detected only by radar images and AIS information, and is identified as the position of an AIS-equipped ship. The target position P5' is located at a relatively intermediate distance from the own ship S, is detected only by the radar image, and is identified as the position of the AIS hidden ship. Target position P6' is located at a relatively short distance from own ship S, is detected only by camera images, and is identified as the position of a ship not equipped with AIS.

The identification results of target T at times T1 and Tn around own ship S are compared, and target positions P1, P2, P3, P4, P5, and P6 are compared to target positions P1', P2', and P3, respectively. ', P4', P5', and P6' are indicated by tracking vectors indicated by arrows. In this way, by combining radar images, camera images, and AIS information, the weaknesses of one can be compensated for by the strengths of the other, and the burden of visual inspection by many sailors can be reduced.

In addition, when own ship surrounding information display system D detects AIS concealed ships and non-AIS equipped ships, it tracks and photographs these ships for a while, stores images of these ships for a long period of time, and displays images of these ships. In particular, AIS concealed vessels may be identified and inferred by image recognition.

The upper part of FIG. 8 shows a superimposed display in which AIS information is not superimposed. The superimposed screen DS uses the PPI screen of the radar R to show target positions P2, P3, P4, and P5 detected in the radar image as circles, and shows target positions P4, P5, and P6 detected in the camera image as circles. is indicated by a triangle.

The target position P2 is located at a relatively intermediate distance from the own ship S, is detected by a radar image, and is identified as the position of a ship that is not an AIS-equipped ship. The target position P3 is located at a relatively intermediate distance from the own ship S, is detected by a radar image, and is identified as the position of a ship that is not an AIS-equipped ship. The target position P4 is located at a relatively short distance from the own ship S, is detected by radar images and camera images, and is identified as the position of a ship that is not an AIS-equipped ship. The target position P5 is located at a relatively short distance from the own ship S, is detected by radar images and camera images, and is identified as the position of a ship that is not an AIS-equipped ship. The target position P6 is located at a relatively short distance from the own ship S, is detected by a camera image, and is identified as the position of a ship that is not equipped with an AIS.

The lower part of FIG. 8 shows a superimposed display in which AIS information is superimposed. The superimposed screen DS uses the PPI screen of the radar R to show target positions P2, P3, P4, and P5 detected in the radar image as circles, and shows target positions P4, P5, and P6 detected in the camera image as circles. are indicated by triangle marks, and target positions P1, P2, and P4 detected by AIS information are indicated by square marks.

The target position P1 is identified as the position of the AIS-equipped ship, as in the upper row of FIG. The target position P2 is identified as the position of an AIS-equipped ship among ships that are not non-AIS-equipped ships. The target position P3 is identified as the position of an AIS concealed ship among ships that are not AIS-equipped ships. The target position P4 is identified as the position of an AIS-equipped ship among ships that are not non-AIS-equipped ships. Target position P5 is identified as the position of an AIS concealed ship among ships that are not AIS-equipped ships. The target position P6 is identified as the position of a ship not equipped with AIS, as in the upper row of FIG.

In this way, the type of target T can be narrowed down and identified using the radar image and the camera image together with AIS information. Then, using the AIS information transmitted by other ships, it is possible to guarantee the target position accuracy of the radar images and camera images generated by the own ship S.

FIG. 9 shows how the present disclosure improves the accuracy of time stamps of radar images and camera images. Since a plurality of cameras C are installed on the horizontally rotating antenna of the radar R, the image acquisition unit 11 can acquire the radar image and the camera image at approximately the same time.

Here, if the time stamp _tS,RC of the radar image and camera image is not synchronized with the generation time tG _,RC of the radar image and camera image, the true position of the other ship at the time stamp _tS,RC. is in front of the measurement position of the other ship at the generation time tG _,RC .

However, if the time stamp _tS,RC of the radar image and camera image is synchronized with the generation time tG _,RC of the radar image and camera image, the true position of the other ship at the time stamp _tS,RC is , it is possible to almost match the measured position of another ship at the generation time tG _,RC .

Therefore, the time adding unit 12 synchronizes the time t _S,RC at which the same time stamp is added to the radar image and the camera image with the time t _G,RC at which the radar image and the camera image are generated on the own ship.

In this way, in order to synchronize the time stamps added to radar images and camera images with the generation time of the radar images and camera images on the own ship, the accuracy of the time stamps added to the radar images and camera images, and even the ship drive recorder. The accuracy can be increased.

FIG. 10 shows high precision of the time stamp of AIS information according to the present disclosure. If the reception time of the AIS information substantially coincides with the generation time of the radar image and the camera image, the image acquisition unit 11 can obtain the AIS information at substantially the same time as the radar image and the camera image. Here, the acquisition time of the radar image and the camera image almost coincides with the generation time of the radar image and the camera image, but the reception time of the AIS information is a time later than the generation time of the AIS information.

Then, if the time stamp _tS,A of the AIS information is the same time as the time stamp _tS,RC of the radar image and camera image, the measured position of the other ship at the time stamp _{tS,A is the same as the time stamp tS,} RC of the radar image and camera image. The time stamp t will be behind the measurement position of the other ship at _{S, RC} .

However, if the time stamp _tS,A of the AIS information is earlier than the time stamp _tS,RC of the radar image and camera image, the measured position of the other ship at the time stamp _tS,A is The time stamp t _{S, can be made to almost match the measured position of another ship at RC} .

Therefore, the time assigning unit 12 calculates the time _tR,A at which the AIS information was acquired by the own ship, the traveling direction and traveling speed v of the other ship measured based on the time changes of the radar image and the camera image, and the radar Based on the difference Δd between the position of the other ship measured based on the image, the camera image, and the AIS information, the time t _G,A when the AIS information was generated on the other ship is expressed as t _R,A −Δd/ Measure as v. Note that the deviation Δd between the positions of other ships measured based on radar images, camera images, and AIS information is considered to be small. The positions (circle marks, triangle marks, and square marks) are considered to be the positions of the same other ships.

Then, the time assigning unit 12 assigns a time stamp t _S,A to the AIS information as a time stamp t S, _{RC that is approximately the same time but earlier than the same time stamp t S, RC} of the radar image and the camera image. -Δt, the time _tG,A at which the AIS information was generated on the other ship.

In this way, in order to synchronize the time stamp given to AIS information with the generation time of the AIS information on another ship, which is a slightly earlier time than the same time stamp, the accuracy of the time stamp given to AIS information and the ship The accuracy of drive recorders can be improved.

A specific example of the own ship surrounding information display system of the present disclosure is shown in FIG. In FIG. 1, the own ship S is viewed from the side, but in FIG. 11, the own ship S is viewed from above.

In the upper part of FIG. 11, a bridge is installed near the stern of the own ship S, a radar R is installed on the upper part of the bridge, and a plurality of cameras C are installed on the horizontal rotating antenna of the radar R. However, with only the plurality of cameras C installed on the horizontally rotating antenna of the radar R, a blind spot occurs near the bow of the own ship S having a long hull. Therefore, by using the auxiliary cameras C installed on the starboard and port sides near the bow of the own ship S, no blind spot occurs even near the bow of the own ship S which has a long hull. Then, the image acquisition unit 11 acquires camera images from the plurality of cameras C installed on the horizontally rotating antenna of the radar R and the auxiliary cameras C installed on the starboard and port sides near the bow of the own ship S. good.

In the lower part of FIG. 11, a bridge is installed near the bow of the own ship S, a radar R is installed on the upper part of the bridge, and a plurality of cameras C are installed on the horizontally rotating antenna of the radar R. However, with only the plurality of cameras C installed on the horizontally rotating antenna of the radar R, a blind spot occurs near the stern of the own ship S having a long hull. Therefore, the auxiliary cameras C installed on the starboard and port sides near the stern of the own ship S prevent blind spots from occurring even near the stern of the own ship S, which has a long hull. Then, the image acquisition unit 11 acquires camera images from the plurality of cameras C installed on the horizontally rotating antenna of the radar R and the auxiliary cameras C installed on the starboard and port sides near the stern of the own ship S. good.

In this embodiment, a simultaneous time stamp is attached to a radar image, a camera image, and AIS information, and the radar image, camera image, AIS information, and simultaneous time stamp are stored in association with each other, and the radar image, camera image, and camera image are stored in association with the same time stamp. Display images and AIS information.

As a first modification, a simultaneous time stamp is attached to a radar image and a camera image, the radar image, the camera image, and the same time stamp are stored in association with each other, and the radar image and camera image stored in association with the same time stamp are displayed. You can. As a second modification, a camera image and AIS information are given a same time stamp, the camera image, AIS information, and the same time stamp are stored in association with each other, and the camera image and AIS information stored in association with the same time stamp are displayed. You can.

The own ship surroundings information storage device and own ship surroundings information display device of the present disclosure are useful for visual inspection by a large number of sailors when detecting small ships, wooden ships, people falling overboard, marine life, etc. that cannot be detected using radar images. In addition to reducing the burden on ships, it is possible to realize a drive recorder for ships.

S: Own ship T: Target D: Own ship surrounding information display system R: Radar C: Camera M1, Mn: Memory data R1, Rn: Radar image C1, Cn: Camera image A1, An: AIS information DS: Superimposed screen DA1, DA2, DA3, DA4: Around view screen P1, P2, P3, P4, P5, P6: Target position P1', P2', P3', P4', P5', P6': Target position 1: Self Ship surroundings information storage device 2: Own ship surroundings information display device 11: Image acquisition section 12: Time giving section 13: Image storage section 21: Image reading section 22: Image display section 23: Object marking section

Claims (14)

A radar image and a camera image around the own ship are acquired at the same time, and by superimposing or combining the radar image and the camera image , a target that cannot be detected using one is detected using the other. an image acquisition unit that allows the image to be supplemented with a target that can
a time stamp that stamps the radar image and the camera image at the same time;
an image storage unit that stores the radar image, the camera image, and the same time stamp in association with each other;
An own ship peripheral information storage device comprising: The time adding unit synchronizes the time at which the same time stamp is added to the radar image and the camera image with the time at which the radar image and the camera image are generated by the own ship. Own ship peripheral information storage device according to item 1. The image acquisition unit further acquires AIS (Automatic Identification System) information around the own ship at the same time, and displays one of the radar image, camera image, and AIS information in a superimposed manner or in a composite manner. A target object that cannot be detected using one method can be complemented with a target object that can be detected using the other method,
The time adding unit further adds the same time stamp to the AIS information,
The own ship surrounding information storage device according to claim 1 or 2, wherein the image storage unit further stores the AIS information and the same time stamp in association with each other. The time assigning unit includes (1) the time at which the AIS information was acquired by the own ship, the traveling direction and traveling speed of the other ship measured based on the time changes of the radar image and the camera image, and the (2) measuring the time when the AIS information was generated on the other ship based on a deviation between the position of the other ship measured based on the radar image, the camera image, and the AIS information; The time stamp added to the AIS information is set to a time when the AIS information was generated on the other ship, which is approximately the same time as the same time stamp, but an earlier time. The own ship surrounding information storage device according to claim 3. Any one of claims 1 to 4, wherein the image acquisition unit acquires the camera image from a horizontally rotating camera attached to a horizontally rotating antenna of a radar from which the radar image is acquired. Own ship surrounding information storage device described in . an image reading unit that reads out the radar image and the camera image stored in association with the same time stamp from the own ship surrounding information storage device according to any one of claims 1 to 5;
an image display unit that displays the radar image and the camera image around the own ship at the same time, which are stored in association with the same time stamp, in a superimposed manner or in a composite manner;
An own ship surrounding information display device comprising: In the superimposed display or composite display of the radar image and the camera image around the own ship at the same time, an object is detected based on whether it is detected in one or both of the radar image and the camera image. The own ship surrounding information display device according to claim 6, further comprising an object mark separate section for identifying the type of mark. Obtain camera images and AIS (Automatic Identification System) information around the own ship at the same time, and use superimposed display or composite display of the camera image and the AIS information to detect targets that cannot be detected using either one. an image acquisition unit capable of supplementing with a target that can be detected using the other;
a time stamp unit that stamps the camera image and the AIS information at the same time;
an image storage unit that stores the camera image, the AIS information, and the same time stamp in association with each other;
An own ship peripheral information storage device comprising: 9. The vicinity of own ship according to claim 8, wherein the time adding unit synchronizes the time at which the same time stamp is added to the camera image with the time when the camera image is generated on the own ship. Information storage device. The time assigning unit includes (1) the time when the AIS information was acquired by the own ship, the traveling direction and traveling speed of the other ship measured based on the time change of the camera image, the camera image and the ship; (2) measuring the time when the AIS information was generated on the other ship based on the deviation between the positions of the other ships measured based on the AIS information; and (2) a time stamp to be added to the AIS information. As set forth in claim 8 or 9, the AIS information is set to a time at which the AIS information was generated on the other ship, which is approximately the same time but earlier than the same time stamp. Own ship surrounding information storage device. The image acquisition unit further acquires a radar image around the own ship at the same time, and displays the camera image, the AIS information, and the radar image in a superimposed manner or in a composite manner , and performs detection using one of them. It is possible to complement a target that cannot be detected with a target that can be detected using the other,
The time adding unit further adds the same time stamp to the radar image,
The own ship surrounding information storage device according to any one of claims 8 to 10, wherein the image storage unit further stores the radar image and the same time stamp in association with each other. The own ship according to claim 11, wherein the image acquisition unit acquires the camera image from a horizontally rotating camera attached to a horizontally rotating antenna of a radar from which the radar image is acquired. Peripheral information storage device. an image reading unit that reads the camera image and the AIS information stored in association with the same time stamp from the own ship surrounding information storage device according to any one of claims 8 to 12;
an image display unit that superimposes or displays the AIS information and the camera image around the own ship at the same time stored in association with the same time stamp;
An own ship surrounding information display device comprising: In the superimposed display or composite display of the camera image and the AIS information around the own ship at the same time, an object is detected based on whether it is detected by one or both of the camera image and the AIS information. The own ship surrounding information display device according to claim 13, further comprising an object mark separate section for identifying the type of mark.

Images