KR102675081B1 - Apparatus and method for monitoring and reporting bycatch of porpoise during fishing activities - Google Patents

Abstract

A monitoring device is provided. The monitoring device includes an acquisition unit that acquires image information in which a setting area is captured; It may include a monitoring unit that monitors the presence or absence of specific fish through analysis of the image information.

Description

Apparatus and method for monitoring and reporting bycatch of porpoise during fishing activities}

The present invention relates to devices and methods for monitoring and reporting bycatch.

Bycatch, in which other species are accidentally caught in a net set to catch a specific fish, continues to occur.

Bycatch of jellyfish and other fish can cause great damage to the fishing industry. Bycatch of sharks and other sharks can pose a threat to the protection of endangered species.

Therefore, in order to prevent various bycatch, a system for monitoring and reporting bycatch is required.

Korean Patent Publication No. 2023-0126986 discloses a technology for classifying bycatch fish using size.

Korean Patent Publication No. 2023-0126986

The present invention is intended to provide a device, system, and method for monitoring bycatch of sharks.

The monitoring device of the present invention includes an acquisition unit that acquires image information in which a setting area is captured; It may include a monitoring unit that monitors the presence or absence of specific fish through analysis of the image information.

At this time, the acquisition unit may acquire the image information from a camera that photographs the setting area in a two-dimensional manner.

The monitoring unit may monitor the presence or absence of the specific fish through analysis of the image information in the form of a flat image viewed from above.

According to the present invention, specific bycatch fish can be automatically monitored.

For example, fish species such as Finless Porpoise are protected. However, some of them are suffering damage due to bycatch.

To minimize the damage, bycatch needs to be prevented.

Although it is difficult to fundamentally prevent bycatch in reality, it is possible to significantly reduce the possibility of additional bycatch by monitoring information on the area and time of bycatch and disseminating the monitoring results to each vessel. The monitoring device of the present invention is proposed for this purpose. According to the present invention, the population, location of bycatch, and time of bycatch of specific fish bycaught can be monitored in real time.

Considering the environment of ships such as fishing boats operating at sea away from the central management server, it is recommended that the monitoring device be installed on the ship.

This eliminates the need for huge amounts of video information captured of the work environment to be transmitted to the management server. It is sufficient if only monitoring results containing limited information, such as specific fish type information, population information, bycatch location information, and bycatch time information, are provided to the management server. In this way, according to an environment in which only monitoring results are provided to the management server, the communication load and processing load of the management server targeting numerous ships can be greatly reduced.

In addition, the monitoring device of the present invention can monitor specific fish using image information in which a setting area corresponding to the work site is captured. This could mean that it is possible to monitor specific fish using only surveillance cameras. Accordingly, an environment in which specific fish can be monitored can be established by using existing surveillance cameras installed on ships.

1 is a schematic diagram showing a monitoring device of the present invention.
Figure 2 is a schematic diagram showing the installation position of the camera and the range of the setting area captured by the camera.
Figure 3 is a photo actually taken of the setting area.
Figure 4 is a schematic diagram showing a state in which a shark corresponding to a specific fish is extracted from a photo taken of a setting area.
Figure 5 is a schematic diagram showing monitoring results.
Figure 6 is a flowchart showing the monitoring method of the present invention.
7 is a diagram illustrating a computing device according to an embodiment of the present invention.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts unrelated to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

In this specification, duplicate descriptions of the same components are omitted.

Also, in this specification, when a component is mentioned as being 'connected' or 'connected' to another component, it may be directly connected or connected to the other component, but may be connected to the other component in the middle. It should be understood that may exist. On the other hand, in this specification, when it is mentioned that a component is 'directly connected' or 'directly connected' to another component, it should be understood that there are no other components in between.

Additionally, the terms used in this specification are merely used to describe specific embodiments and are not intended to limit the present invention.

Also, in this specification, singular expressions may include plural expressions, unless the context clearly dictates otherwise.

In addition, in this specification, terms such as 'include' or 'have' are only intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and one or more It should be understood that this does not exclude in advance the presence or addition of other features, numbers, steps, operations, components, parts, or combinations thereof.

Also, in this specification, the term 'and/or' includes a combination of a plurality of listed items or any of the plurality of listed items. In this specification, 'A or B' may include 'A', 'B', or 'both A and B'.

Additionally, in this specification, detailed descriptions of well-known functions and configurations that may obscure the gist of the present invention will be omitted.

1 is a schematic diagram showing a monitoring device of the present invention. Figure 2 is a schematic diagram showing the installation position of the camera 10 and the range of the setting area photographed by the camera 10. Figure 3 is a photo actually taken of the setting area. Figure 4 is a schematic diagram showing a state in which a shark corresponding to a specific fish is extracted from a photo taken of a setting area. Figure 5 is a schematic diagram showing monitoring results.

The monitoring device shown in FIG. 1 may include an acquisition unit 110 and a monitoring unit 130.

The acquisition unit 110 may acquire image information in which the setting area a is captured. As an example, the acquisition unit 110 may acquire corresponding image information from the camera 10 that photographs the setting area a. Alternatively, the acquisition unit 110 may include the camera 10 itself that directly photographs the setting area a.

The monitoring unit 130 can monitor the presence or absence of specific fish through analysis of image information.

As an example, the setting area a may include the vessel 90 where the caught fish is unloaded and the bottom surface of the vessel. At this time, the monitoring unit 130 can analyze video images of fish exposed to the bottom surface of the ship and monitor the presence or absence of specific fish. Analysis of fish included in a video image may increase the amount of calculation or learning depending on the angle of the fish included in the video image. In order to reduce the amount of computation or learning, it is better for the acquisition unit 110 to acquire video images and video information in a form where the fish is viewed from only one direction.

As an example, the acquisition unit 110 may acquire image information from the camera 10 that photographs the setting area a in a two-dimensional manner. At this time, the monitoring unit 130 can monitor the presence or absence of a specific fish through analysis of image information in the form of a flat image viewed from the top down. In this case, the monitoring unit 130 can extract a specific fish from image information as long as it has analysis techniques for the top view of the fish and the side view of the fish.

Looking at Figure 2, which corresponds to a photo in which the setting area a is taken, for example, CCTV (Closed-circuit Television) image data and image information, various obstacles m, fish f, specific fish t, on the bottom of the ship 90 Worker p is mixed.

The monitoring unit 130 may extract a specific fish t included in the image through analysis of image information. For example, when a video image is input, a learned identification model can be prepared to determine the presence or absence of a specific fish. At this time, the monitoring unit 130 can determine or monitor the presence or absence of a specific fish using the identification model. In Figure 3, a bounding box representing the extraction state of a specific fish t represents the extraction state of a specific fish.

If it is determined that a specific fish t exists, the monitoring unit 130 may analyze the population of the specific fish included in the image information.

The monitoring unit 130 can determine the current point in time corresponding to the time when a specific fish was found to exist, and the current location corresponding to the location where the specific fish was found to exist.

The monitoring unit 130 can monitor at least one of the number of objects, current time, and current location in real time.

The monitoring unit 130 may determine whether to transmit the monitoring results to the management server according to the number of objects. For example, depending on a particular species of fish, it may be necessary to restrict fishing in an area even if only one bycatch is caught. However, for other types of fish, it may be advantageous to continue fishing in the area even if a few bycatch are present. In the latter case, the monitoring unit 130 or the communication unit 150 may not transmit the monitoring results to the management server if the number of entities is less than the set value. Of course, differently from this, as long as a specific fish is caught bycatch, monitoring results including the number of fish can be reported to the management server regardless of the number of fish. In this case, the decision on whether to continue operations based on the number of individuals may be delegated to the management server.

A communication unit 150 that communicates with a remote management server may be provided. Unlike the ship 90 that generally operates at sea, the management server may be installed on land.

When the communication unit 150 determines that a specific fish exists by the monitoring unit 130, the current location corresponds to the time at which the specific fish was identified as existing, and the current location corresponding to the location at which the specific fish was identified to exist. can be transmitted to the management server in real time. If necessary, the communication unit 150 may additionally transmit the monitored population of specific fish to the management server. In this case, the communication unit 150 may additionally transmit the original video image from which the specific fish is extracted to the management server so that the management server can re-monitor the specific fish.

At this time, the monitoring information d may take the form of a text file as shown in FIG. 5. The text file is reported to the management server, and accordingly, the monitoring device of the present invention may be referred to as a reporting device.

Monitoring information d includes the current time corresponding to the bycatch time, such as '2022-10-25 01:47'; The current location corresponding to the bycatch location may be included. In addition, the file name 'Test No. 1_1.jpg' of the original image containing a specific fish may be included in the monitoring information d.

For the convenience of the administrator who manages the management server, the monitoring unit 130 may generate map information indicating the current time and location. As an example, the monitoring unit 130 may display the current time and current location on pre-stored map information.

The communication unit 150 may transmit map information including the current time and current location to the management server.

Meanwhile, it may be unreasonable for the above monitoring work to be performed 24 hours a day. For example, if the actual net lifting time is less than a few hours, it is better to monitor specific fish species only during that time.

To this end, the monitoring unit 130 can distinguish in time between a fishing section in which fishing work is performed and a non-fishing section in which fishing work is not performed. Additionally, the monitoring unit 130 can monitor the presence or absence of specific fish only in the fishing section.

The monitoring unit 130 may perform a primary analysis of image information captured at the time of each setting cycle. For example, assume the setting cycle is 1 minute. The monitoring unit 130 may perform the primary analysis at 3:25 p.m. Additionally, the primary analysis can be performed again one minute later at 3:26 PM. The object (analysis target) of the first analysis performed at 3:25 p.m. may be image information captured in setting area a at 3:25 p.m. The object (analysis target) of the first analysis performed at 3:26 p.m. may be image information captured in setting area a at 3:26 p.m.

If it is determined through primary analysis that fish exist in the set area a, the monitoring unit 130 may determine that the fishing section has started.

The monitoring unit 130 may determine that the fishing section has ended when all fish disappear from the set area.

As a result, according to the primary analysis, the operation status at that point in each set cycle can be identified.

The monitoring unit 130 can perform secondary analysis of video information in real time from the start of the operation section to the end of the operation section.

The monitoring unit 130 can monitor the presence or absence of specific fish through secondary analysis. If the monitoring unit 130 determines that the operation section has ended, the secondary analysis may be terminated and the primary analysis may be resumed.

Figure 6 is a flowchart showing the monitoring method of the present invention.

The monitoring method of FIG. 6 can be performed by the monitoring device shown in FIG. 1.

The monitoring method may include an acquisition step (S 510) and a monitoring step (S 520).

In the acquisition step (S510), image information in which the setting area is captured may be acquired. The acquisition step (S510) may be performed by the acquisition unit 110.

The monitoring step (S 520) can monitor the presence or absence of specific fish through analysis of image information. The monitoring step (S520) may be performed by the monitoring unit 130.

According to the present invention discussed above, bycatch of sharks can be monitored during fishing activities. Monitoring at this time may mean determining at least one of the following: determining whether bycatch of sharks, the number of bycaught individuals, location of bycatch, or time of bycatch.

Although the marine mammal, the shark, is designated as a protected marine organism, bycatch frequently occurs during fishing, so protection and conservation measures need to be established. According to the present invention, bycatch monitoring information of sharks can be reported to the management server in real time. Through this, measures such as suspending fishing in areas where sharks are caught bycatch for a set period of time can be taken quickly. In addition, monitoring information accumulated over a long period of time becomes a means of understanding the distribution status of sharks over time, and through this means, a method can be developed to select fishing areas to avoid sharks.

According to the present invention, bycatch information (corresponding to monitoring information including time, location, number of individuals, etc.) can be transmitted to the management server when a bycatch occurs during fishing activities on a CCTV installed on a fishing boat.

According to the present invention, it is possible to recognize and determine the number of sharks by using an AI algorithm (e.g., a grasp model) learned from a video of shark bycatch.

According to the present invention, the location of the bycatch of the shad can be identified using the location information of a separately manufactured black box for a fishing boat.

According to the present invention, monitoring information can be transmitted to the management server using the communication unit 150, such as a router installed on the ship 90.

7 is a diagram illustrating a computing device according to an embodiment of the present invention. The computing device TN100 of FIG. 7 may be a device (e.g., a monitoring device, etc.) described herein.

In the embodiment of FIG. 7, the computing device TN100 may include at least one processor TN110, a transceiver device TN120, and a memory TN130. Additionally, the computing device TN100 may further include a storage device TN140, an input interface device TN150, an output interface device TN160, etc. Components included in the computing device TN100 may be connected by a bus TN170 and communicate with each other.

The processor TN110 may execute a program command stored in at least one of the memory TN130 and the storage device TN140. The processor TN110 may refer to a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. Processor TN110 may be configured to implement procedures, functions, and methods described in connection with embodiments of the present invention. The processor TN110 may control each component of the computing device TN100.

Each of the memory TN130 and the storage device TN140 can store various information related to the operation of the processor TN110. Each of the memory TN130 and the storage device TN140 may be comprised of at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory TN130 may be comprised of at least one of read only memory (ROM) and random access memory (RAM).

The transceiving device TN120 can transmit or receive wired signals or wireless signals. The transmitting and receiving device (TN120) can be connected to a network and perform communication.

Meanwhile, the embodiments of the present invention are not only implemented through the apparatus and/or method described so far, but may also be implemented through a program that realizes the function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded. This implementation can be easily implemented by anyone skilled in the art from the description of the above-described embodiments.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also possible. It falls within the scope of invention rights.

10...camera 90...ship
110...Acquisition Department 130...Surveillance Department
150...Department of Communications

Claims (10)

An acquisition unit that acquires image information in which a set area is captured;
It includes a monitoring unit that monitors the presence or absence of specific fish through analysis of the image information,
The monitoring unit first analyzes the video information captured at the time of the setting period for each setting period,
If the monitoring unit determines that fish exist in the set area through the primary analysis, it determines that the fishing section has started,
The monitoring unit determines that the fishing section has ended when all fish disappear from the set area,
The monitoring unit determines the operation status at the relevant time for each set cycle through the primary analysis,
The monitoring unit secondaryly analyzes the video information in real time from the start of the operation section to the end of the operation section,
The monitoring unit monitors the presence or absence of the specific fish through the secondary analysis,
A monitoring device wherein the monitoring unit terminates the secondary analysis and resumes the primary analysis when it is determined that the operation section has ended.
According to paragraph 1,
The acquisition unit acquires the image information from a camera that photographs the setting area in a two-dimensional manner,
The monitoring unit is a monitoring device that monitors the presence or absence of the specific fish through analysis of the image information in the form of a flat image viewed from above.
According to paragraph 2,
The set area includes the bottom surface of the boat where the caught fish are placed,
The monitoring unit is a monitoring device that analyzes video images of fish exposed to the bottom of the ship and monitors the presence or absence of the specific fish.
According to paragraph 3,
When the video image is input, a learned identification model is prepared to determine the presence or absence of the specific fish,
The monitoring unit is a monitoring device that monitors the presence or absence of the specific fish using the identification model.
According to paragraph 1,
When the monitoring unit determines that the specific fish exists, the monitoring unit analyzes the population of the specific fish included in the image information,
The monitoring unit determines the current point in time corresponding to the time when the specific fish was found to exist, and the current location corresponding to the location where the specific fish was found to exist,
The monitoring unit is a monitoring device that monitors at least one of the number of objects, the current time, and the current location in real time.
According to paragraph 1,
A communication unit is provided to communicate with a remote management server,
If the communication unit determines that the specific fish exists by the monitoring unit, the communication unit determines the current time corresponding to the time when the specific fish was identified as existing and the current location corresponding to the location where the specific fish was identified as existing. A monitoring device that transmits data to the management server in real time.
According to clause 6,
The monitoring unit generates map information showing the current time point and the current location,
The communication unit is a monitoring device that transmits the map information including the current time and the current location to the management server.
According to paragraph 1,
The monitoring unit distinguishes between a fishing section in which fishing work is performed and a non-fishing section in which fishing work is not performed,
The monitoring unit is a monitoring device that monitors the presence or absence of the specific fish only in the fishing section.
delete In a monitoring method performed by a monitoring device,
An acquisition step of acquiring image information in which a set area is captured;
It includes a monitoring step of monitoring the presence or absence of specific fish through analysis of the image information,
The monitoring step is,
For each setting cycle, the video information captured at the time of the setting cycle is first analyzed,
If it is determined that fish exist in the set area through the primary analysis, it is determined that the fishing section has started,
When all fish disappear from the set area, it is determined that the fishing section has ended,
Through the primary analysis, the operation status is determined at each set cycle,
Secondary analysis of the video information in real time from the start of the operation section to the end of the operation section,
Monitoring the presence or absence of the specific fish through the secondary analysis,
A monitoring method for terminating the secondary analysis and resuming the primary analysis when it is determined that the operation section has ended.

Images