JP2020170319A - Detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a detection device for reducing a processing amount and improving the detection accuracy of an individual object. SOLUTION: A detection device is installed in a moving body and acquires image information around the moving body, a storage unit for storing information about a plurality of trained models, and image information acquired in the imaging unit. As input data, using the trained model stored in the storage unit, the detection unit that detects the target object, the sensor that acquires the surrounding environment information of the moving object, and the information including the surrounding environment information acquired by the sensor. Based on this, a switching determination unit for switching the trained model used in the detection unit is provided. [Selection diagram] Fig. 1

Description

The present invention relates to a detection device.

In recent years, technologies for speeding up CPUs (Central Processing Units) and GPUs (Graphics Processing Units), increasing memory capacities, and machine learning have been rapidly advancing. For this reason, machine learning using learning data on the order of hundreds of thousands to millions has become possible, and highly accurate identification technology and classification technology are being established (see Non-Patent Document 1).

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia (pp. 675-678) ). ACM.

In deep learning, in order to detect or recognize a wide variety of objects in a general-purpose manner, the same network is used, and learning data is changed or added according to the purpose to recognize various objects. There is.

However, although the above method has high versatility, there is a problem that the detection accuracy of each object is low. Further, as the number of types of objects to be detected increases, there is a problem that the amount of calculation for determining which of the detection objects is the detection object after being detected in the network increases.
Therefore, the present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a detection device for reducing the processing amount and improving the detection accuracy of individual objects.

Embodiment 1; One or more embodiments of the present invention include an imaging unit that is installed in a moving body and acquires image information around the moving body, a storage unit that stores information about a plurality of learned models, and the imaging unit. A detection unit that detects an object using a learned model stored in the storage unit using image information acquired in the unit as input data, a sensor that acquires information on the surrounding environment of the moving object, and the sensor. We propose a detection device including a switching determination unit for switching a learned model used in the detection unit based on the information including the surrounding environment information acquired by the above-mentioned.

Embodiment 2; One or more embodiments of the present invention are characterized in that the plurality of trained models stored in the storage unit are specialized trained models according to the image information or the object. We are proposing a detection device.

Embodiment 3; In one or more embodiments of the present invention, the sensor obtains the surrounding environment information including at least one of traveling position information of the moving body, time information, and illuminance information around the moving body. We are proposing a detection device that is characterized by acquisition.

Embodiment 4; One or more embodiments of the present invention include a switching information generation unit that outputs switching information of the learned model to the switching determination unit, and the switching information generation unit can be used at arbitrary time intervals. Of the plurality of learned models stored in the storage unit, switching information for switching the learned model used in the detection unit to another learned model is output to the switching determination unit, and a deep layer in the detection unit. We propose a detection device that outputs switching information to the learned model having the highest score for learning similarity to the switching determination unit.

Embodiment 5; In one or more embodiments of the present invention, the switching information generation unit may use the switching information generation unit for an arbitrary time when the image information acquired by the imaging unit has a predetermined range of similarity in time series. At intervals, switching information for switching the learned model used in the detection unit to another learned model among the plurality of learned models stored in the storage unit is output to the switching determination unit, and the detection is performed. We are proposing a detection device that outputs switching information to the learned model having the highest score for the similarity of deep learning in the unit to the switching determination unit.

Embodiment 6; In one or more embodiments of the present invention, the moving body is a forklift that travels indoors or outdoors, and the storage unit is at least a daytime specialized trained model and a nighttime specialized trained model. The sensor stores at least time information or illuminance information around the forklift as the surrounding environment information of the forklift, and the detection unit detects the object that the forklift may collide with. We have proposed a detection device including a notification unit that detects the above and notifies the operator of the forklift of the detection result of the object in the detection unit.

Embodiment 7; In one or more embodiments of the present invention, the moving body is a forklift traveling indoors, and the storage unit is at least an indoor specialized trained model or a person-specific trained model. The sensor stores at least the traveling position information as the peripheral environment information of the forklift, and the detection unit detects the object that the forklift may collide with, and the detection unit in the detection unit. The detection device according to claim 1, further comprising a notification unit for notifying the operator of the forklift of the detection result of the object.

According to one or more embodiments of the present invention, there is an effect of reducing the amount of processing and improving the detection accuracy of individual objects.

It is a figure which shows the structure of the detection apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows the structure in the storage part in the detection apparatus which concerns on 1st Embodiment of this invention. It is a figure which illustrated the specialized learning model which concerns on 1st Embodiment of this invention. It is a figure which illustrated the specialized learning model which concerns on 1st Embodiment of this invention. It is a figure which illustrated the specialized learning model which concerns on 1st Embodiment of this invention. It is a figure which shows the structure of the sensor in the detection apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows the structure in the control part in the detection apparatus which concerns on 1st Embodiment of this invention. It is a processing flow of the detection apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows the structure in the control part in the detection apparatus which concerns on 2nd Embodiment of this invention. It is a processing flow of the detection apparatus which concerns on 2nd Embodiment of this invention. It is a processing flow between frames in the detection apparatus which concerns on 2nd Embodiment of this invention. It is a figure which shows the structure in the control part in the detection apparatus which concerns on 3rd Embodiment of this invention. It is a processing flow of the detection apparatus which concerns on 3rd Embodiment of this invention. It is a processing flow between frames in the detection apparatus which concerns on 3rd Embodiment of this invention.

<First Embodiment>
Hereinafter, the first embodiment of the present invention will be described with reference to FIGS. 1 to 8.
In the following, a forklift will be described as an example of a moving body.

<Configuration of detection device>
As shown in FIG. 1, the detection device 1 according to the present embodiment includes an imaging unit 10, a storage unit 20, a control unit 30, a sensor 40, and a notification unit 50, and is composed of a control unit. 30 is configured to include a detection unit 37 and a switching determination unit 35.

The image pickup unit 10 is installed on the forklift as a moving body, and acquires image information around the forklift as a moving body. The imaging unit 10 is, for example, a camera, and is attached at a position where an image in the forward traveling direction of the inner or outer portion of the head guard of the forklift as a moving body can be captured. Further, the mounting direction of the camera is not particularly limited as long as an image in the forward traveling direction of the forklift as a moving body can be taken. In addition, when one camera is affected by a blind spot due to the mast or fork of the forklift as a moving body, two or more cameras comprehensively capture an image of the forklift as a moving body in the forward traveling direction. You may try to do it. Further, a pedestal may be provided on the outside of the head guard to fix the camera. Further, a mechanism may be provided so that the camera rotates according to the traveling direction (for example, forward and backward) of the forklift as a moving body.

Further, the camera may be a wireless camera or a wired camera. Further, the camera may be a still camera or a movie camera. Further, the imaging unit 10 may be a sensor such as LIDAR, depending on the object.

The storage unit 20 stores information about a plurality of trained models. The storage unit 20 may be a non-volatile memory or the like, or a hard disk or the like. Further, it may be a server or the like connected to the detection device 1 in the network.

Here, as a plurality of trained models stored in the storage unit 20, as shown in FIG. 2, a vehicle-specific trained model 20A, a daytime-specialized trained model 20B, and a night-time-specialized trained model are used. 20C, indoor specialized learning model 20D, outdoor specialized learned model 20E, obstacle specialized learned model 20F, person specialized learned model 20G, and the like can be exemplified.

The learning model used for general deep learning was constructed from a wide variety of data such as people, cars, animals, buildings, landscapes, etc. in order to have versatility, but in the present embodiment, the detection device 1 For example, as shown in FIG. 3, a vehicle-specific trained model 20A based on learning data consisting only of cars such as an image of a passenger car, an image of a large truck, an image of a light vehicle, and an image of a bus, FIG. As shown, a daytime specialized trained model 20B based on training data consisting only of daytime data such as daytime images, weather images, sky color images, and brightness images, as shown in FIG. Night-specific trained model 20C based on training data consisting only of night data such as night images, headlight images, taillight images, and street images, as shown in Fig. 5, storage shelf images and Indoor specialized trained model 20D based on training data consisting only of indoor data such as palette images, indoor equipment images, indoor sign images, truck images and pedestrian images, as shown in FIG. Outdoor specialized trained model 20E based on training data consisting only of outdoor data such as bike image, building image, electric pole image, pylon image, concrete block image, palette image as shown in Fig. 3. Obstacle-specific trained model 20F based on training data consisting only of obstacle data such as images, barricade images, cushion drum images, guard fence images, male images and female images as shown in Fig. 3. Storage unit 20 stores a person-specific trained model 20G based on training data consisting only of person data such as images of people, adults, children, people wearing hats, and images of people wearing helmets. Remembers.

The above-mentioned trained model is generated by inputting specific training data in advance into a neural network (NN) and performing deep learning. For example, the vehicle-specific trained model 20A is generated by inputting training data consisting only of vehicles into a neural network (NN) and performing deep learning. The trained model may be generated from the captured image data input from the imaging unit 10 during a test run before full-scale operation, or the image information of the actual operating site is listed for each detection target. A trained model may be generated based on the data.

The control unit 30 controls the operation of the entire detection device 1 according to a control program stored in a ROM (Read Only Memory) or the like (not shown). In the present embodiment, for example, the detection process in the detection unit 37 and the switching determination process in the switching determination unit 35, which will be described later, are controlled. The details of these controls will be described later.

The sensor 40 acquires information including information on the surrounding environment of the forklift as a moving body. For example, as shown in FIG. 6, the sensor 40 detects the time information 40B by combining a GPS receiver that detects the traveling position information 40A of the forklift as a moving body, image information and map information, a steering angle sensor, and the like. A timer, a clock, an illuminance sensor that acquires illuminance information 40C around the forklift as a moving body, an acceleration sensor that detects acceleration information 40D during forklift running as a moving body, and speed information 40E during forklift running as a moving body. An example of a speed sensor or the like can be used.

The notification unit 50 notifies, for example, the operator of the forklift as a moving body of the detection result of the target body in the detection unit 37. Here, the notification unit 50 may be a display device that visually notifies the danger as a result of detection of the target body by the detection unit 37, a speaker that notifies by voice, or, for example, a forklift as a moving body. It may be something like a vibrator that notifies the operator of the detection result by vibrating the seat, and is not particularly limited.

<Structure of control unit 30>
As shown in FIG. 7, the control unit 30 in the detection device 1 according to the present embodiment includes a collection unit 31, an image capture image database (DB) 32, an extraction unit 33, a detection target data set 34, and a switching determination unit. It includes 35, a trained data set 36, a detection unit 37, and a detection result database (DB) 38.

The collecting unit 31 collects the captured image from the imaging unit 10 via a communication unit 11 (not shown). Here, the collected image information is stored in the captured image database (DB) 32.

The extraction unit 33 extracts a captured image as a positive example used for deep learning from the captured image database (DB) 32 and stores it in the detection target data set 34.

On the other hand, the switching determination unit 35 selects a suitable learned model stored in the storage unit 20 based on the information including the surrounding environment information of the forklift as a moving body obtained from the sensor 40, and the trained model. Is stored in the trained data set 36.

For example, when the vehicle is traveling in a dangerous place indoors based on the traveling position information 40A from the sensor 40, the switching determination unit 35 receives the indoor specialized learning model 20D or the obstacle specialized learning from the storage unit 20. The trained model 20F is selected, and the trained model is stored in the trained data set 36.

Further, when the time is in the night zone based on the time information 40B from the sensor 40, the switching determination unit 35 selects the night-specialized learning model 20C from the storage unit 20 and has learned the learned model. Store in dataset 36.

Further, when the illuminance is low due to the illuminance information 40C around the forklift as a moving body from the sensor 40, the switching determination unit 35 has already learned the indoor specialized learning model 20D or the night specialized learning from the storage unit 20. Model 20C is selected and the trained model is stored in the trained dataset 36.

The detection unit 37 detects the target body using the image information acquired by the imaging unit 10 as input data and the trained model stored in the trained data set 36 by the switching determination unit 35. More specifically, the detection unit 37 includes a detection target data set 34 stored by the extraction unit 33 and a learned data set (for example, a vehicle-specific trained model 20A) stored by the switching determination unit 35. For example, when the degree of deviation of the score with respect to the similarity is calculated and the degree of deviation is equal to or less than a predetermined value, the data in the detection target data set 34 for which the degree of deviation is calculated is the desired detector. It detects that there is, and stores the data in the detection result DB 38. Here, the target body includes an object, a human body, a moving body, a lane, and the like.

In the above, an example in which the switching of the specialized learned model stored in the storage unit 20 is executed by the determination process of the switching determination unit 35 has been shown, but the specialized learned model stored in the storage unit 20 has been shown. A manual switching unit for manually switching the model may be provided so that the specialized trained model can be switched at the discretion of the operator of the forklift as a moving body.

<Processing of detection device>
The process of the detection device 1 according to the present embodiment will be described with reference to FIG.

The image capturing unit 10 acquires image information around the forklift as a moving body (step S101). The collecting unit 31 collects the captured image from the imaging unit 10 and stores the collected image information in the captured image database (DB) 32 (step S102).

The switching determination unit 35 selects a suitable learned model stored in the storage unit 20 based on the information including the surrounding environment information of the forklift as a moving body obtained from the sensor 40, and learns the learned model. It is stored in the completed data set 36 (step S103).

The extraction unit 33 extracts a captured image as a positive example used for deep learning from the captured image database (DB) 32 and stores it in the detection target data set 34 (step S104).

The detection unit 37 detects the target body using the image information acquired by the imaging unit 10 as input data and the trained model stored in the trained data set 36 by the switching determination unit 35 (step S105).

The detection unit 37 determines the detection target data set based on, for example, the degree of deviation of the score with respect to the similarity between the detection target data set 34 stored by the extraction unit 33 and the learned data set stored by the switching determination unit 35. The data in which it is detected that the data in 34 is a desired detector is stored in the detection result database (DB) 38 (step S106).

Here, the control unit 30 shifts to processing for the next image information (step S107), and confirms whether or not there is image information to be processed (step S108). Then, when the control unit 30 determines that there is no image information to be processed (“Yes” in step S108), the series of processes is terminated. On the other hand, when the control unit 30 determines that there is image information to be processed (“No” in step S108), the processing is returned to step S105.

As described above, according to the present embodiment, the detection device 1 is installed on a forklift as a moving body, and has an imaging unit 10 for acquiring image information around the forklift as a moving body, and a plurality of trained models. A storage unit 20 that stores information about the subject, a detection unit 37 that detects an object using a learned model stored in the storage unit 20 using image information acquired by the imaging unit 10 as input data, and a moving body. A sensor 40 that acquires the surrounding environment information of the forklift as a device, and a switching determination unit 35 that switches the learned model used in the detection unit 37 based on the information including the surrounding environment information acquired by the sensor 40. There is.
That is, the image information around the forklift as a moving body acquired by the imaging unit 10 is input, and the switching determination unit 35 is stored in the storage unit 20 based on the information including the surrounding environment information acquired by the sensor 40. The detection unit 37 detects the target body using the trained model determined to be suitable among the plurality of trained models.
Therefore, compared to the method of performing detection using a general general-purpose trained model, detection is performed using a trained model suitable for the current situation. Therefore, in particular, the target body corresponds to any of the image information. It is possible to reduce the amount of processing for determining whether or not, and to improve the detection accuracy of individual objects.

Further, according to the present embodiment, the plurality of trained models stored in the storage unit 20 are specialized trained models according to image information or an object. Here, examples of the specialized trained model include a daytime specialized trained model, a nighttime specialized trained model, an indoor specialized trained model, and a person-specific trained model. ..
That is, the detection unit 37 inputs the image information around the forklift as a moving body acquired by the imaging unit 10, and the switching determination unit 35 stores the storage unit based on the information including the surrounding environment information acquired by the sensor 40. The detection unit 37 detects the target object by using the image information stored in 20 or the trained model determined to be suitable among the specialized trained models corresponding to the target body.
Therefore, compared to the method of performing detection using a general-purpose trained model, detection is performed using a specialized trained model suitable for the current situation. Therefore, in particular, which of the image information is the object. It is possible to further reduce the amount of processing for determining whether or not the object corresponds to the above, and further improve the detection accuracy of individual objects.

Further, according to the present embodiment, the sensor 40 acquires the surrounding environment information including at least one of the traveling position information of the forklift as a moving body, the time information, and the illuminance information around the forklift as a moving body.
That is, according to the surrounding environment information acquired by the sensor 40, the traveling position information of the forklift as a moving body is, for example, whether it is a dangerous area or a caution area, whether it is the daytime or the nighttime, and the forklift as a moving body. It is possible to obtain information linked to the specialized trained model stored in the storage unit 20, such as whether it is indoors or outdoors.
Therefore, the switching determination unit 35 selects the specialized trained model based on the surrounding environment information acquired by the sensor 40, so that a suitable specialized trained model suitable for the current situation is used. Since the detection unit 37 can detect the target body, it is for determining which of the image information the target body corresponds to, as compared with the method of performing the detection using a general general-purpose learning model. It is possible to further reduce the amount of processing and further improve the detection accuracy of individual objects.

Further, according to the present embodiment, the forklift as a moving body travels indoors or outdoors, and the storage unit 20 stores at least the daytime specialized trained model and the nighttime specialized trained model, and the sensor 40. Detects at least time information or illuminance information around the forklift as a moving body as the surrounding environment information of the forklift as a moving body, and the detection unit detects an object that the forklift as a moving body may collide with. However, it is provided with a notification unit that notifies the operator of the detection result of the target object in the detection unit.
Therefore, for example, even for an object that is difficult to be recognized by human vision at night, the detection device 1 accurately recognizes the object and notifies the detection result to prevent the occurrence of a collision or the like. Can function as an auxiliary device for.

Further, according to the present embodiment, the forklift as a moving body travels indoors, the storage unit 20 stores at least an indoor specialized learned model, and the sensor 40 is the surrounding environment of the forklift as a moving body. As information, at least the traveling position information is detected, the detection unit 37 detects the target body that the forklift as a moving body may collide with, and the detection result of the target body in the detection unit 37 is the forklift as a moving body. The notification unit 50 for notifying the operator of the above is provided.
Therefore, for example, even indoors where there are many objects and structures that can be blind spots, the detection device 1 accurately recognizes the target object and notifies the detection result in order to prevent the occurrence of a collision or the like. Can function as an auxiliary device for.

In the present embodiment, it has been described that the target body is detected by using the specialized learned model corresponding to the surrounding environment information of the forklift as the moving body acquired by the sensor 40. It is also possible to detect an object using a specialized trained model that meets the needs of.
As an example, a vehicle-specific trained model may be used to acquire detailed information on a vehicle and apply it to determine which truck to load a load when a load-loading mode is detected. ..
In addition, detailed information on a person may be acquired by a person-specific trained model and applied to detect a suspicious person.
Similarly, it may be applied to the purpose of acquiring detailed information of a person by a person-specific trained model and determining the operator, for example, determining the presence or absence of an operator. By applying it to such an application, it is possible to display the discrimination result regarding the operator, and when the forklift as a moving body is a manned and unmanned vehicle, the discrimination result of the operator can be used for vehicle allocation control.

<Second embodiment>
Hereinafter, the detection device 1A according to the second embodiment of the present invention will be described with reference to FIGS. 9 to 11.
The detection device 1 according to the present embodiment differs from the first embodiment only in the configuration of the control unit 30A in its configuration.

<Structure of control unit 30A>
As shown in FIG. 9, the control unit 30A in the detection device 1A according to the present embodiment includes the collection unit 31, the captured image database (DB) 32, the extraction unit 33, the detection target data set 34, and the switching determination unit. It includes 35A, a learned data set 36, a detection unit 37, a detection result database (DB) 38, a switching information generation unit 39A, and a timer unit 39B.
Since the components having the same reference numerals as those in the first embodiment have the same functions, detailed description thereof will be omitted.

The switching determination unit 35A selects a suitable learned model stored in the storage unit 20 based on the information including the surrounding environment information of the forklift as a moving body obtained from the sensor 40, and learns the learned model. Store in the completed data set 36. Further, the switching determination unit 35A switches the learned model based on the switching information from the switching information generation unit 39A described later. Further, the switching determination unit 35A switches the learned model based on the switching information based on the detection result of the detection unit 37A.

The switching information generation unit 39A stores the trained model used in the detection unit 37 at an arbitrary time interval, for example, an image frame interval, among a plurality of trained models stored in the storage unit 20, and another trained model. The switching information for switching to is output to the switching determination unit 35A. Further, the switching information generation unit 39A outputs the switching information to the learned model having the highest score for the similarity of deep learning in the detection unit 37 to the switching determination unit 35A.

The timer unit 39B clocks an arbitrary time interval, for example, an image frame interval.

<Processing of detection device>
The processing of the detection device 1A according to the present embodiment will be described with reference to FIGS. 10 and 11.

The imaging unit 10 acquires image information around the forklift as a moving body (step S201). The collecting unit 31 collects the captured image from the imaging unit 10 and stores the collected image information in the captured image database (DB) 32 (step S202).

The switching determination unit 35A selects a suitable learned model stored in the storage unit 20 based on the information including the surrounding environment information of the forklift as a moving body obtained from the sensor 40, and learns the learned model. It is stored in the completed data set 36 (step S203).

The extraction unit 33 extracts a captured image as a positive example used for deep learning from the captured image DB 32 and stores it in the detection target data set 34 (step S204).

The detection unit 37 detects the target body using the image information acquired by the imaging unit 10 as input data and the trained model stored in the trained data set 36 by the switching determination unit 35 (step S205).

When the timer unit 39B clocks an arbitrary time interval, for example, an image frame interval, the switching information generation unit 39A and the detection unit 37 execute inter-frame processing (step S206). Specifically, as shown in FIG. 11, the switching information generation unit 39A switches the trained model used by the detection unit 37 to another trained model among the plurality of trained models stored in the storage unit 20. The switching information is output to the switching determination unit 35A, and the trained model is stored in the trained data set 36 (step S2061). Then, the extraction unit 33 extracts the captured image which is a positive example used for deep learning from the captured image DB 32 and stores it in the detection target data set 34 (step S2062), and the detection unit 37 acquires it in the imaging unit 10. Using the generated image information as input data, the switching determination unit 35A detects the target object using the trained model stored in the trained data set 36 (step S2063).

The detection unit 37 determines the detection target data set based on, for example, the degree of deviation of the score with respect to the similarity between the detection target data set 34 stored by the extraction unit 33 and the learned data set stored by the switching determination unit 35. The data in which it is detected that the data in 34 is a desired detector is stored in the detection result database (DB) 38 (step S207).

The switching information generation unit 39A outputs the switching information to the trained model having the highest score for the similarity of deep learning in the detection unit 37 to the switching determination unit 35A (step S208).

Here, the control unit 30A shifts to processing for the next image information (step S209), and confirms whether or not there is image information to be processed (step S210). Then, when the control unit 30A determines that there is no image information to be processed (“Yes” in step S210), the series of processes is terminated. On the other hand, when the control unit 30A determines that there is image information to be processed (“No” in step S210), the processing is returned to step S205.

As described above, according to the present embodiment, the detection device 1A includes a switching information generation unit 39A that outputs switching information of the learned model to the switching determination unit 35A, and the switching information generation unit 39A is arbitrary. At time intervals, switching information for switching the learned model used in the detection unit 37 to another learned model among the plurality of learned models stored in the storage unit 20 is output to the switching determination unit 35A, and the detection unit The switching information to the trained model having the highest score for the similarity of the deep learning in 37 is output to the switching determination unit 35A.
That is, the switching information generation unit 39A stores the trained model used in the detection unit 37 at an arbitrary time interval, for example, at a frame interval of image information, among a plurality of trained models stored in the storage unit 20. The switching information for switching to the trained model of is output to the switching determination unit 35A. Then, the switching information to the learned model having the highest score for the similarity of deep learning in the detection unit 37 is output to the switching determination unit 35A.
Therefore, the detection unit 37 detects the target body by the trained model with high detection accuracy, and as a result, the detection accuracy of the target body can be further improved. In addition, compared to the method of performing detection using a general general-purpose learning model, detection is performed using a learned model that is most suitable for the current situation, so that the object corresponds to any of the image information. It is possible to reduce the amount of processing for determining whether or not.

<Third embodiment>
Hereinafter, the detection device 1B according to the third embodiment of the present invention will be described with reference to FIGS. 12 to 14.
The detection device 1B according to the present embodiment differs from the first embodiment and the second embodiment only in the configuration of the control unit 30B in its configuration.

<Structure of control unit 30B>
As shown in FIG. 12, the control unit 30B in the detection device 1 according to the present embodiment includes the collection unit 31, the captured image database (DB) 32, the extraction unit 33, the detection target data set 34, and the switching determination unit. It includes 35A, a learned data set 36, a detection unit 37, a detection result database (DB) 38, a switching information generation unit 39C, a timer unit 39B, and an image similarity determination unit 39D. ..
Since the components having the same reference numerals as those in the first embodiment have the same functions, detailed description thereof will be omitted.

The switching information generation unit 39C stores the learned model used in the detection unit 37 at an arbitrary time interval when the image information acquired by the imaging unit 10 has a similarity within a predetermined range in time series. Among the plurality of trained models stored in, the switching information for switching to another trained model is output to the switching determination unit 35A, and the trained model having the highest score for the similarity of deep learning in the detection unit 37 is obtained. The switching information is output to the switching determination unit 35A.

The image similarity determination unit 39D determines the similarity between the image information with respect to the time-series image information in the detection target data set 34 extracted by the extraction unit 33, and the similarity is within a predetermined range. In this case, a signal is output to the switching information generation unit 39C.

<Processing of detection device>
The processing of the detection device 1 according to the present embodiment will be described with reference to FIGS. 13 and 14.

The imaging unit 10 acquires image information around the forklift as a moving body (step S301). The collecting unit 31 collects the captured image from the imaging unit 10 and stores the collected image information in the captured image database (DB) 32 (step S302).

The switching determination unit 35A selects a suitable learned model stored in the storage unit 20 based on the information including the surrounding environment information of the forklift as a moving body obtained from the sensor 40, and learns the learned model. It is stored in the completed data set 36 (step S303).

The extraction unit 33 extracts a captured image as a positive example used for deep learning from the captured image database (DB) 32 and stores it in the detection target data set 34 (step S304).

The detection unit 37 detects the target body using the image information acquired by the imaging unit 10 as input data and the trained model stored in the trained data set 36 by the switching determination unit 35 (step S305).

When the timer unit 39B clocks an arbitrary time interval, for example, an image frame interval, the switching information generation unit 39C and the detection unit 37 execute inter-frame processing (step S306). Specifically, as shown in FIG. 14, the image similarity determination unit 39D determines the degree of similarity between the image information with respect to the time-series image information in the detection target data set 34 extracted by the extraction unit 33. It is determined, and it is determined whether or not the similarity is within a predetermined range (step S2064). Then, when it is determined that the similarity is not within the predetermined range (“NO” in step S2064), the process is stopped. On the other hand, when it is determined that the similarity is within the predetermined range (“YES” in step S2064), a signal is output to the switching information generation unit 39C. The switching information generation unit 39C outputs switching information for switching the learned model used by the detection unit 37 to another learned model among the plurality of learned models stored in the storage unit 20 to the switching determination unit 35A. The trained model is stored in the trained dataset 36 (step S2061). Then, the extraction unit 33 extracts the captured image which is a positive example used for deep learning from the captured image DB 32 and stores it in the detection target data set 34 (step S2062), and the detection unit 37 in the imaging unit 10 Using the acquired image information as input data, the switching determination unit 35A detects the target object using the trained model stored in the trained data set 36 (step S2063).

The detection unit 37 determines the detection target data set based on, for example, the degree of deviation of the score with respect to the similarity between the detection target data set 34 stored by the extraction unit 33 and the learned data set stored by the switching determination unit 35. The data in which it is detected that the data in 34 is a desired detector is stored in the detection result DB 38 (step S207).

The switching information generation unit 39C outputs the switching information to the trained model having the highest score for the similarity of deep learning in the detection unit 37 to the switching determination unit 35A (step S208).

Here, the control unit 30A shifts to processing for the next image information (step S209), and confirms whether or not there is image information to be processed (step S210). Then, when the control unit 30A determines that there is no image information to be processed (“Yes” in step S210), the series of processes is terminated. On the other hand, when the control unit 30A determines that there is image information to be processed (“No” in step S210), the processing is returned to step S205.

As described above, according to the present embodiment, in the switching information generation unit 39C of the detection device 1B, the image information acquired by, for example, the image pickup unit 10 is in a predetermined range in time series at an arbitrary time interval. When there is a degree of similarity, the switching information for switching the learned model used in the detection unit 37 to another trained model among the plurality of trained models stored in the storage unit 20 is switched at the frame interval of the image information. Output to the determination unit 35A.
Therefore, by applying the learned model with high detection accuracy to the image information having the similarity within a predetermined range in time series, the detection unit 37 detects the target body, and as a result, the target body is detected. The accuracy can be further improved. In addition, compared to the method of performing detection using a general general-purpose learning model, detection is performed using a learned model that is most suitable for the current situation, so that the object corresponds to any of the image information. It is possible to reduce the amount of processing for determining whether or not.

In addition, another imaging unit (for example, a drive recorder) that captures an image in the same region is provided, and the image captured by the imaging unit is used as teacher data to perform matching evaluation with the detection result in the detection unit 37, and this evaluation result is used. Based on this, it may have a function to update a part of the trained model.

Further, in the above embodiment, the forklift has been illustrated as a moving body, but it can also be applied to a construction machine such as an excavator car and an industrial machine such as an agricultural machine such as a tractor.

Further, the detection device 1 of the present invention is realized by recording the processing of the detection device 1 on a recording medium that can be read by a computer system, causing the detection device 1 to read the program recorded on the recording medium, and executing the process. Can be done. The computer system referred to here includes hardware such as an OS and peripheral devices.

Further, the "computer system" includes a homepage providing environment (or a display environment) if a WWW (World Wide Web) system is used. Further, the program may be transmitted from a computer system in which this program is stored in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

Further, the above program may be for realizing a part of the above-mentioned functions. Further, it may be a so-called difference file (difference program) that can realize the above-mentioned function in combination with a program already recorded in the computer system.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs and the like within a range that does not deviate from the gist of the present invention.

1; Detection device 10; Image pickup unit 20; Storage unit 30; Control unit 31; Collection unit 32; Image capture image database (DB)
33; Extractor 33
34; Detection target data set 35, 35A; Switching judgment unit 36; Learned data set 37; Detection unit 38; Detection result database (DB)
39A, 39C; Switching information generation unit 39B; Timer unit 39D; Image similarity determination unit 40; Sensor 50; Notification unit

Claims (7)

An imaging unit that is installed on a moving body and acquires image information around the moving body,
A storage unit that stores information about multiple trained models,
Using the image information acquired by the imaging unit as input data and using the learned model stored in the storage unit, a detection unit that detects an object and a detection unit.
A sensor that acquires information on the surrounding environment of the moving object,
A switching determination unit that switches the learned model used in the detection unit based on the information including the surrounding environment information acquired by the sensor.
A detection device characterized by being equipped with. The detection device according to claim 1, wherein the plurality of trained models stored in the storage unit are specialized trained models corresponding to the image information or the object. Any one of claims 1 or 2, wherein the sensor acquires the surrounding environment information including at least one of the traveling position information, the time information, and the illuminance information around the moving body. The detection device described in. The switching determination unit is provided with a switching information generation unit that outputs switching information of the learned model.
The switching information generation unit switches information for switching the learned model used in the detection unit to another learned model among the plurality of learned models stored in the storage unit at an arbitrary time interval. Any of claims 1 to 3, wherein the switching determination unit is output, and the switching information to the learned model having the highest score for the similarity of deep learning in the detection unit is output to the switching determination unit. The detection device described in. The switching information generation unit stores the learned model used in the detection unit at an arbitrary time interval when the image information acquired in the imaging unit has a similarity within a predetermined range in time series. Among the plurality of trained models stored in the unit, switching information for switching to another trained model is output to the switching determination unit, and the trained model has the highest score for the similarity of deep learning in the detection unit. The detection device according to claim 4, wherein switching information to the model is output to the switching determination unit. The moving body is a forklift that travels indoors or outdoors.
The storage unit stores at least a daytime specialized trained model and a nighttime specialized trained model.
The sensor detects at least time information or illuminance information around the forklift as the surrounding environment information of the forklift.
The detection unit detects the object that the forklift may collide with, and
The detection device according to claim 1, further comprising a notification unit that notifies the forklift operator of the detection result of the target body in the detection unit. The moving body is a forklift that travels indoors.
The storage unit stores at least an indoor specialized trained model or a person-specific trained model.
The sensor detects at least traveling position information as the surrounding environment information of the forklift.
The detection unit detects the object that the forklift may collide with, and
The detection device according to claim 1, further comprising a notification unit that notifies the forklift operator of the detection result of the target body in the detection unit.

Images