CN109063761B - Diffuser shedding detection method, device and electronic equipment - Google Patents

Abstract

The invention provides a method and a device for detecting the falling of a diffuser and electronic equipment, and relates to the technical field of 3D (three-dimensional) shooting, wherein the method comprises the following steps: acquiring an image to be detected through a camera device; inputting an image to be detected into a pre-trained feature extraction network so that the feature extraction network extracts features of the image to be detected; wherein the feature extraction network is generated by training a new sample set; the new sample set is formed by a new sample group obtained by combining given samples; inputting the features into a pre-trained classifier to obtain a classification result; and determining whether the diffuser dropping occurs according to the classification result. The embodiment of the invention can carry out more efficient diffuser falling detection on the camera device, and has higher detection accuracy.

Description

Diffuser shedding detection method, device and electronic equipment

technical field

The present invention relates to the technical field of 3D imaging, and in particular, to a method, device and electronic device for detecting the falling off of a diffuser.

Background technique

With the wide application of 3D modules, more and more mobile terminals will be equipped with 3D camera modules, such as the common Time-of-flight module technology, which emits modulated near-infrared light through sensors, After encountering the reflection of the object, the sensor converts the distance of the captured scene by calculating the time difference or phase difference between light emission and reflection to generate depth information. In addition, combined with traditional camera shooting, the three-dimensional outline of the object can be represented by different colors. A topographic map of distances is presented.

The above-mentioned camera module is usually equipped with a diffuser (Diffuser, also called a diffuser or a diffuser) in front of the infrared light source, so that the infrared light can be uniformly irradiated to the entire shooting scene. Without a diffuser, the infrared light emitted by the infrared light source would gather into a bundle, which would prevent the 3D camera from properly perceiving the depth of the object being photographed in the scene. In the actual use process, if the 3D camera module is bumped or vibrated, there will be a certain probability that the diffuser will fall off, making the 3D camera module invalid.

In view of the problem of poor accuracy in detecting the falling off of the diffuser in the prior art, no effective solution has been proposed yet.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present invention is to provide a method, device and electronic device for detecting the falling off of a diffuser, which can improve the accuracy of detecting the falling off of the diffuser.

In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present invention are as follows:

In a first aspect, an embodiment of the present invention provides a method for detecting the falling off of a diffuser, where the diffuser is arranged on a camera, and the method includes: acquiring an image to be detected through the camera; inputting the image to be detected into pre-training feature extraction network, so that the feature extraction network extracts the features of the image to be detected; wherein, the feature extraction network is generated by training a new sample set; the new sample set is obtained by combining the given samples It is composed of a new sample group; the feature is input into a pre-trained classifier, and a classification result is obtained; according to the classification result, it is determined whether the diffuser falls off.

Further, the method further includes: inputting one of the new sample groups in the new sample set into the feature extraction network; the new sample group includes at least two of the given samples; extracting features of each of the given samples; if two of the given samples belong to the same class of samples, minimize the features of the two given samples to train the feature extraction network; The classes include non-dropout classes and dropout classes; if two of the given samples do not belong to the same class of samples, the features of the two given samples are maximized to train the feature extraction network ; Input new sample groups in the new sample set in turn, and stop when the feature extraction network converges.

Further, the step of inputting one of the new sample groups in the new sample set into the feature extraction network includes: forming a dropout sample, a normal sample and an attack sample into the new sample group and inputting it into the feature extraction network ; The step of minimizing the features of the two given samples includes: minimizing the features of the attack samples and the normal samples; The step of maximizing the feature of a given sample includes: maximizing the feature of the dropout sample and the normal sample; maximizing the feature of the dropout sample and the attack sample. maximize processing.

Further, the step of inputting one of the new sample groups in the new sample set into the feature extraction network includes: randomly selecting two of the given samples to form the new sample group, and inputting the new sample group into the feature extraction network. the feature extraction network.

Further, the step of minimizing the features of the two training samples includes: calculating a distance between vectors of the two features, and minimizing the distance to perform the minimization process.

Further, the step of maximizing the features of the two training samples includes: calculating a distance between vectors of the two features, and maximizing the distance to perform maximization processing.

Further, the method further includes: performing feature extraction on the drop-off samples, normal samples and attack samples through the pre-trained feature extraction network; dividing the features of the drop-off samples into drop-off categories, and classifying the normal samples and attack samples The features of the samples are divided into non-dropout categories, and the classifier is trained.

Further, the method further includes: when it is determined that the diffuser falls off, performing an alarm reminder.

In a second aspect, an embodiment of the present invention further provides a diffuser falling off detection device, the diffuser is arranged on a camera device, and the device includes: an acquisition module for acquiring an image to be detected through the camera device; a feature extraction module , which is used to input the image to be detected into a pre-trained feature extraction network, so that the feature extraction network extracts the features of the image to be detected; wherein, the feature extraction network is generated by training a new sample set; The new sample set is composed of a new sample group obtained by combining the given samples; the classification module is used to input the features into a pre-trained classifier to obtain a classification result; the judgment module is used to determine according to the classification results. Whether the diffuser has fallen off.

In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, the memory stores a computer program that can run on the processor, and the processor implements the computer program when the processor executes the computer program. The steps of any one of the methods of the first aspect.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, any one of the foregoing first aspects is executed. steps of the method.

Embodiments of the present invention provide a method, device, and electronic device for detecting diffuser shedding, which can perform feature extraction on an image to be detected through a pre-trained feature extraction network, and the image to be collected is acquired by a camera device to be detected. The network is trained by a new sample set obtained by combining the given samples. After the features are extracted, the classification is performed to determine whether the diffuser falls off. The camera device can be more efficiently detected by the diffuser falling off, and The detection accuracy is higher.

Additional features and advantages of the present disclosure will be set forth in the description that follows, or some may be inferred or unambiguously determined from the description, or may be learned by practicing the above-described techniques of the present disclosure.

In order to make the above-mentioned objects, features and advantages of the present disclosure more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to illustrate the specific embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the specific embodiments or the prior art. Obviously, the accompanying drawings in the following description The drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

FIG. 1 is a schematic structural diagram of a processing device according to an embodiment of the present invention;

2 is a flowchart of a method for detecting a diffuser falling off according to an embodiment of the present invention;

3 is a flowchart of a feature extraction network training method provided by an embodiment of the present invention;

4 is a schematic diagram of a given sample provided by an embodiment of the present invention;

5 is a schematic diagram of a training feature extraction network provided by an embodiment of the present invention;

6 is a schematic diagram of another training feature extraction network provided by an embodiment of the present invention;

FIG. 7 is a structural block diagram of a diffuser falling-off detection device according to an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of them. example. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

With the application of 3D sensing technology in smart terminals, the detection of diffuser shedding of 3D camera modules has become a new problem. Smart terminal manufacturers or camera module manufacturers require a detection accuracy of more than 99.9% for diffusers, especially to minimize misidentification of good products. When the smart terminal user actually uses the smart terminal, there is also a need to detect whether the diffuser falls off. The existing methods for detecting whether the diffuser falls off have poor precision and cannot meet the needs of manufacturers or users. In order to improve this problem, the embodiments of the present invention provide a method, device and electronic device for detecting the falling off of a diffuser, and the embodiments of the present invention will be described in detail below.

Example 1:

First, an example electronic device 100 for implementing the diffuser shedding detection method, apparatus, and electronic device according to an embodiment of the present invention will be described with reference to FIG. 1 .

As shown in FIG. 1 is a schematic structural diagram of an electronic device, the electronic device 100 includes one or more processors 102, one or more storage devices 104, an input device 106, an output device 108, and an image acquisition device 110. These components are The bus system 112 and/or other form of connection mechanism (not shown) are interconnected. It should be noted that the components and structures of the electronic device 100 shown in FIG. 1 are only exemplary and not restrictive, and the electronic device may also have other components and structures as required.

The processor 102 may be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA), and the processor 102 may be a central processing unit. Unit (CPU) or one or a combination of other forms of processing units with data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.

The storage device 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and/or cache memory, or the like. The non-volatile memory may include, for example, read only memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 102 may execute the program instructions to implement the client functions (implemented by the processor) in the embodiments of the present invention described below. and/or other desired functionality. Various application programs and various data, such as various data used and/or generated by the application program, etc. may also be stored in the computer-readable storage medium.

The input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, mouse, microphone, touch screen, and the like.

The output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.

The image capture device 110 may capture images (eg, photos, videos, etc.) desired by the user, and store the captured images in the storage device 104 for use by other components. The image acquisition device 110 includes a 3D camera module, and the 3D camera module includes a diffuser.

Exemplarily, exemplary electronic devices for implementing the method, apparatus, and system for detecting diffuser shedding according to embodiments of the present invention may be implemented as smart terminals such as smart phones, tablet computers, and computers.

Embodiment 2:

Referring to the flow chart of a method for detecting the falling off of a diffuser shown in FIG. 2 , the diffuser is arranged in a camera device, and the method can be performed by the electronic equipment provided in the foregoing embodiment, and the method specifically includes the following steps:

Step S202, the image to be detected is acquired by the camera device.

The camera device can be installed on the smart terminal, or can be used independently. When the camera device is subjected to the detection of the diffuser falling off, the camera device captures an image as the image to be detected. For example, the to-be-detected image is received and stored by the above-mentioned intelligent terminal, or the to-be-detected image is received and stored by other external devices connected to the camera device.

Step S204, input the image to be detected into a pre-trained feature extraction network, so that the feature extraction network extracts the features of the image to be detected.

Among them, the feature extraction network is generated by training a new sample set obtained by combining the given samples. The method of deep learning by the feature extraction network has higher accuracy of feature extraction and does not require manual adjustment of parameters, but the operation speed is relatively slow, and the model needs to be trained in advance.

Since it is uncommon for the diffuser to fall off, there are few samples that fall off and many cases to deal with. The above given samples include shedding samples and non-shedding samples. In this embodiment, shedding samples and non-shedding samples can be combined, for example, a shedding sample is combined with a non-shedding sample, a shedding sample is combined with two non-shedding samples, and two A new sample group is obtained by a combination of dropped samples or a combination of two non-dropped samples. The number of the new sample group is greater than the sum of the number of dropped samples and normal samples, and the new sample set formed by the above new sample group is used as the training of the feature extraction network. sample set. The above method increases the number of training samples of the feature extraction network, so that a more accurate extraction model can be obtained only from the original small sample data.

Step S206, input the feature into the pre-trained classifier to obtain a classification result.

When training the classifier, the input samples are the features obtained by the above-mentioned feature extraction network extracting the training samples, and each training sample can be represented as a feature vector. The feature vectors corresponding to the shedding samples are labeled as one class, and the feature vectors corresponding to the non-dropping samples are labeled as another class, and the classifier is trained by both. Among them, the detached sample is the image collected by the camera when the diffuser is detached; the non-detached sample includes a normal sample and an attack sample, and the normal sample is the image collected by the camera when the diffuser is working normally; the attack sample is an approximate detachment signal, which is in the The diffuser is installed and working normally, but the image is captured when the user is improperly operated or other factors cause the shooting to be abnormal.

After the above classifier is trained, the features extracted by the feature extraction network can be classified.

In step S208, it is determined whether the diffuser falls off according to the classification result.

After the classifier outputs the classification result of the features of the image to be detected, it can be determined whether the diffuser falls off in the camera device according to the classification result. When it is determined that the diffuser falls off, an alarm reminder can be given, so that it is convenient for the user of the smart terminal to have a problem and need to be repaired, or for the manufacturer to remove or repair the faulty camera device.

The above-mentioned diffuser shedding detection method provided by the embodiment of the present invention can perform feature extraction on the image to be detected through a pre-trained feature extraction network, the image to be collected is acquired by the camera device to be detected, and the feature extraction network is based on a given sample. The new sample group obtained by the combination is obtained by training, and after the features are extracted, the classification is performed, and whether the diffuser falls off is determined, and the camera device can be more efficiently detected.

The feature extraction network used in the above method is trained based on the feature learning scheme of small samples. Referring to the flowchart of the feature extraction network training method shown in FIG. 3, the method specifically includes the following steps:

Step S302, a new sample group in the new sample set is input into the feature extraction network. The new sample set includes at least two of the given samples.

The feature extraction network may be an existing neural network capable of image feature extraction. The given samples are combined, and the combination method can be a random sample combination, such as any combination of two given samples or three given samples, and the combined multiple given samples are used as a new sample group and input to feature extraction. in the network. The number of training samples can be increased by incorporating new samples in the form of groups.

In step S304, the features of each given sample are respectively extracted through the feature extraction network.

Before training, the feature extraction network uses the initial parameters to perform feature extraction for a given sample.

Step S306, if the two given samples belong to the same category of samples, minimize the features of the two given samples to train a feature extraction network.

The above categories include non-drop-off classes and drop-off classes, normal samples and attack samples in a given sample are classified as non-drop-off classes, and drop-off samples are classified as drop-off classes. Referring to the schematic diagram of a given sample shown in FIG. 4 , from left to right are a normal sample, an attack sample, and a shedding sample, taking an IR (Infrared Radiation, infrared) image as an example. Among them, the normal sample is the image collected by the camera device when the diffuser is working normally; the attack sample refers to the approximate shedding signal, which is the image collected when the diffuser is installed and worked normally, but the user's operation is improper or other factors cause abnormal shooting. There will be a relatively obvious light spot in the image, for example, the user's finger is very close to the lens of the camera device; the detached sample is the image collected by the camera device when the diffuser falls off. Since there is no diffuser in front of the light source of the camera device, the light source emits light. The light is a narrow beam, and only a small part of the collected image has content, and other parts have no content.

Step S308, if the two given samples do not belong to the same category of samples, the features of the two given samples are maximized to train a feature extraction network.

For the given samples belonging to the same category in the above new sample group, the features of the two given samples are minimized; for the given samples belonging to different categories in the above new sample group, the features of the two given samples are processed. Maximize processing; thereby optimizing the parameters of the feature extraction network described above. The maximization and minimization processing can be performed by the distance of the feature vector corresponding to the feature, for example: calculating the distance between two feature vectors, maximization is to maximize the distance, and minimization is to minimize the distance. For example, the feature obtained from sample 1 is F ₁ (multi-dimensional vector), and the feature obtained from sample 2 is F ₂ (feature vector of the same dimension), maximizing the distance between the vectors to make the maximum distance between the vectors Max|F ₁ -F ₂ |^2, the minimum It becomes Min|F ₁ -F ₂ |^2. After maximizing/minimizing the distance of the above feature vectors, the feature extraction network can be returned, and the parameters of the feature extraction network can be finally optimized, so that the results of classification of the extracted features are more accurate. The method for optimizing the above network may use an existing network optimization method, such as a gradient descent optimization algorithm.

Step S310 , sequentially inputting new sample groups in the new sample set, and stopping when the feature extraction network converges.

In the training process, each time a new sample group is input for training, the optimized feature extraction network is obtained, and then the second batch of samples is selected, and the optimization is performed on the basis of the above-mentioned optimized feature extraction network until the network converges.

Considering that a given sample may include three types: normal samples, attack samples and dropout samples, a dropout sample, a normal sample and an attack sample can be used as a new sample group input feature extraction network for training, or two can be randomly selected. The samples are used as a new sample group to input the feature extraction network for training.

Referring to the schematic diagram of the training feature extraction network shown in FIG. 5 , the input feature extraction network of a new sample group composed of a shedding sample, a normal sample and an attack sample is taken as an example for description. The feature extraction network performs feature extraction on the above-mentioned samples, and obtains shedding features, attack features and normal features respectively. As shown in Figure 5, each two features are maximized or minimized, specifically: the attack feature and the normal feature are minimized; the shedding feature and the normal feature are maximized; the shedding feature and the attack feature are maximized to maximize processing. In the above method, three samples are input each time, and the feature extraction network is continuously trained until it converges.

Referring to the schematic diagram of the training feature extraction network shown in FIG. 6 , the description is given by taking two randomly selected samples to input the feature extraction network as an example. Two samples are input each time. In Figure 6, sample 1 and sample 2 are used as examples. The two samples are randomly selected from all samples. The feature extraction network performs feature extraction on the above samples to obtain sample 1 features and sample 2 features respectively. As shown in Figure 6, the two features are maximized or minimized, specifically: the same samples are minimized, for example, sample 1 and sample 2 belong to the above non-dropout category or dropout category; different types of samples are maximized For example, sample 1 and sample 2 belong to the above-mentioned non-dropout category and dropout category, respectively.

Before using the above classifier for classification, it needs to be trained. The classifier may be an existing SVM (Support Vector Machine, Support Vector Machine), or other applicable classifiers, which are not limited in this embodiment. . The training process of the classifier can be carried out as follows:

(1) Feature extraction is performed on the shedding samples, normal samples and attack samples through a pre-trained feature extraction network;

(2) The features of the shedding samples are divided into shedding categories, the features of normal samples and attack samples are divided into non-shedding categories, and the classifier is trained. Among them, each sample is represented as a feature vector, and the feature vectors of the dropped samples are classified into one class, and the feature vectors of normal samples and attack samples are classified into one class for training.

The above method increases the training sample set by combining the shedding samples and non-shedding samples (including normal samples and attack samples) in pairs, and learns the characteristics of the shedding samples and the non-shedding samples, and thus performs shedding detection, which can be The network is trained based on the dropout samples of small samples, and efficient dropout detection can be performed based on the network.

Embodiment three:

For the method for detecting the detachment of a diffuser provided in the second embodiment, the embodiment of the present invention provides a device for detecting the detachment of a diffuser. Referring to the structural block diagram of a device for detecting the detachment of a diffuser shown in FIG. 7 , the device includes:

an acquisition module 701, configured to acquire an image to be detected through a camera;

The feature extraction module 702 is used to input the image to be detected into a pre-trained feature extraction network, so that the feature extraction network extracts the features of the image to be detected; wherein, the feature extraction network is generated by training a new sample set; the new sample set is composed of It consists of a new sample group obtained by combining the given samples;

A classification module 703, for inputting the feature extraction into a pre-trained classifier to obtain a classification result;

The judging module 704 is configured to determine whether the diffuser falls off according to the classification result.

The above-mentioned diffuser shedding detection device provided by the embodiment of the present invention can perform feature extraction on the image to be detected through a pre-trained feature extraction network. The new sample set obtained by combining is obtained by training, and after the features are extracted, the classification is performed, and whether the diffuser falls off is determined.

In one embodiment, the above-mentioned apparatus further includes a feature extraction network training module for: inputting a new sample group in the new sample set into the feature extraction network; the new sample group includes at least two given samples; Extract the features of each given sample; if two given samples belong to the same class of samples, the features of the two given samples are minimized to train the feature extraction network; the classes include normal classes and dropout classes; if two If the given samples do not belong to the same category, the features of the two given samples are maximized to train the feature extraction network; new sample groups in the new sample set are sequentially input until the feature extraction network converges.

Among them, the above feature extraction network training module is also used to: form a new sample group of a shedding sample, a normal sample and an attack sample into the feature extraction network; minimize the features of the two given samples to optimize the features The steps of extracting the parameters of the network include: minimizing the features of the attack samples and the normal samples; maximizing the features of the two given samples to optimize the parameters of the feature extraction network, including: removing the dropped samples and the features of normal samples are maximized; the features of shedding samples and attack samples are maximized.

The above feature extraction network training module is also used for: randomly selecting two given samples to form a new sample group, and inputting the new sample group into the feature extraction network. The above feature extraction network training module is also used for: calculating the distance between vectors of two features, and minimizing the distance, so as to perform minimization processing. The above feature extraction network training module is also used for: calculating the distance between vectors of two features, and maximizing the distance, so as to perform maximization processing. The above feature extraction network training module is also used to: calculate the distance between vectors of two features according to the following formula: |F1–F2|^2; where F1 and F2 represent feature vectors of features respectively.

In another embodiment, the above-mentioned apparatus further includes a classifier training module, which is used for: extracting features from the drop-off samples, normal samples and attack samples through a pre-trained feature extraction network; classifying the features of the drop-off samples into drop-off categories , the features of normal samples and attack samples are divided into non-dropout categories, and the classifier is trained.

In another embodiment, the above-mentioned device further includes a reminder module, configured to issue an alarm reminder when it is determined that the diffuser falls off.

The implementation principle and the technical effects of the device provided in this embodiment are the same as those in the foregoing embodiments. For brief description, for the parts not mentioned in the device embodiment, reference may be made to the corresponding content in the foregoing method embodiments.

In addition, this embodiment also provides an electronic device, including a memory and a processor, the memory stores a computer program that can run on the processor, and when the processor executes the computer program, the diffuser shedding detection provided in the second embodiment above is implemented. steps of the method.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the device described above, reference may be made to the corresponding process in the foregoing embodiments, and details are not repeated here.

This embodiment also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the method provided in the second embodiment above are executed.

A computer program product of a method, an apparatus, and a processing device for a diffuser shedding detection provided by the embodiments of the present invention includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the foregoing method embodiments. For the specific implementation of the method described in , please refer to the method embodiment, which will not be repeated here. The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, removable hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes.

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present invention, and are used to illustrate the technical solutions of the present invention, but not to limit them. The protection scope of the present invention is not limited thereto, although referring to the foregoing The embodiment has been described in detail the present invention, those of ordinary skill in the art should understand: any person skilled in the art who is familiar with the technical field within the technical scope disclosed by the present invention can still modify the technical solutions described in the foregoing embodiments. Or can easily think of changes, or equivalently replace some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be covered in the present invention. within the scope of protection. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (10)

1. A method for detecting the dropping of a diffuser, wherein the diffuser is provided in an image pickup device, the method comprising:

acquiring an image to be detected through the camera device;

inputting the image to be detected into a pre-trained feature extraction network so that the feature extraction network extracts the features of the image to be detected; wherein the feature extraction network is generated by a new sample set training; the new sample set is formed by a new sample group obtained by combining given samples;

inputting the features into a pre-trained classifier to obtain a classification result;

and determining whether the diffuser falling-off occurs according to the classification result.

2. The method of claim 1, further comprising:

inputting one of the new sample sets into a feature extraction network; the new set of samples comprises at least two of the given samples;

respectively extracting the features of each given sample through the feature extraction network;

if the two given samples belong to the same class of samples, performing minimization processing on the features of the two given samples to train the feature extraction network; the categories include a non-shedding category and a shedding category;

if the two given samples do not belong to the same class of samples, performing maximization processing on the features of the two given samples to train the feature extraction network;

and sequentially inputting new sample groups in the new sample set until the feature extraction network converges.

3. The method of claim 2, wherein said step of inputting one of said new sample sets into a feature extraction network comprises:

inputting a new sample group consisting of a shedding sample, a normal sample and an attack sample into the feature extraction network;

said step of minimizing said features of both said given samples comprises: minimizing the features of the attack and normal samples;

said step of maximizing said features of both said given samples comprises: maximizing the features of the shed and normal samples; maximizing the features of the shedding sample and the attacking sample.

4. The method of claim 2, wherein said step of inputting one of said new sample sets into a feature extraction network comprises:

randomly selecting two of the given samples to form the new sample set, and inputting the new sample set into the feature extraction network.

5. The method of claim 2, wherein said step of minimizing said features of said two given samples comprises:

and calculating the distance between the vectors of the two features, and minimizing the distance to perform minimization processing.

6. The method of claim 2, wherein said step of maximizing said features of said two given samples comprises:

and calculating the distance between the vectors of the two features, and maximizing the distance to perform maximization processing.

7. The method of claim 1, further comprising:

extracting the characteristics of the shedding sample, the normal sample and the attack sample through the pre-trained characteristic extraction network;

and dividing the characteristics of the shedding sample into shedding categories, dividing the characteristics of the normal sample and the attacking sample into non-shedding categories, and training the classifier.

8. A device for detecting the dropping of a diffuser, wherein the diffuser is provided in an image pickup device, the device comprising:

the acquisition module is used for acquiring an image to be detected through the camera device;

the characteristic extraction module is used for inputting the image to be detected into a pre-trained characteristic extraction network so that the characteristic extraction network extracts the characteristics of the image to be detected; wherein the feature extraction network is generated by a new sample set training; the new sample set is formed by a new sample group obtained by combining given samples;

the classification module is used for inputting the characteristics into a pre-trained classifier to obtain a classification result;

and the judging module is used for determining whether the diffuser falls off or not according to the classification result.

9. An electronic device comprising a memory and a processor, the memory having stored therein a computer program operable on the processor, wherein the processor implements the steps of the method of any of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, having stored thereon a computer program, characterized in that the computer program, when being executed by a processor, is adapted to carry out the steps of the method of any of the claims 1 to 7.

Images