KR20230009651A - Ai apparatus and method for automatically reading test results of multiple diagnostic kits - Google Patents

Abstract

The present invention aims to enhance the accuracy of reading results through machine learning algorithms based on images. Disclosed is an electronic device configured to: acquire images taken of a plurality of diagnostic kits; generate a first image patch including a first subset of the plurality of diagnostic kits; generate a second image patch including a second subset of the plurality of diagnostic kits, wherein one or more diagnostic kits included in the first subset and the second subset are duplicated; acquire a first test result set for the first image patch and a second test result set for the second image patch; and if the test results of the one or more duplicate diagnostic kits included in the first test result set match the test results of the one or more duplicate diagnostic kits included in the second test result set, determine the first test result set and the second test result set as a final test result set for the plurality of diagnostic kits. In addition, various embodiments identified through the specification are possible.

Description

AI device and method for automatically reading test results of multiple diagnostic kits

Embodiments disclosed in this document are related to an AI model that automatically reads test results of a diagnostic kit.

Diagnostic kits are widely used as one of the simple and quick diagnostic methods. For example, diagnostic kits are used to test whether a person is infected with a specific disease or not, and whether antibodies are formed. Since the reading of the test result of the diagnostic kit is performed by a person, the person analyzing the result must have basic knowledge such as information on the diagnostic kit and how to read the diagnostic kit. In addition, when the read result needs to be digitized, since the read result is input by a person, an input error may occur in the middle of inputting the result.

Furthermore, if a test is performed on a large group, a large number of diagnostic kits must be analyzed simultaneously, and a correspondingly large-scale manpower is required to computerize the test results. Therefore, a lot of cost, such as manpower for inspection and time required for inspection, is required.

It is intended to automate the reading process performed by the human eye and the reporting process for the reading result. In addition, it is intended to increase the accuracy of the reading result through machine learning performed based on the image. Furthermore, it is intended to be able to read several diagnostic kits at once and computerize the test results of several diagnostic kits.

Various embodiments disclosed in this document intend to provide a technology for generating and providing an AI model capable of automatically reading test results from images of a plurality of diagnostic kits.

An electronic device according to an embodiment disclosed in this document acquires an image of a plurality of diagnostic kits, generates a first image patch including a first partial set of the plurality of diagnostic kits, and A second image patch including a second partial set of diagnostic kits is generated, wherein the first partial set and one or more diagnostic kits included in the second partial set overlap, and a first examination result for the first image patch obtaining a second test result set for the set and the second image patch, and the overlapping test results of one or more diagnostic kits included in the first test result set and the duplicated test results included in the second test result set When the test results of the one or more kits are matched, the first test result set and the second test result set may be set to be determined as final test result sets for the plurality of diagnostic kits.

In addition, a method according to an embodiment disclosed in this document is performed by an electronic device, a plurality of user terminals, and a server device, and the server device provides a plurality of users for a plurality of users from the plurality of user terminals. Obtaining identification information, mapping the plurality of user identification information and a plurality of diagnostic kit identification information for a plurality of diagnostic kits to be distributed to each of the plurality of users, and storing the mapping in a user DB, the electronic device Obtaining a test result for each of the one or more diagnostic kits based on an image in which one or more of the diagnostic kits are screened, and acquiring identification information on the one or more diagnostic kits, the electronic device performing the test result and transmitting a test result set including the identification information to the server device, and the server device permits or disallows entry to one or more user terminals corresponding to the one or more diagnostic kits based on the test result set and the user DB. It may include an operation of sending a notification.

According to embodiments disclosed in this document, an electronic device can automatically read test results of a diagnostic kit and computerize the test results. The electronic device can accurately and efficiently read test results for a plurality of diagnostic kits. In addition to this, various effects identified directly or indirectly through this document may be provided.

1 is a block diagram of an electronic device for reading test results of a diagnostic kit according to an embodiment of the present invention.
2 is a flowchart illustrating a method of learning and generating a neural network by an electronic device according to an embodiment of the present invention.
3 is a flowchart illustrating a method of determining a test result for one diagnostic kit according to an embodiment.
4 is a flowchart illustrating a method of determining test results of a plurality of diagnostic kits according to an embodiment.
5 is a diagram for explaining a method of reading test results of a plurality of diagnostic kits and obtaining identification information of the diagnostic kits according to an embodiment.
6 is a flowchart illustrating a method of transmitting to a user whether entry is permitted according to a test result of a diagnostic kit by a system according to an exemplary embodiment.
In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar elements.

Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that this is not intended to limit the present invention to the specific embodiments, and includes various modifications, equivalents, and/or alternatives of the embodiments of the present invention.

1 is a block diagram of an electronic device 100 (hereinafter referred to as electronic device 100) for reading test results of a diagnostic kit according to an embodiment of the present invention.

1 is a block diagram of an electronic device for reading test results of a diagnostic kit according to an embodiment of the present invention. Referring to FIG. 1 , an electronic device 100 according to an embodiment may include a memory 110, a processor 120, an input interface 130, a display 140, and a communication interface 150.

The memory 110 may include a learning data DB 111 , a neural network model 113 , and a command DB 115 .

The learning data DB 111 may include a plurality of images of the diagnostic kit. The image may be obtained through an external server or an external DB, or may be downloaded from the Internet.

The neural network model 113 may be an artificial intelligence model learned based on an object detection algorithm that analyzes an input image, detects an object included in the image, and determines a class. The artificial intelligence model may be generated by the operation of the processor 120 to be described later and stored in the memory 110.

The command DB 115 may store commands capable of executing the operation of the processor 120 . For example, the command DB 115 may store computer codes for performing operations corresponding to operations of the processor 120 to be described later.

The processor 120 may control overall operations of elements included in the electronic device 100, the memory 110, the input interface 130, the display 140, and the communication interface 150.

The processor 120 may include a labeling module 121 , a learning module 123 and a learning module 125 . The processor 120 may execute instructions stored in the memory 110 to drive the labeling module 121, the learning module 123, and the learning module 125, and the labeling module 121, the learning module 123 and An operation performed by the learning module 125 may be understood as an operation performed by the processor 120 .

The labeling module 121 may label (map) a class corresponding to a test result with respect to an image in which a diagnostic kit is photographed. The labeled image may be used for training of an artificial intelligence model. The labeled image may be stored in the training data DB 111 as training data.

The electronic device 100 may acquire an image using a camera (not shown) provided in the electronic device 100 . The electronic device 100 may obtain an image file through an external server or an external DB or obtain an image file on the Internet. In various embodiments, the electronic device 100 may obtain labeled learning data.

Classes for test results may be classified into positive, negative, and invalid. For example, the learning data is an image of a diagnostic kit and may be understood as data labeled with test results.

class 0 One 2 Described invalidity voice positivity

The learning module 123 may input training data to the neural network model 113 designed based on an object detection algorithm and learn weights for deriving a correlation between an image included in the training data and a class labeled in the image. . The neural network model 113 may determine a class for a newly input image based on a correlation derived through a learning process. For example, the learning module 123 may set an input layer of a neural network designed based on an object detection algorithm to input an image included in training data, and set a class labeled for each image to be input to an output layer. The learning module 123 may generate the neural network model 113 by learning the weights of the neural network to derive a correlation between the images included in the training data and the classes labeled for each image.

An object detection algorithm can detect an object in an image and classify the object's class. For example, object detection algorithms include R-CNN, SPP-Net*, Fast R-CNN* Faster R-CNN*, R-FCN, FPN, Mask R-CNN, DetectoRS, YOLO, SSD, YOLOv2, RetinaNet, YOLOv3 , CenterNet, EfficientDet-D2, YOLOv4, Swin-L.

An artificial intelligence model may refer to an overall model that has problem-solving capabilities and is composed of nodes that form a network by combining synapses. An artificial intelligence model can be defined by a learning process that updates model parameters, which is a weight between layers constituting the model, and an activation function that generates an output value.

Model parameters refer to parameters determined through learning, and include weights of layer connections and biases of neurons. In addition, hyperparameters mean parameters that must be set before learning in a machine learning algorithm, and include a learning rate, number of iterations, mini-batch size, initialization function, and the like.

The purpose of training an AI model can be seen as determining the model parameters that minimize the loss function. The loss function can be used as an indicator for determining optimal model parameters in the learning process of an artificial intelligence model.

The learning module 125 may input a new image to the artificial intelligence model for which learning has been completed, and derive a class determined by the artificial intelligence model for the input image as a corresponding inspection result. Accordingly, the learning module 125 may output the inspection result through the display 140 or transmit it to the outside through the communication interface 150 .

The input interface 130 may receive a user's input. For example, when labeling a class for training data, a user's input may be received.

The display 140 may include a hardware component including a display panel to output an image.

The communication interface 150 allows information to be transmitted and received by communicating with an external device (ex. an external server, a user terminal, etc.). To this end, the communication interface 150 may include a wireless communication module or a wired communication module.

In one embodiment, the electronic device 100 may cooperate with an external server and a user terminal through the communication interface 150 to provide a notification service according to a test result. For example, whether or not admission to a specific space (eg, a concert hall or stadium) is permitted may be transmitted to the user. The system including the electronic device 100, an external server, and a user terminal transmits an alarm through the user terminal allowing entry to a user with a negative test result based on the test result, and disallows entry to a user with a positive test result. The alarm may be transmitted through the user terminal. A related embodiment will be described later with reference to FIG. 6 .

In various embodiments, the electronic device 100 may further include a camera 160 . For example, the electronic device 100 may capture learning data or an image to be read using the camera 160 .

2 is a flowchart illustrating a method of generating a neural network by the electronic device 100 according to an embodiment of the present invention.

In an embodiment, the electronic device 100 may acquire a plurality of images of the diagnostic kit. The plurality of images may be labeled with classes corresponding to test results of the diagnostic kit. The electronic device 100 may acquire the plurality of images as learning data.

The electronic device 100 may generate the neural network model 113 based on an object detection algorithm and the learning data. The neural network model 113 may be trained to derive a correlation between at least some regions of each image included in the training data and a labeled class.

Referring to FIG. 2 , an image 200 of a diagnostic kit having a positive test result is shown, for example. The electronic device 100 may label (map) the test result 'class' to '1 (positive)' for the image 200 in which the diagnostic kit is photographed (S210 and S215).

For example, the labeling may be performed based on user input. A user may input information necessary for labeling through the input interface 130, and the electronic device 100 may perform labeling based on the input information. In another embodiment, the electronic device 100 may obtain an image on which labeling is completed from an external source (eg, an external server, etc.).

The labeled images 200-1 and 300-2 may further include bounding boxes 250 and 255 corresponding to at least a partial region of the diagnostic kit. In FIG. 2 , the first image 200-1 includes a first bounding box 201 corresponding to the entire area of the diagnostic kit. The second image 200-2 includes a second bounding box 203 corresponding to the result display area of the diagnostic kit. The result display area corresponds to a physical area (eg, an area in which a membrane inside the diagnostic kit is exposed through an opening of a housing of the diagnostic kit) indicating test results performed by the diagnostic kit. The first image 250 and the second image 255 are examples of training data for training the neural network. ), the bounding box 250 can be assigned to the entire area of the diagnostic kit, and class values can be labeled. These plurality of images may be first learning data for learning the first neural network 113-1 for deriving a correlation between the entire area of the diagnostic kit and the labeled class (S220).

The electronic device 100 acquires, as first learning data, a plurality of images labeled with classes corresponding to test results of the diagnostic kit for the entire area (first area) of the diagnostic kit, and uses a predetermined object detection algorithm and the first training data. Based on 1 training data, a first neural network model 113-1 for deriving a correlation between the first region of each image included in the first training data and the class may be generated.

The electronic device 100 may designate a bounding box 255 in a result display area of the diagnostic kit and label a class value, similar to the second image 200-2, for a plurality of images in which the diagnostic kit is photographed. there is. These plurality of images may be first learning data for learning the second neural network 113-2 for deriving a correlation between the result display area of the diagnostic kit and the labeled class (S225).

The electronic device 100 acquires, as second training data, a plurality of images labeled with classes corresponding to test results of the diagnostic kit with respect to the result display area (second area) of the diagnostic kit, and uses a predetermined object detection algorithm and the first training data. 2 Based on the training data, a second neural network model 113-2 may be generated that derives a correlation between the second region of each image included in the second training data and the class.

Through the process of FIG. 2 , the electronic device 100 uses neural networks 113-1 and 113-2 that derive a correlation between at least some regions 250 and 255 of the image 200 in which the diagnostic kit is captured and the labeled class. ) can be created.

In various embodiments, the electronic device 100 may designate a bounding box 257 for at least a portion of the result display area of the diagnostic kit and label a class value (S217). The labeled image 200-3 may include a third bounding box 257 corresponding to at least a partial area of the result display area of the diagnostic kit. The result display area of the diagnostic kit includes an area including the 'control line (C)' and an area including the 'test line (T)'. The control line is an index for determining whether a test result is valid, and the test line is an index for indicating whether a test result is negative or positive. The third bounding box 257 may be referred to as a region (third region) including the inspection line.

The image including the third bounding box 257 is the first for learning the third neural network 113-3 for deriving a correlation between the labeled class and the region including the control line among the result display areas of the diagnostic kit. It may be learning data (S227).

The electronic device 100 may generate a third neural network 113 - 3 that derives a correlation between the third bounding box 257 and the class for an image belonging to class 1 or class 2 . The third neural network 113-3 may be used to increase the accuracy of judgment for the 1st or 2nd class. When the test results of the diagnostic kit are read, in particular, the shape of the test line may appear in various ways. In some cases, the test line is obvious, but in other cases it is faint or smudged. Such ambiguous cases may be difficult to discern with the human eye. By learning the third neural network 113-3, it is possible to precisely derive a correlation between the 'test line' and positive and negative results. When the test result is read by a person, a more sophisticated test result can be derived.

In various embodiments, the neural network 113 of the electronic device 100 may include a plurality of neural networks 113-1, 113-2, and 113-3. In various embodiments, the plurality of neural networks may be composed of different layers within one neural network.

3 is a flowchart illustrating a method of determining a test result for one diagnostic kit according to an embodiment.

The electronic device 100 may obtain an image of the diagnostic kit (S310). The image may be understood as a newly photographed image of a diagnostic kit for which a test result is to be read.

The electronic device 100 may determine the first class corresponding to the test result based on the first area (eg, the first area 350 of FIG. 3 ) corresponding to the entire area of the diagnostic kit in the image ( S320). The electronic device 100 may determine a second class corresponding to a test result based on a second region (eg, the second region 355 of FIG. 3 ) corresponding to a partial region of the diagnostic kit in the image. (S330). The order of operations S320 and S330 is not essential, and operations S320 or S330 may be performed first, and operations S320 and S330 may be performed simultaneously.

The electronic device 100 may perform operation S320 using the first neural network 113-1 and may perform operation S330 using the second neural network 113-2.

The electronic device 100 may compare the determined first class and second class (S340). If the first class and the second class match, the electronic device 100 may determine the first class as the final class (S350).

When the class is 0, the electronic device 100 may generate a message requesting retesting. When the class is 1, the electronic device 100 may generate a message indicating that the test result is negative. If the class is 2, a message indicating that the test result is positive may be generated. The electronic device 100 may output the generated message through the display unit 140 or transmit it to an external device through the communication interface 150 . Accordingly, the test result of the diagnostic kit may be digitized and stored in the electronic device 100, and the test result may be transmitted in the form of data to an external device that requires the test result.

In order to increase the accuracy of the reading result, the electronic device 100 compares the class determined based on the entire area of the diagnostic kit with the class determined based on the result display area of the diagnostic kit, and derives the final test result only when both are identical. can do.

If the first class and the second class do not match, the electronic device 100 may additionally obtain another image of the diagnostic kit (S310). For example, when an image suitable for reading is not obtained, such as when the resolution of the image is low or the brightness of the image is low, the determination of the inspection result may not be accurate. Accordingly, the electronic device 100 may generate a message requesting re-photographing of the diagnostic kit, output the message through the display unit 140, or transmit the message to an external device through the communication interface 150.

In various embodiments, with respect to the image in which the first class and the second class match in operation S340, the electronic device 100 uses the third neural network 113-3 to generate a third third layer including the inspection line of the result display area. An operation of determining a third class corresponding to the test result may be additionally performed based on the region (eg, the third region 357 of FIG. 3 ). The electronic device 100 may increase reading accuracy by determining the first class as the final class only when the first class, the second class, and the third class match each other.

4 is a flowchart illustrating a method of determining test results of a plurality of diagnostic kits according to an embodiment. 5 is a diagram for explaining a method of reading test results of a plurality of diagnostic kits and acquiring identification information of the diagnostic kits according to an embodiment.

The electronic device 100 may obtain an image in which a plurality of diagnostic kits are photographed (S410). The image is a photographed image of a plurality of diagnostic kits for which test results are to be read. The image may be acquired through an external server or the like, and may be acquired by the electronic device 100 taking a picture using the camera 160 . In the latter case, the electronic device 100 may include a physical space in which a plurality of diagnostic kits can be accommodated. A camera 160 may be provided to face the physical space.

The electronic device 100 may generate a first image patch including the first subset of the plurality of diagnostic kits (S420). The electronic device 100 includes the second subset of the plurality of diagnostic kits. A second image patch may be generated (S430). One or more diagnostic kits included in the first subset and the second subset may overlap.

Accuracy may be reduced when test results for each diagnostic kit are read from an image including a plurality of diagnostic kits. Accordingly, the electronic device 100 may configure a plurality of image patches based on images in order to perform a reading procedure quickly and efficiently.

Referring to FIG. 5 , an image 500 in which a plurality of diagnostic kits are photographed is shown as an example. The electronic device 100 generates four image patches 500 - 1 , 500 - 2 , 500 - 3 , and 500 - 4 from the image 500 .

The number of diagnostic kits to be included in the image patch may be experimentally determined according to the accuracy of the final test result set. In this example, each of the first image patch 500-1 to the fourth image patch 500-4 includes four diagnostic kits.

In various embodiments, the sizes of the image patches 500-1, 500-2, 500-3, and 500-4 may be preset. For example, a size optimized for the processing performance of the neural network 113 may be experimentally determined. Thus, image patches can be created with a fixed size. In this case, the number of diagnostic kits included in the image patch may be different for each image patch.

The first image patch 500-1 to the fourth image patch 500-4 may include at least some diagnostic kits redundantly. In various embodiments, the electronic device 100 executes reading twice or more for each of the overlapping diagnostic kits. The electronic device 100 can increase the accuracy of test results by repeatedly reading one diagnostic kit.

In various embodiments, when a specific pixel portion of an image is dark or blurry, image patches may be generated such that diagnostic kits located at the specific pixel are included in a plurality of image patches.

The electronic device 100 may obtain a first inspection result set for the first image patch and a second inspection result set for the second image patch (S440). A test result set may be understood as a set of test result values for a plurality of diagnostic kits included in an image patch.

The electronic device 100 determines whether the test results of the duplicated one or more diagnostic kits included in the first test result set match the test results of the duplicated one or more diagnostic kits included in the second test result set. It can be determined (S450). If the test results of the one or more diagnostic kits match, the electronic device 100 may determine the first test result set and the second test result set as the final test result set for the plurality of diagnostic kits (S460). ).

In operation S440, the electronic device 100 may read test results of each diagnostic kit included in the first subset or the second subset by the method described above with reference to FIGS. 2 and 3 .

The electronic device 100 determines the first region corresponding to the test result based on the entire area (eg, the first bounding box 250 of FIG. 2 ) of each diagnostic kit included in the first subset or the second subset. 1 class can be determined. The electronic device 100 may determine the second class corresponding to the test result based on the result display area (eg, the second bounding box 255 of FIG. 2 ) of each diagnostic kit. When the first class and the second class match, the electronic device 100 may determine the first class as the final class.

When the final class is determined for each diagnostic kit, the electronic device 100 may add the final class to the first test result set or the second test result set.

The result display area of each diagnostic kit may include a test line area including a test line. In various embodiments, the electronic device 100 determines the inspection result based on the inspection line area when the first class and the second class match and the first class and the second class are previously designated classes. A third class corresponding to may be determined. When the first class and the third class match, the electronic device 100 may determine the first class as the final class.

In various embodiments, each of the plurality of diagnostic kits may include identifier information corresponding to each diagnostic kit. Since the test is performed on a plurality of diagnostic kits, each diagnostic kit can be configured to include its identifier information. Referring to the image 510 of FIG. 5 , a diagnostic kit 550 and a QR code 551 provided in the diagnostic kit 550 are shown. The QR code 551 may be configured to include identifier information of the diagnostic kit 550. The QR code 551 is an example, and an identification mark including a serial number and a barcode may be used.

In an embodiment, the electronic device 100 may acquire a first identifier information set corresponding to the first partial set included in the first image patch, and a second identifier information set corresponding to the second partial set. can be obtained. For example, the electronic device 100 obtains the first identifier information set by recognizing the QR codes of a plurality of diagnostic kits included in the first subset, and the plurality of diagnostic kits included in the second subset. The second identifier information set may be obtained by recognizing the QR codes of the kits.

In various embodiments, the electronic device 100 may transmit the final test result set corresponding to the first identifier information set and the second identifier information set to an external server device. The electronic device 100 causes the external server device to obtain the first identifier information set and the second identifier information set based on the final inspection result corresponding to the first identifier information set and the second identifier information set. It is possible to determine whether access is permitted for corresponding users. An embodiment related to this will be described later with reference to FIG. 6 .

6 is a flowchart illustrating a method of transmitting to a user whether entry is permitted according to a test result of a diagnostic kit by a system according to an exemplary embodiment. As described above in FIG. 1 , the system may include the electronic device 100 , the server device 10 and a plurality of user terminals. 6 shows an arbitrary user terminal 50 among a plurality of user terminals as an example. The server device 10 and the user terminal 50 include a processor (eg, the processor 120 of FIG. 1 , a memory (eg, the memory 110 of FIG. 1 ), a display (eg, the display 140 of FIG. 1 ), A camera (eg, the camera 160 of Fig. 1) may be included. can

For example, the system may provide the notification service as described above for large-scale facilities such as concert halls and stadiums that require many people to enter at the same time. The system may include an electronic device 100, user terminals 50 of a plurality of users who need to enter a large-scale facility, and a server device 10 installed for facility management of a large-scale facility.

When the server device 10 receives the user identifier from the user terminal 50 (S610), the server device 10 may map the identifier of the diagnostic kit to be distributed to the user and the received user identifier and store them in a user DB. (S612). The user identifier may be understood as an identifier for a user who owns the user terminal 50 and has to enter the large-scale facility. The user identifier may be, for example, a unique serial number such as a phone number or a resident registration number. For example, the user terminal 50 may output the user identifier through a display, and the server device 10 may acquire the user identifier by tagging the information. Alternatively, the user identifier may be uploaded to the server device 10 through a separate program installed in the user terminal 50 . The user DB may be stored in the memory of the server device 10 . The user DB may include a list of users (participants of large-scale facilities) managed by the server device 10, a list of diagnostic kit identifiers to be distributed to users, and mapping information between specific users and specific diagnostic kits. Mapping information may be understood as information about identification information of a diagnostic kit to be distributed to a specific user. The mapping information may be determined by a manager of the server device 10 (manager of a large-scale facility). For example, a diagnostic kit to be distributed to a specific user may be determined in advance, and mapping information therefor may be previously stored in a user DB.

By repeating operations S610 to S612, the server device 10 obtains a plurality of user identifiers for a plurality of users from the plurality of user terminals, and obtains the plurality of user identifiers and a plurality of user identifiers to be distributed to the plurality of users. Identification information of diagnostic kits may be mapped and stored in a user DB.

The electronic device 100 may generate a test result set including test results and identification information for a plurality of diagnostic kits (S620). For example, the electronic device 100 may read the test results in units of n arbitrary pieces of diagnostic kits distributed to users. At this time, the electronic device 100 may read the test results of the n diagnostic kits based on the image in which the n diagnostic kits are printed. The electronic device 100 may obtain identification information for n diagnostic kits from the images. For example, the electronic device 100 may obtain the identification information based on an identification mark (eg, the QR code 551 of FIG. 5 ) provided in the diagnostic kit. The test result set includes test results and identification information for the n diagnostic kits. The server device 10 may transmit the test result set to the server device 10 (S622).

In various embodiments, the server device 10 may photograph n diagnostic kits as a unit of examination and transmit the images to the electronic device 100 . Alternatively, the electronic device 100 may take pictures in n units through the camera 160 and perform operations S620 and S622 in units of the captured images.

Based on the test result set, the server device 10 may determine whether to allow or deny entry to a user corresponding to a specific diagnostic kit (S624). The server device 10 may obtain user information mapped to the identifier of the diagnostic kit included in the test result set from the user DB. In various embodiments, the server device 10 may map access information on whether admission is permitted or denied to the user information and store it in a user DB.

When it is determined whether to permit or deny admission to a specific diagnostic kit, the server device 10 may transmit an alarm of permitting or disallowing admission to a user terminal corresponding to the specific diagnostic kit (S634). Operation S634 may be repeated until admission information is transmitted to all users.

For example, the server device 10 may identify the user terminal 50 to which admission information is to be transmitted based on user information stored in the user DB, and transmit a notification to the identified user terminal 50 . In another example, the user outputs the user identifier through the user terminal 50 (S630), and the server device 10 may obtain the user identifier by tagging the output screen (S632). In this case, the server device 10 may transmit access information corresponding to the user identifier to the user terminal 50 through a short-range communication channel between the user terminals 50 formed during tagging. For example, operations S630 and 632 may be performed when a user enters a large-scale facility and controls the entrance according to the inspection result at the entrance of the facility.

Electronic devices according to various embodiments disclosed in this document may be devices of various types. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. An electronic device according to an embodiment of the present document is not limited to the aforementioned devices.

Various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, like reference numbers may be used for like or related elements. The singular form of a noun corresponding to an item may include one item or a plurality of items, unless the relevant context clearly dictates otherwise. In this document, "A or B", "at least one of A and B", "at least one of A or B," "A, B or C," "at least one of A, B and C," and "A Each of the phrases such as “at least one of , B, or C” may include all possible combinations of items listed together in the corresponding one of the phrases. Terms such as "first", "second", or "first" or "secondary" may simply be used to distinguish a given component from other corresponding components, and may be used to refer to a given component in another aspect (eg, importance or order) is not limited. A (e.g., first) component is said to be "coupled" or "connected" to another (e.g., second) component, with or without the terms "functionally" or "communicatively." When mentioned, it means that the certain component may be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term "module" used in this document may include a unit implemented by hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be an integrally constructed component or a minimal unit of components or a portion thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of this document may be implemented as software including one or more instructions stored in a storage medium (eg, memory 110) readable by a machine (eg, electronic device 100). can For example, a processor (eg, the processor 120 ) of a device (eg, the electronic device 100 ) may call at least one command among one or more instructions stored from a storage medium and execute it. This enables the device to be operated to perform at least one function according to the at least one command invoked. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' only means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and this term refers to the case where data is stored semi-permanently in the storage medium. It does not discriminate when it is temporarily stored.

According to one embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. A computer program product is distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store (eg Play Store ^TM ) or between two user devices ( It can be distributed (eg downloaded or uploaded) online, directly between smartphones. In the case of online distribution, at least part of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium such as a manufacturer's server, an application store server, or a relay server's memory.

According to various embodiments, each component (eg, module or program) of the components described above may include a singular entity or a plurality of entities. According to various embodiments, one or more components or operations among the aforementioned corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by a corresponding component of the plurality of components prior to the integration. . According to various embodiments, the actions performed by a module, program, or other component are executed sequentially, in parallel, iteratively, or heuristically, or one or more of the actions are executed in a different order, or omitted. or one or more other actions may be added.

Claims (9)

In electronic devices,
one or more memories for storing instructions to perform a predetermined operation; and
at least one processor operably connected with the one or more memories and configured to execute the instructions;
The at least one processor,
Obtaining an image in which a plurality of diagnostic kits are photographed;
generating a first image patch including a first subset of the plurality of diagnostic kits;
generating a second image patch including a second partial set of the plurality of diagnostic kits, wherein at least one diagnostic kit included in the first partial set and the second partial set overlap;
obtain a first inspection result set for the first image patch and a second inspection result set for the second image patch; and
When the test results of the one or more overlapping diagnostic kits included in the first test result set and the test results of the overlapping one or more diagnostic kits included in the second test result set match, the first test result set and The electronic device configured to determine the second test result set as a final test result set for the plurality of diagnostic kits. The method of claim 1,
The image includes identifier information for each of the plurality of diagnostic kits,
The at least one processor,
obtaining a first identifier information set corresponding to the first partial set; and
An electronic device configured to obtain a second identifier information set corresponding to the second partial set. The method of claim 2,
The image includes a QR code provided in each of the plurality of diagnostic kits,
The at least one processor,
obtaining the first identifier information set by recognizing the QR codes of the plurality of diagnostic kits included in the first partial set; and
The electronic device configured to acquire the second identifier information set by recognizing the QR codes of the plurality of diagnostic kits included in the second partial set. The method of claim 2,
The electronic device further includes a communication interface configured to communicate with an external server device;
The at least one processor,
The external server allows access to users corresponding to the first identifier information set and the second identifier information set based on the final inspection result corresponding to the first identifier information set and the second identifier information set. The electronic device configured to transmit the final test result set corresponding to the first identifier information set and the second identifier information set to the external server device to determine permission. The method of claim 1,
The at least one processor,
determining a first class corresponding to a test result based on the entire region of the first diagnostic kit included in the first partial set or the second partial set;
determining a second class corresponding to a test result based on a result display area of the first diagnostic kit;
if the first class and the second class match, determining the first class as a final class; and
The electronic device configured to add the final class to the first test result set or the second test result set. The method of claim 5,
The at least one processor,
The sample diagnostic kit acquires a sample image,
Creating first learning data by designating a first bounding box corresponding to the entire area of the sample diagnostic kit in the sample image and mapping a result class corresponding to a test result of the sample diagnostic kit to the first bounding box; ;
Based on a predetermined object detection algorithm and the first training data, a first neural network model for deriving a correlation between the first bounding box of the sample image included in the first training data and the result class is generated; ;
designating a second bounding box corresponding to a result display area of the sample diagnostic kit in the sample image, and generating second learning data by mapping the result class to the second bounding box; and
generate a second neural network model that derives a correlation between the second bounding box of the sample image included in the second training data and the result class, based on a predetermined object detection algorithm and the second training data; Set, electronic device. The method of claim 5,
The result display area includes an inspection line area including an inspection line,
The at least one processor,
determining a third class corresponding to the inspection result based on the inspection line area when the first class and the second class match, and the first class and the second class are previously designated classes; and
The electronic device configured to determine the first class as the final class when the first class and the third class match. In a method performed by an electronic device, a plurality of user terminals, and a server device,
The server device obtains a plurality of user identification information for a plurality of users from the plurality of user terminals, and maps mapping information of the plurality of user identification information and a plurality of diagnostic kit identification information to a plurality of diagnostic kits into a user DB. the operation of saving to;
obtaining, by the electronic device, test results and diagnostic kit identification information for the plurality of diagnostic kits based on the captured images of the plurality of diagnostic kits;
transmitting, by the electronic device, a test result set including the test result and the diagnostic kit identification information to the server device; and
and transmitting, by the server device, an entry permission or non-permission notification to one or more user terminals corresponding to the one or more diagnostic kits based on the test result set and the user DB. The method of claim 8,
The electronic device generates a first image patch including a first partial set of the plurality of diagnostic kits of the image and generates a second image patch including a second partial set of the plurality of diagnostic kits of the image. including, but one or more diagnostic kits included in the first subset and the second subset overlap;
obtaining, by the electronic device, a first inspection result set for the first image patch and a second inspection result set for the second image patch; and
When the electronic device matches the test result of the one or more overlapping diagnostic kits included in the first test result set and the test result of the duplicated one or more diagnostic kits included in the second test result set, the first test result set The method further comprising determining a test result set and the second test result set as the test result set for the plurality of diagnostic kits.

Images