KR20220076398A - Object recognition processing apparatus and method for ar device - Google Patents

Abstract

An object recognition processing apparatus for an AR (Augmented Reality) device according to an embodiment of the present invention includes a communication module; a memory in which an object recognition program is stored; and a processor executing the object recognition program. In this case, the object recognition program includes an AR information collection module that receives an AR image, a shooting time of the AR image, location information of a place where the AR image was shot, and information on the AR device that captured the image from at least one AR device; An object detection module for discriminating each object in the AR image, receiving object category information divided through the object detection module, tracking the object based on the degree of similarity of each object, and extracting a data set for learning the object Object tracking and dataset collection module, an object learning module that receives a data set from the object tracking and dataset collection module, and performs learning to classify each object based on the similarity of the received data set, and an input AR image and an object recognition module for inputting an object identified through the object detection module from the object learning module to output identification information of the object.

Description

Object recognition processing apparatus and method for AR device {OBJECT RECOGNITION PROCESSING APPARATUS AND METHOD FOR AR DEVICE}

The present invention relates to an object recognition processing apparatus and method for an AR device.

Augmented reality (AR) systems start from an event-type usage method that recognizes a scene through a smartphone screen only at a special event (museum, machine assembly, etc.), and is recently changing to a wearable device such as smart glasses. .

For the existing event-type AR system, it was possible to provide identification means (QR code, license plate, location coordinates) for recognizing specific conditions or learn about a small number of components (exhibits, machine parts) to obtain information to be augmented. However, when applying the existing learning and inference method to a new usage method, if the scope of the entire object is not limited to a small number of components as in the existing event-type AR system, the category of objects (person, car, chair) is a problem of learning difficulty. etc.), it is possible to obtain only a level of classifying (classifying).

In order to overcome this, an efficient alternative that considers the characteristics of AR, rather than simply increasing the number of learning and models, is needed. The reason is that, first, it is very difficult to create a dataset that can sufficiently learn about all objects. This is because there is a limit to modifying the model for a new object in a short time every time in the situation of

In addition, unlike the existing deep learning learning in a controlled situation, the AR environment where damage to the object image (blur, occlusion, etc.) is frequent must also be considered.

There are two major requirements for deep learning algorithms for new AR systems.

The first is to secure the performance capable of classifying each object beyond the category of the object, and secondly, it must be able to overcome this even when the object image is damaged.

In relation to object detection/classification, the widely known image deep learning technique is a technique that simultaneously infers the area where a specific object exists (Bounding box) and the category of the object within the entire image. Mask RCNN, YOLO v3, Mask RCNN, YOLO v3, Various studies such as SSD are in progress. Also, if the input data is a video, it is inefficient to use the image deep learning technique for every frame, so if a bounding box of a specific object is given regardless of the object category (class-agnostic), the An object tracking deep learning technique that detects the movement of an object and provides a moved bounding box is used, and various studies such as MDNet, SiamFC, and SiamRPN++ are in progress.

In contrast to image-based object detection, object tracking is easier to deal with object deformation, in which the same object takes on a different appearance due to changes in angle, lighting, etc. over time. It has a characteristic that

Therefore, by utilizing these features, the AR system uses object tracking to build a training data set, and at the same time, through a combination of a classification model and a Siamese similarity model It is possible to classify by more specific object ID, and it is thought that object image deformation can also be overcome.

An object of the present invention is to provide an object recognition processing apparatus to which a new learning and inference method capable of enhancing the performance of an object recognition algorithm of an augmented reality system is applied.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

As a technical means for solving the above technical problem, an object recognition processing apparatus for an AR (Augmented Reality) device according to an embodiment of the present invention includes: a communication module; a memory in which an object recognition program is stored; and a processor executing the object recognition program, wherein the object recognition program includes an AR image from at least one AR device, a recording time of the AR image, location information of a place where the AR image was captured, and an AR device that captured the image An AR information collection module for receiving information on, an object detection module for discriminating each object in the AR image, and an object category information divided through the object detection module and tracking the object based on the degree of similarity of each object, , an object tracking and dataset collection module for extracting a data set for learning the corresponding object, receiving a data set from the object tracking and dataset collection module, and classifying each object based on the similarity of the received data set and an object learning module for performing learning, and an object recognition module for outputting identification information of an object by inputting an object separated from the input AR image through the object detection module into the object learning module.

According to another embodiment of the present invention, an object recognition processing method using an object recognition processing apparatus for an AR (Augmented Reality) device includes an AR image from at least one AR device, a recording time of the AR image, and a place where the AR image is captured. receiving location information of the AR device and information about an AR device that has captured the corresponding image; classifying each object in the AR image and generating classified object category information; receiving the divided object category information, tracking the object based on the degree of similarity of each object, and extracting a data set for learning the object; building an object learning module by receiving the extracted data set and performing learning to classify each object based on the similarity of the received data set; and inputting the object divided from the input AR image into the object learning module and outputting identification information of the object.

According to the above-described means for solving the problem of the present application, a training data set is constructed by utilizing object tracking in the AR system, and at the same time, a classification model and a Siamese similarity model learned through this data set are Through the combination, more specific object ID classification than category is possible, and object image deformation can also be overcome.

1 is a block diagram illustrating the configuration of an object recognition processing apparatus for an AR device according to an embodiment of the present invention.
2 is a block diagram illustrating the configuration of an object recognition program according to an embodiment of the present invention.
3 is a diagram for explaining an operation of an object detection module according to an embodiment of the present invention.
4 and 5 are diagrams for explaining the operation of the object tracking and data set collection module according to an embodiment of the present invention.
6 is a diagram for explaining an operation of an object learning module according to an embodiment of the present invention.
7 is a diagram for explaining an operation of an object recognition module according to an embodiment of the present invention.
8 is a flowchart illustrating an object recognition method according to an embodiment of the present invention.

Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present application pertains can easily carry out. However, the present application may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present application in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout this specification, when a part is "connected" with another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element interposed therebetween. do.

Throughout this specification, when a member is said to be located “on” another member, this includes not only a case in which a member is in contact with another member but also a case in which another member is present between the two members.

Throughout this specification, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated. As used throughout this specification, the terms "about," "substantially," and the like are used in a sense at or close to the numerical value when the manufacturing and material tolerances inherent in the stated meaning are presented, and are intended to enhance the understanding of this application. To help, precise or absolute figures are used to prevent unfair use by unconscionable infringers of the stated disclosure. The term “step of” or “step of” to the extent used throughout this specification does not mean “step for”.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings and the content to be described later. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Like reference numerals refer to like elements throughout.

Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present application pertains can easily carry out. However, the present application may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present application in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout this specification, when a part is "connected" with another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element interposed therebetween. do.

Throughout this specification, when a member is said to be located “on” another member, this includes not only a case in which a member is in contact with another member but also a case in which another member is present between the two members.

Throughout this specification, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated. As used throughout this specification, the terms "about," "substantially," and the like are used in a sense at or close to the numerical value when the manufacturing and material tolerances inherent in the stated meaning are presented, and are intended to enhance the understanding of this application. To help, precise or absolute figures are used to prevent unfair use by unconscionable infringers of the stated disclosure. The term “step of” or “step of” to the extent used throughout this specification does not mean “step for”.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings and the content to be described later. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Like reference numerals refer to like elements throughout.

Hereinafter, an industrial accident event extraction system according to an embodiment of the present invention will be described.

1 is a block diagram illustrating the configuration of an object recognition processing apparatus for an AR device according to an embodiment of the present invention.

Referring to FIG. 1 , the object recognition processing device 100 processes object recognition using AR images collected from a plurality of AR devices 200 to 204 , and may perform an operation like a server. have. To this end, the object recognition processing apparatus 100 includes a communication module 110 , a memory 120 , a processor 130 , and a database 140 .

Next, the communication module 110 is a wired network such as a local area network (LAN), a wide area network (WAN), or a value added network (VAN), or a mobile radio communication network (mobile radio communication network). ) or a communication module using all kinds of wireless networks, such as satellite communication networks. In particular, the communication module 110 provides a communication interface with each AR device 200 to 204 connected through an external communication network.

The memory 120 stores an object recognition program. The object recognition program collects AR information for receiving at least the AR image, the shooting time of the AR image, location information of the place where the AR image was taken, and information about the AR device that took the image from each of the AR devices 200 to 204 The module 310, the object detection module 320 for discriminating each object in the AR image, receives the object category information divided through the object detection module 320, and tracks the object based on the similarity of each object, Receive a data set from the object tracking and dataset collection module 330, which extracts a data set for object learning, and the object tracking and dataset collection module 330, and collects each object based on the similarity of the received data set. An object learning module 340 that performs learning to classify and an object recognition module that inputs an object classified from the input AR image through the object detection module 320 into the object learning module 340 and outputs identification information of the object (350).

The memory 120 should be interpreted as a generic term for a non-volatile storage device that continuously maintains stored information even when power is not supplied, and a volatile storage device that requires power to maintain the stored information. In addition, the memory 120 may perform a function of temporarily or permanently storing data processed by the processor 130 . The memory 120 may include magnetic storage media or flash storage media in addition to the volatile storage device that requires power to maintain stored information, but the scope of the present invention is not limited thereto. not.

The processor 130 executes an object recognition program stored in the memory 120 . The processor 130 may include various types of devices for controlling and processing data. The processor 130 may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed as a code or an instruction included in a program. In one example, the processor 200 is a microprocessor (microprocessor), central processing unit (CPU), processor core (processor core), multiprocessor (multiprocessor), ASIC (application-specific integrated circuit), FPGA ( field programmable gate array), but the scope of the present invention is not limited thereto.

In addition, the database 140 may manage various AR images collected from each AR device, the shooting time of the AR image, location information of a place where the AR image was shot, information about the AR device that captured the image, and the like.

Meanwhile, the object recognition processing device 100 may receive an AR image from each AR device 200 to 204 and operate as a server that provides information on an object identified through an object recognition program. In this case, the object recognition processing apparatus 100 may operate in a cloud computing service model such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS). In addition, the object recognition processing apparatus 100 may be built in a form such as a private cloud, a public cloud, or a hybrid cloud.

2 is a block diagram illustrating the configuration of an object recognition program according to an embodiment of the present invention.

The object recognition program collects AR information for receiving at least the AR image, the shooting time of the AR image, location information of the place where the AR image was taken, and information about the AR device that took the image from each of the AR devices 200 to 204 The module 310, the object detection module 320 for discriminating each object in the AR image, receives the object category information divided through the object detection module 320, and tracks the object based on the similarity of each object, Receive a data set from the object tracking and dataset collection module 330, which extracts a data set for object learning, and the object tracking and dataset collection module 330, and collects each object based on the similarity of the received data set. An object learning module 340 that performs learning to classify and an object recognition module that inputs an object classified from the input AR image through the object detection module 320 into the object learning module 340 and outputs identification information of the object (350).

The AR information collection module 310, as defined above, includes not only the AR image from each AR device 200 to 204, but also the time at which the AR image is captured, location information of the place where the AR image is captured (eg, GPS information, etc.), and information (eg, terminal unique number) about the AR device that captured the image. In particular, information on AR devices may be used to classify users who use each AR device in the future.

3 is a diagram for explaining an operation of an object detection module according to an embodiment of the present invention.

The object detection module 320 generates a bounding box for each object in the AR image, gives object category information to each object-specific bounding box, and transmits it to the object tracking and dataset collection module 330 .

As shown in FIG. 3 , a bounding box is generated for each object in the AR image, and a conventional technique such as YOLO is used as an algorithm for generating the bounding box. Then, object category information is given to the bounding box for each object and transmitted to the object tracking and data set collection module 330 . The object tracking and dataset collection module 330 performs object tracking based on the object category information received from the object detection module 320 , and transmits the object tracking performance result back to the object detection module 320 , to be used for detection.

On the other hand, when the object category classification result is derived in the object detection step, the object category information is transmitted to the object recognition module 350, and the object recognition module 350 additionally utilizes the object category classification result to obtain identification information of the object. print out The object detection module 320 may provide classification information of, for example, a person, a car, an animal, etc. as category information of an object.

In addition, as shown in FIG. 3 , when an object that has been previously detected and tracked in the AR image and an object that has not been tracked exist together, the remaining area except for the area of the object being tracked, that is, a newly appeared object is detected. carry out In this way, the newly detected object may be recognized as a new object and transmitted to the object tracking and data set collection module 330 and the object recognition module 350 , respectively.

4 and 5 are diagrams for explaining the operation of the object tracking and data set collection module according to an embodiment of the present invention.

The object tracking and dataset collection module 330 receives the object category information divided through the object detection module 320 , tracks the object based on the similarity of each object, and extracts a data set for learning the object do.

As shown in FIG. 4 , images of a plurality of divided frames are received from the AR image, and information on objects for each frame divided through the object detection module 320 for each frame is received. In addition, the object tracking and data set collection module 330 may track an object by applying a Siamese network to extract an object having the greatest similarity for each frame.

In particular, as shown in Equation 1 below, a plurality of frames in which the sum of similarities between objects among all frames used for object tracking is minimized or a predetermined number of frames in which the similarity between frames is less than or equal to a threshold is selected. Select the data set for training.

[Equation 1]

i, j indicate frame number, S _ij indicates similarity between two frames i, j

K is a subset of U, which is the set of all frames, and n(K) is the number of frames belonging to the set K

In this way, since a data set for each object is collected for a frame with a large difference in similarity within an image in which a specific object is tracked, it is possible to build a learning model for various types of deformation of each object.

In this case, the configuration of the Siamese network for determining the similarity is as shown in FIG. 5 , and is known as an algorithm for calculating the similarity for each object included in each frame.

Then, the object tracking and data set collection module 330 adds additional identification information of each object extracted from characteristic information of the object identified in the object detection or object tracking process to the data set for learning, and object learning The module 340 learns each object based on the data set to which additional identification information is added.

For example, as the feature information of the object, in the case of a person, feature information of the face of the object is used, in the case of a car, feature information identified through a license plate or information such as a barcode attached to each article can be used as feature information. can

Since such information is characteristic information representing an object, by using it, it is possible to more accurately identify an object, and based on this, it is possible to more easily set identification information of each object.

6 is a diagram for explaining an operation of an object learning module according to an embodiment of the present invention.

The object learning module 340 receives a data set from the object tracking and data set collection module 330 and performs learning to classify each object based on the similarity of the received data set.

As shown in FIG. 6 , the object learning module 340 provides object identification information for classifying each object based on the similarity of the received data set, but the AR image capture time and AR image location Learning is performed by updating the data set by additionally considering location information.

For example, based on location information of a place where an AR image collected from each AR device was captured, a data set to which object identification information is assigned is collected for each separated location (position A, location B). In this case, the data set includes similar frame images and object detection results (information about object categories) for each object identification information.

Then, the preceding steps are sequentially performed, and when a new data set is input, the data set is updated with a focus on the data set having a high similarity through similarity comparison with the existing data set. In this process, the data set is updated to include the data sets collected from different AR devices, so that the data set includes various features and has an even distribution.

Meanwhile, the Siamese network described above may be used for determining the similarity between data sets or for updating the data sets in a form suitable for each region.

7 is a diagram for explaining an operation of an object recognition module according to an embodiment of the present invention.

The object recognition module 350 inputs the object identified through the object detection module 320 from the input AR image into the object learning module 340 and outputs identification information of the object.

At this time, the object recognition module 350 classifies the object classified through the object detection module 320 and the object learning module 340 into existing object identification information based on the similarity between the object-specific data set 340, or a new object. Identification information may be assigned or classified as unconfirmed. In this case, the Siamese network described above may be used to determine the similarity between the input AR image and the data set of the object learning module 340 .

When the similarity between the input AR image and the specific data set is greater than or equal to the threshold, the object learning module 340 classifies it as corresponding to object identification information (ID) representing the data set.

If the similarity between the input AR image and the entire data set does not reach the threshold, the object learning module 340 considers that the corresponding data set has not been secured, and after assigning new object identification information (ID), the corresponding object identification information Build a data set based on (ID).

When the similarity between the input AR image and the plurality of data sets is greater than or equal to the threshold, specific object identification information (ID) is not assigned and is classified as unconfirmed.

8 is a flowchart illustrating an object recognition method according to an embodiment of the present invention.

First, an AR image, a recording time of the AR image, location information of a place where the AR image was captured, and information on the AR device that captured the image are received from at least one AR device ( S810 ). As an operation performed by the AR information collection module 310 of the object recognition processing device 100, the AR data thus received is used to build a learning module, and then used to perform an inference process on a newly input AR image.

Next, each object is divided in the AR image, and divided object category information is generated ( S820 ). As an operation performed by the object detection module 320 of the object recognition processing apparatus 100, a bounding box is created for each object in an image, and category classification is performed on the object. The detected object information is then transmitted to the object tracking and data set collection step (S830) and the object recognition step (S850), respectively.

Next, the object category information is received, the object is tracked based on the degree of similarity of each object, and a data set for learning the object is extracted (S830). As an operation performed in the object tracking and data set collection module 330 of the object recognition processing apparatus 100, the similarity is calculated using a Siamese network, and extracted as an optimal frame data set through Equation 1 described above. , for example, a plurality of frames in which the sum of similarities between objects is minimized or a predetermined number of frames in which the similarity between frames is equal to or less than a threshold value is selected as a data set for learning. In addition, by adding additional identification information of each object extracted from characteristic information of the object identified in the object detection or object tracking process to the data set for learning, it is possible to improve the accuracy of the data set for each ID.

Next, the extracted data set is received, and an object learning module is constructed by performing learning to classify each object based on the degree of similarity of the received data set (S840). Based on the similarity of the received data set, the object identification information for classifying each object is given, and the data set is updated by additionally considering the recording time of the AR image and the location information of the place where the AR image was captured. carry out

Next, an object recognition step of outputting identification information of an object by inputting an object separated from the input AR image into the object learning module is performed (S850). Based on the degree of similarity between the object and the data set for each object of the object learning module, the object may be classified as existing object identification information, new object identification information may be assigned, or classified as unconfirmed.

The object recognition method according to an embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Although the methods and systems of the present invention have been described with reference to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.

The foregoing description of the present application is for illustration, and those of ordinary skill in the art to which the present application pertains will understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present application. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present application is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present application.

100: object recognition processing unit
110: communication module
120: memory
130: processor
140: database
300: object recognition program
310: AR information collection module
320: object detection module
330: object tracking and data set collection module
340: object learning module
350: object recognition module

Claims (19)

In the object recognition processing device for AR (Augmented Reality) device,
communication module;
a memory in which an object recognition program is stored; and
A processor executing the object recognition program,
The object recognition program includes an AR information collection module that receives an AR image, a recording time of the AR image, location information of a place where the AR image was captured, and information on an AR device that captured the image from at least one AR device, the AR An object detection module for discriminating each object in an image, an object receiving object category information divided through the object detection module, tracking the object based on the degree of similarity of each object, and extracting a data set for learning the object From a tracking and dataset collection module, an object learning module that receives a data set from the object tracking and dataset collection module, and performs learning to classify each object based on the similarity of the received data set, and an input AR image and an object recognition module for inputting the object classified through the object detection module into the object learning module and outputting identification information of the object. According to claim 1,
The object detection module generates a bounding box for each object in the AR image, gives the object category information to each object-specific bounding box, and transmits the object category information to the object tracking and dataset collection module. According to claim 1,
The object detection module generates a bounding box for each object in the AR image, gives the object category information to each object-specific bounding box, and transmits the object category information to the object recognition module;
The object recognition module further utilizes the object category information to output identification information of the object. According to claim 1,
Wherein the object detection module performs object detection on the remaining area except for the area of the object when there is an object being detected and tracked in the AR image. According to claim 1,
The object tracking and dataset collection module applies a Siamese network to each frame-by-frame object classified through the object detection module from the AR image, and extracts the object with the greatest degree of similarity for each frame. Tracking the object recognition processing device. 6. The method of claim 5,
The object tracking and data set collection module selects a plurality of frames in which the sum of similarities between objects among all frames used for object tracking is the minimum or a predetermined number of frames in which the similarity between frames is less than or equal to a threshold value. An object recognition processing device that is selected as a data set for learning. According to claim 1,
The object tracking and data set collection module adds additional identification information of each object extracted from the characteristic information of the object identified in the object detection or object tracking process to the data set for learning,
The object learning module is to learn each object based on the data set to which the additional identification information is added, the object recognition processing apparatus. According to claim 1,
The object learning module provides the object identification information for classifying each object based on the similarity of the received data set, and additionally considers the recording time of the AR image and the location information of the place where the AR image is captured. To perform learning in a way to update the object recognition processing device. According to claim 1,
The object recognition module
Based on the similarity between the object classified through the object detection module and the data set for each object of the object learning module, classification as existing object identification information, new object identification information, or classification as unconfirmed, object recognition processing unit. In the object recognition processing method using the object recognition processing device for AR (Augmented Reality) device,
(a) receiving an AR image, a recording time of the AR image, location information of a place where the AR image was captured, and information on the AR device that captured the image from at least one AR device;
(b) classifying each object in the AR image and generating classified object category information;
(c) receiving the divided object category information, tracking the object based on the degree of similarity of each object, and extracting a data set for learning the object;
(d) receiving the extracted data set and constructing an object learning module by performing learning to classify each object based on the degree of similarity of the received data set; and
(e) inputting the object divided from the input AR image into the object learning module and outputting identification information of the object; 11. The method of claim 10,
The step (b) is to create a bounding box for each object in the AR image, and to give the object category information to the bounding box for each object, the object recognition processing method. 11. The method of claim 10,
In the step (b), a bounding box is created for each object in the AR image, the object category information is given to the bounding box for each object, and the object category information is transmitted to the step (e),
The step (e) is to output the identification information of the object by further utilizing the object category information, object recognition processing method. 11. The method of claim 10,
In the step (b), if there is an object being detected and being tracked in the AR image, object detection is performed on the remaining area except for the area of the object. 11. The method of claim 10,
The step (c) applies a Siamese network to the object for each frame from the AR image, and the object is tracked by extracting the object with the greatest similarity for each frame. 15. The method of claim 14,
The step (c) selects a plurality of frames in which the sum of similarities between objects is the minimum among all frames used for object tracking, or a predetermined number of frames in which the similarity between frames is less than or equal to a threshold value for learning. An object recognition processing method, which is selected as a data set. 11. The method of claim 10,
In the step (c), additional identification information of each object extracted from the characteristic information of the object identified in the object detection or object tracking process is added to the data set for learning,
The step (d) is to learn each object based on the data set to which the additional identification information is added, the object recognition processing method. 11. The method of claim 10,
In the step (d), the object identification information for classifying each object is given based on the degree of similarity of the received data set, and data is additionally taken into account in consideration of the recording time of the AR image and the location information of the place where the AR image is captured. An object recognition processing method that performs learning by updating the set. 11. The method of claim 10,
In step (e), based on the degree of similarity between the object classified through step (b) and the data set for each object of the object learning module, classifying the object into existing object identification information, or assigning new object identification information, An object recognition processing method that classifies as unconfirmed. A computer-readable medium in which a program for executing the method according to any one of claims 10 to 18 is recorded.

Images