JP7267974B2 - IDENTIFICATION DEVICE, SCANNER SYSTEM, IDENTIFICATION METHOD, AND IDENTIFICATION PROGRAM - Google Patents

Description

The present disclosure relates to an identification device, a scanner system including the identification device, an identification method, and an identification program.

2. Description of the Related Art Conventionally, in the field of dentistry, a 3D scanner with a built-in 3D camera for acquiring a 3D shape of a tooth is known for digital designing of a prosthesis or the like on a computer. For example, Patent Literature 1 discloses a technique for recording the shape of a tooth by capturing an image of the tooth using a three-dimensional camera.

JP-A-2000-74635

Thus, three-dimensional scanners have been used to digitally design prostheses and the like. However, no consideration has been given to using the information obtained by the three-dimensional scanner for other purposes.

An object of the present disclosure is to expand the use of three-dimensional data obtained by a three-dimensional scanner, and to enhance the possibilities of three-dimensional scanners in the dental field.

According to the present disclosure, an identification device is provided for identifying a processed site with respect to a living body site in the oral cavity. The identification device includes an input unit into which three-dimensional data including three-dimensional position information at each of a plurality of points representing the three-dimensional shape of the oral cavity is input, three-dimensional data input from the input unit, and three-dimensional data. and an estimating model including a neural network for estimating the machined portion based on the identification unit for identifying the machined portion, and an output unit for outputting the identification result of the identification unit. The identification unit directly inputs at least three-dimensional position information of the three-dimensional data to a neural network included in the estimation model.

According to the present disclosure, a scanner system is provided for acquiring intraoral shape information. The scanner system includes a three-dimensional scanner that acquires three-dimensional data including three-dimensional position information at each of a plurality of points that indicate the three-dimensional shape of the oral cavity using a three-dimensional camera, and a three-dimensional scanner acquired by the three-dimensional scanner. and an identification device that identifies the processed part of the living body part in the oral cavity based on the original data. An identification unit for identifying a machined portion based on three-dimensional data and an estimation model including a neural network for estimating the machined portion based on the three-dimensional data, and an output unit for outputting the identification result of the identification unit. The identification unit directly inputs at least three-dimensional position information of the three-dimensional data to a neural network included in the estimation model.

According to the present disclosure, a computer-assisted identification method is provided for identifying a processed portion for a biological portion within an oral cavity. The identification method includes a step of inputting three -dimensional data including three-dimensional position information at each of a plurality of points representing a three-dimensional shape of the oral cavity; a step of identifying a machined portion based on an estimation model including a neural network for estimating a machined portion based on data; and a step of outputting an identification result obtained by the identifying step. At least three-dimensional position information among the three-dimensional data is directly input to a neural network included in the estimation model.

According to the present disclosure, an identification program is provided for identifying a processed site relative to an intraoral biometric site. The identification program includes a step of inputting three-dimensional data including three-dimensional position information of each of a plurality of points representing a three-dimensional shape of the oral cavity into the computer, and the three-dimensional data input by the input step. and an estimation model including a neural network for estimating the machined part based on the three-dimensional data. At least three-dimensional position information among the three-dimensional data is directly input to a neural network included in the estimation model.

According to the present disclosure, it is possible to identify a processed site in the oral cavity from three-dimensional data including three-dimensional position information at each of a plurality of points that constitute the shape of the oral cavity, and to identify the processed site. can provide a variety of uses.

1 is a schematic diagram showing the overall configuration of a scanner system 10 according to a first embodiment; FIG. 1 is a schematic diagram showing an application example of the scanner system 10 according to the first embodiment; FIG. 1 is a schematic diagram showing the overall configuration of a system to which the scanner system 10 according to the first embodiment can be applied; FIG. 1 is a schematic diagram showing a hardware configuration of an identification device 100 according to Embodiment 1; FIG. 1 is a schematic diagram showing a functional configuration of an identification device 100 according to Embodiment 1; FIG. FIG. 4 is a schematic diagram for explaining identification processing by the identification device 100 according to the first embodiment; 4 is a schematic diagram for explaining generation of learning data according to the first embodiment; FIG. 4 is a schematic diagram for explaining an example of learning data according to the first embodiment; FIG. 4 is a flowchart for explaining an example of processing type learning processing executed by the identification device 100 according to the first embodiment. 5 is a flowchart for explaining an example of service providing processing executed by the identification device 100 according to the first embodiment; FIG. 10 is a diagram showing an output example during measurement; FIG. 10 is a diagram showing a modified example of an output during measurement; FIG. 10 is a schematic diagram showing a functional configuration that functions during measurement of the scanner system 10a according to the second embodiment; FIG. 11 is a schematic diagram showing a functional configuration that functions when a medical chart is generated after measurement of the identification device 100a according to the second embodiment; FIG. 4 is a schematic diagram for explaining a method of estimating the type of tooth of a machined region; FIG. 11 is a schematic diagram for explaining an example of a learning data set 116a according to the second embodiment; FIG. FIG. 11 is a schematic diagram for explaining generation of a trained model 115a based on a learning data set 116a according to the second embodiment; 10 is a flowchart for explaining an example of tooth type learning processing executed by the identification device 100a according to the second embodiment. 9 is a flowchart for explaining an example of service providing processing executed by the identification device 100a according to the second embodiment; 10 is a flowchart for explaining an example of chart generation processing executed by the identification device 100a according to the second embodiment; FIG. 11 is a schematic diagram showing a functional configuration functioning during measurement of the scanner system 10b according to the third embodiment; FIG. 3 is a schematic diagram showing a functional configuration of an identification device 100b that functions when generating a chart after measurement; FIG. 11 is a schematic diagram for explaining generation of a trained model 115b based on a learning data set 116b according to the third embodiment; FIG. 12 is a flowchart for explaining an example of a lesion learning process executed by the identification device 100b according to the third embodiment; FIG. FIG. 12 is a flowchart for explaining an example of service providing processing executed by the identification device 100b according to the third embodiment; FIG. FIG. 12 is a flowchart for explaining an example of a medical record generation process executed by the identification device 100b according to the third embodiment; FIG. FIG. 11 is a schematic diagram for explaining an example of a learning data set according to a modification; It is a figure for demonstrating the data set for learning concerning a modification. FIG. 11 is a flowchart for explaining an example of service providing processing executed by an identification device 100c according to a modification; FIG.

Embodiments of the present disclosure will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are given the same reference numerals, and the description thereof will not be repeated.

<Embodiment 1>
[Overview of scanner system 10]
An overview of the scanner system 10 according to the first embodiment will be described with reference to FIGS. 1 to 3. FIG. FIG. 1 is a schematic diagram showing the overall configuration of a scanner system 10 according to the first embodiment. FIG. 2 is a schematic diagram showing an application example of the scanner system 10 according to the first embodiment. FIG. 3 is a schematic diagram showing the overall configuration of a system to which the scanner system 10 according to the first embodiment can be applied.

Referring to FIG. 1 , scanner system 10 according to the first embodiment includes three-dimensional scanner 200 , identification device 100 and display 300 . The three-dimensional scanner 200 acquires three-dimensional data 122 to be scanned by a built-in three-dimensional camera. Specifically, the three-dimensional scanner 200 scans the intraoral cavity to generate three-dimensional data 122 of positional information (longitudinal direction, lateral direction, height direction) of each of a plurality of points forming the shape of the intraoral cavity. ) are acquired using an optical sensor or the like. Alternatively, the three-dimensional scanner 200 scans a model created by taking an impression of the oral cavity, and obtains positional information (for example, longitudinal X-axis along the direction, Y-axis along the horizontal direction, and Z-axis along the height direction) are acquired using an optical sensor or the like. Based on the three-dimensional data 122 acquired by the three-dimensional scanner 200, the identification device 100 generates a two-dimensional image including a tooth image from an arbitrary viewpoint, or a three-dimensional image including a three-dimensional image such as a hologram. An image is generated and the generated three-dimensional image is displayed on the display 300 .

The scanner system 10 according to the first embodiment uses AI (Artificial Intelligence) possessed by the identification device 100 to automatically identify the processed region based on the three-dimensional data 122 acquired by the three-dimensional scanner 200. It is configured to execute a process that “Identifying a machined portion” means identifying whether or not a portion corresponding to the three-dimensional data 122 is a machined portion, and if it is a machined portion, identifying the type of the machined portion. Note that hereinafter, the process of identifying what kind of part the part corresponding to the three-dimensional data 122 is by the identification device 100 is also referred to as "identification process".

The term "processed site" means a site that has been processed to treat a living body site in the oral cavity. In addition, the “biological site in the oral cavity” is not limited to teeth, and may include gums. The processed site is not limited to the site where the prosthesis or artificial tooth is attached to the tooth, and may include the site where the implant body is attached to the jawbone. In addition, the processed part is not limited to the part where an artificially produced object (artificial object) such as an implant body or prosthesis is attached, but also the part where the natural tooth is shaved such as an abutment tooth or a cavity-formed tooth. can contain.

With reference to FIG. 2 , when user 1 scans the intraoral biological part of subject 2 using three-dimensional scanner 200 , three-dimensional data 122 is input to identification device 100 . The identification device 100 performs identification processing for identifying what kind of part the part corresponding to the input three-dimensional data 122 is, based on the input three-dimensional data 122 and an estimation model including a neural network. In the first embodiment, the identification device 100 identifies whether or not the part corresponding to the three-dimensional data 122 is a machined part, and if it is a machined part, executes identification processing to identify the type of the machined part. .

An "estimation model" includes a neural network and parameters used by the neural network. For example, the "estimation model" is parameter optimized using teacher data including three-dimensional data of the machined part, which is input data, and correct data indicating the type of the machined part corresponding to the three-dimensional data. .

Here, Table 1 shows types of machined parts that can be set as correct data. As shown in Table 1, the types of processed parts are broadly classified into, for example, "implant", "cavity forming tooth", "abutment tooth", and "prosthesis". An “implant” can be classified into an “implant body” corresponding to a tooth root portion, an “abutment” corresponding to an abutment portion, and the like. Also, "implant body" and "abutment" can be classified according to what kind of prosthesis is attached. "Abutment teeth" and "cavity-forming teeth" can be further classified according to what prosthesis is attached. A "prosthesis" can be further classified into inlays, onlays, crowns, bridges, veneers, dentures, and the like.

As shown in Table 1, the "processing site" can be arbitrarily subdivided, and can be classified as "abutment tooth", "cavitated tooth", "prosthesis", and "implant". Also, each of "abutment teeth", "cavitated teeth", "prostheses" and "implants" may be further subdivided and classified. In other words, the method of classifying processed parts can be arbitrarily designed.

The method of classifying processed parts can be arbitrarily designed. Therefore, the information indicating the types of machined parts included in the training data may include the types classified by a specific classification method, as well as the types classified by another classification method. For example, the information indicating the type of machined portion may include "abutment tooth" and "inlay-applied abutment tooth".

Also, the processed region can be broadly classified into a processed region that is being treated and a processed region that has been completely treated. Therefore, in addition to information indicating the type of processed region, the correct data may include information that can specify whether the treatment is "under treatment" or "treated".

In addition, the content of treatment applied to the tooth differs depending on the type of processed portion. Therefore, the correct answer data may include information indicating "treatment content" in addition to information indicating the type of processed region.

Also, the teacher data can include three-dimensional data of a living body part (for example, natural tooth, gum) that is not a machined part, and correct data indicating that the three-dimensional data is not a machined part.

Note that processing for learning such an estimation model is also referred to as “learning processing”. In addition, the estimation model optimized by the learning process is particularly called a "learned model". That is, pre-learning estimation models and trained estimation models are collectively referred to as "estimation models", while trained estimation models are also particularly referred to as "learned models".

A scene to which the scanner system 10 can be applied will be described with reference to FIG. By using the scanner system 10, the user 1 can acquire three-dimensional shape data (hereinafter also referred to as "three-dimensional data") including the teeth that the subject 2 has. It should be noted that the “user” is any person who uses the scanner system 10, such as an operator such as a dentist, a dental assistant, a teacher or student at a dental college, a dental technician, an engineer at a manufacturer, or an operator at a manufacturing plant. may be The “subject” may be any person to whom the scanner system 10 is applied, such as a patient at a dental clinic or a subject at a dental college.

When identification processing is executed by identification device 100 using the trained model, the identification result is output to display 300 .

Display 300 displays at least one of images, letters, numbers, icons, and symbols corresponding to the identification result.

Based on the three-dimensional data 122 and the identification result, the identification device 100 generates a three-dimensional intraoral image reflecting the identification result and outputs the image to the display 300 . For example, the identification device 100 reflects the identification result in the three-dimensional image by displaying the processed region on the display 300 in a color corresponding to the type of the processed region.

Three-dimensional scanners 200 have been used to digitally design prostheses and the like. The identification device 100 can provide the three-dimensional scanner 200 with a new application of identifying the type of the processed region by identifying the type of the processed region in the oral cavity from the three-dimensional data.

For example, by identifying the type of processed part for a living body part in the oral cavity and outputting the identification result, the user 1 can read the medical record while comparing the identification result with the diagnosis result determined while scanning. can be entered. In addition, even if the user 1 confirms the processed site again because the diagnosis result and the identification result are different from each other, the user 1 can check the recognition result displayed on the display 300 to obtain a rough estimate of the processed site in the oral cavity. position can be determined. As a result, the time during which the subject 2 has his/her mouth open can be shortened, and the burden on the subject 2 can be reduced.

Also, when the user 1 performs medical examination after scanning, information such as the type of processed part that assists the medical examination can be obtained.

In addition, the user 1 can explain the diagnosis result to the subject 2 while causing the display 300 to display the three-dimensional image reflecting the identification result. This allows the subject 2 to objectively check the condition of his or her teeth.

1 and 3, the identification result by identification device 100 may be output as scan information to server device 500 located in the dental laboratory and management center along with the three-dimensional data used in identification processing. good.

For example, as shown in FIG. 3, scanner system 10 is located at each of a plurality of locals AC. For example, local A and local B are dental clinics, and in the dental clinics, an operator or a dental assistant who is user 1 uses scanner system 10 to at least scan teeth and gums of a patient who is subject 2. Acquire 3D data of the oral cavity including Also, the local C is a dental college, and at the dental college, a teacher or a student who is the user 1 acquires intraoral three-dimensional data of a subject who is the subject 2 . The scan information (three-dimensional data, identification results) acquired by each of the locals A to C is output via the network 5 to the server device 500 located in the dental laboratory and management center as the local D.

If the identification result by the identification device 100 and the medical examination result by the user 1 do not match, the user 1 may correct the identification result by inputting it into a chart. In such a case, the scanner systems 10 arranged in each of the locals A to C may include both the pre-correction identification result and the post-correction identification result in the scan information and output it to the server device 500. good. Further, the scanner systems 10 arranged in each of the locals A to C may include the corrected identification result in the scan information and output it to the server apparatus 500 without including the identification result before the correction.

In the dental laboratory, a dental technician or the like creates a prosthesis or the like to replace the missing tooth portion of the subject 2 based on the scan information obtained from each of the locals A to C. In the management center, the server device 500 accumulates and stores the scan information obtained from each of the locals A to C, and holds it as big data.

Note that the server device 500 is not limited to being arranged in a management center different from the local one in the dental clinic, and may be arranged locally. For example, the server apparatus 500 may be arranged within any one of the locals A to C. FIG. Also, a plurality of identification devices 100 may be arranged within one local, and a server device 500 capable of communicating with the plurality of identification devices 100 may be arranged within the one local. Moreover, the server device 500 may be implemented in the form of a cloud service.

In the dental laboratory, scan information is aggregated from various locations, such as local A-C. For this reason, the scan information held at the dental laboratory may be transmitted to the management center via the network 5, or may be sent to a removable disk 550 such as a CD (Compact Disc) and USB (Universal Serial Bus) memory. may be sent to the management center via

The scan information may also be sent to the management center from each of the local A to C via the removable disk 550 without going through the network 5 . Also, between each of the locals A to C, scan information may be sent to each other via the network 5 or the removable disk 550 .

Each identification device 100 of local A to C has its own estimated model, and uses its own estimated model to identify the type of machined part during the identification process. Each local identification device 100 generates a trained model by learning its own estimation model through its own learning process. Furthermore, in the present embodiment, server device 500 also holds an estimation model. The server device 500 generates a trained model by learning an estimated model through learning processing using scan information acquired from the identification device 100 of each of the locals A to C and the dental laboratory. The trained model is distributed to the identification device 100 .

Further, when the identification result is corrected by the user 1, the identification devices 100 of the local A to C may re-learn their own estimation models using the corrected identification result as correct data.

In the present embodiment, both local identification devices 100 and server devices 500 of local A to C execute learning processing, but only local identification devices 100 of local A to C execute learning processing. Alternatively, only the server device 500 may execute the learning process. In the case where only the server device 500 executes the learning process, the estimation model (learned model) held by the identification devices 100 of the local A to C is shared by the identification devices 100 of the local A to C. be done.

Also, the server device 500 may have the function of identification processing in the identification device 100 . For example, each of the locals A to C transmits the acquired three-dimensional data to the server device 500, and the server device 500 determines the type of machined region in each based on the three-dimensional data received from each of the locals A to C. An identification result may be calculated. Then, server device 500 may transmit each identification result to each of locals A to C, and each of locals A to C may output the estimation result received from server device 500 to a display or the like. In this way, each local A to C and the server device 500 may be configured in the form of a cloud service. In this way, as long as server device 500 holds an estimation model (learned model), each of locals A to C can obtain an estimation result without holding an estimation model (learned model). can.

As described above, according to the scanner system 10 according to the present embodiment, the AI of the identification device 100 is used to identify the type of processed region based on the three-dimensional data acquired by the three-dimensional scanner 200 .

Although the identification device 100 of the scanner system 10 arranged in each of the locals A to C will be described below, the server device 500 is assumed to be capable of executing the same processing that the identification device 100 can execute. The description of the processing executed by is omitted. Similarly, in the second and third embodiments, the server device 500 can similarly execute processing that can be executed by each of the identification device 100a and the identification device 100b.

[Hardware Configuration of Identification Device 100]
An example of the hardware configuration of the identification device 100 according to the first embodiment will be described with reference to FIG. FIG. 4 is a schematic diagram showing the hardware configuration of the identification device 100 according to the first embodiment. The identification device 100 may be realized, for example, by a general-purpose computer or a computer dedicated to the scanner system 10 .

As shown in FIG. 4, the identification device 100 includes, as main hardware elements, a scanner interface 102, a display interface 103, a peripheral device interface 105, a media reader 107, a PC display 108, a memory 109, It includes a storage 110 and an arithmetic unit 130 .

The scanner interface 102 is an interface for connecting the three-dimensional scanner 200 and realizes input/output of data between the identification device 100 and the three-dimensional scanner 200 .

The display interface 103 is an interface for connecting the display 300 and realizes input/output of data between the identification device 100 and the display 300 . Display 300 is configured by, for example, an LCD (Liquid Crystal Display) or an organic ELD (Electroluminescence) display.

The peripheral device interface 105 is an interface for connecting peripheral devices such as the keyboard 601 and the mouse 602, and realizes input/output of data between the identification device 100 and the peripheral devices.

The media reader 107 reads various data such as scan information stored in the removable disk 550 .

The PC display 108 is a dedicated display for the identification device 100 . PC display 108 is configured by, for example, an LCD or an organic EL display. Although the PC display 108 is separate from the display 300 in the first embodiment, it may be shared with the display 300 .

The memory 109 provides a storage area for temporarily storing program codes, work memory, etc. when the arithmetic unit 130 executes an arbitrary program. The memory 109 is composed of, for example, a volatile memory device such as a DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory).

The storage 110 provides storage areas for storing various data necessary for identification processing, learning processing, and the like. The storage 110 is composed of a non-volatile memory device such as a hard disk or SSD (Solid State Drive), for example.

Storage 110 includes scan information 112, processed species estimation model 114 (learned model 115), learning data set 116, color classification data 118, identification program 120, learning program 121, OS (Operating System) 127 is stored.

The scan information 112 includes three-dimensional data 122 acquired by the three-dimensional scanner 200 and an identification result 124 indicating the type of machined region obtained by identification processing executed based on the three-dimensional data 122 . The identification result 124 is stored in the storage 110 in association with the three-dimensional data 122 used for identification processing. The learning data set 116 is a group of learning data used for learning processing of the processed species estimation model 114 . The color classification data 118 is data used for generating the learning data set 116 and for learning processing. The identification program 120 is a program for executing identification processing. The learning program 121 is a program for executing the learning process of the processed species estimation model 114, and part of it also includes a program for executing the identification process.

The arithmetic unit 130 is an arithmetic entity that executes various types of processing such as identification processing and learning processing by executing various programs, and is an example of a computer. Arithmetic device 130 includes, for example, a CPU (Central Processing Unit) 132, an FPGA (Field-Programmable Gate Array) 134, a GPU (Graphics Processing Unit) 136, and the like.

[Identification processing by identification device 100]
An example of identification processing by the identification device 100 according to the first embodiment will be described with reference to FIGS. 5 and 6. FIG. FIG. 5 is a schematic diagram showing the functional configuration of the identification device 100 according to the first embodiment. FIG. 6 is a schematic diagram for explaining identification processing by the identification device 100 according to the first embodiment.

As shown in FIG. 5, the identification device 100 has an input unit 1102, a processing type identification unit 1132, and an output unit 1103 as functional units involved in identification processing. Each of these functions is realized by executing the OS 127 and the identification program 120 by the arithmetic device 130 of the identification device 100 .

Three-dimensional data acquired by the three-dimensional scanner 200 is input to the input unit 1102 . Based on the three-dimensional data input to the input unit 1102, the processing type identification unit 1132 uses the processing type estimation model 114 (learned model 115) to identify the region corresponding to the input three-dimensional data as the processing region. Identification processing is performed to identify whether or not the machined portion is a machined portion, and to identify the type of the machined portion if the machined portion.

The processed species estimation model 114 includes a first neural network (NNW) 1142 and parameters 1144 used by the first NNW 1142 . Parameters 1144 include weighting factors used in calculations by first NNW 1142 and decision values used to determine identity. Output unit 1103 outputs the result of identification by processing type identification unit 1132 to display 300 .

Here, as shown in FIG. 6, the three-dimensional data input to the input unit 1102 includes three-dimensional position information at each point that constitutes the shape of the oral cavity, and and color information. Location information is used in the identification processing according to the first embodiment. The position information includes the coordinates of the absolute position in three dimensions with reference to a predetermined position. For example, the position information includes absolute position coordinates on each of the X-axis, Y-axis, and Z-axis, with a predetermined position in the oral cavity as the origin. The origin may be, for example, a predetermined measurement point at the start of measurement. Note that the position information is not limited to coordinates of a three-dimensional absolute position based on a predetermined position, and may include, for example, coordinates of a three-dimensional relative position indicating a distance from an adjacent point.

The user 1 can identify the processed part at a glance if it is in the middle of treatment, such as an implant body or a cavity formed in a tooth. However, in the field of dentistry, from the viewpoint of ensuring aesthetics, the development of machined parts with excellent aesthetics comparable to natural teeth is progressing. Therefore, it is gradually becoming difficult for the user 1 to visually identify the processed site after the treatment is completed.

Although it is difficult for the human eye to distinguish between an artificial material and a natural material, there may be slight differences in surface unevenness and overall shape. In addition, when a prosthesis is placed over a tooth, there may be a difference in the shape of the boundary between the gingiva and the prosthesis compared to when the prosthesis is not placed.

Based on the three-dimensional data in which the characteristic shape and size of the machined part are digitized, the identification device 100 uses the machining type estimation model 114 to determine whether the part corresponding to the three-dimensional data is a machined part. is identified, and if it is a machined part, the type of machined part is identified.

In this way, by using AI, it is possible to find out the features of a workpiece that are difficult to identify with the human eye. Then, the machined part can be accurately identified without relying on the knowledge of the user 1, which varies from person to person. In addition, by identifying the type of processed part, it is possible to assist the user 1 in entering a medical record, thereby facilitating the examination at the first visit.

In addition, the prosthesis is classified into a plurality of types according to the covering range, covering method, and the like. The type of prosthesis to be placed over the cavity portion and the abutment tooth differs depending on the shape of the cavity portion and the abutment tooth, and the content of treatment to be performed may differ. In addition, the course of treatment changes depending on the type of prosthesis that is put on. However, in order to identify the type of prosthesis, a high level of knowledge of the user 1 is required. Moreover, the user 1 needs to observe and identify the narrow oral cavity, which increases the burden on the subject 2 .

A finer identification of the type of work site, such as the type of prosthesis fitted and the type of prosthesis attached to the cavity or abutment, allows for a more detailed examination.

As shown in FIG. 5 , processing species estimation model 114 includes first NNW 1142 . In the first NNW 1142, the value of the position information included in the three-dimensional data input to the input unit 1102 is input to the input layer. In the first NNW 1142, for example, the intermediate layer multiplies the value of the input positional information by a weighting factor or adds a predetermined bias, and performs calculation using a predetermined function. is compared with the judgment value. Then, in the first NNW 1142, the result of the calculation and determination is output from the output layer as the identification result. Regarding the calculation and determination by the first NNW 1142, it is possible to identify whether or not the part corresponding to the three-dimensional data is a machined part based on the three-dimensional data, and to identify the type of machined part if it is a machined part. Either method may be used.

In the first NNW 1142 of the processing species estimation model 114, processing by deep learning is performed because the intermediate layer has a multi-layer structure. In the first embodiment, examples of the identification program 120 that performs identification processing specialized for three-dimensional images include VoxNet, 3DShapeNets, Multi-View CNN, RotationNet, OctNet, FusionNet, PointNet, PointNet++, SSCNet, and MarrNet. used, but other programs may also be used. Also, an existing one may be applied to the mechanism of the first NNW 1142 .

In such a configuration, when three-dimensional data corresponding to a three-dimensional image including a plurality of teeth is input, the identification device 100 identifies the feature of the processed region based on the three-dimensional data, and calculates the processed species estimation model 114 based on the three-dimensional data. can be extracted using the first NNW 1142 of , and the type of machined portion can be identified based on the extracted features. Note that the identification device 100 uses the first NNW 1142 to extract the features of the unprocessed living body part (for example, natural teeth and gums) in addition to the features of the processed part, and based on the extraction results, You can identify whether there is

In the first embodiment, the identification device 100 uses the first NNW 1142 to extract features of the machined part, and identifies the type of the machined part from the extracted features.

[Generation of learning data]
An example of generating learning data will be described with reference to FIGS. 7 and 8. FIG. FIG. 7 is a schematic diagram for explaining generation of learning data according to the first embodiment. FIG. 8 is a schematic diagram for explaining an example of learning data according to the first embodiment;

As shown in FIG. 7, first, three-dimensional data is acquired by the three-dimensional scanner 200 (STEP 1). The three-dimensional data acquired by the three-dimensional scanner 200 includes three-dimensional position information at each point constituting the intraoral shape corresponding to the three-dimensional data and color information (RGB values) at each point. be When a 3D image is generated based on the 3D data acquired by the 3D scanner 200, a 3D image including actual colored teeth is generated as shown in FIG. 7(a).

Next, noise removal processing is performed in preparation for color-coding processing for each tooth, which will be described later. For example, in Embodiment 1, a three-dimensional image corresponding to three-dimensional data is grayscaled (STEP 2). The grayscaling of the three-dimensional image is performed by the user 1 (in this case, an engineer of the manufacturer who generates the learning data, a worker of the manufacturing plant, or the like). When the three-dimensional image is grayscaled, a grayscaled three-dimensional image is generated as shown in FIG. 7(b). In addition, according to the grayscaling of the three-dimensional image, the color information (RGB values) at each point corresponding to the three-dimensional data is changed to a value corresponding to the grayscale.

Next, a predetermined color is applied to each part included in the three-dimensional image corresponding to the three-dimensional data, so that each part is color-coded according to the type of the processed part (STEP 3). The color to be applied is assigned in advance to each part. The allocation details are stored in storage 110 as color classification data 118 . Also, in the example shown in FIG. 7, it is assumed that white, which is not assigned to any type of processed portion, is applied to the unprocessed portion. The color classification data 118 may include information on colors assigned to unprocessed regions.

Color application to each part is performed by the user 1 (particularly, a dentist, a dental technician, or other person who has knowledge of dentistry). Specifically, the user 1 identifies whether or not each part included in the three-dimensional image is a machined part based on his/her own knowledge, and if it is a machined part, identifies the type of the machined part, and identifies the identified part. The corresponding color is specified while referring to the color classification data 118, and the specified color is applied to the image of the site.

For example, when the user 1 identifies that the site included in the three-dimensional image is a living body site that has not been processed, the user 1 paints the image of the site in white. Further, when the user 1 identifies that the site included in the three-dimensional image is an implant, the user 1 applies color a to the image of the site. Further, when the user 1 identifies that the site included in the three-dimensional image is a prosthesis, the user 1 applies the color b to the image of the site. When a color is applied to each tooth and gum included in the three-dimensional image, a three-dimensional image in which a predetermined color is applied to each part as shown in FIGS. 7(c) and 8 is obtained. generated. For easy understanding, the colors a and b are hatched on the drawing.

Further, the color information (RGB values) at each point of the tooth corresponding to the three-dimensional data is changed to a value corresponding to the applied color according to the color coding of each part. Thereby, predetermined color information (RGB values) is associated with each point corresponding to the three-dimensional data.

When predetermined color information is associated with each part, the three-dimensional data includes position information and color information corresponding to the applied color. Adopted as training data. That is, in the learning data according to the first embodiment, the position information referred to in the identification process is associated with color information indicating to which part the position information corresponds (labeled). ). A plurality of such learning data are generated, and a collection of the generated learning data is held in the identification device 100 as a learning data set 116 .

In the first embodiment, the user 1 manually applies colors to each part included in the three-dimensional image based on his/her own knowledge. Color may be applied manually to each part included in the three-dimensional image. It is also possible to supplement some of the work with software.

Further, in the first embodiment, grayscaling is performed as noise removal processing, but noise removal processing may not be performed, and may be realized by processing different from grayscaling.

Although color information is associated with each piece of position information as identification information that is correct data, identification information is not limited to color information. For example, learning data may be created by inputting correct data by applying a color and then converting the input correct data into identification information different from color information. In this case, it is preferable to use information having a data amount smaller than the RGB values as the correct data. As a result, the user 1 can be made to associate the correct data with the intuitively understandable work of applying the color, and the data amount of the learning data can be reduced.

Note that the method of generating the correct answer data shown in FIGS. 7 and 8 is an example. For example, a color corresponding to the type of the processed portion may be applied only to the portion identified as the processed portion, and the three-dimensional data of the portion to which the color is not applied may not be used as the correct data.

In addition, in the example shown in FIG. 8, the processed parts are classified according to the major classifications of "implant", "cavity", "abutment tooth", and "prosthesis". may be sorted according to

[Learning processing of identification device 100]
A learning process performed by the identification device 100 will be described with reference to FIG. 9 . FIG. 9 is a flowchart for explaining an example of processing type learning processing executed by the identification device 100 according to the first embodiment. Each step shown in FIG. 9 is realized by executing the OS 127 and the learning program 121 by the arithmetic device 130 of the identification device 100 .

As shown in FIG. 9, the identification device 100 selects learning data to be used for learning from the learning data set 116 (S1). Note that the identification device 100 may select one learning data or a plurality of learning data from the learning data set 116 . Further, the identification device 100 is not limited to automatically selecting learning data, and may use learning data selected by the user 1 for learning processing.

The identification device 100 inputs the position information of the three-dimensional data included in the selected learning data to the processed species estimation model 114 (S2). At this time, the identification device 100 does not input the correct data labeled with the three-dimensional data. The identification device 100 extracts features of the three-dimensional data using the processing type estimation model 114, and executes identification processing for identifying the type of the processed portion of the portion corresponding to the three-dimensional data (S3).

The identification device 100 updates the parameter 1144 of the processed species estimation model 114 based on the error between the identification result of the identification process and the correct data corresponding to the learning data used in the learning process (S4).

For example, as a result of identification based on the input position information, the identification device 100 estimates color information corresponding to the part indicated by the input position information. The identification device 100 compares the color information (correct data) corresponding to the part indicated by the input position information included in the learning data with the color information estimated by itself, and if they match, the processed species estimation model 114 while maintaining the parameter 1144 of the processing species estimation model 114 so that the two match if the answer is incorrect.

Next, the identification device 100 determines whether or not it has learned based on all the learning data (S5). If the identification device 100 has not learned based on all the learning data (NO in S5), the process returns to S1.

On the other hand, if the identification device 100 has learned based on all the learning data (YES in S5), the identification device 100 stores the learned processed species estimation model 114 as the learned model 115 (S6), and ends this process.

In this way, the identification device 100 uses the color information associated with the three-dimensional data included in the learning data as correct data, and based on the identification result of the identification processing using the three-dimensional data, the processing species estimation model 114 is generated. By learning, a trained model 115 is generated.

[Service provision processing of identification device 100]
A service providing process executed by the identification device 100 will be described with reference to FIG. FIG. 10 is a flowchart for explaining an example of service providing processing executed by the identification device 100 according to the first embodiment. Each step shown in FIG. 10 is realized by executing the OS 127 and the identification program 120 by the arithmetic device 130 of the identification device 100 . By executing the service providing process, the identification result is associated with the three-dimensional data acquired by the three-dimensional scanner 200 .

As shown in FIG. 10, the identification device 100 determines whether three-dimensional data corresponding to a predetermined point has been input (S11). For example, if the storage 110 stores three-dimensional data that is not associated with an identification result, the identification device 100 determines that three-dimensional data corresponding to a predetermined point has been input.

When the three-dimensional data is sent from the three-dimensional scanner 200 , the identification device 100 stores the three-dimensional data in the storage 110 . The service providing process is executed in an arbitrary calling cycle, and is repeatedly executed until all three-dimensional data stored in the storage 110 are associated with identification results. When the identification results are associated with all the three-dimensional data stored in the storage 110, the identification device 100 determines that the three-dimensional data corresponding to the predetermined point has not been input (NO in S11). ) to end the service providing process.

On the other hand, when three-dimensional data corresponding to a predetermined point is input (YES in S11), the identification device 100 inputs the three-dimensional data (position information) to the learned model 115, and uses the learned model 115. identification processing for identifying the type of the machined portion is executed (S12).

The identification device 100 associates the identification result obtained by the identification process with the three-dimensional data input in S11 (S13). Specifically, the identification device 100 associates color information indicating identification results (color a to color d or color information indicating white) with the position information included in the three-dimensional data. As a result, the point corresponding to the input three-dimensional data is associated with the identification result indicating the state of the portion corresponding to the point (whether or not it has been processed, the details of processing).

The identification device 100 outputs the association result to the outside (for example, the display 300) (S14). Specifically, the display 300 displays the result of the association of the points corresponding to the input three-dimensional data. After that, the identification device 100 terminates this process.

[Example of display output to display 300 during measurement]
An example of a display output to display 300 during measurement will be described with reference to FIGS. 11 and 12. FIG. FIG. 11 is a diagram showing an output example during measurement. FIG. 12 is a diagram showing a modified example of an output during measurement.

The identification device 100 executes the service providing process, and at the timing when the three-dimensional data corresponding to a predetermined point is input, associates the three-dimensional data based on the identification result. The user 1 scans in order while gradually moving the three-dimensional scanner 200 in the oral cavity. Therefore, when the three-dimensional scanner 200 is moved to acquire new three-dimensional data, the identification device 100 associates the identification result with each of the plurality of points corresponding to each acquired three-dimensional data. The three-dimensional data associated with the identification result is output to the display 300 in a manner showing the associated identification result.

For example, as shown in FIG. 11(a), when an unprocessed natural tooth is scanned, each of a plurality of points corresponding to each three-dimensional data obtained by scanning is displayed in white. In addition, in FIG. 11A, for convenience, points output in white are represented by triangles.

After that, when the living body part including the processed part is newly scanned, as shown in FIG. The original data are indicated by points of color a (dots of black circles in the figure), while the three-dimensional data corresponding to unprocessed parts are indicated by white points (dots of triangles in the figure). be done. Although not shown in FIG. 11, three-dimensional data corresponding to other types of processed parts such as prostheses, cavity-forming teeth, and abutment teeth are displayed with dots colored according to each type. .

Although the identification device 100 displays a plurality of points corresponding to three-dimensional data on the display 300 as points, the obtained three-dimensional data is displayed on the display 300 using a polygon mesh. good too. A polygon mesh is a method of displaying an object by arranging polygons such as triangles and quadrilaterals on the screen.

Specifically, as shown in FIG. 12, the identification device 100 replaces a plurality of points corresponding to the input three-dimensional data with polygons, and displays the results of associating the identification results with the plurality of points. 300 may be displayed.

In this way, the identification results for the acquired three-dimensional data are sequentially displayed on the display 300 at the timing when the three-dimensional data is acquired, so that the user 1 can respond to the three-dimensional data input at the input timing. You can check the type of part to be treated. In addition, since the identification device 100 displays on the display 300 in a display mode according to the type of the processed part, it is possible to show the user 1 the boundary between the processed parts in an easy-to-understand manner.

Note that at the beginning of intraoral scanning by the three-dimensional scanner 200, the number of obtained three-dimensional data is small, so the identification device 100 cannot output identification results based on the three-dimensional data. Therefore, the identification device 100 may display an image (point or polygon) corresponding to the three-dimensional data on the display 300 in a manner indicating that no association has been established, that is, no identification result has been obtained. Then, when the scanning of the oral cavity by the three-dimensional scanner 200 progresses and the number of obtained three-dimensional data increases, the service providing process may be executed again to try to obtain the identification result. . In this case, when the identification result is obtained, the identification device 100 switches the display mode of the image corresponding to the three-dimensional data that has already been displayed to a mode according to the identification result.

As described above, the identification apparatus 100 according to the first embodiment uses AI to find features of a workpiece that are difficult to identify with the human eye, and to detect the accuracy without relying on the user's knowledge, which varies from person to person. The type of machined part can be identified well.

<Embodiment 2>
An identification device 100a according to the second embodiment will be described with reference to FIGS. 13 to 20. FIG. In the first embodiment, the user 1 generates a chart by referring to the identification result. The identification device 100a according to the second embodiment may automatically generate a medical record according to the identification result.

Compared with the identification device 100, the identification device 100a identifies the type of tooth in addition to identifying the type of machined region. Accordingly, the identification device 100a estimates the type of tooth corresponding to the machined portion. Further, the identification device 100a estimates the content of the treatment given to the processed region based on the identification result indicating the type of the processed region. In this way, the identification device 100a generates the chart of the subject 2 by estimating the type of tooth corresponding to the processed region and the details of treatment given to the processed region.

The "types of teeth" are upper right central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars, upper left side. central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars, lower right central incisors, lateral incisors, canines, 1st premolar, 2nd premolar, 1st molar, 2nd molar, and 3rd molar, mandibular left central incisor, lateral incisor, canine, 1st premolar, 2nd premolar, 3rd molar It refers to each tooth type such as 1 molar, 2nd molar, and 3rd molar.

[Configuration that works during measurement]
FIG. 13 is a schematic diagram showing a functional configuration functioning during measurement of the scanner system 10a according to the second embodiment. Referring to FIG. 13, a scanner system 10a differs from the scanner system 10 according to the first embodiment in that an identification device 100a is provided instead of the identification device 100. FIG. Further, the identification device 100a is different from the identification device 100 according to the first embodiment in that the identification device 100a further includes a tooth type identification unit 1134 and uses the profile data 119 for identification processing in addition to the three-dimensional data.

The profile data 119 is information indicating the profile of the subject 2, such as age, sex, race, height, weight, and place of residence. Note that the profile data 119 is not necessarily required information, but is information that is used to improve accuracy when identifying the type of tooth. Also, the profile data 119 is information obtained from the subject 2 at the first visit, and is included in the medical record information.

The tooth type identification unit 1134 identifies the tooth type based on the three-dimensional data input to the input unit 1102 and the profile data 119, using the tooth type estimation model 114a (learned model 115a). The tooth type estimation model 114a includes a second NNW 1142a and parameters 1144a used by the second NNW 1142a. Note that the second NNW 1142a and the parameters 1144a are common to the first NNW 1142 and the parameters 1144 described in the first embodiment, so description thereof will be omitted.

Since the processed part is a part obtained by processing a living body part, it differs from a natural tooth in features such as shape. Therefore, when the tooth type identification unit 1134 performs identification processing for identifying the type of tooth based on three-dimensional data obtained by scanning the processed region, the type of tooth may not be identified. In the present embodiment, when the tooth type identification unit 1134 cannot identify the type of tooth, the identification device 100a determines that the portion corresponding to the three-dimensional data input to the input unit 1102 is the processed portion. The species identification unit 1132 executes identification processing for identifying the type of processed portion.

Output unit 1103 outputs identification results from tooth type identification unit 1134 and processing type identification unit 1132 to display 300 .

In addition, the storage 110 provided in the identification device 100a stores three-dimensional data 122, an identification result 124a indicating the type of tooth in the region corresponding to the three-dimensional data 122, and the type of the processed region corresponding to the three-dimensional data 122. The identification result 124 is stored.

[Configuration that functions after measurement]
Functions related to chart generation will be described with reference to FIGS. 14 and 15. FIG. FIG. 14 is a schematic diagram showing a functional configuration that functions when generating a chart after measurement of the identification device 100a according to the second embodiment. 15A and 15B are schematic diagrams for explaining a method of estimating the type of tooth of the machined portion.

Referring to FIG. 14, identification device 100a includes treatment estimation unit 1152, tooth type estimation unit 1154, and chart generation unit 1156 as functions for generating a chart after measurement.

Based on the identification result 124 indicating the type of the processed part, the treatment estimation unit 1152 estimates the content of the treatment given to the processed part.

The processing type identification unit 1132 identifies the type of processing region of the region corresponding to the three-dimensional data. Processing of a living body part is performed for treatment. Therefore, there is a correspondence relationship between the type of processed region and the content of treatment. Table 2 shows the relationship between the type of processed site and the content of treatment.

Referring to Table 2, the "processed site" can be classified into a processed site in the middle of treatment and a processed site that has already been treated. For example, "implant", "cavity-forming tooth" and "abutment tooth" are working sites under treatment, and "prosthesis" are working sites that have been treated. Since the type of processed part and the type of treatment content correspond to each other, the treatment estimating unit 1152 can estimate the content of treatment performed on the subject 2 based on the identification result 124 indicating the type of the processed part. . In addition, the processed region is classified into a processed region in the middle of treatment and a processed region that has already been treated. Therefore, the treatment estimation unit 1152 can estimate whether the treatment is in progress or has been completed based on the identification result 124 indicating the type of the processed part.

The treatment estimating unit 1152 may refer to a table as shown in Table 2 to estimate the content of treatment, and performs deep learning using the identification result 124 indicating the type of processed part as an input value to obtain the identification result 124 . You may presume the contents of treatment corresponding to. When the treatment content is estimated by deep learning, the treatment estimation unit 1152 uses a neural network that optimizes parameters using training data that associates treatment content as correct data with respect to the type of processed part. to estimate the treatment content.

The tooth type estimation unit 1154 estimates the type of tooth corresponding to the machining site. The tooth type estimating unit 1154 estimates the type of tooth corresponding to the machining site from the type of the tooth corresponding to the machining site or the adjacent tooth. The "tooth corresponding to the processed part" is not limited to the tooth to which the prosthesis is attached, and means the tooth of the original natural tooth when the artificial tooth is attached instead of the natural tooth. In addition, the "tooth corresponding to the processed site" means, when an implant is attached, the tooth that existed before the implant was attached.

The chart generation unit 1156 generates the chart 150 based on the tooth type corresponding to the processed region estimated by the tooth type estimation unit 1154 and the treatment content estimated by the treatment estimation unit 1152 .

The medical record 150 includes a profile display area 152 showing basic information of the subject 2 specified from the profile data 119, a tooth type display area 154 showing the type of tooth corresponding to the processed part, and information about the displayed tooth. It includes a comment display area 156 showing the details of the treatment performed and a dental formula display area 158 showing the dental formula. Note that the medical record generation unit 1156 displays a two-dimensional image of the tooth including the processed part viewed from an arbitrary viewpoint based on the three-dimensional data instead of or together with the tooth formula in the tooth formula display area 158. may be displayed.

A method for estimating the type of tooth corresponding to the machined portion will be specifically described with reference to FIG. First, three-dimensional data is acquired by the three-dimensional scanner 200 (STEP 1). Next, the identification device 100a identifies the type of tooth through identification processing by the tooth type identification unit 1134 (STEP 2). For example, in the example shown in STEP2 of FIG. 15, the identification device 100a identifies the type of each tooth located on the right side of the lower jaw and the right side of the upper jaw. Specifically, the identification device 100a performs identification processing to identify the second molar, first molar, second premolar, and first premolar on the right side of the lower jaw, and the second molar, first molar, and second molar on the right side of the upper jaw. Identify the second premolar and the first premolar, respectively.

Here, when there is a processed site in the oral cavity, the tooth type identification unit 1134 of the identification device 100a may not be able to identify the type of tooth corresponding to the processed site by the identification processing. However, the tooth type identifying unit 1134 can estimate the type of adjacent teeth adjacent to the tooth corresponding to the processed region and opposing teeth facing the tooth corresponding to the processed region through identification processing. The tooth type estimating section 1154 estimates the type of tooth corresponding to the processed portion based on the types of adjacent teeth and opposing teeth (STEP 3).

For example, in the example shown in STEP 3 of FIG. 15, the tooth type identification unit 1134 cannot identify the type of tooth corresponding to the processed region between the second molar and the second premolar on the right side of the upper jaw. However, the tooth type identifying section 1134 identifies at least the second molar and the second premolar on the right side of the upper jaw. Therefore, the tooth type estimation unit 1154 can estimate that the tooth between them is the upper right first molar. Alternatively, the tooth type identification unit 1134 identifies at least the first molar on the right side of the mandible facing the tooth corresponding to the processed region. Therefore, the tooth type estimation unit 1154 can estimate that the tooth facing the lower right first molar is the upper right first molar.

In addition, when the type of tooth type estimation unit 1154 is a prosthesis attached to a part of the tooth, such as an inlay, an onlay, or a crown, the teeth around the three-dimensional data identified as the processed part The type of tooth corresponding to the machining site may be identified based on the three-dimensional data in which the type of tooth is identified. For example, the tooth type corresponding to the three-dimensional data of the machined portion is estimated to be the same as the tooth type associated with the three-dimensional data of the surrounding teeth of the three-dimensional data of the machined portion.

[Generation of tooth type estimation model 114a (learned model 115a)]
The learning process for generating the trained model 115a will be described with reference to FIGS. 16 to 18. FIG. FIG. 16 is a schematic diagram for explaining an example of the learning data set 116a according to the second embodiment. FIG. 17 is a schematic diagram for explaining generation of the trained model 115a based on the learning data set 116a according to the second embodiment. FIG. 18 is a flowchart for explaining an example of tooth type learning processing executed by the identification device 100a according to the second embodiment. Note that the learning process for generating the learned model 115 of the processed species estimation model 114 is common to the learning process (see FIG. 9) for generating the learned model 115 described in the first embodiment. omitted.

When generating learning data for identifying tooth types, user 1 (particularly, a person with dental knowledge such as a dentist or a dental technician) confirms the three-dimensional image and identifies the tooth types. is identified, and a color corresponding to the type of the identified tooth is applied to the three-dimensional image, thereby associating the position information with the identification information. The color to be applied is assigned in advance for each tooth type. The allocation contents are stored in the storage 110 as the color classification data 118a.

For example, when the user 1 identifies that the tooth included in the three-dimensional image is the second molar, the user 1 paints the image of the tooth in red. Further, when the tooth included in the three-dimensional image is identified as the first molar, the image of the tooth is painted green. When each tooth included in the three-dimensional image is painted with a predetermined color, a three-dimensional image is generated in which each tooth is painted with a predetermined color as shown in FIG. For easy understanding, each color is represented by hatching on the drawing.

When predetermined color information (identification information) is associated with each tooth, the three-dimensional data includes position information and color information (identification information) corresponding to the applied color. , such three-dimensional data is employed as learning data. That is, in the learning data of the tooth type estimation model 114a according to the second embodiment, color information corresponding to the tooth type is associated (labeled) with the position information referred to in the identification process. Further, color information is associated with the three-dimensional data so that each range of a plurality of teeth corresponding to the three-dimensional data can be specified. Specifically, the same color information is associated with each piece of position information corresponding to each tooth. A collection of such learning data is held in the identification device 100a as a learning data set 116a.

In the second embodiment, the user 1 manually applies the color to each tooth included in the three-dimensional image based on his/her own knowledge, but it is also possible to supplement part of the work with software. For example, the boundary between the tooth to be labeled and the tooth adjacent to it, and the boundary between the tooth to be labeled and the gingiva may be identified by edge detection, such that: Only teeth to be labeled can be extracted.

As shown in FIG. 17, the learning data set 116a can be classified into categories based on the profile of the subject 2 who was scanned when the learning data set 116a was generated. For example, age (minor, working generation, elderly), gender (male, female), race (Asian, Western, African), height (less than 150 cm, 150 or more), weight (less than 50 kg, 50 kg) above), and place of residence (resident in Japan, residing outside of Japan) can be assigned a learning data set generated from three-dimensional data including the teeth of the corresponding subject 2 . Note that the stratification of each category can be set as appropriate. For example, with respect to age, for each predetermined age difference (every three years in this case), specifically, more detailed information such as 0 to 3 years old, 4 to 6 years old, 7 to 9 years old, etc. can be stratified into

The identification device 100a generates a trained model 115a by training the tooth type estimation model 114a using a plurality of learning data sets 116a that can be classified by category. Note that the learning data may overlap depending on how the categories are classified. If the learning data overlaps, only one of the learning data is used to train the tooth type estimation model 114a. Just do it.

In general, tooth shape differs depending on genetics or living environment such as age, sex, race, height, weight, place of residence, and the like. For example, the permanent teeth of adults are generally larger than the milk teeth of children, and the two have different shapes. In general, male teeth are larger than female teeth, and the two have different shapes. In general, the teeth of Westerners tend to have sharp tips so that they can easily bite off hard meat and bread, whereas the teeth of Japanese people tend to have sharp tips so that they can easily grind soft rice and vegetables. It tends to be smooth. Therefore, by performing learning processing based on profile data as in the second embodiment, it is possible to generate a trained model that can identify the type of tooth in consideration of heredity or living environment.

As shown in FIG. 18, the identification device 100a performs learning processing for the tooth type to generate a trained model 115a. The learning process for the tooth type is based on the fact that the estimation model to be used is the tooth type estimation model 114a, and the information input to the tooth type estimation model 114a is the position information of the three-dimensional data, and the learning data is generated. This differs from the processing type learning process (see FIG. 9) executed by the identification device 100 according to the first embodiment in that the profile data of the subject 2 who was actually scanned is input (S2a).

Specifically, the identification device 100a selects learning data to be used for learning from the learning data set 116a (S1a). The identification device 100a inputs the position information of the three-dimensional data included in the selected learning data and the profile data of the subject 2 who was scanned when the learning data was generated into the tooth type estimation model 114a. (S2a). The identification device 100 executes identification processing for identifying the type of tooth using the tooth type estimation model 114a based on the characteristics of the tooth corresponding to the three-dimensional data (S3a). In the identification process, the identification device 100 identifies the type of tooth using the tooth type estimation model 114a based on profile data in addition to three-dimensional data.

The identification device 100a updates the parameter 1144a of the tooth type estimation model 114a based on the error between the tooth type identification result identified by the identification process and the correct data corresponding to the learning data used in the learning process (S4a ).

Next, the identification device 100a determines whether or not it has learned based on all the learning data (S5a). If the identification device 100a has not learned based on all the learning data (NO in S5a), the identification device 100a returns to the process of S1a.

On the other hand, if the identification device 100a has learned based on all the learning data (YES in S5a), the identification device 100a stores the learned tooth type estimation model 114a as the learned model 115a (S6a), and ends this process.

In this way, the identification device 100a uses the identification information (for example, color information) corresponding to the type of tooth associated with the three-dimensional data included in the learning data as correct data, and uses the three-dimensional data obtained by the identification process. A learned model 115a can be generated by learning the tooth type estimation model 114a based on the tooth type identification result.

Furthermore, in the learning process, the identification device 100a learns the tooth type estimation model 114a in consideration of profile data in addition to learning data, so it is possible to generate a trained model 115a in consideration of the profile of the subject 2.

Although the color information is associated with each positional information as identification information, which is correct data, the identification information may be information different from the color information, such as the name of the tooth or the number of the tooth.

[Service provision processing]
FIG. 19 is a flowchart for explaining an example of service providing processing executed by the identification device 100a according to the second embodiment. The service providing process executed by the identification device 100a according to the second embodiment differs from the service providing process executed by the identification device 100 according to the first embodiment shown in FIG. 10 in that S121 to S126 are executed instead of S12. . Differences from the service providing process executed by the identification device 100 will be mainly described below.

When three-dimensional data corresponding to a predetermined point is input (YES in S11), the identification device 100a determines whether there is profile data (S121). If there is no profile data (NO in S121), the identification device 100a inputs the three-dimensional data (positional information) to the learned tooth type estimation model 114a (learned model 115a) (S123). On the other hand, if there is profile data (YES in S121), the identification device 100a inputs the three-dimensional data (position information) and profile data to the learned tooth type estimation model 114a (learned model 115a) (S122).

After S122 and S123, the identification device 100a performs identification processing to identify the type of tooth using the trained tooth type estimation model 114a (learned model 115a) based on the characteristics of the tooth corresponding to the three-dimensional data. Execute (S124). At this time, if profile data has been input to the learned model 115a in S122, the identification device 100a identifies the type of tooth using the learned model 115a based on profile data in addition to three-dimensional data.

The identification device 100a determines whether or not the type of tooth has been identified (S125). Whether or not it has been identified is determined, for example, based on a comparison between the determination value included in the parameter 1144a and the calculation result.

If the identification device 100a can identify the type of tooth (YES in S125), the identification device 100a executes the processes from S13 onward. On the other hand, if identification device 100a fails to identify the type of tooth (NO in S125), identification device 100a inputs the three-dimensional data to trained processed species estimation model 114 (learned model 115), and uses learned model 115 as input. Identification processing for identifying the type of the machined portion is executed (S126).

After S125 and S126, the identification device 100a associates the identification result with the three-dimensional data input in S11 (S13). For example, when the identification device 100a can identify the type of tooth, the color information (color information indicating red, green, etc.) indicating the identification result 124a indicating the type of tooth is used for the position information included in the three-dimensional data. associate. On the other hand, if the type of the tooth cannot be identified but the type of the machined portion can be identified, the identification device 100a generates color information indicating the identification result 124 indicating the type of the machined portion for the position information included in the three-dimensional data. (color information indicating color a, color b, etc.) is associated.

The identification device 100a outputs the association result to the outside (for example, the display 300) (S14). Specifically, for points corresponding to the input three-dimensional data, display 300 displays the results of associating parts corresponding to the points. After that, the identification device 100a terminates this process. For example, as in the output examples during measurement shown in FIGS. 11 and 12, the identification results for the acquired three-dimensional data are sequentially displayed on the display 300 at the timing of acquiring the three-dimensional data.

As described above, in the second embodiment, when the tooth type identification unit 1134 fails to identify the type of tooth, the identification device 100a executes the process of identifying the type of the processed portion by the processing type identification unit 1132, If the tooth type identification unit 1134 can identify the type of tooth, the processing type identification unit 1132 does not perform the process of identifying the type of the processed portion.

In the service providing process shown in FIG. 19, if the identifying device 100a cannot identify the type of tooth and identifies that it is not a machined part in S126, it indicates that it could not be identified in S13. Information may be associated with three-dimensional data.

[Cart generation process]
FIG. 20 is a flowchart for explaining an example of chart generation processing executed by the identification device 100a according to the second embodiment. The medical record generation process is performed after the acquisition of the three-dimensional data by the three-dimensional scanner 200 is completed and when the identification process (service provision process) for the three-dimensional data is completed.

In the chart generating process, the identification device 100a estimates the tooth type corresponding to the processed site and the details of the treatment performed on the processed site, and creates a chart of the details of the treatment performed on the processed site in association with the estimated tooth type. output as

Referring to FIG. 20, identification device 100a determines whether or not there is a machined portion for which tooth type and treatment details have not been estimated (S21). If it is determined that there is no processed portion for which the tooth type and treatment content have not been estimated (NO in S21), that is, for all processed portions, the tooth type corresponding to the processed portion is estimated and the treatment content is determined. If it is determined that the estimation has been made, the identification device 100a executes the process of S26 and ends the chart generation process.

If it is determined that there is a processed portion for which the tooth type and treatment details have not been estimated (YES in S21), the identification device 100a identifies the type of tooth adjacent to the processed portion based on the three-dimensional data of the processed portion. Obtained from the result 124a (identification result indicating the type of tooth) (S22).

The identification device 100a estimates the type of tooth corresponding to the processed region from the acquired tooth type (S23). Based on the identification result 124 indicating the type of the machined part, the identification device 100a estimates the content of treatment given to the machined part (S24). The identification device 100a associates the estimated treatment content with the estimated tooth type and reflects them in the chart (S25). The identification device 100a executes the process of S21 after reflecting it in the medical chart, repeats S22 to S25 until the tooth type and treatment details are estimated for all processed parts, and then outputs the medical chart (S26). . For example, identification device 100a displays chart 150 shown in FIG. 14 on display 300, or prints chart 150. FIG.

As described above, in the second embodiment, the details of the treatment performed on the processed region are estimated, and the type of tooth corresponding to the processed region is estimated. A chart is automatically generated by outputting the estimated treatment content and the estimated tooth type in association with each other.

Note that in the second embodiment, the treatment estimation unit 1152 is assumed to function after measurement. Note that the processing executed by the treatment estimation unit 1152 may be incorporated in the service providing processing shown in FIG. 19 . For example, after executing S126, the identification device 100a may execute a process of estimating the content of treatment based on the obtained identification result.

<Embodiment 3>
An identification device 100b according to the third embodiment will be described with reference to FIGS. 21 to 26. FIG. The identifying device 100b identifies a lesion site in addition to identifying a tooth type and a processed site. It should be noted that "identifying a lesion site" means identifying whether or not a site corresponding to the three-dimensional data 122 is a lesion site, and if so, identifying the content of the lesion of the lesion site. means.

The “lesion site” is a lesion site in the oral cavity, and includes, for example, a carious site and a site showing symptoms of periodontal disease. In addition, the "lesion site" is not limited to a site that is actually diagnosed as having a disease, and may include a site that is likely to cause a disease, such as a site where dental calculus is accumulated.

[Configuration that works during measurement]
FIG. 21 is a schematic diagram showing a functional configuration functioning during measurement of the scanner system 10b according to the third embodiment. Referring to FIG. 21, a scanner system 10b differs from the scanner system 10a according to the second embodiment in that an identification device 100b is provided instead of the identification device 100a. Further, the identification device 100b differs from the identification device 100a according to the second embodiment in that a lesion identification unit 1136 is further provided. The lesion identification unit 1136 identifies the type of lesion based on the three-dimensional data input to the input unit 1102, using the lesion estimation model 114b (learned model 115b).

The lesion estimation model 114b includes a third NNW 1142b and parameters 1144b used by the third NNW 1142b. Note that the third NNW 1142b and the parameters 1144b are common to the first NNW 1142 and the parameters 1144 described in the first embodiment, so description thereof will be omitted.

Output unit 1103 outputs identification results from each of tooth type identification unit 1134 , processing type identification unit 1132 , and lesion identification unit 1136 to display 300 .

In addition, the storage 110 provided in the identification device 100b stores three-dimensional data 122, an identification result 124a indicating the type of tooth in the region corresponding to the three-dimensional data 122, and the type of the processed region corresponding to the three-dimensional data 122. An identification result 124 and an identification result 124b indicating the content of a lesion at a site corresponding to the three-dimensional data 122 are stored.

[Configuration that functions after measurement]
With reference to FIG. 22, functions related to chart generation will be described. FIG. 22 is a schematic diagram showing the functional configuration of the identification device 100b that functions when generating a chart after measurement.

Referring to FIG. 22, as functions for generating a chart after measurement, the identification device 100b includes a treatment estimation unit 1152, a tooth type estimation unit 1154, and a chart generation unit 1156, similarly to the identification device 100a according to the second embodiment. Prepare.

The tooth type estimation unit 1154 according to the third embodiment estimates the type of tooth corresponding to the lesion site in addition to the type of tooth corresponding to the processed site. The tooth type estimation unit 1154 estimates the type of tooth corresponding to the processed site or lesion site from the type of the tooth corresponding to the processed site or lesion site or the adjacent tooth. A “tooth corresponding to a lesion site” means a carious tooth or a tooth in the vicinity of a lesioned gingiva. The function of the treatment estimation unit 1152 is common to that of the treatment estimation unit 1152 included in the identification device 100a according to the second embodiment.

The chart generating unit 1156 generates information on the type of tooth corresponding to the processed region estimated by the tooth type estimating unit 1154, the treatment given to the processed region estimated by the treatment estimating unit 1152, and the lesion estimated by the tooth type estimating unit 1154. A chart 150 is generated based on the type of tooth corresponding to the site and the content of the lesion indicated by the identification result 124b.

The chart 150 displays a profile display area 152 showing the profile of the subject 2 specified from the profile data 119, and a tooth type display area 154 showing the type of tooth corresponding to the processed site or lesion site. It includes a comment display area 156 showing details of treatments or lesions applied to the tooth, and a tooth formula display area 158 showing the tooth formula.

The method for estimating the type of tooth corresponding to the lesion site is the same as the method for estimating the type of tooth corresponding to the processed site described in the second embodiment. For example, when the lesion site is a tooth, the tooth type estimation unit 1154 estimates the type of tooth corresponding to the lesion site from the type of the tooth corresponding to the lesion site or adjacent teeth. Further, when the lesion site is the gingiva, the tooth type estimation unit 1154 assumes that the tooth adjacent to the gingiva corresponds to the lesion site, and estimates the type of tooth corresponding to the lesion site (gingiva). .

[Generation of lesion estimation model 114b (learned model 115b)]
The learning process for generating the trained model 115b will be described with reference to FIGS. 23 and 24. FIG. FIG. 23 is a schematic diagram for explaining generation of the trained model 115b based on the learning data set 116b according to the third embodiment. FIG. 24 is a flowchart for explaining an example of a lesion learning process performed by the identification device 100b according to the third embodiment. The learning process for generating the trained model 115a of the tooth type estimation model 114a and the trained model 115 of the processing type estimation model 114 are common to the learning process described in the above embodiment, so description thereof will be omitted. .

Referring to FIG. 23 , when generating learning data for identifying lesion content, user 1 (particularly, a dentist, a dental technician, or other person with dental knowledge) uses a three-dimensional image to generate learning data. The positional information and the identification information are associated with each other by identifying the lesion site while confirming and applying a color corresponding to the content of the identified lesion to the three-dimensional image. The color to be applied is assigned in advance for each lesion content. The allocation contents are stored in the storage 110 as the color classification data 118b.

For example, when the user 1 identifies that the site included in the three-dimensional image is a tooth with no lesion, the user 1 paints the image of the tooth white. Further, when the user 1 identifies that a site included in the three-dimensional image is an early dental caries, the user 1 applies color A to the image of the site. For easy understanding, the colors A and C are hatched on the drawing.

When color information (identification information) indicating lesion content is associated with position information as correct data, the three-dimensional data includes position information and color information (identification information) corresponding to the applied color. Thus, such three-dimensional data is adopted as learning data. In other words, in the training data of the lesion estimation model 114b according to the third embodiment, color information corresponding to lesion details is associated (labeled) with position information referred to in the identification process. Furthermore, color information is associated with the three-dimensional data so that the lesion range corresponding to the three-dimensional data can be specified. Specifically, the same color information is associated with each position information corresponding to each lesion site. A collection of such learning data is held in the identification device 100b as a learning data set 116b.

In the third embodiment, the user 1 manually applies colors to each part included in the three-dimensional image based on his/her own knowledge. A color may be applied to each part included in the three-dimensional image in the operation. It is also possible to supplement some of the work with software. For example, the configuration may be such that the color to be applied is automatically selected based on the chart.

The identification device 100b generates a trained model 115b by training the lesion estimation model 114b using one or more training data sets 116b generated in this way.

Note that the lesion content classification shown in the color classification data 118b of FIG. 23 is an example. For example, an example was shown in which teeth are classified into three categories: no abnormality, early caries, and caries requiring treatment. can be classified according to the index of Similarly, although the gingiva is classified into two categories, ie, no abnormality and periodontal disease, the periodontal disease may be further classified into a plurality of stages indicating the degree of progression. Specifically, it may be classified into gingivitis, mild periodontitis, moderate periodontitis, and severe periodontitis.

As shown in FIG. 24, the identification device 100b performs a lesion learning process to generate a trained model 115b. In the lesion learning process, the lesion estimation model 114b is used as the estimation model, and the input learning data is the learning data set 116b. It differs from the processing (see FIG. 9).

Specifically, the identification device 100b selects learning data to be used for learning from the learning data set 116b (S1b). The identification device 100b inputs the position information of the three-dimensional data included in the selected learning data to the lesion estimation model 114b (S2b). The identification device 100b uses the lesion estimation model 114b to extract features of the three-dimensional data, identifies whether or not the site corresponding to the three-dimensional data is a lesion site, and if so, identifies the content of the lesion. Identification processing for identification is executed (S3b).

The identification device 100b updates the parameter 1144b of the lesion estimation model 114b based on the error between the identification result of the identification processing and the correct data corresponding to the learning data used in the learning processing (S4b).

Next, the identification device 100b determines whether or not it has learned based on all the learning data (S5b). If the identification device 100b has not learned based on all the learning data (NO in S5b), the identification device 100b returns to the process of S1b.

On the other hand, if the identification device 100b has learned based on all the learning data (YES in S5b), the identification device 100b stores the learned lesion estimation model 114b as the learned model 115b (S6b), and terminates this process.

[Service provision processing]
FIG. 25 is a flowchart for explaining an example of service providing processing executed by the identification device 100b according to the third embodiment. The service providing process executed by the identification device 100b according to the third embodiment differs from the service providing process executed by the identification device 100a according to the second embodiment shown in FIG. 19 in that S127 is further executed. Differences from the service providing process executed by the identification device 100a according to the second embodiment will be mainly described below.

If the identification device 100b fails to identify the type of tooth (NO in S125), the three-dimensional data is input to the learned processed species estimation model 114 (learned model 115), and the learned model 115 is used. An identification process for identifying the type of machined portion is executed (S126).

Further, the identification device 100b inputs the three-dimensional data to the learned lesion estimation model 114b (learned model 115b), and executes identification processing for identifying lesion content using the learned model 115b (S127).

After S125 and S127, the identification device 100b associates the identification result with the three-dimensional data input in S11 (S13). For example, if the type of tooth cannot be identified and the type of the processed part could not be identified in S126 for the part corresponding to the input three-dimensional data, but the type of lesion could be identified in S127, the identification device 100b associates color information (color information indicating color A, color B, etc.) indicating the identification result 124b with the position information included in the three-dimensional data. In addition, if the type of tooth cannot be identified and the type of the processed region can be identified in S126 and the type of lesion cannot be identified in S127 for the region corresponding to the input three-dimensional data, the identification device 100b , position information included in the three-dimensional data is associated with color information indicating the identification result 124 (color information indicating color a, color b, etc.). In addition, if the type of tooth cannot be identified and the type of the processed region can be identified in S126 and the type of lesion can be identified in S127 for the region corresponding to the input three-dimensional data, the identification device 100b is color information indicating the identification result 124 (color information indicating color a, color b, etc.) and color information indicating the identification result 124b (color A, color B, etc.) for the position information included in the three-dimensional data. color information).

The identification device 100b then outputs the association result to the outside (for example, the display 300) (S14). Specifically, for points corresponding to the input three-dimensional data, display 300 displays the results of associating parts corresponding to the points. After that, the identification device 100b terminates this process. For example, as in the output examples during measurement shown in FIGS. 11 and 12, the identification results for the acquired three-dimensional data are sequentially displayed on the display 300 at the timing of acquiring the three-dimensional data. Note that when two identification results (for example, identification result 124 indicating the type of processed site and identification result 124b indicating lesion details) are associated with the position information included in the three-dimensional data, two colors are used. may be displayed on the display 300 in such a manner as to alternately blink at .

As described above, in the third embodiment, the identification device 100b identifies the type of tooth in the region corresponding to the three-dimensional data, and then, if the type of tooth cannot be identified, determines whether the type of treatment applied to the region is identified. An identification process for identifying the content and an identification process for identifying the content of the lesion at the site are performed.

[Cart generation process]
FIG. 26 is a flowchart for explaining an example of chart generation processing executed by the identification device 100b according to the third embodiment. The chart generation process executed by the identification device 100b according to the third embodiment differs from the chart generation process executed by the identification device 100a according to the second embodiment shown in FIG. 20 in that S251 to S254 are further executed. Differences from the chart generation process executed by the identification device 100a according to the second embodiment will be mainly described below.

The identification device 100b repeats S22 to S25 until the tooth type and treatment details have been estimated for all processed parts, and when it determines that the tooth type has been estimated for all processed parts (NO in S21), The processing after S251 is executed.

The identification device 100b determines whether or not there is a lesion site for which the tooth type has not been estimated (S251). If it is determined that there is no lesion site for which the tooth type has not been estimated (NO in S251), that is, the processes of S251 to S254 are executed for all lesion sites to estimate the tooth type corresponding to the lesion site. If yes, the identification device 100b executes the process of S26 and ends the medical record generation process.

If it is determined that there is a lesion site for which the tooth type has not been estimated (YES in S251), the identification device 100b identifies the type of tooth adjacent to the lesion site from the identification result 124a based on the three-dimensional data of the lesion site. Acquire (S252).

The identification device 100b estimates the type of tooth corresponding to the lesion site from the acquired type of tooth (S253). The identification device 100b acquires the lesion content of the lesion site from the identification result 124b, associates the acquired lesion content with the estimated tooth type, and reflects them in the chart (S254).

The identification device 100b reflects the data in the medical chart, executes the process of S251, repeats S252 to S254 until the tooth types are estimated for all lesion sites, and then outputs the medical chart (S26). For example, identification device 100a displays chart 150 shown in FIG. 22 on display 300, or prints chart 150. FIG.

As described above, the identification device 100b according to the third embodiment also identifies the lesion site in addition to the processed site. Therefore, the identification device 100b can perform more accurate identification processing and can provide the user 1 with a large amount of information.

For example, in the case where the shape of a hole is greatly changed as the disease progresses, such as a hole formed by progressing caries, the user 1 can identify the lesion site at a glance. However, early caries and slightly missing parts may not be apparent at first glance. Also, periodontal disease is diagnosed from swollen and drooping gums. Therefore, the diagnosis of periodontal disease may vary depending on the knowledge of the user 1 .

Therefore, by using AI, it is possible to reduce the variability in diagnosis and to find out the characteristics of the lesion site even if it is not apparent at first glance.

In addition, since the user 1 can confirm the shape of the caries three-dimensionally, it is easier for the user 1 to examine the cutting range, the treatment method, and the like than when observing the inside of the oral cavity. Further, the identification device 100b can estimate the shape of the caries by identifying the content of the lesion and the lesion site. Therefore, the identification device 100b can automatically set the cutting range based on the caries shape, express the set cutting range on the displayed three-dimensional data of the oral cavity (tooth), and display it on the display 300. . Further, the identification device 100b can automatically propose a treatment method by identifying the content of the lesion and the lesion site.

In addition, when the identification device 100b identifies tartar or plaque and displays it on the display 300, the user 1 can explain to the subject 2 the condition of the teeth of the subject 2 three-dimensionally in an easy-to-understand manner. Instruction on how to brush teeth can be easily performed.

Further, by checking the three-dimensional image of the oral cavity displayed on the display 300, the user 1 can three-dimensionally recognize the distance of the gap between the gum and the tooth, thereby diagnosing the progress of periodontal disease. Cheap. Further, the degree of progression of periodontal disease may be estimated by the identification device 100b based on the distance between the gingiva and the tooth. Then, the identification device 100b may display the estimated degree of progression of periodontal disease on the display 300 .

Further, as an early symptom of periodontal disease, there is swelling of the gums. Swollen gums are a symptom that the user 1 tends to overlook. Therefore, early detection of periodontal disease is realized by the identification device 100b identifying the site where the initial symptoms of periodontal disease appear based on the shape characteristics (swelling) of the gingiva.

In the chart generation process, the identification device 100b estimates the type of tooth for each processed site and lesion site. Note that the identification device 100 b may estimate the type of tooth for each three-dimensional data 122 . In the third embodiment, for one piece of three-dimensional data, a part corresponding to the three-dimensional data may be identified as both a processed part and a lesion part. In such a case, estimating the tooth type for each processed site and diseased site results in duplication of processing. Therefore, the identification device 100b may estimate the type of tooth based on the three-dimensional data 122 in which the type of adjacent tooth can be identified for the three-dimensional data 122 in which the type of tooth cannot be identified.

[Modification]
<Modified example of identification processing>
The identification device 100 according to the first embodiment identifies whether or not a portion corresponding to three-dimensional data is a processed portion, and if it is a processed portion, identifies the type of the processed portion. Note that the identification device 100 may further estimate the treatment content as in the second and third embodiments. Further, the identification device 100 may perform identification processing for identifying the type of tooth as in Embodiments 2 and 3, and estimate the type of tooth corresponding to the machining site based on the type of tooth. good.

The identification device 100a according to the second embodiment executes identification processing for identifying the type of tooth, and if the type of tooth cannot be identified, identifies the site corresponding to the three-dimensional data as a processed site. , an identification process was performed to identify the type of machined portion.

Note that the identification device 100a may execute identification processing for identifying the type of the processed region, and then execute identification processing for identifying the type of the tooth if the region is identified as not being processed.

In addition, the identification device 100a performs both identification processing for identifying the tooth type and identification processing for identifying the machined portion on each of the obtained three-dimensional data, so as to obtain two identification results. can be In this case, the tooth type estimating section 1154 may estimate the type of tooth corresponding to the portion whose tooth type could not be identified based on the type of adjacent tooth. The order of execution of each process for obtaining the identification result may be determined in advance, and may be either from the identification result 124a regarding the tooth type or from the identification result 124 regarding the machined part. Also, each process for obtaining the identification result may be executed under task control.

The identification apparatus 100b according to the third embodiment executes identification processing for identifying the type of tooth, and if the type of tooth cannot be identified, identification processing for identifying the type of processed portion and identification for identifying lesion content. processed. Note that the identification device 100b may perform identification processing for identifying the type of tooth, identification processing for identifying treatment content, and identification processing for identifying lesion content, and obtain three identification results.

Further, the identification device 100b may perform identification processing for identifying the type of tooth when the type of the processed region cannot be identified and the contents of the lesion cannot be identified.

When obtaining a plurality of identification results for one piece of three-dimensional data, the order of executing each process for obtaining the identification results may be determined in advance, and may be executed in any order. , may be executed under task control.

Further, in the second embodiment, the identification device 100a identifies the type of the processed part using the processed type estimation model 114, and then estimates the treatment details based on the type of the processed part. Note that the identification device 100a may use one estimation model to estimate the type of the processed region and the content of treatment. In addition, although the identification device 100a has two estimation models, it may be configured to identify the type of tooth and the type of processed site (or treatment content) using one estimation model. Similarly, the identification device 100b according to the third embodiment uses three estimation models to identify three types of items, but one or two estimation models may be used to identify three types of items. .

Further, in Embodiments 1 to 3, the identification device 100b identifies the processed site and the content of treatment given to the processed site. In addition to these, a configuration for identifying the material of the part used for the processed site or the product number of the implant manufacturer may be provided.

<Modified example of training data set>
In any one of the first to third embodiments, identification processing is performed based on the position information included in the three-dimensional data. The identification process may be performed using color information in addition to position information. FIG. 27 is a schematic diagram for explaining an example of a learning data set according to the modification; When color information is input to a learned model in addition to position information to obtain a classification result, the learning process is executed with the color information included in the learning data as shown in FIG. 27 . In the learning process, color information before color-coding is input to the estimated model in addition to position information, so that a learned model is generated after considering the actual color of each part.

A prosthesis is usually attached to a tooth in a shape comparable to that of a natural tooth from the viewpoint of esthetics and functionality. Therefore, it may be difficult to sufficiently identify whether or not the site corresponding to the input three-dimensional data is a site corresponding to the prosthesis only with the positional information where the shape features appear. By using the estimation model, it is possible to extract color features in addition to shape features, so that the machined part can be identified with higher accuracy.

In addition, early symptoms of periodontal disease include swollen gums and changes in the color of the gums. Since the estimation model can extract color features in addition to shape features, periodontal disease can be identified more accurately. Also, as an early symptom of dental caries, there are white spots that form on the teeth (white spots). By using the estimation model, it is possible to extract color features in addition to shape features, so that caries can be identified with higher accuracy.

In any of Embodiments 1 to 3, the learning data set was generated by scanning the oral cavity of the subject 2 . Note that the learning data set used to identify the machined part may be generated by scanning the part used for machining.

FIG. 28 is a diagram for explaining a learning data set according to the modification; For example, the learning data set may be generated using three-dimensional data obtained by scanning parts used for treatment. For example, the learning data set may be generated from the collected three-dimensional data by collecting three-dimensional data obtained by scanning an implant body used for implant treatment. Also, the learning data set may be generated by collecting three-dimensional data obtained by scanning the abutment tooth and generating the collected three-dimensional data. The identification device may generate a learned model by inputting each learning data set generated for each part used in treatment into an estimation model.

<Modified example of the medical chart generator>
In Embodiments 2 and 3 above, the medical chart generation unit 1156 includes a tooth type estimated by the tooth type estimation unit 1154, and an identification data indicating treatment content or lesion content estimated by the treatment estimation unit 1152 based on the identification result 124. A chart 150 was generated based on the results 124b. Note that the chart generation unit 1156 does not need to generate the final product of the chart 150, and may provide a function of assisting the input of the chart by the user 1, for example. Specifically, information (for example, tooth type) indicating a site identified as a processed site or a diseased site may be output to prompt user 1 to enter chart 150 .

<Modified example of service provision processing>
As shown in FIGS. 10, 19, and 25, the identification devices according to Embodiments 1 to 3 do not perform learning processing in service providing processing. However, as shown in FIG. 29, the identification device may perform the learning process in the service providing process. The identification device may perform so-called Federated Learning.

FIG. 29 is a flowchart for explaining an example of service providing processing executed by the identification device 100c according to the modification. Since the processing of S11 to S14 shown in FIG. 29 is the same as the processing of S11 to S14 shown in FIG. 10, detailed description of the processing of S11 to S14 is omitted in FIG. In the following description, it is assumed that the identification device 100 according to the first embodiment executes learning processing in service providing processing.

As shown in FIG. 29, the identification device 100c performs the processing of S11 to S14 to determine the state of the part corresponding to the point corresponding to the input three-dimensional data (whether it has been processed, whether it has been After outputting the result of association with the point based on the identification result indicating the content), the learning process at the time of service provision is executed. Specifically, after S14, the identification device 100c determines whether correct data for error correction has been input (S15). For example, when the identification result obtained in S12 (whether or not it has been processed, details of processing) is different from the determination result determined by the user 1 based on the part that was actually scanned, the identification device 100c actually It is determined whether or not the error has been corrected by the user 1 inputting correct data, which is the determination result of the user 1 for the part to be scanned.

If the correct data for error correction is not input (NO in S15), the identification device 100c ends this process. On the other hand, when the correct data for error correction is input (YES in S15), the identification device 100c gives a reward based on the identification result and the correct data (S16).

For example, the smaller the degree of dissociation between the identification result and the correct answer data, the smaller the negative points given as a reward, and the greater the degree of dissociation between the two, the larger the negative points given as a reward. Just do it. Note that the reward is not limited to minus points, and may be plus points.

The method of rewarding can be designed arbitrarily. For example, the identification device 100c may give a reward based on a preset reward size for each of a plurality of possible combinations of identification results and correct data. Also, the identification device 100c may reward according to a predetermined rule. Specifically, the identification device 100c may give negative points with a small degree of dissociation and a small value if the identification result and the correct data are in different minor categories but share the same major category (see Table 1). . On the other hand, if the identification result and the correct data are different in both the small classification and the large classification, the identification device 100c may give a large negative point with a large degree of dissociation. Further, the identification device 100c may give a reward by a method combining a reward giving method according to such a rule and a giving method in which a reward is set in advance for each of a plurality of types of combinations. good.

The identification device 100c updates the parameter 1144 of the trained model 115 based on the given reward (S17). For example, the identification device 100c updates the parameter 1144 of the trained model 115 so that the negative points given as rewards approach zero. After that, the identification device 100c ends this process.

Although the learning process executed during the service providing process for identifying only the type of machined part has been described, the identification device is capable of handling the plurality of items shown in the second and third embodiments (the type of machined part and tooth type). The learning process can be executed even during the service providing process for identifying the type, type of machined part, type of tooth, and content of lesion).

For example, in the service providing process for identifying the type of machined part and the type of tooth, when correct data for another tooth type is input in response to the identification result in which the type of tooth is identified, the identification device The parameter 1144a of the tooth type estimation model 114a is updated based on the identification result 124a indicating the type of teeth and the correct answer data. In this case, as an example, if the tooth output as the identification result 124a and the tooth input as the correct answer data are adjacent to each other, the identification device gives a small minus point, and if the two are separated, a value give a big minus point.

As another example, in the service providing process for identifying the type of processed part, the type of tooth, and the content of the lesion, when correct data of different content of the lesion is input with respect to the content of the identified lesion, the identification device: The parameter 1144b of the lesion estimation model 114b is updated based on the identification result 124b indicating the content of the lesion and the correct data. In this case, for example, if the degree of inflammation (degree of caries progression or progression of gingivitis) is close, the identification device gives a small minus point, and if the degree of inflammation is far, a large minus point. give.

Further, an identification device having a plurality of estimation models may select an estimation model to be re-learned according to a combination of identification results and correct data. For example, if both the identification result and the correct data relate to the tooth type, the identification device executes processing to update the parameter 1144a of the tooth type estimation model 114a. If both the identification result and the correct data are related to the content of processing, the identification device executes processing to update the parameters 1144 of the processing type estimation model 114 . If both the identification result and the correct data are matters related to lesion content, the identification device executes processing to update the parameter 1144b of the lesion estimation model 114b. Further, the identification device may update a plurality of parameters out of the parameter 1144 of the processed species estimation model 114, the parameter 1144a of the tooth species estimation model 114a, and the parameter 1144b of the lesion estimation model 114b.

In this way, the identification device according to the modification executes learning processing even in the service providing processing. The type, or lesion content, can be identified. Such processing is a kind of so-called collaborative learning.

It should be considered that the embodiments disclosed this time are illustrative in all respects and not restrictive. The scope of the present disclosure is indicated by the scope of claims rather than the above description, and is intended to include all modifications within the meaning and scope of equivalents of the scope of claims. Note that the configurations exemplified in this embodiment and the configurations exemplified in the modifications can be combined as appropriate.

1 user, 2 subject, 10, 10a, 10b scanner system, 100, 100a, 100b, 100c identification device, 102 scanner interface, 103 display interface, 105 peripheral device interface, 107 media reader, 108 PC display, 109 memory, 110 storage, 112 scan information, 114 processed species estimation model, 114a tooth species estimation model, 114b lesion estimation model, 115, 115a, 115b learned model, 116, 116a, 116b learning data set, 118, 118a, 118b color classification data, 119 profile data, 120 identification program, 121 learning program, 122 three-dimensional data, 124, 124a, 124b identification result, 130 arithmetic device, 150 chart, 152 profile display area, 154 tooth type display area, 156 comment display Area 158 Dental formula display area 200 Three-dimensional scanner 300 Display 550 Removable disk 601 Keyboard 602 Mouse 1102 Input unit 1103 Output unit 1132 Processing type identification unit 1134 Tooth type identification unit 1136 Lesion identification unit , 1142 first NNW, 1142a second NNW, 1142b third NNW, 1144, 1144a, 1144b parameters, 1152 treatment estimation unit, 1154 tooth type estimation unit, 1156 chart generation unit.

Claims (10)

An identification device for identifying a processed part with respect to a living body part in the oral cavity,
an input unit for inputting three -dimensional data including three-dimensional position information at each of a plurality of points representing a three-dimensional shape of the oral cavity;
an identification unit that identifies the machined portion based on the three-dimensional data input from the input unit and an estimation model including a neural network for estimating the machined portion based on the three-dimensional data;
An output unit that outputs the identification result by the identification unit ,
The identification device, wherein the identification unit directly inputs at least the three-dimensional position information of the three-dimensional data to the neural network included in the estimation model. The identification device according to claim 1, wherein the three-dimensional data includes color information at each of a plurality of points representing a three-dimensional shape in the oral cavity. The neural network is a neural network for estimating the processed part based on the three-dimensional data input from the input unit and estimating the type of tooth of the part corresponding to the three-dimensional data,
The identification unit directly inputs at least the three-dimensional positional information of the three-dimensional data of the teeth around the tooth corresponding to the machining site into the neural network included in the estimation model , and identify the type of
3. The identifying device according to claim 1, further comprising a tooth type estimating section that estimates the type of tooth corresponding to the machining site based on the type of the peripheral tooth identified by the identifying section. 4. The identifying device according to claim 3, wherein the output unit outputs the machined part identified by the identifying part in association with the tooth type corresponding to the machined part estimated by the tooth type estimating part. The identification device according to any one of claims 1 to 4, further comprising a treatment estimating unit that estimates, based on the processed portion identified by the identifying unit, details of treatment given to the processed portion. . The neural network is a neural network for estimating the processed part based on the three-dimensional data input from the input unit and estimating the type of tooth of the part corresponding to the three-dimensional data,
The identification unit directly inputs at least the three-dimensional positional information of the three-dimensional data of the teeth around the tooth corresponding to the machining site into the neural network included in the estimation model , and identify the type of
a treatment estimating unit for estimating, based on the processed part identified by the identifying unit, details of treatment given to the processed part;
a tooth type estimating unit for estimating the type of tooth corresponding to the processed portion based on the type of the peripheral tooth identified by the identifying unit;
The output unit converts the processed region identified by the identifying unit to the details of the treatment performed on the processed region estimated by the treatment estimation unit, and the type of tooth corresponding to the processed region estimated by the tooth type estimation unit. 3. The identification device according to claim 1 or 2, which outputs in association with and. The estimation model is
a first estimation model including a first neural network for estimating a machined part based on the three-dimensional data input from the input unit;
a second estimation model including a second neural network for estimating the type of tooth of the part corresponding to the three-dimensional data based on the three-dimensional data input from the input unit;
The identification unit directly inputs at least the three-dimensional positional information of the three-dimensional data input from the input unit into the second neural network , and determines the type of tooth of the site corresponding to the input three-dimensional data. is not estimated, at least the three-dimensional positional information among the three-dimensional data input from the input unit is directly input to the first neural network to identify the processed part. 7. Identification device according to any one of 6. A scanner system for acquiring intraoral shape information,
A three-dimensional scanner that acquires three-dimensional data including three-dimensional positional information at each of a plurality of points representing the three-dimensional shape of the oral cavity using a three-dimensional camera;
an identification device that identifies a processed part of a living body part in the oral cavity based on the three-dimensional data acquired by the three-dimensional scanner;
The identification device
an input unit into which three-dimensional data acquired by the three-dimensional scanner is input;
an identification unit that identifies the machined portion based on the three-dimensional data input from the input unit and an estimation model including a neural network for estimating the machined portion based on the three-dimensional data;
an output unit that outputs a result of identification by the identification unit ;
The scanner system , wherein the identification unit directly inputs at least the three-dimensional positional information of the three-dimensional data to the neural network included in the estimation model. An identification method for identifying a processed part of a living body part in the oral cavity by a computer,
a step of inputting three-dimensional data including three-dimensional position information at each of a plurality of points representing a three-dimensional shape of the oral cavity;
a step of identifying the machined portion based on the three-dimensional data input in the inputting step and an estimation model including a neural network for estimating the machined portion based on the three-dimensional data;
and outputting an identification result from the identifying step ,
The identification method , wherein at least the three-dimensional positional information of the three-dimensional data is directly input to the neural network included in the estimation model. An identification program for identifying a processed part with respect to a living body part in the oral cavity,
The identification program, in a computer,
a step of inputting three-dimensional data including three-dimensional position information at each of a plurality of points representing a three-dimensional shape of the oral cavity;
a step of identifying the machined portion based on the three-dimensional data input in the inputting step and an estimation model including a neural network for estimating the machined portion based on the three-dimensional data;
a step of outputting an identification result obtained by the identifying step ;
A program for identification, wherein at least the three-dimensional positional information of the three-dimensional data is directly input to the neural network included in the estimation model.

Images