JP2022012198A - Identification device, scanner system, identification method, and identification program - Google Patents

Abstract

To enhance possibility of a three-dimensional scanner in a dental field, and widen an application of three-dimensional data acquired by a three-dimensional scanner.SOLUTION: An identification device 100 comprises: an input part 1102 into which three-dimensional data including positional information on each of plural points forming a shape of an oral cavity, is input; a processing kind identification part 1132 for identifying a processing site, on the basis of the three-dimensional data input from the input part 1102, and a processing kind estimation model 114 including a first NNW 1142 for estimating the processing site on the basis of the three-dimensional data; and an output part 1103 for outputting the identification result by the processing kind identification part 1132.SELECTED DRAWING: Figure 5

Description

The present disclosure relates to an identification device, a scanner system including the identification device, an identification method, and an identification program.

Conventionally, in the field of dentistry, a three-dimensional scanner having a built-in three-dimensional camera that acquires a three-dimensional shape of a tooth has been known in order to digitally design a prosthesis or the like on a computer. For example, Patent Document 1 discloses a technique for recording the shape of a tooth by photographing the tooth with a three-dimensional camera.

Japanese Unexamined Patent Publication No. 2000-74635

In this way, three-dimensional scanners have been used for digitally designing prostheses and the like. However, it has not been considered that the information obtained by the three-dimensional scanner can be used for other purposes.

The present disclosure is to expand the use of the three-dimensional data obtained by the three-dimensional scanner, and aims to enhance the possibility of the three-dimensional scanner in the dental field.

According to the present disclosure, there is provided an identification device for identifying a processed site with respect to a biological site in the oral cavity. The identification device is used for an input unit in which three-dimensional data including three-dimensional position information at each of a plurality of points constituting the shape in the oral cavity is input, a three-dimensional data input from the input unit, and three-dimensional data. Based on an estimation model including a neural network for estimating a processed portion based on the machined portion, an identification unit for identifying the processed portion and an output unit for outputting the identification result by the identification unit are provided.

According to the present disclosure, a scanner system for acquiring shape information in the oral cavity is provided. The scanner system uses a three-dimensional camera to acquire three-dimensional data including three-dimensional position information at each of a plurality of points constituting the shape of the oral cavity, and a three-dimensional scanner acquired by the three-dimensional scanner. Based on the data, it is equipped with an identification device that identifies the processed part with respect to the biological part in the oral cavity, and the identification device includes an input unit into which the three-dimensional data acquired by the three-dimensional scanner is input and a tertiary input from the input unit. Based on the original data and an estimation model including a neural network for estimating the processed portion based on the three-dimensional data, the identification unit for identifying the processed portion and the output unit for outputting the identification result by the identification unit are included.

According to the present disclosure, a computer-based identification method for identifying a processed site with respect to a biological site in the oral cavity is provided. The identification method includes a step in which three-dimensional data including three-dimensional position information at each of a plurality of points constituting the shape in the oral cavity is input, a three-dimensional data input by the input step, and three-dimensional data. It includes a step of identifying a machined part and a step of outputting an identification result by the step of identifying the machined part based on an estimation model including a neural network for estimating a machined part based on.

According to the present disclosure, an identification program for identifying a processed site with respect to a biological site in the oral cavity is provided. The identification program includes a step in which three-dimensional data including three-dimensional position information at each of a plurality of points constituting the shape in the oral cavity is input to the computer, and a step in which the three-dimensional data input by the input step is input. , The step of identifying the machined part and the step of outputting the identification result by the step of identifying the machined part are executed based on the estimation model including the neural network for estimating the machined part based on the three-dimensional data.

According to the present disclosure, it is possible to identify a processed portion in the oral cavity from three-dimensional data including three-dimensional position information at each of a plurality of points constituting the shape of the oral cavity, and it is possible to identify the processed portion. Can provide various uses.

It is a schematic diagram which shows the whole structure of the scanner system 10 which concerns on Embodiment 1. FIG. It is a schematic diagram which shows the application example of the scanner system 10 which concerns on Embodiment 1. FIG. It is a schematic diagram which shows the whole structure of the system to which the scanner system 10 which concerns on Embodiment 1 can apply. It is a schematic diagram which shows the hardware composition of the identification apparatus 100 which concerns on Embodiment 1. FIG. It is a schematic diagram which shows the functional structure of the identification apparatus 100 which concerns on Embodiment 1. FIG. It is a schematic diagram for demonstrating the identification process by the identification apparatus 100 which concerns on Embodiment 1. FIG. It is a schematic diagram for demonstrating the generation of learning data which concerns on Embodiment 1. FIG. It is a schematic diagram for demonstrating an example of learning data which concerns on Embodiment 1. FIG. It is a flowchart for demonstrating an example of the learning process for the processing type executed by the identification apparatus 100 which concerns on Embodiment 1. FIG. It is a flowchart for demonstrating an example of the service provision process executed by the identification apparatus 100 which concerns on Embodiment 1. FIG. It is a figure which shows the output example during measurement. It is a figure which shows the modification of the output example during measurement. It is a schematic diagram which shows the functional structure which functions during the measurement of the scanner system 10a which concerns on Embodiment 2. It is a schematic diagram which shows the functional structure which functions when the medical record is generated after the measurement of the identification apparatus 100a which concerns on Embodiment 2. It is a schematic diagram for demonstrating the method of estimating the type of the tooth of a processed part. It is a schematic diagram for demonstrating an example of the learning data set 116a which concerns on Embodiment 2. FIG. It is a schematic diagram for demonstrating the generation of the trained model 115a based on the learning data set 116a which concerns on Embodiment 2. FIG. It is a flowchart for demonstrating an example of the learning process for the tooth type executed by the identification apparatus 100a which concerns on Embodiment 2. FIG. It is a flowchart for demonstrating an example of the service provision process executed by the identification apparatus 100a which concerns on Embodiment 2. FIG. It is a flowchart for demonstrating an example of the medical record generation processing executed by the identification apparatus 100a which concerns on Embodiment 2. FIG. It is a schematic diagram which shows the functional structure which functions during the measurement of the scanner system 10b which concerns on Embodiment 3. FIG. It is a schematic diagram which shows the functional structure of the identification apparatus 100b which functions at the time of generating a medical record after measurement. It is a schematic diagram for demonstrating the generation of the trained model 115b based on the learning data set 116b which concerns on Embodiment 3. FIG. It is a flowchart for demonstrating an example of the learning process for a lesion performed by the identification apparatus 100b which concerns on Embodiment 3. FIG. It is a flowchart for demonstrating an example of the service provision process executed by the identification apparatus 100b which concerns on Embodiment 3. FIG. It is a flowchart for demonstrating an example of the medical record generation processing executed by the identification apparatus 100b which concerns on Embodiment 3. FIG. It is a schematic diagram for demonstrating an example of the learning data set which concerns on a modification. It is a figure for demonstrating the learning data set which concerns on a modification. It is a flowchart for demonstrating an example of a service provision process executed by the identification apparatus 100c which concerns on a modification.

The embodiments of the present disclosure will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are designated by the same reference numerals and the description thereof will not be repeated.

<Embodiment 1>
[Outline of Scanner System 10]
The outline of the scanner system 10 according to the first embodiment will be described with reference to FIGS. 1 to 3. FIG. 1 is a schematic diagram showing the overall configuration of the scanner system 10 according to the first embodiment. FIG. 2 is a schematic diagram showing an application example of the scanner system 10 according to the first embodiment. FIG. 3 is a schematic diagram showing an overall configuration of a system to which the scanner system 10 according to the first embodiment can be applied.

Referring to FIG. 1, the scanner system 10 according to the first embodiment includes a three-dimensional scanner 200, an identification device 100, and a display 300. The three-dimensional scanner 200 acquires the three-dimensional data 122 to be scanned by the built-in three-dimensional camera. Specifically, the three-dimensional scanner 200 scans the inside of the oral cavity to obtain the three-dimensional data 122, and the position information (vertical direction, horizontal direction, height direction) of each of the plurality of points constituting the shape in the oral cavity. (Coordinates of each axis) are acquired using an optical sensor or the like. Alternatively, the three-dimensional scanner 200 scans the model created by taking an impression in the oral cavity to obtain position information (for example, vertical) of each of a plurality of points constituting the scan target portion as three-dimensional data. The X-axis along the direction, the Y-axis along the horizontal direction, and the Z-axis coordinates along the height direction) are acquired using an optical sensor or the like. The identification device 100 is based on the three-dimensional data 122 acquired by the three-dimensional scanner 200, and has a two-dimensional image including an image of teeth or the like from an arbitrary viewpoint, or a three-dimensional image including a three-dimensional tooth or the like by a hologram or the like. An image is generated, and the generated three-dimensional image is displayed on the display 300.

The scanner system 10 according to the first embodiment automatically identifies a machined portion based on the three-dimensional data 122 acquired by the three-dimensional scanner 200 by using AI (artificial intelligence) possessed by the identification device 100. It is configured to perform the processing to be performed. "Identifying the machined part" means identifying whether or not the part corresponding to the three-dimensional data 122 is a machined part, and if it is a machined part, identifying the type of the machined part. In the following, the process of identifying what kind of site the portion corresponding to the three-dimensional data 122 by the identification device 100 is referred to as "identification process".

The "processed site" means a site processed to treat a biological part in the oral cavity. Further, the "living part in the oral cavity" is not limited to the teeth, but may include the gingival part. The processed site is not limited to the site where the prosthesis or the artificial tooth is attached to the tooth, but may include the site where the implant body is attached to the jawbone. In addition, the processed part is not limited to the part where an artificially generated object (artificial object) such as an implant body or a prosthesis is attached, but also a part where a natural tooth is shaved such as an abutment tooth or a cavity-forming tooth. Can include.

With reference to FIG. 2, when the user 1 scans the biological part in the oral cavity of the subject 2 using the three-dimensional scanner 200, the three-dimensional data 122 is input to the identification device 100. The identification device 100 executes an identification process for identifying what kind of part corresponds to the input three-dimensional data 122 based on the estimation model including the input three-dimensional data 122 and the neural network. In the first embodiment, the identification device 100 discriminates whether or not the part corresponding to the three-dimensional data 122 is a machined part, and if it is a machined part, executes an identification process for identifying the type of the machined part. ..

The "estimation model" includes the neural network and the parameters used by the neural network. For example, in the "estimation model", the parameters are optimized by using the teacher data including the three-dimensional data of the machined part which is the input data and the correct answer data indicating the type of the machined part corresponding to the three-dimensional data. ..

Here, Table 1 shows the types of processed parts that can be set as correct answer data. As shown in Table 1, the types of processed sites include, for example, "implants", "cavity-forming teeth", "abutment teeth", "prostheses" and the like as major categories. The "implant" can be classified into an "implant body" corresponding to a tooth root portion and an "abutment" corresponding to an abutment portion. Also, "implant bodies" and "abutments" can be classified according to what prosthesis is attached. "Abutment teeth" and "cavity-forming teeth" can be further classified according to what prosthesis is attached. "Prosthesis" can be further classified into inlays, onlays, crowns, bridges, veneers, dentures and the like.

As shown in Table 1, the "processed site" can be arbitrarily subdivided, and can be classified into "abutment tooth", "cavity-forming tooth", "prosthesis", and "implant". Often, each of the "abutment tooth", "cavity-forming tooth", "prosthesis", and "implant" may be further subdivided and classified. That is, the method for classifying the processed portion can be arbitrarily designed.

The method for classifying the processed parts can be arbitrarily designed. Therefore, the information indicating the type of the processed portion included in the teacher data may include the type classified by another classification method in addition to the type classified by a specific classification method. For example, the information indicating the type of the machined portion may include "abutment tooth" and "inlay-applied abutment tooth".

Further, the processed portion can be roughly classified into a processed portion being treated and a processed portion having been treated. Therefore, the correct answer data may include information that can identify whether the patient is "under treatment" or "treated" in addition to the information indicating the type of the processed portion.

In addition, the treatment content applied to the teeth differs depending on the type of processed part. Therefore, the correct answer data may include information indicating "treatment content" in addition to information indicating the type of the processed portion.

Further, the teacher data may include three-dimensional data of a biological part (for example, natural tooth, gingiva) that is not a processed part, and correct answer data indicating that the three-dimensional data is not a processed part.

The process of learning such an estimation model is also referred to as "learning process". In addition, the estimation model optimized by the learning process is also referred to as a "trained model" in particular. That is, the pre-trained estimation model and the trained estimation model are collectively referred to as an "estimation model", while the trained estimation model is also referred to as a "trained model".

A scene in which the scanner system 10 can be applied will be described with reference to FIG. By using the scanner system 10, the user 1 can acquire three-dimensional shape data (hereinafter, also referred to as “three-dimensional data”) including teeth possessed by the subject 2. The "user" is any person who uses the scanner system 10, such as a surgeon such as a dentist, a dental assistant, a teacher or student at a dental school, a dental technician, a maker engineer, or a worker at a manufacturing factory. May be. The "subject" may be any person who is the target of the scanner system 10, such as a patient in a dental clinic or a subject in a dental school.

When the identification process is executed by the identification device 100 using the trained model, the identification result is output to the display 300.

The display 300 displays at least one of an image, a letter, a number, an icon, and a symbol corresponding to the identification result.

The identification device 100 generates a three-dimensional image of the oral cavity reflecting the identification result based on the three-dimensional data 122 and the identification result, and outputs the three-dimensional image to the display 300. For example, the identification device 100 reflects the identification result on the three-dimensional image by displaying the processed portion on the display 300 in a color corresponding to the type of the processed portion.

The three-dimensional scanner 200 has been used for digitally designing prostheses and the like. The identification device 100 can provide the three-dimensional scanner 200 with a new application of identifying the type of the processed portion by identifying the type of the processed portion in the oral cavity from the three-dimensional data.

For example, the type of the processed part with respect to the biological part in the oral cavity is identified, and the identification result is output, so that the user 1 can compare his / her own examination result determined while scanning with the identification result and perform a medical record. Can be entered. Further, even if the examination result and the identification result are different from each other and the processed portion is confirmed again, the user 1 confirms the identification result displayed on the display 300 to roughly confirm the processed portion in the oral cavity. You can grasp the position of. As a result, the time during which the subject 2 is open can be shortened, and the burden on the subject 2 can be reduced.

Further, even when the user 1 performs medical examination after scanning, information such as the type of the processed portion that assists the examination can be obtained, so that the difference in the examination result due to the individual ability of the user 1 can be reduced.

Further, the user 1 can explain the examination result to the subject 2 while displaying the three-dimensional image reflecting the identification result on the display 300. As a result, the subject 2 can objectively confirm the condition of his / her teeth.

With reference to FIGS. 1 and 3, the identification result by the identification device 100 may be output as scan information to the server device 500 arranged in the dental laboratory and the management center together with the three-dimensional data used in the identification process. good.

For example, as shown in FIG. 3, the scanner system 10 is arranged in each of a plurality of locals A to C. For example, local A and local B are dental clinics, and in the dental clinic, the surgeon or dental assistant who is the user 1 uses the scanner system 10 to at least remove the teeth and gingiva of the patient who is the subject 2. Acquire 3D data in the oral cavity including. Further, the local C is a dental school, and in the dental school, the teacher or the student who is the user 1 acquires the three-dimensional data in the oral cavity of the subject who is the subject 2. The scan information (three-dimensional data, identification result) acquired in each of the locals A to C is output to the server device 500 located in the dental laboratory and the management center, which is the local D, via the network 5.

If the identification result by the identification device 100 and the examination result by the user 1 do not match, the user 1 may correct the identification result by inputting to a medical record or the like. In such a case, even if the scanner system 10 arranged in each of the locals A to C includes both the identification result before the modification and the identification result after the modification in the scan information and outputs the scan information to the server device 500. good. Further, the scanner system 10 arranged in each of the locals A to C may include the identification result after the modification in the scan information and output it to the server device 500 without including the identification result before the modification.

At the dental laboratory, a dental technician or the like creates a prosthesis or the like that supplements the missing portion of the tooth of the subject 2 based on the scan information acquired from each of the locals A to C. In the management center, the server device 500 accumulates and stores the scan information acquired from each of the locals A to C, and holds it as big data.

The server device 500 is not limited to the one arranged in the management center different from the local one in the dental clinic, and may be arranged in the local one. For example, the server device 500 may be arranged in any one of the locals A to C. Further, a plurality of identification devices 100 may be arranged in one local, and further, a server device 500 capable of communicating with the plurality of identification devices 100 may be arranged in the one local. Further, the server device 500 may be realized in the form of a cloud service.

In the dental laboratory, scan information is collected from various places such as locals A to C. Therefore, the scan information held in the dental laboratory may be transmitted to the management center via the network 5, or the removable disk 550 such as a CD (Compact Disc) and a USB (Universal Serial Bus) memory. May be sent to the management center via.

Note that scan information may be sent from each of the locals A to C to the management center via the removable disk 550 without going through the network 5. Further, scan information may be sent to each other between the locals A to C via the network 5 or the removable disk 550.

The identification device 100 of each local A to C holds an estimation model by itself, and identifies the type of the machined portion by using the estimation model held by each of them during the identification process. The identification device 100 of each local A to C generates a trained model by learning its own estimation model by its own learning process. Further, in the present embodiment, the server device 500 also holds the estimation model. The server device 500 generates a trained model by learning an estimation model by a learning process using the identification device 100 of each local A to C and the scan information acquired from the dental laboratory, and generates a trained model of each local A to C. The trained model is distributed to the identification device 100.

Further, when the identification result is corrected by the user 1, the identification devices 100 of the locals A to C may relearn about their own estimation model using the corrected identification result as correct answer data.

In the present embodiment, both the identification device 100 of each local A to C and the server device 500 execute the learning process, but only the identification device 100 of each local A to C executes the learning process. Or may be a form in which only the server device 500 executes the learning process. When only the server device 500 executes the learning process, the estimation model (learned model) held by the identification devices 100 of each local A to C is common to the identification devices 100 of each local A to C. Will be done.

Further, the server device 500 may have a function of identification processing in the identification device 100. For example, each local A to C transmits the acquired three-dimensional data to the server device 500, and the server device 500 is based on the respective three-dimensional data received from each local A to C, and the type of the processing site in each is different. The identification result may be calculated. Then, the server device 500 may transmit each identification result to each local A to C, and each local A to C may output the estimation result received from the server device 500 to a display or the like. In this way, each local A to C and the server device 500 may be configured in the form of a cloud service. In this way, as long as the server device 500 holds the estimation model (trained model), each local A to C can obtain the estimation result without holding the estimation model (trained model). can.

As described above, according to the scanner system 10 according to the present embodiment, the type of the machined portion is identified based on the three-dimensional data acquired by the three-dimensional scanner 200 by using the AI possessed by the identification device 100.

Although the identification device 100 of the scanner system 10 arranged in each of the locals A to C will be described below, the server device 500 assumes that the identification device 100 can similarly execute the processes that can be executed by the server device 500. The description of the process executed by is omitted. Similarly, in the second and third embodiments, the server device 500 can similarly execute the processes that can be executed by each of the identification device 100a and the identification device 100b.

[Hardware configuration of identification device 100]
An example of the hardware configuration of the identification device 100 according to the first embodiment will be described with reference to FIG. FIG. 4 is a schematic diagram showing a hardware configuration of the identification device 100 according to the first embodiment. The identification device 100 may be realized by, for example, a general-purpose computer or a computer dedicated to the scanner system 10.

As shown in FIG. 4, the identification device 100 has, as main hardware elements, a scanner interface 102, a display interface 103, a peripheral device interface 105, a media reader 107, a PC display 108, a memory 109, and the like. It includes a storage 110 and a computing device 130.

The scanner interface 102 is an interface for connecting the three-dimensional scanner 200, and realizes data input / output between the identification device 100 and the three-dimensional scanner 200.

The display interface 103 is an interface for connecting the display 300, and realizes input / output of data between the identification device 100 and the display 300. The display 300 is composed of, for example, an LCD (Liquid Crystal Display) or an organic ELD (Electroluminescence) display.

The peripheral device interface 105 is an interface for connecting peripheral devices such as a keyboard 601 and a mouse 602, and realizes data input / output between the identification device 100 and the peripheral device.

The media reading device 107 reads various data such as scan information stored in the removable disk 550.

The PC display 108 is a display dedicated to the identification device 100. The PC display 108 is composed of, for example, an LCD or an organic EL display. In the first embodiment, the PC display 108 is separate from the display 300, but may be shared with the display 300.

The memory 109 provides a storage area for temporarily storing a program code, a work memory, or the like when the arithmetic unit 130 executes an arbitrary program. The memory 109 is composed of, for example, a volatile memory device such as a DRAM (Dynamic Random Access Memory) or a SRAM (Static Random Access Memory).

The storage 110 provides a storage area for storing various data necessary for identification processing, learning processing, and the like. The storage 110 is composed of, for example, a hard disk or a non-volatile memory device such as an SSD (Solid State Drive).

The storage 110 includes scan information 112, a machined species estimation model 114 (learned model 115), a learning data set 116, color classification data 118, an identification program 120, a learning program 121, and an OS (Operating). System) 127 and is stored.

The scan information 112 includes the three-dimensional data 122 acquired by the three-dimensional scanner 200 and the identification result 124 indicating the type of the machined portion by the identification process executed based on the three-dimensional data 122. The identification result 124 is stored in the storage 110 in association with the three-dimensional data 122 used for the identification process. The training data set 116 is a group of training data used in the training process of the processed species estimation model 114. The color classification data 118 is data used for generation and training processing of the training data set 116. The identification program 120 is a program for executing the identification process. The learning program 121 is a program for executing the learning process of the processing type estimation model 114, and a part thereof includes a program for executing the identification process.

The arithmetic unit 130 is an arithmetic unit that executes various processes such as identification processing and learning processing by executing various programs, and is an example of a computer. The arithmetic unit 130 is composed of, for example, a CPU (Central Processing Unit) 132, an FPGA (Field-Programmable Gate Array) 134, a GPU (Graphics Processing Unit) 136, and the like.

[Identification processing by the identification device 100]
An example of the identification process by the identification device 100 according to the first embodiment will be described with reference to FIGS. 5 and 6. FIG. 5 is a schematic diagram showing a functional configuration of the identification device 100 according to the first embodiment. FIG. 6 is a schematic diagram for explaining the identification process by the identification device 100 according to the first embodiment.

As shown in FIG. 5, the identification device 100 has an input unit 1102, a processing type identification unit 1132, and an output unit 1103 as functional units related to the identification process. Each of these functions is realized by the arithmetic unit 130 of the identification device 100 executing the OS 127 and the identification program 120.

The three-dimensional data acquired by the three-dimensional scanner 200 is input to the input unit 1102. The machining type identification unit 1132 uses the machining type estimation model 114 (learned model 115) based on the three-dimensional data input to the input unit 1102, and the portion corresponding to the input three-dimensional data is the machining portion. In addition to identifying whether or not it is a machined part, an identification process for identifying the type of the machined part when it is a machined part is executed.

The processing type estimation model 114 includes a first neural network (NNW) 1142 and a parameter 1144 used by the first NNW 1142. The parameter 1144 includes a weighting coefficient used in the calculation by the first NNW 1142 and a determination value used in the determination of identification. The output unit 1103 outputs the identification result by the processing type identification unit 1132 to the display 300.

Here, as shown in FIG. 6, the three-dimensional data input to the input unit 1102 includes three-dimensional position information at each point constituting the shape in the oral cavity and each point constituting the shape in the oral cavity. Contains color information. In the identification process according to the first embodiment, the position information is used. The position information includes the coordinates of the absolute position in three dimensions with respect to the predetermined position. For example, the position information includes the coordinates of the absolute position in each of the X-axis, Y-axis, and Z-axis with the predetermined position in the oral cavity as the origin. The origin may be, for example, a predetermined measurement point at the start of measurement as the origin. The position information is not limited to the coordinates of the absolute position in three dimensions with respect to the predetermined position, and may include, for example, the coordinates of the relative position in three dimensions indicating the distance from the adjacent points.

If the condition is in the middle of treatment, such as an implant body or a tooth cavity formed in a tooth, the user 1 can identify the processing site at a glance. However, in the field of dentistry, from the viewpoint of ensuring aesthetics, the development of processed parts having excellent aesthetics comparable to those of natural teeth is progressing. Therefore, it is gradually becoming difficult for the user 1 to identify the processed portion in the state after the treatment is completed.

Although it is difficult for human eyes to distinguish between artificial and natural products, there may be slight differences in surface irregularities and overall shape. Further, when the tooth is covered with the prosthesis, there may be a difference in the shape of the boundary between the gingiva and the prosthesis as compared with the case where the prosthesis is not covered.

The identification device 100 uses the machined type estimation model 114 based on the three-dimensional data in which the characteristic shape and size of the machined part are quantified, and whether or not the part corresponding to the three-dimensional data is the machined part. And, if it is a machined part, identify the type of machined part.

In this way, by using AI, it is possible to find out the characteristics of the processed product that are difficult for the human eye to identify. Then, the machined portion can be accurately identified without relying on the information that varies from person to person, such as the knowledge of the user 1. Further, by identifying the type of the processed portion, it is possible to assist the user 1 in inputting the medical record, and it is possible to facilitate the medical examination at the time of the first medical examination.

Further, the prosthesis is classified into a plurality of types according to the covering range, the covering method, and the like. The type of prosthesis to be placed on the tooth cavity and abutment tooth differs depending on the shape of the tooth cavity and the abutment tooth, and the treatment content to be performed may differ. In addition, the course of treatment varies depending on the type of prosthesis covered. However, in order to identify the type of prosthesis, the high knowledge of user 1 is required. In addition, the user 1 needs to observe and identify the narrow oral cavity, which increases the burden on the subject 2.

More detailed examination can be realized if the type of machined part can be identified in more detail, such as the type of prosthesis attached and the type of prosthesis to be attached to the tooth cavity or abutment tooth.

As shown in FIG. 5, the processed species estimation model 114 includes the first NNW 1142. In the first NNW 1142, the value of the position information included in the three-dimensional data input to the input unit 1102 is input to the input layer. Then, in the first NNW 1142, for example, the intermediate layer multiplies the input position information value by a weighting coefficient, adds a predetermined bias, and performs a calculation by a predetermined function, and the calculation result is obtained. Is compared with the judgment value. Then, in the first NNW 1142, the result of the calculation and the determination is output from the output layer as the identification result. Regarding the calculation and determination by the first NNW1142, it is possible to identify whether or not the part corresponding to the three-dimensional data is a machined part based on the three-dimensional data, and if it is a machined part, the type of the machined part can be identified. If so, any method may be used.

In the first NNW 1142 of the processing type estimation model 114, the intermediate layer has a multi-layer structure, so that processing by deep learning is performed. In the first embodiment, examples of the identification program 120 that performs identification processing specialized for a three-dimensional image include VoxNet, 3DShapeNets, Multi-View CNN, RotationNet, OctNet, FusionNet, PointNet, PointNet ++, SSCNet, and MarrNet. Although used, other programs may be used. Further, the existing mechanism of the first NNW 1142 may be applied.

In such a configuration, when the three-dimensional data corresponding to the three-dimensional image including a plurality of teeth is input, the identification device 100 determines the characteristics of the processed portion based on the three-dimensional data in the processed type estimation model 114. The type of the processed portion can be identified based on the extracted characteristics obtained by extracting using the first NNW1142 of the above. The identification device 100 uses the first NNW1142 to extract the characteristics of the unprocessed biological part (for example, natural tooth and gingiva) in addition to the characteristics of the processed part, and based on the extraction result, the processed part is used. It may be identified whether or not it exists.

In the first embodiment, the identification device 100 uses the first NNW1142 to extract the characteristics of the processed portion, and identifies the type of the processed portion from the extracted characteristics.

[Generation of training data]
An example of generating learning data will be described with reference to FIGS. 7 and 8. FIG. 7 is a schematic diagram for explaining the generation of learning data according to the first embodiment. FIG. 8 is a schematic diagram for explaining an example of learning data according to the first embodiment.

As shown in FIG. 7, first, three-dimensional data is acquired by the three-dimensional scanner 200 (STEP 1). The three-dimensional data acquired by the three-dimensional scanner 200 includes three-dimensional position information at each point constituting the shape in the oral cavity corresponding to the three-dimensional data, and color information (RGB values) at each point. Is done. When a three-dimensional image is generated based on the three-dimensional data acquired by the three-dimensional scanner 200, a three-dimensional image including the actual colored teeth is generated as shown in FIG. 7 (a).

Next, a noise reduction process is performed in preparation for the color-coded process of each tooth, which will be described later. For example, in the first embodiment, the three-dimensional image corresponding to the three-dimensional data is grayscaled (STEP 2). Grayscale of the three-dimensional image is performed by user 1 (in this case, an engineer of a manufacturer or a worker of a manufacturing factory that generates learning data). When the three-dimensional image is grayscaled, a grayscaled three-dimensional image is generated as shown in FIG. 7 (b). Further, the color information (RGB value) at each point corresponding to the three-dimensional data is changed to the value corresponding to the gray scale according to the gray scale of the three-dimensional image.

Next, by applying a predetermined color to each part included in the three-dimensional image corresponding to the three-dimensional data, each part is color-coded according to the type of the processed part (STEP 3). The color to be applied is pre-assigned to each part. The allocation contents are stored in the storage 110 as the color classification data 118. Further, in the example shown in FIG. 7, it is assumed that the unprocessed portion is coated with white that is not assigned to any type of processed portion. The color classification data 118 may include information about colors assigned to unprocessed portions.

The application of color to each part is performed by user 1 (particularly, a person having dental knowledge such as a dentist or a dental technician). Specifically, the user 1 identifies whether or not each part included in the three-dimensional image is a processed part based on his / her own knowledge, and if it is a processed part, identifies the type of the processed part, and determines the identified part. The corresponding color is specified with reference to the color classification data 118, and the specified color is applied to the image of the relevant portion.

For example, when the user 1 identifies that the part included in the three-dimensional image is an unprocessed biological part, the user 1 applies white to the image of the part. Further, when the user 1 identifies that the site included in the three-dimensional image is an implant, the user 1 applies the color a to the image of the site. Further, when the user 1 identifies that the portion included in the three-dimensional image is a prosthesis, the user 1 applies the color b to the image of the portion. When a color is applied to each tooth and gingiva included in the three-dimensional image, a three-dimensional image in which a predetermined color is applied to each part as shown in FIGS. 7 (c) and 8 is obtained. Generated. For the sake of clarity, the colors a and b are represented by hatching on the drawing.

Further, the color information (RGB value) at each point of the tooth corresponding to the three-dimensional data is changed to the value corresponding to the applied color according to the color coding of each part. As a result, predetermined color information (RGB values) is associated with each point corresponding to the three-dimensional data.

When predetermined color information is associated with each part, the three-dimensional data includes position information and color information corresponding to the applied color, and such three-dimensional data is obtained. It is adopted as learning data. That is, in the learning data according to the first embodiment, the color information indicating which part the position information corresponds to is associated with the position information referred to in the identification process (labeled). Ru). A plurality of such learning data are generated, and a collection of the generated plurality of learning data is held in the identification device 100 as a learning data set 116.

In the first embodiment, the user 1 manually applies a color to each part included in the three-dimensional image based on his / her own knowledge, but the user 1 confirms the treatment history of the subject 2 and the like. Color may be manually applied to each part included in the 3D image. It is also possible to supplement some work with software.

Further, in the first embodiment, grayscale is performed as the noise reduction process, but the noise reduction process may not be performed, and may be realized by a process different from the grayscale process.

Although the color information is associated with each position information as the identification information which is the correct answer data, the identification information is not limited to the color information. For example, learning data may be created by inputting correct answer data by applying a color and then converting the input correct answer data into identification information different from the color information. In this case, it is preferable to use information having a data amount lower than the RGB value as the correct answer data. As a result, the correct answer data can be associated with the user 1 in an intuitively easy-to-understand operation of applying a color, and the amount of learning data can be reduced.

The method of generating the correct answer data shown in FIGS. 7 and 8 is an example. For example, the color corresponding to the type of the processed portion may be applied only to the portion identified as the processed portion, and the three-dimensional data of the portion to which the color is not applied may not be used as the correct answer data.

Further, in the example shown in FIG. 8, the processed parts were sorted according to the major classifications such as “implant”, “cavity cavity”, “abutment tooth”, and “prosthesis”, but the minor classifications shown in Table 1 above. You may sort according to.

[Learning process of identification device 100]
The learning process executed by the identification device 100 will be described with reference to FIG. FIG. 9 is a flowchart for explaining an example of the learning process for the processing type executed by the identification device 100 according to the first embodiment. Each step shown in FIG. 9 is realized by the arithmetic unit 130 of the identification device 100 executing the OS 127 and the learning program 121.

As shown in FIG. 9, the identification device 100 selects learning data to be used for learning from the learning data set 116 (S1). The identification device 100 may select one learning data or a plurality of learning data from the learning data set 116. Further, the identification device 100 is not limited to the one that automatically selects the learning data, and the learning data selected by the user 1 may be used for the learning process.

The identification device 100 inputs the position information of the three-dimensional data included in the selected learning data into the processing type estimation model 114 (S2). At this time, the identification device 100 does not input the correct answer data labeled in the three-dimensional data. The identification device 100 extracts the features of the three-dimensional data using the processing type estimation model 114, and executes an identification process for identifying the type of the processing portion of the portion corresponding to the three-dimensional data (S3).

The identification device 100 updates the parameter 1144 of the processing type estimation model 114 based on the error between the identification result by the identification process and the correct answer data corresponding to the learning data used in the learning process (S4).

For example, the identification device 100 estimates the color information corresponding to the portion indicated by the input position information as a result of identification based on the input position information. The identification device 100 compares the color information (correct answer data) corresponding to the portion indicated by the input position information included in the learning data with the color information estimated by itself, and if they match, the processing type estimation model 114 While maintaining the parameter 1144 of the above, if the answer is incorrect, the parameter 1144 of the machining type estimation model 114 is updated so that the two match.

Next, the identification device 100 determines whether or not learning has been performed based on all the learning data (S5). When the identification device 100 has not learned based on all the learning data (NO in S5), the identification device 100 returns to the process of S1.

On the other hand, when the identification device 100 learns based on all the learning data (YES in S5), the discriminator 100 stores the trained processed species estimation model 114 as the trained model 115 (S6), and ends this process.

As described above, the identification device 100 uses the color information associated with the three-dimensional data included in the learning data as the correct answer data, and uses the processing type estimation model 114 based on the identification result by the identification process using the three-dimensional data. By training, a trained model 115 is generated.

[Service provision process of identification device 100]
The service provision process executed by the identification device 100 will be described with reference to FIG. 10. FIG. 10 is a flowchart for explaining an example of the service providing process executed by the identification device 100 according to the first embodiment. Each step shown in FIG. 10 is realized by the arithmetic unit 130 of the identification device 100 executing the OS 127 and the identification program 120. By executing the service provision process, the identification result is associated with the three-dimensional data acquired by the three-dimensional scanner 200.

As shown in FIG. 10, the identification device 100 determines whether or not three-dimensional data corresponding to a predetermined point has been input (S11). For example, the identification device 100 determines that the three-dimensional data corresponding to a predetermined point has been input when the storage 110 stores the three-dimensional data not associated with the identification result.

When the three-dimensional data is sent from the three-dimensional scanner 200, the identification device 100 stores the three-dimensional data in the storage 110. The service provision process is executed in an arbitrary call cycle, and is repeatedly executed until the identification result is associated with all the three-dimensional data stored in the storage 110. When the identification result is associated with all the three-dimensional data stored in the storage 110, the identification device 100 determines that the three-dimensional data corresponding to the predetermined point is not input (NO in S11). ), End the service provision process.

On the other hand, when the three-dimensional data corresponding to a predetermined point is input (YES in S11), the identification device 100 inputs the three-dimensional data (position information) to the trained model 115 and uses the trained model 115. The identification process for identifying the type of the machined portion is executed (S12).

The identification device 100 associates the identification result obtained by the identification process with the three-dimensional data input in S11 (S13). Specifically, the identification device 100 associates the position information included in the three-dimensional data with the color information indicating the identification result (color a to color d, or color information indicating white). As a result, the identification result indicating the state (whether or not the processing is performed, the processing content) of the portion corresponding to the point is associated with the point corresponding to the input three-dimensional data.

The identification device 100 outputs the result of the association to the outside (for example, the display 300) (S14). Specifically, the result of the association with respect to the points corresponding to the input three-dimensional data is displayed on the display 300. After that, the identification device 100 ends this process.

[Example of display output to display 300 during measurement]
An example of the display output to the display 300 during the measurement will be described with reference to FIGS. 11 and 12. FIG. 11 is a diagram showing an example of output during measurement. FIG. 12 is a diagram showing a modified example of the output example during measurement.

The identification device 100 executes a service provision process and associates with the three-dimensional data based on the identification result at the timing when the three-dimensional data corresponding to a predetermined point is input. The user 1 scans the three-dimensional scanner 200 in sequence while gradually moving it in the oral cavity. Therefore, when the three-dimensional scanner 200 is moved to acquire new three-dimensional data, the identification device 100 associates the identification result with each of the plurality of points corresponding to the acquired three-dimensional data. The three-dimensional data associated with the identification result is output to the display 300 in a manner indicating the associated identification result.

For example, as shown in FIG. 11A, when an unprocessed natural tooth is scanned, each of the plurality of points corresponding to each of the three-dimensional data obtained by scanning is displayed in white. In FIG. 11A, the points output in white are represented by triangles for convenience.

After that, when a biological part including a newly processed part is scanned, as shown in FIG. 11B, among a plurality of points corresponding to each three-dimensional data obtained by scanning, the tertiary part corresponding to the implant is obtained. The original data is displayed as a point of color a (black circle in the figure), while the 3D data corresponding to the unprocessed part is represented by a white point (triangular point in the figure). Will be done. Although not shown in FIG. 11, the three-dimensional data corresponding to other types of processed parts such as the prosthesis, the cavity-forming tooth, and the abutment tooth are displayed as colored dots corresponding to each type. ..

The identification device 100 is supposed to display a plurality of points corresponding to the three-dimensional data on the display 300 at each point, but the obtained three-dimensional data is displayed on the display 300 using a polygon mesh. May be good. A polygon mesh is a method of displaying an object by arranging polygons on the screen, such as triangles and quadrangles.

Specifically, as shown in FIG. 12, the identification device 100 replaces a plurality of points corresponding to the input three-dimensional data with polygons, and displays the result of associating the identification results with the plurality of points. It may be displayed at 300.

In this way, since the identification results for the acquired 3D data are sequentially displayed on the display 300 at the timing when the 3D data is acquired, the user 1 corresponds to the 3D data input at the input timing. You can check the type of part to be used. Further, since the identification device 100 displays on the display 300 in a display mode according to the type of the processed portion, the boundary of each processed portion can be shown to the user 1 in an easy-to-understand manner.

At the beginning of the intraoral scanning by the three-dimensional scanner 200, the number of obtained three-dimensional data is small, so that the identification device 100 cannot produce an identification result based on the three-dimensional data. Therefore, the identification device 100 may display an image (point or polygon) corresponding to the three-dimensional data on the display 300 in a manner indicating that the association is not established, that is, the identification result cannot be obtained. Then, at the timing when the scan in the oral cavity by the three-dimensional scanner 200 progresses and the number of the obtained three-dimensional data increases, the service provision process may be executed again to try to obtain the identification result. .. In this case, when the identification result can be obtained, the identification device 100 switches the display mode of the image corresponding to the three-dimensional data already displayed to the mode corresponding to the identification result.

As described above, the identification device 100 according to the first embodiment finds the characteristics of the workpiece that are difficult for the human eye to identify by using AI, and the accuracy does not depend on the knowledge of the user, which varies from person to person. The type of machined part can be identified well.

<Embodiment 2>
The identification device 100a according to the second embodiment will be described with reference to FIGS. 13 to 20. In the first embodiment, the user 1 generates a medical record by referring to the identification result. The identification device 100a according to the second embodiment may further automatically generate a medical record according to the identification result.

The identification device 100a identifies the type of tooth in addition to identifying the type of the processed portion as compared with the identification device 100. As a result, the identification device 100a estimates the type of tooth corresponding to the processed portion. Further, the identification device 100a estimates the treatment content applied to the processed portion based on the identification result indicating the type of the processed portion. In this way, the identification device 100a generates a medical record of the subject 2 by estimating the type of tooth corresponding to the processed portion and the treatment content applied to the processed portion.

The "types of teeth" include middle premolars on the right side of the upper jaw, lateral cut teeth, dog teeth, first premolars, second premolars, first premolars, second premolars, third premolars, and left side of the upper jaw. Middle premolars, lateral premolars, dog teeth, first premolars, second premolars, first premolars, second premolars, and third premolars, right middle premolars, lateral premolars, dog teeth, 1st premolar, 2nd premolar, 1st premolar, 2nd premolar, and 3rd premolar, left middle premolar, lateral incision, dog tooth, 1st premolar, 2nd premolar, 1st It means the type of each tooth such as 1 premolar, 2nd premolar, and 3rd premolar.

[Configuration that works during measurement]
FIG. 13 is a schematic diagram showing a functional configuration that functions during measurement of the scanner system 10a according to the second embodiment. With reference to FIG. 13, the scanner system 10a differs from the scanner system 10 according to the first embodiment in that the identification device 100a is provided in place of the identification device 100. Further, the identification device 100a is different from the identification device 100 according to the first embodiment in that the tooth type identification unit 1134 is further provided and the profile data 119 is used for the identification process in addition to the three-dimensional data.

The profile data 119 is information showing a profile such as age, gender, race, height, weight, and place of residence of the subject 2. The profile data 119 is not necessarily necessary information, but is information used to improve the accuracy in identifying the type of tooth. Further, the profile data 119 is information obtained from the subject 2 at the time of the first medical examination, and is included in the medical record information.

The tooth type identification unit 1134 identifies the tooth type using the tooth type estimation model 114a (learned model 115a) based on the three-dimensional data input to the input unit 1102 and the profile data 119. The tooth species estimation model 114a includes a second NNW 1142a and a parameter 1144a used by the second NNW 1142a. Since the second NNW 1142a and the parameter 1144a are common to the first NNW 1142 and the parameter 1144 described in the first embodiment, the description thereof will be omitted.

Since the processed part is a part that has been processed for a living body part, its shape characteristics and the like are different from those of natural teeth. Therefore, when the tooth type identification unit 1134 performs an identification process for identifying the tooth type based on the three-dimensional data obtained by scanning the processed portion, the tooth type may not be identified. In the present embodiment, the identification device 100a processes the processing portion assuming that the portion corresponding to the three-dimensional data input to the input unit 1102 is the processing portion when the tooth type cannot be identified by the tooth type identification unit 1134. The species identification unit 1132 executes an identification process for identifying the type of the machined portion.

The output unit 1103 outputs the identification results of each of the tooth type identification unit 1134 and the processing type identification unit 1132 to the display 300.

Further, the storage 110 included in the identification device 100a shows the three-dimensional data 122, the identification result 124a indicating the type of tooth of the portion corresponding to the three-dimensional data 122, and the type of the processed portion corresponding to the three-dimensional data 122. The identification result 124 is stored.

[Configuration that works after measurement]
A function related to the generation of a medical record will be described with reference to FIGS. 14 and 15. FIG. 14 is a schematic diagram showing a functional configuration that functions when a medical record is generated after the measurement of the identification device 100a according to the second embodiment. FIG. 15 is a schematic diagram for explaining a method of estimating the type of tooth at the processing site.

With reference to FIG. 14, as a function for generating a medical record after measurement, the identification device 100a includes a treatment estimation unit 1152, a tooth type estimation unit 1154, and a medical record generation unit 1156.

The treatment estimation unit 1152 estimates the treatment content applied to the processed site based on the identification result 124 indicating the type of the processed site.

The processing type identification unit 1132 identifies the type of the processing portion of the portion corresponding to the three-dimensional data. Processing on biological parts is performed for treatment. Therefore, there is a correspondence between the type of processed part and the treatment content. Table 2 shows the relationship between the type of processed part and the treatment content.

With reference to Table 2, the "processed site" can be classified into a processed site during treatment and a processed site that has been treated. For example, an "implant", a "cavity-forming tooth" and an "abutment tooth" are processed sites being treated and a "prosthesis" is a treated processed site. Since the type of the processed site and the type of the treatment content correspond to each other, the treatment estimation unit 1152 can estimate the treatment content performed on the subject 2 based on the identification result 124 indicating the type of the processed site. .. Further, the processed portion is classified into a processed portion during treatment and a processed portion that has been treated. Therefore, the treatment estimation unit 1152 can estimate whether the treatment is in the middle or the treatment is completed based on the identification result 124 indicating the type of the processed portion.

The treatment estimation unit 1152 may estimate the treatment content by referring to a table as shown in Table 2, and may perform deep learning using the identification result 124 indicating the type of the processed portion as an input value to perform the identification result 124. The treatment content corresponding to the above may be estimated. When estimating the treatment content by deep learning, the treatment estimation unit 1152 uses a neural network in which parameters are optimized by using teacher data associated with the treatment content as correct answer data for the type of processed part. Estimate the treatment content.

The tooth type estimation unit 1154 estimates the type of tooth corresponding to the processed portion. The tooth type estimation unit 1154 estimates the type of tooth corresponding to the processed portion from the type of the tooth corresponding to the processed portion or the adjacent tooth. The "tooth corresponding to the processed part" is not limited to the tooth to which the prosthesis is attached, but means the tooth of the original natural tooth when an artificial tooth is attached instead of the natural tooth. Further, the “tooth corresponding to the processed portion” means a tooth that was present before the implant was attached, if the implant was attached.

The medical record generation unit 1156 generates a medical record 150 based on the type of tooth corresponding to the processed portion estimated by the tooth type estimation unit 1154 and the treatment content estimated by the treatment estimation unit 1152.

The chart 150 includes a profile display area 152 showing basic information of the subject 2 specified from the profile data 119, a tooth type display area 154 showing the type of tooth corresponding to the processing site, and the displayed tooth. It includes a comment display area 156 showing the treatment content and a dentition display area 158 showing the dentition. In addition, the chart generation unit 1156 is a two-dimensional image including an image of a tooth including a processed portion viewed from an arbitrary viewpoint based on three-dimensional data in the tooth type display area 158 by changing to the tooth type or together with the tooth type. May be displayed.

With reference to FIG. 15, a method of estimating the type of tooth corresponding to the processed portion will be specifically described. First, three-dimensional data is acquired by the three-dimensional scanner 200 (STEP 1). Next, the identification device 100a identifies the type of tooth by the identification process by the tooth type identification unit 1134 (STEP 2). For example, in the example shown in STEP 2 of FIG. 15, the identification device 100a identifies the type of each tooth located on the right side of the mandible and the right side of the maxilla. Specifically, the identification device 100a uses the identification process to display the second molars, the first molars, the second premolars, and the first premolars on the right side of the lower jaw, and the second molars and the second premolars on the right side of the upper jaw. Identify the two premolars and the first premolar, respectively.

Here, when the processed portion is present in the oral cavity, the tooth type identification unit 1134 of the identification device 100a may not be able to identify the type of tooth corresponding to the processed portion by the identification process. However, the tooth type identification unit 1134 can estimate the types of adjacent teeth adjacent to the tooth corresponding to the processed portion and opposed teeth facing the tooth corresponding to the processed portion by identification processing. The tooth type estimation unit 1154 estimates the type of tooth corresponding to the processing site based on the types of adjacent teeth and opposite teeth (STEP 3).

For example, in the example shown in STEP 3 of FIG. 15, the tooth type identification unit 1134 cannot identify the type of tooth corresponding to the processing site between the second molar and the second premolar on the right side of the maxilla. However, the tooth species identification unit 1134 at least identifies the second molar and the second premolar on the right side of the maxilla. Therefore, the tooth species estimation unit 1154 can presume that the tooth between them is the maxillary right first molar. Alternatively, the tooth species identification unit 1134 at least identifies the first molar on the right side of the mandible facing the tooth corresponding to the machined site. Therefore, the tooth type estimation unit 1154 can presume that the tooth facing the first molar on the right side of the mandible is the first molar on the right side of the upper jaw.

If the tooth type estimation unit 1154 and the type of the processed part are a prosthesis attached to a part of the tooth such as an inlay, an inlay, or a crown, the tooth around the three-dimensional data identified as the processed part. The type of tooth corresponding to the machined portion may be identified based on the three-dimensional data in which the type of tooth is identified. For example, it is presumed that the type of tooth corresponding to the three-dimensional data of the processed portion is the same as the type of tooth associated with the three-dimensional data of the teeth surrounding the three-dimensional data of the processed portion.

[Generation of tooth type estimation model 114a (learned model 115a)]
A learning process for generating the trained model 115a will be described with reference to FIGS. 16 to 18. FIG. 16 is a schematic diagram for explaining an example of the learning data set 116a according to the second embodiment. FIG. 17 is a schematic diagram for explaining the generation of the trained model 115a based on the learning data set 116a according to the second embodiment. FIG. 18 is a flowchart for explaining an example of the learning process for the tooth type executed by the identification device 100a according to the second embodiment. The learning process for generating the trained model 115 of the processed species estimation model 114 is common to the learning process for generating the trained model 115 described in the first embodiment (see FIG. 9). Omit.

When generating learning data to identify the type of tooth, user 1 (especially a person with dental knowledge such as a dentist or a dental technician) checks the type of tooth while checking the three-dimensional image. Is identified, and the position information and the identification information are associated with each other by applying a color corresponding to the type of the identified tooth to the three-dimensional image. The color to be applied is assigned in advance for each type of tooth. The assigned contents are stored in the storage 110 as the color classification data 118a.

For example, when the user 1 identifies that the tooth included in the three-dimensional image is a second molar, the user 1 applies red color to the image of the tooth. Further, when the tooth included in the three-dimensional image is identified as the first molar, green is applied to the image of the tooth. When a predetermined color is applied to each tooth included in the three-dimensional image, a three-dimensional image in which the predetermined color is applied to each tooth is generated as shown in FIG. For the sake of clarity, each color is represented by hatching on the drawing.

When predetermined color information (identification information) is associated with each tooth, the three-dimensional data includes position information and color information (identification information) corresponding to the applied color. , Such three-dimensional data is adopted as learning data. That is, in the learning data of the tooth type estimation model 114a according to the second embodiment, the color information corresponding to the tooth type is associated (labeled) with the position information referred to in the identification process. Further, the color information is associated with the three-dimensional data so that the range of each of the plurality of teeth corresponding to the three-dimensional data can be specified. Specifically, the same color information is associated with each position information corresponding to each tooth. Such a collection of learning data is held in the identification device 100a as a learning data set 116a.

In the second embodiment, the user 1 manually applies a color to each tooth included in the three-dimensional image based on his / her own knowledge, but it is also possible to supplement some of the work with software. For example, the boundary between the tooth to be labeled and the tooth adjacent to the tooth and the boundary between the tooth to be labeled and the gingiva may be specified by edge detection. Only the teeth to be labeled can be extracted.

As shown in FIG. 17, the learning data set 116a can be classified into categories based on the profile of the subject 2 to be scanned when the learning data set 116a is generated. For example, age (minors, active generation, elderly), gender (male, female), race (Asian, Western, African), height (less than 150 cm, 150 or more), weight (less than 50 kg, 50 kg). The above) and the place of residence (resident in Japan, living outside Japan) can be assigned a learning data set generated from three-dimensional data including the teeth of the corresponding subject 2. The stratification of each category can be set as appropriate. For example, regarding age, more details such as by a predetermined age difference (in this case, every 3 years), specifically, 0 to 3 years, 4 to 6 years, 7 to 9 years, and so on. Can be stratified into.

The identification device 100a generates a trained model 115a by training a tooth type estimation model 114a using a plurality of learning data sets 116a that can be classified by category. The training data may be duplicated depending on how the categories are classified, but if the training data is duplicated, the tooth type estimation model 114a should be trained using only one of the training data. Just do it.

In general, the shape of teeth varies in characteristics depending on heredity or living environment such as age, gender, race, height, weight, and place of residence. For example, in general, adult permanent teeth are larger than children's deciduous teeth, and both differ in shape. Also, in general, male teeth are larger than female teeth, and their shapes are different. Also, in general, Westerners' teeth tend to have sharp tips to make it easier to bite hard meat and bread, while Japanese teeth have tips to make it easier to grind soft rice and vegetables. Tends to be smooth. Therefore, if the learning process is executed based on the profile data as in the second embodiment, it is possible to generate a learned model that can identify the type of tooth in consideration of heredity or living environment.

As shown in FIG. 18, the identification device 100a performs a learning process for the tooth species in order to generate the trained model 115a. In the learning process for tooth species, the estimation model to be used is the tooth species estimation model 114a, and the information input to the tooth species estimation model 114a is added to the position information of the three-dimensional data to generate learning data. It differs from the learning process for the type of processing (see FIG. 9) executed by the identification device 100 according to the first embodiment in that the profile data of the subject 2 to be scanned is input (S2a).

Specifically, the identification device 100a selects learning data to be used for learning from the learning data set 116a (S1a). The identification device 100a inputs the position information of the three-dimensional data included in the selected learning data and the profile data of the subject 2 to be scanned when generating the learning data into the tooth type estimation model 114a. (S2a). The identification device 100 executes an identification process for identifying the tooth type using the tooth type estimation model 114a based on the characteristics of the tooth corresponding to the three-dimensional data (S3a). In the identification process, the identification device 100 identifies the type of the tooth by using the tooth type estimation model 114a based on the profile data in addition to the three-dimensional data.

The identification device 100a updates the parameter 1144a of the tooth type estimation model 114a based on the error between the identification result of the tooth type identified by the identification process and the correct answer data corresponding to the learning data used in the learning process (S4a). ).

Next, the identification device 100a determines whether or not learning has been performed based on all the learning data (S5a). When the identification device 100a has not learned based on all the learning data (NO in S5a), the identification device 100a returns to the process of S1a.

On the other hand, when the identification device 100a learns based on all the learning data (YES in S5a), the learning tooth type estimation model 114a is stored as the learned model 115a (S6a), and this process ends.

As described above, the identification device 100a uses the identification information (for example, color information) corresponding to the tooth type associated with the three-dimensional data included in the learning data as the correct answer data, and uses the three-dimensional data by the identification process. The trained model 115a can be generated by learning the tooth type estimation model 114a based on the identification result of the tooth type.

Further, since the identification device 100a learns the tooth type estimation model 114a in consideration of the profile data in addition to the learning data in the learning process, the learned model 115a in consideration of the profile of the subject 2 can be generated.

Although the color information is associated with each position information as the identification information which is the correct answer data, the identification information may be information different from the color information such as the name of the tooth or the number of the tooth.

[Service provision process]
FIG. 19 is a flowchart for explaining an example of the service providing process executed by the identification device 100a according to the second embodiment. The service providing process executed by the identification device 100a according to the second embodiment is different from the service providing process executed by the identification device 100 according to the first embodiment shown in FIG. 10 in that S121 to S126 are executed instead of S12. .. Hereinafter, the points different from the service provision process executed by the identification device 100 will be mainly described.

When the three-dimensional data corresponding to a predetermined point is input (YES in S11), the identification device 100a determines whether or not there is profile data (S121). When there is no profile data (NO in S121), the identification device 100a inputs three-dimensional data (position information) into the trained tooth type estimation model 114a (learned model 115a) (S123). On the other hand, when the identification device 100a has profile data (YES in S121), the three-dimensional data (position information) and the profile data are input to the trained tooth type estimation model 114a (trained model 115a) (S122).

After S122 and S123, the identification device 100a performs an identification process for identifying the tooth type using the learned tooth type estimation model 114a (learned model 115a) based on the characteristics of the tooth corresponding to the three-dimensional data. Execute (S124). At this time, when the profile data is input to the trained model 115a in S122, the identification device 100a identifies the type of the tooth by using the trained model 115a based on the profile data in addition to the three-dimensional data.

The identification device 100a determines whether or not the type of tooth can be identified (S125). Whether or not the identification was possible is determined based on, for example, a comparison between the determination value included in the parameter 1144a and the calculation result.

When the identification device 100a can identify the type of tooth (YES in S125), the identification device 100a executes the processing after S13. On the other hand, when the identification device 100a cannot identify the type of tooth (NO in S125), the three-dimensional data is input to the trained processed type estimation model 114 (trained model 115), and the trained model 115 is input. It is used to execute an identification process for identifying the type of the machined portion (S126).

After S125 and S126, the identification device 100a associates the identification result with the three-dimensional data input in S11 (S13). For example, when the identification device 100a can identify the type of tooth, the color information indicating the identification result 124a indicating the type of tooth (color information indicating red, green, etc.) with respect to the position information included in the three-dimensional data. To associate. On the other hand, when the type of the tooth cannot be identified and the type of the processed portion can be identified, the identification device 100a indicates the color information indicating the identification result 124 indicating the type of the processed portion with respect to the position information included in the three-dimensional data. (Color information indicating color a, color b, etc.) is associated.

The identification device 100a outputs the result of the association to the outside (for example, the display 300) (S14). Specifically, with respect to the point corresponding to the input three-dimensional data, the result of the association with the portion corresponding to the point is displayed on the display 300. After that, the identification device 100a ends this process. For example, as in the output example during measurement shown in FIGS. 11 and 12, at the timing when the three-dimensional data is acquired, the identification results for the acquired three-dimensional data are sequentially displayed on the display 300.

As described above, in the second embodiment, when the tooth type identification unit 1134 cannot identify the type of tooth, the identification device 100a executes a process of identifying the type of the processed portion by the processed type identification unit 1132. If the tooth type identification unit 1134 can identify the type of tooth, the processing type identification unit 1132 does not execute the process of identifying the type of the processed portion.

In the service providing process shown in FIG. 19, when the identification device 100a cannot identify the type of tooth, and when it identifies that it is not a processed part in S126, it indicates that it could not be identified in S13. Information may be associated with three-dimensional data.

[Medical record generation process]
FIG. 20 is a flowchart for explaining an example of the medical record generation process executed by the identification device 100a according to the second embodiment. The medical record generation process is performed after the acquisition of the three-dimensional data by the three-dimensional scanner 200 is completed and when the identification process (service provision process) for the three-dimensional data is completed.

In the medical record generation process, the identification device 100a estimates the tooth type corresponding to the processed site and the treatment content applied to the processed site, and records the treatment content applied to the processed site in association with the estimated tooth type. Is output as.

With reference to FIG. 20, the identification device 100a determines whether or not there is a processed portion for which the tooth type and the treatment content have not been estimated (S21). When it is determined that there is no processed part for which the tooth type and treatment content have not been estimated (NO in S21), that is, for all processed parts, the tooth type corresponding to the processed part is estimated and the treatment content is estimated. If it is determined that the estimation has been made, the identification device 100a executes the process of S26 and ends the medical record generation process.

When it is determined that there is a processed part for which the tooth type and the treatment content have not been estimated (YES in S21), the identification device 100a identifies the type of tooth adjacent to the processed part based on the three-dimensional data of the processed part. Obtained from the result 124a (identification result indicating the type of tooth) (S22).

The identification device 100a estimates the type of tooth corresponding to the processed portion from the acquired type of tooth (S23). The identification device 100a estimates the treatment content applied to the processed site based on the identification result 124 indicating the type of the processed site (S24). The identification device 100a associates the estimated treatment content with the estimated tooth type and reflects them in the medical record (S25). After reflecting on the medical record, the identification device 100a executes the processing of S21, repeats S22 to S25 until the type of tooth and the treatment content are estimated for all the processed parts, and then outputs the medical record (S26). .. For example, the identification device 100a displays the chart 150 shown in FIG. 14 on the display 300, or prints the chart 150.

As described above, in the second embodiment, the treatment content applied to the processed portion is estimated, and the type of tooth corresponding to the processed portion is estimated. Then, the medical record is automatically generated by outputting the estimated treatment content and the estimated tooth type in association with each other.

In the second embodiment, the treatment estimation unit 1152 functions after the measurement. The process executed by the treatment estimation unit 1152 may be incorporated in the service provision process shown in FIG. For example, the identification device 100a may execute a process of estimating the treatment content based on the obtained identification result after executing S126.

<Embodiment 3>
The identification device 100b according to the third embodiment will be described with reference to FIGS. 21 to 26. The identification device 100b identifies the lesion site in addition to the identification of the tooth type and the identification of the processed site. In addition, "identifying a lesion site" means identifying whether or not the site corresponding to the three-dimensional data 122 is a lesion site, and if it is a lesion site, identifying the lesion content of the lesion site. means.

The "lesion site" includes a lesion site in the oral cavity, for example, a carious site and a site where symptoms of periodontal disease are observed. Further, the "lesion site" is not limited to a site that is actually diagnosed as having a disease, but may include a site that is likely to cause a disease, such as a site where tartar is accumulated.

[Configuration that works during measurement]
FIG. 21 is a schematic diagram showing a functional configuration that functions during measurement of the scanner system 10b according to the third embodiment. With reference to FIG. 21, the scanner system 10b differs from the scanner system 10a according to the second embodiment in that the identification device 100b is provided in place of the identification device 100a. Further, the identification device 100b is different from the identification device 100a according to the second embodiment in that the lesion identification unit 1136 is further provided. The lesion identification unit 1136 identifies the type of lesion using the lesion estimation model 114b (learned model 115b) based on the three-dimensional data input to the input unit 1102.

The lesion estimation model 114b includes a third NNW1142b and a parameter 1144b used by the third NNW1142b. Since the third NNW 1142b and the parameter 1144b are common to the first NNW 1142 and the parameter 1144 described in the first embodiment, the description thereof will be omitted.

The output unit 1103 outputs the identification results of each of the tooth type identification unit 1134, the processed type identification unit 1132, and the lesion identification unit 1136 to the display 300.

Further, the storage 110 included in the identification device 100b shows the three-dimensional data 122, the identification result 124a indicating the type of tooth of the portion corresponding to the three-dimensional data 122, and the type of the processed portion corresponding to the three-dimensional data 122. The identification result 124 and the identification result 124b indicating the lesion content of the site corresponding to the three-dimensional data 122 are stored.

[Configuration that works after measurement]
With reference to FIG. 22, the function related to the generation of the medical record will be described. FIG. 22 is a schematic diagram showing a functional configuration of the identification device 100b that functions when generating a medical record after measurement.

With reference to FIG. 22, as a function of generating a medical record after measurement, the identification device 100b has a treatment estimation unit 1152, a tooth type estimation unit 1154, and a medical record generation unit 1156, similarly to the identification device 100a according to the second embodiment. To prepare for.

The tooth type estimation unit 1154 according to the third embodiment estimates the type of tooth corresponding to the lesion site in addition to the type of tooth corresponding to the processed site. The tooth type estimation unit 1154 estimates the type of tooth corresponding to the processed site or the lesion site from the type of the tooth corresponding to the processed site or the lesion site or the adjacent tooth. "Tooth corresponding to a lesion site" means a carious tooth or a tooth in the vicinity of the gingiva having a lesion. The function of the treatment estimation unit 1152 is common to the treatment estimation unit 1152 included in the identification device 100a according to the second embodiment.

The medical record generation unit 1156 includes the type of tooth corresponding to the processing site estimated by the tooth type estimation unit 1154, the treatment content estimated by the treatment estimation unit 1152 on the processing site, and the lesion estimated by the tooth type estimation unit 1154. A medical record 150 is generated based on the type of tooth corresponding to the site and the lesion content indicated by the identification result 124b.

In the chart 150, a profile display area 152 showing the profile of the subject 2 specified from the profile data 119 and a tooth type display area 154 showing the type of tooth corresponding to the processed site or the lesion site are displayed. It includes a comment display area 156 showing the content of treatment or lesion applied to the tooth and a tooth expression display area 158 showing the tooth type.

The method of estimating the type of tooth corresponding to the lesion site is the same as the method of estimating the type of tooth corresponding to the processed site described in the second embodiment. For example, when the lesion site is a tooth, the tooth type estimation unit 1154 estimates the type of tooth corresponding to the lesion site from the type of the tooth corresponding to the lesion site or the adjacent tooth. Further, when the lesion site is a gingiva, the tooth type estimation unit 1154 estimates the type of tooth corresponding to the lesion site (gingiva) by using the tooth adjacent to the gingiva as the tooth corresponding to the lesion site. ..

[Generation of lesion estimation model 114b (learned model 115b)]
A learning process for generating the trained model 115b will be described with reference to FIGS. 23 and 24. FIG. 23 is a schematic diagram for explaining the generation of the trained model 115b based on the learning data set 116b according to the third embodiment. FIG. 24 is a flowchart for explaining an example of a learning process for a lesion executed by the identification device 100b according to the third embodiment. Since the learning process for generating the trained model 115a of the tooth species estimation model 114a and the trained model 115 of the processed species estimation model 114 is common to the learning process described in the above embodiment, the description thereof will be omitted. ..

With reference to FIG. 23, when generating learning data to identify the lesion content, user 1 (particularly, a person with dental knowledge such as a dentist or a dental technician) displays a three-dimensional image. While confirming, the lesion site is identified, and the position information and the identification information are associated with each other by applying a color corresponding to the identified lesion content to the three-dimensional image. The color to be applied is assigned in advance for each lesion content. The allocation contents are stored in the storage 110 as color classification data 118b.

For example, when the user 1 identifies that the site included in the three-dimensional image is a tooth without a lesion, the user 1 applies white color to the image of the tooth. Further, when the user 1 identifies that the portion included in the three-dimensional image is the initial dental caries, the user 1 applies the color A to the image of the portion. For the sake of clarity, the colors A and C are represented by hatching on the drawing.

When the position information is associated with the color information (identification information) indicating the lesion content as the correct answer data, the three-dimensional data includes the position information and the color information (identification information) corresponding to the applied color. Then, such three-dimensional data is adopted as learning data. That is, in the learning data of the lesion estimation model 114b according to the third embodiment, the color information corresponding to the lesion content is associated (labeled) with the position information referred to in the identification process. Further, color information is associated with the three-dimensional data so that the range of the lesion corresponding to the three-dimensional data can be specified. Specifically, the same color information is associated with each position information corresponding to each lesion site. Such a collection of learning data is held in the identification device 100b as a learning data set 116b.

In the third embodiment, the user 1 manually applies the color to each part included in the three-dimensional image based on his / her own knowledge, but the user 1 manually applies the color while checking the chart of the target person 2. Color may be applied to each part included in the three-dimensional image by work. It is also possible to supplement some work with software. For example, the color to be applied may be automatically selected based on the medical record.

The identification device 100b generates the trained model 115b by training the lesion estimation model 114b using the one or a plurality of learning data sets 116b thus generated.

The classification of the lesion content shown in the color classification data 118b in FIG. 23 is an example. For example, an example was shown in which teeth are classified into three categories: no abnormality, initial dental caries, and dental caries that require treatment. It may be classified by the index of. Similarly, although an example of classifying the gingiva into two categories, normal and periodontal disease, may be further classified into a plurality of stages indicating the degree of progression. Specifically, it may be classified into gingival inflammation, mild periodontitis, moderate periodontitis, and severe periodontitis.

As shown in FIG. 24, the discriminator 100b performs a learning process for the lesion to generate the trained model 115b. In the learning process for lesions, the estimation model to be used is the lesion estimation model 114b, and the learning data to be input is the learning data set 116b. It is different from the processing (see FIG. 9).

Specifically, the identification device 100b selects learning data to be used for learning from the learning data set 116b (S1b). The identification device 100b inputs the position information of the three-dimensional data included in the selected learning data into the lesion estimation model 114b (S2b). The identification device 100b extracts the characteristics of the three-dimensional data using the lesion estimation model 114b, identifies whether or not the site corresponding to the three-dimensional data is the lesion site, and if it is the lesion site, determines the lesion content. The identification process for identification is executed (S3b).

The identification device 100b updates the parameter 1144b of the lesion estimation model 114b based on the error between the identification result by the identification process and the correct answer data corresponding to the learning data used in the learning process (S4b).

Next, the identification device 100b determines whether or not learning has been performed based on all the learning data (S5b). When the identification device 100b has not learned based on all the learning data (NO in S5b), the identification device 100b returns to the process of S1b.

On the other hand, when the identification device 100b learns based on all the learning data (YES in S5b), the learned lesion estimation model 114b is stored as the learned model 115b (S6b), and this process ends.

[Service provision process]
FIG. 25 is a flowchart for explaining an example of the service providing process executed by the identification device 100b according to the third embodiment. The service providing process executed by the identification device 100b according to the third embodiment is different from the service providing process executed by the identification device 100a according to the second embodiment shown in FIG. 19 in that S127 is further executed. Hereinafter, the points different from the service providing process executed by the identification device 100a according to the second embodiment will be mainly described.

When the identification device 100b cannot identify the type of tooth (NO in S125), the three-dimensional data is input to the trained processed species estimation model 114 (trained model 115), and the trained model 115 is used. An identification process for identifying the type of the machined portion is executed (S126).

Further, the identification device 100b inputs three-dimensional data into the trained lesion estimation model 114b (learned model 115b), and executes the discrimination process for discriminating the lesion content using the trained model 115b (S127).

After S125 and S127, the identification device 100b associates the identification result with the three-dimensional data input in S11 (S13). For example, if the type of tooth cannot be identified and the type of processed site cannot be identified in S126 for the site corresponding to the input 3D data, while the type of lesion can be identified in S127, the identification device. Reference numeral 100b associates the position information included in the three-dimensional data with the color information (color information indicating color A, color B, etc.) indicating the identification result 124b. Further, when the type of the tooth cannot be identified, the type of the processed portion can be identified in S126 and the type of the lesion cannot be identified in S127 with respect to the portion corresponding to the input three-dimensional data, the identification device 100b can be used. , Color information indicating the identification result 124 (color information indicating color a, color b, etc.) is associated with the position information included in the three-dimensional data. Further, when the type of the tooth cannot be identified, the type of the processed portion can be identified in S126 and the type of the lesion can be identified in S127 with respect to the portion corresponding to the input three-dimensional data, the identification device 100b Indicates color information indicating the identification result 124 (color information indicating the color a, color b, etc.) and color information indicating the identification result 124b (color A, color B, etc.) with respect to the position information included in the three-dimensional data. Color information) is associated.

Then, the identification device 100b outputs the result of the association to the outside (for example, the display 300) (S14). Specifically, with respect to the point corresponding to the input three-dimensional data, the result of the association with the portion corresponding to the point is displayed on the display 300. After that, the identification device 100b ends this process. For example, as in the output example during measurement shown in FIGS. 11 and 12, at the timing when the three-dimensional data is acquired, the identification results for the acquired three-dimensional data are sequentially displayed on the display 300. When two identification results (for example, the identification result 124 indicating the type of the processed site and the identification result 124b indicating the lesion content) are associated with the position information included in the three-dimensional data, two colors are used. It may be displayed on the display 300 in such a manner that it blinks alternately with.

As described above, in the third embodiment, the identification device 100b identifies the type of tooth at the site corresponding to the three-dimensional data, and if the type of tooth cannot be identified, the treatment applied to the site is performed. The identification process for identifying the content and the identification process for identifying the lesion content at the site are executed.

[Medical record generation process]
FIG. 26 is a flowchart for explaining an example of the medical record generation process executed by the identification device 100b according to the third embodiment. The medical record generation process executed by the identification device 100b according to the third embodiment is different from the medical record generation process executed by the identification device 100a according to the second embodiment shown in FIG. 20 in that S251 to S254 are further executed. Hereinafter, the points different from the medical record generation process executed by the identification device 100a according to the second embodiment will be mainly described.

When the identification device 100b repeats S22 to S25 until the type of tooth and the treatment content are estimated for all the processed parts, and determines that the type of tooth is estimated for all the processed parts (NO in S21). The processing after S251 is executed.

The identification device 100b determines whether or not there is a lesion site for which the tooth type has not been estimated (S251). When it is determined that there is no lesion site for which the tooth type has not been estimated (NO in S251), that is, the treatments S251 to S254 are executed for all the lesion sites to estimate the tooth type corresponding to the lesion site. If is, the identification device 100b executes the process of S26 and ends the medical record generation process.

When it is determined that there is a lesion site for which the tooth type has not been estimated (YES in S251), the identification device 100b identifies the type of tooth adjacent to the lesion site from the identification result 124a based on the three-dimensional data of the lesion site. Acquire (S252).

The identification device 100b estimates the type of tooth corresponding to the lesion site from the acquired type of tooth (S253). The identification device 100b acquires the lesion content of the lesion site from the identification result 124b, associates the acquired lesion content with the estimated tooth type, and reflects it in the medical record (S254).

After reflecting on the medical record, the identification device 100b executes the processing of S251, repeats S252 to S254 until the type of tooth is estimated for all the lesion sites, and then outputs the medical record (S26). For example, the identification device 100a displays the chart 150 shown in FIG. 22 on the display 300, or prints the chart 150.

As described above, the identification device 100b according to the third embodiment also identifies the lesion site in addition to the processed site. Therefore, the identification device 100b can execute more accurate identification processing and can provide a lot of information to the user 1.

For example, when the shape of a hole is significantly changed as the disease progresses, such as a hole formed by the progress of caries, the user 1 can identify the lesion site at first glance. However, the initial caries and the slightly missing parts may not be apparent at first glance. Periodontal disease is diagnosed from swelling and sagging of the gingiva. Therefore, the diagnosis of periodontal disease may vary depending on the knowledge of the user 1.

Therefore, by using AI, it is possible to reduce the variation in diagnosis and to find out the characteristics of the lesion site even if the lesion site is not apparent at first glance.

Further, since the user 1 can confirm the shape of the caries in three dimensions, it is easier to examine the cutting range and the treatment method as compared with the case of observing the inside of the oral cavity. Further, the identification device 100b can estimate the shape of caries by discriminating between the lesion content and the lesion site. Therefore, the identification device 100b can automatically set the cutting range based on the shape of the caries, display the set cutting range on the three-dimensional data of the oral cavity (tooth), and display it on the display 300. .. Further, the identification device 100b can automatically propose a treatment method by discriminating between the lesion content and the lesion site.

Further, when the identification device 100b identifies tartar or plaque and displays it on the display 300, the user 1 can explain the state of the teeth of the subject 2 to the subject 2 in a three-dimensional and easy-to-understand manner. You can easily teach how to brush your teeth.

Further, since the user 1 can three-dimensionally recognize the distance between the gingiva and the tooth by checking the three-dimensional image of the oral cavity displayed on the display 300, the user 1 can diagnose the progress of periodontal disease. Cheap. Further, the degree of progression of periodontal disease may be estimated by the identification device 100b based on the distance between the gingiva and the tooth. Then, the identification device 100b may display the estimated progress of periodontal disease on the display 300.

In addition, gingival swelling is an early sign of periodontal disease. Gingival swelling is a symptom that User 1 can easily overlook. Therefore, the identification device 100b identifies the site where the initial symptom of periodontal disease appears based on the characteristic (swelling) of the shape of the gingiva, so that early detection of periodontal disease is realized.

In the above medical record generation process, the identification device 100b estimated the type of tooth for each processed site and lesion site. The identification device 100b may estimate the type of tooth for each three-dimensional data 122. In the third embodiment, for one three-dimensional data, the site corresponding to the three-dimensional data may be identified as a processed site and a lesion site. In such a case, if the type of tooth is estimated for each processed site and lesion site, the processing will be duplicated. Therefore, the identification device 100b may estimate the tooth type of the three-dimensional data 122 that could not identify the tooth type based on the three-dimensional data 122 that could identify the type of the adjacent tooth.

[Modification example]
<Modification example of identification processing>
The identification device 100 according to the first embodiment identifies whether or not the portion corresponding to the three-dimensional data is a processed portion, and if it is a processed portion, identifies the type of the processed portion. The identification device 100 may further estimate the treatment content as in the second and third embodiments. Further, the identification device 100 may execute the identification process for identifying the tooth type as in the second and third embodiments to estimate the tooth type corresponding to the processed portion based on the tooth type. good.

The identification device 100a according to the second embodiment executes an identification process for identifying the type of tooth, and when the type of tooth cannot be identified, identifies the portion corresponding to the three-dimensional data as a processed portion. , An identification process was performed to identify the type of machined part.

The identification device 100a may execute an identification process for identifying the type of the processed portion and then perform an identification process for identifying the type of the tooth when it is determined that the portion is not the processed portion.

Further, the identification device 100a executes both the identification process for identifying the tooth type and the identification process for identifying the processed portion on each of the obtained three-dimensional data so as to obtain two identification results. You may do it. In this case, the tooth type estimation unit 1154 may estimate the type of tooth corresponding to the site where the tooth type could not be identified based on the type of the adjacent tooth. The order in which each process for obtaining the identification result is executed may be predetermined, and may be either from the identification result 124a regarding the tooth type or from the identification result 124 regarding the processed portion. Further, each process for obtaining the identification result may be executed by task control.

The identification device 100b according to the third embodiment executes an identification process for identifying the type of tooth, and when the type of tooth cannot be identified, the identification process for identifying the type of the processed portion and the identification for identifying the lesion content. The process was executed. The identification device 100b may execute each of the identification process for identifying the type of tooth, the identification process for identifying the treatment content, and the identification process for identifying the lesion content, and obtain three identification results.

Further, the identification device 100b may execute an identification process for identifying the type of tooth when the type of the processed portion cannot be identified and the lesion content cannot be identified.

When a plurality of identification results are obtained for one three-dimensional data, the order in which each process for obtaining the identification results is executed may be predetermined and may be executed in any order. , May be executed by task control.

Further, in the second embodiment, the identification device 100a identifies the type of the processed portion by the processed type estimation model 114, and then estimates the treatment content based on the type of the processed portion. The identification device 100a may estimate the type of the processed portion and the treatment content by using one estimation model. Further, although it is assumed that the identification device 100a has two estimation models, one estimation model may be used to identify the type of tooth and the type of processed portion (or treatment content). Similarly, the identification device 100b according to the third embodiment has identified three types of items using three estimation models, but may identify three types of items using one or two estimation models. ..

Further, in the first to third embodiments, the identification device 100b identifies the processed portion and the treatment content applied to the processed portion. In addition to these, the material of the part used for the processed portion or the structure for identifying the part number of the implant manufacturer may be provided.

<Modification example of learning data set>
In any of the above-described first to third embodiments, the identification process is performed based on the position information included in the three-dimensional data. It should be noted that the identification process may be performed using color information in addition to the position information. FIG. 27 is a schematic diagram for explaining an example of a learning data set according to a modified example. When the color information is input to the trained model in addition to the position information to obtain the identification result, as shown in FIG. 27, the learning process is executed including the color information in the training data. In the learning process, the color information before color coding is input to the estimated model in addition to the position information, so that the trained model is generated after considering the actual color of each part.

The prosthesis is usually attached to the tooth in a shape comparable to that of a natural tooth from an aesthetic and functional point of view. Therefore, it may be difficult to sufficiently identify whether or not the part corresponding to the input three-dimensional data is the part corresponding to the prosthesis only by the position information in which the feature of the shape appears. Since the estimation model can extract color features in addition to shape features, it is possible to identify the machined part more accurately.

In addition, the initial symptoms of periodontal disease include swelling of the gingiva and change in the color of the gingiva. Since the estimation model can extract color features in addition to shape features, periodontal disease can be identified more accurately. In addition, as an initial symptom of dental caries, there are white spots (white spots) on the teeth. The estimation model can extract color features in addition to shape features, so that caries can be identified more accurately.

In any of the above-described first to third embodiments, a learning data set was generated by scanning the oral cavity of the subject 2. The learning data set used to identify the machined portion may be generated by scanning the parts used for machining.

FIG. 28 is a diagram for explaining a learning data set according to a modified example. For example, the training data set may be generated using three-dimensional data obtained by scanning a component used for treatment. For example, the learning data set may be generated from the collected 3D data by collecting the 3D data obtained by scanning the implant body used for implant treatment. Further, the learning data set may be generated from the collected three-dimensional data by collecting the three-dimensional data obtained by scanning the abutment teeth. The identification device may generate a trained model by inputting each training data set generated for each component used in the treatment into the estimation model.

<Modification example of medical record generation part>
In the second and third embodiments, the medical record generation unit 1156 identifies the tooth type estimated by the tooth type estimation unit 1154 and the treatment content or lesion content estimated by the treatment estimation unit 1152 based on the identification result 124. A chart 150 was generated based on the results 124b. The medical record generation unit 1156 does not need to generate the final product of the medical record 150, and may provide, for example, a function of assisting the user 1 in inputting the medical record. Specifically, information indicating a site identified as a processed site or a lesion site (for example, tooth type) may be output, and the user 1 may be prompted to input the medical record 150.

<Modified example of service provision processing>
As shown in FIGS. 10, 19, and 25, the identification devices according to the first to third embodiments do not execute the learning process in the service provision process. However, as shown in FIG. 29, the identification device may execute the learning process in the service providing process. The identification device may be one that performs so-called collaborative learning (Federated Learning).

FIG. 29 is a flowchart for explaining an example of the service providing process executed by the identification device 100c according to the modified example. Since the processes of S11 to S14 shown in FIG. 29 are the same as the processes of S11 to S14 shown in FIG. 10, detailed description of the processes of S11 to S14 will be omitted in FIG. 29. In the following, the identification device 100 according to the first embodiment will be described as executing the learning process in the service providing process.

As shown in FIG. 29, in the identification device 100c, with respect to the point corresponding to the input three-dimensional data by the processing of S11 to S14, the state of the portion corresponding to the point (whether or not it is processed, processing). After outputting the result of the association to the relevant point based on the identification result indicating the content), the learning process at the time of service provision is executed. Specifically, the identification device 100c determines whether or not correct answer data for error correction has been input after S14 (S15). For example, the identification device 100c actually uses the identification device 100c when the identification result (whether or not it is processed, the processing content) obtained by S12 is different from the determination result determined by the user 1 from the portion actually scanned. It is determined whether or not the error has been corrected by the user 1 inputting the correct answer data which is the determination result of the user 1 for the portion to be scanned.

When the correct answer data for error correction is not input (NO in S15), the identification device 100c ends this process. On the other hand, when the correct answer data for error correction is input (YES in S15), the identification device 100c grants a reward based on the identification result and the correct answer data (S16).

For example, the smaller the dissociation degree between the identification result and the correct answer data, the smaller the negative point given as the reward, and the larger the dissociation degree between the two, the larger the negative point given as the reward. Just do it. The reward is not limited to minus points, but may be plus points.

The method of rewarding can be arbitrarily designed. For example, the identification device 100c may give a reward based on a preset size of the reward for each of a plurality of types of combinations that can be considered as a combination of the identification result and the correct answer data. Further, the identification device 100c may give a reward according to a predetermined rule. Specifically, the identification device 100c may give a negative point with a small degree of dissociation and a small value as long as the identification result and the correct answer data have different minor classifications but the major classification (see Table 1) is common. .. On the other hand, the identification device 100c may give a large negative point with a large degree of dissociation if both the minor classification and the major classification are different between the identification result and the correct answer data. Further, the identification device 100c may give a reward by a method in which a reward giving method according to such a rule and a giving method in which a reward is set in advance for each of a plurality of types of combinations are combined. good.

The identification device 100c updates the parameter 1144 of the trained model 115 based on the given reward (S17). For example, the identification device 100c updates the parameter 1144 of the trained model 115 so that the negative points given as rewards approach 0. After that, the identification device 100c ends this process.

Although the learning process executed during the service provision process for identifying only the type of the processed portion has been described, the identification device has a plurality of items (type of the processed portion and teeth) shown in the second and third embodiments. The learning process can be executed even during the service provision process for identifying the type, the type of the processing site, the type of the tooth, and the content of the lesion.

For example, in the service providing process for discriminating between the type of the processed part and the type of tooth, when the correct answer data of another tooth type is input for the identification result in which the tooth type is identified, the discriminating device uses the tooth. The parameter 1144a of the tooth type estimation model 114a is updated based on the identification result 124a indicating the type of the tooth and the correct answer data. In this case, as an example, the identification device gives a small minus point if the tooth output as the identification result 124a and the tooth input as the correct answer data are adjacent to each other, and if they are separated from each other, the value is given. Gives a big negative point.

As another example, in the service providing process for discriminating the type of the processed site, the type of tooth, and the lesion content, when the correct answer data of another lesion content is input for the identified lesion content, the identification device is used. The parameter 1144b of the lesion estimation model 114b is updated based on the identification result 124b indicating the lesion content and the correct answer data. In this case, as an example, the identification device gives a negative point with a small value if the degree of inflammation (the degree of progress of caries or the degree of progression of gingival inflammation) is close, and a negative point with a large value if the degree of inflammation is far. give.

Further, the identification device including a plurality of estimation models may select an estimation model to be retrained according to the combination of the identification result and the correct answer data. For example, if both the identification result and the correct answer data are matters relating to the tooth type, the identification device executes a process of updating the parameter 1144a of the tooth type estimation model 114a. If both the identification result and the correct answer data are matters related to the processing content, the identification device executes a process of updating the parameter 1144 of the processing type estimation model 114. If both the identification result and the correct answer data are matters relating to the lesion content, the identification device executes a process of updating the parameter 1144b of the lesion estimation model 114b. Further, the identification device may update a plurality of parameters from the parameter 1144 of the processed species estimation model 114, the parameter 1144a of the tooth species estimation model 114a, and the parameter 1144b of the lesion estimation model 114b.

As described above, since the identification device according to the modified example also executes the learning process in the service provision process, the accuracy of the identification process is improved as the user 1 uses it, and the type of the processed portion and the tooth are more accurately identified. The type or lesion content can be identified. Such processing is a kind of so-called collaborative learning.

It should be considered that the embodiments disclosed this time are exemplary in all respects and not restrictive. The scope of the present disclosure is shown by the scope of claims rather than the above description, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims. The configurations exemplified in this embodiment and the configurations exemplified in the modified examples can be appropriately combined.

1 user, 2 subjects, 10,10a, 10b scanner system, 100,100a, 100b, 100c identification device, 102 scanner interface, 103 display interface, 105 peripheral device interface, 107 media reader, 108 PC display, 109 memory, 110 storage, 112 scan information, 114 processed species estimation model, 114a tooth species estimation model, 114b lesion estimation model, 115, 115a, 115b trained model, 116, 116a, 116b training dataset, 118, 118a, 118b color classification. Data, 119 profile data, 120 identification program, 121 learning program, 122 three-dimensional data, 124, 124a, 124b identification result, 130 arithmetic unit, 150 chart, 152 profile display area, 154 tooth type display area, 156 comment display Area, 158 Tooth Display Area, 200 Three-Dimensional Scanner, 300 Display, 550 Removable Disk, 601 Keyboard, 602 Mouse, 1102 Input, 1103 Output, 1132 Processing Type Identification, 1134 Tooth Identification, 1136 Disease Identification 1,142 1st NNW, 1142a 2nd NNW, 1142b 3rd NNW, 1144, 1144a, 1144b Parameters, 1152 Treatment estimation unit, 1154 Tooth type estimation unit, 1156 Carte generation unit.

Claims (10)

It is an identification device that identifies the processed part with respect to the living part in the oral cavity.
An input unit for inputting 3D data including 3D position information at each of a plurality of points constituting the shape of the oral cavity.
An identification unit that identifies the processed part based on the three-dimensional data input from the input unit and an estimation model including a neural network for estimating the processed part based on the three-dimensional data.
An identification device including an output unit that outputs an identification result by the identification unit. The identification device according to claim 1, wherein the three-dimensional data includes color information at each of a plurality of points constituting the shape in the oral cavity. The neural network is a neural network for estimating a processed portion based on three-dimensional data input from the input unit and estimating the type of tooth of the portion corresponding to the three-dimensional data.
The identification unit identifies the type of the tooth around the tooth based on the three-dimensional data of the tooth around the tooth corresponding to the processed portion and the estimation model, and identifies the type of the tooth around the tooth.
The identification device according to claim 1 or 2, further comprising a tooth type estimation unit that estimates the type of tooth corresponding to the processed portion based on the type of the peripheral tooth identified by the identification unit. The identification device according to claim 3, wherein the output unit outputs the processed portion identified by the identification unit in association with the type of tooth corresponding to the processed portion estimated by the tooth type estimation unit. The identification device according to any one of claims 1 to 4, further comprising a treatment estimation unit that estimates the treatment content applied to the processing site based on the processing site identified by the identification unit. .. The neural network is a neural network for estimating a processed portion based on three-dimensional data input from the input unit and estimating the type of tooth of the portion corresponding to the three-dimensional data.
The identification unit identifies the type of the tooth around the tooth based on the three-dimensional data of the tooth around the tooth corresponding to the processed portion and the estimation model, and identifies the type of the tooth around the tooth.
A treatment estimation unit that estimates the treatment content applied to the processing site based on the processing site identified by the identification unit, and a treatment estimation unit.
Further provided with a tooth type estimation unit that estimates the type of tooth corresponding to the processing site based on the type of the peripheral tooth identified by the identification unit.
In the output unit, the processing content identified by the identification unit is applied to the processing site estimated by the treatment estimation unit, and the type of tooth corresponding to the processing site estimated by the tooth type estimation unit. The identification device according to claim 1 or 2, which outputs in association with the above. The estimation model is
A first estimation model including a first neural network for estimating a machining site based on three-dimensional data input from the input unit, and a first estimation model.
A second estimation model including a second neural network for estimating the type of tooth at the site corresponding to the three-dimensional data based on the three-dimensional data input from the input unit is included.
When the identification unit cannot estimate the type of tooth of the portion corresponding to the input three-dimensional data based on the three-dimensional data input from the input unit and the second estimation model, the identification unit said. The identification device according to any one of claims 1 to 6, which identifies a machined portion based on the three-dimensional data input from the input unit and the first estimation model. It is a scanner system that acquires shape information in the oral cavity.
A 3D scanner that acquires 3D data including 3D position information at each of multiple points that make up the shape of the oral cavity using a 3D camera.
Based on the 3D data acquired by the 3D scanner, it is equipped with an identification device that identifies the processed part with respect to the biological part in the oral cavity.
The identification device is
An input unit into which 3D data acquired by the 3D scanner is input, and
An identification unit that identifies the processed part based on the three-dimensional data input from the input unit and an estimation model including a neural network for estimating the processed part based on the three-dimensional data.
A scanner system including an output unit that outputs an identification result by the identification unit. It is a method of identifying a processed part with respect to a living part in the oral cavity by a computer.
A step in which 3D data including 3D position information at each of a plurality of points constituting the shape in the oral cavity is input, and a step.
A step of identifying the machined part based on the three-dimensional data input by the input step and an estimation model including a neural network for estimating the machined part based on the three-dimensional data.
An identification method including a step of outputting an identification result by the identification step. It is an identification program that identifies the processed part of the living body in the oral cavity.
The identification program is applied to the computer.
A step in which 3D data including 3D position information at each of a plurality of points constituting the shape in the oral cavity is input, and a step.
A step of identifying the machined part based on the three-dimensional data input by the input step and an estimation model including a neural network for estimating the machined part based on the three-dimensional data.
An identification program that executes a step of outputting an identification result by the identification step.

Images