CN114065744B - ICD automatic coding method and device for Chinese electronic medical record operation text - Google Patents
ICD automatic coding method and device for Chinese electronic medical record operation text Download PDFInfo
- Publication number
- CN114065744B CN114065744B CN202111373822.0A CN202111373822A CN114065744B CN 114065744 B CN114065744 B CN 114065744B CN 202111373822 A CN202111373822 A CN 202111373822A CN 114065744 B CN114065744 B CN 114065744B
- Authority
- CN
- China
- Prior art keywords
- node
- text
- word
- nodes
- semantic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 239000013598 vector Substances 0.000 claims abstract description 84
- 238000012512 characterization method Methods 0.000 claims abstract description 42
- 238000012549 training Methods 0.000 claims abstract description 31
- 230000003993 interaction Effects 0.000 claims abstract description 16
- 238000013528 artificial neural network Methods 0.000 claims abstract description 14
- 201000010099 disease Diseases 0.000 claims description 28
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 28
- 230000006870 function Effects 0.000 claims description 22
- 239000011159 matrix material Substances 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 11
- 230000004927 fusion Effects 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 9
- 238000013459 approach Methods 0.000 claims description 8
- 230000004931 aggregating effect Effects 0.000 claims description 7
- 230000007246 mechanism Effects 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 230000002776 aggregation Effects 0.000 claims description 5
- 238000004220 aggregation Methods 0.000 claims description 5
- 238000011176 pooling Methods 0.000 claims description 5
- 238000003058 natural language processing Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 9
- 238000001356 surgical procedure Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 206010062767 Hypophysitis Diseases 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000007486 appendectomy Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000003759 clinical diagnosis Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 210000001061 forehead Anatomy 0.000 description 2
- 230000001939 inductive effect Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 210000003635 pituitary gland Anatomy 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 102100025594 Guided entry of tail-anchored proteins factor CAMLG Human genes 0.000 description 1
- 101000932902 Homo sapiens Guided entry of tail-anchored proteins factor CAMLG Proteins 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000004907 gland Anatomy 0.000 description 1
- 238000011423 initialization method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000007170 pathology Effects 0.000 description 1
- 230000001817 pituitary effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
- G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/353—Clustering; Classification into predefined classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/258—Heading extraction; Automatic titling; Numbering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Molecular Biology (AREA)
- Epidemiology (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Machine Translation (AREA)
- Medical Treatment And Welfare Office Work (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
The invention relates to the technical field of natural language processing, in particular to an ICD automatic coding method and device for a Chinese electronic medical record operation text, wherein the method comprises the following steps: taking operation texts in the structured electronic medical record as input, and constructing nodes and edges containing the non-supervision context semantic information based on a BERT non-supervision pre-training model to obtain an non-supervision semantic graph of each operation text; inputting the constructed unsupervised semantic graph into a gating graph neural network to perform global information interaction to obtain a semantic graph with context semantic information and global semantic information fused; based on the semantic graph fused with the obtained semantic information, the characterization of each node is aggregated, and an aggregated feature vector is obtained; classifying according to the aggregated feature vectors, and determining ICD codes corresponding to the surgical operation text. The method integrates context semantic information and global information in the operation text, can obtain better characterization performance based on the text, and realizes accurate coding of operation.
Description
Technical Field
The invention relates to the technical field of natural language processing, in particular to an ICD automatic coding method, an ICD automatic coding device, an electronic device and a storage medium for a Chinese electronic medical record operation text.
Background
International disease classification (International Classification of Diseases, ICD) is an internationally unified disease classification method by WHO, which classifies diseases into an ordered combination according to their etiology, pathology, clinical manifestations, anatomical location, etc., and is expressed by a coded method. ICD codes are widely applied to medical care tasks such as medical records management, medical insurance reimbursement and the like in various hospitals all over the world as unified classification standards.
ICD coding is an important task for hospital medical management. However, the conventional encoding process has a lot of manual operations. In the traditional public hospital of China, the ICD coding flow firstly needs to fill in clinical diagnosis description in an electronic medical record according to the condition of a disease caused by an attending medical doctor, then a corresponding coding standard name is determined by a coder specialized in a hospital record room according to the clinical diagnosis description, finally the coding standard name is converted into a corresponding ICD code and is input into a disease Diagnosis Related Group (DRG) system for medical statistics analysis, and the whole process is very time-consuming and easy to make mistakes.
Disclosure of Invention
Based on the problems of time and labor waste and easy error of manual ICD coding work, the invention provides an ICD automatic coding method, device, electronic equipment and storage medium for Chinese electronic medical record operation texts, which can automatically realize ICD accurate coding aiming at operation structured short texts in electronic medical records.
In a first aspect, an embodiment of the present invention provides an ICD automatic coding method for a chinese electronic medical record operation text, including:
Taking operation texts in the structured electronic medical record as input, and constructing nodes and edges containing the non-supervision context semantic information based on a BERT non-supervision pre-training model to obtain an non-supervision semantic graph of each operation text; the nodes of the unsupervised semantic graph comprise chapter-level nodes, document-level nodes and at most four shaft core word nodes; the feature vectors of the chapter level nodes comprise context semantic information of the chapter to which the surgical operation text belongs, the feature vectors of the document level nodes comprise context semantic information of the whole surgical operation text, the feature vectors of the four axis word nodes comprise four axis word semantic information of the part, the operation, the entrance and the disease property of the surgical operation respectively, and the weight of the edge of the unsupervised semantic graph comprises semantic influence information between the two connected nodes in the surgical operation text;
Inputting the constructed unsupervised semantic graph into a gating graph neural network to perform global information interaction to obtain a semantic graph fused with semantic information;
Based on the semantic graph fused with the obtained semantic information, the characterization of each node is aggregated, and an aggregated feature vector is obtained;
Classifying according to the aggregated feature vectors, and determining ICD codes corresponding to the surgical operation text.
Optionally, the building a node containing the unsupervised context semantic information based on the BERT unsupervised pre-training model includes:
inputting the operation text into a first BERT model, searching a coding chapter corresponding to the operation text, and determining a chapter title text; the first BERT model comprises a BERT unsupervised pre-training model and a softmax layer;
Inputting the chapter title text into a second BERT model, wherein the characterization of [ CLS ] characters is used as an initial feature vector of the chapter level node;
Inputting the operation text into a named entity recognition model, respectively extracting the phrase of four axial words of the position, operation type, approach and disease property, and determining the phrase position of each axial word;
And inputting the operation text into a second BERT model, taking the representation of [ CLS ] characters as initial feature vectors of the document level nodes, and taking the average value of the representations of the characters related to each axial word phrase as the initial feature vector of the corresponding axial word node.
Optionally, the building an edge containing the non-supervision context semantic information based on the BERT non-supervision pre-training model includes:
Calculating the influence relation with directivity between every two axis word nodes;
calculating influence relation with directivity between each axis word node and the document level node;
calculating the influence relation with directivity between the chapter level node and the document level node;
And constructing an influence matrix based on the influence relation, converting the constructed influence moment matrix into a normalized adjacency matrix by using an activation function, and determining the weight of each side in the unsupervised semantic graph.
Optionally, calculating an influence relationship with directivity between two axis word nodes includes:
Setting two axial word nodes as a first axial word node and a second axial word node respectively, shielding a phrase corresponding to the first axial word node in the operation text by using [ MASK ] characters, inputting a second BERT model, and obtaining an average value of characterization of characters related to the phrase corresponding to the first axial word node as a first average value;
Using [ MASK ] characters to shade phrases corresponding to a first axis word node and a second axis word node in the operation text, inputting a second BERT model, and obtaining the average value of the representation of the characters involved in the phrases corresponding to the first axis word node as a second average value;
And calculating the difference between the first average value and the second average value based on the Euclidean distance, and taking the difference as the influence weight of the second axis word node on the first axis word node.
Optionally, calculating an influence relationship with directivity between the axis word node and the document level node includes:
Using [ MASK ] characters to shield the phrase corresponding to the axle center word node in the operation text, inputting a second BERT model, using the representation of [ CLS ] characters as a second document parameter, and using the average value of the representations of the characters related to the phrase corresponding to the axle center word node as a second axle center word parameter;
Taking the initial feature vector of the document level node as a first document parameter, and calculating the difference between the first document parameter and the second document parameter based on Euclidean distance to be used as the influence weight of the axis word node on the document level node;
And taking the initial feature vector of the axis word node as a first axis word parameter, and calculating the difference between the first axis word parameter and the second axis word parameter based on Euclidean distance to be used as the influence weight of the document level node on the axis word node.
Optionally, calculating an influence relationship with directivity between the chapter-level node and the document-level node includes:
And assigning a softmax layer in the first BERT model to the weight of the corresponding encoded chapter of the operation text, wherein the softmax layer is used as the influence weight of the chapter level node on the chapter level node and the chapter level node on the document level node.
Optionally, the aggregating the characterization of each node includes:
Each node of the semantic graph is assigned a weight through an attention mechanism, and a representation of each node is extracted based on an average function and a maximum pooling function.
In a second aspect, an embodiment of the present invention further provides an ICD automatic coding device for a chinese electronic medical record surgical operation text, including:
The composition module is used for taking operation texts in the structured electronic medical record as input, constructing nodes and edges containing the non-supervision context semantic information based on the BERT non-supervision pre-training model, and obtaining a non-supervision semantic graph of each operation text; the nodes of the unsupervised semantic graph comprise chapter-level nodes, document-level nodes and at most four shaft core word nodes; the characteristic vector of the chapter level node comprises context semantic information of a chapter to which the operation text belongs, the characteristic vector of the document level node comprises context semantic information of the whole operation text, the characteristic vector of the four axis word nodes comprises four axis word semantic information of operation parts, operation, access and disease properties, and the weight of the edge of the unsupervised semantic graph comprises semantic influence information of the connected two nodes in the operation text;
The fusion module is used for inputting the constructed unsupervised semantic graph into a gating graph neural network to perform global information interaction so as to obtain a semantic graph fused with semantic information;
The aggregation module is used for aggregating the characterization of each node based on the semantic graph fused by the obtained semantic information to obtain an aggregated feature vector;
and the classification module is used for classifying according to the aggregated feature vectors and determining ICD codes corresponding to the surgical operation text.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of the present specification is implemented.
In a fourth aspect, embodiments of the present invention also provide a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform a method according to any of the embodiments of the present specification.
The invention provides an ICD automatic coding method, an ICD automatic coding device, an ICD automatic coding electronic device and a ICD automatic coding storage medium for a Chinese electronic medical record operation text, and the technical scheme of the invention is to construct an unsupervised semantic graph based on the operation text and extract critical semantic information in the operation text; inputting the unsupervised semantic graph into a gating graph neural network to perform global information interaction to obtain a semantic graph fused with semantic information, and further fusing the key semantic information contained in each node; based on the semantic graph fused with semantic information, the characterization of each node is aggregated to obtain an aggregated feature vector, and context semantic information and global information in the operation text are integrated; classifying according to the aggregated feature vectors, and determining the accurate ICD codes corresponding to the surgical operation text. According to the invention, the connotation characteristics of the operation text are fully considered, the context semantic information based on the unsupervised pre-training model and the global information based on the graph in each operation text are effectively fused, the axial word knowledge and the chapter knowledge in the operation text are structurally strengthened, the semantic characterization capability of the operation text is enhanced, the text information characteristics are fully extracted, and then the ICD coding is automatically carried out according to the extracted information characteristics. The invention can effectively save labor and has low error rate.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are needed in the embodiments or the description of the prior art will be briefly described, it will be obvious that the drawings in the following description are some embodiments of the invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a step diagram of an ICD automatic coding method for Chinese electronic medical record operation text according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a node construction process according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an edge building process according to an embodiment of the present invention;
FIG. 4 is a flowchart of an ICD auto-coding method for a Chinese electronic medical record operation text according to an embodiment of the present invention;
FIG. 5 is a step diagram of an ICD automatic coding method for a Chinese electronic medical record operation text according to another embodiment of the present invention;
FIG. 6 is a hardware architecture diagram of an electronic device according to an embodiment of the present invention;
Fig. 7 is a block diagram of an ICD automatic coding device for a chinese electronic medical record operation text according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by persons of ordinary skill in the art without making any inventive effort based on the embodiments of the present invention are within the scope of protection of the present invention.
As previously mentioned, the conventional ICD encoding process involves a large number of manual operations, which are time consuming and prone to error. In addition, the problem that the coding versions adopted by the hospitals of all provinces are different also exists, the coding in each version and the corresponding standard name have larger difference, and the difficulty of accurate coding of the ICD is further increased.
ICD codes can be divided into disease codes and surgical procedures codes. Wherein, the disease codes classify the disease of the patient according to unstructured long text such as disease diagnosis, clinical report and the like in the hospitalization medical records. The surgical procedure code classifies the patient's surgical procedure based on the structured short text describing the surgical procedure in the inpatient medical records. The invention focuses on realizing ICD automatic coding of the operation text aiming at the structured operation short text in the Chinese electronic medical record.
The current ICD automatic coding implementation is mostly based on traditional machine learning methods (such as a Support Vector Machine (SVM) and an edit distance (LEVENSHTEIN DISTANCE)) and time-series deep learning methods (such as a Convolution Neural Network (CNN) and a cyclic neural network (RNN)), and attempts to introduce external knowledge to enrich information of disease cases and texts. However, these methods have limited characterizations for structured short text, such as surgical text, and insufficient extraction of information features, and high automatic coding error rates.
The invention provides a graph neural network-based global characterization method for the operation text without introducing external knowledge. Meanwhile, in view of the strong representation capability of an unsupervised pre-training model (such as a BERT model) in the prior art to context semantics, the invention further effectively fuses global information based on a graph of the surgical operation text and context semantic information based on the unsupervised pre-training model so as to enhance the semantic representation capability of the surgical operation text.
In order to enhance the semantic representation capability of the surgical operation text, the invention fully considers the connotation characteristics of the surgical operation text:
(1) The surgical operation text comprises at most four types of axial words: site, surgery, approach, nature of the disease. For example: the appendectomy consists of two axial words of a part and an operation type, the pituitary adenomatosis excision is performed, and the forehead consists of four axial words of a part, an operation type, an approach type and a disease property. These four kinds of axial words play a decisive role in classifying surgical operation codes of texts.
(2) The operation ICD code is divided into 18 chapters in ICD-9-CM3 according to the difference of the parts to which the operation ICD code belongs, and the operation text itself also implies chapter information to be mined.
In view of this, the present invention constructs a heterogram containing nodes composed of chapter titles, texts and up to four axial words for each surgical operation text, so as to achieve structural reinforcement of axial word knowledge and chapter knowledge in the surgical operation text. Meanwhile, the invention calculates the semantic dependency right edges among all nodes based on the unsupervised BERT model, and initializes the semantic features of all nodes in the graph based on the unsupervised BERT model. After global propagation is carried out through the graph neural network, effective fusion of the global information based on the graph and the context semantic information based on the non-supervision pre-training model is realized, so that inductive characterization learning of the operation text is completed, and finally, accurate automatic coding of the operation text is realized.
Specific implementations of the above concepts are described below.
Referring to fig. 1, an embodiment of the present invention provides an ICD automatic coding method for a chinese electronic medical record operation text (abbreviated as the method of the present invention), which includes:
And 100, taking operation texts in the structured electronic medical record as input, and constructing nodes and edges containing the non-supervision context semantic information based on the BERT non-supervision pre-training model to obtain an non-supervision semantic graph of each operation text.
The nodes of the unsupervised semantic graph comprise chapter-level nodes, document-level nodes and at most four axis word nodes. The feature vector of the chapter level node comprises context semantic information of a chapter to which the operation text belongs, the feature vector of the document level node comprises context semantic information of the whole operation text, the feature vector of the four axis word nodes respectively comprises four axis word semantic information of operation parts, operation type, access and disease properties, the weight of the side of the unsupervised semantic graph can be calculated based on a BERT unsupervised pre-training model, and the feature vector comprises semantic influence information of the connected two nodes in the operation text.
Currently, chinese disease codes are generally referenced to version 1CD-10 and surgical codes are generally referenced to version ICD-9-CM 3. The invention analyzes the characteristics of the operation coding text from the medical angle: 1) All the operation texts can be divided into 18 chapters, and the chapters are related to the height of the part to which the text describes; 2) The standard operation text is composed of four axial words of parts, operation type, access and disease property, and the core knowledge of the text is hidden in the axial words, so that the text plays a decisive role in automatic coding; 3) The original surgical text itself contains hidden semantic information that helps to precisely encode. Therefore, the invention constructs a corresponding unsupervised semantic Graph (SMP-Graph) for each surgical operation text, wherein the nodes in the unsupervised semantic Graph comprise three types: chapter level nodes, axis word nodes, and document level nodes. The chapter level nodes and the document level nodes are respectively one, and the number of the axle center word nodes is four at most. For some surgical operation texts, such as: the appendectomy consists of two axial words of a part and an operation type, after an unsupervised semantic graph is constructed, only two axial word nodes can be formed, and the two axial words of the part and the operation type are respectively corresponding to each other, namely, the characteristic vectors of the two nodes respectively contain the two axial word meaning information of the part and the operation type of the operation type. The weight of an edge connecting nodes reflects the degree of semantic impact (or degree of dependency) between the two nodes to which the edge is connected.
Step 102, inputting the constructed unsupervised semantic graphs of each surgical operation text into a Gating Graph Neural Network (GGNN) respectively, and carrying out global information interaction of each unsupervised semantic graph respectively to realize deep fusion of unsupervised context semantic information and global information of the graph network so as to obtain a semantic graph with fused semantic information.
The invention utilizes GGNN to realize global information interaction between nodes. GGNN learn node representations through a neural network of a gate cycle unit (GRU), so that neighborhood information is fused, and the representations of the nodes are enriched. The information fusion among the nodes is continuously enhanced along with the increase of the interaction time t, and finally the global information interaction of the whole structure can be realized. In this way, the internal fusion is finally realized based on the global information and the unsupervised context semantic information of the graph, the semantic representation capability of the operation text is further enhanced, and the corresponding structure-enhanced unsupervised semantic graph representation output, namely the semantic graph fused with the semantic information, is obtained.
And 104, calculating each surgical operation text based on the obtained semantic graph fused with the corresponding semantic information, and aggregating the characterization of each node to obtain an aggregated feature vector.
After the feature output of the structure-enhanced unsupervised semantic graph is obtained, the features of all nodes are aggregated in a downstream reading stage so as to perform final classification prediction. The characterization of the aggregate nodes is preferably based on attention mechanisms, max pooling, and the like.
And 106, classifying each surgical operation text according to the aggregated feature vectors, and finally determining the ICD codes corresponding to each surgical operation text.
According to the ICD automatic coding method for the Chinese electronic medical record operation text, provided by the invention, an unsupervised semantic graph is built aiming at the connotation characteristics of the operation text without adopting a mode of directly extracting semantic information in the operation text based on an unsupervised pre-training model, global information based on the unsupervised semantic graph and contextual semantic information based on the unsupervised pre-training model are effectively fused, so that inductive representation learning of the operation text is enhanced, semantic information in the text is fully mined, and finally classification is carried out based on the aggregated feature vectors, so that accurate ICD coding is realized. The invention can be applied to ICD automatic coding of operation text in electronic calendar to solve the actual clinical requirement in the current Chinese hospital case management. It should be noted that, after the feature vector with enhanced semantic representation capability of the text of the surgical operation is obtained in step 104, the feature vector may be used for other analysis. The feature vectors which enhance the semantic characterization capability of the surgical operation text can more accurately characterize semantic information contained in the surgical operation text.
The manner in which the individual steps shown in fig. 1 are performed is described below.
First, step 100 constructs each node based on the operation text, and initializes the feature vector of the node using the BERT model. Optionally, in step 100, based on the BERT unsupervised pre-training model, a node containing unsupervised context semantic information is constructed, which specifically includes:
And inputting the operation text into a first BERT model, searching a coded chapter corresponding to the operation text by using the first BERT model, and determining a chapter title text. The first BERT model includes a BERT unsupervised pre-training model and a softmax layer. The first BERT model is used as a section classifier based on BERT, and can roughly search the coding section to which the operation text belongs.
Inputting the determined chapter title text into a second BERT model, obtaining the representation corresponding to each character of the text by using the second BERT model, and initializing the feature vector of the chapter level node by taking the representation of the [ CLS ] character as the initial feature vector of the chapter level node. The second BERT model is a new BERT unsupervised pre-training model, text is input into the second BERT model for processing, each text section relates to a plurality of characters, each character (or token) has a corresponding feature vector output, and the corresponding feature vector is the representation of the character. The BERT model has some special token as a placeholder, the [ CLS ] character is placed at the forefront of the text and used as the first token, and the representation of the [ CLS ] character, namely the feature vector corresponding to the [ CLS ] character, represents the feature vector corresponding to the whole text input and reflects chapter semantic information and implicit context semantic information.
And (3) inputting the operation text into a named entity recognition model (NER), respectively extracting the phrase of the four axial words of the part, the operation type, the approach and the disease property, and determining the phrase position of each axial word. The NER model is based on Bi-LSTM and CRF (random field) constitution, and can be used for extracting accurate phrase of part, operation type, access and disease property in operation text. The number and type of axial words are different in different texts, and the corresponding positions (and lengths) are also different.
Inputting the whole operation text into a second BERT model, obtaining the representation corresponding to each character of the text by using the second BERT model, for the document level node, using the representation of [ CLS ] characters implying the context semantic information of the whole text as the initial feature vector of the document level node, initializing the feature vector of the document level node, and for the axis word node, respectively using the average value of the representations of the characters related to each axis word phrase as the initial feature vector of the corresponding axis word node, and initializing the feature vector of at most four axis word nodes. The average value of the representation of the characters related to one axis word phrase, namely the average value is obtained after the addition of the feature vectors corresponding to a plurality of token related to the phrase.
FIG. 2 is a schematic diagram of a node construction process according to an embodiment of the present invention, in which named entity recognition is implemented by Bi-LSTM and CRF, and four axial words of a part, a surgery, an approach and a disease property in a surgery operation text are determined; then, determining a Chapter Title text by using a Chapter classifier (namely a first BERT model) based on BERT, wherein a Chapter column in fig. 2 represents a Chapter number, a Title column represents the Chapter Title text, and a Code Range column represents a Code to which a corresponding operation text belongs; finally, the characterization of the single character (such as T [cls] corresponding to [ CLS ] character, T Hanging down corresponding to 'vertical' character, T Part of the being the average value of T A part (C) and T Dividing into corresponding to the phrase 'part') is obtained through an unsupervised pre-training model (namely a second BERT model), and the corresponding characterization is output to obtain each node.
The nodes constructed in the unsupervised semantic graph reflect the characteristics of the operation text on the document level, zhang Jieji and the axis word level, so that the diversity of information characterization is greatly enhanced. Meanwhile, compared with the traditional node initialization method using Glove pre-trained as character level node characteristics, the node initialization based on the unsupervised pre-training model is more flexible and contains more context semantic information.
Step 100 then constructs each edge, and determines the weight of the edge according to the semantic influence information between the two nodes connected by the edge. Optionally, constructing an edge containing the semantic information of the unsupervised context based on the BERT unsupervised pre-training model in step 100 specifically includes:
calculating the influence relation with directivity between every two axis word nodes;
calculating the influence relation with directivity between each axis word node and the document level node;
Calculating the influence relation with directivity between the chapter level nodes and the document level nodes;
constructing an influence matrix based on all directional influence relationships obtained through calculation, converting the constructed influence moment matrix into a normalized adjacency matrix by utilizing an activation function, determining the weight of each side in the unsupervised semantic graph, and completing the construction of the directed sides to obtain the unsupervised semantic graph.
Further, in step 100, calculating an influence relationship with directivity between two axis word nodes includes:
Setting two axial word nodes as a first axial word node and a second axial word node respectively, shielding a phrase corresponding to the first axial word node in the operation text by using [ MASK ] characters, inputting the shielded text into a second BERT model, and obtaining an average value of the representation of characters related to the phrase corresponding to the first axial word node as a first average value;
Using [ MASK ] characters to shade phrases corresponding to a first axis word node and a second axis word node in the operation text, inputting the shielded text into a second BERT model, and obtaining an average value of the representation of the characters related to the phrases corresponding to the first axis word node as a second average value;
based on Euclidean distance, the difference between the first average value and the second average value is calculated and used as the influence weight of the second axis word node on the first axis word node.
Through the method, the influence force relation of the directivity of the second axial word node to the first axial word node can be determined. The calculation of the influence relation with directivity between the word pairs formed by every two axis word nodes in the unsupervised semantic graph can be completed by changing the specific pointed nodes of the first axis word node and the second axis word node.
Assuming a sentence whose tokenized input is a list x= [ x 1,…,xT ], thanks to the self-attention mechanism, the BERT model can map each character x i to a representation H θ(x)i of implicit context semantics, i.e. the corresponding feature vector, Where θ represents a parameter in the network. The characterization result of the character x i varies with the change of the context information and the position information, so that the degree of interdependence between characters can be reflected. to capture the dependency between the context character x i and character x j, character x i is first occluded, such as replacing character x i with a MASK character, And inputting the new sequence x\ { x i } obtained after shielding into the BERT model, thereby obtaining H θ(x\{xi})i as a characterization result corresponding to the character x i. Such a characterization includes the effect of the entire context on the character x i. Then, to further reflect the effect of character x j on character x i, character x j is further masked, i.e., character x i and character x j are replaced with the [ MASK ] character, And inputting the new sequence x\ { x i,xj } obtained after shielding into the BERT model, thereby obtaining H θ(x\{xi,xj})i as a new characterization result corresponding to the character x i. This new characterization result contains the effect of the entire context on character x i, except for character x j. Thus, the dependency of character x i on character x j can be represented by calculating the distance between the two representations corresponding to character x i. The definition function f (x i,xj) represents the effect of the character x j on another character x i in the context, expressed as follows:
f(xi,xj)＝dis(Hθ(x\{xi})i,Hθ(x\{xi,xj})i)
Where dis (x, y) is a distance measure representing the difference. The present invention uses Euclidean distance to calculate the characterization difference, the larger the distance, the greater the effect of character x j on character x i.
In the invention, the nodes of the unsupervised semantic graph take words or texts as units, so as to evaluate the dependency relationship between the word level and the document level, thereby constructing corresponding edges. In order to evaluate the directional influence relationship between two axiom nodes, similarly, after replacing the sequence character [ x m:xn ] (m < n) (abbreviated as x m:n) corresponding to the axiom w i corresponding to a certain node with the [ MASK ] character, inputting a second BERT model, and obtaining the representation output of the implicit context semantics from the second BERT model, wherein the new representation of the axiom w i is calculated from the average value of the representation of the corresponding character, and the expression is as follows:
f(wi,wj)＝dis(Avg(Hθ(x\{xm:n})m:n),Avg(Hθ(x\{xm:n,xp:q})m:n))
Wherein x m:n and x p:q respectively represent (sequence) characters corresponding to an axis word w i and an axis word w j, a phrase corresponding to a first axis word node is set as an axis word w i, a phrase corresponding to a second axis word node is set as an axis word w j,Avg(Hθ(x\{xm:n})m:n, an average value of representations of characters (x m:n) related to the axis word w i obtained by inputting a text shielding character x m:n into a BERT model is a first average value, avg (H θ(x\{xm:n, xp:q})m:n) represents an average value of representations of characters related to the axis word w i obtained by inputting a text shielding character x m:n and a character x p:q, namely a second average value, and an influence f (w i,wj) of the axis word w j on the axis word w i is obtained by calculating an euclidean distance, namely, influence weight of the second axis word node on the first axis word node is obtained.
Further, in step 100, calculating an influence relationship with directionality between an axis word node and a document level node includes:
The [ MASK ] characters are used for shielding the phrase corresponding to the axle center word node in the operation text, the shielded text is input into a second BERT model, the representation of the [ CLS ] characters is used as a second document parameter, and the average value of the representations of the characters related to the phrase corresponding to the axle center word node is used as a second axle center word parameter;
Taking an initial feature vector of a document level node as a first document parameter, and calculating the difference between the first document parameter and a second document parameter based on Euclidean distance to be used as the influence weight of the axis word node on the document level node;
the initial feature vector of the axis word node is used as a first axis word parameter, and the difference between the first axis word parameter and a second axis word parameter is calculated based on Euclidean distance and used as the influence weight of the document level node on the axis word node.
In the invention, in order to evaluate the dependency relationship between the axial words and the document, the [ CLS ] characters are used as the corresponding representation of the whole document. Therefore, the influence relationship between the document and the axial word is expressed as:
f(d,wi)＝dis(Hθ(x)[CLS],Hθ(x\xm:n)[CLS])
f(wi,d)＝dis(Avg(Hθ(x)m:n),Avg(Hθ(x\{xm:n})m:n))
Wherein f (d, w i) represents the influence of the axial word w i on the document d, f (w i, d) represents the influence weight of the document d on the axial word w i, Let the document d correspond to the complete operation text, the phrase corresponding to the axis word node is the axis word w i, then H θ(x)[CLS] represents the representation of [ CLS ] characters obtained by inputting the complete operation text into the BERT model, namely the initial feature vector of the document level node, also called the first document parameter, H θ(x\xm:n)[CLS] represents the representation of [ CLS ] characters obtained by inputting the text after shielding the character x m:n corresponding to the axis word w i, I.e., a second document argument; Avg (H θ(x)m:n) represents the average value of the representation of the character (x m:n) involved in the axial word w i, i.e. the initial feature vector of the axial word node, also called the first axial word parameter, Avg (H θ(x\{xm:n})m:n) represents the average value of the character representation related to the axis word w i obtained by inputting the character x m:n corresponding to the text-shielding axis word w i into the BERT model, I.e. the second axial word parameter. Through the formula, f (d, w i) is the influence weight of the axis word node on the document level node, and f (w i, d) is the influence weight of the document level node on the axis word node.
Further, in step 100, calculating an influence relationship with directivity between the chapter level node and the document level node includes:
The softmax layer in the first BERT model is assigned to weights of corresponding encoded chapters of the operation text, and the weights are taken as influence weights of the chapter-level nodes on the chapter-level nodes and the chapter-level nodes on the document-level nodes.
In the invention, a classifier based on a BERT model, which is used before when a node of a graph is constructed, is adopted between chapters and a document to judge the chapters to which a surgical operation text belongs, a softmax layer is assigned to the final weight of the predicted chapter, and the final weight is set as the influence weight between the chapter and the document, because the weight contains the semantic relation between the text and the chapter.
By calculating the influence relationships of all word pairs, all word-documents, and chapter-documents in the context, an influence matrix M εR |W|+2*|W|+2 can be constructed, where |W| represents the number of axial words in the surgical text. Converting an influence matrix M containing semantic relations into a normalized adjacent matrix A through an activation function to obtain corresponding weights, thereby completing the construction of edges in the SMP-Graph, wherein the activation function is modified by sigmoid:
Wherein, adjacency matrix A ε R |W|+2*|W|+2.
FIG. 3 is a schematic diagram of an edge construction process according to an embodiment of the present invention, in which the operation text "vertical gland partial excision, via forehead access" is taken as an example, a second BERT model (abbreviated as BERT in FIG. 3) is used for shielding each word Fu Shuru of "pituitary gland" by the [ MASK ], a second BERT model is input by each of the characters shielding "pituitary gland" and "excision" by the [ MASK ], obtaining an influence matrix, and assigning final weights (i.e. weights W c obtained from the chapter classifier) to the prediction chapters in combination with the softmax layer of the first BERT model to obtain an adjacency matrix a, wherein one |w|x|w|matrix in the adjacency matrix a represents weights of edges between axial words (such asRepresenting the semantic influence weight of the second axis word on the first axis word, the remaining elements representing weights representing edges between the document and each axis word (e.g.)Representing the semantic impact weight of the second axis term on the document), the weight of the edge between the chapter and the document (e.g., E d,c represents the semantic impact weight of the chapter on the document), or empty (i.e., filled in with 0).
Optionally, for step 102, global information interaction is performed, and the detailed interaction formula is as follows:
at＝Aht-1Wa
zt＝σ(Wzat+Uzht-1bz)
rt＝σ(Wrat+Urht-1br)
Wherein σ () represents a sigmoid type function, a t represents global information received from neighboring nodes by each node in the graph network of time step t, a represents an adjacency matrix, h t-1 represents a set of node feature vectors of time step t-1, W a represents a trainable weight, W z represents a trainable weight, W h represents a trainable weight, U z represents a trainable weight, U r represents a trainable weight, U h represents a trainable weight, b z represents a trainable bias, b r represents a trainable bias, b h represents a trainable bias, The candidate hidden state of the time step t is represented, h t represents the node feature vector set of the time step t, and the element multiplication operation is represented by the following, the parameters W, U and b are trainable weights and deviations, and z t and r t represent functions of a control update gate and a reset gate, respectively, so that the contribution degree of neighborhood information to the current node embedding is determined.
Optionally, for step 104, after obtaining the semantic graph of the original surgical operation text semantic information fusion, the present invention aggregates the node characterization at the downstream reading stage to perform the final classification prediction. The read function is designed as follows:
Where f 1 () and f 2 () are two multi-layer perceptrons, as a soft attention weight and nonlinear feature transform, respectively. h w,d,c represents the node feature vector set after the attention mechanism is weighted, and h t w,d,c represents the node feature vector set of time step t.
Optionally, for step 104, aggregating the characterizations of the nodes, including:
And (3) distributing weight to each node of the semantic graph fused with the semantic information through a notice mechanism, and extracting the characterization of each node based on an average function and a maximum pooling function. The method has the advantages that the information of each node contributes to the final aggregate graph representation, and the nodes with higher weights contribute more to the final graph representation output. And finally, adding the extracted characterization results to realize the aggregate characterization of each node.
Optionally, for step 106, classifying according to the aggregated feature vector includes:
The aggregated feature vector is sent to a softmax layer for prediction, and the cross entropy function is used for training parameters, wherein the expression is:
Wherein the method comprises the steps of The i-th element of the one-hot vector,Representing the model predicted one-hot vector,Representing the ith element of the model predicted one-hot vector, W represents a trainable weight, b represents a trainable bias, and Loss represents a Loss function.
As shown in fig. 4 and fig. 5, the embodiment of the present invention further provides an ICD automatic coding method for a chinese electronic medical record operation text, including:
step 200, inputting a surgical operation text into a first BERT model, searching a coded chapter corresponding to the surgical operation text by using the first BERT model, and determining a chapter title text;
Step 202, inputting a chapter title text into a second BERT model, wherein the characterization of [ CLS ] characters is used as an initial feature vector of a chapter level node;
Step 204, inputting the operation text into a named entity recognition model, respectively extracting the phrase of four axial words of the part, operation type, approach and disease property, and determining the phrase position of each axial word;
Step 206, inputting the operation text into a second BERT model, taking the representation of [ CLS ] characters as initial feature vectors of document level nodes, and taking the average value of the representations of the characters related to each axial word phrase as the initial feature vector of the corresponding axial word node;
Step 208, calculating the influence relation with directivity between every two axis word nodes;
step 210, calculating influence relation with directivity between each axis word node and document level node;
Step 212, calculating an influence relation with directivity between the chapter level node and the document level node;
Step 214, constructing an influence matrix based on the influence relation, converting the constructed influence moment matrix into a normalized adjacency matrix by using an activation function, determining the weight of each side in the unsupervised semantic graph, and completing the construction of the unsupervised semantic graph;
Step 216, inputting the constructed unsupervised semantic graph into a Gating Graph Neural Network (GGNN) to perform global information interaction to obtain a semantic graph fused with semantic information;
step 218, aggregating the characterization of each node based on the semantic graph fused by the obtained semantic information to obtain an aggregated feature vector;
step 220, classifying according to the aggregated feature vectors, and determining ICD codes corresponding to the operation text.
As shown in fig. 4, a preprocessing model is formed by a named entity recognition model of an axis word and a chapter classifier (i.e., a first BERT model), an original operation text is preprocessed, a document of the axis word and a chapter heading text in the operation text are input into an unsupervised training model (a second BERT model) to complete the construction of a graph network, and after global information interaction, the characterization of each node is aggregated based on the operations of an attention mechanism, a maximum pooling and the like, and finally the ICD automatic coding of the operation text is realized.
In a specific embodiment, to illustrate the effectiveness of the method of the present invention, a set of data sets is constructed based on Chinese ICD-9-CM3 encoded text collected from real electronic medical records and used to evaluate the performance of the method of the present invention in the task of automatically encoding surgical procedures. Meanwhile, the method is widely compared with other representative text characterization models.
The invention collects a Chinese operation text data set from the structured electronic medical record, which comprises 8400 Chinese operation text sections corresponding to 1400 ICD-9-CM3 operation codes. All the texts are short texts, and the number of the short texts is not more than 80 Chinese characters. All text in the dataset is manually annotated with ICD-9-CM3 codes by the hospital professional encoder to construct text code matches. In addition, considering the long tail effect of the dataset, the 100 pieces of surgical operation text with highest frequency are also selected to reconstruct the sub dataset named CN-100, and the original dataset is named CN-full.
The present invention compares the characteristic performance of the present invention method with the characteristic baseline model in the prior art, and classifies the characteristic prior art baseline model into three types: 1) Traditional machine learning methods, such as edit distance; 2) Sequence-based deep learning methods, such as: textCNN, text RNN, and CAML; 3) Deep learning methods based on unsupervised pre-training models, such as the trimmed BERT model.
As for the evaluation index of the performance, the performance of all methods was evaluated using Accuracy, recall, micro-average, and macro-average F1. To maintain fairness of the comparison, the introduction of any external information is avoided to enrich the features of the original text. Thus, all characterization methods rely on information implicit in the text for processing.
The results show that the performance of the method is superior to that of the baseline model on both data sets, the method obtains 70.13% of accuracy on the data set CN-full and 97.63% of accuracy on the data set CN-100. Meanwhile, the performance of the method is higher than that of a BERT model for learning characterization from the unsupervised contextualization information, which shows that the method can effectively integrate the global information and the unsupervised contextualization information based on the graph into the graph structure. Therefore, the method effectively improves the characterization performance of the operation text.
As shown in fig. 6 and 7, the embodiment of the invention also provides an ICD automatic coding device for the operation text of the chinese electronic medical record. The apparatus embodiments may be implemented by software, or may be implemented by hardware or a combination of hardware and software. In terms of hardware, as shown in fig. 6, a hardware architecture diagram of an electronic device where an ICD automatic coding device for a chinese electronic medical record operation text (ICD automatic coding device for short) provided in an embodiment of the present invention is located, where in addition to a processor, a memory, a network interface, and a nonvolatile memory shown in fig. 6, the electronic device where the embodiment is located may generally include other hardware, such as a forwarding chip responsible for processing a message, and so on. For example, as shown in fig. 7, the device in a logic sense is formed by reading a corresponding computer program in a nonvolatile memory into a memory by a CPU of an electronic device where the device is located. The automatic ICD coding device for the operation text of the Chinese electronic medical record provided by the embodiment comprises a composition module 401, a fusion module 402, an aggregation module 403 and a classification module 404, wherein:
The composition module 401 is configured to construct nodes and edges containing semantic information of an unsupervised context based on a BERT unsupervised pre-training model by taking a surgical operation text in the structured electronic medical record as an input, and obtain an unsupervised semantic graph of each surgical operation text; the nodes of the unsupervised semantic graph comprise chapter-level nodes, document-level nodes and at most four shaft core word nodes; the feature vectors of the chapter level nodes comprise context semantic information of chapters to which the operation text belongs, the feature vectors of the document level nodes comprise context semantic information of the whole operation text, the feature vectors of the four axis word nodes comprise four axis word semantic information of operation parts, operation, access and disease properties respectively, and the weight of the edge of the unsupervised semantic graph comprises semantic influence information between two connected nodes in the operation text;
The fusion module 402 is used for inputting the constructed unsupervised semantic graph into a gating graph neural network to perform total information interaction, so as to obtain a semantic graph fused with semantic information;
The aggregation module 403 is configured to aggregate the features of each node based on the semantic graph fused by the obtained semantic information, so as to obtain an aggregated feature vector;
the classification module 404 is configured to classify according to the aggregated feature vector, and determine the ICD code corresponding to the surgical operation text.
In the present invention, the composition module 401 may be used to implement the step 100, the fusion module 402 may be used to implement the step 102, the aggregation module 403 may be used to implement the step 104, and the classification module 404 may be used to implement the step 106.
It will be appreciated that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on an ICD automatic coding device for chinese electronic medical record surgical operation text. In other embodiments of the present invention, an ICD automatic encoding device for Chinese electronic medical records surgical operation text may include more or less components than illustrated, or may combine certain components, or may split certain components, or may have a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
The content of information interaction and execution process between the modules in the device is based on the same conception as the embodiment of the method of the present invention, and specific content can be referred to the description in the embodiment of the method of the present invention, which is not repeated here.
The embodiment of the invention also provides electronic equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the ICD automatic coding method of the Chinese electronic medical record operation text in any embodiment of the invention is realized.
The embodiment of the invention also provides a computer readable storage medium, and the computer readable storage medium is stored with a computer program, when the computer program is executed by a processor, the processor is caused to execute the ICD automatic coding method of the Chinese electronic medical record operation text in any embodiment of the invention.
Specifically, a system or apparatus provided with a storage medium on which a software program code realizing the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or apparatus may be caused to read out and execute the program code stored in the storage medium.
In this case, the program code itself read from the storage medium may realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code form part of the present invention.
Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-R, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs, DVD+RWs), magnetic tapes, nonvolatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer by a communication network.
Further, it should be apparent that the functions of any of the above-described embodiments may be achieved not only by executing the program code read out by the computer but also by causing an operating system or the like operating on the computer to perform part or all of the actual operations based on the instructions of the program code.
Further, it is understood that the program code read out by the storage medium is written into a memory provided in an expansion board inserted into a computer or into a memory provided in an expansion module connected to the computer, and then a CPU or the like mounted on the expansion board or the expansion module is caused to perform part and all of actual operations based on instructions of the program code, thereby realizing the functions of any of the above embodiments.
It should be noted that relational terms such as first and second are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one …" does not exclude the presence of additional identical elements in a process, method, article, or apparatus that comprises the element.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware related to program instructions, and the foregoing program may be stored in a computer readable storage medium, where the program, when executed, performs steps including the above method embodiments; and the aforementioned storage medium includes: various media in which program codes may be stored, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some of the technical features can be replaced equivalently; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. An ICD automatic coding method for a chinese electronic medical record operation text, comprising:
Taking operation texts in the structured electronic medical record as input, and constructing nodes and edges containing the non-supervision context semantic information based on a BERT non-supervision pre-training model to obtain an non-supervision semantic graph of each operation text; the nodes of the unsupervised semantic graph comprise chapter-level nodes, document-level nodes and at most four shaft core word nodes; the feature vectors of the chapter level nodes comprise context semantic information of chapters to which the operation text belongs, the feature vectors of the document level nodes comprise context semantic information of the whole operation text, the feature vectors of the four axis word nodes comprise four axis word semantic information of operation parts, operation ways, access ways and disease properties, and the weights of the edges of the unsupervised semantic graph comprise semantic influence information between the two connected nodes in the operation text;
Inputting the constructed unsupervised semantic graph into a gating graph neural network to perform global information interaction to obtain a semantic graph fused with semantic information;
Based on the semantic graph fused with the obtained semantic information, the characterization of each node is aggregated, and an aggregated feature vector is obtained;
Classifying according to the aggregated feature vectors, and determining ICD codes corresponding to the surgical operation text;
The construction of the node containing the non-supervision context semantic information based on the BERT non-supervision pre-training model comprises the following steps:
inputting the operation text into a first BERT model, searching a coding chapter corresponding to the operation text, and determining a chapter title text; the first BERT model comprises a BERT unsupervised pre-training model and a softmax layer;
inputting the chapter title text into a second BERT model, wherein the characterization of [ CLS ] characters is used as an initial feature vector of the chapter level node;
Inputting the operation text into a named entity recognition model, respectively extracting the phrase of four axial words of the position, operation type, approach and disease property, and determining the phrase position of each axial word;
And inputting the operation text into a second BERT model, taking the representation of [ CLS ] characters as initial feature vectors of the document level nodes, and taking the average value of the representations of the characters related to each axial word phrase as the initial feature vector of the corresponding axial word node.
2. The method of claim 1, wherein constructing edges containing unsupervised context semantic information based on the BERT unsupervised pre-training model comprises:
Calculating the influence relation with directivity between every two axis word nodes;
calculating influence relation with directivity between each axis word node and the document level node;
calculating the influence relation with directivity between the chapter level node and the document level node;
and constructing an influence matrix based on the influence relation, converting the constructed influence moment matrix into a normalized adjacency matrix by using an activation function, and determining the weight of each side in the unsupervised semantic graph.
3. The method of claim 2, wherein calculating an influence relationship with directivity between two of the axis phrase nodes comprises:
Setting two axial word nodes as a first axial word node and a second axial word node respectively, shielding a phrase corresponding to the first axial word node in the operation text by using [ MASK ] characters, inputting a second BERT model, and obtaining an average value of characterization of characters related to the phrase corresponding to the first axial word node as a first average value;
Using [ MASK ] characters to shade phrases corresponding to the first axial word node and the second axial word node in the operation text, inputting a second BERT model, and obtaining the average value of the representation of the characters related to the phrases corresponding to the first axial word node as a second average value;
and calculating the difference between the first average value and the second average value based on the Euclidean distance, and taking the difference as the influence weight of the second axis word node on the first axis word node.
4. The method of claim 2, wherein calculating an influence relationship having directionality between the axis word node and the document level node comprises:
Using [ MASK ] characters to shield the phrase corresponding to the axle center word node in the operation text, inputting a second BERT model, using the representation of [ CLS ] characters as a second document parameter, and using the average value of the representations of the characters related to the phrase corresponding to the axle center word node as a second axle center word parameter;
Taking the initial feature vector of the document level node as a first document parameter, and calculating the difference between the first document parameter and the second document parameter based on Euclidean distance to be used as the influence weight of the axis word node on the document level node;
and taking the initial feature vector of the axis word node as a first axis word parameter, and calculating the difference between the first axis word parameter and the second axis word parameter based on Euclidean distance to be used as the influence weight of the document level node on the axis word node.
5. The method of claim 2, wherein calculating an influence relationship having directionality between the chapter-level nodes and the document-level nodes comprises:
And assigning a softmax layer in the first BERT model to the weight of the section corresponding to the coding of the operation text, wherein the softmax layer is used as the influence weight of the document level node on the section level node and the section level node on the document level node.
6. The method of claim 1, wherein aggregating the characterization of each node comprises:
Each node of the semantic graph is assigned a weight through an attention mechanism, and a representation of each node is extracted based on an average function and a maximum pooling function.
7. An ICD automatic coding device for chinese electronic medical record operation text, comprising:
The composition module is used for taking the operation text in the structured electronic medical record as input, constructing nodes and edges containing the non-supervision context semantic information based on the BERT non-supervision pre-training model, and obtaining a non-supervision semantic graph of each operation text; the nodes of the unsupervised semantic graph comprise chapter-level nodes, document-level nodes and at most four shaft core word nodes; the feature vectors of the chapter level nodes comprise context semantic information of chapters to which the operation text belongs, the feature vectors of the document level nodes comprise context semantic information of the whole operation text, the feature vectors of the four axis word nodes comprise four axis word semantic information of operation parts, operation ways, access ways and disease properties, and the weights of the edges of the unsupervised semantic graph comprise semantic influence information between the two connected nodes in the operation text;
the fusion module is used for inputting the constructed unsupervised semantic graph into a gating graph neural network to perform global information interaction so as to obtain a semantic graph fused with semantic information;
The aggregation module is used for aggregating the characterization of each node based on the semantic graph fused by the obtained semantic information to obtain an aggregated feature vector;
The classification module is used for classifying according to the aggregated feature vectors and determining ICD codes corresponding to the surgical operation text;
The construction of the node containing the non-supervision context semantic information based on the BERT non-supervision pre-training model comprises the following steps:
inputting the operation text into a first BERT model, searching a coding chapter corresponding to the operation text, and determining a chapter title text; the first BERT model comprises a BERT unsupervised pre-training model and a softmax layer;
inputting the chapter title text into a second BERT model, wherein the characterization of [ CLS ] characters is used as an initial feature vector of the chapter level node;
Inputting the operation text into a named entity recognition model, respectively extracting the phrase of four axial words of the position, operation type, approach and disease property, and determining the phrase position of each axial word;
And inputting the operation text into a second BERT model, taking the representation of [ CLS ] characters as initial feature vectors of the document level nodes, and taking the average value of the representations of the characters related to each axial word phrase as the initial feature vector of the corresponding axial word node.
8. An electronic device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the method of any of claims 1-6 when the computer program is executed.
9. A storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111373822.0A CN114065744B (en) | 2021-11-19 | 2021-11-19 | ICD automatic coding method and device for Chinese electronic medical record operation text |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111373822.0A CN114065744B (en) | 2021-11-19 | 2021-11-19 | ICD automatic coding method and device for Chinese electronic medical record operation text |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114065744A CN114065744A (en) | 2022-02-18 |
| CN114065744B true CN114065744B (en) | 2024-09-27 |
Family
ID=80278275
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111373822.0A Active CN114065744B (en) | 2021-11-19 | 2021-11-19 | ICD automatic coding method and device for Chinese electronic medical record operation text |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114065744B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116226702B (en) * | 2022-09-09 | 2024-04-26 | 武汉中数医疗科技有限公司 | Thyroid sampling data identification method based on bioelectrical impedance |
| CN116188618B (en) * | 2023-04-24 | 2023-08-15 | 清华大学 | Image generation method and device based on structured semantic graph |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112801010A (en) * | 2021-02-07 | 2021-05-14 | 华南理工大学 | Visual rich document information extraction method for actual OCR scene |
| CN113535904A (en) * | 2021-07-23 | 2021-10-22 | 重庆邮电大学 | Aspect level emotion analysis method based on graph neural network |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113377929B (en) * | 2021-08-12 | 2021-12-10 | 北京好欣晴移动医疗科技有限公司 | Unsupervised clustering method, unsupervised clustering device and unsupervised clustering system for special terms |
-
2021
- 2021-11-19 CN CN202111373822.0A patent/CN114065744B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112801010A (en) * | 2021-02-07 | 2021-05-14 | 华南理工大学 | Visual rich document information extraction method for actual OCR scene |
| CN113535904A (en) * | 2021-07-23 | 2021-10-22 | 重庆邮电大学 | Aspect level emotion analysis method based on graph neural network |
Non-Patent Citations (1)
| Title |
|---|
| 基于预训练表征模型的自动ICD编码;王天罡;李晓亮;张晓滨;蔡宏伟;;中国数字医学;20200715(07);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114065744A (en) | 2022-02-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9183466B2 (en) | Correlating videos and sentences | |
| CN111159407B (en) | Method, apparatus, device and medium for training entity recognition and relation classification model | |
| CN109871538A (en) | A kind of Chinese electronic health record name entity recognition method | |
| CN110032632A (en) | Intelligent customer service answering method, device and storage medium based on text similarity | |
| CN112507699B (en) | Remote supervision relation extraction method based on graph convolution network | |
| CN114330354B (en) | Event extraction method and device based on vocabulary enhancement and storage medium | |
| CN111259144A (en) | Multi-model fusion text matching method, device, equipment and storage medium | |
| CN113886571A (en) | Entity identification method, entity identification device, electronic equipment and computer readable storage medium | |
| CN114547230B (en) | Intelligent administrative law enforcement case information extraction and case identification method | |
| CN113392209B (en) | Text clustering method based on artificial intelligence, related equipment and storage medium | |
| CN114065744B (en) | ICD automatic coding method and device for Chinese electronic medical record operation text | |
| CN112434718B (en) | New coronary pneumonia multi-modal feature extraction fusion method and system based on depth map | |
| CN112101031B (en) | Entity identification method, terminal equipment and storage medium | |
| CN115858825A (en) | Equipment fault diagnosis knowledge graph construction method and device based on machine learning | |
| CN114528898A (en) | Scene graph modification based on natural language commands | |
| CN116975256B (en) | Method and system for processing multisource information in construction process of underground factory building of pumped storage power station | |
| CN114818717B (en) | Chinese named entity recognition method and system integrating vocabulary and syntax information | |
| CN114218922A (en) | Aspect emotion analysis method based on dual-channel graph convolution network | |
| CN114997288A (en) | Design resource association method | |
| CN118468061B (en) | Automatic algorithm matching and parameter optimizing method and system | |
| CN116757196A (en) | Event trigger word extraction method and device | |
| CN114550856B (en) | Electronic medical record similarity retrieval method integrating cross knowledge attention | |
| CN116739402A (en) | Health portrait construction method based on data mining | |
| CN115879460A (en) | Method, device, equipment and medium for identifying new label entity facing text content | |
| CN118568568B (en) | Training method of content classification model and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |