CN117218433B - Household multi-cancer detection device and multi-mode fusion model construction method and device - Google Patents
Household multi-cancer detection device and multi-mode fusion model construction method and device Download PDFInfo
- Publication number
- CN117218433B CN117218433B CN202311185051.1A CN202311185051A CN117218433B CN 117218433 B CN117218433 B CN 117218433B CN 202311185051 A CN202311185051 A CN 202311185051A CN 117218433 B CN117218433 B CN 117218433B
- Authority
- CN
- China
- Prior art keywords
- sample
- detection image
- model
- label
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 342
- 230000004927 fusion Effects 0.000 title claims abstract description 156
- 206010028980 Neoplasm Diseases 0.000 title claims abstract description 58
- 201000011510 cancer Diseases 0.000 title claims abstract description 58
- 238000010276 construction Methods 0.000 title claims abstract description 22
- 238000004458 analytical method Methods 0.000 claims abstract description 82
- 238000012216 screening Methods 0.000 claims abstract description 70
- 201000010099 disease Diseases 0.000 claims abstract description 27
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims abstract description 27
- 239000000523 sample Substances 0.000 claims description 428
- 238000000034 method Methods 0.000 claims description 43
- 238000003908 quality control method Methods 0.000 claims description 33
- 230000003750 conditioning effect Effects 0.000 claims description 25
- 238000000605 extraction Methods 0.000 claims description 22
- 238000012545 processing Methods 0.000 claims description 21
- 230000005540 biological transmission Effects 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 11
- 238000007689 inspection Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 9
- 238000011156 evaluation Methods 0.000 claims description 8
- 238000012360 testing method Methods 0.000 claims description 7
- 238000000611 regression analysis Methods 0.000 claims description 6
- 238000013500 data storage Methods 0.000 claims description 4
- 238000012502 risk assessment Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 230000003187 abdominal effect Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000002604 ultrasonography Methods 0.000 description 4
- 238000005070 sampling Methods 0.000 description 3
- 210000001015 abdomen Anatomy 0.000 description 2
- 238000013475 authorization Methods 0.000 description 2
- 210000004369 blood Anatomy 0.000 description 2
- 239000008280 blood Substances 0.000 description 2
- 210000001124 body fluid Anatomy 0.000 description 2
- 239000010839 body fluid Substances 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 210000003608 fece Anatomy 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000011528 liquid biopsy Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 210000003296 saliva Anatomy 0.000 description 2
- 210000002700 urine Anatomy 0.000 description 2
- 206010008342 Cervix carcinoma Diseases 0.000 description 1
- 206010033128 Ovarian cancer Diseases 0.000 description 1
- 206010061535 Ovarian neoplasm Diseases 0.000 description 1
- 208000006105 Uterine Cervical Neoplasms Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 239000000427 antigen Substances 0.000 description 1
- 102000036639 antigens Human genes 0.000 description 1
- 108091007433 antigens Proteins 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 150000001720 carbohydrates Chemical class 0.000 description 1
- 201000010881 cervical cancer Diseases 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013399 early diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000011987 methylation Effects 0.000 description 1
- 238000007069 methylation reaction Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000007170 pathology Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 210000002966 serum Anatomy 0.000 description 1
- 238000010561 standard procedure Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 239000000107 tumor biomarker Substances 0.000 description 1
- 210000001215 vagina Anatomy 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a home multi-cancer detection device and a multi-mode fusion model construction method and device, wherein the home multi-cancer detection device is arranged to conveniently and accurately detect in the home of a user, and the types, the number and the parameters of the device are flexibly determined based on the types of diseases to be detected, so that the device can flexibly cope with complex and various home detection scenes, and can carry out targeted detection on the types of the diseases to be detected in different types; in addition, based on a probability analysis sub-model, multi-label two-classification is carried out on a sample detection image acquired by a household multi-cancer detection device, the probability analysis sub-model is trained, then feature selection is carried out based on a multi-label classification result of the sample detection image of a sample object, a sample physiological detection result and a sample personal attribute, screening features are obtained, and then a fusion classification sub-model is constructed based on the screening features, so that a multi-mode fusion model with accurate multi-mode fusion analysis capability can be obtained.
Description
Technical Field
The invention relates to the technical field of information detection, in particular to a household multi-cancer detection device and a multi-mode fusion model construction method and device.
Background
Cancer threatens the life health of humans. The early diagnosis and treatment can not only greatly improve the survival rate of the cancer of the patient, but also remarkably reduce the social medical cost. However, conventional cancer screening methods generally require patients to go to a hospital for examination at a time when they feel uncomfortable, or by physical examination institutions. For areas with relatively tight medical resources, the method is time-consuming and labor-consuming, and in some cases, patients are difficult to go to an inspection mechanism for inspection, so that a portable home multiple cancer detection device is needed to facilitate the necessary inspection of the patients at home. However, current portable detection apparatuses are generally designed for specific diseases, and are difficult to adapt to current detection requirements in the face of complex and diverse home detection scenes.
In addition, some home cancer primary screening protocols based on liquid biopsies (kits) are also currently emerging. However, the existing home primary screening scheme is often limited to a certain detection means or mode, and multi-mode information fusion is ignored, so that a fusion analysis model capable of accurately fusing multi-mode information and performing fusion analysis is required to be constructed, and the accuracy of home primary screening and the guiding significance for subsequent diagnosis are improved.
Disclosure of Invention
The invention provides a household multi-cancer detection device and a multi-mode fusion model construction method and device, which are used for solving the defects that a current portable detection instrument in the prior art is usually designed aiming at specific diseases, is difficult to adapt to the current detection requirement, and the existing household primary screening scheme is often limited to a certain detection means or mode and is inaccurate in analysis result.
The invention provides a home multi-cancer detection device, comprising:
One or more replaceable ultrasound probe transducers, a multiplexed data transmission unit, a signal conditioning circuit, and a system processing unit;
The type, the number and the parameters of the replaceable ultrasonic probe transducers are determined based on the type of the disease to be detected, and are used for transmitting ultrasonic signals to the position to be detected and collecting echo signals reflected by the position to be detected;
the multiplexing data transmission unit is used for transmitting echo signals reflected by the to-be-detected part acquired by the replaceable ultrasonic probe transducer to the signal conditioning circuit and transmitting detection beam analog signals sent by the signal conditioning circuit to the replaceable ultrasonic probe transducer;
The signal conditioning circuit is used for receiving echo signals reflected by the part to be detected, converting the echo signals into digital signals and transmitting the digital signals to the system processing unit, and receiving detection beams sent by the system processing unit, converting the detection beams into analog signals and transmitting the analog signals to the multiplexing data transmission unit;
The system processing unit is used for carrying out beam forming on the basis of the digital signals of the echo signals transmitted by the signal conditioning circuit to obtain detection images, and is used for generating new detection beams and sending the new detection beams to the signal conditioning circuit.
According to the household multi-cancer detection device provided by the invention, the replaceable ultrasonic probe transducer has a multi-mode scanning function and is used for acquiring scanning images corresponding to a plurality of scanning modes.
According to the household multiple cancer species detection device provided by the invention, the household multiple cancer species detection device further comprises an external data transmission module, a data storage module and a power supply module.
The invention also provides a multi-mode fusion model construction method based on any household multi-cancer detection device, which comprises the following steps:
acquiring a sample detection image of a sample detection part of a sample object acquired by the home multi-cancer detection device;
Performing multi-label two-classification on the sample detection image based on the probability analysis sub-model of the multi-mode fusion model to obtain a multi-label classification result of the sample detection image, and performing parameter adjustment on the probability analysis sub-model based on the multi-label classification result and the label of the sample detection image to obtain a trained probability analysis sub-model;
Performing feature selection based on a multi-label classification result of the sample detection image of the sample object, a sample physiological detection result of the sample object and a sample personal attribute to obtain screening features; the screening characteristics are one or more characteristics of classification probability of various labels indicated by the multi-label classification result, detection results of various examination and verification indicated by the sample physiological detection result and various attributes in the sample personal attributes;
And constructing a fusion classification sub-model of the multi-mode fusion model based on the screening characteristics, and carrying out parameter adjustment on the fusion classification sub-model based on the characteristic values corresponding to the screening characteristics of the sample object to obtain a trained fusion classification sub-model.
According to the multi-mode fusion model construction method provided by the invention, the sample detection image comprises scanning images corresponding to a plurality of scanning modes;
the probability analysis submodel based on the multi-mode fusion model carries out multi-label two-classification on the sample detection image to obtain a multi-label classification result of the sample detection image, and the method specifically comprises the following steps:
based on the feature extraction layer in the probability analysis sub-model, respectively extracting features of the scanned images corresponding to the multiple scanning modes to obtain image features of the scanned images corresponding to the multiple scanning modes;
and carrying out feature fusion on image features of the scanned images corresponding to the multiple scanning modes based on the information fusion layer in the probability analysis submodel to obtain fusion features, and carrying out multi-label two-classification based on the fusion features to obtain a multi-label classification result of the sample detection image.
According to the multi-mode fusion model construction method provided by the invention, a plurality of feature extraction layers are arranged in the probability analysis sub-model, and each feature extraction layer is respectively used for extracting features of the scanning images corresponding to the corresponding scanning modes.
According to the method for constructing the multi-modal fusion model provided by the invention, the feature selection is performed on the basis of the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object and the sample personal attribute, so as to obtain screening features, and the method specifically comprises the following steps:
performing Lasso regression analysis on the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object and the sample personal attribute, and determining the screening characteristic based on the classification probability of various labels indicated by the multi-label classification result, the detection result of various inspection tests indicated by the sample physiological detection result and the characteristic that the regression coefficient in various attributes in the sample personal attribute is not 0.
According to the method for constructing the multi-mode fusion model provided by the invention, the sample detection image of the sample detection part of the sample object acquired by the home multi-cancer detection device is acquired, and then the method further comprises the following steps:
Inputting a sample detection image of the sample object to a data quality control model to obtain the quality grade and the multi-label quality type of the corresponding sample detection image output by the data quality control model; the multi-label quality type of the sample detection image characterizes whether the sample detection image is blurred or not and whether the scanning process of the sample detection image is qualified or not;
And discarding the sample detection image of the sample object if the quality level of the sample detection image of the sample object is lower than a preset threshold value or the multi-label quality type of the sample detection image of the sample object indicates that the sample detection image is blurred or the scanning process of the sample detection image is unqualified.
According to the method for constructing the multi-mode fusion model provided by the invention, the sample detection image of the sample object is input into the data quality control model to obtain the quality grade and the multi-label quality type of the corresponding sample detection image output by the data quality control model, and the method specifically comprises the following steps:
extracting the latent features of the sample detection image based on the latent feature extraction layer of the data quality control model;
Based on the full-connection network of the data quality control model, carrying out feature processing on the latent features of the sample detection image to obtain the full-connection features of the sample detection image;
Based on a quality grade evaluation layer of the data quality control model, carrying out quality grade evaluation on the full-connection characteristics of the sample detection image to obtain the quality grade of the sample detection image;
and based on a multi-label quality type classification layer of the data quality control model, performing multi-label quality type classification on the full-connection characteristics of the sample detection image to obtain the multi-label quality type of the sample detection image.
The invention also provides a multi-mode fusion model construction device based on any household multi-cancer detection device, which comprises:
The image acquisition unit is used for acquiring a sample detection image of a sample detection part of a sample object acquired by the home multi-cancer detection device;
The probability analysis submodel construction unit is used for carrying out multi-label two-classification on the sample detection image based on the probability analysis submodel of the multi-mode fusion model to obtain a multi-label classification result of the sample detection image, and carrying out parameter adjustment on the probability analysis submodel based on the risk analysis result and the label of the sample detection image to obtain a trained probability analysis submodel;
the feature selection unit is used for carrying out feature selection based on the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object and the sample personal attribute to obtain screening features; the screening characteristics are one or more characteristics of classification probability of various labels indicated by the multi-label classification result, detection results of various examination and verification indicated by the sample physiological detection result and various attributes in the sample personal attributes;
And the fusion classification sub-model construction unit is used for constructing a fusion classification sub-model of the multi-mode fusion model based on the screening characteristics, and carrying out parameter adjustment on the fusion classification sub-model based on the characteristic values corresponding to the screening characteristics of the sample object to obtain a trained fusion classification sub-model.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the multi-mode fusion model construction method according to any one of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a multimodal fusion model construction method as described in any of the above.
The invention also provides a computer program product comprising a computer program which when executed by a processor implements a multimodal fusion model building method as described in any one of the above.
According to the home multi-cancer detection device and the multi-mode fusion model construction method and device, accurate and reliable detection can be conveniently carried out in a user's home through the home multi-cancer detection device, and the types, the number and the parameters of the device can be flexibly determined based on the types of the diseases to be detected through the replaceable ultrasonic probe transducer, so that complex and various home detection scenes can be flexibly dealt with, and targeted detection can be carried out for various types of the diseases to be detected; in addition, based on a probability analysis sub-model of the multi-modal fusion model, multi-label two-classification is carried out on a sample detection image acquired by a home multi-cancer detection device to obtain a multi-label classification result of the sample detection image, so that parameter adjustment is carried out on the probability analysis sub-model to obtain a trained probability analysis sub-model, then feature selection is carried out on the multi-label classification result of the sample detection image of a sample object, a sample physiological detection result of the sample object and a sample personal attribute to obtain screening features, then a fusion classification sub-model of the multi-modal fusion model is constructed on the basis of the screening features, parameter adjustment is carried out on the fusion classification sub-model on the basis of feature values corresponding to the screening features of the sample object to obtain a trained fusion classification sub-model, and the multi-modal fusion model with accurate multi-modal fusion analysis capability is obtained, so that the accuracy and the specificity of risk screening are improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a household multiple cancer detection device provided by the invention;
FIG. 2 is a schematic flow chart of a method for constructing a multimodal fusion model according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a device for constructing a multimodal fusion model provided by the invention;
Fig. 4 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a schematic structural diagram of a home multiple cancer detection device provided by the present invention, as shown in fig. 1, the device includes: one or more interchangeable ultrasound probe transducers 110, a multiplexed data transmission unit 120, a signal conditioning circuit 130, and a system processing unit 140.
Wherein the type, number and parameters of the replaceable ultrasonic probe transducers 110 are determined based on the type of disease to be detected, and the scanning criteria of the replaceable ultrasonic probe transducers 110 are also determined based on the type of disease to be detected. For different scenarios, the interchangeable ultrasound probe transducer 110 can be interchanged for different scenarios to obtain better image acquisition quality. When the user purchases the service or submits the use case application on various authorization platforms, the professional can determine the type of the disease to be detected according to the symptoms fed back by the user, and the user can actively fill in the type of the disease to be detected, such as abdominal cancer, female cancer, and the like. Depending on the type of disease to be detected, the specific type, number, relevant reference parameters, and scanning criteria of the interchangeable ultrasonic probe transducer 110 used to detect the type of disease to be detected may be automatically retrieved from the database. Wherein, when the type of disease to be detected is an abdominal disease, the replaceable ultrasonic probe transducer 110 comprises a fan-shaped scanning probe, which is suitable for abdominal organ imaging; when the type of condition to be detected is female condition, the interchangeable ultrasonic probe transducer 110 includes a fan-like scanning probe and a tube scanning probe adapted for transvaginal ultrasound for the examination of ovarian cancer and cervical cancer. In addition, the replaceable ultrasonic probe transducer 110 comprises an on-chip ultrasonic micro-transducer array, has a function of combining multiple scanning modes, and simultaneously has a gray-scale B-mode scanning function and a color Doppler mode scanning function. The professional inspector can select the proper replaceable ultrasonic probe transducer 110 according to the specific types and the number of the replaceable ultrasonic probe transducers 110, and configure the replaceable ultrasonic probe transducer according to the related reference parameters. The configured replaceable ultrasonic probe transducer 110 is used for transmitting ultrasonic signals to a part to be detected of a user and collecting echo signals reflected by the part to be detected.
The multiplexing data transmission unit 120 is configured to transmit echo signals reflected by the portion to be detected acquired by the interchangeable ultrasonic probe transducer 110 to the signal conditioning circuit 130, and to transmit analog signals of the probe beam transmitted by the signal conditioning circuit 130 to the interchangeable ultrasonic probe transducer 110, so as to transmit ultrasonic waves again to the portion to be detected. The interface between the replaceable ultrasonic probe transducer 110 and the multiplexing data transmission unit 120 is a unified interface, and can be used for connecting with different types of ultrasonic probes.
The signal conditioning circuit 130 is configured to receive the echo signal reflected from the portion to be detected and transmitted by the multiplexed data transmission unit 120, convert the echo signal into a digital signal, and transmit the digital signal to the system processing unit 140 for processing, and receive the probe beam sent by the system processing unit 140, convert the probe beam into an analog signal, and transmit the analog signal to the multiplexed data transmission unit 120. The signal conditioning circuit 130 includes a T/R Converter for controlling transmission and reception of data, a DAC (Digital-to-Analog Converter) and an ADC (Analog-to-Digital Converter ), among others.
The system processing unit 140 is configured to perform beam forming based on the digital signal of the echo signal transmitted by the signal conditioning circuit 130 to generate frame data, thereby obtaining a probe image, and is configured to generate a new probe beam and transmit the new probe beam to the signal conditioning circuit 130. The system processing unit 140 includes a system processor (Micro Processor Unit, MPU) and two beamformers.
In addition, the home multiple cancer species detection device further comprises an external data transmission module 150, a data storage module 160 and a power supply module 170. The external data transmission module 150 may include a wireless wifi and a USB interface, so as to be convenient to directly connect with a mobile terminal such as a smart phone, a tablet computer or a computer, and further upload to a database of the cloud server.
After the user submits the home detection application, the service organization can send out the professional detection personnel closest to the user to carry the home multiple cancer detection device and other sampling tools to the user's home. And the professional inspector confirms information and authorization with the user according to the sampling flow and specification, then performs ultrasonic scanning of one or more body areas aiming at the type of the disease to be inspected, finishes scanning after the scanning quality evaluation is qualified, and performs data storage and uploading to the server. The scanning process is to scan based on the scanning standard corresponding to the scanning position (the specific scanning position is determined according to the type of the disease to be detected, for example, the scanning position corresponding to the abdominal cancer is abdomen, the scanning position corresponding to the female cancer is abdomen, vagina, etc.), wherein the scanning process comprises the setting of scanning body position, scanning path, etc., the quality control judgment is carried out on the scanning result after the scanning is finished, and the server is uploaded after the scanning is qualified, otherwise, the scanning is carried out again or the online scanning guidance is carried out by contacting the clinical specialist of the company. During scanning, standardized guidance of the scanning process, such as on-line guidance of scanning area positioning, scanning direction and mode, etc., can be performed in an AR-based manner. In addition, liquid biopsy sampling can be performed for the type of disease to be detected, sufficient blood and other body fluids (such as saliva, urine or feces) are obtained, the sample is packaged and stored according to a standard procedure, and the sample is sent back to the mechanism within a set time.
Therefore, the household multi-cancer detection device provided by the embodiment can conveniently and accurately detect in the home of the user, and can flexibly cope with complex and various household detection scenes by flexibly determining the types, the quantity and the parameters of the types of the diseases to be detected based on the replaceable ultrasonic probe transducer.
Based on any of the above embodiments, fig. 2 is a schematic flow chart of a method for constructing a multimodal fusion model according to an embodiment of the present invention, where the method is based on the home multiple cancer detection device provided in the above embodiment, as shown in fig. 2, and the method includes:
Step 210, obtaining a sample detection image of a sample detection part of a sample object acquired by the home multi-cancer detection device;
Step 220, performing multi-label two-classification on the sample detection image based on the probability analysis sub-model of the multi-mode fusion model to obtain a multi-label classification result of the sample detection image, and performing parameter adjustment on the probability analysis sub-model based on a risk analysis result and a label of the sample detection image to obtain a trained probability analysis sub-model;
Step 230, performing feature selection based on the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object and the sample personal attribute to obtain screening features; the screening characteristics are one or more characteristics of classification probability of various labels indicated by the multi-label classification result, detection results of various examination and verification indicated by the sample physiological detection result and various attributes in the sample personal attributes;
And 240, constructing a fusion classification sub-model of the multi-mode fusion model based on the screening features, and carrying out parameter adjustment on the fusion classification sub-model based on the feature values corresponding to the screening features of the sample object to obtain a trained fusion classification sub-model.
In particular, in order to construct a multi-modal fusion model, a sample dataset may be prepared in advance that trains the model. The sample data set includes a sample detection image of a sample detection portion of the sample object collected by the home multi-cancer detection device provided by the above embodiment, and a label of the sample detection image. The labels of the sample detection image are mainly pixel-level labels and case-level labels, the pixel-level labels are mainly labels of organs and target areas, the case-level labels are multi-disease types (such as multi-cancer types) for labeling the case, and the multi-disease types are given by a professional imaging doctor in combination with pathology. In addition, the sample data set can also contain sample detection images of sample detection parts of sample objects collected in the public data set and labels of the sample detection images so as to enrich the number and types of training samples of the sample data set and improve the multi-mode analysis capability of the multi-mode fusion model. Here, the sample detection image of the sample detection site of the sample object collected from the public dataset will be mainly used for model pre-training in the early stage, while the sample detection image of the sample detection site of the sample object collected by the home multi-cancer detection device is used for fine tuning in the later stage.
In some embodiments, after obtaining a sample detection image of a sample detection part of a sample object collected by the home multi-cancer detection device, the sample detection image of the sample object may be input to a data quality control model for data quality analysis, so as to obtain a quality grade and a multi-label quality type of a corresponding sample detection image output by the data quality control model. Wherein the multi-label quality type of the sample detection image characterizes whether the sample detection image is blurred and whether the scanning process of the sample detection image is qualified. And discarding the sample detection image if the quality level of the sample detection image of the sample object is lower than a preset threshold value or the multi-label quality type of the sample detection image of the sample object indicates that the sample detection image is blurred or the scanning process of the sample detection image is unqualified. The sample detection images with different quality are screened out by carrying out data quality analysis on the sample detection images collected by the household multi-cancer detection device, so that the training effect and efficiency of the multi-mode fusion model are improved.
In other embodiments, the data quality control model may extract latent features of the input sample detection image based on a latent feature extraction layer of the data quality control model when performing data quality analysis on the sample detection image. The latent feature extraction layer may be constructed based on an encoder network, which may be a conventional convolutional neural network such as a Resnet, a VGG, or the like, or may be a Transformer network, which is not particularly limited in the embodiment of the present invention. And carrying out feature integration on the latent features of the sample detection image based on the full-connection network of the data quality control model to obtain the full-connection features of the sample detection image. Then, on one branch, based on a quality grade evaluation layer of the data quality control model, carrying out quality grade evaluation on the full-connection characteristics of the sample detection image to obtain the quality grade of the sample detection image; and on the other branch, based on a multi-label quality type classification layer of the data quality control model, performing multi-label quality type classification on the full-connection characteristics of the sample detection image to obtain the multi-label quality type of the sample detection image.
The multi-modal fusion model mainly comprises two parts: the probability analysis sub-model and the fusion classification sub-model are performed in steps. The probability analysis submodel is used for performing multi-label two-classification on the sample detection image to obtain a multi-label classification result of the corresponding sample detection image. The multi-label classification result of the sample detection image comprises probabilities of various disease types, which are predicted by the probability analysis sub-model, of the sample detection image.
In some embodiments, to enhance the capability of the probability analysis sub-model to analyze the image modes, the home multi-cancer detection device may be controlled to perform multi-mode scanning, and the collected sample detection images may include scan images corresponding to multiple scan modes, for example, a scan image corresponding to a gray-scale B-mode scan mode and a scan image corresponding to a color doppler mode scan mode. On the basis, when the multi-label two-classification is carried out on the sample detection image based on the probability analysis submodel of the multi-mode fusion model, the characteristic extraction layer in the probability analysis submodel can be specifically used for respectively extracting the characteristics of the scanning images corresponding to the multiple scanning modes to obtain the image characteristics of the scanning images corresponding to the multiple scanning modes, the information fusion layer in the probability analysis submodel is used for carrying out the characteristic fusion on the image characteristics of the scanning images corresponding to the multiple scanning modes to obtain the fusion characteristics, and the multi-label two-classification is carried out based on the fusion characteristics to obtain the multi-label classification result of the corresponding sample detection image. In order to improve the feature extraction accuracy of the scanned images corresponding to different scanning modes so as to improve the image analysis capability of the probability analysis sub-model, a plurality of feature extraction layers can be arranged in the probability analysis sub-model, wherein each feature extraction layer is respectively used for carrying out feature extraction on the scanned images corresponding to the corresponding scanning modes, namely, the feature extraction layers are in one-to-one correspondence with the scanned images corresponding to the scanning modes so as to carry out targeted feature extraction.
And carrying out parameter adjustment on the probability analysis sub-model based on the multi-label classification result and the labels of each sample detection image to obtain a trained probability analysis sub-model. Based on the trained probability analysis sub-model, the detection images collected by the household multi-cancer detection device (the multi-mode fusion model training stage is a sample detection image of a sample object, and the running stage is a detection image to be analyzed of an object to be detected) can be analyzed and classified in an image mode.
However, the accuracy of the analysis classification result in the single mode of the image mode is still insufficient, so that the multi-mode fusion model can accurately fuse the multi-mode information to fuse the analysis model, thereby improving the accuracy of home primary screening and the guiding significance for subsequent diagnosis, and the fusion analysis can be performed by combining information of other modes outside the image mode. In the training stage of the multimodal fusion model, the information of other modalities includes a sample physiological detection result and a sample personal attribute, the sample physiological detection result may include various detection and detection information such as blood and other body fluids (such as saliva, urine or feces, etc.), for example, tumor biomarkers such as serum AFP and saccharide antigen CA19-9, etc., and a Circulatory Abnormal Cell (CAC) detection result, methylation data, etc., and the sample personal attribute may include clinical indexes such as age, family history and weight change of the corresponding sample object. Therefore, the multi-label classification result of the sample detection image of the sample object, each sample physiological detection result of the sample object and the sample personal attribute can be used as a feature for the subsequent fusion analysis of the sample object, but excessive features can cause a certain overfitting phenomenon, so that the fusion analysis capability with good generalization is difficult to acquire.
Therefore, feature selection can be performed based on the multi-label classification result of the sample detection image of each sample object, the sample physiological detection result of the corresponding sample object and the sample personal attribute, and screening features can be obtained. The classifying probability of the multi-label classifying result indicating various disease type labels is regarded as a characteristic respectively, and the corresponding classifying probability value is the characteristic value of the characteristic; similarly, the detection results of various examination tests indicated by the physiological detection results of the sample are respectively regarded as a feature, and the index value of the corresponding detection result is the feature value of the feature; each type of attribute in the sample personal attribute is regarded as a feature, and the corresponding attribute value is the feature value of the feature. Therefore, the screening feature obtained after feature selection based on the feature values of the respective features corresponding to the respective sample objects is one or more features of the classification probability of various labels indicated by the multi-label classification result, the detection result of various inspection tests indicated by the sample physiological detection result, and various attributes in the sample personal attributes.
In some embodiments, when feature selection is performed based on the multi-label classification result of the sample detection image of each sample object and the sample physiological detection result and the sample personal attribute of the same sample object, lasso regression analysis can be performed on the multi-label classification result of the sample detection image of each sample object and the feature values corresponding to the sample physiological detection result and the sample personal attribute of the same sample object, and then based on the Lasso regression analysis result, features with regression coefficients different from 0 in the classification probability of various labels indicated by the multi-label classification result, the detection result of various inspection tests indicated by the sample physiological detection result and various attributes in the sample personal attribute are screened out as screening features.
After the feature screening is finished, a fusion classification sub-model of the multi-mode fusion model can be constructed based on the determined screening features. The fusion classification sub-model is used for classifying the sample object corresponding to the characteristic value of each screening characteristic based on the sample object input into the current model, and obtaining a fusion classification result of the sample object. Here, the fusion classification result of the sample object includes probabilities of the sample object corresponding to various disease types predicted by the fusion classification sub-model. And carrying out parameter adjustment on the fusion classification sub-model based on the fusion classification result and the label of each sample object, so as to obtain the trained fusion classification sub-model. In some embodiments, the fused classification sub-model may be a random forest model, a multi-layer perceptron, a support vector machine, etc., which is not particularly limited in embodiments of the present invention. After the fusion classification sub-model is trained, the whole multi-mode fusion model is built. The built multi-mode fusion model can perform fusion analysis of multiple disease types on the object to be tested, and a fusion classification result of the object to be tested is obtained. Furthermore, the multi-mode fusion model can be visualized based on the fusion classification result of the object to be detected based on the thermodynamic diagram technology, and a specific risk area is presented for the user.
In summary, the method provided by the embodiment of the invention carries out multi-label two-classification on the sample detection image acquired by the home multi-cancer detection device based on the probability analysis sub-model of the multi-modal fusion model to obtain the multi-label classification result of the sample detection image, so as to carry out parameter adjustment on the probability analysis sub-model to obtain the trained probability analysis sub-model, then carries out feature selection based on the multi-label classification result of the sample detection image of the sample object and the sample physiological detection result and the sample personal attribute of the sample object to obtain screening features, then constructs the fusion classification sub-model of the multi-modal fusion model based on the screening features, carries out parameter adjustment on the fusion classification sub-model based on the feature value corresponding to the screening features of the sample object to obtain the trained fusion classification sub-model, and thus obtains the multi-modal fusion model with accurate multi-modal fusion analysis capability, thereby being beneficial to improving the accuracy and specificity of risk screening.
The multi-modal fusion model construction device provided by the invention is described below, and the multi-modal fusion model construction device described below and the multi-modal fusion model construction method described above can be referred to correspondingly.
Based on any of the above embodiments, fig. 3 is a schematic structural diagram of a device for constructing a multimodal fusion model according to the present invention, as shown in fig. 3, the device includes:
An image acquisition unit 310, configured to acquire a sample detection image of a sample detection portion of a sample object acquired by the home multi-cancer detection device;
The probability analysis sub-model construction unit 320 is configured to perform multi-label two-classification on the sample detection image based on the probability analysis sub-model of the multi-mode fusion model to obtain a multi-label classification result of the sample detection image, and perform parameter adjustment on the probability analysis sub-model based on the risk analysis result and the label of the sample detection image to obtain a trained probability analysis sub-model;
A feature selection unit 330, configured to perform feature selection based on a multi-label classification result of the sample detection image of the sample object, a sample physiological detection result of the sample object, and a sample personal attribute, so as to obtain a screening feature; the screening characteristics are one or more characteristics of classification probability of various labels indicated by the multi-label classification result, detection results of various examination and verification indicated by the sample physiological detection result and various attributes in the sample personal attributes;
The fusion classification sub-model construction unit 340 is configured to construct a fusion classification sub-model of the multi-mode fusion model based on the screening feature, and perform parameter adjustment on the fusion classification sub-model based on a feature value corresponding to the screening feature of the sample object, so as to obtain a trained fusion classification sub-model.
According to the device provided by the embodiment of the invention, based on the probability analysis submodel of the multi-modal fusion model, multi-label two-classification is carried out on the sample detection image obtained by the home multi-cancer detection device to obtain the multi-label classification result of the sample detection image, so that the parameter adjustment is carried out on the probability analysis submodel to obtain the trained probability analysis submodel, then, based on the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object and the sample personal attribute, the feature selection is carried out to obtain the screening feature, then, the fusion classification submodel of the multi-modal fusion model is constructed based on the screening feature, and based on the feature value corresponding to the screening feature of the sample object, the parameter adjustment is carried out on the fusion classification submodel to obtain the trained fusion classification submodel, and the multi-modal fusion model with accurate multi-modal fusion analysis capability is obtained, and the accuracy and the specificity of risk screening are improved.
Based on any one of the above embodiments, the sample detection image includes scan images corresponding to a plurality of scan modes;
the probability analysis submodel based on the multi-mode fusion model carries out multi-label two-classification on the sample detection image to obtain a multi-label classification result of the sample detection image, and the method specifically comprises the following steps:
based on the feature extraction layer in the probability analysis sub-model, respectively extracting features of the scanned images corresponding to the multiple scanning modes to obtain image features of the scanned images corresponding to the multiple scanning modes;
and carrying out feature fusion on image features of the scanned images corresponding to the multiple scanning modes based on the information fusion layer in the probability analysis submodel to obtain fusion features, and carrying out multi-label two-classification based on the fusion features to obtain a multi-label classification result of the sample detection image.
Based on any one of the above embodiments, the probability analysis sub-model includes a plurality of feature extraction layers, and each feature extraction layer is respectively configured to perform feature extraction on a scan image corresponding to a corresponding scan mode.
Based on any of the foregoing embodiments, the selecting a feature based on the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object, and the sample personal attribute to obtain a screening feature specifically includes:
performing Lasso regression analysis on the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object and the sample personal attribute, and determining the screening characteristic based on the classification probability of various labels indicated by the multi-label classification result, the detection result of various inspection tests indicated by the sample physiological detection result and the characteristic that the regression coefficient in various attributes in the sample personal attribute is not 0.
Based on any of the above embodiments, the apparatus further includes a quality control unit configured to perform, after the acquiring the sample detection image of the sample detection portion of the sample object acquired by the home multi-cancer species detection device:
Inputting a sample detection image of the sample object to a data quality control model to obtain the quality grade and the multi-label quality type of the corresponding sample detection image output by the data quality control model; the multi-label quality type of the sample detection image characterizes whether the sample detection image is blurred or not and whether the scanning process of the sample detection image is qualified or not;
And discarding the sample detection image of the sample object if the quality level of the sample detection image of the sample object is lower than a preset threshold value or the multi-label quality type of the sample detection image of the sample object indicates that the sample detection image is blurred or the scanning process of the sample detection image is unqualified.
Based on any one of the above embodiments, the inputting the sample detection image of the sample object to the data quality control model to obtain the quality level and the multi-label quality type of the corresponding sample detection image output by the data quality control model specifically includes:
extracting the latent features of the sample detection image based on the latent feature extraction layer of the data quality control model;
Based on the full-connection network of the data quality control model, carrying out feature processing on the latent features of the sample detection image to obtain the full-connection features of the sample detection image;
Based on a quality grade evaluation layer of the data quality control model, carrying out quality grade evaluation on the full-connection characteristics of the sample detection image to obtain the quality grade of the sample detection image;
and based on a multi-label quality type classification layer of the data quality control model, performing multi-label quality type classification on the full-connection characteristics of the sample detection image to obtain the multi-label quality type of the sample detection image.
Fig. 4 is a schematic structural diagram of an electronic device according to the present invention, as shown in fig. 4, the electronic device may include: processor 410, memory 420, communication interface (Communications Interface) 430, and communication bus 440, wherein processor 410, memory 420, and communication interface 430 communicate with each other via communication bus 440. The processor 410 may invoke logic instructions in the memory 420 to perform a multi-modal fusion model building method based on the above-described home multi-cancer detection apparatus, the method comprising: acquiring a sample detection image of a sample detection part of a sample object acquired by the home multi-cancer detection device; performing multi-label two-classification on the sample detection image based on the probability analysis sub-model of the multi-mode fusion model to obtain a multi-label classification result of the sample detection image, and performing parameter adjustment on the probability analysis sub-model based on the multi-label classification result and the label of the sample detection image to obtain a trained probability analysis sub-model; performing feature selection based on a multi-label classification result of the sample detection image of the sample object, a sample physiological detection result of the sample object and a sample personal attribute to obtain screening features; the screening characteristics are one or more characteristics of classification probability of various labels indicated by the multi-label classification result, detection results of various examination and verification indicated by the sample physiological detection result and various attributes in the sample personal attributes; and constructing a fusion classification sub-model of the multi-mode fusion model based on the screening characteristics, and carrying out parameter adjustment on the fusion classification sub-model based on the characteristic values corresponding to the screening characteristics of the sample object to obtain a trained fusion classification sub-model.
Further, the logic instructions in the memory 420 described above may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, including a computer program stored on a non-transitory computer readable storage medium, the computer program including program instructions, which when executed by a computer, are capable of executing the method for constructing a multimodal fusion model based on the above-mentioned home multiple cancer species detection device, provided by the methods, the method comprising: acquiring a sample detection image of a sample detection part of a sample object acquired by the home multi-cancer detection device; performing multi-label two-classification on the sample detection image based on the probability analysis sub-model of the multi-mode fusion model to obtain a multi-label classification result of the sample detection image, and performing parameter adjustment on the probability analysis sub-model based on the multi-label classification result and the label of the sample detection image to obtain a trained probability analysis sub-model; performing feature selection based on a multi-label classification result of the sample detection image of the sample object, a sample physiological detection result of the sample object and a sample personal attribute to obtain screening features; the screening characteristics are one or more characteristics of classification probability of various labels indicated by the multi-label classification result, detection results of various examination and verification indicated by the sample physiological detection result and various attributes in the sample personal attributes; and constructing a fusion classification sub-model of the multi-mode fusion model based on the screening characteristics, and carrying out parameter adjustment on the fusion classification sub-model based on the characteristic values corresponding to the screening characteristics of the sample object to obtain a trained fusion classification sub-model.
In still another aspect, the present invention further provides a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, is implemented to perform the above-provided multi-modal fusion model construction method based on the home multiple cancer species detection device, the method comprising: acquiring a sample detection image of a sample detection part of a sample object acquired by the home multi-cancer detection device; performing multi-label two-classification on the sample detection image based on the probability analysis sub-model of the multi-mode fusion model to obtain a multi-label classification result of the sample detection image, and performing parameter adjustment on the probability analysis sub-model based on the multi-label classification result and the label of the sample detection image to obtain a trained probability analysis sub-model; performing feature selection based on a multi-label classification result of the sample detection image of the sample object, a sample physiological detection result of the sample object and a sample personal attribute to obtain screening features; the screening characteristics are one or more characteristics of classification probability of various labels indicated by the multi-label classification result, detection results of various examination and verification indicated by the sample physiological detection result and various attributes in the sample personal attributes; and constructing a fusion classification sub-model of the multi-mode fusion model based on the screening characteristics, and carrying out parameter adjustment on the fusion classification sub-model based on the characteristic values corresponding to the screening characteristics of the sample object to obtain a trained fusion classification sub-model.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. The method for constructing the multi-mode fusion model is characterized by comprising the following steps of:
Acquiring a sample detection image of a sample detection part of a sample object acquired by a home multi-cancer detection device; the household multiple cancer detection device comprises one or more replaceable ultrasonic probe transducers, a multiplexing data transmission unit, a signal conditioning circuit and a system processing unit; the type, the number and the parameters of the replaceable ultrasonic probe transducers are determined based on the type of the disease to be detected, and are used for transmitting ultrasonic signals to the position to be detected and collecting echo signals reflected by the position to be detected; the multiplexing data transmission unit is used for transmitting echo signals reflected by the to-be-detected part acquired by the replaceable ultrasonic probe transducer to the signal conditioning circuit and transmitting detection beam analog signals sent by the signal conditioning circuit to the replaceable ultrasonic probe transducer; the signal conditioning circuit is used for receiving echo signals reflected by the part to be detected, converting the echo signals into digital signals and transmitting the digital signals to the system processing unit, and receiving detection beams sent by the system processing unit, converting the detection beams into analog signals and transmitting the analog signals to the multiplexing data transmission unit; the system processing unit is used for carrying out beam forming on the basis of the digital signals of the echo signals transmitted by the signal conditioning circuit to obtain a detection image, and generating a new detection beam and transmitting the new detection beam to the signal conditioning circuit;
Performing multi-label two-classification on the sample detection image based on the probability analysis sub-model of the multi-mode fusion model to obtain a multi-label classification result of the sample detection image, and performing parameter adjustment on the probability analysis sub-model based on the multi-label classification result and the label of the sample detection image to obtain a trained probability analysis sub-model;
Performing feature selection based on a multi-label classification result of the sample detection image of the sample object, a sample physiological detection result of the sample object and a sample personal attribute to obtain screening features; the screening characteristics are one or more characteristics of classification probability of various labels indicated by the multi-label classification result, detection results of various examination and verification indicated by the sample physiological detection result and various attributes in the sample personal attributes;
Constructing a fusion classification sub-model of the multi-mode fusion model based on the screening characteristics, and carrying out parameter adjustment on the fusion classification sub-model based on characteristic values corresponding to the screening characteristics of the sample object to obtain a trained fusion classification sub-model;
The feature selection is performed based on the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object and the sample personal attribute, so as to obtain screening features, which specifically includes:
performing Lasso regression analysis on the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object and the sample personal attribute, and determining the screening characteristic based on the classification probability of various labels indicated by the multi-label classification result, the detection result of various inspection tests indicated by the sample physiological detection result and the characteristic that the regression coefficient in various attributes in the sample personal attribute is not 0.
2. The method for constructing a multi-modal fusion model according to claim 1, wherein the sample detection image includes scan images corresponding to a plurality of scan modes;
the probability analysis submodel based on the multi-mode fusion model carries out multi-label two-classification on the sample detection image to obtain a multi-label classification result of the sample detection image, and the method specifically comprises the following steps:
based on the feature extraction layer in the probability analysis sub-model, respectively extracting features of the scanned images corresponding to the multiple scanning modes to obtain image features of the scanned images corresponding to the multiple scanning modes;
and carrying out feature fusion on image features of the scanned images corresponding to the multiple scanning modes based on the information fusion layer in the probability analysis submodel to obtain fusion features, and carrying out multi-label two-classification based on the fusion features to obtain a multi-label classification result of the sample detection image.
3. The method for constructing a multi-modal fusion model according to claim 2, wherein the probability analysis sub-model has a plurality of feature extraction layers, each feature extraction layer being used for extracting features of a scanned image corresponding to a corresponding scan mode.
4. The method for constructing a multimodal fusion model as defined in claim 1, wherein the acquiring the sample detection image of the sample detection portion of the sample object acquired by the home multi-cancer species detection device further comprises:
Inputting a sample detection image of the sample object to a data quality control model to obtain the quality grade and the multi-label quality type of the corresponding sample detection image output by the data quality control model; the multi-label quality type of the sample detection image characterizes whether the sample detection image is blurred or not and whether the scanning process of the sample detection image is qualified or not;
And discarding the sample detection image of the sample object if the quality level of the sample detection image of the sample object is lower than a preset threshold value or the multi-label quality type of the sample detection image of the sample object indicates that the sample detection image is blurred or the scanning process of the sample detection image is unqualified.
5. The method for constructing a multi-modal fusion model according to claim 4, wherein the inputting the sample detection image of the sample object to the data quality control model to obtain the quality level and the multi-label quality type of the corresponding sample detection image output by the data quality control model specifically includes:
extracting the latent features of the sample detection image based on the latent feature extraction layer of the data quality control model;
Based on the full-connection network of the data quality control model, carrying out feature processing on the latent features of the sample detection image to obtain the full-connection features of the sample detection image;
Based on a quality grade evaluation layer of the data quality control model, carrying out quality grade evaluation on the full-connection characteristics of the sample detection image to obtain the quality grade of the sample detection image;
and based on a multi-label quality type classification layer of the data quality control model, performing multi-label quality type classification on the full-connection characteristics of the sample detection image to obtain the multi-label quality type of the sample detection image.
6. The method for constructing a multi-modal fusion model according to claim 1, wherein the replaceable ultrasonic probe transducer has a multi-mode scanning function for acquiring scan images corresponding to a plurality of scan modes.
7. The method for constructing a multi-modal fusion model according to claim 1, wherein the home multi-cancer species detection device further comprises an external data transmission module, a data storage module and a power supply module.
8. A multi-modal fusion model building apparatus, comprising:
The image acquisition unit is used for acquiring a sample detection image of a sample detection part of a sample object acquired by the home multi-cancer detection device; the household multiple cancer detection device comprises one or more replaceable ultrasonic probe transducers, a multiplexing data transmission unit, a signal conditioning circuit and a system processing unit; the type, the number and the parameters of the replaceable ultrasonic probe transducers are determined based on the type of the disease to be detected, and are used for transmitting ultrasonic signals to the position to be detected and collecting echo signals reflected by the position to be detected; the multiplexing data transmission unit is used for transmitting echo signals reflected by the to-be-detected part acquired by the replaceable ultrasonic probe transducer to the signal conditioning circuit and transmitting detection beam analog signals sent by the signal conditioning circuit to the replaceable ultrasonic probe transducer; the signal conditioning circuit is used for receiving echo signals reflected by the part to be detected, converting the echo signals into digital signals and transmitting the digital signals to the system processing unit, and receiving detection beams sent by the system processing unit, converting the detection beams into analog signals and transmitting the analog signals to the multiplexing data transmission unit; the system processing unit is used for carrying out beam forming on the basis of the digital signals of the echo signals transmitted by the signal conditioning circuit to obtain a detection image, and generating a new detection beam and transmitting the new detection beam to the signal conditioning circuit;
The probability analysis submodel construction unit is used for carrying out multi-label two-classification on the sample detection image based on the probability analysis submodel of the multi-mode fusion model to obtain a multi-label classification result of the sample detection image, and carrying out parameter adjustment on the probability analysis submodel based on the risk analysis result and the label of the sample detection image to obtain a trained probability analysis submodel;
the feature selection unit is used for carrying out feature selection based on the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object and the sample personal attribute to obtain screening features; the screening characteristics are one or more characteristics of classification probability of various labels indicated by the multi-label classification result, detection results of various examination and verification indicated by the sample physiological detection result and various attributes in the sample personal attributes;
the fusion classification sub-model construction unit is used for constructing a fusion classification sub-model of the multi-mode fusion model based on the screening characteristics, and carrying out parameter adjustment on the fusion classification sub-model based on the characteristic values corresponding to the screening characteristics of the sample object to obtain a trained fusion classification sub-model;
The feature selection is performed based on the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object and the sample personal attribute, so as to obtain screening features, which specifically includes:
performing Lasso regression analysis on the multi-label classification result of the sample detection image of the sample object, the sample physiological detection result of the sample object and the sample personal attribute, and determining the screening characteristic based on the classification probability of various labels indicated by the multi-label classification result, the detection result of various inspection tests indicated by the sample physiological detection result and the characteristic that the regression coefficient in various attributes in the sample personal attribute is not 0.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311185051.1A CN117218433B (en) | 2023-09-13 | 2023-09-13 | Household multi-cancer detection device and multi-mode fusion model construction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311185051.1A CN117218433B (en) | 2023-09-13 | 2023-09-13 | Household multi-cancer detection device and multi-mode fusion model construction method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117218433A CN117218433A (en) | 2023-12-12 |
| CN117218433B true CN117218433B (en) | 2024-08-27 |
Family
ID=89038398
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311185051.1A Active CN117218433B (en) | 2023-09-13 | 2023-09-13 | Household multi-cancer detection device and multi-mode fusion model construction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117218433B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103202712A (en) * | 2012-01-17 | 2013-07-17 | 三星电子株式会社 | Probe Device, Server, System For Diagnosing Ultrasound Image, And Method Of Processing Ultrasound Image |
| CN111096766A (en) * | 2020-02-26 | 2020-05-05 | 深圳市威尔德医疗电子有限公司 | Portable color Doppler ultrasound device and color Doppler ultrasound system |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003319938A (en) * | 2002-04-30 | 2003-11-11 | Matsushita Electric Ind Co Ltd | Ultrasonic diagnostic device |
| US7998072B2 (en) * | 2003-12-19 | 2011-08-16 | Siemens Medical Solutions Usa, Inc. | Probe based digitizing or compression system and method for medical ultrasound |
| KR101798082B1 (en) * | 2013-09-27 | 2017-11-15 | 삼성전자주식회사 | Probe device, server, ultrasound image diagnosis system, and ultrasound image processing method |
| CN103784166B (en) * | 2014-03-03 | 2015-08-19 | 哈尔滨工业大学 | Multifunctional all digital ultrasound diagnostic system |
| CN107616784B (en) * | 2017-09-19 | 2020-09-22 | 华南师范大学 | Wide-field photoacoustic ultrasonic breast coronal plane scanning imaging device and method based on 1024 linear array detection |
| CN112004478B (en) * | 2018-01-19 | 2024-09-10 | 皇家飞利浦有限公司 | Automatic path correction during multi-modality fusion targeted biopsy |
| WO2020047803A1 (en) * | 2018-09-06 | 2020-03-12 | 深圳迈瑞生物医疗电子股份有限公司 | Ultrasonic probe and method for processing ultrasonic echo signal thereby, and ultrasonic imaging device |
| CN110313941B (en) * | 2019-08-01 | 2021-02-19 | 无锡海斯凯尔医学技术有限公司 | Data processing method, device, equipment and storage medium |
| CN115019405A (en) * | 2022-05-27 | 2022-09-06 | 中国科学院计算技术研究所 | Multi-modal fusion-based tumor classification method and system |
| CN115035347A (en) * | 2022-06-24 | 2022-09-09 | 微梦创科网络科技（中国）有限公司 | Picture identification method and device and electronic equipment |
| CN115082437B (en) * | 2022-07-22 | 2023-04-07 | 浙江省肿瘤医院 | Tumor prediction system and method based on tongue picture image and tumor marker and application |
| CN115662504A (en) * | 2022-11-02 | 2023-01-31 | 大连理工大学 | Multi-angle fusion-based biological omics data analysis method |
| CN116452851A (en) * | 2023-03-17 | 2023-07-18 | 中山大学肿瘤防治中心（中山大学附属肿瘤医院、中山大学肿瘤研究所） | Training method and device for disease classification model, terminal and readable storage medium |
| CN116098652B (en) * | 2023-04-12 | 2023-07-11 | 中国医学科学院北京协和医院 | Ultrasonic contrast blood pressure measuring device and method based on subharmonic resonance frequency |
-
2023
- 2023-09-13 CN CN202311185051.1A patent/CN117218433B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103202712A (en) * | 2012-01-17 | 2013-07-17 | 三星电子株式会社 | Probe Device, Server, System For Diagnosing Ultrasound Image, And Method Of Processing Ultrasound Image |
| CN111096766A (en) * | 2020-02-26 | 2020-05-05 | 深圳市威尔德医疗电子有限公司 | Portable color Doppler ultrasound device and color Doppler ultrasound system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117218433A (en) | 2023-12-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11950959B2 (en) | Ultrasound system with automated dynamic setting of imaging parameters based on organ detection | |
| CN114343715B (en) | Method for measuring parameters in ultrasonic image and ultrasonic imaging system | |
| US7244230B2 (en) | Computer aided diagnostic assistance for medical imaging | |
| US20110245623A1 (en) | Medical Diagnosis Using Community Information | |
| CN112040876A (en) | Adaptive ultrasound scanning | |
| CN111971688A (en) | Ultrasound system with artificial neural network for retrieving imaging parameter settings of relapsing patients | |
| US20210177374A1 (en) | Biometric measurement and quality assessment | |
| CN112201336B (en) | Remote consultation system and method based on AI diagnosis | |
| US12133764B2 (en) | Systems and methods for an adaptive interface for an ultrasound imaging system | |
| CN113194837B (en) | System and method for frame indexing and image review | |
| KR20190001489A (en) | Ultrasound Imaging Apparatus and Controlling Method Thereof | |
| CN115813434A (en) | Method and system for automated assessment of fractional limb volume and fat lean mass from fetal ultrasound scans | |
| CN117218433B (en) | Household multi-cancer detection device and multi-mode fusion model construction method and device | |
| CN114202504A (en) | Carotid artery ultrasonic automatic Doppler method, ultrasonic equipment and storage medium | |
| CN111528918B (en) | Tumor volume change trend graph generation device after ablation, equipment and storage medium | |
| CN111696085B (en) | Rapid ultrasonic evaluation method and equipment for lung impact injury condition on site | |
| CN115813433A (en) | Follicle measuring method based on two-dimensional ultrasonic imaging and ultrasonic imaging system | |
| CN112614123A (en) | Ultrasonic image identification method and related device | |
| RU2814539C1 (en) | System for diagnosing malignant and benign skin new growths | |
| US20240363205A1 (en) | System for determining dynamic examination process | |
| US20220101518A1 (en) | System and method for stylizing a medical image | |
| WO2023185030A1 (en) | Imaging parameter configuration method | |
| CN112513674B (en) | Ultrasound system for automatically and dynamically setting imaging parameters based on organ detection | |
| KR20230005633A (en) | Calculating method for volume of ovarian endometrioma, predicting method and apparatus for prognosis of endometriosis | |
| CN115281717A (en) | Examination protocol statistical method, examination protocol statistical system, imaging equipment and data processing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |