CN114627136B - Tongue image segmentation and alignment method based on feature pyramid network - Google Patents
Tongue image segmentation and alignment method based on feature pyramid network Download PDFInfo
- Publication number
- CN114627136B CN114627136B CN202210106423.6A CN202210106423A CN114627136B CN 114627136 B CN114627136 B CN 114627136B CN 202210106423 A CN202210106423 A CN 202210106423A CN 114627136 B CN114627136 B CN 114627136B
- Authority
- CN
- China
- Prior art keywords
- tongue
- mapping
- picture
- formula
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000003709 image segmentation Methods 0.000 title claims abstract description 11
- 238000013507 mapping Methods 0.000 claims abstract description 47
- 238000012545 processing Methods 0.000 claims abstract description 8
- 230000011218 segmentation Effects 0.000 claims description 20
- 238000011176 pooling Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000003745 diagnosis Methods 0.000 abstract description 10
- 230000004927 fusion Effects 0.000 abstract description 4
- 238000011161 development Methods 0.000 abstract description 3
- 230000018109 developmental process Effects 0.000 abstract description 3
- 238000001514 detection method Methods 0.000 abstract description 2
- 210000000746 body region Anatomy 0.000 abstract 1
- 239000003814 drug Substances 0.000 description 7
- 230000008569 process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000002405 diagnostic procedure Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000001717 pathogenic effect Effects 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 210000001835 viscera Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a tongue image segmentation and alignment method based on a feature pyramid network, which comprises the steps of constructing a feature pyramid, carrying out multi-scale fusion on the detail features of a lower layer and the semantic features of a higher layer of the network, intercepting a tongue body region by utilizing a target detection network, and then further generating a mask for each feature image to finish tongue image segmentation; the segmented tongue images are further processed, alignment of the tongue images is achieved through conformal mapping, an efficient and accurate method is provided for malformed medical image processing and diagnosis, and the method has a certain significance for objectification development of tongue diagnosis.
Description
Technical Field
The invention relates to the fields of computer vision, image processing and traditional Chinese medicine tongue diagnosis, in particular to a tongue image segmentation and alignment method based on a characteristic pyramid network.
Background
The theory of traditional Chinese medicine considers that the tongue is a window for mapping the viscera changes in the human body, and reflects the pathogenic qi nature, advance and retreat of the disease and the healthy qi. The tongue inspection of traditional Chinese medicine is an important component of inspection in traditional Chinese medicine, and refers to a method for judging the health condition of a human body by observing tongue features such as tongue body, tongue quality, tongue fur, sublingual collaterals and the like, and is one of main basis in the clinical diagnosis process of traditional Chinese medicine.
The tongue diagnosis is painless and noninvasive, and is one of the most commonly used diagnostic methods in traditional Chinese medicine. The appearance characteristics of the tongue body such as color, texture, shape, tongue fur and the like reveal a great deal of information of the health condition of the human body. However, traditional tongue diagnosis is highly dependent on the experience of the clinician, and different doctors diagnosing the same patient may draw different conclusions, which makes traditional Chinese medicine tongue diagnosis less objective. These deficiencies can be improved by a computer-aided tongue diagnosis method, but the collected tongue picture also comprises background areas such as faces, lips and the like, and some tongue bodies are askew, which affects the subsequent auxiliary diagnosis and treatment, so that the tongue bodies need to be segmented and aligned first.
At present, the traditional tongue segmentation method is roughly divided into three types, namely an edge-based method, a region-based method and a region-and edge fusion-based method. The method based on the region is characterized in that a watershed algorithm is used for dividing the tongue image into a plurality of small regions, and then the regions based on the color similarity are combined to obtain the final tongue body segmentation. For edge-based methods, edge initialization is required to segment the final tongue. The method is based on the fusion method of the region and the edge, firstly, the region of interest of the tongue body is extracted by utilizing color information, and then, the region of interest is used for replacing an original image to carry out subsequent segmentation. Although these approaches have met with some success, certain limitations remain. For example, the lips and the tongue are very similar in color and require additional pretreatment, which complicates the whole process and makes it impossible to accurately segment the tongue with smooth edges.
In recent years, with the rapid development of deep learning, a method based on a deep convolutional neural network is developed to improve the robustness of tongue segmentation. Because the deep neural network has strong characteristic learning capability, the deep learning-based method has higher performance than the traditional tongue segmentation method. The shallow layer part in the deep network has more detail information, the high layer part has rich semantic information, however, most networks are segmented by utilizing the last high layer characteristic, but the shallow layer detail characteristic is ignored, and the shallow layer detail information can improve the segmentation accuracy to a certain extent. The ideal segmentation effect cannot be achieved using a single high-level feature.
Disclosure of Invention
In order to solve the problems of the traditional method and the depth network in tongue image segmentation, through the thought of learning a residual error network, the invention provides a tongue image segmentation and alignment method based on a feature pyramid network.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: a tongue image segmentation and alignment method based on a feature pyramid network comprises the following steps:
step 1: collecting a required tongue picture by using tongue picture collecting equipment, and carrying out data enhancement processing on the picture;
step 2: manually marking the tongue bodies in all the picture data;
step 3: preprocessing the marked picture, and then sending the preprocessed picture into a constructed feature pyramid network to extract features to obtain an effective feature layer;
step 4: sending the effective feature layer extracted in the step 3 into an RPN to obtain a suggestion frame;
step 5: carrying out maximum pooling operation on all the suggestion boxes obtained in the step 4;
step 6: connecting two fully-connected network layers after the suggestion frame subjected to the maximum pooling operation, respectively judging whether an object is contained in the suggestion frame, and then adjusting the suggestion frame to obtain a prediction frame;
step 7: taking the prediction frame obtained in the step 6 as a region interception part of the mask model, and classifying pixel points by using the mask model after interception to obtain a semantic segmentation result;
step 8: calculating boundary areas and Fourier coefficients of the segmented tongue images;
step 9: calculating corresponding areas of the boundaries of the unit disc and the tongue picture;
step 10: after determining the boundary value, calculating the mapping of the internal region by using a Kexil integral formula;
step 11: constructing a conformal map, which updates the real part and the imaginary part of the function through iteration;
step 12: outputting the aligned image.
Further, in the step 1, when the data enhancement processing is performed on the picture, the data set is expanded through rotation and horizontal overturn.
Further, step 2 adopts lableme software when manually marking the tongue body in all the picture data.
Further, in step 5, all the suggestion boxes are fixed to 7x7 features after the maximum pooling operation.
Further, the feature pyramid network is constructed from the middle 4 bnecks of mobiletv 2.
Compared with the prior art, the invention has the beneficial effects that: according to the invention, by constructing the feature pyramid, multi-scale fusion is carried out on the low-level detail features and the high-level semantic features of the network, a tongue region is firstly intercepted by utilizing a target detection network, and then a mask is further generated for each feature map, so that tongue image segmentation is completed; the segmented tongue images are further processed, alignment of the tongue images is achieved through conformal mapping, an efficient and accurate method is provided for malformed medical image processing and diagnosis, and the method has a certain significance for objectification development of tongue diagnosis.
Drawings
FIG. 1 is a flow chart of a tongue segmentation and alignment method based on a feature pyramid network of the present invention;
FIG. 2 is a schematic view of the shape of the unit disks mentioned in the examples;
FIG. 3 is a schematic illustration of the shape of the standard tongue as referred to in the examples;
fig. 4 is a tongue labeling diagram given in the example.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without making any inventive effort based on the embodiments of the present invention are within the scope of protection of the present invention.
In the invention, a tongue image segmentation and alignment method based on a feature pyramid network is provided, wherein a Mobilenetv2 is used as a main feature extraction network, the feature pyramid network is constructed by taking the output of 4 bnecks in the middle of the Mobilenetv2, and the attention of the network to detail features is enhanced through multi-scale feature fusion, so that more accurate segmentation is facilitated. The segmented images are aligned using a conformal mapping based tongue alignment method, the main flow of which is described in detail below.
The invention is mainly used for dividing the tongue picture shot in the standard environment, and the shot picture comprises a face part and a lip part. Therefore, in order to divide the tongue more conveniently and rapidly, the specific position of the tongue is detected first, and the required tongue part is selected by a frame, so that the complete tongue is divided. The invention is mainly divided into two parts, wherein the first part is used for positioning and dividing a tongue picture, the second part is used for realizing tongue picture alignment through conformal mapping on the divided tongue picture, the flow diagram of the invention is shown in figure 1, and the specific steps are as follows:
1. tongue positioning and segmentation
The positioning of the tongue body is to firstly detect the position of the tongue body in the image, wherein the tongue body in the image is a foreground point, the rest part is a background point, and more accurate prediction frames are obtained through continuous adjustment, and meanwhile, a mask is generated for each prediction frame, so that the tongue body is segmented. And replacing a backup in the network with a mobilent v2 by using a Mask R-CNN network model, and combining the high-resolution and low-resolution feature images to make full use of low-layer detail information and high-layer semantic information to realize more accurate segmentation. The specific work is as follows:
(1) Basic feature extraction: firstly, inputting a tongue picture into a network, firstly, obtaining a feature picture through standard convolution, then expanding a channel of the feature picture by using 1×1 convolution, enriching feature quantity, then performing feature integration by using 3×3 convolution, finally compressing by using 1×1 convolution, and finally connecting the input feature picture with the compressed feature picture by using jump connection, thereby enriching feature quantity and reducing parameter quantity. For the extracted high-level feature images, the spatial resolution of the high-level feature images is improved to be the same as that of the feature images of the upper layer in a deconvolution mode, the high-level feature images and the low-level feature images are fused in an addition mode, then the fused feature images are convolved to generate a common feature layer, and the fused feature images with multiple scales are continuously and iteratively generated for subsequent use.
(2) Obtaining a prediction frame: and sending the extracted multi-scale effective feature map into an RPN (remote procedure network) to generate a suggestion frame, removing some overlapped suggestion frames through non-maximum value suppression operation, sending the suggestion frame generated by the RPN and the multi-scale effective feature layer to an ROI (region planning) layer together, fixing all region planning to be the same size, carrying out concat connection on the ROI generated in the process, and sending the ROI to a full-connection layer and a mask module. The fully connected layer can conduct category prediction and regression of the bounding box to generate a more accurate prediction box.
(3) mask semantic segmentation: the Mask module outputs a binary Mask for each region of interest selected by the box while performing classification and bounding box regression. The mask branch first performs a max pulling operation on the region of interest to a size of 14×14×256, then performs two deconvolutions, where the convolution kernels used are all 3×3, and outputs a 28×28×80 mask through the final deconvolution operation. Sigmoid is applied to each pixel in the region of interest, and then the average of the cross entropy of all pixels over the region of interest is taken as the segmentation penalty. The class probability of each pixel can be obtained through the part, so that semantic segmentation is realized; pixels of the tongue image are classified into 2 classes, namely, a tongue region and a background region.
2. Tongue alignment
For the segmented tongue images, a self-adaptive alignment method is used, and standard alignment can be realized for the segmented tongue images with different shapes. Firstly, constructing regional mapping of the segmented tongue boundary by utilizing a Fourier descriptor, then expanding the mapping to an internal region by utilizing the combination of a Kexil method and a finite difference method, and finally realizing mapping from the segmented inclined tongue image to the standard tongue image by utilizing Riemann mapping. Fig. 2 and 3 are schematic views of the shapes of the unit disks and standard tongue images mentioned herein, respectively.
In the whole mapping process, two mappings are obtained, namely, the mapping from the segmented tongue picture to the unit circle and the mapping from the segmented tongue picture to the standard tongue picture. In the Riemann mapping, there is a mapping from the original tongue to the unit disk, and there is a mapping from the standard tongue to the unit disk, and at this time there will be an inverse mapping from the unit disk to the standard tongue, and by this inverse mapping, a composite mapping from the segmented original tongue to the standard tongue is obtained.
Assuming that the tongue picture area is omega 1, the unit disc is D, and the standard tongue picture area is omega 2; firstly, the mapping from D to omega 1 and the mapping from D to omega 2 are obtained; the mapping from Ω 1 to Ω 2 is then found, i.e. what we finally need. The method mainly comprises the following steps:
the first step: firstly, calculating a corresponding tongue picture boundary, and calculating a boundary region by using a Fourier descriptor through a formula (1):
and a second step of: on the basis of obtaining the boundary correspondence, further calculating the mapping of the internal region, wherein the cauchy integral formula is shown in formula (2), and when Γ is a unit circle, the cauchy integral formula becomes formula (3):
wherein z and z 0 Points on the boundary and interior, respectively, f (z 0 ) And F (z) 0 ) Mapping representing boundary and interior regions, respectively:
wherein z is k ＝e 2jπ(k/K) Represents the form of a dot on a unit circle, phi k Representing the corresponding tongue boundary.
And a third step of: constructing a conformal mapping, and setting functions of the conformal mapping as follows:
F＝U+j·V (5)
fourth step: calculating the mapping of the original tongue image to the unit circle:
assume that the original tongue is denoted as I 0 [u,v,w]The image of the original image mapped onto the unit circle is expressed as J [ x, y, w]The expression can be expressed as follows:
J[x,y,w]＝I 0 [U(x,y),V(x,y),w] (6)
wherein (u, v) ε Ω 1 (original tongue area), (x, y) ∈d (unit circle), w=1, 2,3 represents RGB three channels.
Fifth step: omega determination 1 To omega 2 Mapping of I 0 [u,v,w]The original tongue picture is represented by the image,representing the standard tongue picture after alignment; for arbitrary->Its corresponding pixel position on the unit circle is denoted +.>The mapping from unit circle to standard tongue image is thus expressed as:
the mapping of the original image to the unit disk is calculated by the formula (6), and the formula (6) is substituted into the formula (7) to obtain the following formula:
thus far, the original image I can be obtained by the formula (8) 0 I to standard tongue picture 1 Is mapped to the mapping of (a).
The specific implementation manner of this embodiment is as follows:
step 1: collecting a required tongue picture by using tongue picture collecting equipment, carrying out data enhancement processing on the picture, and expanding the data by rotation and horizontal overturning;
step 2: manually marking the tongue bodies in all the picture data by using lableme software; the tongue picture annotation diagram given in the embodiment is shown in fig. 4;
step 3: preprocessing the marked picture, and then sending the preprocessed picture into a constructed feature pyramid network to extract features to obtain an effective feature layer;
the feature pyramid network is constructed by the output of the middle 4 bnecks of the Mobilenetv 2;
step 4: sending the effective feature layer extracted in the step 3 into an RPN to obtain a suggestion frame;
step 5: performing maximum pooling operation on all the suggestion frames obtained in the step 4, and fixing all the suggestion frames to 7x7 features after the maximum pooling operation;
step 6: connecting two fully-connected network layers after the suggestion frame subjected to the maximum pooling operation, respectively judging whether an object is contained in the suggestion frame, and then adjusting the suggestion frame to obtain a prediction frame;
step 7: taking the prediction frame obtained in the step 6 as a region interception part of the mask model, and classifying pixel points by using the mask model after interception to obtain a semantic segmentation result;
step 8: calculating boundary areas and Fourier coefficients of the segmented tongue images;
step 9: calculating corresponding areas of the boundaries of the unit disc and the tongue picture;
step 10: after determining the boundary value, calculating the mapping of the internal region by using a Kexil integral formula;
step 11: constructing a conformal map, which updates the real part and the imaginary part of the function through iteration;
step 12: outputting the aligned images, and ending the algorithm.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (5)
1. A tongue image segmentation and alignment method based on a feature pyramid network is characterized by comprising the following steps:
step 1: collecting a required tongue picture by using tongue picture collecting equipment, and carrying out data enhancement processing on the picture;
step 2: manually marking the tongue bodies in all the picture data;
step 3: preprocessing the marked picture, and then sending the preprocessed picture into a constructed feature pyramid network to extract features to obtain an effective feature layer;
step 4: sending the effective feature layer extracted in the step 3 into an RPN to obtain a suggestion frame;
step 5: carrying out maximum pooling operation on all the suggestion boxes obtained in the step 4;
step 6: connecting two fully-connected network layers after the suggestion frame subjected to the maximum pooling operation, respectively judging whether an object is contained in the suggestion frame, and then adjusting the suggestion frame to obtain a prediction frame;
step 7: taking the prediction frame obtained in the step 6 as a region interception part of the mask model, and classifying pixel points by using the mask model after interception to obtain a semantic segmentation result;
step 8: calculating boundary areas and Fourier coefficients of the segmented tongue images;
step 9: calculating corresponding areas of the boundaries of the unit disc and the tongue picture;
step 10: after determining the boundary value, calculating the mapping of the internal region by using a Kexil integral formula;
step 11: constructing a conformal map, which updates the real part and the imaginary part of the function through iteration;
step 12: outputting the aligned image;
in the above steps, the specific steps for obtaining the mapping are as follows: the tongue picture area is set to be omega 1, the unit disc is set to be D, and the standard tongue picture area is set to be omega 2; firstly, the mapping from D to omega 1 and the mapping from D to omega 2 are obtained, and then the mapping from omega 1 to omega 2 is obtained;
the first step: firstly, calculating a corresponding tongue picture boundary, and calculating a boundary region by using a Fourier descriptor through a formula (1):
and a second step of: on the basis of obtaining the boundary correspondence, further calculating the mapping of the internal region, wherein the cauchy integral formula is shown in formula (2), and when Γ is a unit circle, the cauchy integral formula becomes formula (3):
wherein z and z 0 Points on the boundary and interior, respectively, f (z 0 ) And F (z) 0 ) Mapping representing boundary and interior regions, respectively:
wherein z is k ＝e 2jπ(k/K) Represents the form of a dot on a unit circle, phi k Representing the corresponding tongue boundary;
and a third step of: constructing a conformal mapping, and setting functions of the conformal mapping as follows:
F＝U+j·V (5)
fourth step: calculating the mapping of the original tongue image to the unit circle:
let the original tongue picture be represented as I 0 [u,v,w]The image of the original image mapped onto the unit circle is expressed as J [ x, y, w]Expressed by the following formula:
J[x,y,w]＝I 0 [U(x,y),V(x,y),w] (6)
wherein (u, v) ε Ω 1 (original tongue area), (x, y) ∈d (unit circle), w=1, 2,3 represents RGB three channels;
fifth step: omega determination 1 To omega 2 Mapping of I 0 [u,v,w]The original tongue picture is represented by the image,representing the standard tongue picture after alignment; for arbitrary->Its corresponding pixel position on the unit circle is denoted +.>The mapping from unit circle to standard tongue image is thus expressed as:
the mapping of the original image to the unit disk is calculated by the formula (6), and the formula (6) is substituted into the formula (7) to obtain the following formula:
thus far, the original image I can be obtained by the formula (8) 0 I to standard tongue picture 1 Is mapped to the mapping of (a).
2. The method for tongue segmentation and alignment based on feature pyramid network as claimed in claim 1, wherein step 1 expands the set by rotation and horizontal flip when the data enhancement processing is performed on the picture.
3. The method for segmenting and aligning tongue images based on a feature pyramid network as claimed in claim 1, wherein step 2 adopts lableme software when manually marking the tongue bodies in all picture data.
4. The method for tongue segmentation and alignment based on feature pyramid network according to claim 1, wherein in step 5, all suggested frames are fixed to 7x7 features after maximum pooling operation.
5. The tongue segmentation and alignment method based on feature pyramid network of claim 1, wherein the feature pyramid network is constructed from the outputs of the middle 4 bnecks of mobiletv 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210106423.6A CN114627136B (en) | 2022-01-28 | 2022-01-28 | Tongue image segmentation and alignment method based on feature pyramid network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210106423.6A CN114627136B (en) | 2022-01-28 | 2022-01-28 | Tongue image segmentation and alignment method based on feature pyramid network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114627136A CN114627136A (en) | 2022-06-14 |
| CN114627136B true CN114627136B (en) | 2024-02-27 |
Family
ID=81899071
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210106423.6A Active CN114627136B (en) | 2022-01-28 | 2022-01-28 | Tongue image segmentation and alignment method based on feature pyramid network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114627136B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115908464B (en) * | 2023-01-09 | 2023-05-09 | 智慧眼科技股份有限公司 | Tongue image segmentation method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107316307A (en) * | 2017-06-27 | 2017-11-03 | 北京工业大学 | A kind of Chinese medicine tongue image automatic segmentation method based on depth convolutional neural networks |
| CN107977671A (en) * | 2017-10-27 | 2018-05-01 | 浙江工业大学 | A kind of tongue picture sorting technique based on multitask convolutional neural networks |
| CN108109160A (en) * | 2017-11-16 | 2018-06-01 | 浙江工业大学 | It is a kind of that interactive GrabCut tongue bodies dividing method is exempted from based on deep learning |
| WO2020029915A1 (en) * | 2018-08-06 | 2020-02-13 | 深圳市前海安测信息技术有限公司 | Artificial intelligence-based device and method for tongue image splitting in traditional chinese medicine, and storage medium |
-
2022
- 2022-01-28 CN CN202210106423.6A patent/CN114627136B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107316307A (en) * | 2017-06-27 | 2017-11-03 | 北京工业大学 | A kind of Chinese medicine tongue image automatic segmentation method based on depth convolutional neural networks |
| CN107977671A (en) * | 2017-10-27 | 2018-05-01 | 浙江工业大学 | A kind of tongue picture sorting technique based on multitask convolutional neural networks |
| CN108109160A (en) * | 2017-11-16 | 2018-06-01 | 浙江工业大学 | It is a kind of that interactive GrabCut tongue bodies dividing method is exempted from based on deep learning |
| WO2020029915A1 (en) * | 2018-08-06 | 2020-02-13 | 深圳市前海安测信息技术有限公司 | Artificial intelligence-based device and method for tongue image splitting in traditional chinese medicine, and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 面向舌体分割的两阶段卷积神经网络设计;王丽冉;汤一平;陈朋;何霞;袁公萍;;中国图象图形学报;20181016(10);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114627136A (en) | 2022-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021036616A1 (en) | Medical image processing method, medical image recognition method and device | |
| Chan et al. | Texture-map-based branch-collaborative network for oral cancer detection | |
| WO2023137914A1 (en) | Image processing method and apparatus, electronic device, and storage medium | |
| Choi et al. | Boosting proximal dental caries detection via combination of variational methods and convolutional neural network | |
| CN108053417B (en) | lung segmentation device of 3D U-Net network based on mixed rough segmentation characteristics | |
| CN105957063B (en) | CT image liver segmentation method and system based on multiple dimensioned weighting similarity measure | |
| CN110288597B (en) | Attention mechanism-based wireless capsule endoscope video saliency detection method | |
| US20210118144A1 (en) | Image processing method, electronic device, and storage medium | |
| WO2020133636A1 (en) | Method and system for intelligent envelope detection and warning in prostate surgery | |
| CN107492071A (en) | Medical image processing method and equipment | |
| CN108205806B (en) | Automatic analysis method for three-dimensional craniofacial structure of cone beam CT image | |
| CN110930416A (en) | MRI image prostate segmentation method based on U-shaped network | |
| CN106023151B (en) | Tongue object detection method under a kind of open environment | |
| CN108765392B (en) | Digestive tract endoscope lesion detection and identification method based on sliding window | |
| CN112750531A (en) | Automatic inspection system, method, equipment and medium for traditional Chinese medicine | |
| Liu et al. | Extracting lungs from CT images via deep convolutional neural network based segmentation and two-pass contour refinement | |
| Yu et al. | FS-GAN: Fuzzy Self-guided structure retention generative adversarial network for medical image enhancement | |
| CN114627136B (en) | Tongue image segmentation and alignment method based on feature pyramid network | |
| Malathi et al. | Active contour based segmentation and classification for pleura diseases based on Otsu’s thresholding and support vector machine (SVM) | |
| CN113643281B (en) | Tongue image segmentation method | |
| CN108765431B (en) | Image segmentation method and application thereof in medical field | |
| CN115393239A (en) | Multi-mode fundus image registration and fusion method and system | |
| CN112766332A (en) | Medical image detection model training method, medical image detection method and device | |
| Tang et al. | DE-Net: dilated encoder network for automated tongue segmentation | |
| Nasim et al. | Review on multimodality of different medical image fusion techniques |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |