CN117934510A - Colon polyp image segmentation method based on shape perception and feature enhancement - Google Patents

Abstract

The invention discloses a colon polyp image segmentation method based on shape perception and feature enhancement, which comprises the following steps: obtaining and dividing colon polyp images to obtain a training set and a testing set, and carrying out image enhancement preprocessing operation on the training set and the testing set; constructing an initial segmentation model based on PVTv a2 trunk extraction network, a shape perception module, a feature enhancement module, a parallel partial decoder and a multi-modal attention module; training the initial segmentation model through the training set after the pretreatment operation, and verifying through the test set after the pretreatment operation to obtain a segmentation model with weight parameters adjusted; and performing image segmentation on the image to be detected through the segmentation model to obtain a colon polyp image. The invention can solve the problems of under-segmentation and over-segmentation of polyps caused by complex size, shape and color of polyps and unclear boundaries of polyps and mucous membranes, realizes automatic segmentation of polyp images, and has good application prospect.

Description

Colon polyp image segmentation method based on shape perception and feature enhancement

Technical Field

The present invention relates to the technical field of medical image processing based on deep learning, and in particular to a colon polyp image segmentation method based on shape perception and feature enhancement.

Background technique

Polyp segmentation in colonoscopic images is a complex challenge due to various factors: (1) colon polyps vary greatly in shape, size, color, and quantity; and (2) the appearance and color of polyps and the background are very similar, so the boundary between polyps and the background is usually difficult to distinguish.

To overcome the challenge of polyp segmentation, many scholars have tried a lot of efforts and have developed various techniques for the polyp segmentation task, which can be mainly divided into two categories: (1) manual feature based methods and (2) deep learning based methods.

Early traditional methods used handcrafted features to segment polyps. These methods mainly relied on features such as color, shape, texture, and edge, and trained classifiers to distinguish polyps from surrounding normal mucosa. However, these methods can hardly capture the global contextual information of polyps and are not robust to complex scenes, so they are inefficient and inaccurate. The main reason is that polyps vary greatly in shape, size, and color, and the ability of manual feature extraction is quite limited. Therefore, it is still a long way from clinical application.

Compared with traditional methods, the colon polyp segmentation method based on deep learning can save more time and improve segmentation accuracy. Therefore, more and more people are committed to using convolutional neural networks (CNNs) for polyp segmentation tasks. Among them, the network based on encoder and decoder has shown excellent performance in the semantic segmentation of medical images, such as UNet, UNet++, ResUNet, ResUNet++, etc. The encoder is used to extract feature representation, and the decoder is used to generate segmentation masks. The jump connection can pass the complex and rich feature information in the encoder to the decoder sub-network, which not only promotes the propagation of high-level semantic features, but also enables the network to retain low-level detail information. In order to further improve the performance of the polyp segmentation network, a specific network specifically designed for polyp segmentation is also designed, such as PraNet, ACSNet, MSNet, etc. These networks introduce ideas such as reverse attention and multi-scale subtraction into the polyp segmentation problem, and the effect is very good. Although convolutional neural networks (CNNs) have made great progress in polyp segmentation, there is still a big gap from clinical application. These models have good performance for the prediction of most simple polyps, but the prediction resolution of complex polyps is not ideal, resulting in rough segmentation results and inaccurate boundaries. In addition, due to the locality of convolution operations, the network can only obtain sufficient receptive field by stacking convolution layers. The global modeling capability is limited and CNN cannot model long-distance dependencies.

Summary of the invention

In order to solve the problems existing in the above-mentioned prior art, the present invention provides the following technical solutions:

A colon polyp image segmentation method based on shape perception and feature enhancement comprises the following steps:

Acquire and segment colon polyp images to obtain a training set and a test set, and perform image enhancement preprocessing operations on the training set and the test set;

Construct an initial segmentation model based on the PVTv2 backbone extraction network, shape perception module, feature enhancement module, parallel partial decoder and multimodal attention module;

The initial segmentation model is trained by the training set after the preprocessing operation, and verified by the test set after the preprocessing operation to obtain the segmentation model with adjusted weight parameters;

The image to be detected is segmented by using the segmentation model to obtain a colon polyp image.

Preferably, the method for acquiring colon polyp images comprises: extracting endoscopic images of colon polyps from Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, CVC-T and ETIS.

Preferably, the image enhancement preprocessing method includes: adjusting the resolution of the images in the colon polyp image set to 352×352, and performing data enhancement operations, wherein the data enhancement operations include horizontal flipping, vertical flipping, 90-degree rotation, and random cropping with a size of 224×224.

Preferably, the method for training the initial segmentation model by the training set after the preprocessing operation includes: extracting multi-scale features of the colon polyp image in the training set by the PVTv2 backbone extraction network to generate four levels of features, wherein the four levels of features are _E1 , _E2 , _E3 , and _E4 , respectively, inputting the features _E1 , _E2 , and _E3 into the shape perception module to generate a segmentation map _S1 , inputting the features _E2 , _E3 , and _E4 into the feature enhancement module to obtain enhanced features _F2 , _F3 , and _F4 , inputting _F2 , _F3 , and _F4 into the parallel partial decoder to generate an initial segmentation map _S5 ;

_F4 and the initial segmentation map _S5 are input into the multimodal attention module to generate feature map _M4 , feature map _M4 is added to the initial segmentation map _S5 to generate segmentation map _S4 , _F3 and _S4 are input into the multimodal attention module to generate feature map _M3 , feature map _M3 is added to segmentation map _S4 to generate segmentation map _S3 , _F2 and segmentation map _S3 are input into the multimodal attention module to generate feature map _M2 , segmentation map _S1 is added to feature map _M2 and segmentation map _S3 to generate segmentation map _S2 , segmentation map _S2 is upsampled to the same size as the input polyp image and a Sigmoid operation is performed to obtain the final segmentation map.

Preferably, the method for generating the segmentation map _S1 includes: inputting features _E1 , _E2 , and _E3 into a 3×3 convolution layer respectively to obtain convolution features and performing feature subtraction operations between adjacent features in sequence to obtain feature differences of different scales at adjacent levels in the encoder, and then highlighting important channel features through channel attention based on the feature differences to obtain _{E1_2} and _{E2_3} , performing a 3×3 convolution layer, feature subtraction, and channel attention operation on _{E1_2} and _{E2_3} to obtain _{E1_3} , adding _{E1_2} and _{E1_3} to obtain _{E1_2_3} , and finally performing channel splicing and convolution layer operations on the result of 3×3 convolution of the feature map _{E2_3} and _{E1_2_3} to obtain complementary enhanced features, and obtaining the segmentation map _S1 through spatial attention based on the complementary enhanced features.

Preferably, the feature enhancement module includes a first branch, a second branch, a third branch, a fourth branch and a residual branch;

The first branch includes a 1×1 convolutional layer and a cross-semantic attention module connected in sequence;

The second branch includes a 1×1 convolutional layer, a 1×3 convolutional layer, a 3×1 convolutional layer, a dilated convolution with a dilation rate of 3, and a cross-semantic attention module connected in sequence;

The third branch includes a 1×1 convolutional layer, a 1×5 convolutional layer, a 5×1 convolutional layer, a dilated convolution with a dilation rate of 5, and a cross-semantic attention module connected in sequence;

The fourth branch includes a 1×1 convolutional layer, a 1×7 convolutional layer, a 7×1 convolutional layer, a dilated convolution with a dilation rate of 7, and a cross-semantic attention module connected in sequence;

The residual strip branch includes sequentially connected 1×1 convolutional layers and non-local operations;

The first branch, the second branch, the third branch and the fourth branch are spliced, and a feature enhancement module is obtained by performing channel splicing through a 3×3 convolutional layer and a residual branch.

Preferably, the process of generating a feature map by a multimodal attention module comprises:

Calculate the foreground attention, background attention, and boundary attention of the previous level segmentation map _Si+1 , i=2, 3, 4 respectively. The calculation formula is as follows:

Among them, pred∈R ^H×W×1 represents the previous level segmentation map, S _foreground represents foreground attention, S _background represents background attention, and S _boundary represents boundary attention;

The foreground attention, background attention and boundary attention are multiplied with the feature _Fi respectively to obtain three result branches, and the result branches are subjected to 1×1 and 3×3 convolution operations respectively, and then added to the segmentation map S _i+1 . The three result branches are channel-connected and subjected to squeeze excitation and 3×3 convolution operations, and finally added to the feature _Fi to generate the feature map _Mi , where i=2,3,4.

Preferably, deep supervision is performed during the training process through a loss function, and the expression of the loss function is:

in, represents the weighted IoU loss,/> represents the weighted BCE loss;

The total loss expression is:

Where G represents the true label, Represents the upsampled prediction map.

The present invention has the following technical effects:

1. The shape perception module SAM proposed in the present invention utilizes the differences between the backbone features at each level to distinguish polyps from the background, and removes background noise to extract clearer shape, boundary, texture and other detail information.

2. The feature enhancement module FEM proposed in the present invention is used to improve the segmentation of polyps of different sizes and shapes and to select and refine features.

3. The multimodal attention module MMAM proposed in the invention not only focuses on background information, but also expands its focus to the three areas of background, foreground and boundary, so as to cross-consider the role of these three areas in local area segmentation and achieve weighted balance, thereby improving the network's discrimination ability.

4. The present invention adopts a deep supervision method to calculate the loss of the segmentation maps generated at each stage, thereby accelerating the convergence of the model and improving the training efficiency of the network.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

FIG1 is a schematic flow chart of a method for segmenting colon polyps based on shape perception and feature enhancement proposed by the present invention;

FIG2 is a schematic diagram of the framework structure of the colon polyp segmentation method based on shape perception and feature enhancement proposed by the present invention;

FIG3 is a schematic diagram of a shape perception module SAM of the colon polyp segmentation method based on shape perception and feature enhancement proposed by the present invention;

FIG4 is a schematic diagram of a feature enhancement module FEM of the colon polyp segmentation method based on shape perception and feature enhancement proposed by the present invention;

FIG5 is a schematic diagram of a cross-semantic attention module CSA of the colon polyp segmentation method based on shape perception and feature enhancement proposed in the present invention;

FIG6 is a schematic diagram of a parallel partial decoder PD of the colon polyp segmentation method based on shape perception and feature enhancement proposed by the present invention;

FIG7 is a schematic diagram of the multimodal attention module MMAM of the colon polyp segmentation method based on shape perception and feature enhancement proposed in the present invention.

Detailed ways

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

Embodiment 1

FIG1 is a flow chart of a method for segmenting colon polyps based on shape perception and feature enhancement according to an embodiment of the present invention. The method for segmenting colon polyps based on shape perception and feature enhancement comprises the following steps:

Step S1, obtaining a public colon polyp dataset, dividing the dataset into a training set and a test set, and performing data enhancement operations on the colon polyp data;

Step S2, the model consists of a PVTv2 backbone extraction network, a shape perception module SAM, a feature enhancement module FEM, a parallel partial decoder PD, and a multimodal attention module MMAM;

Step S3, input the training set after the enhancement operation into the colon polyp segmentation method based on shape perception and feature enhancement for multi-scale training to improve the reliability of the training, set the training data set size scaling ratio to {0.75, 1.0, 1.25}, use the loss function for deep supervision, calculate the loss between the segmentation map generated in each stage and the true label, adjust the parameters, and save the weight parameter file of the epoch with the best performance when the model converges;

Step S4: import the saved weight parameter file into the model, then input the test image into the model, calculate the model performance and save the segmentation result image.

The following are detailed descriptions:

The specific process of step S1 is as follows:

Among them, the collected public colon polyp datasets include Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, ETIS and CVC-T. For the division of the dataset, 900 images in Kvasir-SEG and 550 images in CVC-ClinicDB, a total of 1450 images, are used as training sets, and the remaining 100 images in Kvasir-SEG and 62 images in CVC-ClinicDB as well as images in ETIS, CVC-ColonDB, and CVC-T are used as test datasets. The resolution of the colon polyp dataset is uniformly adjusted to 352×352, and the data enhancement operations include horizontal flipping, vertical flipping, 90-degree rotation, and random cropping of size 224×224.

The following table gives detailed information of each colon polyp dataset in this embodiment:

data set Resolution quantity Training samples Test samples Kvasir-SEG 332×487～1920×1072 1000 900 100 CVC-ClinicDB 384×288 612 550 62 CVC-ColonDB 574×500 380 n/a 380 ETIS 1225×966 196 n/a 196 CVC-T 574×500 60 n/a 60

The specific process of step S2 is as follows:

As shown in Figure 2, a schematic diagram of the framework structure of a colon polyp segmentation method based on shape perception and feature enhancement according to an embodiment of the present invention is shown. The PVTv2 backbone extraction network is used to extract multi-scale features of polyps. The shape perception module SAM utilizes the differences in features at each level of the backbone to remove noise in the colonoscopy image, extract clearer shape features and generate a segmentation map. The feature enhancement module FEM is used to enhance and enrich the encoder feature semantic information, capture polyps of different shapes and sizes, select and refine features, and then input them into the parallel partial decoder PD to generate an initial segmentation map. The designed multimodal attention module MMAM is used to use the global feature map formed by the parallel partial decoder PD or the high-level segmentation map generated by the previous decoder as a guide map to achieve attention to the background, foreground, and boundaries, so that the network pays more attention to suspicious and complex polyp areas.

Specific steps are as follows:

The PVTv2 backbone extraction network extracts multi-scale features of colon polyps and generates four levels of features, namely _E1 , _E2 , _E3 , and _E4 . Low-level features contain more detailed information. Therefore, in order to capture more detailed shape, texture, boundary and other information of polyps, the backbone features _E1 , _E2 , and _E3 at each level are input into the shape perception module SAM to generate a segmentation map _S1 . _E2 , _E3 , and _E4 are input into the feature enhancement module FEM to generate enhanced features _F2 , _F3 , and _F4 respectively. _F2 , _F3 , and _F4 are input into the parallel partial decoder PD to aggregate high-level features and capture global context information to generate a coarsely located global image segmentation map _S5 . Next, _F4 and the initial segmentation map _S5 are input into the multimodal attention module MMAM to generate feature _M4 . _M4 is then added to the initial segmentation map _S5 to generate segmentation map _S4 . Then, _F3 and S4 are added in the same way. ₄ is input into the multimodal attention module MMAM to generate a feature map M ₃ , which is then added to the segmentation map S ₄ of the previous level to generate a segmentation map S _3. Finally, the segmentation map S ₁ generated by the shape perception module SAM is added to the feature map M ₂ and the segmentation map S ₃ to generate a segmentation map S ₂ , where the feature map M ₂ is generated by inputting F ₂ and S ₃ into the multimodal attention module MMAM. The segmentation map S ₂ is upsampled to the same size as the input polyp image and a Sigmoid operation is performed to obtain the final probability map.

Furthermore, the PVTv2 backbone extraction network is PVTv2 pre-trained on ImageNet, where in order to adapt PVTv2 to the polyp segmentation task, the last classification layer is removed to obtain backbone features _E1 , _E2 , _E3 , and _E4 at each level. _E1, as the lowest-level feature, contains detailed detail features, and _E4, as the highest-level feature, contains rich semantic information.

Further, as shown in FIG3 , a schematic diagram of a shape perception module SAM of a colon polyp segmentation method based on shape perception and feature enhancement described in an embodiment of the present invention is shown. The input of the shape perception module SAM is the main features E ₁ , E ₂ , and E ₃ , which are respectively input into the 3×3 convolution layer. The obtained features are sequentially subjected to feature subtraction operations between adjacent features to obtain feature differences of different scales at adjacent levels in the encoder. Then, the difference feature maps are respectively subjected to channel attention CA to highlight important channel features to obtain E _{1_2} and E _{2_3} . Next, E _{1_2} and E _{2_3} are subjected to 3×3 convolution, feature subtraction, and channel attention CA operations to obtain E _{1_3} . The obtained E _{1_2} and E _{1_3} are added to obtain E _{1_2_3} . Finally, the result of 3×3 convolution of the feature map E _{2_3} and E _{1_2_3} are subjected to channel splicing and convolution operations to generate complementary enhanced features, and spatial attention SA is applied to obtain a feature segmentation map S ₁ with detailed location information of the polyp.

Further, as shown in FIG4 , a schematic diagram of a feature enhancement module FEM of a method for segmenting colon polyps based on shape perception and feature enhancement according to an embodiment of the present invention is shown, wherein the feature enhancement module FEM comprises: a first branch, a second branch, a third branch, a fourth branch, and a residual branch;

The first branch of the feature enhancement module FEM includes: sequentially connecting a 1×1 convolutional layer and a cross-semantic attention module CSA;

The second branch of the feature enhancement module FEM includes: a 1×1 convolution layer, a 1×3 convolution layer, a 3×1 convolution layer, a dilated convolution with a dilation rate of 3, and a cross-semantic attention module CSA connected in sequence;

The third branch of the feature enhancement module FEM includes: sequentially connecting a 1×1 convolution layer, a 1×5 convolution layer, a 5×1 convolution layer, a dilated convolution with a dilation rate of 5, and a cross-semantic attention module CSA;

The fourth branch of the feature enhancement module FEM includes: sequentially connecting a 1×1 convolution layer, a 1×7 convolution layer, a 7×1 convolution layer, a dilated convolution with a dilation rate of 7, and a cross-semantic attention module CSA;

The feature enhancement module FEM residual strip branch includes: sequentially connecting a 1×1 convolutional layer and a non-local operation Non-Local;

The above four branches are concatenated, then passed through a 3×3 convolutional layer, and then channel-concatenated with the residual branch to obtain the output features.

Furthermore, as shown in FIG5 , a schematic diagram of a cross-semantic attention module CSA of a colon polyp segmentation method based on shape perception and feature enhancement described in an embodiment of the present invention is shown. The input features of the cross-semantic attention module CSA are divided into E _ic and E _is along the channel direction. E _ic is used to extract one-dimensional channel global information M _ic , wherein E _ic is globally averaged pooled, and then the obtained 2D tensor is converted to 1D, followed by a one-dimensional convolution operation, and finally the obtained vector is converted back to 2D space and a Sigmoid operation is performed; E _is used to extract two-dimensional spatial information M _is , and a 3×3 convolution operation and a Sigmoid operation are performed on E _is , and the two parts M _ic and _Mis are combined in an interactive manner. The specific operation of the interaction is two branches, the first branch performs a matrix multiplication operation on _Mis and E _ic ; the second branch performs a 3×3 convolution operation on E _is , and then performs a matrix multiplication operation with M _ic ; finally, the two branches are spliced on the channel.

Further, as shown in FIG6 , a schematic diagram of a parallel partial decoder PD of a method for segmenting colon polyps based on shape perception and feature enhancement according to an embodiment of the present invention is shown. The input of the schematic diagram of the parallel partial decoder PD is the output results from the feature enhancement module FEM, which are F ₂ , F ₃ , and F ₄ , respectively. One branch upsamples F ₄ by 2 times to make its size the same as F ₃ , multiplies the upsampled result of F ₄ by F _{3 , and} then performs channel splicing with the upsampled result of F ₄ to obtain f _{3_4} ; another branch upsamples F ₃ by 2 times and the upsampled result of F ₄ by 4 times by F ₂ , and then performs channel splicing with f _{3_4} that is upsampled and subjected to a 3×3 convolution operation to obtain f _{2_3_4} , and then performs two 3×3 convolutions and one 1×1 convolution on f _{2_3_4} , and the obtained feature map is upsampled to the size of the input image to obtain the initial segmentation map S ₅

Further, as shown in FIG7 , a schematic diagram of a modal attention module MMAM of a colon polyp segmentation method based on shape perception and feature enhancement according to an embodiment of the present invention is shown. The multimodal attention module MMAM calculates the foreground attention, background attention, and boundary attention of the previous level segmentation map S _i+1 , i=3, 4, 5 respectively. The calculation formula is as follows:

Where pred∈R ^H×W×1 represents the previous level segmentation map, S _foreground represents foreground attention, S _background represents background attention, and S _boundary represents boundary attention;

These three attentions are then multiplied with the feature _Fi respectively, and then the resulting branches are subjected to 1×1 and 3×3 convolution operations respectively, and then added to the segmentation map Si ₊₁ . The three branches are channel-connected and subjected to squeeze-excitation SE and 3×3 convolution operations, and finally added to the feature _Fi to generate the feature map _Mi , where i=2,3,4.

The specific process of step S3 is as follows:

The preprocessed training data set is input into the colon polyp segmentation method based on shape perception and feature enhancement for training. The initial learning rate in the hyperparameters is 0.0001, and the weight decay is also adjusted to 0.0001. The AdamW optimization algorithm is used, the batch size is 8, and the loss function is used for deep supervision. The loss between the segmentation map generated in each stage and the true label is calculated. The loss function is defined as:

in represents the weighted IoU loss,/> represents the weighted BCE loss;

The total loss is:

Where G represents the true label, represents the upsampled segmentation map;

Finally, save the weight parameter file of the epoch with the best performance until the model converges.

The specific process of step S4 is as follows:

Input the test image into the trained model, test the performance of the model with the saved weight parameters, and output and save the segmentation map.

In order to verify the effectiveness of the present invention compared with the prior art, we conducted comparative experiments with UNet, UNet++, SFA, PraNet, ACSNet, EU-Net, SANet, and MSNet technologies in the test data set. The experimental results are shown in the following table:

From the comparison results, it can be seen that our invention has good learning and generalization capabilities compared with the above 8 existing classic methods, and has the ability to accurately segment polyp images.

The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have several changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims (8)

1. A method for segmenting a colon polyp image based on shape perception and feature enhancement, comprising the steps of:

obtaining and dividing colon polyp images to obtain a training set and a testing set, and carrying out image enhancement preprocessing operation on the training set and the testing set;

Constructing an initial segmentation model based on PVTv a2 trunk extraction network, a shape perception module, a feature enhancement module, a parallel partial decoder and a multi-modal attention module;

Training the initial segmentation model through the training set after the pretreatment operation, and verifying through the test set after the pretreatment operation to obtain a segmentation model with weight parameters adjusted;

And performing image segmentation on the image to be detected through the segmentation model to obtain a colon polyp image.

2. The method for segmentation of a colon polyp image based on shape perception and feature enhancement as set forth in claim 1,

The method for acquiring the colon polyp image comprises the following steps: endoscopic images of colon polyps were extracted from Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, CVC-T, and ETIS.

3. The method for segmentation of a colon polyp image based on shape perception and feature enhancement as set forth in claim 1,

The image enhancement preprocessing method comprises the following steps: the resolution of the image in the set of colon polyp images is adjusted to 352 x 352 and a data enhancement operation is performed including horizontal flip, vertical flip, 90 degree rotation and random cropping of 224 x 224 size.

4. The method for segmentation of a colon polyp image based on shape perception and feature enhancement as set forth in claim 1,

The method for training the initial segmentation model through the training set after the preprocessing operation comprises the following steps: PVTv 2a trunk extraction network extracts multi-scale features of a colon polyp image in a training set to generate 4 layers of features, wherein the 4 layers of features are E ₁、E₂、E₃、E₄ respectively, the features E ₁、E₂、E₃ are input into a shape sensing module to generate a segmentation map S ₁, the features E ₂、E₃、E₄ are input into a feature enhancement module respectively to obtain enhancement features F ₂、F₃、F₄, the features F ₂、F₃、F₄ are input into a parallel partial decoder to generate an initial segmentation map S ₅;

The method comprises the steps of inputting the F ₄ and the initial segmentation map S ₅ into a multi-mode attention module to generate a feature map M ₄, adding the feature map M ₄ and the initial segmentation map S ₅ to generate a segmentation map S ₄, inputting the F ₃ and the S ₄ into the multi-mode attention module to generate a feature map M ₃, adding the feature map M ₃ and the segmentation map S ₄ to generate a segmentation map S ₃, inputting the F ₂ and the segmentation map S ₃ into the multi-mode attention module to generate a feature map M ₂, adding the segmentation map S ₁ and the feature map M ₂ and the segmentation map S ₃ to generate a segmentation map S ₂, upsampling the segmentation map S ₂ to the same size as the input polyp image and performing a Sigmoid operation to obtain a final segmentation map.

5. The method for segmentation of a colon polyp image based on shape perception and feature enhancement as set forth in claim 4,

The method for generating the segmentation map S ₁ includes: the method comprises the steps of inputting characteristics E ₁、E₂、E₃ into a 3X 3 convolution layer respectively to obtain convolution characteristics, sequentially carrying out characteristic subtraction operation between adjacent characteristics to obtain characteristic differences of different scales of adjacent levels in an encoder, carrying out important channel characteristic highlighting through channel attention based on the characteristic differences to obtain E _{1_2},E_{2_3}, carrying out 3X 3 convolution layers, characteristic subtraction and channel attention operation on E _{1_2} and E _{2_3} to obtain E _{1_3}, adding E _{1_2} and E _{1_3} to obtain E _{1_2_3}, carrying out channel splicing and convolution layer operation on a result of 3X 3 convolution on a characteristic diagram E _{2_3} to obtain complementary enhancement characteristics, and carrying out space attention based on the complementary enhancement characteristics to obtain a segmentation diagram S ₁.

6. The method for segmentation of a colon polyp image based on shape perception and feature enhancement as set forth in claim 4,

The characteristic enhancement module comprises a first branch, a second branch, a third branch, a fourth branch and a residual branch;

the first branch comprises a1 multiplied by 1 convolution layer and a cross-semantic attention module which are connected in sequence;

The second branch comprises a 1 multiplied by 1 convolution layer, a 1 multiplied by 3 convolution layer, a 3 multiplied by 1 convolution layer, an expansion convolution and cross-semantic attention module with expansion rate of 3 which are connected in sequence;

The third branch comprises a 1 multiplied by 1 convolution layer, a 1 multiplied by 5 convolution layer, a 5 multiplied by 1 convolution layer, an expansion convolution and cross-semantic attention module with expansion ratio of 5 which are connected in sequence;

the fourth branch comprises a 1 multiplied by 1 convolution layer, a 1 multiplied by 7 convolution layer, a 7 multiplied by 1 convolution layer and an expansion convolution and cross-semantic attention module with expansion rate of 7 which are connected in sequence;

The residual stripe branch comprises a1 multiplied by 1 convolution layer and non-local operation which are connected in sequence;

And splicing the first branch, the second branch, the third branch and the fourth branch, and performing channel splicing through a 3X 3 convolution layer and a residual branch to obtain the characteristic enhancement module.

7. The method for segmentation of a colon polyp image based on shape perception and feature enhancement as set forth in claim 4,

The process of generating a feature map by the multimodal attention module includes:

The foreground attention, background attention, and boundary attention of the previous stage segmentation map S _i+1, i=2, 3, and 4 are calculated respectively, and the calculation formula is as follows:

S_foreground＝pred,S_background＝1-pred,

Wherein pred ε R ^H×W×1 represents the previous segmentation map, S _foreground represents foreground attention, S _background represents background attention, and S _boundary represents boundary attention;

The foreground attention, the background attention and the boundary attention are multiplied by a feature F _i respectively to obtain three result branches, the result branches are subjected to 1×1 convolution operation and 3×3 convolution operation respectively, then added with a segmentation map S _i+1, the three result branches are subjected to channel connection, extrusion excitation and 3×3 convolution operation, and finally added with a feature F _i to generate a feature map M _i, wherein i=2, 3 and 4.

8. The method for segmentation of a colon polyp image based on shape perception and feature enhancement as set forth in claim 1,

And carrying out deep supervision through a loss function in the training process, wherein the expression of the loss function is as follows:

wherein, Representing weighted IoU loss,/>Representing weighted BCE loss;

The total loss expression is:

wherein G represents the actual label and wherein, Representing the upsampled prediction map.