CN108830788A - A kind of plain splice synthetic method of histotomy micro-image - Google Patents

Abstract

The invention discloses a plane splicing synthesis method of microscopic images of tissue sections, which comprises the following steps: (1) acquiring a single microscopic image to be spliced; (2) extracting features of the microscopic images; (3) matching the microscopic images (4) Stitching and synthesis of microscopic images. The invention has the advantages of solving the deficiencies in the existing method for synthesizing microscopic images of slices and achieving efficient and accurate plane splicing and synthesizing of microscopic images of animal tissue slices.

Description

A Plane Stitching Synthesis Method for Microscopic Images of Tissue Sections

technical field

The invention relates to the field of image processing, in particular to a method for plane splicing and synthesis of microscopic images of tissue sections.

Background technique

In the process of observation and 3D reconstruction of animal tissues, it is necessary to obtain complete microscopic images of tissue slices. However, in order to ensure that the images are clear and reach a certain pixel degree during the shooting process, it is impossible to directly obtain a complete tissue slice image by taking a single photo. It can take pictures of some areas of tissue slices under the microscope field of view. Therefore, in order to obtain a clear and complete microscopic image of a paraffin section, multiple high-resolution images need to be taken. In order to obtain a complete microscopic image, all photos need to be spliced and synthesized to generate a complete high-resolution image. Microscopic images of tissue sections. At present, most of the splicing and synthesis of microscopic images of slices are realized manually by using software such as Photoshop, but the artificial synthesis of microscopic images through software is time-consuming and labor-intensive, and the requirements for image synthesis personnel are very high. Therefore, the existing microscopic slices Image synthesis methods have certain limitations.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method for plane splicing and synthesis of microscopic images of tissue slices to solve the deficiencies in the existing synthesis methods of microscopic images of slices and achieve efficient and accurate completion of microscopic images of animal tissue slices. Planar stitching of images.

In order to achieve the above object, the technical solution adopted in the present invention is: a method for plane splicing and synthesis of microscopic images of tissue sections, comprising the following steps:

(1) Obtain a single microscope image to be stitched;

(2) Extract the features of the microscopic image;

(3) matching the features of the microscopic image;

(4) Stitching and synthesis of microscopic images.

(5) Perform post-processing on the synthesized image.

In step (1), tissue sections are prepared through steps such as tissue fixation, dehydration, transparency, paraffin embedding, sectioning, staining, and sealing, and then the prepared tissue sections are observed and photographed to obtain single microscopic images under different fields of view.

The acquisition of microscopic images includes the following steps:

(a) Making paraffin sections of animal tissues: select biological tissue blocks whose size and thickness are not more than 5 mm, and then use paraffin to embed the animal tissue blocks; use a Leica microtome to slice the embedded paraffin blocks, and keep each slice The thickness is 7 microns; then the slices are unfolded in the slide tank, stained with a staining agent, dehydrated, transparent, and sealed, and finally made into paraffin sections that are convenient for storage and long-distance transportation;

(b) Take microscopic images of animal tissue paraffin sections under different visual fields: use a complete set of optical microscope equipment to take microphotographs to obtain plane photos; use a computer to observe microscopic images; Take microscopic images in sequence from left to right and from top to bottom, and ensure that there are overlapping parts of two adjacent images until the sum of single microscopic images under different fields of view is enough to cover the area of animal tissue samples in paraffin sections; After the single microscopic image of the field of view is taken, the captured microscopic images are screened, and unqualified images need to be re-shot in time or replaced with a new paraffin section to obtain a single microscopic image again.

The extraction of microscopic image features includes the following steps:

(a) Read n microscopic images of different fields of view under the same magnification;

(b) extracting the SURF feature vectors of the input n microscopic images under different fields of view;

(c) Build the Hessian matrix: After filtering, the Hessian matrix can be simplified as: Calculate the H matrix discriminant and judge whether it is greater than 0 or less than 0 to determine whether the point is an extreme point or not;

(d) Use the Hessian matrix to construct the Gaussian pyramid scale space, and convolve the box filters of different scales with the original picture so that the original microscopic image remains unchanged and only the size of the filter is changed;

(e) First use the Hessian matrix to determine the candidate points, then preliminarily determine the feature points through non-maximum value suppression, and then filter out the key points with relatively weak energy and wrongly positioned key points to further define the feature points precisely;

(f) Finally, calculate the haar wavelet response in the x and y directions of each sector in the circular neighborhood of the feature point, find the sector direction with the largest modulus and use it as the main direction of the feature point, that is, complete the 64-dimensional SURF feature vector extract.

Step (3) Microscopic image feature matching steps are as follows:

(a) Select two adjacent images A and B from multiple images, use step (2) to generate key point descriptors of A and B, and use the standardized Euclidean distance of the key point SURF feature vector as the key of the two images The basis of point descriptor similarity detection;

(b) Find the closest 2 matching points in B for each key point in A through brute force search;

(c) For the two matching points in (b), if the standard Euclidean distance of the closest point divided by the standard Euclidean distance of the next closest point is less than a certain ratio threshold, then this pair of matching points is accepted. In order to ensure the stability of feature matching, here we specifically define the ratio threshold as 0.5.

Step (4) The microscopic image mosaic synthesis steps are as follows:

(a) Use the ICP algorithm to find a consistent match, so that the key points of the two images correspond one-to-one;

(b) Use VGG16 training to obtain the feature matrix of all microscopic images, and then verify the effectiveness of image matching; VGG16 is a 16-layer CNN, including 13 convolutional layers and 3 full-link layers ;

(c) After verifying that the matching is valid, find out the connected components in the matching image;

(d) For each connected component, find out a suitable rotation angle and a suitable matching connection point;

(e) Filtering is used for rendering optimization, and smooth connection is used to complete the splicing and synthesis of microscopic images;

(f) Obtain a complete microscopic image of the slice.

Step (5) The post-processing steps of the composite image are as follows:

(a) Firstly, perform color gradation processing on the complete microscopic image of the slice after splicing and synthesis, remove the influence caused by background light and white balance, enhance the contrast of the image, and make the layers more distinct;

(b) Then use edge softening to make the background and edges more natural;

(c) The final cropped image with uniform resolution and centered microscopic image.

The advantages of the present invention are as follows: 1. The present invention can carry out planar splicing and synthesis of microscopic images of animal tissue slices, and can output complete microscopic images of slices with uniform resolution, which is convenient for experimenters to observe the image structure, and can meet the requirements of subsequent three-dimensional reconstruction .

2. The present invention can overcome the disadvantages of large number of image stitching, uneven brightness, and few repeated parts on the basis of high image resolution when plane splicing and synthesis of animal tissue slice microscopic images, and achieve rapid and correct completion of microscopic images. The purpose of image plane stitching synthesis.

3. The present invention breaks through the time-consuming and labor-intensive process of splicing and synthesizing plane images using artificial software, and at the same time reduces the technical requirements of image compositing personnel, and realizes the efficient and precise splicing and synthesizing of microscopic images of animal tissue slices.

Description of drawings

The content expressed in each accompanying drawing of the description of the present invention and the marks in the figure are briefly described below:

Fig. 1 is the flowchart of splicing method of the present invention;

Fig. 2 is a flow chart of micro image feature extraction in the present invention.

Detailed ways

The specific implementation manner of the present invention will be described in further detail below by describing the best embodiment with reference to the accompanying drawings.

A method for plane splicing and synthesis of microscopic images of animal tissue sections, comprising the following steps:

(1) Acquisition of a single microscopic image. Tissue sections are made through the steps of tissue fixation, dehydration, transparency, paraffin embedding, sectioning, staining, and sealing, and then the prepared tissue sections are observed and photographed to obtain single microscopic images under different fields of view. The Olympus BX51 optical microscope complete set of equipment was used to take microphotographs to obtain planar photos. Microscopic images were observed with the help of Image-Pro Plus 6.0 software on the computer, and several suitable and clear single-section microscopic images were selected for subsequent stitching and synthesis of complete section microscopic images.

(2) Microscopic image feature extraction. For the screened high-quality microscopic images, first construct the Hessian matrix and calculate the Hessian matrix discriminant, then use the Hessian matrix to construct the Gaussian pyramid scale space, then use the Hessian matrix to determine the candidate points and accurately define the feature points, and finally select the feature points The main direction constructs a 64-dimensional SURF (Speeded Up Robust Features) feature point description operator.

(3) Microscopic image feature matching. Take the feature matching of two images A and B as an example to illustrate. After the feature extraction of the above microscopic images is completed, the key point descriptors of A and B will be generated respectively (that is, the SURF feature vector of K1*64 dimension and K2*64 Dimensional SURF feature vector), using the standardized Euclidean distance of the key point SURF feature vector as the basis for the similarity detection of the key points of the two images, and find the two closest matching points for each key point through brute force search.

(4) Microscopic image splicing and synthesis. Use ICP (Iterative Closest Point, Iterative Closest Point) to find consistent matching, so that the key points of the two images correspond one-to-one, and use CNN (Convolutional Neural Network, Convolutional Neural Network) model to verify the validity of image matching , find out the appropriate rotation angle and the appropriate matching connection point, use filtering to optimize the rendering, smooth connection to complete the splicing and synthesis of microscopic images, and obtain microscopic images with complete slices.

(5) Synthetic image post-processing. Perform color gradation processing on the complete microscopic image after splicing and synthesis, remove the influence caused by background light and white balance, use edge softening to make the background and edges more natural, and finally crop it into a uniform resolution and place the microscopic image in the center image.

The steps of acquiring a single microscopic image in the above step (1) are as follows:

(a) First make paraffin sections of animal tissues. A biological tissue block whose size and thickness is not more than 5 mm is selected, and then the animal tissue block is embedded in paraffin. The embedded paraffin blocks were sectioned with a Leica microtome, and the thickness of each section was kept at 7 μm. Then unfold the slices in the slide tank, stain the slices with a staining agent (HE staining method or Nissl staining method), and then perform dehydration, transparency, sealing, etc., and finally make the slices into paraffin wax for storage and long-distance transportation slice;

(b) Microscopic images of paraffin sections of animal tissues were then taken at different fields of view. The Olympus BX51 optical microscope complete set of equipment was used to take microphotographs to obtain planar photos. Microscopic images were observed with the help of Image-Pro Plus 6.0 software on the computer. When taking microscopic images under different fields of view, it is necessary to ensure that any one image overlaps with at least one other image. Therefore, microscopic images are taken in order from left to right and top to bottom, and two adjacent images are guaranteed to overlap until the sum of single microscopic images under different fields of view is enough to cover the area of animal tissue samples in paraffin sections . After the shooting of single microscopic images of different fields of view is completed, the captured microscopic images are manually screened. For unqualified images, it is necessary to re-shoot in time or replace a new paraffin section to obtain a single microscopic image again.

The above step (2) microscopic image feature extraction steps are as follows:

(a) Read n microscopic images of different fields of view under the same magnification;

(b) Extract the SURF feature vectors of the microscopic images of the input n different fields of view, as shown in Figure 2;

(c) Build the Hessian matrix: After filtering, the Hessian matrix can be simplified as: Calculate the H matrix discriminant and judge whether it is greater than 0 or less than 0 to determine whether the point is an extreme point or not; where F(x, y) represents a certain point on the image. A certain point X=(x, y) in the H(x, σ) image, the Hessian matrix on the σ scale of point X, σ represents the nearest neighbor. Lxx(X,σ) represents the convolution of the Gaussian second-order partial derivative at X with the image I. Lxy(X,σ), Lyy(X,σ) have similar meanings

(d) Use the Hessian matrix to construct the Gaussian pyramid scale space, and convolve the box filters of different scales with the original picture so that the original microscopic image remains unchanged and only the size of the filter is changed;

(e) First use the Hessian matrix to determine the candidate points, then preliminarily determine the feature points through non-maximum value suppression, and then filter out the key points with relatively weak energy and wrongly positioned key points to further define the feature points precisely;

(f) Finally, calculate the haar wavelet response in the x and y directions of each sector in the circular neighborhood of the feature point, find the sector direction with the largest modulus and use it as the main direction of the feature point, that is, complete the 64-dimensional SURF feature vector extract.

The above step (3) microscopic image feature matching steps are as follows:

(a) Select two adjacent images A and B from multiple images, and use step (2) to generate key point descriptors of A and B (that is, K1*64-dimensional SURF feature vector and K2*64-dimensional SURF feature vector SURF feature vector), using the standardized Euclidean distance of the key point SURF feature vector as the basis for similarity detection of key point descriptors of two images;

(b) Find the closest 2 matching points in B for each key point in A through brute force search;

(c) For the two matching points in (b), if the standard Euclidean distance of the closest point divided by the standard Euclidean distance of the next closest point is less than a certain ratio threshold, then this pair of matching points is accepted. In order to ensure the stability of feature matching, here we specifically define the ratio threshold as 0.5.

The above step (4) microscopic image mosaic synthesis steps are as follows:

(a) Use ICP (Iterative Closest Point, Iterative Closest Point) to find a consistent match, so that the key points of the two images correspond one-to-one;

(b) Using VGG16 (a 16-layer CNN, including 13 convolutional layers and 3 fully-connected layers) to train to obtain the feature matrix of all microscopic images, and then verify the validity of image matching on this basis;

(c) find the connected components in the matching image;

(d) For each connected component, find out a suitable rotation angle and a suitable matching connection point;

(e) Filtering is used for rendering optimization, and smooth connection is used to complete the splicing and synthesis of microscopic images;

(f) Obtain a complete microscopic image of the slice.

Above-mentioned steps (5) composite image post-processing steps are as follows:

(a) Firstly, perform color gradation processing on the complete microscopic image of the slice after splicing and synthesis, remove the influence caused by background light and white balance, enhance the contrast of the image, and make the layers more distinct;

(b) Then use edge softening to make the background and edges more natural;

(c) Finally, the image is cropped to a uniform resolution and the microscopic image is placed in the center, and the resolution of the complete microscopic image of the final output slice is fixed at 4000*3000.

Apparently, the specific implementation of the present invention is not limited by the above methods, as long as various insubstantial improvements are made by adopting the method concept and technical solutions of the present invention, they all fall within the protection scope of the present invention.

Claims (8)

1. a kind of plain splice synthetic method of histotomy micro-image, which is characterized in that include the following steps：

(1) individual MIcrosope image to be spliced is obtained；

(2) feature of micro-image is extracted；

(3) feature of micro-image is matched；

(4) splicing of micro-image is synthesized.

2. a kind of plain splice synthetic method of histotomy micro-image as described in claim 1, which is characterized in that also wrap Include step

(5) post-processing is carried out to the image after synthesis.

3. a kind of plain splice synthetic method of histotomy micro-image as described in claim 1, which is characterized in that step (1) in, by tissue fixation, dehydration, transparent, paraffin embedding, slice, dyeing, mounting and etc. make histotomy, it is subsequent right The histotomy made is observed, shoots individual micro-image obtained under the different visuals field.

4. a kind of plain splice synthetic method of histotomy micro-image as claimed in claim 3, which is characterized in that micro- The acquisition of image includes the following steps：

(a) animal tissue's paraffin section is made：It selects and chooses biological tissue's block that size thickness is no more than 5 millimeters, then use Paraffin embeds animal tissue's block；Embedded paraffin mass is sliced using Lycra slicer, keeps every thickness of slice It is 7 microns；Then slice is unfolded in exhibition film trap, carries out slice dyeing with coloring agent, then be dehydrated, be transparent, mounting mistake Journey finally slice is made convenient for saving the paraffin section with long distance transportation；

(b) micro-image of animal tissue's paraffin section under the different visuals field is shot：It is carried out by optical microscopy complete set of equipments Photomicrograph obtains planar picture；Micro-image is observed by computer；When shooting the micro-image under the different visuals field, according to from Sequential shoot micro-image left-to-right, from top to bottom, and guarantee two adjacent images there are intersection, until different views The summation of individual micro-image under wild is enough to cover the region of animal tissue's sample in paraffin section；Complete the different visuals field individual After the shooting of micro-image, the micro-image of shooting is screened, timely retake or more is needed for underproof image The paraffin section renewed re-starts the acquisition of individual micro-image.

5. a kind of plain splice synthetic method of histotomy micro-image as described in claim 1, which is characterized in that micro- The extraction of characteristics of image includes the following steps：

(a) micro-image in the different visuals field under n identical amplification factors is read；

(b) the SURF feature vector of the micro-image under the n different visuals field of input is extracted；

(c) Hessian matrix is constructed：Hessian matrix can simplify after filtering For：H-matrix discriminate is calculated, and judges to be greater than 0 or less than 0, to differentiate this Point yes or no extreme point；

(d) gaussian pyramid scale space is constructed using Hessian matrix, and to the box filters of different scale and original Picture convolution makes original micro-image remain unchanged and only change the size of filter；

(e) candidate point is determined first with Hessian matrix, characteristic point is then primarily determined by non-maxima suppression, using Filter out the further explication characteristic point of key point of the weak key point of energy comparison and location of mistake；

(f) the haar small echo response that the direction x, y in each fan-shaped range is finally calculated in the circle shaped neighborhood region of characteristic point, finds mould Maximum sector direction and as characteristic point principal direction, that is, complete the extraction of the SURF feature vector of 64 dimensions.

6. a kind of plain splice synthetic method of histotomy micro-image as claimed in claim 5, which is characterized in that step (3) micro-image character matching step is as follows：

(a) it for two adjacent two image A, B are chosen in multiple image, is retouched using the key point that step (2) generate A and B Son is stated, the standardization Euclidean distance of key point SURF feature vector is used to describe sub- similitude detection as two images key point Foundation；

It (b) is that each key point finds immediate 2 match points in B in A by force search；

(c) for two match points in (b), if the standard European distance of closest approach is divided by the standard Euclidean distance of secondary near point Less than some proportion threshold value, then receive this pair of of match point.In order to guarantee the stability of characteristic matching, we are particularly herein Defining proportion threshold value is 0.5.

7. a kind of plain splice synthetic method of histotomy micro-image as claimed in claim 6, which is characterized in that

Step (4) Microscopic Image Mosaicing synthesis step is as follows：

(a) consistency matching is found out using ICP algorithm, so that the key point of two kinds of images corresponds；

(b) eigenmatrix of all micro-images is obtained using VGG16 training, carries out images match validity on this basis Verification；

(c) after verifying matching effectively, the connected component in matching image is found out；

(d) it is directed to each connected component, finds out suitable rotation angle and suitable matching tie point；

(e) rendering optimization is carried out using filtering, is smoothly connected the splicing synthesis for completing micro-image；

(f) it obtains being sliced complete micro-image.

8. a kind of plain splice synthetic method of histotomy micro-image as claimed in claim 2, which is characterized in that

Step (5) composograph post-processing steps are as follows：

(a) color range processing is carried out to the complete micro-image of slice after splicing synthesis first, removal bias light and white balance etc. are made At influence, enhance picture contrast, keep its level clearly more demarcated；

(b) then edge softening is used to make background and edge more natural；

(c) finally it is cut into unified resolution and image that micro-image is placed centrally.