JP6337629B2 - Diagnosis support information generation method, image processing apparatus, diagnosis support information generation system, and image processing program - Google Patents

Description

  The present invention relates to a diagnosis support information generation method, an image processing apparatus, a diagnosis support information generation system, and an image processing program.

  In recent years, with the spread of molecular targeted drug treatment centering on antibody drugs, in order to design molecular targeted drugs more effectively, quantification of biological substances on cells to be observed has been demanded. As a method for confirming the presence of a biological material, a tissue analysis method based on the binding of a fluorescent material to which a biological material recognition site is bound and a biological material corresponding to the biological material recognition site is known.

For example, Patent Document 1 describes a method in which a specific antigen of a tissue section is stained using nanoparticles containing a plurality of fluorescent substances, and information on the specific antigen in the section is generated based on the fluorescence signal of the nanoparticles. ing.
According to the method described in Patent Document 1, by using high-luminance fluorescent substance-containing nanoparticles as a fluorescent labeling material, a fluorescent signal is observed in a dot shape without being buried in the autofluorescence of the tissue or the like. Thereby, the number of biological substances can be easily measured based on the fluorescence signal of the nanoparticles.

JP 2012-208106 A

  However, as a matter of fact, in the acquired image, a large number of overlapping regions in which measurement target regions such as a plurality of cells or cell nuclei are superimposed are observed. In the method described in Patent Document 1, when a bright spot is present in the overlapping region, it is impossible to determine which of the superimposed cells the bright spot belongs. Could not be generated and could lead to misdiagnosis.

  A main object of the present invention is to determine to which cell a bright spot existing in a superimposed region belongs, and to generate accurate diagnosis support information.

In order to solve the above problem, according to the invention described in claim 1,
Diagnosis of generating diagnostic support information by quantifying the biological material from a tissue section or cultured cell specimen stained using a fluorescent material combined with a biological material recognition site that recognizes a specific biological material as a staining reagent In the support information generation method,
A fluorescence image input step of inputting a fluorescence image representing the expression of the biological material in the specimen as a fluorescence bright spot;
A biological material position calculating step for calculating the position of the biological material from the fluorescent image based on the fluorescence of the bright spot region centered on the fluorescent bright spot;
A cell region extraction step for extracting a specific part of a cell as a cell region from a bright field image obtained by photographing the same visual field range as the fluorescent image or the fluorescent image;
A superimposed region extraction step for extracting a superimposed region in which the cell regions of a plurality of cells are superimposed;
A biological material determination step of determining whether or not the biological material exists in the superimposed region from the superimposed region and the biological material position;
When the biological material determination step determines that the biological material is present in the overlap region, the overlap cell information acquisition acquires, as overlap cell information, information related to a plurality of cells in which the cell region is overlapped in the overlap region. Steps,
From the biological material position and the superimposed cell information, an affiliation determination step of determining which of the plurality of cell regions superimposed in the superimposed region the biological material existing in the superimposed region belongs,
A diagnostic support information generation method characterized by comprising:

According to the invention described in claim 2, in the diagnosis support information generation method according to claim 1,
The biological material position calculating step includes:
A diagnostic support information generation method is provided, which calculates the height of a biological substance existing in the bright spot region.

According to the invention described in claim 3, in the diagnosis support information generation method according to claim 1 or 2,
The biological material position calculating step includes:
Create a brightness profile showing the brightness distribution of the bright spot area,
Based on the luminance profile and a reference profile having a fluorescence luminance distribution derived from the fluorescent material and position information in the height direction of the fluorescent material prepared in advance, the number of biological materials present in the bright spot region and A diagnostic support information generation method characterized by calculating a height is provided.

According to the invention described in claim 4, in the diagnosis support information generating method according to claim 2 or 3,
The superimposed cell information includes relative position information in the height direction of the plurality of cell regions superimposed in the superimposed region,
In the affiliation determination step, a diagnosis support information generation method is provided in which the determination is performed based on the height of the biological material and relative position information in the height direction of the cell region.

According to the invention described in claim 5, in the diagnosis support information generation method according to any one of claims 1-4,
The superimposed cell information includes position information of contour lines of the plurality of cell regions that are superimposed in the superimposed region,
In the affiliation determination step, a diagnosis support information generation method is provided in which it is determined that the biological material belongs to the cell region having the contour line closest to the bright spot region.

According to the invention described in claim 6, in the diagnosis support information generating method according to any one of claims 1 to 5,
The superimposed cell information includes physiological information of a plurality of cells on which the cell region is superimposed in the superimposed region,
The affiliation determining step identifies the cell region of a cell to which the biological material is likely to belong based on the physiological information, and determines that the biological material belongs to the specified cell region. A diagnostic support information generation method is provided.

According to the invention described in claim 7, in the diagnosis support information generating method according to claim 6,
The physiological information includes the area ratio of cytoplasm and nucleus, cell circularity, cell area, number of bright spots in the cell region, the staining reagent in a plurality of cells in which the cell region is superimposed in the overlap region. There is provided a method for generating diagnosis support information, characterized in that any one or a plurality of non-uniformity of staining due to.

According to the invention described in claim 8,
An image for generating diagnostic support information by quantifying the biological material from a tissue section or cultured cell specimen stained using a fluorescent material combined with a biological material recognition site that recognizes a specific biological material as a staining reagent In the processing device,
A fluorescence image input means for inputting a fluorescence image representing the expression of the biological material in the specimen as a fluorescence bright spot;
A biological material position calculating means for calculating the biological material position from the fluorescent image based on the fluorescence of the bright spot region centered on the fluorescent bright spot;
A cell region extracting means for extracting a specific part of a cell as a cell region from a bright field image obtained by photographing the same visual field range as the fluorescent image or the fluorescent image;
A superimposed region extracting means for extracting a superimposed region where the cell regions of a plurality of cells are superimposed;
Biological material determination means for determining whether or not the biological material exists in the superimposed region from the superimposed region and the biological material position;
Superimposed cell information that acquires, as superimposed cell information, information related to a plurality of cells in which the cell region is superimposed in the superimposed region when the biological material determining unit determines that the biological material is present in the superimposed region. Acquisition means;
From the biological material position and the superimposed cell information, belonging determination means for determining which biological cells existing in the overlapping region belong to which of the plurality of cell regions superimposed in the overlapping region;
An image processing apparatus is provided.

According to the invention of claim 9,
An image processing apparatus according to claim 8 ,
Expression of the biological material in a tissue section or cultured cell specimen stained with a fluorescent material used as a staining reagent that is combined with a fluorescent material that recognizes a specific biological material used in the image processing apparatus. An image acquisition device for acquiring a fluorescent image represented by a fluorescent luminescent spot ;
A diagnostic support information generation system is provided.

According to the invention of claim 10,
A computer that generates diagnostic support information by quantifying the biological material from a tissue section or cultured cell specimen stained using a fluorescent material combined with a biological material recognition site that recognizes a specific biological material as a staining reagent The
Fluorescence image input means for inputting a fluorescence image representing the expression of the biological material in the specimen as a fluorescent luminescent spot,
A biological material position calculating means for calculating the biological material position from the fluorescent image based on the fluorescence of the bright spot region centered on the fluorescent bright spot;
A cell region extraction means for extracting a specific portion of a cell as a cell region from a bright field image obtained by photographing the same visual field range as the fluorescent image or the fluorescent image;
A superimposed region extracting means for extracting a superimposed region in which the cell regions of a plurality of cells are superimposed;
A biological material determination means for determining whether or not the biological material is present in the superimposed region from the superimposed region and the biological material position;
Superimposition cell information acquisition that acquires, as superimposed cell information, information related to a plurality of cells in which the cell region is superimposed in the overlap region when the biological material determination unit determines that the biological material exists in the overlap region means,
From the biological material position and the superimposed cell information, affiliation determination means for determining which biological cell existing in the superimposed region belongs to which of a plurality of cell regions superimposed in the superimposed region;
An image processing program for functioning as

  According to the present invention, it is possible to determine to which cell the bright spot existing in the superimposition region belongs, so that accurate diagnosis support information can be generated and appropriate treatment can be performed.

It is a figure which shows the system configuration | structure of a diagnostic assistance information generation system. It is a block diagram which shows the functional structure of the image processing apparatus of FIG. It is a figure which shows an example of a bright field image. It is a figure which shows an example of a fluorescence image. It is a flowchart which shows the image analysis process performed by the control part of FIG. It is a flowchart which shows the detail of the process of step S2 of FIG. It is a figure which shows the image from which (a) bright field image and (b) cell area | region were extracted. It is a figure which shows the image from which (a) fluorescence image and (b) luminescent spot area | region were extracted. It is a flowchart which shows the detail of the process in step S8 of 1st embodiment. It is a figure for demonstrating the discrimination | determination method of the relative height of a cell. It is a figure for demonstrating schematically the calculation method of a luminance integration value. It is a schematic diagram which shows the example of the brightness | luminance profile of one fluorescent particle. It is a schematic diagram which shows the example of the brightness | luminance profile of two fluorescent particles which exist close to the same height. It is a schematic diagram which shows the example of the brightness | luminance profile of two fluorescent particles which exist in proximity to the same XY coordinate position and different height. It is a schematic diagram which shows the example of the brightness | luminance profile of two fluorescent particles which have XY coordinate positions and heights which are shifted and close to each other. It is a flowchart which shows the detail of the process in step S8 of 2nd embodiment. It is a schematic diagram explaining the method to discriminate | determine the affiliation cell of a fluorescent particle from the distance of the outline of a cell, and a bright spot area | region. It is a flowchart which shows the detail of the process in step S8 of 3rd embodiment.

  Hereinafter, although the form for implementing this invention with reference to figures is demonstrated, this invention is not limited to these.

<Configuration of Diagnosis Support Information Generation System 100>
FIG. 1 shows an example of the overall configuration of a diagnostic support information generation system 100 using the fluorescent substance quantification method of the present invention. The diagnosis support information generation system 100 acquires a microscopic image of a human tissue sample stained with a predetermined staining reagent, and analyzes the acquired microscopic image, thereby expressing a specific biological substance in the tissue sample to be observed. This is a system that outputs a feature quantity that quantitatively represents.

  As shown in FIG. 1, the diagnostic support information generation system 100 is configured by connecting a microscope image acquisition device 1A and an image processing device 2A so that data can be transmitted and received via an interface such as a cable 3A. The connection method between the microscope image acquisition device 1A and the image processing device 2A is not particularly limited. For example, the microscope image acquisition apparatus 1A and the image processing apparatus 2A may be connected via a LAN (Local Area Network) or may be connected wirelessly.

The microscope image acquisition apparatus 1A is a known optical microscope with a camera, and acquires a microscope image of a tissue specimen on a slide placed on a slide fixing stage, and transmits the microscope image to the image processing apparatus 2A.
The microscope image acquisition apparatus 1A includes an irradiation unit, an imaging unit, an imaging unit, a communication I / F, and the like. The irradiation means includes a light source, a filter, and the like, and irradiates the tissue specimen on the slide placed on the slide fixing stage with light. The imaging means may be either an upright type or an inverted type constituted by an eyepiece, an objective lens, etc., and forms an image of transmitted light, reflected light, or fluorescence emitted from the tissue specimen on the slide by the irradiated light. . The imaging means is a microscope-installed camera that includes a CCD (Charge Coupled Device) sensor and the like, captures an image formed on the imaging surface by the imaging means, and generates digital image data of the microscope image. The communication I / F transmits image data of the generated microscope image to the image processing apparatus 2A.
In the present embodiment, the microscope image acquisition apparatus 1A includes a bright field unit that combines an irradiation unit and an imaging unit suitable for bright field observation, and a fluorescence unit that combines an irradiation unit and an imaging unit suitable for fluorescence observation. It is possible to switch between bright field / fluorescence by switching units. Any light source such as a mercury lamp, a xenon lamp, an LED, or a laser beam can be used as a light source for fluorescence observation.

  Note that the microscope image acquisition apparatus 1A is not limited to a microscope with a camera. For example, a virtual microscope slide creation apparatus (for example, a special microscope microscope) that acquires a microscope image of an entire tissue specimen by scanning a slide on a microscope slide fixing stage. Table 2002-514319 gazette) etc. may be used. According to the virtual microscope slide creation device, it is possible to acquire image data that allows the entire tissue specimen image on the slide to be viewed on the display unit at a time.

The image processing apparatus 2A calculates the expression distribution of a specific biological substance in the tissue specimen to be observed by analyzing the microscope image transmitted from the microscope image acquisition apparatus 1A.
FIG. 2 shows a functional configuration example of the image processing apparatus 2A. As shown in FIG. 2, the image processing apparatus 2 </ b> A includes a control unit 21, an operation unit 22, a display unit 23, a communication I / F 24, a storage unit 25, and the like, and each unit is connected via a bus 26. Yes.

  The control unit 21 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like. The control unit 21 executes various processes in cooperation with various programs stored in the storage unit 25, and performs image processing 2A. Overall control of the operation. For example, the control unit 21 executes image analysis processing (see FIG. 5) in cooperation with a program stored in the storage unit 25, and performs image input means, biological material position calculation means, cell region and superimposed region extraction means. The function as the biological material discrimination means, the superposition cell information acquisition means, and the affiliation discrimination means in the superposition region is realized.

  The operation unit 22 includes a keyboard having character input keys, numeric input keys, various function keys, and the like, and a pointing device such as a mouse, and a key pressing signal pressed by the keyboard and an operation signal by the mouse. Are output to the control unit 21 as an input signal.

  The display unit 23 includes a monitor such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display), for example, and displays various screens according to instructions of a display signal input from the control unit 21. In the present embodiment, the display unit 23 functions as an output unit for outputting an image analysis result.

  The communication I / F 24 is an interface for transmitting and receiving data to and from external devices such as the microscope image acquisition apparatus 1A. The communication I / F 24 functions as a bright field image and fluorescent image input unit.

The storage unit 25 includes, for example, an HDD (Hard Disk Drive), a semiconductor nonvolatile memory, or the like. As described above, the storage unit 25 stores various programs, various data, and the like.
In addition, the image processing apparatus 2A may include a LAN adapter, a router, and the like and be connected to an external device via a communication network such as a LAN.

The image processing apparatus 2A in the present embodiment preferably performs analysis using the bright field image and the fluorescence image transmitted from the microscope image acquisition apparatus 1A.
The bright field image is a microscope obtained by enlarging and photographing a tissue specimen stained with an H (hematoxylin) staining reagent and an HE (hematoxylin-eosin) staining reagent in a bright field in the microscope image acquisition apparatus 1A. It is an image and is a cell form image showing the form of the cell in the tissue specimen. Hematoxylin is a blue-violet pigment that stains cell nuclei, bone tissue, part of cartilage tissue, serous components, etc. (basophilic tissue, etc.). Eosin is a red to pink pigment that stains cytoplasm, soft tissue connective tissue, red blood cells, fibrin, endocrine granules, and the like (eosinophilic tissue, etc.). FIG. 3 shows an example of a bright field image obtained by photographing a tissue specimen subjected to HE staining.
The fluorescent image is stained using a staining reagent including nanoparticles (referred to as fluorescent substance-containing nanoparticles) encapsulating a fluorescent substance bound with a biological substance recognition site that specifically binds and / or reacts with a specific biological substance. This is a microscope image obtained by irradiating the tissue specimen with excitation light of a predetermined wavelength in the microscope image acquisition device 1A to emit fluorescent substance-containing nanoparticles (fluorescence), and enlarging and photographing this fluorescence. . That is, the fluorescence appearing in the fluorescence image indicates the expression of a specific biological material corresponding to the biological material recognition site in the tissue specimen. FIG. 4 shows an example of the fluorescence image.

<Acquisition of fluorescence image>
Here, a method for acquiring a fluorescent image will be described in detail, including a staining reagent (fluorescent substance-containing nanoparticles) used for acquiring the fluorescent image, a staining method for a tissue specimen using the staining reagent, and the like.

[Fluorescent substance]
Examples of the fluorescent substance used in the staining reagent for obtaining a fluorescent image include fluorescent organic dyes and quantum dots (semiconductor particles). When excited by ultraviolet to near infrared light having a wavelength in the range of 200 to 700 nm, it preferably emits visible to near infrared light having a wavelength in the range of 400 to 1100 nm.

  Fluorescent organic dyes include fluorescein dye molecules, rhodamine dye molecules, Alexa Fluor (Invitrogen) dye molecules, BODIPY (Invitrogen) dye molecules, cascade dye molecules, coumarin dye molecules, and eosin dyes. Examples include molecules, NBD dye molecules, pyrene dye molecules, cyanine dye molecules, aromatic hydrocarbon molecules, and the like.

Specifically, 5-carboxy-fluorescein, 6-carboxy-fluorescein, 5,6-dicarboxy-fluorescein, 6-carboxy-2 ′, 4,4 ′, 5 ′, 7,7′-hexachlorofluorescein, 6 -Carboxy-2 ', 4,7,7'-tetrachlorofluorescein, 6-carboxy-4', 5'-dichloro-2 ', 7'-dimethoxyfluorescein, naphthofluorescein, 5-carboxy-rhodamine, 6-carboxy Rhodamine, 5,6-dicarboxy-rhodamine, rhodamine 6G, tetramethylrhodamine, X-rhodamine, sulforhodamine B, sulforhodamine 101, and Alexa Fluor
350, Alexa Fluor 405, Alexa Fluor 430, Alexa Fluor 488, Alexa Fluor 500, Alexa Fluor 514, Alexa Fluor 532, Alexa Fluor 546, Alexa Fluor 555, Alexa Fluor 568, Alexa Fluor 594, Alexa Fluor 610, Alexa Fluor 633, Alexa Fluor 635, Alexa Fluor 647, Alexa Fluor 660, Alexa Fluor 680, Alexa Fluor 700, Alexa Fluor 750, BODIPY FL, BODIPY TMR, BODIPY 493/503, BODIPY 530/550, BODIPY 558/568, BODIPY 564/57
0, BODIPY 576/589, BODIPY 581/591, BODIPY 630/650, BODIPY 650/665 (manufactured by Invitrogen), methoxycoumarin, eosin, NBD, pyrene, Cy5, Cy5.5, Cy7, HiLyte Fluor 594 (registered) Trademark, manufactured by Anaspec), DyLight 594 (registered trademark, manufactured by Thermo Scientific), dye molecule, ATTO 594 (registered trademark, manufactured by ATTO-TEC), MFP 594 (registered trademark, manufactured by Mobitec), 5 , 10,15,20-tetraphenylporphine tetrasulfonic acid, zinc 5,10,15,20-tetraphenylporphine tetrasulfonic acid, phthalocyanine tetrasulfonic acid, zinc phthalocyanine tetrasulfonic acid, N, N-Bis- (2, 6-diisopropylphenyl) -1,6,7,12- (4-tert-butylphenoxy) -perylen-3,4,9,10-tetracarbonacid diimide, N, N'-Bis (N, N'-diisopropylphenyl) -1 , 6,7,12-tetraphenoxyperylene-3,4: 9,10-tetracarboxdiimide, Benzenesulfonic acid 4,4 ', 4``, 4'''-[(1,3,8,10-tetrahydro-1,3 , 8,10-tetraoxoperylo [3,4-cd: 9,10-c 'd'] dipyran-5,6,12,13-tetrayl) tetralis (oxy)] tetrakis- and the like. You may use individually or what mixed multiple types.

  Quantum dots include II-VI group compounds, III-V group compounds, or quantum dots containing group IV elements as components ("II-VI group quantum dots", "III-V group quantum dots", " Or “Group IV quantum dots”). You may use individually or what mixed multiple types.

  Specific examples include, but are not limited to, CdSe, CdS, CdTe, ZnSe, ZnS, ZnTe, InP, InN, InAs, InGaP, GaP, GaAs, Si, and Ge.

It is also possible to use a quantum dot having the above quantum dot as a core and a shell provided thereon. Hereinafter, as a notation of a quantum dot having a shell in this specification, when the core is CdSe and the shell is ZnS, it is expressed as CdSe / ZnS. For example, CdSe / ZnS, CdS / ZnS, InP / ZnS, InGaP / ZnS, Si / SiO2, Si / ZnS, Ge / GeO2, and Ge / ZnS can be used, but not limited thereto.
As the quantum dots, those subjected to surface treatment with an organic polymer or the like may be used as necessary. Examples thereof include CdSe / ZnS having a surface carboxy group (manufactured by Invitrogen), CdSe / ZnS having a surface amino group (manufactured by Invitrogen), and the like.

[Fluorescent substance-containing nanoparticles]
In the present embodiment, the fluorescent substance-encapsulating nanoparticles are those in which the fluorescent substance is dispersed inside the nanoparticles, whether the fluorescent substance and the nanoparticles themselves are chemically bonded or not. Good.
The material constituting the nanoparticles is not particularly limited, and examples thereof include polystyrene, polylactic acid, silica, and melamine.

  The fluorescent substance-containing nanoparticles used in the present embodiment can be produced by a known method. For example, silica nanoparticles encapsulating a fluorescent organic dye can be synthesized with reference to the synthesis of FITC-encapsulated silica particles described in Langmuir 8, Vol. 2921 (1992). Various fluorescent organic dye-containing silica nanoparticles can be synthesized by using a desired fluorescent organic dye instead of FITC.

  Silica nanoparticles encapsulating quantum dots can be synthesized with reference to the synthesis of CdTe-encapsulated silica nanoparticles described in New Journal of Chemistry, Vol. 33, p. 561 (2009).

Polystyrene nanoparticles encapsulating a fluorescent organic dye can be obtained by copolymerization using an organic dye having a polymerizable functional group described in US Pat. No. 4,326,008 (1982), or US Pat.
692 (1992), and can be prepared by using a method of impregnating polystyrene nanoparticles with a fluorescent organic dye.

  The polymer nanoparticles encapsulating the quantum dots can be prepared by using the method of impregnating the quantum nanoparticles into polystyrene nanoparticles described in Nature Biotechnology, Vol. 19, page 631 (2001).

The average particle diameter of the fluorescent substance-containing nanoparticles used in this embodiment is not particularly limited, but the accessibility of the antigen and the signal of the fluorescent particles are background noise (camera noise and cell autofluorescence). The range that is not buried is usually about 30 to 800 nm.
Also, the coefficient of variation indicating the variation in particle diameter is not particularly limited, but it is usually preferable to use a coefficient of 20% or less.
The average particle size is obtained by taking an electron micrograph using a scanning electron microscope (SEM), measuring the cross-sectional area of 1000 particles, and taking each measured value as the area of the circle as the diameter of the circle. The arithmetic average was taken as the average particle size. The coefficient of variation was also a value calculated from the particle size distribution of 1000 particles.

[Bonding of biological substance recognition sites and fluorescent substance-containing nanoparticles]
The biological material recognition site according to the present embodiment is a site that specifically binds and / or reacts with the target biological material. The target biological substance is not particularly limited as long as a substance that specifically binds to the target biological substance exists, but typically, protein (peptide), nucleic acid (oligonucleotide, polynucleotide), antibody, etc. Is mentioned. Accordingly, substances that bind to the target biological substance include antibodies that recognize the protein as an antigen, other proteins that specifically bind to the protein, and nucleic acids having a base sequence that hybridizes to the nucleic acid. Is mentioned. Specifically, an anti-HER2 antibody that specifically binds to HER2, which is a protein present on the cell surface, an anti-ER antibody that specifically binds to an estrogen receptor (ER) present in the cell nucleus, and actin that forms a cytoskeleton And an anti-actin antibody that specifically binds to. Among them, those in which anti-HER2 antibody and anti-ER antibody are bound to fluorescent substance-containing nanoparticles can be used for selection of breast cancer medication, and are preferable.

  The mode of binding between the biological substance recognition site and the fluorescent substance-encapsulating nanoparticles is not particularly limited, and examples thereof include covalent bonding, ionic bonding, hydrogen bonding, coordination bonding, physical adsorption, and chemical adsorption. A bond having a strong bonding force such as a covalent bond is preferred from the viewpoint of bond stability.

  Moreover, there may be an organic molecule that connects between the biological substance recognition site and the fluorescent substance-encapsulating nanoparticles. For example, in order to suppress non-specific adsorption with a biological substance, a polyethylene glycol chain can be used, and SM (PEG) 12 manufactured by Thermo Scientific can be used.

  When the biological substance recognition site is bound to the fluorescent substance-encapsulating silica nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. For example, a silane coupling agent that is a compound widely used for bonding an inorganic substance and an organic substance can be used. This silane coupling agent is a compound having an alkoxysilyl group that gives a silanol group by hydrolysis at one end of the molecule and a functional group such as a carboxyl group, an amino group, an epoxy group, an aldehyde group at the other end, Bonding with an inorganic substance through an oxygen atom of the silanol group. Specifically, mercaptopropyltriethoxysilane, glycidoxypropyltriethoxysilane, aminopropyltriethoxysilane, silane coupling agent having a polyethylene glycol chain (for example, PEG-silane no. SIM6492.7 manufactured by Gelest), etc. Is mentioned. When using a silane coupling agent, you may use 2 or more types together.

A known procedure can be used for the reaction procedure of the fluorescent organic dye-encapsulated silica nanoparticles and the silane coupling agent. For example, the obtained fluorescent organic dye-encapsulated silica nanoparticles are dispersed in pure water, aminopropyltriethoxysilane is added, and the mixture is reacted at room temperature for 12 hours. After completion of the reaction, fluorescent organic dye-encapsulated silica nanoparticles whose surface is modified with an aminopropyl group can be obtained by centrifugation or filtration. Subsequently, by reacting an amino group with a carboxyl group in the antibody, the antibody can be bound to the fluorescent organic dye-encapsulated silica nanoparticles via an amide bond. If necessary, a condensing agent such as EDC (1-Ethyl-3- [3-Dimethylaminopropyl] carbodiimide Hydrochloride: manufactured by Pierce (registered trademark)) can also be used.

  If necessary, a linker compound having a site capable of directly binding to a fluorescent organic dye-encapsulated silica nanoparticle modified with an organic molecule and a site capable of binding to a molecular target substance can be used. As a specific example, sulfo-SMCC (Sulfosuccinimidyl 4 [N-maleimidomethyl] -cyclohexane-1-carboxylate: manufactured by Pierce) having both a site that selectively reacts with an amino group and a site that reacts selectively with a mercapto group When used, by binding the amino group of the fluorescent organic dye-encapsulated silica nanoparticles modified with aminopropyltriethoxysilane and the mercapto group in the antibody, antibody-bound fluorescent organic dye-encapsulated silica nanoparticles can be produced.

When the biological substance recognition site is bound to the fluorescent substance-encapsulated polystyrene nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. That is, by impregnating polystyrene nanoparticles having functional groups such as amino groups with fluorescent organic dyes and quantum dots, it is possible to obtain polystyrene nanoparticles encapsulating functional groups with fluorescent substances, and thereafter using EDC or sulfo-SMCC. Thus, antibody-bound fluorescent substance-encapsulated polystyrene nanoparticles can be produced.
When the biological substance recognition site is bound to the fluorescent substance-encapsulated melamine nanoparticles, the same procedure as that for the fluorescent substance-encapsulated silica nanoparticles can be applied. In order to further improve the reactivity, the number of surface amino groups may be increased by reacting melamine nanoparticles with a polyfunctional amine compound in advance.

Antibodies that recognize specific antigens include M. actin, MS actin, SM actin, ACTH, Alk-1, α1-antichymotrypsin, α1-antitrypsin, AFP, bcl-2, bcl-6, β-catenin, BCA 225, CA19-9, CA125, calcitonin, calretinin, CD1a, CD3, CD4, CD5, CD8, CD10, CD15, CD20, CD21, CD23, CD30, CD31, CD34, CD43, CD45, CD45R, CD56, CD57, CD61, CD68, CD79a, "CD99, MIC2", CD138, chromogranin, c-KIT, c-MET, collagen type IV, Cox-2, cyclin D1, keratin, cytokeratin (high molecular weight), pankeratin, pankeratin, cytokeratin 5/6, cytokeratin 7, cytokeratin 8, cytokeratin 8/18, cytokeratin 14, cytokeratin 19, cytokeratin 20, CMV, E-cadherin, EGFR, ER, EMA, EBV, factor VIII related antigen, Fassin, FSH, galectin-3, gastrin, GFAP, guru Gon, glycophorin A, granzyme B, hCG, hGH, Helicobacter pylori, HBc antigen, HBs antigen, hepatocyte specific antigen, HER2, HSV-I, HSV-II, HHV-8, IgA, IgG, IgM, IGF-1R, Inhibin, insulin, kappa light chain, Ki67, lambda light chain, LH, lysozyme, macrophage, melan A, MLH-1, MSH-2, myeloperoxidase, myogenin, myoglobin, myosin, neurofilament, NSE, p27
(Kip1), p53, p53, P63, PAX 5, PLAP, Pneumocystis carini, podoplanin (D2-40), PGR, prolactin, PSA, prostatic acid phosphatase, Renal Cell Carcinoma, S100, somatostatin, spectrin, synaptophysin, TAG-72, TdT, thyroglobulin, TSH, TTF-1, TRAcP, tryptase, villin, vimentin, WT1, and Zap-70.

[Dyeing method]
Hereinafter, a method for staining a tissue specimen will be described, but the present invention is not limited to a tissue specimen, and can also be applied to a specimen such as a cell fixed on a substrate.
Moreover, the preparation method of the tissue specimen which can apply the dyeing | staining method demonstrated below is not specifically limited, What was produced by the well-known method can be used.

1) Deparaffinization process The tissue specimen is immersed in a container containing xylene to remove paraffin. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, xylene may be exchanged during the immersion.
Next, the tissue specimen is immersed in a container containing ethanol to remove xylene. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. Further, if necessary, ethanol may be exchanged during the immersion.
Next, the tissue specimen is immersed in a container containing water to remove ethanol. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. Moreover, you may exchange water in the middle of immersion as needed.

2) Activation process The activation process of the target biological substance is performed according to a known method. The activation conditions are not particularly defined, but as the activation liquid, 0.01 M citrate buffer (pH 6.0), 1 mM EDTA solution (pH 8.0), 5% urea, 0.1 M Tris-HCl buffer, etc. Can be used. As the heating device, an autoclave, a microwave, a pressure cooker, a water bath, or the like can be used. The temperature is not particularly limited, but can be performed at room temperature. The temperature can be 50 to 130 ° C. and the time can be 5 to 30 minutes.
Next, the tissue specimen after the activation treatment is immersed in a container containing PBS (Phosphate Buffered Saline) and washed. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, PBS may be exchanged during the immersion.

3) Staining using fluorescent substance-encapsulating nanoparticles combined with biological substance recognition sites Place PBS dispersion of fluorescent substance-encapsulating nanoparticles combined with biological substance recognition sites on a tissue specimen and react with the target biological substance . By changing the biological material recognition site to be combined with the fluorescent substance-containing nanoparticles, staining corresponding to various biological materials becomes possible. When using fluorescent substance-encapsulated nanoparticles to which several types of biological substance recognition sites are bound, each fluorescent substance-encapsulated nanoparticle PBS dispersion may be mixed in advance or separately placed on a tissue specimen separately. Also good.
The temperature is not particularly limited, but can be performed at room temperature. The reaction time is preferably 30 minutes or more and 24 hours or less.
It is preferable to drop a known blocking agent such as BSA-containing PBS before staining with fluorescent substance-encapsulating nanoparticles.
Next, the stained tissue specimen is immersed in a container containing PBS to remove unreacted fluorescent substance-containing nanoparticles. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, PBS may be exchanged during the immersion. Place the cover glass on the tissue specimen and enclose it. A commercially available encapsulant may be used as necessary.
In addition, when dyeing | staining using HE dyeing reagent, HE dyeing is performed before enclosure with a cover glass.

(Fluorescence image acquisition)
A microscope image (fluorescence image) with a wide field of view is acquired from the stained tissue specimen using the microscope image acquisition device 1A. In the microscope image acquisition apparatus 1A, an excitation light source and a fluorescence detection optical filter corresponding to the absorption maximum wavelength and fluorescence wavelength of the fluorescent material used for the staining reagent are selected.

<Operation of Diagnosis Support Information Generation System 100 (Including Image Processing Method)>
Hereinafter, an operation of acquiring and analyzing the above-described fluorescence image and bright field image in the diagnosis support information generation system 100 will be described with specific embodiments. Here, staining is performed using a staining reagent containing fluorescent substance-encapsulating nanoparticles bound to a biological substance recognition site that recognizes a specific protein (here, HER2 protein in breast cancer tissue, hereinafter referred to as a specific protein). However, the present invention is not limited to this example.

(First embodiment)
First, the operator uses two types of staining reagents, an HE staining reagent and a staining reagent using fluorescent substance-encapsulated nanoparticles bound with a biological substance recognition site that recognizes a specific protein as a fluorescent labeling material. Stain.
Thereafter, in the microscope image acquisition apparatus 1A, a bright field image and a fluorescence image are acquired by the procedures (a1) to (a5). In the first embodiment, the microscope image acquisition apparatus 1A is an upright microscope that observes the specimen from above, and the acquired image has the thickness direction (the direction in which the depth of focus of the microscope moves) in the height direction. These two-dimensional coordinates are defined as an X coordinate and a Y coordinate.
(A1) An operator places a tissue specimen stained with a hematoxylin staining reagent and a staining reagent containing fluorescent substance-containing nanoparticles on a slide, and places the slide on a slide fixing stage of the microscope image acquisition apparatus 1A.
(A2) Set to the bright field unit, adjust the imaging magnification and focus, and place the observation target region on the tissue specimen in the field of view.
(A3) Shooting is performed by the imaging unit to generate bright field image data, and the image data is transmitted to the image processing apparatus 2A.
(A4) Change the unit to a fluorescent unit.
(A5) Shooting is performed by the imaging means without changing the field of view and the shooting magnification to generate image data of a fluorescent image, and the image data is transmitted to the image processing apparatus 2A.

In the image processing apparatus 2A, image analysis processing is executed based on the bright field image and the fluorescence image.
FIG. 5 shows a flowchart of image analysis processing in the image processing apparatus 2A. The image analysis processing shown in FIG. 5 is executed in cooperation with the control unit 21 and the program stored in the storage unit 25.

First, when a bright field image is input from the microscope image acquisition device 1A through the communication I / F 24 (step S1), a specific part (cell region) of a cell is extracted from the bright field image (step S2). The specific part of the cell is, for example, the cell nucleus, the whole cell, etc., and is arbitrarily determined according to the type of the specific protein. In the first embodiment, the entire cell surrounded by the cell membrane is extracted as a cell region.
FIG. 6 shows a detailed flow of the process in step S2. The process of step S2 is executed in cooperation with the control unit 21 and the program stored in the storage unit 25.

In step S2, first, a bright-field image is converted into a monochrome image (step S201). FIG. 7A shows an example of a bright field image.
Next, threshold processing is performed on the monochrome image using a predetermined threshold, and the value of each pixel is binarized (step S202).

Next, noise processing is performed (step S203). Specifically, the noise process can be performed by performing a closing process on the binary image. The closing process is a process in which the contraction process is performed the same number of times after the expansion process is performed. The expansion process is a process of replacing a target pixel with white when at least one pixel in the range of n × n pixels (n is an integer of 2 or more) from the target pixel is white. The contraction process is a process of replacing a target pixel with black when at least one pixel in the range of n × n pixels from the target pixel contains black. By the closing process, a small area such as noise can be removed. FIG. 7B shows an example of an image after noise processing. As shown in FIG. 7B, after noise processing, an image (cell image) from which cells are extracted is generated.

  Next, a labeling process is performed on the image after the noise process, and a label is assigned to each of the extracted cells (step S204). The labeling process is a process for identifying an object in an image by assigning the same label (number) to connected pixels. By the labeling process, each cell can be identified from the image after the noise process and a label can be applied.

  Next, in step S3, a superimposed region where a plurality of cells are superimposed is extracted. Although the method for extracting the overlapping region is arbitrary, for example, the method described in Japanese Patent Laid-Open No. 2000-321031 can be used. Specifically, first, a pixel that is the cell center is selected from the input image according to a predetermined criterion, and pixels that form the outline of the cell are selected from the direction of the density gradient of the peripheral pixels of all the selected cell center pixels. The presence / absence of the overlapping region is determined. When there is an overlapping area, the number of cells superimposed in the overlapping area and the outline that forms the outline of the overlapping area are extracted.

On the other hand, when a fluorescence image is input from the microscope image acquisition apparatus 1A through the communication I / F 24 (step S4), fluorescence of fluorescent substance-containing nanoparticles (hereinafter simply referred to as “fluorescent particles”) from the fluorescence image. A bright spot region derived from is extracted (step S5).
The process of step S5 is executed in cooperation with the control unit 21 and the program stored in the storage unit 25.
In step S5, first, a color component corresponding to the wavelength of the fluorescent bright spot is extracted from the fluorescent image as shown in FIG. For example, when the emission wavelength of the fluorescent particles is 550 nm, only the fluorescent bright spot having the wavelength component is extracted as an image.

Next, threshold processing is performed on the extracted image, a binary image is generated, and a bright spot region is extracted. FIG. 8B shows an example of an image from which the bright spot area is extracted.
Note that noise removal processing such as cell autofluorescence and other unnecessary signal components may be performed before the threshold processing, and a low-pass filter such as a Gaussian filter or a high-pass filter such as a second derivative is preferably used.

  After the processing of step S3 and step S5 is completed, addition processing of the cell image (FIG. 7B) and the bright spot area image (FIG. 8B) is performed (step S6), and the bright spot area on the cell is determined. The distribution is displayed on the display unit 23 of the image processing apparatus 2A.

Next, it is determined whether or not each of the extracted bright spot areas overlaps with the overlapping area of cells (step S7). If it is determined that the bright spot area does not overlap with the overlapping area, the process proceeds to step S9, and the number and position of fluorescent particles in the bright spot area are calculated by a known method.
If it is determined that the bright spot region overlaps the overlapping region, the process proceeds to step S8, and the cells to which the fluorescent particles existing in the overlapping region belong are determined by the method described below.

FIG. 9 shows a detailed flow of the process in step S8 of the first embodiment. In step S8, first, information of a plurality of cells (superimposed cell information) existing in the superimposed region is extracted. In the first embodiment, the relative positional relationship (relative height of the cells) in the height direction of the cells existing in the overlap region is extracted as the overlap cell information (step S801).

The method for determining the relative height of the cells is arbitrary, but for example, cells near the lower surface of the tissue specimen have another cell or other tissue between the cells and the objective lens. This can be determined by utilizing the blurring of the image. Specifically, for example, as shown in the schematic diagram of the cross section of the tissue specimen in FIG. 10A, when two cells 40 and 41 overlap in the overlapping region, The line due to the outline of the upper cell 40 is clearly observed as compared with the line due to the outline of the lower cell 41, and the contrast between the inside and outside of the overlapping region sandwiching the outline is high. Can be judged.
FIGS. 10B and 10C show an example in which two cells are superimposed. In the overlapping region where the cells 42 and 43 in FIG. 10B are superimposed, the outline of the cell 42 is clearer than the outline of the cell 43, and the contrast inside and outside the overlapping region across the outline is high. From this, it is determined that the cell 42 is on the top. In the overlapping region where the cells 44 and 45 in FIG. 10C are superimposed, the outline of the cell 44 is clearly observed, but the outline of the cell 45 is very thin, and the inside and outside of the overlapping region sandwiching the contour line. Therefore, it is determined that the cell 44 is on the upper side.

Next, a brightness profile is created by mapping the brightness signal value information in the bright spot area from the fluorescent image from which the bright spot area is extracted. From the brightness profile, the number of fluorescent particles in each bright spot area and each fluorescent particle The position is calculated (step S802).
The “luminance profile” is two-dimensional distribution information of luminance signal values created based on an image extracted from a fluorescence image using an image from which the luminescent spot region is extracted as a mask. This range (brightness distribution spread) is shown.

  That is, as shown in FIG. 11A, when an image in which a bright spot area is extracted from a fluorescence image is generated, an image in which a bright spot area is extracted for each bright spot area (FIG. 11B). And a fluorescent image (FIG. 11C) of a part corresponding to the bright spot region are overlaid, and the bright spot image corresponding to the bright spot region is obtained from the fluorescent image using the image from which the bright spot region is extracted as a mask. Generated (FIG. 11 (d)), a luminance distribution on the XY coordinate plane is created based on the bright spot image (FIG. 11 (e)), and this becomes a luminance profile.

  The luminance profile may be a two-dimensional representation of the luminance value on the XY coordinate plane as shown in FIG. 11 (e), and each luminance profile as shown in FIG. 11 (f). The luminance value (height) at the XY coordinate position may be expressed three-dimensionally.

  Actually, one bright spot region includes one or a plurality of fluorescent particles, and the luminance profile shows the luminance value and its distribution according to the number of fluorescent particles and the position of each fluorescent particle. . 12 to 15, the left figure shows the position of the fluorescent particles, and the right figure shows the shape of the luminance profile when the fluorescent particles are at the position shown in the left figure.

  For example, as shown in FIG. 12A, the luminance profile in the case where one fluorescent particle is on the upper side of the tissue sample in one bright spot region has a feature that the maximum luminance value is large and the distribution range is narrow. There is. Then, as shown in FIGS. 12B and 12C, the closer the fluorescent particle is to the lower surface of the tissue specimen, the more the fluorescent signal of the fluorescent particle is scattered while passing through the tissue. Smaller and wider distribution range.

In addition, a plurality of fluorescent particles that are close to the same height may be measured as one bright spot region, and the shape of the luminance profile depends on the height direction position of the fluorescent particles and the distance between the fluorescent particles. Will change. For example, as shown in FIG. 13 (a), when two fluorescent particles are adjacent to the upper side of the tissue specimen, compared with the luminance profile in the case of one fluorescent particle,
The maximum luminance value is large, and the distribution range is slightly widened. Further, as shown in FIG. 13B, when the fluorescent particles are slightly apart, the maximum luminance value is slightly smaller than that in FIG. 13A, and the distribution range is widened. When the fluorescent particles are further apart, two peaks are seen as shown in FIG. 13C, and the maximum luminance value of each peak is when there is one fluorescent particle on the upper surface side (FIG. 12). It is almost the same as (a)).

  In addition, a plurality of fluorescent particles that exist at the same XY coordinate position and at different heights may be measured as one bright spot region, and the shape of the luminance profile depends on the position of the fluorescent particles and the distance between the fluorescent particles. Will change. For example, as shown in FIG. 14A, when two fluorescent particles are present on the upper side of the tissue specimen, the maximum luminance is compared with the luminance profile of the fluorescent particles in FIG. The value increases and the spread of the distribution range does not change. In addition, as shown in FIG. 14B, when two fluorescent particles are present apart on the upper side and the lower side of the tissue specimen, the maximum value of luminance is slightly higher than that in FIG. 14A. The distribution range of high luminance is almost unchanged, but the distribution range of low luminance is widened. In addition, as shown in FIG. 14C, when two fluorescent particles are present on the lower side of the tissue specimen, the maximum value of luminance is smaller than that in FIG. It is larger than the case where there is one below the specimen (FIG. 12C).

  In addition, a plurality of fluorescent particles that are close to each other with their XY coordinate positions and heights shifted may be measured as one bright spot region, and the shape of the luminance profile is determined according to the position of the fluorescent particles and the distance between the fluorescent particles. Will change. For example, as shown in FIG. 15A, when two fluorescent particles are present on the upper side of the tissue specimen and in the center in the height direction, two peaks having different heights are seen. In addition, as shown in FIG. 15B, when two fluorescent particles are present on the upper side and the lower side of the tissue specimen, the difference in height between the two peaks compared to FIG. 15A. Increases, and the distribution range of the low-brightness region widens.

  In the first embodiment, the brightness profile data as described above is stored in the storage unit 25 in advance as a reference profile. By analyzing the luminance profile obtained from the fluorescence image of the tissue specimen based on the reference profile, the number and position (XY coordinate position and height) of the fluorescent particles included in each bright spot region are calculated in step S802. Is done.

Next, in step S803, the information on the relative height of the cells extracted in step S801 is compared with the information on the number and position of the fluorescent particles calculated in step S802, and the cells to which the fluorescent particles present in the overlapping region belong are compared. Is determined.
For example, if it is calculated in step S802 that there are two fluorescent particles on the upper side of the tissue sample in the overlapping region where two cells are superimposed, it is determined that the two fluorescent particles belong to the upper cell. .
Thus, fluorescence on the cell is determined based on the belonging cell information of the fluorescent particles existing in the overlapping region determined in step S8 and the information on the number and position of the fluorescent particles in the region where the cells are not superimposed calculated in step S9. Particle distribution is generated as diagnosis support information (step S10).

(Second embodiment)
Hereinafter, the operation of the diagnosis support information generation system 100 of the second embodiment will be described. Microscope image acquisition and steps S1 to S7 and steps S9 to S10 in FIG. 5 are performed in the same manner as in the first embodiment described above.
FIG. 16 shows a detailed flow of the process in step S8 of the second embodiment. In step S8 of the second embodiment, first, the distance between each bright spot area and the outline of the overlapping area where the bright spot area exists is calculated (step S811), and the outline closest to the bright spot area belongs. A cell is discriminated (step S812).
Next, in step S813, the number and position of fluorescent particles in each bright spot region are calculated by the same process as step S802 of the first embodiment.
It is determined that the number of fluorescent particles calculated in step S813 belongs to the cell to which the contour line closest to the bright spot region determined in step S812 belongs.
Thus, fluorescence on the cell is determined based on the belonging cell information of the fluorescent particles existing in the overlapping region determined in step S8 and the information on the number and position of the fluorescent particles in the region where the cells are not superimposed calculated in step S9. Particle distribution is generated as diagnosis support information (step S10).

  The diagnosis support information generation system 100 of the second embodiment is particularly suitable for quantifying biological substances that are specifically expressed on the cell membrane for the reasons described below.

FIG. 17A is a schematic diagram of a two-dimensional image in which the bright spot region 5 having the area S is observed at the left end of the overlap region where the substantially spherical cells 42 and 43 are overlapped. FIG. 17B is a schematic diagram of the cross section of FIG. 17A, and the region sandwiched between two dotted lines indicates the range of the bright spot region 5.
Assuming that the fluorescent particles are present on the cell membrane, in the cell 42 where the bright spot region 5 is observed in the vicinity of the outline, the region where the fluorescent particles can exist is greatly expanded in the height direction (FIG. 17B). , Black thick line), and its area T is about 4 to 5 times S. On the other hand, in the cell 43 in which the bright spot region 5 is observed near the center, the region where the fluorescent particles can be present hardly expands in the height direction (FIG. 17 (b), gray thick line), and the area T is the cell. The upper side and the lower side of 43 are combined and become about twice as large as S. Accordingly, in FIG. 17, the area 42 where the fluorescent particles can exist is larger than that of the cells 43 in the cell 42, and thus the probability that the fluorescent particles exist in the cell 42 is high.

In other words, when fluorescent particles are specifically present on the cell membrane, cells with a contour line closer to the bright spot region have a larger area where fluorescent particles can exist, so the fluorescent particles belong to them. It is thought that there is a high probability.
When a plurality of fluorescent particles are included in one bright spot region, the area T is calculated for each of the plurality of cells superimposed in the overlapping region, and belongs to each cell based on the ratio of the area T. The number of fluorescent particles may be determined.

  The second embodiment for discriminating cells to which fluorescent particles belong based on the distance between the contour line of the overlapping region and the bright spot region is easier to measure than the first embodiment for calculating the relative height of the overlapping cells. There is an advantage of being. Moreover, the number of fluorescent particles can be accurately calculated by using not only the information on the contour line but also the luminance profile.

In the second embodiment, the method for measuring the distance between the bright spot area and the outline of the overlapping area is arbitrary. For example, the coordinate position with the highest luminance in the bright spot area may be set as the center point of the bright spot area, and the distance from the center point to the contour line may be measured, or from the outer edge of the bright spot area It may be the distance to the contour line.
In addition, the process of step 813 is performed before the process of step S811, and the distance between each fluorescent particle and the contour line is measured instead of the distance between the bright spot region and the contour line, and the contour line closest to the fluorescent particle is determined. The cell to which it belongs may be determined as the cell to which the fluorescent particle belongs.

(Third embodiment)
Hereinafter, the operation of the diagnosis support information generation system 100 of the third embodiment will be described. Microscope image acquisition and steps S1 to S7 and steps S9 to S10 in FIG. 5 are performed in the same manner as in the first embodiment described above.
FIG. 18 shows a detailed flow of the process in step S8 of the third embodiment. In step S8 of the third embodiment, first, physiological information of cells superimposed in the overlapping region is extracted (step S821), and then the possibility that fluorescent particles belong based on the physiological information of cells. A cell having a high value is discriminated (step S822).

  The physiological information of the cells extracted in step S821 and the method of identifying the cells to which the fluorescent particles are likely to belong in step S822 are based on the arbitrary physiological information depending on the type of biological material or tissue specimen to be quantified. One or more may be extracted and determined based on one or more combinations of the extracted physiological information. For example, when HER2 protein in breast cancer tissue is quantified, the following examples are given.

  In general, it is known that the higher the malignancy of cancer, the larger the cell nucleus area ratio (N / C ratio) to the cytoplasm area. Therefore, the N / C ratio of the cell is calculated in step S821, and the cell having the maximum N / C ratio is determined in step S822.

  In addition, when cells having different areas on the two-dimensional image are observed, it is considered that the larger the area, the larger the height direction. Therefore, when there is a bright spot region in the overlapping region where cells of different sizes are superimposed, it is considered that the larger the cell, the larger the region where the fluorescent particle can exist, and thus the higher the possibility that the fluorescent particle exists. Therefore, the area of the cell is extracted in step S821, and the cell having the largest area is determined in step S822.

In general, it is known that cells with higher malignancy of cancer are not circular (low circularity). Therefore, the circularity of the cell is extracted in step S821, and the cell having the lowest circularity is determined in step S822.
The definition of the circularity is arbitrary. For example, when the cell area is S and the perimeter is L, the closer the value calculated by the formula of 4πS / L ² is to 1, the closer to the circularity, the circularity Can be defined as high.

  In addition, it is considered that the higher the malignancy of cancer, the greater the number of fluorescent particles. Therefore, the number of fluorescent particles in the region where the cells are not superimposed is extracted in step S821, and the cell having the largest number of fluorescent particles is determined in step S822.

  In general, it is known that cells with higher malignancy of cancer tend to have uneven staining and uneven staining. Therefore, in step S821, the degree of staining unevenness is quantified and extracted, and in step S822, cells with large staining unevenness are determined.

  After the process of step S822, the number of fluorescent particles and the position of each fluorescent particle in each bright spot region are calculated by the same process as step S802 of the first embodiment (step S823), and determined in step S822. It is determined that the number of fluorescent particles calculated in step S823 belongs to the cell.

  Thus, fluorescence on the cell is determined based on the belonging cell information of the fluorescent particles existing in the overlapping region determined in step S8 and the information on the number and position of the fluorescent particles in the region where the cells are not superimposed calculated in step S9. Particle distribution is generated as diagnosis support information (step S10).

According to the first to third embodiments described above, the overlapping region of cells is extracted by the processing of Steps S1 to S3, the bright spot region is extracted by the processing of Steps S4 to S5, and then Steps S6 to Step S3. By the process of S10, the belonging cells of the fluorescent particles existing in the overlapping region of the cells are discriminated, and the distribution of the fluorescent particles belonging to each cell is displayed.
This makes it possible to identify the cells to which the fluorescent particles in the overlapping region belong, so that even if the cells are superimposed on the acquired image, the number of specific proteins expressed for each observation target cell can be accurately quantified This improves the accuracy of diagnosis.

  In addition, the description content in the said embodiment is a suitable example of this invention, and is not limited to this.

  In the above-described embodiment, the cell region and the overlap region are extracted by performing the processing of step S2 to step S3 on the bright field image input in step S1, but the cell region is stained with a fluorescent substance and input in step S4. The processing in steps S2 to S3 in FIG. 5 may be performed on the same image as the fluorescent image to be performed.

In the above embodiment, only one type of specific protein is targeted for quantification, but two or more types of fluorescent particles having different emission wavelengths may be used for a plurality of specific proteins.
In such a case, each color component is extracted using a filter work or the like in step S5, and the processing in steps S6 to S9 may be executed for each extracted color component (wavelength component). In addition, when dyeing | staining using a some fluorescent substance, the combination from which the excitation light and fluorescence wavelength of a fluorescent substance do not interfere mutually is selected.

  Moreover, you may perform the process in step S8 combining the method as described in 1st-3rd embodiment. For example, by combining the relative position information of the cells described in the first embodiment and the information on the size of the cells described in the third embodiment, the height direction position of the superimposed cells can be calculated in detail, It becomes possible to more accurately determine the cell to which the fluorescent particle belongs.

In step S802, the luminance profile corresponding to the number and position of the fluorescent particles is created in advance as a reference profile, and the number and position of the fluorescent particles are calculated by collating with the luminance profile created from the fluorescent image. For example, only a reference profile based on the fluorescence of one fluorescent particle is created in advance, the luminance profile created from the fluorescence image is subjected to processing such as two-dimensional Fourier transform, and the waveform is decomposed. In comparison, the number and position of fluorescent particles may be calculated.
Further, not only the number and position of the fluorescent particles but also a luminance profile that takes into account the presence or absence of tissues such as cells above the fluorescent particles may be used as the reference profile.

  Moreover, in the said embodiment, although the HER2 protein in a breast cancer was mentioned as an example of a specific protein, it is not limited to this. Depending on the type of lesion (cancer) to be diagnosed, if the biological material recognition site when acquiring a fluorescence image is different, the feature quantity that quantitatively indicates the expression level of the specific protein corresponding to the lesion type It can be provided to a doctor.

In the above description, an example in which an HDD, a semiconductor nonvolatile memory, or the like is used as a computer-readable medium for the program according to the present invention is disclosed, but the present invention is not limited to this example. As other computer-readable media, a portable recording medium such as a CD-ROM can be applied. Further, a carrier wave (carrier wave) is also applied as a medium for providing program data according to the present invention via a communication line.
In addition, the detailed configuration and detailed operation of each device constituting the diagnosis support information generating system 100 can be changed as appropriate without departing from the spirit of the invention.

DESCRIPTION OF SYMBOLS 1A Microscope image acquisition apparatus 2A Image processing apparatus 21 Control part 22 Operation part 23 Display part 24 Communication I / F
25 storage unit 26 bus 3A cable 40, 41, 42, 43, 44, 45 cell 100 diagnosis support information generation system

Claims (10)

Diagnosis of generating diagnostic support information by quantifying the biological material from a tissue section or cultured cell specimen stained using a fluorescent material combined with a biological material recognition site that recognizes a specific biological material as a staining reagent In the support information generation method,
A fluorescence image input step of inputting a fluorescence image representing the expression of the biological material in the specimen as a fluorescence bright spot;
A biological material position calculating step for calculating the position of the biological material from the fluorescent image based on the fluorescence of the bright spot region centered on the fluorescent bright spot;
A cell region extraction step for extracting a specific part of a cell as a cell region from a bright field image obtained by photographing the same visual field range as the fluorescent image or the fluorescent image;
A superimposed region extraction step for extracting a superimposed region in which the cell regions of a plurality of cells are superimposed;
A biological material determination step of determining whether or not the biological material exists in the superimposed region from the superimposed region and the biological material position;
When the biological material determination step determines that the biological material is present in the overlap region, the overlap cell information acquisition acquires, as overlap cell information, information related to a plurality of cells in which the cell region is overlapped in the overlap region. Steps,
From the biological material position and the superimposed cell information, an affiliation determination step of determining which of the plurality of cell regions superimposed in the superimposed region the biological material existing in the superimposed region belongs,
A diagnostic support information generating method characterized by comprising: The diagnostic support information generation method according to claim 1,
The biological material position calculating step includes:
A method for generating diagnosis support information, comprising calculating a height of a biological substance existing in the bright spot region. The diagnostic support information generation method according to claim 1 or 2,
The biological material position calculating step includes:
Create a brightness profile showing the brightness distribution of the bright spot area,
Based on the luminance profile and a reference profile having a fluorescence luminance distribution derived from the fluorescent material and position information in the height direction of the fluorescent material prepared in advance, the number of biological materials present in the bright spot region and A diagnostic support information generation method characterized by calculating a height. The diagnostic support information generation method according to claim 2 or 3,
The superimposed cell information includes relative position information in the height direction of the plurality of cell regions superimposed in the superimposed region,
The diagnosis determination information generation method according to claim 1, wherein the affiliation determination step determines from the relative position information of the biological material height and the cell region in the height direction. In the diagnostic assistance information generation method according to any one of claims 1 to 4,
The superimposed cell information includes position information of contour lines of the plurality of cell regions that are superimposed in the superimposed region,
The affiliation determination step determines that the biological material belongs to the cell region having the contour line closest to the bright spot region. In the diagnostic support information generation method according to any one of claims 1 to 5,
The superimposed cell information includes physiological information of a plurality of cells on which the cell region is superimposed in the superimposed region,
The affiliation determining step identifies the cell region of a cell to which the biological material is likely to belong based on the physiological information, and determines that the biological material belongs to the specified cell region. To generate diagnosis support information. The diagnostic support information generation method according to claim 6,
The physiological information includes the area ratio of cytoplasm and nucleus, cell circularity, cell area, number of bright spots in the cell region, the staining reagent in a plurality of cells in which the cell region is superimposed in the overlap region. Any one or a plurality of non-uniformity of staining due to the above. An image for generating diagnostic support information by quantifying the biological material from a tissue section or cultured cell specimen stained using a fluorescent material combined with a biological material recognition site that recognizes a specific biological material as a staining reagent In the processing device,
A fluorescence image input means for inputting a fluorescence image representing the expression of the biological material in the specimen as a fluorescence bright spot;
A biological material position calculating means for calculating the biological material position from the fluorescent image based on the fluorescence of the bright spot region centered on the fluorescent bright spot;
A cell region extracting means for extracting a specific part of a cell as a cell region from a bright field image obtained by photographing the same visual field range as the fluorescent image or the fluorescent image;
A superimposed region extracting means for extracting a superimposed region where the cell regions of a plurality of cells are superimposed;
Biological material determination means for determining whether or not the biological material exists in the superimposed region from the superimposed region and the biological material position;
Superimposed cell information that acquires, as superimposed cell information, information related to a plurality of cells in which the cell region is superimposed in the superimposed region when the biological material determining unit determines that the biological material is present in the superimposed region. Acquisition means;
From the biological material position and the superimposed cell information, belonging determination means for determining which biological cells existing in the overlapping region belong to which of the plurality of cell regions superimposed in the overlapping region;
An image processing apparatus comprising: An image processing apparatus according to claim 8 ,
Expression of the biological material in a tissue section or cultured cell specimen stained with a fluorescent material used as a staining reagent that is combined with a fluorescent material that recognizes a specific biological material used in the image processing apparatus. An image acquisition device for acquiring a fluorescent image represented by a fluorescent luminescent spot ;
A diagnostic support information generation system comprising: A computer that generates diagnostic support information by quantifying the biological material from a tissue section or cultured cell specimen stained using a fluorescent material combined with a biological material recognition site that recognizes a specific biological material as a staining reagent The
Fluorescence image input means for inputting a fluorescence image representing the expression of the biological material in the specimen as a fluorescent luminescent spot,
A biological material position calculating means for calculating the biological material position from the fluorescent image based on the fluorescence of the bright spot region centered on the fluorescent bright spot;
A cell region extraction means for extracting a specific portion of a cell as a cell region from a bright field image obtained by photographing the same visual field range as the fluorescent image or the fluorescent image;
A superimposed region extracting means for extracting a superimposed region in which the cell regions of a plurality of cells are superimposed;
A biological material determination means for determining whether or not the biological material is present in the superimposed region from the superimposed region and the biological material position;
Superimposition cell information acquisition that acquires, as superimposed cell information, information related to a plurality of cells in which the cell region is superimposed in the overlap region when the biological material determination unit determines that the biological material exists in the overlap region means,
From the biological material position and the superimposed cell information, affiliation determination means for determining which biological cell existing in the superimposed region belongs to which of a plurality of cell regions superimposed in the superimposed region;
Image processing program to function as