JP2002521685A - Spectral topography of mammalian material - Google Patents

Abstract

(57) Abstract: Mammalian matter (18) is illuminated by a light source (43) and a multispectral image prism and mirror spectrograph (30) that spectrally disperse the transmitted light substantially simultaneously. A multispectral morphology system and method for transmitting the image of (20) is provided. Spectral images are acquired and prepared for further processing. A processor (37), typically included in a computer system, processes the digital spectral data using a neural network to provide a material diagnosis. The light source may be filtered light that is re-emitted (ie, fluoresces) light and is absorbed by a specially prepared substance. The substance may be magnified and analyzed by a microscope (transmission and fluorescence) or in vivo (live) where the image is transmitted by another conventional image transmitter such as an endoscope. May be. The system can also display the transmitted visual image as well as the spectral image in conjunction with the diagnostic output.

Description

DETAILED DESCRIPTION OF THE INVENTION

The present invention relates to medical pathology, and more particularly, to a system and method for assisting in the automated analysis and diagnosis of diseased cells and tissues.

BACKGROUND OF THE INVENTION [0002] Pathologists are studying tissue and cell samples for the presence of malignancies and other diseases and abnormalities. As described below, recently, these three traditional disciplines of microscopy, spectroscopy, and digital image processing have merged, resulting in more than was possible with traditional microscopy alone. Even faster, medical pathology tools have emerged that assist pathologists in analyzing and diagnosing these conditions, automatically and accurately.

[0003] 1. Microscopy In one method of performing conventional histopathology and cytopathology, one or more slides are prepared by removing tissue biopsy or cell smear from suspicious areas of the patient. After that, the pathologist said Wright,
Or send and study the sample by microscope. Tissue biopsies are often sliced into very thin pieces and stained using one of the well-known tinning techniques, such as, for example, H and E (hematoxylin & eosin) tinning, or a combination thereof. Cell smears are similarly colored. Each region with an abnormality is visually compared to an indication of a known disease to determine whether the disease is being given and what disease is being given.

[0004] Fluorescence microscopy is another diagnostic tool often used by pathologists.
Instead of passing incandescent light, “white” light through a sample from a light source such as a xenon or mercury lamp, ie, “high energy excitation”
Irradiation of "rgy exit" light passes through one or more filter sets that only pass light of a certain wavelength that strikes (does not pass) the sample. In response to light incident on the sample, the normally, but not necessarily, colored sample emits a variety of fluorescence. That is, a low energy emission of measurable wavelength (color) and intensity is performed to provide diagnostically evaluable information about a tissue or cell.

[0005] Fluorescence microscopy techniques are of two categories: endogenous (or natural) phosphors or fluorophores, or those involving reactions, and exogenous (derived from outside the sample). Can be classified into categories. The former involves two types of reactions: autofluorescence in which tissue compounds such as elastin and collagen fluoresce spontaneously without any additives from the outside of the sample, and the combination of specific antigens against antibodies ( (Antigen-antibody reaction), whereby naturally fluorescing antibodies stop the antigen in the tissue and include immunofluorescence which fluoresces naturally for this type of combination.

The latter refers to fluorescent tagging, or labeling, of cells. In this technique, molecules such as peptides, proteins or antigens that attach to highly specific targets are "tagged" with fluorophores such as fluorscein to determine if there is a positive interaction. The decision is made. In one method, for example, fluorescein conjugated to an antibody, which is a cell derived from a single cell, is used to identify a particular antigen. This technique is often used to assist in the identification and diagnosis of atom-based diseases such as viruses, but can also be used, for example, to identify bacteria and malignant tissues. Another technology in this category, fluorescence in situ hybridization (
"FISH": fluoresent in situ hybridiza
tion) utilizes the following a) or b): a) for the labeled probe ("antisense strand") and endogenous mRNA ("sense strand"). Paired nucleotide interactions between, or b) a target binding protein or acceptor (r
proteins-protein interactions in which tissues are labeled and which tissues lie dormant in the tissues containing the receptors. Extrinsic reactions also include general fluorescent stining, such as the application of auramine / rhodamine preparations for smears suspected of tissues such as tuberculosis. This type of stining can enhance certain features, such as the nuclear envelope from the cytoplasm.

[0007] 2. Image Processing The increasing processing power, speed, and miniaturization of digital systems and computers have revolutionized the field of pathology. Pathologists have traditionally relied exclusively on microscopy, their own vision and experience to diagnose pathological diseases, but current imaging and processing techniques have greatly increased the accuracy and speed with which today's pathologists can diagnose. It has been raised to. For example, a charge-coupled device (CCD: charge-coupled device)
e) Coupling the array to a microscope enabled high resolution capture of data representing the microscope image. These video images are analog-digital (AD: a
It can be digitized by a nalog-to-digital converter, displayed on a display device, magnified or otherwise manipulated, and stored on other digital media. The video image data can be further processed by various image processing software. A neural network system, which is non-linear, wherein the system separates each pattern from the provided data and learns to identify and extract the mathematical relationships underlying the data comprises one such processing structure. It is. When applied to pathology, in order to automatically characterize and evaluate samples for specific diseases, the neural network system converts the sample data collected by the CCD array into healthy tissue and cells as well as diseased tissue and diseased tissue. The data representing the cells is compared with each learned pattern.

[0008] 3. Spectroscopy Spectroscopy is the study of the spectral properties of each object, and in particular, the study of the light components (individual wavelengths) of each object and the intensity at these wavelengths. Spectroscopy has long been used as a tool in the field of chemistry to identify each element because each element has its own unique spectrum. Spectroscopy has also shown great promise in the field of medical diagnostics, as it was first realized that spectrometers could provide detailed information on both the chemical and physical properties of materials. For example, it has been emphasized that certain spectral characteristics, such as fluorescence, indicate the presence of a malignant tumor or metabolic state of the tissue. This information can be related to the position of the object in the field of view of the target. It is often possible to determine the boundaries of ambiguous edges by relating each spectrum with the "clustering" of each spectral object. Thus, the field has recently become an evaluable partner with microscopy to analyze the spectral properties of certain suspicious pathological samples.

Unfortunately, the potential of spectroscopy to improve or revolutionize the field of medical pathology has not yet been fully exploited. First, as mentioned above, conventional fluorescence microscopes have one filter that blocks all light except for a single wavelength of light that excites the sample, and passes only the re-emitted light to the optical output. (And rejects higher energy excitation light). Since the sample is usually stained with a fluorophore that fluoresces at a single wavelength in the sample, the method provides valuable diagnostic information about the sample at this single wavelength only.

[0010] To gather even more meaningful information about the sample, the entire image must be recaptured at a second wavelength through a second set of filters. The process is such that the desired number of spectral frames is the "spectral envelope (sp
This is repeated a number of times until an “envelope” is obtained.
Each frame is stored on a computer and the composite image can be analyzed by the processing software described above and / or displayed on a display device. This is a multispectral image ("MSI").
ing).

However, this particular technique for MSI is impractical for a number of reasons. First, this technique requires the availability of a number of costly filters. Second, replacing the filter set inside and outside the fluorescence microscope is time consuming. Third, the sample must typically be colored with a number of dyes that generate fluorescence at each excitation wavelength, a phenomenon that can diminish or degrade the diagnostic value of the sample, "bleaching (bleaching) the sample. bleaching) ".

Rotatable filter wheel, acousto-optic tunable filter (AOTF:
Acoustic-optical tunable filter, liquid crystal tunable filter (LCTF)
Several new automated filter systems, such as filters and interferometers, are effective, all of which capture the entire image at each wavelength continuously until the entire spectrum is obtained. However, these systems are extremely costly and the data processing of these images requires a typical 512 × 51, for example, capturing 80 wavelengths.
This is a tremendous task, considering that a two-pixel array would result in 21 MB of data per scene. It is not unusual for a researcher to try to force a reduction in the number of wavelengths obtained, primarily due to complex and time-consuming data processing. While the 80 wavelengths seem to be a large number, it is not unusual for an analytical chemist to simultaneously obtain up to 1,024 wavelengths using a laboratory set-up CCD detector. Thus, obtaining and classifying so many consecutive frames can be prepared using many fluorophores (also be prepared to optically decolorize each sample) if computational speed is required or desired. It is not practical for the analysis of samples that do. Also, the physical cost of these instruments is extremely high, showing only a small trace of reduction. Furthermore, each of the above-described methods requires that the target object be stationary, and is not subject to chemical changes due to the environment during spectrum acquisition.

[0013] Point spectroscopy is one known method of obtaining spectral data from an object. This technique, as the name implies, captures the entire spectrum of only a single small point of the object at a time. However, to be practical and meaningful for the field of medical pathology, the spectroscopy system must be able to spectrally disperse, display and analyze the entire sample, or at least each fragment of the sample. Thus, point spectroscopy is not an ideal solution to the problem described above.

Therefore, it is possible to obtain MSI data of a meaningful quality of a mammalian substance including a pathological sample quickly and efficiently, and to automatically analyze the data for identifying a disease of the substance. There is a clear need for a low cost system that can do this.

SUMMARY OF THE INVENTION [0015] The present invention provides a method for automatically identifying one or more cells or tissues for signs of acute disease through the multispectral acquisition of each image of a mammarian matter. It addresses these needs by providing a system and method for evaluating. In the broadest embodiment, the system comprises 3
It has two main components: an image transmitter, a unique multispectral image spectroscopy subsystem and a processor. The image transmitter includes a light source for irradiating the substance and an optical output, and is capable of transmitting an image of a fragment of the substance to the optical output. The multispectral image spectroscopy subsystem is connected to the optical output and is adapted to substantially simultaneously spectrally disperse the transmitted image to generate a spectral image. The processor then processes the spectral image to provide diagnostic data representing the image.

This system eliminates the need for continuous filtration, but in a single acquisition, rather than just a single point, of a substantial fragment of the material. This provides the advantage of obtaining a complete multispectral image.

In a more detailed embodiment, the multispectral image spectroscopy subsystem comprises an image spectrograph and a charge-coupled device (CCD) camera. The spectrograph is an entrance slit that allows the passage of light from a portion of the transmitted image of the fragment of the substance, and disperses the light passing through the entrance slit into a number of component wavelengths in a predetermined spectral range. , A spectral dispersion prism for generating a spectral image and a mirror arrangement. The charge coupled device (CCD) camera couples to the spectrograph to obtain and prepare a spectral image. Preparing an acquired image generally involves two steps, as is well known in the art of image processing, digitizing the image using a suitable digital processing algorithm and preprocessing the digitized image. . The processor resides in a computer subsystem that processes the digitized spectral image and provides diagnostic data representing a portion of the image. This system offers several important advantages. The spectrograph of the present invention comprises:
A low cost device that supports simultaneous capture of an entire sample to eliminate the time of intensive and continuous capture of multiple wavelengths of a given portion of the sample. This allows for high-speed data processing that contributes to the real-time diagnostic capabilities of the system. Along with digital cameras and computer data processing, spectrographs provide extremely simple, efficient and low cost tools that can be used to diagnose any suspicious material whose images can be sent to the spectrograph. Further, the entire system is a small part of the cost of other MSI systems such as interferometers.

In a still more detailed embodiment, the image transmitter is a lens-based image magnifying system or telescope, broadly defined as any of a variety of magnifying optical instruments. In one of these embodiments, the present invention also provides a method for illuminating, magnifying, and transmitting each image through a microscope, through the multispectral acquisition of each image of these samples to the optical output, to prepare the prepared cells. Provide a low cost and efficient system for automatically assessing the pathology of pathology and histopathology samples. The optical output of the microscope may also include a standard camera interface, such as a "C-mount" connection to quickly connect and disconnect an image spectrograph.

In yet another more detailed embodiment, the microscope includes an automatic xy stage that can controllably translate a slide for continuous MSI acquisition of the entire sample. After one slice has been obtained by the system, the computer-controlled xy stage automatically aligns or moves the sample so that an image of an adjacent portion of the sample has a spectral variance and a portion thereof. Through the image acquisition entrance slit for the acquisition of This process is repeated until the entire sample is obtained and studied as desired.

In yet another embodiment, a second image is captured from the microscope to capture a video image of an enlarged fragment of the sample and provide the image for display and / or further processing.
Are provided. In this further embodiment, a beam is directed between the microscope and the spectrograph to selectively direct an expanded optical output to either the first CCD camera or the second CCD camera. The assembly can be deployed. In this way, the system provides two diagnostic tools integrated into the pathologist, one for traditional image (video) imaging and the other for spectral morphology (topography) data. The latter can be provided, wherein the latter is a graphical representation, called a spectral graph (after such data has been performed by the neural network), tubular morphology, actual diagnostic printout, prognosis or suggestion and / or all of these It is provided in the form of a combination.

In an alternative embodiment, a beam splitter cube is placed between the microscope and the spectrograph to direct optical output from the microscope to both the spectrograph and the second CCD camera simultaneously. Can be arranged.

In a more preferred detailed aspect of the invention, when the microscope is set at 40 ×, the area of the sample passing through the 5 mm long entrance slit is approximately 1.25 μm width × 125
It is μm long. In this particular embodiment, the spectrograph is approximately 0.5 μm along a portion.
m enables spectral data to be obtained. Such 0.5μm × 1.25μ
Each of the m-wide fragments is called an "object." In the preferred spectrograph and the particular design of the CCD array volume, up to 240 objects can be simultaneously captured from each portion of the sample of interest. In addition, 400n
At a spectral resolution of 1 nm to 700 nm and a wavelength of approximately 15 nm at a wavelength of m, each spectrum of each object contains up to 740 wavelength data points in the range of 380 nm to 800 nm.

The neural network algorithm may alternatively be an unsupervised neural network (USNN: unsuperv)
ised neutral network, surveillance neural network (SNN: super)
rvised neutral network) or a combination of the two. The USNN functions to automatically identify and map (adjust) the presence of the "fingerprint" spectrum. The SNN can be implemented to automate the system and operate to perform routine automatic calibration.

One preferred method of spectrally analyzing a substance for the presence of disease based on a spectral analysis of the histomorphological and physiological bias of the substance from the mean is:
Illuminating the substance with a light source, transmitting the image of the substance to a multispectral image spectrograph, and converting the transmitted image through a prism and mirror arrangement to a number of component wavelengths in a predetermined spectral range. Spectrally dispersing; obtaining the spectrally dispersed image of the multiple component wavelengths; digitizing the obtained image and manipulating the digitized image; and Processing the spatially dispersed image to provide a diagnosis.

In a still more particular embodiment, the image transmitter provides a source of filtered light to the material and transmits an image of the fluorescent light re-emitted from the piece of material to the optical output.
Generally, the material comprises a prepared pathology sample and the image transmitter is a fluorescence microscope. In this particular embodiment, the present invention eliminates the need for multiple filters by simultaneously obtaining the entire spectrum of all fluorophores from the fluorescing fragment.

A method of practicing this more particular embodiment comprises irradiating the substance with filtered light of a specified wavelength such that the substance emits fluorescence, and wherein the substance is emitting fluorescent light. Transmitting an image of the fragment to an optical output; dispersing a portion of the transmitted image spectrally over a number of component wavelengths in a predetermined spectral range; and obtaining and preparing the spectrally dispersed image. And processing the data representing the image with a processor to classify the portion of the image into one of a predetermined number of categories representing a state of the portion of the image.

The systems and methods of the present invention are useful for assessing disease through the analysis of the fluorescent properties of mammalian cells and tissues that fluoresce through endogenous or exogenous reactions. The former, also called natural fluorescence, includes biological substances that have autofluorescent compounds such as elastin found in the aortic wall, arteries and other tissues. Analysis of this type of material using the present invention yields diagnostically evaluable information without any fluorescent tinning preparation. Immunofluorescence methods are also in the endogenous category.
In this case, the substance is generally prepared with at least one immunoglobulin antibody labeled with a tag, such as fluorescein, which is useful, for example, for kidney and heart analysis.

As an alternative to endogenous reactants, the substance, or sample, can be treated with at least one fluorescent dye. Specifically, the sample can be processed using a histochemical probe labeled with a fluorescent dye. More specifically, fluorescence in situ hybridization ("FISH assay") is performed on a sample, and the processor is a genetic disease, a malignant tumor, a bacterium, and a virus such as the human papilloma virus (HPV). Providing data indicative of the presence of at least one lineage of one of the following: Alternatively, the sample is a single cell-derived antibody and one of a polyclonal antibody conjugated to a fluorescent tag. An immunocomposition chemical stain with markers of immune histology can be prepared, including one example of which is the direct immunofluorescent assay (DIF).
y) and colored with Chlamydia trachomatis (CT: chlamyd)
ia trachomatis). Even more beneficially, the system is designed to analyze different stains, immunochemical markers or all of these markers to analyze the condition in which each stain or fluorophore probe is designed to separate individually. Samples treated with multiple probes labeled in combination can be quantified simultaneously.

Thus, the present invention provides a critical tool in the field of cervical cytology by providing a tool for automatically assessing Pap Smear or cervical biopsy against both various HPV strains and CT bacteria. Great progress. In particular, pulp smear samples were prepared using direct immunofluorescence (DIF: direct) using FISH analysis for HPV discrimination and directed to detect CT.
It can be prepared using an immunofluorescent stain. In this way, a single, dual-colored papu smear analyzed by the system of the present invention provides a rapid, robust, and robust indication of some of the major cytological diseases and precursors to cervical cancer. Resulting in lower cost identification.

Conventional stinging for biological substances that elicit a fluorescent response is another category of reactions applicable to the present invention. For example, smears can be fitted with well-known fluorescent stains, such as rhodamine / auramine, to generate a fluorescent response that can be exploited by the present invention for the identification of some diseases, such as tuberculosis.

Other features and advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention, as outlined above and defined by the enumerated claims, provides the following detailed description, which should be read in conjunction with the accompanying drawings. It can be better understood by reference. This detailed description of the specific preferred embodiments set forth below so that specific implementations of the invention can be constructed, used, and practiced is not intended to limit the recited claims, but rather to include It is intended to be useful as an example. The specific examples described below are for the morphology or topography (multi-topography) of the multispectral response of mammalian materials, ie, cells and tissues.
a preferred specific implementation of a system that provides automated spectral data acquisition, analysis and diagnosis of each image of a substance suspected of having a disease. However, the invention also applies to other types of mammalian substances, such as those which are not suspected to be diseased, but which nonetheless require identification of their morphological and physiological properties. can do. Further, in the presently described invention, the material is illuminated by a light source, one particular light source providing filtered light to enable analysis of the fluorescent properties of the material.

Before describing the present invention in more detail, ie, before describing in detail the method of multispectral analysis used by the present invention, its various components and experimental results, the “hidden morphology” of tissues and cells The theory behind the application of spectral morphology as a means to show "hidden morphology" and its physiology as well as its implementation for medical pathology will now be described.

I. Histomorphological effects on normal and abnormal cells and tissues Morphology, such as cell size and shape, their environment (cell morphology), and tissue biopsy studies can provide diagnostic information on the state of these structures that can be evaluated. It has long been recognized that you can. General biological activity is best reflected in the cell structure of the nucleus. Functional activity is mainly reflected in cytoplasmic morphology. The "healthy" basal morphology of each cell is the normal cellular morphology and structure that is stress free from the pathological process, eupracia (e)
upsia) can be considered as a reference level. In Euprasia, the key nucleus structures are rounded or rounded under light microscopy, appear to be uniform and have a regular pattern with respect to nucleus components such as chromatin, and the degree of predictability from one nucleus to another. have.
In tissue analysis, predictability from one cell of the same type to another would be expected.

A number of morphological effects of processes associated with malignancies and precursors thereto have been identified. For example, significant irregularities in the shape of the nuclear envelope of cells, confusion in the structural orderlines and shape of chromatin and parachromatin (blue regions of the cell nucleus), enlargement and irregularities in nucleoli shape. And, importantly, the increasing ratio of nuclear area to cytoplasmic area (N / C ratio) is some of the morphological factors that have been found to help find cancer.

Unfortunately, to date, this type of morphological analysis under transmission or fluorescence microscopy has itself been of limited value in diagnosing diseased cells and tissues for several reasons. Was. First, morphological changes in cells and tissues occur at varying degrees. Many changes cannot be recognized (clearly or at all) by an experienced pathologist, even through the eyepiece of a light microscope, or even in manipulating the video images of the slides of each sample. These hidden morphologies are due to either the intrinsic nature of change, the stage of morphological activity, or a combination of the two. Second, to date, the presence of cancer characteristics has clearly indicated that the particular cell or tissue under observation has the cancer, while the absence of it means that there is no cancer. There were no observable absolute morphological features-or criteria for malignancy.

The present invention irradiates these with a light source, irrespective of in vivo or prepared samples, and obtains their spectral information with the help of a unique multispectral image spectrograph and a sophisticated processing algorithm. The analysis is used to indicate the hidden morphology of suspect cells and tissues.

II. Methods and components Push Bloom Multispectral Image There are several traditional ways to obtain multispectral information from a separate field of view. As mentioned above, one method obtains the entire field through a series of wavelength filters. The number of filters will be equal to the number of wavelength data points required to identify the spectral characteristics of each component of the field. Since the field of view is fixed, data cannot be obtained if the object moves or changes anyway.

In another method developed for remote earth surveillance and implemented according to the present invention, the system obtains a small portion of the field and simultaneously obtains the entire spectrum of the portion in a specialized chromatic dispersion spectrograph. Through the small part. Moving to the next adjacent part is necessary to cover the entire field. By linking each acquisition, the entire field can be covered. This method is often referred to as "push bloom" spectral morphology. This is because the sample is "pushed" across the entrance aperture, or slit, of the spectrograph, and in many respects is more versatile than the previously described methods of multispectral imaging acquisition. Is widely and efficiently used.

The present invention uses push-bloom multispectral image acquisition of suspicious mammalian tissues and cells to show their hidden morphology in the case of each sample prepared on a glass slide and to analyze in vivo assays. In some cases, they show their hidden physiology, and finally result in a medical diagnosis of tissues and cells.

[0041] 2. Basic Components and Method FIG. 1A is a block diagram showing the basic components of the system. Image transmitter 1
Comprises a light source 2 and an optical output 3. A light source 2 illuminates a mammalian substance 4 to be analyzed, the image of which is transmitted to an optical output. The image spectroscopy subsystem 6 spectrally disperses the transmitted image substantially simultaneously over a predetermined range of multiple component wavelengths. The method of spectral dispersion uses spectrographs originally designed for remote, telescopic earth monitoring and astronomy, and is currently used for both military and civilian applications. The original spectrograph was Warren et al.
al. ) Was granted a patent right (Patent No. 5,127,728).
It is designed for use in the infrared wavelength range of 5 μm and is incorporated herein by reference. For life science applications of the present invention, optics are primarily
Redesigned for use in the 60 to 800 nm wavelength range. A processor 8, such as a PC computer, loaded with the appropriate software, operates on the data representing the spectrally dispersed image, and finally provides a diagnostic output 8 on the condition of the substance 4. This diagnostic output 10 could be provided to a computer screen, printed output, or any conventional output device.

FIG. 1A illustrates an image transmitter arrangement in which the light source 2 transmits light through the substance 4. For example,
This is generally the case when the transmission microscope is an image transmitter that provides white light for relatively thin samples, such as cell smears or very thin tissue biopsies. FIG. 1B shows an alternative arrangement for the image transmitter 1 representing one of two optical scenarios. In one scenario, light from light source 2 is reflected by material 4 to reach optical output 3. Reflection can be used in some situations, one situation where the suspicious tissue biopsy is too thick to allow it to penetrate the light in a meaningful way, and another situation to analyze It is when the substance is not removed from the patient, but rather is in vivo. In a second scenario, represented by FIG. 1B, light from a light source is filtered and the filtered light is absorbed by a substance. In response, the tissue or cell re-emits light at a lower energy level. This fluorescence is provided to a multispectral image spectroscopy subsystem for dispersion and analysis.

FIG. 2 is a schematic diagram of a more detailed embodiment of the present invention in which the image transmitter is a fluorescence microscope. The main components of the multispectral morphology system 12 are a fluorescence microscope 40, a prism and mirror image spectrograph 30, a first CCD camera 34.
, A processor 70 and a diagnostic output device 80.

Fluorescent microscopes 40 found in many laboratories and research facilities include an eyepiece 41 and an optical output 42 with a standard camera interface such as a video port with a “C-type” mount. And The light source 43 of the microscope is filtered by an internal filter and absorbed by the sample 18 prepared on the slide glass 16. The sample is then at a lower energy level and re-emits light of varying intensity across the sample, and the lens magnifies the image of the cross-section of the fluorescent sample 18 as well as these images. To both the eyepiece 41 and the optical output 42. Samples generally don't have much to do,
Using conventional techniques, they are prepared using conventional dyes and / or are labeled with conventional tags. The spectroscopy subsystem connected to the microscope 40 comprises a chromatic dispersion image spectrograph 30 having a prism and a mirror and an entrance slit 32. This slit allows a portion (s) thereof, as can be seen in more detail in FIG. 2A.
sample), which itself has many “objects” 21
The image of this portion 20 passes through to reach the spectrograph 30 and the first CCD camera 34. The new and modified spectrograph 30 provides good image quality over a wide range of simultaneous operation of each wavelength so that large spectral spacings can be viewed without moving any components of the system. .

An assembly 60 configured with a “flip” mirror 62 to direct light rays, also referred to as a “beam splitter”
Provides both video image and spectrum acquisition. Thus, when the mirror is flipped in one direction, the spectrally dispersed light is focused and acquired by the first CCD matrix array 36 of the camera 34. Analog images are prepared for viewing and further processing, as is done in conventional CCD cameras. In particular, images are analog-to-digital (A to D: analog-to-dig).
It is digitized by an ital converter and then manipulated or pre-processed by a spectral image preprocessor 37, often called a digital signal processor (DSP). The prepared spectral image can be sent to a processor 70 and / or a CRT. It can be displayed on a display device 38 such as an LCD screen or other conventional display device.

When the flip mirror 62 is rotated in the second orientation, a visual, microscopically magnified image, ie, a video image, becomes a matrix array 56 of the second CCD camera 54.
Captured by This image is also digitized by an analog-to-digital converter, and an advanced image processing algorithm 57 (another DSP) is used in a conventional display device 58.
Manipulate the image for a high quality display on top. The conventional image pre-processing algorithm in the DSP includes smoothing, normalization, and background subst.
fraction, principal component analysis (PCA: principal component)
ent analysis) and partial least squares (PLS)
ast square). In this way, both the spectral image data and the visual image data of the sample can be displayed, stored, analyzed and / or further manipulated for the pathologist. In an alternative embodiment, not shown, the beam-splitter cube is replaced with a beam-directing assembly 60 to send light to the spectrograph 30 and the observed image CCD camera 54 simultaneously.

The spectrograph 30 uses a prism made of inexpensive flint glass. The support and body of the unit can be constructed of cast aluminum or fluted metal plates or other materials that are prepared for weight reduction and increased rigidity.
The optics are fully ray tracing. This optics may be a standard microscope, typically an image port, with or without a beam director 62, using a standard "C-type" implementation or any other acceptable connection means. The optical output is easily assembled.

In one preferred embodiment, when the fluorescence microscope is set at 40 × magnification, each portion 20 captured by the spectrograph 30 through the slit 32 is approximately 1.25 μm wide × 125 μm long. Further, the system can acquire spectral data at approximately 0.5 μm along portion 20. Preferred spectrograph and C
For a particular design of CD array capacity, up to 240 objects (objects)
) Can be simultaneously captured from each portion of the sample intended for The paired spectrum for each object is 700 nm from a spectral resolution of 1 nm at 400 nm wavelength.
It includes up to 740 wavelength data points in the range of 380 nm to 800 nm at a resolution of about 15 nm at nm wavelength. However, it is understood that different (and larger) object and spectral envelope resolutions can be obtained using CCD cameras with other slit sizes and other resolution capacities.

Continuous scanning of the entire image of the sample 18 is achieved by scanning the microscope stage under computer control with histology and cell morphology settings.
The first CCD matrix array detector 36 collects individual spectra along rows of pixels from each object located at the entrance slit 32. This format is sometimes referred to as "open image." This is because there is no restriction on the area of the object to be inspected. Also, this format is certainly the least costly and most flexible method for pathological samples undergoing microscopic examination.

All objects in a slice are acquired within a few milliseconds depending on the signal strength. Photo-bleach is particularly reduced if objects such as multiple single cells or glomeruli are selected with automatic pattern recognition and associated with certain wavelengths.

Processor 70, typically a part of a computer system, such as a PC, includes a powerful neural network 72 that provides near-instantaneous recognition of the spectral fingerprint of each object 21 of sample 18. In one preferred embodiment, the neural network can process substantially simultaneously. There are up to 240 objects, each as small as 0.5 μm. The relatively small file size for each acquisition greatly increases computational speed and simplifies memory management.

FIG. 3 illustrates how the push-bloom methodology is applied to the present invention for multispectral analysis mammalian material. At step 100, an image of the substance, whether prepared on a glass slide or in vivo, is transmitted to an optical output. It is understood that in the broadest embodiment, the images can be transmitted via any known means, such as transillumination or reflection from materials, or fluorescence from each material. In the case of microscopy (transmission or fluorescence), a special field, or fragment, of the enlarged sample is brought into the entrance slit 32 of the spectrograph 30. In step 102, up to 100 objects 21
A single portion of the fragment containing the light passes through slit 32 and strikes the first curved surface of the prism at step 104, is refracted, and is refracted, as described in the Warren et al. Patent. It hits the surface and goes out and hits a spherical mirror. The chromatically dispersed light is then focused back through the prism and stored in a first CCD matrix array, which in step 106 acquires and prepares the light. As used herein, but not necessarily, this preparation (tissue specimen) involves digitizing and pre-processing the spectrally dispersed light. Preprocessing or manipulation of the digital image is achieved using conventional DSP algorithms to improve its appearance. More than 80% to more than 90% of all light, over the entire wavelength range, is transmitted through the system.

Next, at step 108, the processor processes the spectral data representing the spectral envelope of each object in a portion using the neural network to classify the tissue morphological pattern of each object of the cell. The system then proceeds to step 1
At 10, the status of each acquisition is investigated. If only a single acquisition is required or desired, the process may pause at step 112 and diagnostic data may be output for examination by a pathologist. However, often, an additional portion of the sample should be spectrally dispersed and analyzed. That is, in step 114, the answer 110 to the survey is "no," and an adjacent portion of each image is transmitted through the entrance slit. In the case of microscopy, the glass slide of the sample is moved by the xy stage of the microscope 40 to allow the passage of light representing a portion adjacent to the previously dispersed portion. The process then returns to step 104 for spectral variance, acquisition, and analysis of each object in its second portion. This process is repeated until all or a sufficient area of the sample has been analyzed in this way. Once this successive acquisition process is complete, the spectral data for each object is stored in the appropriate memory "bin (
bin), the processor can combine this data to provide a complete diagnosis for the sample, as described below.

There are many advantages to using push-bloom spectral data acquisition for cell analysis. First, the diagnosis is practically instantaneous. Obtaining sufficient spectral data using alternative techniques will require the creation of large digital files prior to data processing. The total acquisition according to one preferred embodiment of the present invention is still only 185 KB, but each includes 240 spectra with 740 data points. Files of this size are easily processed, allowing very fast data processing and decisions. Further, the entire system is much less costly than other methods of analysis such as interferometers.

The method also allows the coarse parameters to be determined prior to the high resolution scan by the fast low resolution evaluation. In addition to obtaining a multispectral morphology map, the system also provides a temporary color "fingerprint" indicating normal and diseased tissue.
"print" the image image of the sample with "spectrum" spectrum
can do. The system also makes a distinction between chronic and acute illnesses, as well as evidence of anti-rejection drug toxicity.

[0056] 3. Neural Networks Traditional mathematical algorithms perform calculations continuously and deliver results based on linear transformations. A neural network (NN) performs calculations in parallel in a manner similar to that of the human brain and performs non-linear transformation. To date, the system includes unsupervised neural net (USNN).
work). However, the system has two neural networks: the USNN and the supervised neural network (SNN).
network). The USNN collects, characterizes, classifies, and places each spectrum from each object in its own "bin" of spectral characteristics. Thus, the USNN automatically recognizes and maps (ie, adjusts) for the presence of the “fingerprint” spectrum. That is, bins containing the same spectral characteristics are identified. The SNN can be used to determine special features that distinguish the spectrum of each bin from spectra from other bins. Thus, the system can compare the indicated spectrum of each object of the substance under analysis to the map to identify the presence of the disease.

The USNN can be thought of as a “filter” or “sieve” that characterizes and classifies each spectrum as each spectrum is collected, and treats each spectrum as a tissue form of similar spectral characteristics. Put in a bin or "class". The process is similar to alphabetizing the words in a book by combining words that have the same or a common root. By the end of this procedure, the number of bins represents the number of spectral objects defined by the spectral morphological features. This process is referred to as "digital chromatography". Because the function is almost identical to the process used in analytical chemistry for the separation of a mixture of chemical compounds. The major difference is that the USNN program adds extra dimensions that provide spatial information by mapping a special class of spectra back into the sample itself.

The operator may decide to combine some spectral classes, such as background features, or remove others. This is achieved by "thresholding" either by the user or the computer. This is a process that allows the USNN to delineate, identify and separate identical and non-identical spectra within the limits of user-defined applications. Human "wet neuron" to set up threshold
To allow the human experience to control the operating limits of the software.
The USNN becomes a “white box” rather than a “black box”. This is because the sensitivity and operating parameters of the network are always effective against the influence of the operator.

Once the USNN has self-adjusted, in a well-limited sample, the USNN
N compares each newly acquired spectrum with a spectrum class and category that recognizes it. The USNN identifies matched, near-match, and mismatched spectra in a new acquisition. Each class of spectrum is coded into a few bytes of data and stored in memory. Any future spectrum acquisitions are similarly coded and compared to stored data. This process reduces the spectrum of 740 data points to blocks of memory by a small byte in size, so that recognition of hundreds of spectra occurs in near real time. Adjustments can take up to two minutes for complex materials and seconds for simple spectra.

The processor 70 operates in cooperation with the USNN, and operates as a supervised neural network (SNN).
vised Neural Net). This SNN identifies the unique characteristics of each bin, automates the system and performs routine auto-calibration. Past experience has shown that combining these two neural network architectures is very effective in controlling many variables, some or all of which can change over time. Most humans tend to monitor or "accommodate" certain changes. In multi-parametric systems such as cytopathology this is very dangerous. This is because a change in one variable will result in a non-linear affect somewhere in the process, which will impair the accuracy of the evaluation. The two neural network systems can form a transparent affinity to automatically ensure that the conditions required for accurate and repeatable diagnosis are not compromised by temperature changes, mechanical misalignments or operator errors. it can.

One or more plausible systems of the present invention provide for spectrally characterizing all displayed pathological disorder and morphology of diseased cells,
It can be adjusted and readjusted by the USNN, thus generating a "library" of spectral fingerprints representing these indicated states. In the end, it is expected that the entire and finite field of pathological disease categories will be spectrally characterized by the USNN. If this milestone is achieved, each system could be equipped with an SNN with a database containing all of the pre-tuned spectral fingerprints in various possible states. If a sample of unknown pathology is provided to the system of the present invention, the SNN will quickly compare its spectral fingerprints to the spectra of all fingerprints stored in the database or in the appropriate category of the database. In the potential of powerful, fast and low cost personal computer systems with large amounts of memory capacity,
A single personal computer system is a "spectral library (spe
ctral library) "
It will result in an automatic diagnosis of the status of any cell or tissue sample given to the system.

Those skilled in the art have developed protocols that allow specific tissue structuring assessments to be selected from a computer menu to simplify and minimize continuous human interaction with the system, as well as to perform auto-calibration. , It is possible to continuously monitor the state of the whole system and to record the operation of the user.

[0063] 4. Outputs The system of the preferred embodiment can provide several diagnostic outputs. 4, 5, 6, 7 and 8, described in more detail below, show some of the outputs from the analysis of mammalian material, ie, spectral samples from patients positive for tuberculosis. The sample is colored with a chemical dye preparation, ie, auramine / rhodamine. Outside luminescence (epi-fluor)
The slides seen under the esence allowed the system to provide spectral properties and mapping of the fluorescent tuberculosis bacteria. The neural network monitors the intensity, and if multiple spectra are provided (although not necessary for this example where only a single fluorophore and a single emitted wavelength are present), the deconvolution returned to the computer ( deconvolution)
The request and paint monitor the location of the identified and processed spectrum on the sample.

In particular, FIG. 4 shows a greyscale image of a fluorescently magnified fragment of a sample on the image display device 58, ie, an “observed image. The image display device is, for example, a CRT. Or it could be any conventional display device such as an LCD screen or the like, etc. Fig. 5 shows the same image of the false color representing the spectrum underlying a portion. Was identified in FIG. 5,
After profiling using USNN, "a more refined version of a spectral image of a portion partitioned into channels. Figure 7 is a graph of 25 of the 240 possible spectra, the y-axis of which is the y-axis. Represents the intensity, or magnitude, of light emitted from an object in a portion of the sample over the spectrum, with the fluorescence envelope between the baseline between 540 nm and 700 nm (peaking at about 580 nm). The difference in intensity is captured as a function of position on the sample with a clear depiction, and shows how the USNN correlated each spectrum with similar properties to the object, in particular FIG. FIG. 4 is a diagram showing each object in the left half and right adjustment sets, that is, each object of another sample.

All of this visual information can assist a pathologist in analyzing and diagnosing a substance. However, in the end, the system is designed to provide, via the neural network, an actual diagnosis of the condition presented by the substance under observation.
The diagnostics may be provided on any conventional output medium, such as a display device, a hard copy stored in memory and transmitted to another computer system, or any combination thereof.

III. Application of Multispectral Morphology to Fluorescent Materials Many fluorescent techniques have been found to be extremely effective in the analysis of various types of biological mammalian materials. Some techniques involve the endogenous properties of special substances (ie, natural fluorescence), including autofluorescence (eg, for aortic walls and arteries) and immunofluorescence (eg, for analysis of kidney and heart tissue). Light).
Other non-endogenous fluorescent techniques include fluorescent labeling, antibodies derived from single cells or polyclonal antibodies (markers of immune histology), in situ hybridization techniques, and standard dye methodology. Uses immunofluorescence containing. Mammalian materials as used in this section include materials of any of the above types provided by any of these fluorescent techniques, and, in the case of self-fluorescent materials, fluorescent tissue Can have no specimen at all.

However, in all these cases, the analysis for such substances is greatly enhanced by incorporating the automated spectral morphology system of the present invention. The remaining description details certain applications of the invention and the tests performed to ascertain their efficacy in diagnosing pathological diseases.

1. Application to Substances That Fluoresce Naturally (Endogenous Substances) Any substance found to contain substantial self-fluorescent compounds such as elastin present in the aortic wall and arteries or collagen present in other substances Analyzes without the use of light stining can be used to assess the autofluorescence pattern of diseased aorta and arteries as well as normal aorta and arteries, as well as generate basic diagnostic information. In addition, native organs such as kidneys, hearts and other organs that use immunofluorescent staining techniques, including antisera that fluoresce against immunoglobulins, to diagnose both the underlying disease and rejection And the analysis of natural fluorescence of cell and tissue material from transplanted organs can also be greatly enhanced by using the multispectral image spectrometer system of the present invention.

A. Application to Autofluorescent Fluorescent Material (Fluorophora) FIG. 9 shows the spectral characteristics of two samples of the aortic wall with a self-fluorescent elastin layer. These samples were tested on an epi-fluorescence microscopy excited at 436 nm and showed intense blue-green fluorescence. The spectral features provided by the spectroscopy subsystem of the present invention and shown in the figures make a clear distinction between the normal aortic wall (curve 1) and the aortic wall showing degradation (curve 2). Eventually, once the neural network has fully tuned the system with many such samples, identification and classification for the entire area of the normal and degraded aortic wall is obtained, and the system of the present invention is an unknown diagnostic tool. Any aortic wall can be analyzed and diagnosed.

B. Application to immunofluorescence.Diagnosis of certain primary organ diseases, such as end-stage renal disease, certain heart diseases, and rejection of transplanted organs, may be based on immunofluorescence findings in frozen sections of biopsy samples. Partly relies on effective evaluation. For example, the ESRD population is progressively increasing year by year. Both dialysis and kidney transplantation are effective techniques for extending life with ESRD. Commonly used diagnostic tests (ie, kidney transplant ultrasound and hippocampus scintigram) help distinguish rejection from other causes of transplant malfunction. Pulsed-Doppler is an excellent tool for studying vascular complications, including kidney transplants, and helps distinguish vascular rejection from other complications. The diagnostic value of quantitative double Doppler acoustic holography (DS) in the evaluation of renal allografts is increasingly being examined critically. Thus, renal biopsies that establish a particular diagnosis by histopathological assessment of allograft dysfunction remain mandatory.

Indeed, diseases of certain renal parenchymal tissues, such as acute or chronic rejections (primarily due to cyclosporin A), which contribute to the malfunction of the transplanted tissue, the “de novo
) "Or recurrent renal glomerular disease, immunosuppressive niphrotoxicity (nephro
To-xicity can generally only be diagnosed by renal histopathological studies.

Recognizing the importance of effective and efficient histopathological analysis for these conditions, the system of the present invention provides a method for analyzing immunoglobulins, complement components, and plasma nitrogen compounds in organ samples stained. Used to evaluate light patterns and correlated with light microscopic and standard immunofluorescence findings and medical data. It has been found that spectral analysis of the following histopathological samples using the system of the present invention has yielded extremely valuable qualitative as well as quantitative information in the diagnosis of these samples.

I. Application to Kidney Biopsy The system of the present invention has been tested and found to assist in the early diagnostic assessment of biopsy from patients with kidney disease, as well as to assist in the regular monitoring of transplanted patients.

In particular, immunofluorescence studies of renal allografts with rejection show a non-diagnostic pattern of immunoglobulin deposits using existing fluorescence microscopy. The application of the spectral morphology microscope of the present invention has been shown to aid in the detection of immunoglobulin deposition as a function of spectral fingerprint and spatial correlation in tissue, and is a powerful digital method for current observation methods. It brings progress.

In patients with native kidneys, spectral images from the system can be compared to medical findings and standard and immunofluorescent studies to categorize the disease state and quantify its suffering I do. Currently, immunofluorescence is graded as-is (0 to +4). This system provides new quantitative and qualitative insights into the underlying structural abnormalities of immune and non-immune tissues based on kidney disease.

The experiment involved analyzing slides in the archives of patients with suspected kidney or heart transplants or native organs showing evidence of disease. Samples were treated with immunofluorescent stains specific for IgA, IgG or IgM. These are also available from Hematoxylin / Eosin (H & E).
n) Each sample was observed and analyzed by transmission (white light) microscopy, treated with staining. The system operating parameters were as follows:

[Table 1]

The excitation wavelength was 405 nm, and the spectrum was obtained through a long pass filter that passes all light at wavelengths greater than 470 nm and blocks all light at wavelengths shorter than 470 nm.

All glass slides were pre-inspected and light bleached to some extent or even roughly. Nevertheless, as shown below, the system has evidence that spectral morphology can distinguish between native kidneys with chronic disease and transplanted kidneys that show evidence of chronic rejection or cyclosporin toxicity. Brought.

(1) Comparison of Original Kidney with Chronic Disease and Transplanted Kidney with Chronic Rejection FIG. 10 shows that the spectral morphology is the same as that of the original kidney with chronic disease (curve 1), but with or without cyclosporine toxicity. It also shows a distinction between transplanted kidneys (curves 2 and 3) which also show signs of rejection. All of these biopsy samples were fluoresceinated for humans.
(anti-sera) IgA.

[Table 2]

(2) Differentiation between IgA, IgG and IgM FIG. 11 shows IgA and IgM in the spectral range from 550 nm to 650 nm.
And a clear distinction between immunofluorescent staining spectra for IgG and IgG.

[Table 3]

(3) Summary of Results In general, the results are most fascinating even at low signal strengths, highlighting the strength of the technique. These tests have provided strong indication that it is possible to distinguish between acute and chronic kidney transplant rejection and cyclosporin toxicity.

Ii. Application to heart transplant biopsies Traditional medical treatment physiotherapy and newer surgical interventions may result in short-term pain relief, but heart transplants are still a natural progression and poor progression for patients with end-stage cardiomyopathy It is only an intervention that can change the prognosis. Despite medical advances, rejection and deterioration of transplanted hearts still result in significant morbidity and mortality, and the progression of disease of the occlusive effect of transplanted hearts limits even longer survival. ing. Especially lethality from rejection, called humoral rejection, usually occurs early in the transplantation. The diagnosis leading to accurate treatment is best determined from immunofluorescence evaluation of frozen fragments of cardiac biopsy from the transplanted heart.

Non-rejection pathology is often seen as the next transplant, with ischemic or alcoholamine effects, interstitial fibrosis, myocardial calcification, and quility effects
and cyclosporine, which is associated with an infiltrate in the heart called f. Therefore, it is extremely important to be able to automatically diagnose for evidence of immunosuppressive disease even when there is no evidence of cell rejection.

The International Society for Heart Transplantation (ISHT: International Society)
y of Heart Translation provides a criterion that allows grading of cardiac rejection ranging from grade 0 to +4. The figure shows two spectra from two patients, both negative for fluid rejection with grade 0 of ISHT, and also negative for the quality effect. Both are intracardiac biopsies of the septum of a healthy ventricle of a heart transplant patient. Each slide was stained with an immunoglobulin, a third component of complement, and an antiserum conjugated to FITC against fibrinogen. Curve 1: (16757) negative quality effect, negative rejection, ISHT grade 0. Curve 2: (16471) negative quality effect, negative rejection, ISHT grade 0. This example provides a good indication that the spectrum is consistent between patients with similar findings.

In general, immunoglobulin G, immunoglobulin A, immunoglobulin M, Clq,
Immunofluorescence studies on immunoperoxidase on C'3, HLA-DR, and fibrinogen, as well as on cells in the heart (Factor VIII for antigen) and macrophages (KP1 [CD68]) have been well performed. Enhanced by the invention.
This is because spatially resolved immunofluorescent features can be quickly, accurately and cheaply studied and evaluated. In connection with neural network-enhanced data analysis, a spectrograph provides a direct, digital, objective interpretation of spectral objects present in such tissue to assist pathologists in making assessments. Thus, the spectrometer / microscope combination increases the speed and specificity for early signs of organ rejection.

[0086] 2. Application to Other Fluorescent Techniques As mentioned above, the biological material is colored with a fluorescent dye or labeled with a fluorescent dye. The immunocomposition chemistry comprises one category of each reaction and is a cell derived from a single cell with markers of immune histology, including polyclonal antibodies. Fluorescence in situ hybridization (FISH: Fluorescen)
tin situ hybridization) is another category,
It can be directed to genetic or nitrogen analysis. In the former case, hybridization is accomplished by nucleotide-labeled probes (called "antisense strands") and endogenous nucleotides (eg, mRN
A, called the "sense strand"). This is called a paired nucleotide interaction. In the latter case, the nitrogen substance is
Labeled and cultured with tissue containing the bound nitrogen substance of interest in what is called a nitrogen-to-nitrogen substance interaction. Examples of the application of the present invention to the diagnosis of diseases associated with such substances will now be described.

A) Application to Cervical and Vaginal Disorders Two special diseases, namely human papillomavirus (HPV: Human Pa
pilloma Virus) and Chlamydia trachomatis (CT: Ch)
Lamydia Trachomatis) has become a growing problem for pathologists. The first is believed to be a precursor of cervical carcinoma and the second is the most common sexually transmitted bacterial pathogen in the United States, causing substantial morbidity It has been recognized that. However, the diagnosis of both diseases is technically subject to the aforementioned limitations. PAP smears are currently well suited for the early detection of cervical carcinoma and other abnormalities by preparing colored and detached cervical cell slides for analysis on an optical microscope. Although a valuable screening tool, PAP smear detects only 50-80% of the abnormalities found by histopathological experiments on biopsy samples. Furthermore, these two diseases are not well tested against using conventional PAP smear selection.

[0088] Siadat-Pajou et al. And other scholars report that cancer-associated HPD genotypes in the nucleus of cervical vaginal cells are based on fluorescence-based in situ hybridization (FISH).
Through the use of an in situ hybridization assay, PAP smears have been shown to be detectable by DNA analysis of cells on glass slides. Images of the (filtered) cells obtained by fluorescence microscopy are then
Captured digitally by a CD camera and analyzed using an algorithm that detects all cell nuclei from DNA counterstained images. The image of the cell nucleus can be used as a mask and mapped onto one side of a FISH image of the same microscope scene, so that the corresponding fluorescent HPV signal from each cell nucleus is quantified (Siadato -Pajuu et al.
al): "Detection of HPV type 16/18 DNA in cervical vaginal cells by fluorescence-based in situ hybridization and automated image cytometry (Defection of HPV).
Type 16/18 DNA in Cericovoginal Cel
ls by Fluorescence Based In Situ Hyb
authorization and Automated Image Cytom
etry) ", Cytometry, 15: 245-257 (1994)). This method can be described as a single wavelength test of a fluorescence signal using a single wavelength filter. Cell nucleus DN
A large number of fluorescent tags that require the use of multiple filters for fluorescence microscopy in order to obtain more data on the A structure and obtain improved speed, specificity and accuracy for HPV identification Must be used. However, as mentioned above, single wavelength testing with multiple filters is time consuming, carries the risk of sample bleaching, and loses useful multi-wavelength data.

Furthermore, Chlamydia Tractis (CT: Chlamydia Trac)
direct immunization of smears with antibodies that are cells derived from fluorescein-conjugated single cells, although detection of the most commonly sexually transmitted disease in the United States was difficult using standard PAP smear techniques. Fluorescence (DIF: di
correct immunofluorescence stinging has been shown to be effective in its diagnosis (Garrozzo et al.).
al. ): "Diagnosis of Chlamydia trachomatis: Correlation study of papu smears and direct immunofluorescence (Chlamydia trachomatis diagnosis)
osis: a correlative study of paper smear
r and direct immuno-fluorescence ”),
clin, Exp. Obst. Gyn. Volume 20 (4): Pages 259 to 26
6 (1993)).

The present invention, in all single tests, together with (otherwise) PAP smear selection can help identify these "early risk" diseases, low cost, fast and efficient Useful as a diagnostic tool and method. In particular, the present invention automates the analysis of normal and diseased cells that can be identified using a sensitive FISH assay and DIF staining to further enhance the presence of HPV and CT from PAP smears. Objective, cost effective, and quick identification.

I. Identification of Human Papillomavirus (HPV) One major purpose of the new cytology instrument is to reduce the false negative ratio (FNR) of sorted PAP smears. Of the various techniques presented to increase the rate of detection of cytologically undetected false negative cases, it has been found that simultaneous screening for HPV in epithelial cell DNA in addition to PAP smear is the most effective. It has been shown to be one of the most effective. When used together, they complement each other and represent a high detection rate.

[0092] The reason for this is that substantial evidence has been accumulating HPV associated with human upper genital disease, ie, cervical carcinoma. To date, over 60 HPV genotypes have been described, of which about 20 are associated with genital disorders.
These 20 species can be characterized as "high risk" and "low risk" according to their association with benign and malignant tumors. Shiadat-Pajou et a
l. ), Reference is made to the above, a FISH assay has been developed to detect HPV 16 and 18 in cervical smears, where these two sequences are almost exclusively malignant disorders. Be identified. The speed, specificity and accuracy of the identification of multiple HPV types can be greatly enhanced by the use of multiple fluorescent tag probes. While the traditional FISH assay is one relatively inexpensive method of HPV identification, a single wavelength test of multiple fluorescent signals with an equal number of wavelength filters (filters) is time consuming and bleaches the sample. At the risk of discarding the multi-wavelength data defining the illuminating light features.

The multispectral morphology system collects the entire spectrum and removes the convolution (
It is a tool that allows the use of multiple tagging of this kind by using a deconvolution algorithm to accurately distinguish between the various spectral signals that contribute to the observed fluorescence envelope.

Ii. Identification of Chlamydia trachomatis (CT) As mentioned above, DIF has been found to be a very effective technology among the technologies used to identify CT (Chlamydia Trachomatis). In particular, CT specific monoclonal antibodies conjugated to fluorescein isothiocyanate yield high specific fluorescein. This test is well suited for sampling and handling, and is consistent with that of PAP smear, as in the FISH assay.

Iii. Dual Stinning Method Conventional FISH and DIF assays require a high quality fluorescence microscope equipped with a special objective lens and multiple filters. Testing for these conditions makes these techniques unsuitable for most office settings. However, using the multispectral imaging system of the present invention, a single neck PAP smear glass slide that is double-colored with FISH labeling and DIF tinting,
By collecting all the wavelength data that defines these conditions, a diagnosis can be made automatically and simultaneously for both HPV sequencing and CT identification.

B. Tissue infected with tuberculosis Bacillus. The analysis of substances colored with fluorescent dyes such as rhodamine / auramine (exogenous fluorophora) is greatly improved in the application of the present invention. The following is a description of one such test.

It is known to infect tuberculosis (TB),
Tissue slides from any patient released from B were analyzed. Each slide was prepared using Auramine / Rhodamine stain in tissue specimens for observation under an Olympus BH2 external fluorescence microscope. The excitation wavelength was 510 nm and the spectrum was obtained through a long pass filter that passes all light with wavelengths greater than 540 nm and blocks all wavelengths shorter than 540 nm.

FIG. 4 shows a video image of a calibration smear known to be positive for TB coded CTB 11299sp3. "Observed" image cameras use false color enhancement to facilitate identification of each area in the smear by high intensity radiation. FIG.
Shows a spectral file obtained by spectrograph and with false color enhancement, as viewed on a computer monitor. Each acquisition has captured a portion of the smear as shown by the rectangular box in FIG. Each object marked 1 and 2 correlates with the spectral images of FIGS. 5 and 6 and the spectrum of FIG. The video camera of this system does not have any difficulty in observing the area of intense light for those samples infected with TB and the significant lack of light in each sample not infected with TB. . FIG. 6 shows a spectrum image divided into “channels”. Each channel is the spectrum of a region in a portion. To simplify presenting this concept, only 25 channels are shown to represent an 8 × 2.5 μm smeared area.

[0099] Under normal operating conditions, there are up to 240 channels, each corresponding to one row of pixels. At this resolution, each of the 240 objects is 1 μm × 2.5 μm when the magnification of the objective lens is 20 ×, and all are obtained simultaneously. Segmentation is set up automatically by the operator or by computer. If the filter set is in the system and blocks wavelengths below a certain value, it is normal to establish each channel to remove most of the missing wavelength data. Each spectrum corresponds to a particular location on the sample smear that is correlated to the channel number. The Y-axis provides a signal strength that can be calibrated to quantify the signal present for the object emitting the signal. Each band in FIG. 6 points to the same spectral location that can be colored back to the first observed image. Each of these spectra has a T
B and was identified using USNN as described in the following section.

In most fluorescence inspections, the fluorescence signal is composed of natural fluorescence as well as “tags”. The signal intensity level of smears positive for TB was found to greatly exceed the signal from natural fluorescence (also referred to as self-fluorescence). The system produces a large number of spectra simultaneously, many of which can be different from their neighbors and can show different conditions and conditions at particular locations.

To illustrate how the USNN concept works on TB identification,
USNN was calibrated using the calibration smear ctb11299sp3 and the smear-presented TB as shown earlier in FIGS.
Recognized the spectral characteristics of Next, each image whose spectrum was observed was designated as tb1
It was coded as 2110sp2 and was captured from a smear known to be positive for TB. The same channel selection was used as for calibration acquisition.

FIG. 8 shows a USNN representation of the analysis of the calibration smear ctb11299sp3 and the encoded patient sample tb12110sp2. On the top left, the first acquisition is a tuning set, indicating that there were nine identified spectral features. If the threshold (sensitivity) of the USNN were adjusted, less spectra would be located only by looking for overall differences, or more spectra by looking for fine differences.

The first column shows the channel numbers and the second column shows the spectral objects associated with a particular channel. The third and fourth columns show the search for the recognized spectral object at tb12110sp2. The pathologist confirmed that the area emission target 1 was clearly that of TB. The fourth column shows that there are 12 reproductions of the objects 1, 3 of the object 8 and that there are several spectra leading to the object 8. NMO. Numbers such as 0607823 indicate percent similarity to the indicated object. Given the signal strength of the object 8, this is probably a small amount of natural fluorescent emission that is expected to be present. Spectral intensity and image data provide sufficient information for quantification and digitization.

While the basic principles and exemplary embodiments of the present invention have been described above, it will be clear that further variations, changes, modifications and improvements will occur to those skilled in the art.
For example, it will be appreciated that the system can be designed with modified optics that capture a wider or different range of wavelength spectra as compared to the identified wavelength spectrum. Further, it will be appreciated that the invention is not limited to the use of a microscope. The spectrographic subsystem uses any type of optical image transmitter capable of transmitting imaged mammalian cells and tissues, such as any lens-based telescope system or fiber optic based imaging system such as an endoscope. Can work. Further, the substance is not limited to the prepared slide glass. For example, the system could automatically analyze the spectral properties of cervical vaginal tissue during a gynecological examination by connecting the spectroscopy subsystem and a computer, for example, to a vaginal magnifier. Finally, the system is not limited to use as a primary diagnostic tool. This system can be used as a tool to improve quality control in pathological experiments. For example, if the cell technician observes a sample from a special patient with abnormal or atypical cellular characteristics during the selection of PAP smear, the previous sample was reported as "negative" All previous screened samples should be reviewed by a cytopathologist to determine that no "false negative" reports were reported. In addition, in certain experiments, a percentage of "negative" samples can be presented for re-sorting by an older cytopathologist. The system of the present invention can be used to re-sort such samples, thereby increasing the objective validity of the analysis (by reducing the dependence of human judgment), and Increase quality assurance by increasing speed and reducing the cost of the sort / resort process. Accordingly, the foregoing description is by way of example only, and the invention is limited and defined only by the various claims and equivalents thereof.

[Brief description of the drawings]

FIG. 1A is a basic schematic diagram illustrating an automated multispectral morphological topography system of the present invention in which light passes through a mammalian material such as a tissue or cell.

1B is a schematic diagram illustrating a variation of the image transmitter of FIG. 1A, wherein light incident on the material is reflected by the material or absorbed by the material re-emits (fluoresces) lower energy light. is there.

FIG. 2 is a schematic diagram of a more detailed embodiment of the present invention where the material is prepared on a glass slide and the image transmitter is a fluorescence microscope.

FIG. 2A is a schematic diagram showing the substance prepared on the slide glass shown in FIG. 2 in more detail.

FIG. 3 is a flowchart illustrating one preferred method of the present invention.

FIG. 4 shows an observed video image of a fragment of an auramine / rhodamine colored fluorescent TB-positive spectral sample in a portion of a fragment surrounded by a rectangle.

FIG. 5 is a diagram showing a pseudo color expression of a spectrum present in a part shown in FIG. 4;

6 shows a more accurate version of the spectral image of FIG. 5 after delineation using an unsupervised neural network.

FIG. 7 is a graphical representation of 25 spectra representing the spectral fingerprints of the 25 channels of the image fragment shown in FIGS. 5 and 6;

FIG. 8 is a chart showing the unsupervised neural network representation of the analysis of the calibration smears shown in FIGS. 4 to 7 in the left two columns, and the analysis of the patient sample in the right two columns.

FIG. 9 is a graph showing a spectral curve of a sample that emits self-fluorescence, one of which is a healthy aortic wall and the other is a degenerated aortic wall.

FIG. 10 is a graph showing spectral curves of an original diseased kidney and a transplanted kidney.

FIG. 11 is a graph showing spectral curves of three portions of an immunofluorescent kidney sample, each stained for IgA, IgG, and IgM.

FIG. 12 is a graph showing a spectral curve for each fragment of two cardiac biopsies where no rejection was recorded for any fragment.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G01N 21/64 G01N 21/64 F 33/483 33/483 C G06T 1/00295 G06T 1/00295 ( 81) Designated countries EP (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, IT, LU, MC, NL, PT, SE), OA (BF, BJ, CF, CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG), AP (GH, GM, KE, LS, MW, SD, SL, SZ, UG, ZW) , EA (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), AL, AM, AT, AU, AZ, BA, BB, BG, BR, BY, CA, CH CN, CU, CZ, DE, DK, EE, ES, FI, GB, GD, GE, GH, GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ , LC, LK, LR, LS, LT, LU, LV, MD, MG, MK, MN, MW, MX, NO, NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, SL, TJ, TM, TR, TT, UA, UG, UZ, VN, YU, ZWF terms (reference) 2G020 AA03 AA04 CA01 CA02 CB04 CC13 CC26 CC42 CC47 CC63 CD03 CD12 CD14 CD24 CD27 CD33 CD52 2G043 AA03 BA16 DA02 EA01 FA02 HA02 HA09 JA02 JA05 KA01 KA02 LA03 LA04 NA05 2G045 CB01 CB21 FA16 FB03 FB12 GC15 2G059 AA05 BB12 CC16 DD03 DD13 EE02 EE07 EE12 FF03 HH01 HH02 JJ02 JJ06 JJ22 KK04 KK06 DC05 DA05 DB05 DA05 5B057

Claims (40)

[Claims] 1. A multi-spectral topography system for automatically assessing mammalian material for evidence of disease, comprising: an image transmitter having an optical output and a light source illuminating the material, wherein the fragment comprises a fragment of the material. A transmitter adapted to transmit the image of the optical output to the optical output; and a multispectral imaging spectroscopy subsystem coupled to the optical output, the subsystem converting the transmitted image to a number of components. A subsystem adapted to spectrally disperse spectral wavelengths substantially simultaneously to produce a spectral image; and a processor for processing the spectral image to provide diagnostic data representative of the image. The multispectral fine-configuration system according to any one of the preceding claims. 2. A multi-spectral topography system for automatically assessing mammalian material for evidence of disease, comprising: an image transmitter having an optical output and a light source illuminating said material, said fragment comprising: And a multi-spectral imaging spectroscopy subsystem coupled to the optical output, the subsystem adapted to transmit the image of the optical output to the optical output, the subsystem comprising: An image spectrograph having an entrance slit that allows the passage of light from a portion of the transmitted image, and a spectrally dispersing prism and mirror arrangement for dispersing the light passing through the entrance slit into a number of component wavelengths in a predetermined spectral range. And a first charge coupled device (C) coupled to the spectrograph for obtaining and preparing the spectral image. D: charge-coupled-device)
Wherein the system further comprises: a computer subsystem comprising a data processor for processing the prepared spectral image and diagnostic data representing a portion of the image. system. 3. The system according to claim 1, wherein the substance to be evaluated is in a living body. 4. The system according to claim 1, wherein the image transmitter is a lens-based image magnifying system. 5. The system of claim 4, wherein the material comprises a prepared pathology sample on a glass slide, and the image transmitter is a microscope that transmits an image of an enlarged fragment of the sample to the optical output. The above-mentioned system, characterized in that: 6. The system of claim 5, wherein the microscope is capable of continuously moving the sample, and wherein the entrance slit of the spectrograph is configured to pass light from an adjacent portion of the sample. The system described above comprising an xy stage adapted to be acceptable. 7. The system of claim 6, wherein the xy stage is controlled automatically by the computer. 8. The system according to claim 2, further comprising an image director disposed between said image transmitter and said spectroscopy subsystem, said optical system including an entrance slit of said spectrograph and an optical director. A first output for communicating,
The system comprising: a second CCD camera for capturing an observed image of the sample fragment; and a second output in optical communication. 9. The system of claim 8, wherein the image director is a beam splitter that selectively directs an image of the sample to the spectrograph or the second CCD camera. . 10. The system of claim 8, wherein said image director is a beam splitter cube for directing an image of said sample to said spectrograph and said second CCD camera simultaneously. . 11. The system according to claim 5, wherein said sample is a tissue biopsy. 12. The system according to claim 5, wherein the sample is a cell smear. 13. The system according to claim 5, wherein the optical output of the microscope comprises a standard camera interface, and the image spectrograph is connected to the interface. 14. The system according to claim 2, wherein said data processor comprises an unsupervised neural network. 15. The system of claim 2, wherein said data processor comprises a supervised neural network. 16. The system according to claim 2, wherein said data processor comprises an unsupervised neural network element and a monitored neural network element. 17. A multispectral topography system for automatically assessing mammalian material for evidence of disease, comprising: irradiating the material; and transmitting an image of a fragment of the material to an optical output. Means for simultaneously spectrally dispersing the image into a number of component wavelengths in a predetermined spectral range; obtaining the dispersed image from the dispersing means; and means for preparing the obtained image; and Means for processing using a neural network to provide a diagnosis of said substance. 18. A method of spectrally analyzing a substance for the presence of a disease based on a spectral analysis of tissue morphology and physiological bias of a mammalian substance from an average, comprising: irradiating the substance with a light source; Transmitting an image of the substance to a multispectral image spectrograph; spectrally dispersing the transmitted image through a prism and mirror arrangement to a number of component wavelengths in a predetermined spectral range; Obtaining said spectrally dispersed image of wavelengths and preparing the same; and processing the prepared spectrally dispersed image to provide a diagnosis. . 19. The method of claim 18, further comprising the step of providing a visual representation of the observed image obtained at an optical output. 20. The method of claim 19, further comprising the step of providing a visual representation of the processed spectrally dispersed image. 21. The method of claim 18, wherein said processing comprises implementing a neural network. 22. A method of spectrally analyzing a pathological sample for the presence of disease based on spectral analysis of biological and functional bias of a pathological sample from an average, comprising: an enlarged image of a fragment of the sample. Transmitting light to an optical output; and enabling light from a portion of the image to be transmitted through an entrance slit, wherein the portion of the image comprises a number of objects; Spectrally dispersing the transmitted image of the portion through a prism and mirror arrangement into multiple component wavelengths of a predetermined spectral range; and obtaining the spectrally dispersed image of the multiple component wavelengths for processing. Preparing, using a neural network, processing the spectrally dispersed image to convert a portion of the image to a portion of the image It said method characterized by comprising the steps of classifying into one of several categories preset showing a pathological condition, the. 23. The method of claim 22, further comprising providing a first visual representation including the spectrally dispersed image and a second visual representation of the observed image. The above method. 24. The method according to claim 22 or 23, wherein: (a) linearly moving the sample along an axis so that light of a part of the image adjacent to a part of the image transmitted in advance is transmitted to the entrance slit. (B) spectrally dispersing the transmitted light in adjacent portions of the image into a number of component wavelengths in a predetermined spectral range; and (c) adjoining the image. Obtaining and preparing a portion of the spectrally dispersed light to be obtained; and (d) processing the prepared spectrally dispersed light of an adjacent portion of the image using a neural network to obtain an image of the image. Classifying each object in an adjacent portion into one of a predetermined number of categories indicative of the state of each object. 25. The method of claim 24, wherein steps (a), (b), (c) and (d) are repeated until a desired number of portions of the image of the sample have been processed. Providing a medical diagnosis from now on. 26. A multispectral sensitive microstructure system for automatically assessing the fluorescent properties of a mammalian substance for evidence of disease, wherein the optical output and the substance are illuminated and absorbed by the substance An image transmitter having a source of filtered light, the image transmitter adapted to transmit an image of the fluorescent light re-emitted from the piece of material to the optical output; and A multi-spectral sensitive imaging spectroscopy subsystem, wherein the transmitted image is spectrally dispersed substantially simultaneously into a number of component wavelengths to generate a spectral image; and And a processor adapted to provide diagnostic data representative of the image. Responsive microstructure system. 27. A multispectral sensitive microstructure system for automatically assessing the fluorescent properties of a mammalian material prepared as a pathological sample for evidence of disease, comprising: an optical output and irradiating said material; A fluorescence microscope having a source of filtered light absorbed by the material, wherein the fluorescence microscope is adapted to transmit an image of the fluorescence re-emitted from the sample fragment to the optical output. A multispectral sensitive image spectroscopy subsystem connected to the optical output;
An entrance slit that allows light from a portion of the transmitted image of the sample fragment to pass therethrough, and passes the light passing through the entrance slit to a number of component wavelengths in a predetermined spectral range. And an image spectrograph having a prism and mirror arrangement for spectrum dispersion so as to generate a spectrum image by dispersing the spectrum image in combination with the spectrograph to obtain the spectrum image and convert the image into digital data. And a first charge-coupled device (CCD: charge) for operating the data
e-coupled-device), the system further comprising: a neural network processor that processes the manipulated data representing the spectral image and provides diagnostic data representing a pathological condition of a portion of the image. The above system, comprising a computer subsystem. 28. The system of claim 27, wherein the sample is an endogenous fluorescent material. 29. The system of claim 28, wherein the sample comprises a self-fluorescent compound such as elastin or collagen. 30. The system of claim 28, wherein the sample is prepared using immunofluorescent staining techniques, including fluorescent antisera to immunoglobulins. 31. The system of claim 27, wherein said sample is treated with at least one fluorescent dye. 32. The system of claim 31, wherein the sample is processed using a fluorescent dye-labeled compositional chemical probe. 33. The system of claim 32, wherein fluorescence in situ hybridization is performed on the sample, and wherein the processor is configured to detect genetic disease, malignancy (tumor),
The system described above, wherein the system provides data indicative of the presence of at least one strain of one of bacteria and viruses. 34. The system of claim 32, wherein the sample comprises one of an antibody and a polyclonal antibody that are cells derived from a single cell conjugated to a fluorescent tag. The above system, wherein the system is prepared using an immunochemical chemistry stain having a marker. 35. The system of claim 34, wherein said sample is colored by DIF analysis and said processor is a Chlamydia trachomatis (C).
T. The system as described above, wherein the system provides data indicative of the presence of chlamydia trachomatis. 36. The system according to claim 32, wherein the sample is FISH.
Colored by both analysis and DIF analysis, and the processor is HPV and CIF
The system described above, wherein the system provides data indicative of the presence of both T. 37. A method of analyzing the fluorescent properties of a mammalian substance against the presence of a disease based on spectral analysis of tissue morphology and physiological bias of the mammalian substance from the average, comprising: Illuminating a substance so that the substance emits fluorescence; transmitting an image of the fragment of the substance emitting the fluorescence to an optical output; Spectrally dispersing the plurality of component wavelengths in a range; obtaining and preparing the spectrally dispersed image of the multiple component wavelengths; andusing the prepared spectrally dispersed image with a processor. Processing to classify the portion of the image into one of a predetermined number of categories indicating a state of the portion of the image. . 38. The method of claim 37, wherein preparing the spectrally dispersed image comprises digitizing the image by digital signal processing and manipulating the digitized image. The above method. 39. The method of claim 37, wherein: (a) directing the material along an axis such that light of a portion of the image adjacent to a portion of the previously transmitted image is transmitted through the entrance slit. (B) spectrally dispersing the transmitted light of adjacent portions of the image into a number of component wavelengths in a predetermined spectral range; and (c) adjacent portions of the image. Obtaining and preparing the spectrally dispersed light of (a), and (d) processing the prepared spectrally dispersed light of an adjacent portion of the image using a neural network to generate an adjacent portion of the image. Classifying each subject of the portion into one of a predetermined number of categories indicative of the status of each subject. 40. The method of claim 39, wherein steps (a), (b), (c) and (d) are repeated until a desired number of portions of the image of the substance have been processed. Providing a medical diagnosis from now on.