NL2008873C2

NL2008873C2 - Method and apparatus for multiple points of view three-dimensional microscopy.

Info

Publication number: NL2008873C2
Application number: NL2008873A
Authority: NL
Inventors: Pierre Mangeol; Erwin Peterman
Original assignee: Stichting Vu Vumc
Priority date: 2012-05-24
Filing date: 2012-05-24
Publication date: 2013-11-26
Also published as: WO2013176549A1

Description

Method and apparatus for multiple points of view three-dimensional microscopy

Field of the invention

The present invention relates to three-dimensional optical microscopy, 5 and particularly to a method and apparatus for three-dimensional optical microscopy to obtain views under different angles of the same object with high level of resolution.

Background of the invention

Optical microscopy is the tool of choice in medical and biological 10 science. Cell-biology pioneers used straightforward, bright-field microscopy to distinguish cells. Their successors applied more advanced methods such as phase-contrast and differential-interference-contrast microscopy to reveal substructures in cells. Over the last twenty years cell biology has again been revolutionized by fluorescence microscopy in combination with 15 autofluorescent proteins and immunolabeling, which allow localization of biomolecules in cells with high specificity. Localization in three dimensions is achieved by confocal fluorescence microscopy, using one- or two-photon excitation, light sheet based fluorescence microcopy, computational deconvolution, three-dimensional stochastic optical reconstruction microscopy, 20 stimulated emission depletion microscopy and three-dimensional structured illumination microscopy.

In confocal fluorescence microscopy, a focused laser excites a fluorescent sample while fluorescent signal is detected through a pinhole, focused on the same position as the laser, with a sensitive detector. Used in 25 laser-scanning mode, the focused laser beam is scanned point by point through the sample and fluorescence is detected through a pinhole with a photomultiplier or an avalanche photodiode; a two-dimensional image is obtained after a plane of interest has been fully scanned and a three-dimensional volume reconstructed after scanning several of these images.

30 Used in spinning-disk mode, an array of lenses focuses the excitation laser simultaneously in thousands of confocal spots while fluorescence is detected through an array of pinholes focused on these spots with a sensitive camera; a single plane is obtained after rotation of the lens and pinhole arrays and three-dimensional volumes are reconstructed by combining planes obtained 2 with different focus settings. Confocal microscopy suffers from several drawbacks. First, due to the pinholes, which are essential for suppressing out-of-focus light and enabling sectioning, a substantial amount of fluorescence light is discarded and consequently longer exposure times are required.

5 Second, the depth resolution is at least twofold worse than the lateral resolution. Third, the reconstruction of a three-dimensional volume requires scanning through the sample, which is inherently slow.

With light-sheet based fluorescence microcopy, only a single plane in the sample is excited, by illumination with a thin sheet of light from the 10 direction perpendicular to the optical axis of detection. The key advantage of this technique is that the sample is only excited close to the detection plane, resulting in less background signals and photo-bleaching due to illumination of out-of-focus regions. The spatial resolution of this technique is identical to traditional microscopy, but its sectioning capabilities enable it to reconstruct 15 three-dimensional volume by combining a stack of planes recorded. Again, because it requires scanning through the sample, light-sheet based fluorescence microcopy is inherently slow. Key limitation is that two objectives need to be used perpendicular, with crossing focal planes. This is not compatible with all samples and difficult with high-resolution, short-working 20 distance objectives.

Deconvolution microscopy is a combination of wide-field microscopy with computational deconvolution. Based on the knowledge of the point spread function of the microscope, it extracts information from images taken at different depth to reconstruct the distribution of fluorescent molecules.

25 Results of computational deconvolution depend largely on the sample and on the signal quality, but can surpass confocal microscopy capabilities in some cases. Limitations are that it is an image-processing based method, which requires prior knowledge and can result in artifacts.

Three-dimensional stochastic optical reconstruction microscopy (3D-30 STORM), stimulated emission depletion microscopy (STED) and three-dimensional structured illumination microscopy (3D-SIM) are three superresolution techniques enabling to overpass the limit of resolution of conventional microscopy.

3 3D-ST0RM is based on wide-field microscopy used with an astigmatic lens and photo-switchable dyes. The basic step in 3D-STORM consists in switching a few fluorophores at a time and to report precisely their three-dimensional position by fitting. A three dimensional volume is reconstructed 5 from repeating this step hundreds of time. The gain of resolution is substantial, but the volume reported is limited to a few hundreds of nanometer in depth and acquisition of such a volume takes several minutes.

STED is a scanning confocal microscopy technique based: a confocal volume is excited by a first laser while a second laser, used in a doughnut 10 mode, causes stimulated emission of most of the volume except for a small, non diffraction-limited volume. The fluorescence of the molecule in this volume is then detected just like in confocal microscopy. Although the gain of resolution is substantial, the reconstruction of a three-dimensional volume takes at least several minutes for a single cell.

15 3D-SIM, a development of structured illumination microscopy, consists in reconstituting a two-dimensional image featuring enhanced lateral and axial resolution from several images taken with different illumination structures. A three-dimensional volume is obtained by scanning through the sample. This technique offers double the lateral resolution of a conventional microscope, 20 but its axial resolution is a bit smaller than the one of the invention. Moreover, the fluorescent samples have to stay static while the different illumination structures are projected. Overall this technique is rather slow and cannot by construction access fast features of living organisms .

The methods discussed above are well suited to describe “life at rest” 25 or dynamics occurring at a timescale of about ten of seconds or more with fluorescence microscopy. Faster dynamics, on the second or sub-second timescale are very difficult to follow quantitatively since cells are three-dimensional objects, and scanning the sample in three dimensions is inherently slow and results in poor axial resolution. Except with scanning 30 through the sample at high rate, these techniques only provide access to motion in the focal plane and the axial component is lacking. High rate scanning is not a solution either, since it leads to an important loss of sensitivity. In studies of processes in single cells or cells in living, multicellular 4 organisms, this issue makes quantitative imaging of dynamics on the subsecond scale hardly possible.

The present invention differs and has advantages compared to all previously known three-dimensional microscopy. A major difference to the 5 invention is that, to retrieve three-dimensional information on specimen thicker than typically one micrometer, previously known three-dimensional microscopy require scanning through the sample, which is slow; the invention can provide three-dimensional information without scanning and is only limited by the frame rate of the camera used. A second important difference is that 10 the invention provides an axial resolution about two times higher than conventional microscopy, whereas, except for computational deconvolution and very slow super-resolution methods, the axial resolution of these techniques is comparable to conventional microscopy. A third difference is that the present invention provides access to three-dimensional information 15 not only in fluorescence microscopy, but also in dark-field microscopy, phase-contrast microscopy and differential-interference-contrast microscopy; this feature is unattainable by any kind of the known microscopy technique.

Thus there is a need for a method and apparatus for three-dimensional optical microscopy which provides live three-dimensional information, which 20 has an enhanced axial resolution and which is not limited to fluorescence microscopy, but provides live three-dimensional information in bright field microscopy, dark-field microscopy, phase-contrast microscopy and differential-interference-contrast microscopy.

The present invention satisfies these needs, as well as others, and 25 generally overcomes the deficiencies found in known optical microscopy devices and methods.

An object of the invention is to provide a method and apparatus for three-dimensional microscopy which provides live three-dimensional information and live three-dimensional recording.

30 Another object of the invention is to provide a method and apparatus for three-dimensional microscopy which provides an enhanced axial resolution.

Another object of the invention is to provide a method and apparatus for three-dimensional microscopy which allows observation of an object under 5 any desired angle with bright-field microscopy, dark-field microscopy, phase-contrast microscopy and differential-interference-contrast microscopy.

Another object of the invention is to provide a method and apparatus for three-dimensional microscopy which allows the three-dimensional 5 reconstruction of an object with enhanced resolution.

Another object of the invention is to provide a method and apparatus for three-dimensional microscopy which provides confocal microscopy with enhanced resolution.

Summary of the Invention 10 The present invention generally pertains to a method and apparatus for three dimensional optical microscopy which employs an objective and a reflector about a sample and one or two additional objectives to achieve far focusing.

The present invention generally pertains a method and apparatus for 15 three dimensional optical microscopy which employs an objective and a reflector about a sample and one or two additional objectives to achieve far focusing.

There are four preferred embodiments of the invention which, employing essentially the same apparatus, allow imaging a sample with two or 20 more points of view thanks to one or more reflectors about the sample and far focusing achieved by one or two additional objectives.

By way of example and not of limitation, the present invention generally includes an objective lens and a reflector which are mounted about a sample, the reflector being preferably in the form of a mirror or a faceted mirror, with 25 the reflector mount including translation and rotation means. For far focusing means, the invention generally includes one or two extra objective lenses combined with one mirror and one quarter-wave plate each, objectives and mirrors being preferably mounted with translation and/or rotation means. Illumination means, preferably in the form of one or more LED lamps or other 30 extended spatially incoherent light source and one or more lasers, provides illumination for the sample.

The invention preferably generally includes beam splitter and combiner means, preferably in the form of beam splitter-recombiner cubes, for splitting and combining the light coming from the sample for recording. A plurality of 6 adjustable mirrors allows the direction of illuminating and/or observed light to and from the objective lenses and image recording means. The image recording means preferably includes a CCD or EMCCD camera. Means for selectively transmitting and reflecting light of different wavelengths, preferably 5 in the form of dichroic mirrors and/or optical filters, are generally included in the invention. Optical path length adjustments means preferably in the form of a translating stage with one or more suitably positioned mirrors, allows tuning of the optical path lengths. Phase and/or amplitude alteration means, preferably in the form of a spatial light modulator or phase plates, may be 10 included for alteration of the phase and/or amplitude of the illumination and observed or emitted light. Alignment means for positioning the sample relative to the objectives lenses are provided, which preferably includes a translational stage or a translational-rotational stage, an objective and/or lenses combined with a CCD camera including translation means. The invention may employ 15 vibration and/or light isolation supporting means such as vibration isolated platform or housing.

In a first embodiment of the present invention, the reflector, which is mounted about the sample, creates one or more mirror images of the sample. Illuminating light is generally directed to the sample from the objective lens or 20 from the side of the sample. The objective lens is used to image the volume of the sample and the images of this volume. The light coming from the objective is split and then projected by the far focusing objectives. A plane of observation is selected in these far focusing objectives by the mean of a mirror. The images selected by the far focusing objective lenses are spatially 25 filtered and then combined on a CCD camera or other imaging means. This first embodiment applies primarily to fluorescence, phosphorescence and bright-field microscopy.

In a second embodiment of the invention, the first embodiment is combined with Fourier optics to generate dark-field, phase-contrast and 30 differential-interference-contrast multiple points of view microscopy. To this mean, the images created by the far focusing objective lenses are combined and filtered in amplitude and/or phase with a spatial light modulator. The Fourier conjugated planes of these images can be used to apply this filtering. Different amplitudes and/or phases are used to achieve dark-field, phase- 7 contrast or differential-interference-contrast multiple points of view microscopy by finally conjugating the images on a CCD camera or other imaging means.

In a third embodiment of the invention, which applies primarily to fluorescence and phosphorescence microscopy, the first embodiment is 5 combined with computational means to reconstruct a three-dimensional image of the sample. For this mean, the volume is scanned by one or more of the far focusing objectives, and volumes observed under different angles are recorded. The three-dimensional reconstruction consists in combining numerically the information in these volumes. Reconstruction is achieved by 10 comparing the volumes in the real space, such as, but not limited to, using the minimum of intensity in each position of the three-dimensional space, or by combining the information of each volume in the Fourier space and then inverting this information, to create a reconstruction of the volume with sectioning and increased resolution.

15 In a fourth embodiment of the invention, the first embodiment is combined with confocal excitation and detection to reconstruct a three-dimensional image of the sample. The excitation is provided by focusing light through the objective close to the sample while detection uses the mirror image of the sample, enabling detection with an angle to the excitation axis.

20 Detection is achieved by recovering the fluorescence signal by one of the far focusing objectives, selecting the mirror image of the excitation spot created by the mirror close to the sample, and imaging it on a pinhole before it is detected by an avalanche photodiode or other detection means.

8

Further objects and advantages of the invention will be brought out in the following portions of the specification, wherein the detailed description is the purpose of fully disclosing preferred embodiments of the invention without placing limitations thereon.

5 Brief description of the drawings

The invention will be more fully understood by reference to the following drawings, which are for illustrative purpose only: FIG. 1 is a schematic diagram of a first embodiment of an optical microscope in accordance to the present invention. A* refers to the Sample, 10 B * to the Objective Lens.

FIG.2 is a schematic diagram of the first embodiment of an optical microscope including an illumination mean. Reference number (30) refers to the Light Source.

FIG.3 is a schematic diagram of the first embodiment of an optical 15 microscope including another illumination mean; C * refers to Reflective Facet.

FIG.4 is a schematic diagram of a reflector mean to be used in the sample plane of the first embodiment of an optical microscope.

FIG.5 is a schematic diagram of another reflector mean to be used in the sample plane of the first embodiment of an optical microscope.

20 FIG.6 is a schematic diagram of another reflector mean to be used in the sample plane of the first embodiment of an optical microscope. Reference number (82) refers to the Image Detector.

FIG.7 is a schematic diagram of a hybrid microfluidic reflector device to be used in the sample plane of the first embodiment of an optical microscope. 25 FIG.8 is a schematic diagram of visualization mean to be used in an optical microscope in accordance to the present invention. D* refers to the Spatial Light Modulator.

FIG.9 is a schematic diagram of the first embodiment of an optical microscope with additional flipping mirrors to provide traditional wide-field and 30 epi-fluorescence microscopy.

FIG. 10 is a schematic diagram of a second embodiment of an optical microscope in accordance to the present invention.

FIG. 11 is a flow diagram showing the general steps of a third embodiment of an optical microscope in accordance to the present invention.

9 FIG. 12 is a graphic representation of the object and its mirror image in the sample plane 11a, where only a single plane reflector 10 is used.

FIG. 13 is a flow diagram showing the general steps of a method based on volume comparison to reconstruct a three dimensional volume from a data 5 set combining image recordings of a volume under several directions.

FIG. 14 is a simplified graphic representation of the iso-intensity (half maximum intensity, in x-z plane, where z is the optical axis of the main objective and x, any axis in the observation plane) of the light emitted by a point source and detected by in a conventional fluorescence microscope.

10 FIG. 15 is a simplified graphic representation of the iso-intensity (half maximum intensity, in x-z plane, where z is the optical axis of the main objective and x, any axis in the observation plane) of the light emitted by a point source, detected and processed by using the apparatus and method of the third embodiment. Reference numbers (174) refer to the Light Source; 15 (175) to the Detector.

FIG. 16 is a flow diagram showing the general steps of a method based on three-dimensional Fourier information to reconstruct a three dimensional volume from a data set combining image recordings of a volume under several directions.

20 FIG. 17 is a schematic diagram of a fourth embodiment of an optical microscope in accordance to the present invention.

FIG. 18 is a simplified graphic representation of the excitation light in a conventional confocal microscope. F* refers to Coverslip Surface, G* to Confocal Spot.

25 FIG. 19 is a simplified graphic representation of the excitation light reflected on the reflector 10 used in the apparatus 170. G* refers to Confocal Spot, H* to Reflector Surface, and I* to Coverslip Surface.

FIG.20 is a simplified graphic representation of the detection method to be used with the apparatus 170. J* refers to Confocal Detection, K* to Image 30 of Confocal Emission Spot; L* to Image of Reflector Surface; M* to Coverslip Surface, and N*to Confocal Emission Spot.

10

Detailed Description of the Invention

Referring more specifically to the drawings, for illustrative purposes, the method and apparatus comprising the present invention and the underlying theory behind the invention are generally shown in FIG 1 through FIG 20.

5 It will be appreciated that the apparatus of the invention may vary as to configuration and as to details of the parts and that the method of the invention may vary as to the steps and their sequence, without departing from the basic concepts as disclosed herein.

FIG. 1 discloses a simplified schematic diagram of the microscope 10 apparatus 1 in accordance with a first embodiment of the present invention:

An objective 16 and a reflector 10 are mounted about a sample 11a. The reflector 10 may be of different forms and is discussed below. The objective 16 is focused on the sample while the reflector 10 is brought close to the sample and reflects the object in the sample. The sample 11a is preferably 15 mounted on a coverglass or in a microfluidics chamber including a reflector (discussed below). Generally, illuminating light for illuminating means is directed to the sample 11a from the side of the sample, partially reflected on reflector 10 or through the objective lens 16 using a beam splitter, which may or may not be dichroic, as will be discussed below. The observed light or 20 images from objective 16 is reflected by a mirror 12a, and then split by a beam splitter, preferably a polarizing cube 14a. The observed light or images from objective 16 are then directed to the objective lenses 17a and 17b for far focusing purpose along paths 19. Along path 19a, light goes through cube 14b and a quarter-wave plate 15a and enters objective lens 17a. Lenses 13a 25 and 13b are placed to ensure that the back focal plane of objective 16 is conjugated with the back focal plane of objective 17a. The translation and rotation of mirror 12b enables to select a plane of observation in the sample 11 b duplicated from the sample 11a. When the light exits the objective 17a, it goes through the quarter-wave plate 15a for a second time. The quarter-wave 30 plate 15a is oriented, so that the polarization of the light passed twice through it undergoes a 90 degrees rotation. The light is therefore reflected by polarizing cube 14b. The lens 13c creates an image of the plane selected with objective 17a and mirror 12b on the plane 18a, where a spatial filter, preferably a razor blade or a mechanical slit, is preferably positioned. The light 11 is then reflected by mirrors 12c, 12d, 12e, 12f and cubes 14a and 14b to be imaged on image detector 20. Lenses 13d and 13g are placed to ensure that the spatial filter 18a is conjugated with the image detector 20 plane.

Preferably, image detection means 20 is a CCD camera or the like, and it may 5 or may not be preceded by dichroic or filtering means (not shown) especially when used in fluorescence or phosphorescence microscopy. The light going along path 19b undergoes similar transformations as the light going along path 19a, with the difference that the polarization of path 19a is 90 degrees rotated compared to path 19b.

10 In the operation of the microscope apparatus 1, two different planes to observe in the sample 11a are selected by objective lenses 17 by positioning mirrors 12b and 12g. These planes are then imaged side by side on the image detector 20. In order to avoid image mixing on the image detection means 20, spatial filters on planes 18 are positioned to select the part of interest in the 15 image while mirrors 12f and 12j are placed and can be tilted in the back focal plane of lens 13g to translate the two images obtained from objective lenses 17a and 17b respectively. Fully flexible imaging of the sample 11a may be ensured by mounting it on a translation stage or a translation rotation stage (not shown), as well as mirror 10 may be mounted on a stage with translation 20 and rotation means (not shown).

Several arrangements of mirrors 12 and lenses 13 may be employed for the microscope apparatus 1, but they should satisfy the conjugation relations defined above.

FIG. 2 shows generally a possible way to illuminate the sample 11a.

25 Illuminating light for illuminating means can be directed to the sample 11a through the objective lens 16 using a beam splitter 23, which may or may not be dichroic. The light source 21, which may or may not be a spatially incoherent light source, may or may not be collimated by a lens 22 into the back focal plane of the objective lens 23.

30 FIG. 3 shows generally another possible way to illuminate the sample 11a. Illuminating light for illuminating means can be directed to the sample 11a from the side of the sample 11a. The light source 30, which may or may not be a spatially incoherent light source, may or may not be collimated by a lens 31 onto the sample 11a and the mirror 10.

12 FIG. 4 shows generally a type of reflector 40 applicable in the sample 11a. The reflector 40 consists of a plane mirror, preferably is a dielectric mirror, coated metallic mirror or the like. There is no limitation to the shape or size of the mirror 40.

5 FIG. 5 shows generally a type of reflector 50 applicable in the sample 11a. The reflector 50 is a beam divider, preferably glass substrate presenting a repeated structure, coated with a dielectric or a metallic layer. The length of repetition of the dihedral structure “a+b” is in the range from 1 micrometer to 5 millimeters, where “a” and “b” might or might not be equal. The angle □ 10 defining the orientation of the structure’s facets is in the range from 45 to 180 degrees. Although intended to be tiltable, reflector 50 is preferably oriented with the main dihedral axis perpendicular to the optical axis of the objective lens 16.

FIG. 6 shows generally a type of reflector 60 applicable in the sample 15 11a. The reflector 60 is a geometrical simplification of beam divider 50, where, all others geometrical and physical properties conserved, only one dihedral structure is present. Several arrangements of these structures are possible, differing in their number, orientation or geometry on the same reflector.

FIG. 7 shows generally the reflector 73 embedded in a microfluidic 20 apparatus 70. The reflector 73 is preferably one of the reflectors 50 and 60 presented above. The creation of channel 75 and the embedding of the reflector 73 are obtained by standard soft lithography with a polymer layer 74, preferably made of low refractive index optical adhesives or coating materials, or poly(dimethylsiloxane) (PDMS). The polymer layer 74, with the embedded 25 reflector 73, is coated on a coverslip 72. Tubing 71 ensure the flow circulation in the channel 75.

FIG. 8 shows generally a possible additional visualization apparatus 80 to the microscope apparatus 1. With the reflector 10 mounted with translation means, it can be translated to enable visualization from the top of the sample 30 11a. A lens 81, preferably a lens or an objective lens is collimated on sample 11a. The sample 11a is then imaged on the image detection means 82, which is preferably a CCD camera or the like. FIG. 8 shows the simplest configuration for visualization, but additional lenses or mirrors may be inserted. An ocular may also be used instead of image detection means 82. Lens 81 13 and image detection means 82 may be mounted on translational means to achieve proper conjugation with the sample.

FIG. 9 shows generally a possible apparatus variation 90 of the microscope apparatus 1. Mirrors 91 may be inserted in the path 93a to 5 shortcut the far focusing paths 19a and 19b, and to direct the light toward the image detection means 20. It enables the microscope to be used as in a traditional wide-field or epi-fluorescence microscope. The mirrors 91 are preferably mounted on translation or flipping means to ensure adequate operation between traditional and multiple points of view visualization. The 10 lens 92 is suitably positioned to image the sample on the image detection means 20. FIG. 9 shows the simplest configuration for traditional visualization, but additional lenses or mirrors may be inserted.

Referring to FIG. 10, a simplified schematic diagram of the microscope apparatus 100 in accordance with the second embodiment of the present 15 invention is generally shown. Path 19c of apparatus 1 is modified to combine multiple points of view microscopy with Fourier filtering optics. Lens 101a is positioned so that its back focal plane coincides with lens 13g front focal plane. The spatial light modulator (SLM) 103 is positioned in lens 101 a front focal plane. Lens 101b back focal plane is positioned on the SLM to image the two 20 different planes selected by objective lenses 17 side by side on the image detection means 104. Preferably, image detection means 104 is a CCD camera, EMCCD camera or the like.

In the operation of the microscope apparatus 100, different amplitude and/or phase can be applied on the SLM 103 to achieve the type of 25 microscopy intended (prior art). The selection of observed planes in the sample 11a is achieved as in the operation of the microscope apparatus 1. Therefore, the apparatus 100 enables dark-field microscopy, phase-contrast microscopy and differential-interference-contrast microscopy to be used and to observe the sample 11a under any angle with these types of microscopy.

30 Referring now to FIG. 11, a simplified flow chart relating the three- dimensional reconstruction method in accordance with the third embodiment of the present invention is generally shown. The following description takes into account the operation of both far focusing objective lenses 17. The operation can be simplified by using only one of these objectives, but it may 14 require longer acquisition time; the reconstruction part is however not changed.

At step 111, the objective lens 16 is focused about the sample. Focusing is preferably carried out by moving the objective lens, or the sample, 5 on precision translating means such as translating stages.

At step 112, far focusing objective lenses 17 are focused upon a section or plane within the sample. Focusing is preferably carried out by moving the objective lenses, or the sample, on precision translating means such as translating stages.

10 At step 113, the images observed by both far focusing objectives are directed to image detection means such as a CCD camera or the like for image recording.

At step 114, mirrors facing far focusing objectives are adjusted to enable imaging of a new set of volume sections. These adjustments are 15 intended to provide volume sections whom distance from sections obtained at step 112 is preferably half of the lateral spatial resolution of one image (i.e. typically 100 nm).

At step 115, steps 113 and 114 are repeated, until each section of the sample has been observed and recorded as related above. The recorded 20 images from each section of sample, under two or more orientations, form a data set for the entire sample, which is stored by a microprocessor interfaced to the image detection means.

At step 116, the data set obtained at step 115 is processed a first time. As shown on figure 12, the data set 120 obtained at step 115, consists of the 25 record of the sample duplicated by mirror transformations. The sample 121 is duplicated here, as a matter of example, in the mirror image 122. One or more than one of these mirror transformations are preferably intended to be used in the invention; the following explanations are limited to one mirror image, but it can be easily generalized to more than one of these mirror images. Each 30 mirror volume image is extracted from the data set, enabling independent transformations of these volumes in reduced data sets for the next steps.

At step 117, the sample and its mirror images are reoriented in the same base. As an example, following the schematic proposed on FIG. 12, the mirror transformation which gives the image 122 of the sample 121 has to be 15 inversed to combine the sample and its image. For this purpose, different options can be taken. A first option is to use a direct mirror transformation through a matrix operation addressing each voxel (three-dimensional pixel) in the mirror image obtained at step 115 to a new voxel or a new set of voxels in 5 the volume created by the inverse mirror transformation. The intensity value of each addressed voxel may be interpolated from neighbouring source voxels to enable adapted addressing. Another option is to transform the volume in the Fourier space: the volume is first Fourier transformed in three-dimension; it undergoes several transformations necessary to perform the desired final 10 transformation (combination of rotations, translations or planar symmetry can be considered). The advantage of this latter technique is that it enables subpixel transformation. In each case the transformation characteristics can be determined beforehand through calibration or by volume comparison.

At step 118, the sample and its mirror images are combined to create a 15 new volume image of the sample with enhanced resolution. Several methods can be used; as a matter of example, two are proposed.

The first method is generally presented in FIG. 13. As the sample and its mirror images have been reoriented in step 117, these volumes can be compared. At step 131, the average intensity of each volume is first equalized, 20 because of possible losses which might have happened while recording the data set. At step 132, the voxel at the same exact position in each volume is examined and at step 133 the corresponding intensities are compared. The voxel of the new volume image at this given position is assigned with the minimum of the intensities compared. At step 134, steps 132 and 133 are 25 repeated to assign the intensity value of each voxel in the new volume image. The result of imaging a single fluorescent point in a conventional microscope is shown on FIG. 14; the graph represents the iso-intensity of half maximum intensity in x-z plane, where z is the optical axis of the main objective and x, any axis in the observation plane. The result of imaging a single fluorescent 30 point with the method described in FIG. 13 is shown on FIG. 15; the reflector used is supposed to be the one of the reflector described in FIG. 5 or 6, where the angle pis 60 degrees; the graph (plain line) represents the iso-intensity of half the maximum intensity in the x-z plane, where z is the optical axis of the main objective and x define the x-z plane as perpendicular to the planes of the 16 reflectors. The image resulting from the method described in FIG. 13 has an enhanced axial resolution.

The second method is generally presented in FIG. 16. At step 161, each volume obtained at step 117 is first Fourier transformed. One obtains the 5 three-dimensional Fourier transform of each volume. At step 162, these Fourier transforms are combined in Wiener filter. At step 163, the result of step 162 is Fourier inversed to obtain the complete reconstructed volume.

The advantage of the following embodiment is that one could in principle go deeper in the sample.

10 Referring to FIG. 17, a simplified schematic diagram of the microscope apparatus 170 in accordance with the fourth embodiment of the present invention is generally shown. The apparatus 170 is a modification of the apparatus 1 enabling confocal excitation of the sample 11a and confocal detection. The light source 174, preferably a spatially coherent light source, is 15 focused by a lens 171 on a scanning device 172, preferably a scanning galvanometer mirror system. The light is then directed to the objective 16, as it is collimated by lens 173 and reflected on the dichroic mirror 177. The objective lens 16 converges the light on a confocal spot in the sample 11a. Scanning of the excitation confocal spot is achieved by the scanning device 20 172, but scanning can also be achieved by translating the sample attached to multi-axis scanning stage. Axial scanning is achieved by scanning the objective lens 16 in its axial direction. The emitted light from the sample 11a is partially reflected on reflector 10, collected by the objective 16 and then directed to the objective lens 17a as described in the first embodiment. A 25 section plane is then selected (discussion below). The light is reflected on the mirror 12b and is then directed towards the detector 175. To enable the light to be properly spatially filtered by the pinhole 179, the mirror 12b and the scanning mirror 176 are adequately positioned. The light is filtered by the emission filter 178 and is then detected by the detector 175.

30 FIG. 18 shows the apparatus 170 can be used to excite the sample as it is done in conventional confocal microscopy. FIG. 19 shows the apparatus 170 can also be used to excite the sample with an angle, by reflecting the excitation light with the reflector 10.

17

More importantly, the apparatus 170 enables detection in a novel way. While the sample 11a and its mirror images are conjugated in the objective lens 17a, the far focusing technique described in the first embodiment enables to detect the confocal emission in sample 11a under an angle, by selecting 5 the proper section plane in sample 11 b thanks to mirror 12b. The confocal emission spot is then imaged on the pinhole 179 to enable final confocal detection.

FIG. 20 gives a schematic of how the detection is achieved thanks to the combination of the reflector 10 used in the sample 11a, the far focusing 10 technique and the confocal detection. In the sample 11a, fluorescent emission is essentially reduced to a small region 201. This region has its mirror image 202 due to the reflector 10. These two regions are duplicated in the sample 11 b thanks to refocusing. The imaging plane is selected at the center of the image of the region 202 by positioning mirror 12b. By centering this region on 15 the pinhole 179 thanks to the scanning mirror 176, the emission is detected in a limited volume 203. Detection is therefore achieved with an arbitrary angle, preferably 90 degrees, compared to the excitation.

It will be seen that the present invention provides a method and apparatus for three-dimensional microscopy which provides live three-20 dimensional information, which has an enhanced axial resolution and which is not limited to fluorescence microscopy, but provides live three-dimensional information in bright field microscopy, dark-field microscopy, phase-contrast microscopy and differential-interference-contrast microscopy. Although the description above contains many specificities, these should not be construed 25 as limiting, but as merely providing illustrations of some of the presently preferred embodiments of this invention. Thus, the scope of the invention should be determined by the appended claims and their legal equivalents.

Claims

A three-dimensional optical microscope apparatus, comprising: a) a main objective lens and a reflector; b) means for supporting a microscope sample between the objective lens and the reflector; c) multiple means for splitting bundles and recombining light; D) at least one focusing objective lens; e) at least one focusing reflector facing the front lens of the focusing objective lens; f) at least one focussing trajectory, the focussing trajectory extending from the beam splitting and recombining means, to the focussing objective lens, and further extending from the focussing lens objective lens after reflection on the focusing reflector; g) a quarter-wavelength plate and beam splitting means arranged on the focusing path positioned in front of the focusing objective; h) a spatial filter for spatially filtering light from the focusing lens; i) image-forming means for detecting and recording images, wherein the image-forming means are positioned in such a way that they detect and record perceived light, the perceived light being split by the beam splitting and recombining means, far focused by the far -sharpening objective lens, filtered through the spatial filter, and combined by the beam splitting and recombining means; j) a plurality of means for directing light from the microscope sample to the beam splitting and recombining means, as well as along the focusing path and in the direction of the imaging means; k) a plurality of lenses to suitably conjugate the image of the microscope sample in the direction of the focusing objective lens, the spatial filter, and the imaging means.

Apparatus according to claim 1, further comprising means for visualizing the microscope sample 10

The device of claim 2, further comprising focusing and imaging means.

Apparatus as claimed in claim 3, further comprising means for positioning the focusing and imaging means.

The apparatus of claim 1, further comprising rotation means for rotating the quarter-wavelength plate.

The apparatus of claim 1, further comprising means for adjusting the position and orientation of the reflector relative to the microscope sample.

The apparatus of claim 6, further comprising means for recording the position and orientation of the reflector relative to the microscope sample. 25

The apparatus of claim 1, further comprising rotation means for rotating the microscope sample relative to the reflector.

9. The apparatus of claim 1, further comprising means for adjusting the position of the main objective relative to the microscope sample.

The apparatus of claim 9, further comprising means for recording the position of the main objective relative to the microscope sample.

The apparatus of claim 10, wherein the positioning means is operative in response to the position sensing means.

The apparatus of claim 1, further comprising means for adjusting the position of the spatial filter in the focusing path. 10

The apparatus of claim 1, further comprising means for adjusting the position of the focusing objective lens relative to the focusing reflector.

The apparatus of claim 1, further comprising means for adjusting the position of the focusing reflector relative to the focusing objective lens.

15. The apparatus of claim 14, further comprising means for detecting the position of the focusing reflector relative to the focusing objective lens. Apparatus as claimed in claim 15, wherein the positioning means is operative in response to the position sensing means. An apparatus according to claim 1, further comprising means for illuminating the sample for the purpose of providing illumination light for the microscope sample, wherein the illumination light is positioned in such a way that the illumination light is delivered to the main objective lens.

The apparatus of claim 1, further comprising means for laterally illuminating the sample for the purpose of providing lateral illumination light for the microscope sample, wherein the lateral illumination light is positioned in such a way that the illumination light is delivered to the reflector and to 5 the microscope sample.

19. Device as claimed in claim 18, further comprising means for selectively transmitting and reflecting light, with a view to further guiding the perceived light, from the main objective lens in the direction of the image-forming means, and for directing the image-forming means reflect away from the lighting light.

20. Device as claimed in claim 18, further comprising filtering means for further guiding the perceived light from the objective lens, and for filtering the illuminating light of the illuminating means.

21. Device as claimed in claim 18, further comprising means for selectively transmitting and reflecting light, with a view to reflecting perceived light from the main objective lens in the direction of the imaging means, and further guiding the illumination light away from the imaging means.

The device of claim 1, wherein the reflector is a flat mirror.

The apparatus of claim 1, wherein the reflector is a beam splitter, preferably a glass substrate on which a repeating structure is applied, and coated with a dielectric or metal layer.

An apparatus according to claim 1, wherein the reflector is a substrate that is provided with a single dihedral structure, preferably a glass substrate on which a repeating structure is applied, and coated with a dielectric or metal layer.

An apparatus according to claim 24, wherein the substrate comprises more than one dihedral structure, structures that can cross one another and whose cross direction may vary between 0 to 90 degrees. 5

The apparatus of claim 1, further comprising a microfluidic device in which a reflector of claims 12-25 is embedded, wherein the microfluidic device is provided with at least one channel realized by using a standard soft lithography with a polymer layer, preferably made from optical adhesive or coating materials with a low refractive index, or from poly (dimethylsiloxane) (PDMS).

The apparatus of claim 1, further comprising active or passive means to change the phase and amplitude of the sensed light. 15

The apparatus of claim 27, further comprising means for positioning the active or passive means to change the phase and amplitude of the sensed light.

A method for three-dimensional optical microscopy, comprising the steps of: a) focusing the main objective lens on the microscope sample; b) focusing the focusing objective lenses on a section or a plane in the image of the sample; c) transmitting images detected with the help of the focusing lenses 25 to the image detection means for the purpose of recording the images; d) adjusting the focusing mirrors facing the focusing objectives, to enable imaging of a new set of volume sections; e) repeating steps a), b), c) and d) until a plurality of sections of the sample were observed and recorded, thereby forming a data set of recorded images; f) processing the data set obtained in step e), in order to extract the 5 images of the microscope sample taken from different directions; g) refocusing the images obtained in step f) in the same basis; h) combining the images obtained in step g) to create a new volume image of the sample with an improved resolution.