JP2020086294A - Magnification observation device - Google Patents

Abstract

To display a live image and an unevenness emphasized image of an observation object in an easily comparable manner.SOLUTION: A magnification observation device, while an observation state for an observation object changes, displays, as an observation image, a dynamic image of the observation object acquired by an imaging unit, and when detecting stop of the change of the observation state for the observation object, displays, as an observation image, an unevenness emphasized image created by an image creation unit after the stop of the change of the observation state for the observation object.SELECTED DRAWING: Figure 5

Description

The present invention relates to a magnifying observation apparatus.

In general, a microscope has been required to have an optical system having high optical performance in order to observe an observation object with high accuracy. In addition, a microscope that displays an observation image on a display in real time has appeared, and there has been competition among microscope makers for how to accurately and precisely display the original appearance of an observation target.

On the other hand, the performance of the image processing chip has improved, and it has become possible to adopt it in a microscope. Further, the users of the microscope have spread not only to researchers such as living things but also to inspectors who inspect (enlarge and observe) the products produced in the factory. As a result, market needs for microscopes that are different from conventional optical performance pursuits have arisen. This is sometimes called a magnifying observation device in distinction from a conventional microscope (Patent Document 1).

JP, 2018-013734, A JP, 2017-531201, A

By the way, there is a need to observe the unevenness of the surface of a product produced in a factory with a magnifying observation device. According to Patent Document 2, a method of creating a concave-convex emphasized image is described. The appearance of the unevenness-enhanced image is significantly different from that of a normal image (luminance image). In order to create the unevenness-enhanced image, a plurality of brightness images with different illumination directions are required, and therefore the image processing time becomes longer than that of a normal image. Since fine unevenness can be confirmed in the unevenness-enhanced image, the user takes an image by using a higher-magnification objective lens. Therefore, since there is a large change in appearance between the normal image and the embossed image, it is difficult for the user to notice a minute difference between the normal image and the embossed image. Generally, in order to observe a desired observation site, the user has to repeat the procedures such as field-of-view search, focus adjustment, unevenness-enhanced image creation, confirmation of unevenness-enhanced image, switching to live image, field-of-view search... I won't. At the time of high-magnification observation, blurring of the observed part in the image becomes large (the depth of field is narrow). Since the field of view becomes narrow during high-magnification observation, it becomes difficult for the user to grasp which part of the observation target is being observed. Therefore, it is an object of the present invention to display a live image of an observation object and an unevenness-enhanced image so that they can be easily compared with each other.

The present invention is, for example,
An optical system including an objective lens and an imaging lens,
An XY stage movable relative to the optical system in at least the X and Y directions;
A detection unit for detecting relative movement of the XY stage with respect to the optical system;
An illumination unit that illuminates the observation target placed on the XY stage with illumination light from different directions;
An imaging unit that receives light from the observation object through the optical system and generates a brightness image of the observation object,
A changing unit that changes a focus position of the optical system along the optical axis of the optical system,
A control unit that controls the illumination unit, the imaging unit, and the changing unit;
A display unit for displaying an observation image that is an image of the observation object,
The control unit is
By irradiating the observation object with the illumination light from the first illumination direction by controlling the illumination unit, the changing unit and the image capturing unit are controlled so that a plurality of different combinations are obtained. Obtaining a plurality of first luminance images by imaging the observation object at each of the focal positions,
By irradiating the observation object with the illumination light from a second illumination direction that is symmetric with respect to the first illumination direction with respect to the first illumination direction by controlling the illumination unit. , A plurality of second luminance images are obtained by imaging the observation object at each of a plurality of different focusing positions by controlling the changing unit and the imaging unit,
The plurality of first luminance image and the plurality of second luminance image having an image generating unit that generates a concave-convex emphasized image in which the concave-convex of the surface of the observation object is emphasized by embossing the concave-convex.
The display unit is
While the detection unit is detecting the relative movement of the XY stage, a moving image of the observation object acquired by the imaging unit is displayed as the observation image,
When the detection unit detects that the relative movement of the XY stage is stopped, the unevenness-enhanced image generated by the image generation unit after the relative movement of the XY stage is stopped is displayed as the observation image. And a magnifying observation device.

According to the present invention, the live image of the observation target and the unevenness-enhanced image are displayed so that they can be easily compared with each other.

The figure explaining the outline of the magnifying observation apparatus 100. Diagram explaining the control unit, etc. Figure explaining the image processor Diagram illustrating depth stacking and embossing Diagram showing an example of the user interface Diagram showing an example of the user interface Flow chart explaining the magnifying observation process (A) shows a normal observation image, (B) shows an unevenness emphasized image, and (C) is a diagram showing a colored unevenness emphasized image (drawing substitute photograph).

<Magnification observation device>
FIG. 1 shows a magnifying observation apparatus 100. The magnifying observation device 100 is a device that magnifies and displays a sample such as a minute object, an electronic component, a work such as a workpiece (hereinafter, these are referred to as an observation target). The user can use the magnifying observation device 100 to inspect the external appearance of the observation object and perform dimension measurement and the like. The magnifying observation apparatus 100 may be called a microscope or a digital microscope. The observation object is not limited to the above-mentioned example, and various objects can be the observation object.

The observation unit 1 includes a base unit 10, a stand unit 20, a head unit 22, and a mounting table 30. The base part 10 is a part for placing the observation part 1 on a desk or the like without wobbling, and constitutes a substantially lower half part of the observation part 1. A mounting table 30 is provided on the base portion 10. The mounting table 30 is supported by a portion on the front side from the vicinity of the central portion of the base portion 10 in the front-rear direction, and projects upward from the base portion 10. The mounting table 30 is a part for mounting an observation target, and is configured by an electric mounting table in this embodiment. That is, the observation object can be supported movably in both the width direction (X direction) and the depth direction (Y direction) of the electric mounting table, and can be rotated in the vertical direction (Z direction) and around the Z axis. It is like this. The stand portion 20 is swingable with respect to the base portion 10. For example, the stand unit 20 can swing the observation unit 1 clockwise and counterclockwise when viewed from the front. When the stand unit 20 swings, the head unit 22 also swings. The stand section 20 and the head section 22 are attached so as to be movable in the Z-axis direction. The head unit 22 has an objective lens, an imaging lens, an illuminating device, an image sensor, and the like. The head unit 22 illuminates the observation target placed on the mounting table 30 with illumination light, detects the amount of reflected light or transmitted light of the illumination light from the observation target, and detects the image of the observation target. To generate. The details of the configuration and function of the observation unit 1 are disclosed in Japanese Patent Application No. 2018-161347 of the same applicant as the present application, the entire disclosure of which is incorporated as a part of this specification (incorporation herein by reference). It

The display unit 2 has a display screen 2a capable of color display, such as a liquid crystal display panel or an organic EL panel, and is supplied with electric power from the outside. A touch operation panel (an example of a reception unit) may be incorporated in the display screen 2a. Further, in this embodiment, an example in which the control unit 60 is incorporated in the display unit 2 has been described, but the present invention is not limited to this, and the control unit 60 may be incorporated in the observation unit 1 or the console unit 3. It may be incorporated or may be an external unit separate from the display unit 2, the observation unit 1 and the console unit 3. The display unit 2 and the observation unit 1 are connected by a cable 5 so that signals can be transmitted and received. The power supply to the observation unit 1 may be performed by the cable 5 or a power cable (not shown).

The console unit 3 is connected to the control unit 60, and is different from a general keyboard and mouse, and operates the observation unit 1, inputs various information, performs selection operations, selects images, specifies areas, and positions. This is a dedicated operation device that can be used to specify items. The mouse 4 is connected to the control unit 60 as a pointing device. The console unit 3 and the mouse 4 may be, for example, a touch panel type input device, a voice input device, or the like, as long as they can operate the magnifying observation device 100. In the case of a touch panel type input device, it can be integrated with the display unit 2 and can be configured to be able to detect an arbitrary position on the display screen displayed on the display unit 2. The console unit 3 and the mouse 4 are reception units that receive an input at an arbitrary position designated by the user on the image displayed on the display unit 2.

In addition to the above-described devices and apparatuses, the magnifying observation apparatus 100 can be connected to devices for operating and controlling, printers, computers for performing various other processes, storage devices, peripheral devices, and the like. The connection in this case is, for example, serial connection such as IEEE 1394, RS-232x, RS-422, or USB, parallel connection, or electrical or magnetic via a network such as 10BASE-T, 100BASE-TX, 1000BASE-T. And optical connection. In addition to wired connection, IEEE802. A wireless connection using radio LAN such as x or radio waves such as Bluetooth (registered trademark), infrared rays, optical communication, or the like may be used. Further, as a storage medium used for a storage device for exchanging data and storing various settings, for example, various memory cards, magnetic disks, magneto-optical disks, semiconductor memories, hard disks, etc. can be used. The magnifying observation device 100 can also be referred to as a magnifying observation system in which the above various units, devices, and devices are combined.

<Control part>
As shown in FIG. 2, the control unit 60 has a CPU 61, an image processor 66, a storage unit 67, and the like. The CPU 61 realizes various functions by executing the programs stored in the ROM area of the storage unit 67. The imaging control unit 62 controls the revolver driving unit 24 to rotate the revolver 21 provided on the head unit 22. As a result, the magnification of the objective lens 23 is changed. That is, the objective lens 23 with a certain magnification is switched to the objective lens 23 with another magnification. The imaging control unit 62 controls the Z-direction drive unit 28 that moves the head unit 22 up and down in order to change the focus position of the objective lens 23. The imaging control unit 62 moves the mounting table 30 in the X direction, the Y direction, and the Z direction through the mounting table driving unit 29, and rotates the mounting table 30 by θ. The imaging control unit 62 controls the imaging unit 25 provided in the head unit 22, causes the imaging unit 25 to image the observation target object W, and acquires an image of the observation target object W. The display control unit 63 causes the display unit 2 to display an image of the observation object W and the like. The illumination control unit 64 controls turning on and off of the ring illumination 26 and the coaxial incident illumination 27 provided on the head unit 22. The UI unit 65 displays a UI (user interface) on the display unit 2 through the display control unit 63 and receives a user instruction input from the console unit 3 or the mouse 4. The image processor 66 creates various image data from the image signal acquired by the imaging unit 25. The image processor 66 may be realized by the CPU 61. The storage unit 67 has a ROM area, a RAM area, a memory card, and the like. The detection unit 68 detects whether the mounting table 30 is stationary or moving, or whether the revolver 21 has changed the magnification of the objective lens 23. The console unit 3 may have a joystick for instructing to move the mounting table 30 in the X direction or the Y direction. In this case, the detection unit 68 detects that the mounting table 30 is moving if the joystick of the console unit 3 is tilted in either direction, and detects that the mounting table 30 is moving if the joystick is stationary at the neutral position. It detects that it is stationary. When the rotation of the revolver 21, that is, the magnification change of the objective lens 23 is instructed through the console unit 3 or the mouse 4 and the revolver drive unit 24 rotates the revolver, the detection unit 68 detects that the magnification has been changed. If the magnification change is not instructed, the detection unit 68 detects that the magnification has not been changed. The detection unit 68 may detect manual adjustment of the focus position by the user.

<Image processor>
As shown in FIG. 3, in the image processor 66, the brightness image generation unit 31 creates a brightness image from the image signal acquired by the imaging unit 25 through the imaging control unit 62. The HDR processing unit 32 controls the imaging unit 25 through the imaging control unit 62 to acquire a plurality of sub-luminance images having different exposure times, and performs a HDR process on the plurality of sub-luminance images to generate a luminance image. .. HDR is an abbreviation for high dynamic range. The depth synthesis unit 33 controls the imaging unit 25 through the imaging control unit 62 to acquire a plurality of sub-luminance images having different in-focus positions, and depth-synthesizes the plurality of sub-luminance images to form a depth synthesis image. To generate. The sub-luminance image may be an HDR-processed luminance image. The unevenness emphasizing unit 34 combines the first luminance image of the observation object W illuminated from the first illumination direction and the second luminance image of the observation object W illuminated from the second illumination direction to perform observation. An unevenness-enhanced image in which unevenness on the surface of the object W is emphasized is created. The first illumination direction and the second illumination direction are symmetrical with each other with the optical axis A1 interposed therebetween. The first luminance image and the second luminance image may be depth-combined images that are depth-combined. For example, the unevenness emphasizing unit 34 may generate the unevenness emphasized image based on the difference in brightness between the first depth combined image and the second depth combined image. Based on the brightness value i1 of the pixel (coordinates x, y) in the first depth composite image and the brightness value i2 of the pixel (coordinates x, y) in the second depth composite image, The pixel value Ixy may be obtained from the following equation.
Ixy=(i1-i2)/(i1+i2) (1)
The pixel value of the unevenness-enhanced image is obtained by dividing the difference between the pair of pixel values by the sum of the pair of pixel values. An expression different from the expression (1) may be adopted as long as the difference between the pixel values is normalized.

The difference between the luminance value i2 and the luminance value i1 which are the numerator reflects the inclination (inclination) of the surface of the local position of the observation object W corresponding to the pixel. The unevenness is emphasized by the inclination of the surface of the observation object W.

The absolute value of the brightness value changes depending on the reflectance of the color and material of the observation object W. In order to emphasize and reflect the concavo-convex component, normalization is performed by dividing the numerator by the sum of the luminance value i2 and the luminance value i1 which are denominators.

The coloring unit 36 acquires color information from the color image of the observation object W acquired by the imaging unit 25 and colorizes the unevenness-enhanced image. The height image generation unit 35 obtains the height of the surface of the observation target W for each pixel by integrating each pixel of the unevenness-enhanced image, and generates a height image having the height as each pixel. The height image generation unit 35 may generate a color height image by coloring each pixel of the height image according to the height of each pixel.

In this way, in the unevenness-enhanced image, the color component in the image of the observation object W is removed. If there is no color component, it is difficult to check the unevenness. Therefore, the coloring unit 36 executes coloring.

FIG. 8 shows an observation object marked with a laser marker as an example of the observation object W. FIG. 8A shows a normal observation image. FIG. 8B shows an unevenness-enhanced image. FIG. 8C shows an image generated by the height image generation unit 35 coloring the unevenness-enhanced image according to the height.

In FIG. 8(A), it is not possible to grasp the unevenness of the observation object W, but in FIG. 8(B) it is possible to understand the unevenness. In FIG. 8B, the concave and the convex may appear to be reversed due to an illusion. Therefore, by coloring as shown in FIG. 8C, it is possible to accurately grasp the concaves and convexes.

In this manner, the unevenness-enhanced image is obtained by switching the direction of illumination and obtaining the brightness value according to the local inclination of the surface of the observation object W.

<Principle of depth synthesis and embossing>
FIG. 4A illustrates that the observation object W is illuminated with illumination light from the first illumination direction by turning on a part of the ring illumination 26. The ring illumination 26 has four light source regions 140A, 140B, 140C, 140D. That is, the ring illumination 26 can illuminate the observation object W from the four illumination directions by selectively turning on and off the four light source regions 140A, 140B, 140C, and 140D. The illumination direction of the light source region 140A and the illumination direction of the light source region 140C are symmetrical with respect to the optical axis A1. The illumination direction of the light source region 140B and the illumination direction of the light source region 140D are symmetrical with respect to the optical axis A1. In FIG. 4A, only the light source area 140A is turned on.

FIG. 4B shows a luminance image I1 obtained by irradiating the observation object W with illumination light from the first illumination direction. SH indicates the shadow of the observation object W.

FIG. 4C shows that the observation object W is irradiated with the illumination light from the second illumination direction by turning on only the light source region 140C. FIG. 4C shows a luminance image I2 obtained by irradiating the observation object W with illumination light from the second illumination direction.

FIG. 4E shows n first luminance images I11 to I1n acquired by irradiating the observation object W with illumination light from the first illumination direction and changing the focus position. The depth synthesis unit 33 creates a first depth synthesis image Ia1 by performing depth synthesis on the n first luminance images I11 to I1n. For example, the depth synthesis unit 33 analyzes a plurality of pixels having the same pixel position in each of the plurality of first luminance images I11 to I1n, and determines the pixel having the highest degree of focus among the plurality of pixels as the pixel concerned. By selecting the focus pixel at the position, the first depth composite image I1a including the focus pixels at a plurality of pixel positions is generated.

FIG. 4F shows n second luminance images I21 to I2n acquired by irradiating the observation object W with illumination light from the second illumination direction and changing the focus position. The depth synthesis unit 33 creates a second depth synthesis image I2a by performing depth synthesis on the n second luminance images I21 to I2n. For example, the depth synthesis unit 33 analyzes a plurality of pixels having the same pixel position in each of the plurality of second luminance images I21 to I2n, and determines the pixel with the highest degree of focus among the plurality of pixels as the pixel concerned. By selecting the focus pixel at the position, the second depth composite image I2a composed of the focus pixels at a plurality of pixel positions is generated.

The unevenness emphasizing unit 34 generates an unevenness emphasized image using the first depth combined image I1a and the second depth combined image I2a. The unevenness emphasizing unit 34 generates the unevenness emphasized image based on the difference in luminance between the first depth combined image I1a and the second depth combined image I2a.

In addition, in FIG. 4, an observation target object W that causes a shadow SH is used. The observation object W, which makes it easier to grasp the unevenness by the unevenness-enhanced image, has minute unevenness as shown in FIG.

An image is acquired by switching the illumination light from the first illumination direction and the illumination light from the second illumination direction, and the unevenness-enhanced image is generated by the method as described above. Thereby, the unevenness on the surface of the observation object W is emphasized.

At point p in FIG. 8(A), it is not possible to determine whether there is unevenness. However, in the unevenness-enhanced image as shown in FIG. 8B, it is possible to grasp whether there is unevenness at the same point p.

In this way, the unevenness-enhanced image can be obtained by performing the arithmetic processing for emphasizing the minute inclination of the observation target by using the image in which the illumination direction is switched.

<Moving the mounting table and displaying images>
The UI unit 65 displays the unevenness emphasized image generated by the unevenness emphasizing unit 34 on the display unit 2. The user uses the unevenness-enhanced image to magnify and observe a part of the observation object W. Here, there are cases where a plurality of portions of the observation object W are targets for magnified observation. In this case, the user operates the console unit 3 to move the mounting table 30 in the X direction or the Y direction or rotate it in the θ direction. When a movement instruction in the X direction is input from the console unit 3, the CPU 61 causes the mounting table drive unit 29 to move the mounting table 30 in the X direction, and the CPU 61 inputs a movement instruction in the Y direction from the console unit 3. Then, the mounting table driving unit 29 moves the mounting table 30 in the Y direction. As long as such a movement instruction is input, the CPU 61 continuously moves the mounting table 30 according to the movement instruction. When the input of the movement instruction by the user is stopped, the CPU 61 stops the movement of the mounting table 30 by the mounting table drive unit 29.

Here, since a certain amount of processing time is required to generate the unevenness-enhanced image, the UI unit 65 causes the unevenness-enhancement unit 34 to generate the unevenness-enhancement process while the mounting table 30 is moving. The image cannot be displayed on the display unit 2. Therefore, the UI unit 65 displays the brightness image of the observation target object W generated by the brightness image generation unit 31 on the display unit 2 while the mounting table 30 is moving. Thereby, the user can confirm whether or not the next observation site is located in the visual field range of the image capturing unit 25 by confirming the brightness image displayed on the display unit 2. Here, the UI unit 65 updates the luminance image of the observation target object W at regular intervals, so that the display unit 2 displays the luminance image of the observation target object W like a moving image. When the CPU 61 detects that the input of the movement instruction is stopped (stop of the mounting table 30), the CPU 61 instructs the image processor 66 to generate the unevenness-enhanced image. As a result, the acquisition of the brightness image, the depth combination, and the unevenness emphasized image described above are executed again, and the unevenness emphasized image at the new observation position is displayed on the display unit 2. Even if the user does not instruct to regenerate the unevenness-enhanced image, the unevenness-enhanced image is regenerated according to the stop of the mounting table 30. Therefore, the user will be able to efficiently observe a plurality of observation sites.

<Coloring of the embossed image>
Each pixel forming the unevenness-enhanced image shows unevenness on the surface of the observation object W and does not include surface color information. Therefore, it is difficult for the user to grasp the relationship between the color of the observed region and the position of the unevenness. Therefore, the coloring unit 36 turns on all the light source regions 140A to 140D of the ring illumination 26, and causes the imaging unit 25 to image the observation target object W. As a result, the luminance image generation unit 31 generates a color image (luminance image) of the observation object W. The coloring unit 36 acquires color information from the color image, maps the color information on the unevenness emphasized image to colorize the unevenness emphasized image, and displays the color on the display unit 2. This makes it easier for the user to understand the relationship between the color of the observed region and the position of the unevenness.

<HDR processing>
When the surface of the observation object W is a metal, whiteout pixels or blackout pixels may occur in the luminance image. In such a case, it becomes difficult to confirm the unevenness in the finally generated unevenness-enhanced image. Therefore, the HDR processing unit 32 may be adopted. The HDR processing unit 32 acquires a plurality of sub-luminance images having different exposure times for one in-focus position, and performs HDR processing on the plurality of sub-luminance images to generate one luminance image. The HDR processing unit 32 acquires a plurality of sub-luminance images having different exposure times each time the focus position is changed, and performs HDR processing on the plurality of sub-luminance images to generate one luminance image. As a result, the first luminance images I11 to I1n and the second luminance images I21 to I2n are all HDR-processed images. Therefore, the depth synthesis unit 33 performs depth synthesis on the HDR-processed first luminance images I11 to I1n to generate a first depth image I1a, and performs depth synthesis on the HDR-processed second luminance images I21 to I2n. , The second depth image I2a is generated. Further, the unevenness emphasizing unit 34 synthesizes the HDR-processed first depth image I1a and the HDR-processed second depth image I2a to generate the HDR-processed unevenness emphasis image. As a result, overexposure and underexposure are less likely to occur, so that the user can more easily grasp the unevenness of the observation object W by checking the unevenness-enhanced image.

<Height image>
The unevenness-enhanced image described above does not include the height information of the surface of the observation object W. Therefore, a crater illusion occurs. The crater illusion is a phenomenon in which an observer mistakenly recognizes a concave portion and a convex portion because the concave shape and the convex shape cannot be distinguished as an image. Therefore, the height image generation unit 35 obtains the height of the surface of the observation object W for each pixel by integrating each pixel of the unevenness-enhanced image, and generates a height image having the height as each pixel. However, it may be displayed on the display unit 2. Further, the height image generation unit 35 converts the height data of each pixel of the height image into color information and maps the color information on the unevenness-enhanced image so that the height image generation unit 35 colors with a different color depending on the height. The unevenness-enhanced image thus generated may be generated and displayed on the display unit 2. This makes it easier for the user to distinguish between the concave shape and the convex shape on the surface of the observation object W based on the color information.

<Other>
Although the light source areas 140A and 140C have been described in FIG. 4, this description is also applicable to the light source areas 140B and 140D. That is, the image processor 66 may generate an unevenness-enhanced image using a pair with the light source regions 140A and 140C, or may generate an unevenness-enhanced image using a pair with the light source regions 140B and 140D. The illumination direction of the light source region 140B may be referred to as the third illumination direction. The illumination direction of the light source region 140D may be referred to as the fourth illumination direction. The third illumination direction and the fourth illumination direction are symmetrical (axisymmetric) with the optical axis A1 interposed.

The luminance image generation unit 31 controls the ring illumination 26 to irradiate the observation object W with illumination light from the third illumination direction different from the first illumination direction and the second illumination direction. . The brightness image generation unit 31 controls the Z-direction drive unit 28 and the imaging unit 25, and acquires a plurality of third brightness images by imaging the observation object W at each of a plurality of different in-focus positions. Similarly, the luminance image generation unit 31 controls the ring illumination 26 to irradiate the observation target W with illumination light from the fourth illumination direction. The brightness image generation unit 31 controls the Z direction drive unit 28 and the imaging unit 25, and acquires a plurality of fourth brightness images by imaging the observation target W at each of a plurality of different focus positions. The image processor 66 outputs the unevenness-enhanced image corresponding to the illumination direction selected by the UI unit 65 among the first illumination direction, the second illumination direction, the third illumination direction, and the fourth illumination direction to a plurality of first luminance images It is generated using a plurality of luminance images corresponding to the selected illumination direction among the plurality of second luminance images, the plurality of third luminance images, and the plurality of fourth luminance images. The display unit 2 displays the unevenness-enhanced image corresponding to the selected illumination direction. Although the first illumination direction is selected in the above embodiment, the second illumination direction may be selected. In this case, i1 and i2 in the equation (1) are interchanged. When the third illumination direction is selected, the brightness value of each pixel of the third brightness image is i1 and the brightness value of each pixel of the fourth brightness image is i2 in the expression (1). When the fourth illumination direction is selected, the luminance value of each pixel of the fourth luminance image becomes i1 and the luminance value of each pixel of the third luminance image becomes i2 in the formula (1).

<Modification>
In the above-described embodiment, the unevenness enhancement is performed after the depth stacking is performed. However, the unevenness enhancement may be performed first, and then the depth stacking may be performed.

The unevenness emphasizing unit 34 uses the first brightness image I11 and the second brightness image I21 to generate a sub unevenness emphasis image I1x. Similarly, the unevenness emphasizing unit 34 uses the first brightness image I12 and the second brightness image I22 to generate a sub unevenness emphasis image I2x. By repeating the same process as described below, finally, the unevenness emphasizing unit 34 generates the sub unevenness-enhanced image Ixn using the first luminance image I1n and the second luminance image I2n. As a result, the depth combination unit 33 performs depth combination of the n sub unevenness-enhanced images Ix1 to Ixn to generate a final unevenness-enhanced image, which is displayed on the display unit 2.

The ring illumination 26 may be arranged around the objective lens 23, or may be configured to irradiate the observation object W with illumination light through the objective lens 23. In addition, as the layout of the ring illumination 26, the one disclosed in US2018/024346 of the same applicant as this case may be used. The entire disclosure of which is incorporated herein by reference.

Even when the head unit 22 of the observation unit 1 is swung with respect to the mounting table 30 to perform tilt observation, the unevenness-enhanced image is displayed in the same manner as when the head unit 22 is perpendicular to the mounting table 30. Can be generated. As described above, the depth stacking may be further performed on the unevenness-enhanced image. At this time, the head portion 22 and the mounting table 30 may be drive-controlled so that the upper surface of the work W is aligned with the eucentric position. The control at this time is also disclosed in Japanese Patent Application No. 2018-161347, and the entire disclosure thereof is incorporated as a part of this specification (incorporation here by reference).

<User interface>
FIG. 5 shows a user interface 70 displayed on the display unit 2 by the UI unit 65. The image display area 71 displays an image output from the image processor 66 through the display control unit 63. This image is a live image of the observation target W, a depth-combined unevenness enhancement image, an unevenness enhancement image colored according to the color of the surface of the observation target W, or an unevenness enhancement image colored based on height data ( Color height image). When the detection unit 68 detects that the mounting table 30 is moving or that switching of the objective lens 23 is instructed, the UI unit 65 displays a live image of the observation object W in the image display area 71. When the change of the magnification is completed and the detection unit 68 detects that the mounting table 30 is stationary, the UI unit 65 displays the depth-combined unevenness-enhanced image in the image display area 71. When it is detected that the mounting table 30 moves while the unevenness-enhanced image is displayed, the UI unit 65 displays the live image in the image display area 71. The illumination direction selection unit 72 has four buttons corresponding to the light source areas 140A to 140D. As described above, since the light source region 140A and the light source region 140C form a pair, when the button corresponding to the light source region 140A is selected by the pointer 74, the illumination direction of the light source region 140A becomes the first illumination direction. Then, the illumination direction of the light source region 140C is determined as the second illumination direction. Similarly, when the button corresponding to the light source region 140D is selected by the pointer 74, the illumination direction of the light source region 140D is determined as the first illumination direction, and the illumination direction of the light source region 140B is determined as the second illumination direction. The magnification selection unit 73 is a pull-down list for selecting any one of the objective lenses 23 mounted on the revolver 21. For example, the UI unit 65 acquires the magnification of each of the plurality of objective lenses 23 attached to the revolver 21 from the setting information of the objective lenses 23 and creates a pull-down list. When the user operates the pointer 74 to select one of the magnifications, the UI unit 65 notifies the revolver drive unit 24 of the selected magnification (objective lens 23). The revolver drive unit 24 rotates the revolver 21 so that the objective lens 23 having the notified magnification is located on the photographing optical axis A1. When the revolver drive unit 24 has finished rotating, it outputs a rotation end signal to the CPU 61. Upon receiving the rotation end signal, the detection unit 68 determines that the magnification change has been completed. Normally, when the magnification change is completed, it is necessary to adjust the position of the observation site within the visual field range. The user positions the observation site at a desired position within the visual field range by moving the mounting table 30. When the mounting table 30 stands still, the UI unit 65 instructs the image processor 66 to generate the unevenness-enhanced image. The UI unit 65 displays, in the image display area 71, the unevenness-enhanced image that has been depth-synthesized instead of the live image.

It should be noted that the user interface 70 may be provided with a switching button for explicitly switching between displaying the embossed image and displaying the live image. Thus, the user may compare the both of the unevenness-enhanced image and the live image by switching between them after the completion of the unevenness-enhanced image.

FIG. 6 shows another example of the user interface 70 displayed on the display unit 2 by the UI unit 65. In this example, the image display area 71 displays a live image, and the image display area 75 displays depth-combined unevenness-enhanced images, unevenness-enhanced images colored according to the color of the surface of the observation object W, and height data. An image (color height image) or the like that is colored based on is displayed. Although the image display areas 71 and 75 are arranged in the vertical direction in FIG. 6, they may be arranged in the horizontal direction.

The coloring designating section 76 uses radio buttons for instructing to color the depth-combined unevenness-enhanced image based on the color information of the surface of the observation target W, and height information of the surface of the observation target W. Based on this, it has a radio button for instructing to color the depth-combined unevenness-enhanced image and a radio button for instructing not to color. The UI unit 65 notifies the image processor 66 of the coloring instruction designated by the coloring designation unit 76. The coloring designating section 76 may also be provided in the user interface 70 shown in FIG. The depth stacking instruction unit 77 is a check box for designating whether to enable or disable depth stacking. When the depth stacking instruction unit 77 disables the depth stacking, the unevenness emphasizing unit 34 generates the unevenness emphasized image using the first brightness image and the second brightness image according to the equation (1). In this case, the image display area 75 displays an unevenness-enhanced image that has not been depth-combined.

Although it has been described that the live image is displayed in the image display area 71, the present invention is not limited to this. A still image or the like may be displayed as long as the entire state of the observation target object W can be confirmed.

<Flowchart>
FIG. 7 is a flowchart showing the magnification observation process. When the start is instructed by the mouse 4 or the console section 3, the CPU 61 executes the following processing.

The case where the unevenness emphasized image is displayed in the image display area 71 will be described.

In S0, the CPU 61 controls the mounting table driving unit 29 according to the movement instruction input from the mouse 4 or the console unit 3 to move the mounting table 30.

When the movement start is detected in S1, a live image is displayed in the image display area 71. That is, the CPU 61 controls the imaging unit 25 and the ring illumination 26 via the imaging control unit 62 and the illumination control unit 64 to cause the observation target object W to be imaged, and controls the image processor 66 to display a live image of the observation target object W ( A brightness image that is updated at regular time intervals) is generated, and a live image is displayed on the display unit 2 via the display control unit 63. The live image is displayed in the image display area 71. The illumination control unit 64 turns on all four light source regions 140A to 140D for the live image. While viewing the live image, the user moves the mounting table 30 in the X direction, the Y direction, the Z direction, and the θ direction so that the observation target object W enters the visual field range of the imaging unit 25.

In S2, the CPU 61 uses the detection unit 68 to determine whether the movement of the mounting table 30 is completed. For example, when the input of the movement instruction by the mouse 4 or the console unit 3 is finished and a certain time has elapsed or an explicit imaging instruction is input, the CPU 61 (detection unit 68) causes the mounting table 30 It is determined that the movement of is completed. When the CPU 61 determines that the movement of the mounting table 30 is completed, the CPU 61 proceeds to S3.

In S3, the CPU 61 displays the live image at the end of the movement in the image display area 71. If the image of the observation target object W is not displayed in the display area during the period from S4 to S10, it becomes difficult for the user to grasp the observation target object W. Therefore, the CPU 61 may display the image of the observation object W immediately before the unevenness-enhanced image is generated in the image display area 71, if necessary, while generating the unevenness-enhanced image.

In S4, the CPU 61 generates a plurality of first luminance images I11 to I1n having different focusing positions in the first illumination direction. The illumination control unit 64 turns on one light source region corresponding to the first illumination direction among the light source regions 140A to 140D. The imaging control unit 62 causes the imaging unit 25 and the brightness image generation unit 31 to acquire the first brightness images I11 to I1n while gradually changing the focus position of the objective lens 23 through the Z-direction drive unit 28.

In S5, the CPU 61 generates a plurality of second luminance images I21 to I2n having different focusing positions in the second illumination direction. The illumination control unit 64 turns on one light source region corresponding to the second illumination direction among the light source regions 140A to 140D. The imaging control unit 62 causes the imaging unit 25 and the luminance image generation unit 31 to acquire the second luminance images I21 to I2n while gradually changing the focus position of the objective lens 23 through the Z-direction driving unit 28.

In S6, the CPU 61 uses the depth combination unit 33 to perform depth combination of the first luminance images I11 to I1n to generate the first depth image I1a in the first illumination direction.

In S7, the CPU 61 uses the depth combination unit 33 to perform depth combination of the second luminance images I21 to I2n to generate the second depth image I2a in the second illumination direction.

In S8, the CPU 61 uses the unevenness emphasizing unit 34 to generate an unevenness-enhanced image based on the first depth image I1a and the second depth image I2a.

In S9, the CPU 61 uses the coloring unit 36 or the height image generation unit 35 to color the concave-convex emphasized image. Thereby, the color of the surface of the observation object W is added to the unevenness-emphasized image, or the unevenness-enhanced image is colored according to the height data acquired from the height image. Whether or not to execute the coloring process and what to perform the coloring process depend on which radio button is pressed in the coloring designation unit 76.

In S10, the CPU 61 displays the concave-convex emphasized image on the display unit 2 through the UI unit 65 and the display control unit 63. That is, in the image display area 71, the embossed image is displayed instead of the live image of the observation object W. In the user interface 70 shown in FIG. 6, a live image of the observation object W is displayed in the image display area 71, and a depth-combined unevenness-enhanced image is displayed in the image display area 75.

In S11, the CPU 61 determines whether or not an instruction to end the magnifying observation process has been input. When the end instruction is input from the mouse 4 or the like, the CPU 61 ends the magnifying observation process. If the end instruction has not been input, the CPU 61 proceeds to S12.

In S12, the CPU 61 uses the detection unit 68 to determine whether or not an instruction to move the mounting table 30 or an instruction to change the magnification is input from the mouse 4 or the like. If neither the movement instruction nor the magnification change instruction has been input, the CPU 61 returns to S11. When the movement instruction or the magnification change instruction is input, the CPU 61 returns to S1 and displays the live image of the observation object W on the display unit 2. After that, when the movement of the mounting table 30 is stopped, the CPU 61 executes the generation of the unevenness-enhanced image again.

<Summary>
As described with reference to FIG. 1 and the like, the head unit 22 functions as an optical system including an objective lens and an imaging lens. The mounting table 30 is an example of an XY stage that can move relative to the optical system in at least the X and Y directions. The detection unit 68 functions as a detection unit that detects the relative movement of the XY stage with respect to the optical system. The ring illumination 26 functions as an illumination unit that illuminates the observation object W placed in the visual field of the optical system with illumination light from different directions. The imaging unit 25 receives light from the observation object W via the optical system and generates a brightness image of the observation object W. The Z-direction drive unit 28 functions as a changing unit that changes the focus position of the optical system along the optical axis A1 of the optical system. The control unit 60 functions as a control unit that controls the illumination unit, the imaging unit, and the changing unit. The display unit 2 functions as a display unit that displays an observation image that is an image of the observation object W. The image processor 66 (i) irradiates the observation target with illumination light from the first illumination direction by controlling the illumination unit, controls the changing unit and the imaging unit, and controls a plurality of different units. A plurality of first luminance images are acquired by imaging the observation object at each of the in-focus positions, and (ii) the illumination unit is controlled to control the optical axis of the observation object with respect to the first illumination direction. By irradiating the observation object with illumination light from the second illumination direction that is symmetrical with respect to, and controlling the changing unit and the imaging unit to image the observation object at each of a plurality of different in-focus positions. Acquiring a plurality of second luminance images, and (iii) depth combination and unevenness enhancement of the plurality of first luminance images and the plurality of second luminance images enhances the unevenness of the surface of the observation target and captures the image. It functions as an image generation unit that generates an unevenness-enhanced image with a wider depth of field than a single luminance image that can be acquired by the unit. The display unit 2 displays the embossed image as the observation image. This provides a depth-combined unevenness-enhanced image.

While the detection unit 68 is detecting a change in the observation state with respect to the observation target W (eg, relative movement of the XY stage), the display unit 2 displays the observation target as the observation image acquired by the imaging unit 25. A moving image (live image) of the object W is displayed. When the detection unit 68 detects the stop of the change of the observation state with respect to the observation target W (eg, the relative movement of the XY stage), the display unit 2 stops the change of the observation state with respect to the observation target W as an observation image. After that, the unevenness-enhanced image generated by the image generation unit is displayed. As a result, the user can move the XY stage while confirming the relationship between the visual field range of the imaging unit 25 and the observation target object W. Further, since the user can immediately see the depth-combined unevenness-enhanced image when the XY stage is stationary, the user can efficiently observe the observation object W. The change of the observation state with respect to the observation target is, for example, the change of the focus position by the changing portion and the movement of the XY stage (eg, movement in the X direction, movement in the Y direction, movement in the Z direction, θ rotation). Etc.), switching of illumination by the illumination unit, and change in relative angle between the optical axis of the optical system and the observation object W may be included.

The detection unit 68 may be configured to further detect a change in the magnification of the optical system. The display unit 2 displays the moving image of the observation target object W acquired by the imaging unit 25 as the observation image while the detection unit 68 detects the relative movement of the XY stage. Further, when the detection unit 68 detects that the relative movement of the XY stage is stopped and the magnification of the optical system is changed, the display unit 2 stops the relative movement of the XY stage and changes the magnification as an observation image. The unevenness-enhanced image generated by the image generation unit may be displayed later.

As shown in FIGS. 5 and 6, the display unit 2 may have a display area for a moving image and a display area for a concave-convex emphasized image.

The detection unit 68 may detect that the focus adjustment of the objective lens 23 by manual or auto focus is completed. When the magnification of the objective lens 23 is changed, the imaging control unit 62 controls the Z direction drive unit 28 to execute auto focus. Therefore, the concave-convex emphasized image may be generated with the magnification changed and the completion of the focus adjustment of the objective lens 23 as a trigger.

The UI unit 65 may further display a separate window for displaying a navigation image on the display unit 2 separately from the user interface 70. The navigation image is, for example, an image of the observation object W acquired using the objective lens 23 having a low magnification. When the user designates the observation site in the navigation image with the pointer 74, the UI unit 65 converts the designated position in the navigation image into the coordinate position of the mounting table 30, and moves the mounting table 30 to the position. As a result, it becomes possible to easily fit the observation site within the visual field range.

The depth stacking unit 33 creates a first depth stacking image by depth stacking a plurality of first brightness images, and a depth stacking process that creates a second depth stacking image by depth stacking a plurality of second brightness images. Function as a department. The unevenness emphasizing unit 34 functions as an emphasized image generation unit that generates an unevenness emphasized image based on the difference in brightness between the first depth combined image and the second depth combined image.

The depth synthesizing unit 33 analyzes a plurality of pixels having the same pixel position in each of the plurality of first luminance images, and determines the pixel with the highest degree of focusing among the plurality of pixels as the focused pixel at the pixel position. The selection may be performed to generate the first depth composite image including the in-focus pixels at each of a plurality of pixel positions. Note that the focus degree may be obtained for each pixel area including a plurality of adjacent pixels. The depth synthesizing unit 33 analyzes a plurality of pixels having the same pixel position in the plurality of second luminance images, and determines the pixel with the highest degree of focus among the plurality of pixels as the focus pixel at the pixel position. The selection may be performed to generate a second depth composite image including in-focus pixels at each of a plurality of pixel positions.

The mounting table 30 is an example of an XY stage on which the observation object W is mounted and which is movable in at least the X direction and the Y direction. The mounting table drive unit 29 is an example of a drive unit that drives the XY stage. As described in relation to S12, the CPU 61 may function as a detection unit that detects that the XY stage has moved and then stopped. The CPU 61 and the image processor 66 may execute the generation of the unevenness-enhanced image again when the detection unit detects that the XY stage is stationary after moving. This allows the user to save the trouble of explicitly instructing the generation of the unevenness-enhanced image.

The detection unit may make the determination based on the image of the observation object W. The detection unit may determine that the XY stage is stopped when there is no difference or a difference between the previous time and the current time.

When the CPU 61 detects that the XY stage is moving, the CPU 61 may be configured to display the moving image (live image) acquired by the imaging unit 25 on the display unit 2. When the CPU 61 detects that the XY stage is stationary, the CPU 61 may cause the image processor 66 to again generate the unevenness-enhanced image and display the unevenness-enhanced image on the display unit 2.

The coloring unit 36 may function as a color combining unit that acquires color information from the color image of the observation object W acquired by the imaging unit 25 and colorizes the unevenness-enhanced image. This makes it easier for the user to understand the relationship between the unevenness and the surface color.

The image processor 66 acquires a plurality of sub-luminance images each having a different exposure time when generating the first luminance image, and performs HDR processing on the plurality of sub-luminance images to generate the first luminance image. Good. Similarly, the image processor 66 acquires a plurality of sub-luminance images each having a different exposure time when generating the second luminance image, and performs a HDR process on the plurality of sub-luminance images to generate the second luminance image. May be. By applying the HDR processing in this way, it is possible to reduce whiteout and underexposure and obtain a depth-combined unevenness-enhanced image.

The height image generation unit 35 obtains the height of the surface of the observation target for each pixel by integrating each pixel of the unevenness-enhanced image, and generates a height image having the height as each pixel. Good. Further, the height image generation unit 35 may generate a color height image by coloring each pixel of the height image according to the height of each pixel. This makes it difficult for the crater illusion to occur, and allows the user to more accurately distinguish between the concave shape and the convex shape.

The mouse 4, the console unit 3, and the CPU 61 may function as a selection unit that selects the illumination direction of the illumination unit. The image processor 66 irradiates the observation target W with illumination light from a third illumination direction different from the first illumination direction and the second illumination direction by controlling the illumination unit, and the changing unit. The plurality of third luminance images may be acquired by controlling the image capturing unit and capturing the observation target object at each of a plurality of different focus positions. The image processor 66 irradiates the observation object with illumination light from a fourth illumination direction that is symmetrical with respect to the third illumination direction with respect to the third illumination direction by controlling the illumination unit. You may acquire a some 4th brightness|luminance image by controlling a change part and an image pick-up part, and image|photographing an observation target object in each of a plurality of different focus positions, respectively. Further, the image processor 66 outputs a plurality of first luminance images, the unevenness-enhanced image corresponding to the illumination direction selected by the selection unit among the first illumination direction, the second illumination direction, the third illumination direction and the fourth illumination direction. Alternatively, it may be generated using a plurality of brightness images corresponding to the illumination direction selected by the selection unit among the plurality of second brightness images, the plurality of third brightness images, and the plurality of fourth brightness images. The display unit 2 may display the unevenness-enhanced image corresponding to the illumination direction selected by the selection unit.

The unevenness emphasizing unit 34 synthesizes a pair of brightness images having the same in-focus position from the plurality of first brightness images and the plurality of second brightness images based on the difference in brightness, and then combines each of the plurality of focus positions. A plurality of sub unevenness-enhanced images in which unevenness is emphasized may be generated. The depth synthesis unit 33 may function as a depth synthesis image generation unit that generates a depth-synthesized unevenness-enhanced image by depth-synthesizing a plurality of sub-unevenness-enhanced images. As described above, the order of the embossing and the depth stacking may be first.

The illumination unit may be a ring illumination 26 arranged around the objective lens 23. The illumination unit may be an illumination light source provided inside the head unit 22 that illuminates the observation object W with illumination light through the objective lens 23.

Claims (15)

An optical system including an objective lens and an imaging lens,
An XY stage movable relative to the optical system in at least the X and Y directions;
A detection unit for detecting relative movement of the XY stage with respect to the optical system;
An illumination unit that illuminates the observation target placed on the XY stage with illumination light from different directions;
An imaging unit that receives light from the observation object through the optical system and generates a brightness image of the observation object,
A changing unit that changes a focus position of the optical system along the optical axis of the optical system,
A control unit that controls the illumination unit, the imaging unit, and the changing unit;
A display unit for displaying an observation image that is an image of the observation object,
The control unit is
By irradiating the observation object with the illumination light from the first illumination direction by controlling the illumination unit, the changing unit and the image capturing unit are controlled so that a plurality of different combinations are obtained. Obtaining a plurality of first luminance images by imaging the observation object at each of the focal positions,
By irradiating the observation object with the illumination light from a second illumination direction that is symmetric with respect to the first illumination direction with respect to the first illumination direction by controlling the illumination unit. , A plurality of second luminance images are obtained by imaging the observation object at each of a plurality of different focusing positions by controlling the changing unit and the imaging unit,
The plurality of first luminance image and the plurality of second luminance image having an image generating unit that generates a concave-convex emphasized image in which the concave-convex of the surface of the observation object is emphasized by embossing the concave-convex.
The display unit is
While the detection unit is detecting the relative movement of the XY stage, a moving image of the observation object acquired by the imaging unit is displayed as the observation image,
When the detection unit detects that the relative movement of the XY stage is stopped, the unevenness-enhanced image generated by the image generation unit after the relative movement of the XY stage is stopped is displayed as the observation image. Magnification observation device characterized by. The detection unit is configured to further detect a change in magnification of the optical system,
The display unit is
While the detection unit is detecting the relative movement of the XY stage, a moving image of the observation object acquired by the imaging unit is displayed as the observation image,
When the detection unit detects that the relative movement of the XY stage is stopped and the magnification of the optical system is changed, the observation image is displayed after the relative movement of the XY stage is stopped and the magnification is changed. The magnifying observation apparatus according to claim 1, wherein the unevenness-enhanced image generated by the image generation unit is displayed. The magnifying observation apparatus according to claim 1, wherein the display unit has a display area for the moving image and a display area for the embossed image. The image generation unit,
A depth synthesizing unit that generates a second depth synthesizing image by depth synthesizing the plurality of second luminance images while generating a first depth synthesizing image by depth synthesizing the plurality of first luminance images.
4. The enhanced image generation unit that generates the unevenness enhanced image based on a difference in luminance between the first depth synthetic image and the second depth synthetic image, and the enhanced image generation unit according to claim 1. The magnifying observation device described. The depth combination unit,
By analyzing a plurality of pixels having the same pixel position in each of the plurality of first luminance images and selecting a pixel having the highest degree of focusing among the plurality of pixels as a focused pixel at the pixel position. , Generating the first depth composite image composed of focused pixels at a plurality of pixel positions,
By analyzing a plurality of pixels having the same pixel position in each of the plurality of second luminance images and selecting the pixel having the highest degree of focusing among the plurality of pixels as the focused pixel at the pixel position. The magnifying observation apparatus according to claim 4, wherein the magnifying observation apparatus is configured to generate the second depth composite image including focused pixels at a plurality of pixel positions. 6. The color synthesizing unit that further acquires color information from the color image of the observation target object acquired by the imaging unit and colorizes the unevenness-enhanced image, further comprising: a color synthesizing unit. The magnifying observation device according to the item. The image generation unit,
Acquiring a plurality of sub-luminance images each having a different exposure time when generating the first luminance image, and generating the first luminance image by performing HDR processing on the plurality of sub-luminance images,
When generating the second luminance image, a plurality of sub-luminance images having different exposure times are acquired, and the second luminance image is generated by performing HDR processing on the plurality of sub-luminance images. The magnifying observation apparatus according to any one of claims 1 to 6, wherein It further has a height image generation unit that calculates the height of the surface of the observation object for each pixel by integrating each pixel of the unevenness-enhanced image, and generates a height image having the height as each pixel. The magnifying observation apparatus according to any one of claims 1 to 7, wherein 9. The height image generation unit is configured to generate a color height image by coloring each pixel of the height image according to the height of each pixel. The magnifying observation apparatus described in 1. Further comprising a selection unit for selecting an illumination direction of the illumination unit,
The image generation unit,
By irradiating the observation object with the illumination light from the third illumination direction different from the first illumination direction and the second illumination direction by controlling the illumination unit, the changing unit And a plurality of third luminance images are acquired by controlling the image capturing unit and capturing the observation object at each of a plurality of different focus positions.
By irradiating the observation object with the illumination light from a fourth illumination direction that is symmetrical with respect to the third illumination direction with respect to the third illumination direction with respect to the third illumination direction by controlling the illumination unit. , Acquiring a plurality of fourth luminance images by imaging the observation object at each of a plurality of different focus positions by controlling the changing unit and the imaging unit,
The image generation unit, further, the unevenness emphasized image corresponding to the illumination direction selected by the selection unit among the first illumination direction, the second illumination direction, the third illumination direction and the fourth illumination direction, A plurality of brightness images corresponding to the illumination direction selected by the selection unit among the plurality of first brightness images, the plurality of second brightness images, the plurality of third brightness images and the plurality of fourth brightness images. Generated using
10. The magnifying observation apparatus according to claim 1, wherein the display unit is configured to display an unevenness-enhanced image corresponding to the illumination direction selected by the selection unit. .. The image generation unit,
From the plurality of first luminance images and the plurality of second luminance images, a pair of luminance images having the same in-focus position is synthesized based on the difference in luminance, and unevenness is formed for each of the plurality of in-focus positions. An emphasized image generation unit that generates a plurality of emphasized images, and
The magnifying observation apparatus according to claim 1, further comprising: a depth-combined image generation unit that generates the unevenness-enhanced image by depth-compositing the plurality of emphasized images. The magnifying observation apparatus according to any one of claims 1 to 11, wherein the illumination unit is a ring illumination arranged around the objective lens. The magnifying observation apparatus according to any one of claims 1 to 11, wherein the illumination unit irradiates the observation object with the illumination light through the objective lens. An optical system including an objective lens and an imaging lens,
An XY stage movable relative to the optical system in at least the X and Y directions;
An illumination unit that illuminates the observation target placed on the XY stage with illumination light from different directions;
An imaging unit that receives light from the observation object through the optical system and generates a brightness image of the observation object,
A changing unit that changes a focus position of the optical system along the optical axis of the optical system,
A control unit that controls the illumination unit, the imaging unit, and the changing unit;
A display unit that displays an observation image that is an image of the observation object,
A detection unit that detects a change in the observation state for the observation target using the optical system,
Have
The control unit is
By irradiating the observation object with the illumination light from the first illumination direction by controlling the illumination unit, the changing unit and the image capturing unit are controlled so that a plurality of different combinations are obtained. Obtaining a plurality of first luminance images by imaging the observation object at each of the focal positions,
By irradiating the observation object with the illumination light from a second illumination direction that is symmetric with respect to the first illumination direction with respect to the first illumination direction by controlling the illumination unit. , A plurality of second luminance images are obtained by imaging the observation object at each of a plurality of different focusing positions by controlling the changing unit and the imaging unit,
The plurality of first luminance image and the plurality of second luminance image having an image generating unit that generates a concave-convex emphasized image in which the concave-convex of the surface of the observation object is emphasized by embossing the concave-convex.
The display unit is
While the detection unit is detecting a change in the observation state of the observation target, as the observation image, a moving image of the observation target acquired by the imaging unit is displayed,
When the detection unit detects the stop of the change of the observation state for the observation target object, the unevenness-enhanced image generated by the image generation unit after the stop of the change of the observation state for the observation target object is displayed as the observation image. A magnifying observation device characterized by: The change in the observation state detected by the detection unit is
A change in the focus position by the change unit,
Movement of the XY stage,
Switching of lighting by the lighting unit,
The magnifying observation apparatus according to claim 14, further comprising at least one of a change in a relative angle between the optical axis of the optical system and the observation target.