CN114983334A - A slit lamp self-adjustment control method and slit lamp based on machine vision - Google Patents

Abstract

The invention provides a slit lamp self-adjustment control method based on machine vision, which includes steps 1) using a camera to collect eye images; 2) comparing the collected eye images with standard data in a database , determine the eye lesion area, and mark the lesion area; 3) establish a two-dimensional coordinate system with the center of the pupil as the coordinate origin, and obtain the coordinates of the lesion area; 4) generate manipulation data according to the coordinates of the lesion area, and manipulate the slit lamp to obtain The optimal shooting angle determines the type of eye lesions according to the lesion image of the marked lesion area, and generates data on the filtering method, the width of the projection light source, and the irradiation angle required for the slit lamp to capture the lesion. The invention can quickly capture the eye lesion area and identify the lesion category, provide a basis for the operation of the slit lamp, ensure the imaging effect of the slit lamp, and facilitate the diagnosis of diseases by doctors.

Description

A slit lamp self-adjustment control method and slit lamp based on machine vision

technical field

The invention relates to a slit lamp self-adjustment control method and slit lamp based on machine vision.

Background technique

Patients with ophthalmic diseases need to conduct a comprehensive and in-depth examination of the eyes during ophthalmic treatment to obtain accurate analysis, and treat patients in the way that best meets the needs of the patient. Among the eye tests, the most commonly used is The slit lamp microscope consists of an illumination system and a binocular microscope. It can not only observe the superficial lesions very clearly, but also adjust the focus and the width of the light source to make an "optical section", so that the lesions in the deep tissue can also be observed. It can be clearly displayed, but in the prior art, there are still the following problems:

1) It cannot assist the doctor to determine the location of the lesion and adjust the microscope so that the slit lamp is aligned with the location of the lesion;

2) The adjustment of the slit lamp parameters requires manual adjustment, and the parameters of the slit lamp cannot be adjusted according to the image of the lesion area to determine the type of the lesion and the image of the lesion.

SUMMARY OF THE INVENTION

In view of the defects of the prior art, the present invention provides a slit lamp self-adjustment control method based on machine vision and a slit lamp, which can quickly capture the eye lesion area and identify the lesion type, provide a basis for the operation of the slit lamp, and ensure the Slit lamp imaging effects to facilitate doctors to diagnose diseases.

In order to achieve the above object, the present invention provides a machine vision-based slit lamp self-adjustment control method, which includes the following steps:

1) Using a camera to collect eye images;

2) Compare the collected eye image with the standard data in the database, determine the eye lesion area, and mark the lesion area;

3) Establish a two-dimensional coordinate system with the center of the pupil as the coordinate origin, and obtain the coordinates of the lesion area;

4) Generate manipulation data according to the coordinates of the lesion area, control the slit lamp to obtain the best shooting angle, determine the type of eye lesion according to the lesion image of the marked lesion area, and generate the filter method and projection required for the slit lamp to photograph the lesion The data of the width and illumination angle of the light source;

The process of determining the ocular lesion type of the lesion image in the marked lesion area is as follows:

Select eye images of various types of eye diseases, extract specific information on the lesion images in each lesion area as identification information through an algorithm, and then generate a judgment matrix P that can determine the lesion images corresponding to each type of eye disease (P1, P2... The lesion image is compared with the decision matrix one-to-one to determine the type of eye disease corresponding to the lesion image in the marked lesion area.

As another specific embodiment of the present invention, data extraction is performed on the eye image collected in step 1) to divide the collected eye image into canthus image, pupil image, eyelid image and Image of eyelid area.

As another specific embodiment of the present invention, in step 2), the collected eye image is first subjected to binarization processing, the contrast threshold is preset, and the area where the grayscale change exceeds the preset contrast threshold is marked as lesion area.

As another specific embodiment of the present invention, the process of generating manipulation data according to the coordinates of the lesion area in step 4) is:

Establish a two-dimensional coordinate set f(x, y) of the lesion area according to the acquired coordinates of the lesion area;

A coordinate set adjustment matrix Si (Si1, Si2, Si3, Si4, i=1, 2...n) is established according to the relationship between the position and shooting angle of the microscope and the lesion area, where Si1 represents the coordinate area, and Si2 represents the microscope adjustment X axis To coordinate, Si3 represents the Y-axis coordinate of the microscope adjustment, Si4 represents the microscope adjustment angle;

Determine whether the area represented by the two-dimensional coordinate set f(x, y) of the lesion area exists in the coordinate area Si1, and if so, call the microscope to adjust the X-axis coordinate Si2, the Y-axis coordinate Si3, and the microscope adjustment angle Si4, prompting the microscope The position should be adjusted to the X-axis coordinate Si1, the Y-axis coordinate Si2, and the angle should be adjusted to the microscope adjustment angle Si4.

As another specific embodiment of the present invention, the specific information on the lesion image in the lesion area in step 4) includes contour features and chromaticity features.

As another specific embodiment of the present invention, in step 4), when judging the type of eye lesion according to the lesion image of the marked lesion area, the lesion image is divided into the lesion edge area and the lesion center area, and the lesion edge area and the lesion center area are respectively divided into The contour features and chromaticity features of the central area are compared with the i-th (i=1, 2...n) disease identification information in the judgment matrix P, and the matching area and the non-matching area are recorded to determine the matching rate of the lesion image. Sz:

Among them, S1 represents the contour feature matching rate of the edge area of the lesion, S2 represents the contour feature matching rate of the middle area of the lesion, S3 represents the chromaticity matching rate of the edge area of the lesion, S4 represents the chromaticity matching rate of the center area of the lesion, and α1 is the weight parameter of S1 to Sz, α2 is the weight parameter of S2 to Sz, α3 is the weight parameter of S3 to Sz, and α4 is the weight parameter of S4 to Sz;

Compare the matching rate Sz of the lesion image with the set matching rate parameter S of the eye disease image:

When Sz>S, it is determined that the eye disease image and the eye disease represented by the i-th disease identification information are the same eye disease;

When Sz≤S, it is determined that the eye disease image and the eye disease represented by the i-th disease identification information are different eye diseases.

As another specific embodiment of the present invention, in step 4), the process of generating the data of the filter mode, the width of the projection light source, and the illumination angle required for imaging the lesion by the slit lamp is as follows:

A control matrix Ki (Ki1, Ki2, Ki3, Ki4, i=1, 2...n) is established based on the relationship between the type of eye lesions and the initial filtering method, the width of the initial illumination light source and the initial illumination angle, where Ki1 represents The type of eye lesions, Ki2 represents the initial filtering method, Ki3 represents the width of the initial illumination light source, and Ki4 represents the initial illumination angle. Initial illumination angle.

As another specific embodiment of the present invention, it further includes step 5) collecting the eye image again, and correcting the width and illumination angle of the projection light source based on the clarity and chromaticity of the eye image collected again.

As another specific embodiment of the present invention, the camera device in step 1) includes a wide-angle camera and an infrared camera.

The present invention also provides a slit lamp for realizing the above-mentioned machine vision-based slit lamp self-adjustment control method.

The present invention has the following beneficial effects:

The invention captures image information through the camera device, which is convenient for capturing the eye lesion area, and at the same time, calculates the moving coordinates and irradiation angle when the microscope is aimed at the lesion area, so that the doctor can quickly check the lesion area, and determine the disease type according to the preset algorithm, and correspondingly The filter method of the slit lamp, the width of the projection light source and the irradiation angle are adjusted according to the type of disease to obtain higher image quality, which is convenient for doctors to quickly align and correct the slit lamp and target the lesion area.

In addition, after identifying the lesion area, the invention adjusts the operation data of the slit lamp correspondingly according to the type of the lesion, so as to ensure that the imaging effect of the slit lamp is convenient for doctors to diagnose diseases.

The present invention will be further described in detail below in conjunction with the accompanying drawings.

Description of drawings

Fig. 1 is the schematic flow sheet of the present invention;

Figure 2 is a schematic structural diagram of a slit lamp that can be used in the present invention;

Fig. 3 is the photo that the present invention takes by wide-angle camera;

Fig. 4 is the photo that the present invention takes through infrared camera;

Figure 5 is a photograph of the present invention showing a lesion area.

Detailed ways

In order to understand the above objects, features and advantages of the present invention more clearly, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments may be combined with each other in the case of no conflict.

Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific details disclosed below. Example limitations.

Example 1

This embodiment provides a machine vision-based self-adjustment control method for the slit lamp. As shown in Figures 1-2, the slit lamp has an illumination system and a microscope system. The specific structure of the slit lamp is not limited in this embodiment. Control the multi-parameters of the slit lamp (irradiation angle, light source width, filter method, etc.) to obtain the best use effect.

It includes the following steps:

1) Using a camera to collect eye images;

The camera device includes a wide-angle camera and an infrared camera, and the eye image is collected by the wide-angle camera and the infrared camera, as shown in Figure 3-4 (the color to be displayed in Figure 3 is processed in grayscale), where the wide-angle camera and the infrared camera are used. The cameras work at the same time. The wide-angle camera receives visible light to determine lesions that are easy to observe under visible light, and the infrared camera receives infrared light to determine lesions that are easy to observe under infrared light.

2) Compare the collected eye image with the standard data in the database to determine the eye lesion area, and mark the lesion area;

The standard database is established based on the healthy eye database, and the eye lesion area is obtained through image comparison technology analysis, as shown in Figure 5, where the framed area is the observed lesion area of suspected lesions.

Specifically, after acquiring the collected eye image, first perform binarization processing on the collected eye image, and determine the lesion area by presetting a contrast threshold in the processor, specifically for grayscale changes The area exceeding the preset contrast threshold was marked as the lesion area.

3) Establish a two-dimensional coordinate system with the center of the pupil as the coordinate origin, and obtain the coordinates of the lesion area;

As shown in Figure 5, according to the location of the suspected lesion in the photo, the (X, Y) coordinates of the lesion area can be obtained. At the same time, according to the position difference of the image captured by the wide-angle camera and the infrared camera on the same part, the (X, Y) coordinates of the lesion area can be determined. Z) coordinate.

4) Generate manipulation data according to the coordinates of the lesion area, control the slit lamp to obtain the best shooting angle, determine the type of eye lesion according to the lesion image of the marked lesion area, and generate the filter method and projection required for the slit lamp to photograph the lesion The data of the width and illumination angle of the light source;

The process of generating manipulation data according to the coordinates of the lesion area is as follows:

Establish a two-dimensional coordinate set f(x, y) of the lesion area according to the acquired coordinates of the lesion area;

A coordinate set adjustment matrix Si (Si1, Si2, Si3, Si4, i=1, 2...n) is established according to the relationship between the position and shooting angle of the microscope and the lesion area, where Si1 represents the coordinate area, and Si2 represents the microscope adjustment X axis To coordinate, Si3 represents the Y-axis coordinate of the microscope adjustment, Si4 represents the microscope adjustment angle;

Determine whether the area represented by the two-dimensional coordinate set f(x, y) of the lesion area exists in the coordinate area Si1, and if so, call the microscope to adjust the X-axis coordinate Si2, the Y-axis coordinate Si3, and the microscope adjustment angle Si4, prompting the microscope The position should be adjusted to the X-axis coordinate Si1, the Y-axis coordinate Si2, and the angle should be adjusted to the microscope adjustment angle Si4.

That is to say, after the coordinates of the lesion area are obtained, the X-axis coordinates of the microscope system, which correspond to the coordinates of the lesion area one-to-one, and Y-axis coordinate and adjustment angle data.

The process of determining the ocular lesion type of the lesion image in the marked lesion area is as follows:

Select eye images of various types of eye diseases, extract specific information on the lesion image in each lesion area as identification information through the Ai algorithm, and then generate a judgment matrix that can determine the lesion image corresponding to each type of eye disease P(P1, P2... A one-to-one comparison is performed between the lesion image of the marked lesion area and the decision matrix to determine the type of eye disease corresponding to the lesion image of the marked lesion area.

Specifically, the process of generating the decision matrix P may be:

First select an eye lesion picture of a type of eye disease, extract the characteristic information of the lesion area in the eye lesion picture through the Ai algorithm, generate identification information that can determine this type of disease, and store it;

The above process is repeated to generate identification information of various types of lesion areas, and then generate the decision matrix P corresponding to the lesion images of various eye diseases.

The process of generating the data of the filter method, the width of the projection light source, and the illumination angle required for imaging the lesion by the slit lamp is as follows:

A control matrix Ki (Ki1, Ki2, Ki3, Ki4, i=1, 2...n) is established based on the relationship between the type of eye lesions and the initial filtering method, the width of the initial illumination light source and the initial illumination angle, where Ki1 represents The type of eye lesions, Ki2 represents the initial filtering method, Ki3 represents the width of the initial illumination light source, and Ki4 represents the initial illumination angle. Initial illumination angle.

For example, when the disease is determined to be the first type of disease, the processor can determine that the initial filtering mode is K12, the initial light source width is K13, and the initial illumination angle is K14;

A specific process of the Ai algorithm is:

Transformer and Unet are integrated into a model to achieve image segmentation and lesion recognition in the anterior segment; among them, the encoder module of U-net uses vision transformer to encode; Transformer encodes the tokenized image blocks from the convolutional neural network (CNN) feature map To extract the input sequence of global context; the decoder upsamples the encoded features and combines them with high-resolution CNN feature maps for precise localization.

The specific information on the image of the lesion in the lesion area used in this embodiment is the contour feature and the chromaticity feature. In order to facilitate the image comparison processing, the eye image collected in step 1) The image is divided into canthus image, pupil image, eyelid image and eyelid part image. The controller performs data processing on the image information of the eye lesion area in each type of image, and the direction of the image processing process. Sex is stronger.

Further, in step 4), when judging the type of eye lesion according to the lesion image of the marked lesion area, the lesion image is divided into the lesion edge area and the lesion center area, and the contour features and chromaticity of the lesion edge area and the lesion center area are respectively divided into. The features are compared with the i-th (i=1, 2...n) disease identification information in the judgment matrix P, and the matching area and the non-matching area are recorded to determine the matching rate Sz of the lesion image:

Among them, S1 represents the contour feature matching rate of the edge area of the lesion, S2 represents the contour feature matching rate of the middle area of the lesion, S3 represents the chromaticity matching rate of the edge area of the lesion, S4 represents the chromaticity matching rate of the center area of the lesion, and α1 is the weight parameter of S1 to Sz, α2 is the weight parameter of S2 to Sz, α3 is the weight parameter of S3 to Sz, and α4 is the weight parameter of S4 to Sz;

Compare the matching rate Sz of the lesion image with the set matching rate parameter S of the eye disease image:

When Sz>S, it is determined that the eye disease image and the eye disease represented by the i-th disease identification information are the same eye disease;

When Sz≤S, it is determined that the eye disease image and the eye disease represented by the i-th disease identification information are different eye diseases.

In this embodiment, after determining the initial filtering mode, the width of the initial projection light source, and the initial illumination angle, the method further includes:

Step 5): The eye image is collected again, and the width and illumination angle of the projection light source are corrected based on the clarity and chromaticity of the eye image collected again. A specific correction process is:

Establish a brightness matrix C0 and a light source width adjustment parameter matrix D0;

For the brightness matrix C0, C0 (C1, C2, C3, C4), where C1 is the first preset brightness parameter, C2 is the second preset brightness parameter, C3 is the third preset brightness parameter, and C4 is the fourth preset brightness parameter Brightness parameter, each preset brightness parameter increases in sequence;

For the light source width adjustment parameter matrix D0, D0 (D1, D2, D3, D4), D1 is the first preset light source width adjustment parameter, D2 is the second preset light source width adjustment parameter, D3 is the third preset light source Width adjustment parameter, D4 is the fourth preset light source width adjustment parameter;

The processor detects the brightness Ci of the ith area of the eye image (the ith area can be any area, in most cases, it refers to the lesion area or the area suspected of having lesions) and compares Ci with the parameters in the C0 matrix:

When Ci≤C1, it is determined that the brightness of the i-th area is insufficient and D1 is selected from the D0 matrix as the light source width adjustment parameter;

When C1<Ci≤C2, it is determined that the brightness of the i-th area is insufficient and D2 is selected from the D0 matrix as the light source width adjustment parameter;

When C2<Ci≤C3, it is determined that the brightness of the i-th area is in a standard state;

When C3<Ci≤C4, it is determined that the brightness of the i-th area is too high and D3 is selected from the D0 matrix as the light source width adjustment parameter;

When Ci>C4, it is determined that the ith region of brightness is too high and D4 is selected from the D0 matrix as the light source width adjustment parameter;

When it is determined that the brightness of the ith area is not in the standard state (the standard state is the state of C2<Ci≤C3), adjust the width of the light source in the ith area to Ci';

When it is determined that the width of the light source in the i-th area is insufficient, Ci'=Ci+(C3-Ci)×Dj, j=1, 2;

When it is determined that the width of the light source in the i-th area is too high, Ci'=Ci-(Ci-C2)×Dr, r=3,4;

When the width of the light source in the i-th area is adjusted to Ci', compare Ci' with the parameters in the C0 matrix:

When C2<Ci'≤C3, it is determined that the brightness of the i-th area is in a standard state;

When Ci' is not in C2 to C3, repeat the above operation until C2<Ci'≤C3.

Although the present invention is disclosed above with preferred embodiments, it is not intended to limit the scope of implementation of the present invention. Any person of ordinary skill in the art can make some improvements without departing from the scope of the present invention, that is, all equivalent improvements made according to the present invention should be covered by the scope of the present invention.

Claims (10)

1. A slit lamp self-adjustment control method based on machine vision, comprising the following steps: 1) Using a camera to collect eye images; 2) Compare the collected eye image with the standard data in the database, determine the eye lesion area, and mark the lesion area; 3) Establish a two-dimensional coordinate system with the center of the pupil as the coordinate origin, and obtain the coordinates of the lesion area; 4) Generate manipulation data according to the coordinates of the lesion area, control the slit lamp to obtain the best shooting angle, determine the type of eye lesion according to the lesion image of the marked lesion area, and generate the filter method and projection required for the slit lamp to photograph the lesion The data of the width and illumination angle of the light source; The process of determining the ocular lesion type of the lesion image in the marked lesion area is as follows: Select eye images of various types of eye diseases, extract specific information on the lesion images in each lesion area as identification information through an algorithm, and then generate a judgment matrix P that can determine the lesion images corresponding to each type of eye disease (P1, P2... The lesion image is compared with the decision matrix one-to-one to determine the type of eye disease corresponding to the lesion image in the marked lesion area. 2. The slit lamp self-adjustment control method based on machine vision as claimed in claim 1, wherein, data extraction is performed on the eye image collected in step 1) to divide the collected eye image into canthus images , eye pupil image, eyelid image and eyelid part image. 3. The slit lamp self-adjustment control method based on machine vision as claimed in claim 1, wherein, in step 2), the collected eye image is first subjected to binarization processing, and a preset contrast threshold is used for grayscale changes. The area exceeding the preset contrast threshold was marked as the lesion area. 4. The slit lamp self-adjustment control method based on machine vision as claimed in claim 1, wherein, in step 4), the process of generating manipulation data according to the coordinates of the lesion area is: Establish a two-dimensional coordinate set f(x, y) of the lesion area according to the acquired coordinates of the lesion area; A coordinate set adjustment matrix Si (Si1, Si2, Si3, Si4, i=1, 2...n) is established according to the relationship between the position and shooting angle of the microscope and the lesion area, where Si1 represents the coordinate area and Si2 represents the microscope adjustment X axis To coordinate, Si3 represents the Y-axis coordinate of the microscope adjustment, Si4 represents the microscope adjustment angle; Determine whether the area represented by the two-dimensional coordinate set f(x, y) of the lesion area exists in the coordinate area Si1, and if so, call the microscope to adjust the X-axis coordinate Si2, the Y-axis coordinate Si3, and the microscope adjustment angle Si4, prompting the microscope The position should be adjusted to the X-axis coordinate Si1, the Y-axis coordinate Si2, and the angle should be adjusted to the microscope adjustment angle Si4. 5. The machine vision-based slit lamp self-adjustment control method according to claim 1, wherein the specific information on the lesion image in the lesion area in step 4) includes contour features and chromaticity features. 6. The slit lamp self-adjustment control method based on machine vision as claimed in claim 5, wherein, in step 4), when judging the eye lesion type according to the lesion image of the marked lesion area, the lesion image is divided into the lesion edge area and the In the central area of the lesion, the contour features and chromaticity features of the lesion edge area and the central area of the lesion were compared with the identification information of the i-th (i=1, 2...n) type of disease in the judgment matrix P, and the anastomotic area was recorded. Non-matching area to determine the matching rate Sz of the lesion image:

Among them, S1 represents the contour feature matching rate of the edge area of the lesion, S2 represents the contour feature matching rate of the middle area of the lesion, S3 represents the chromaticity matching rate of the edge area of the lesion, S4 represents the chromaticity matching rate of the center area of the lesion, and α1 is the weight parameter of S1 to Sz, α2 is the weight parameter of S2 to Sz, α3 is the weight parameter of S3 to Sz, and α4 is the weight parameter of S4 to Sz; Compare the matching rate Sz of the lesion image with the set matching rate parameter S of the eye disease image: When Sz>S, it is determined that the eye disease image and the eye disease represented by the i-th disease identification information are the same eye disease; When Sz≤S, it is determined that the eye disease image and the eye disease represented by the i-th disease identification information are different eye diseases. 7. The slit lamp self-adjustment control method based on machine vision as claimed in claim 6, wherein in step 4), the filter mode, the width of the projection light source, and the data of the irradiation angle required when the slit lamp is photographed for the lesion are generated. The process is: A control matrix Ki (Ki1, Ki2, Ki3, Ki4, i=1, 2...n) is established based on the relationship between the type of eye lesions and the initial filtering method, the width of the initial illumination light source and the initial illumination angle, where Ki1 represents The type of eye lesions, Ki2 represents the initial filtering method, Ki3 represents the width of the initial illumination light source, and Ki4 represents the initial illumination angle. Initial illumination angle. 8. The slit lamp self-adjustment control method based on machine vision as claimed in claim 7, further comprising step 5) collection of eye images again, based on the sharpness and chromaticity of the re-collected eye images for projection. The width and illumination angle of the light source are corrected. 9. The machine vision-based slit lamp self-adjustment control method according to claim 1, wherein the camera device in step 1) comprises a wide-angle camera and an infrared camera. 10. A slit lamp implementing the method for self-adjusting control of a slit lamp based on machine vision according to any one of claims 1-9.

Images