CN118266854B - Stereoscopic vision detection control system and method - Google Patents

Abstract

The present invention discloses a stereoscopic vision detection control system and method, and relates to the field of stereoscopic vision detection technology. The method comprises: after semantic encoding the acquired text description of vision ability and the text description of vision test requirements, a semantically enhanced vision ability semantic representation vector and a semantically enhanced vision test requirement semantic representation vector are obtained through an important content attention integration network; based on the semantic fusion representation between the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector, a recommendation result of a stereoscopic electronic sight mark is determined. In this way, by using a stereoscopic electronic sight mark and establishing an intelligent adjustment control scheme for the stereoscopic electronic sight mark, the consumption of paper materials can be reduced, and a more convenient and personalized inspection experience can be provided.

Description

A stereoscopic vision detection control system and method

Technical Field

The present invention relates to the technical field of stereoscopic vision detection, and in particular to a stereoscopic vision detection control system and method.

Background technique

Stereoscopic vision is a high-level visual function unique to the human body and an important component of binocular vision. It is of great significance for judging the binocular vision function and treatment of patients with strabismus and amblyopia, and is one of the important examination items in ophthalmology. At present, most clinical examinations of stereoscopic vision are performed through close-range stereoscopic examinations using printed paper stereoscopic examination albums.

However, paper-printed random dot stereopsis examination charts have disadvantages such as single color in terms of brightness, color, shape, etc., and are not suitable for clinical examination of patients with low vision such as amblyopia. Therefore, a solution is expected.

Summary of the invention

This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

In a first aspect, the present invention provides a stereoscopic vision detection control method, the method comprising:

Obtain a text description of the customer's vision ability and a text description of the vision test requirements;

Semantically encoding the vision ability text description and the vision test requirement text description respectively to obtain a sequence of vision ability word granularity semantic encoding feature vectors and a sequence of vision test requirement word granularity semantic encoding feature vectors;

The sequence of the granular semantic encoding feature vectors of the vision ability words and the sequence of the granular semantic encoding feature vectors of the vision test requirement words are passed through an important content attention integration network to obtain a semantically enhanced vision ability semantic representation vector and a semantically enhanced vision test requirement semantic representation vector;

Based on the semantic fusion representation between the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector, the recommendation result of the stereoscopic electronic sight mark is determined.

Optionally, the text description of vision ability and the text description of vision test requirements are semantically encoded respectively to obtain a sequence of semantically encoded feature vectors of vision ability word granularity and a sequence of semantically encoded feature vectors of vision test requirement word granularity, including: performing word segmentation processing on the text description of vision ability and then passing it through a semantic encoder including a word embedding layer to obtain a sequence of semantically encoded feature vectors of vision ability word granularity; performing word segmentation processing on the text description of vision test requirements and then passing it through the semantic encoder including a word embedding layer to obtain a sequence of semantically encoded feature vectors of vision test requirement word granularity.

Optionally, after word segmentation, the text description of the vision ability is passed through a semantic encoder including a word embedding layer to obtain a sequence of semantic encoding feature vectors of the vision ability word granularity, including: word segmentation of the text description of the vision ability to convert the text description of the vision ability into a word sequence composed of multiple words; using the word embedding layer of the semantic encoder including the word embedding layer to map each word in the word sequence to a word vector to obtain a sequence of word vectors; using the semantic encoder including the word embedding layer to perform global contextual semantic encoding on the sequence of word vectors to obtain a sequence of semantic encoding feature vectors of the vision ability word granularity.

Optionally, the sequence of the vision ability word granularity semantic encoding feature vectors and the sequence of the vision test requirement word granularity semantic encoding feature vectors are passed through an important content attention integration network to obtain a semantically enhanced vision ability semantic representation vector and a semantically enhanced vision test requirement semantic representation vector, including: calculating the significance coefficient of each vision ability word granularity semantic encoding feature vector in the sequence of the vision ability word granularity semantic encoding feature vectors to obtain a sequence of significance coefficients; normalizing the sequence of significance coefficients to obtain a sequence of autocorrelation significance coefficients; using the sequence of autocorrelation significance coefficients as weights, calculating the vector-by-vector weighted sum of the sequence of the vision ability word granularity semantic encoding feature vectors to obtain the semantically enhanced vision ability semantic representation vector.

Optionally, the significance coefficient of each vision ability word granularity semantic coding feature vector in the sequence of vision ability word granularity semantic coding feature vectors is calculated to obtain a sequence of significance coefficients, including: processing each vision ability word granularity semantic coding feature vector in the sequence of vision ability word granularity semantic coding feature vectors through a fully connected layer to obtain a sequence of vision ability word granularity semantic fully connected coding feature vectors; multiplying each vision ability word granularity semantic coding feature vector in the sequence of vision ability word granularity semantic fully connected coding feature vectors with the transposed vector of the corresponding weight coefficient vector to obtain the sequence of significance coefficients.

Optionally, the fully connected layer uses a Selu activation function.

Optionally, normalizing the sequence of significance coefficients to obtain a sequence of autocorrelation significance coefficients includes: performing normalization processing based on a softmax activation function on the sequence of significance coefficients to obtain the sequence of autocorrelation significance coefficients.

Optionally, based on the semantic fusion representation between the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector, a recommendation result of the stereoscopic electronic sight mark is determined, including: passing the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector through a CLIP model-based vision ability-vision test requirement collaborative encoder to obtain a vision ability-vision test requirement collaborative semantic representation matrix; passing the vision ability-vision test requirement through a decoder-based stereoscopic electronic sight mark parameter recommender to obtain the recommendation result, and the recommendation result includes the color value and brightness value of the stereoscopic electronic sight mark.

In a second aspect, the present invention provides a stereoscopic vision detection control system, the system comprising:

A text description acquisition module, used to acquire a text description of the vision ability of a customer object and a text description of vision test requirements;

A semantic encoding module, used for performing semantic encoding on the vision ability text description and the vision test requirement text description respectively to obtain a sequence of vision ability word granularity semantic encoding feature vectors and a sequence of vision test requirement word granularity semantic encoding feature vectors;

An important content attention integration module, used for passing the sequence of the granular semantic encoding feature vectors of the vision ability words and the sequence of the granular semantic encoding feature vectors of the vision test requirement words through an important content attention integration network to obtain a semantically enhanced vision ability semantic representation vector and a semantically enhanced vision test requirement semantic representation vector;

The semantic fusion module is used to determine the recommendation result of the stereoscopic electronic sight mark based on the semantic fusion representation between the semantic enhancement vision ability semantic representation vector and the semantic enhancement vision test requirement semantic representation vector.

Optionally, the semantic encoding module includes: a word segmentation processing unit, used for performing word segmentation processing on the text description of vision ability and then passing it through a semantic encoder including a word embedding layer to obtain a sequence of semantic encoding feature vectors of the vision ability word granularity; a semantic encoding unit, used for performing word segmentation processing on the text description of vision test requirements and then passing it through the semantic encoder including a word embedding layer to obtain a sequence of semantic encoding feature vectors of the vision test requirement word granularity.

The above technical solution is adopted to obtain the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector through the important content attention integration network after semantic encoding of the obtained vision ability text description and the vision test requirement text description; based on the semantic fusion representation between the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector, the recommendation result of the stereoscopic electronic sight mark is determined. In this way, by using the stereoscopic electronic sight mark and establishing an intelligent adjustment and control scheme for the stereoscopic electronic sight mark, the consumption of paper materials can be reduced, providing a more convenient and personalized inspection experience.

Other features and advantages of the present invention will be described in detail in the following detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:

Fig. 1 is a flow chart showing a stereoscopic vision detection control method according to an exemplary embodiment.

Fig. 2 is a block diagram showing a stereoscopic vision detection control system according to an exemplary embodiment.

Fig. 3 is a block diagram of an electronic device according to an exemplary embodiment.

Fig. 4 is a diagram showing an application scenario of a stereoscopic vision detection control method according to an exemplary embodiment.

Detailed ways

Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and/or in parallel. In addition, the method embodiments may include additional steps and/or omit the steps shown. The scope of the present invention is not limited in this respect.

The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

It should be noted that the concepts such as "first" and "second" mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

Stereoscopic vision is an advanced visual function that enables people to perceive the depth and spatial relationship of objects, so as to make accurate positioning and judgment in daily life. Stereoscopic vision is an important part of binocular vision. It relies on the coordinated work of both eyes. By comparing the slightly different images received by the two eyes, the brain can synthesize a three-dimensional image with a sense of depth.

Stereoscopic vision plays a vital role in clinical ophthalmology examinations, especially in the evaluation and treatment of patients with strabismus and amblyopia. Strabismus is the inability of the eyes to focus on the same target, while amblyopia is the subnormal vision of one or both eyes that cannot be improved by corrective lenses. Stereoscopic vision examinations help doctors understand the patient's binocular vision function, determine whether there is a stereoscopic vision disorder, and develop appropriate treatment plans accordingly.

At present, one of the commonly used methods for clinical stereopsis examination is to use paper stereopsis examination albums. These albums usually contain random dot patterns, and patients can perceive the stereoscopic images hidden in the patterns through special glasses or naked eye observation. However, this method has some limitations. Paper-printed albums may not be rich enough in brightness, color, and shape, which may result in a single color and affect the accuracy of the examination. Especially for patients with low vision such as amblyopia, this examination method may not be suitable because their visual system is less sensitive to details and colors.

To overcome these limitations, ophthalmologists and researchers are exploring more advanced stereoscopic examination methods. For example, digital stereoscopic examination systems can provide more diverse images and richer colors to adapt to the vision levels of different patients. In addition, the application of virtual reality (VR) technology also shows potential in stereoscopic examinations, which can provide an immersive stereoscopic visual experience and help to more accurately assess patients' stereoscopic vision ability.

In order to solve the above problems, the present invention provides a stereoscopic vision detection control system and method, which respectively semantically encodes the obtained text description of vision ability and the text description of vision test requirements, and then obtains a semantically enhanced vision ability semantic representation vector and a semantically enhanced vision test requirement semantic representation vector through an important content attention integration network; based on the semantic fusion representation between the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector, the recommendation result of the stereoscopic electronic sight mark is determined. In this way, by using stereoscopic electronic sight marks and establishing an intelligent adjustment control scheme for stereoscopic electronic sight marks, the consumption of paper materials can be reduced, providing a more convenient and personalized inspection experience.

The specific implementation modes of the present invention are described in detail below with reference to the accompanying drawings.

FIG. 1 is a flow chart of a stereoscopic vision detection control method according to an exemplary embodiment. As shown in FIG. 1 , the method includes:

S101. Obtain a text description of the vision ability and vision test requirements of the customer object;

S102, semantically encoding the vision ability text description and the vision test requirement text description respectively to obtain a sequence of vision ability word granularity semantic encoding feature vectors and a sequence of vision test requirement word granularity semantic encoding feature vectors;

S103, passing the sequence of the granular semantic encoding feature vectors of the vision ability words and the sequence of the granular semantic encoding feature vectors of the vision test requirement words through an important content attention integration network to obtain a semantically enhanced vision ability semantic representation vector and a semantically enhanced vision test requirement semantic representation vector;

S104. Determine a recommendation result of a stereoscopic electronic sight mark based on a semantic fusion representation between the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector.

In response to the above technical problems, the technical concept of this application is to capture the customer's vision and test requirements from the text description of the customer's vision ability and the text description of the vision test requirements through text semantic understanding and semantic analysis based on natural language processing technology, and then use deep learning algorithms to enhance and integrate the key semantic information of the two to form a semantic collaborative correlation feature representation of the customer's vision and vision test requirements, and based on this, to achieve intelligent recommendation of the color value and brightness value of the stereoscopic electronic sight mark. In this way, by using stereoscopic electronic sight marks and establishing an intelligent adjustment and control scheme for stereoscopic electronic sight marks, the consumption of paper materials can be reduced, providing a more convenient and personalized inspection experience.

Based on this, in the technical solution of the present application, first, a text description of the customer's vision ability and a text description of the vision test requirements are obtained. It should be understood that the vision condition and test requirements of each customer (patient) are unique. Specifically, the text description of vision ability can provide the patient's current level of visual function, including vision, visual field, stereoscopic vision ability, etc., which helps the system understand the patient's vision condition. The text description of vision test requirements involves the specific type of test that the patient needs to undergo, such as close or long-distance stereoscopic vision tests, and the difficulty level of the test. This helps to customize and personalize the test content.

In one embodiment of the present invention, the vision ability text description and the vision test requirement text description are semantically encoded respectively to obtain a sequence of vision ability word granularity semantic encoding feature vectors and a sequence of vision test requirement word granularity semantic encoding feature vectors, including: performing word segmentation processing on the vision ability text description and then passing it through a semantic encoder including a word embedding layer to obtain a sequence of vision ability word granularity semantic encoding feature vectors; performing word segmentation processing on the vision test requirement text description and then passing it through the semantic encoder including a word embedding layer to obtain a sequence of vision test requirement word granularity semantic encoding feature vectors.

Then, after word segmentation processing is performed on the text description of vision ability, a sequence of semantically encoded feature vectors of the word granularity of vision ability is obtained through a semantic encoder including a word embedding layer; and after word segmentation processing is performed on the text description of vision test requirements, a sequence of semantically encoded feature vectors of the word granularity of vision test requirements is obtained through a semantic encoder including a word embedding layer. Specifically, in the technical solution of the present application, the continuous text characters in the text description of vision ability and the text description of vision test requirements can be segmented into meaningful units, such as words or phrases, respectively, through word segmentation processing, so that the subsequent model can perform semantic understanding based on the basic structure in the text description; the word embedding layer can embed and transform each word or phrase obtained after word segmentation of the text description of vision ability and the text description of vision test requirements, so as to convert them into structured vector representations; and then the semantic encoder is used to capture the semantic features and text content in the text description of vision ability and the text description of vision test requirements, respectively.

Furthermore, in one embodiment of the present invention, after word segmentation, the text description of the vision ability is passed through a semantic encoder including a word embedding layer to obtain a sequence of semantic encoding feature vectors of the vision ability word granularity, including: word segmentation of the text description of the vision ability to convert the text description of the vision ability into a word sequence composed of multiple words; using the word embedding layer of the semantic encoder including the word embedding layer to map each word in the word sequence to a word vector to obtain a sequence of word vectors; using the semantic encoder including the word embedding layer to perform global context-based semantic encoding on the sequence of word vectors to obtain a sequence of semantic encoding feature vectors of the vision ability word granularity.

However, since the semantic features expressed by different words or phrases in the text description of vision ability and the text description of vision test requirements usually have different importance and influence on the understanding of the context. In view of this feature, in the technical solution of the present application, it is expected that the sequence of the granular semantic encoding feature vectors of the vision ability words and the sequence of the granular semantic encoding feature vectors of the vision test requirements words are integrated through an important content attention network to manifest the key semantic information in the text description, thereby obtaining a semantically enhanced vision ability semantic representation vector and a semantically enhanced vision test requirement semantic representation vector.

In an embodiment of the present application, the process of passing the sequence of the vision ability word granularity semantic encoding feature vectors and the sequence of the vision test requirement word granularity semantic encoding feature vectors through an important content attention integration network to obtain a semantically enhanced vision ability semantic representation vector and a semantically enhanced vision test requirement semantic representation vector includes: first calculating the significance coefficient of each vision ability word granularity semantic encoding feature vector in the sequence of the vision ability word granularity semantic encoding feature vector to obtain a sequence of significance coefficients; then, normalizing the sequence of significance coefficients to obtain a sequence of autocorrelation significance coefficients; and then using the sequence of autocorrelation significance coefficients as weights, calculating the vector-by-vector weighted sum of the sequence of the vision ability word granularity semantic encoding feature vectors to obtain the semantically enhanced vision ability semantic representation vector. That is, the full-text semantic autocorrelation in the sequence of the vision ability word granularity semantic encoding feature vectors is captured through the important content attention integration network, and the importance and influence of each vision ability word granularity semantic encoding feature vector on context understanding is quantified based on this full-text semantic autocorrelation.

Furthermore, in one embodiment of the present invention, the significance coefficient of each vision ability word granularity semantic coding feature vector in the sequence of the vision ability word granularity semantic coding feature vector is calculated to obtain a sequence of significance coefficients, including: processing each vision ability word granularity semantic coding feature vector in the sequence of the vision ability word granularity semantic coding feature vector through a fully connected layer to obtain a sequence of vision ability word granularity semantic fully connected coding feature vectors; multiplying each vision ability word granularity semantic coding feature vector in the sequence of the vision ability word granularity semantic fully connected coding feature vector with the transposed vector of the corresponding weight coefficient vector to obtain the sequence of significance coefficients.

Furthermore, in one embodiment of the present invention, the fully connected layer uses a Selu activation function.

Furthermore, in one embodiment of the present invention, the sequence of significance coefficients is normalized to obtain a sequence of autocorrelation significance coefficients, including: performing normalization processing based on a softmax activation function on the sequence of significance coefficients to obtain the sequence of autocorrelation significance coefficients.

Specifically, in one embodiment of the present invention, the sequence of the granular semantic encoding feature vectors of the vision ability words and the sequence of the granular semantic encoding feature vectors of the vision test requirement words are integrated through an important content attention integration network to obtain a semantically enhanced vision ability semantic representation vector and a semantically enhanced vision test requirement semantic representation vector; wherein the integration formula is:

;

;

;

in, For the The significance coefficient, For the A weight coefficient matrix, For the The transposed vector of the weight coefficient vector, For the A bias vector, represents the Selu activation function, is the first in the sequence of the granular semantic encoding feature vectors of the vision ability word The granular semantic encoding feature vector of vision ability words, and are hyperparameters with different values, represents the value input into the Selu activation function, express Activation function, For the The autocorrelation significance coefficient is is the length of the sequence of the granular semantic encoding feature vectors of the vision ability word, A vision-enhanced semantic representation vector is provided for the semantics.

It should be understood that the process of calculating the significance coefficient is to learn the text semantic features of each vision ability word granular semantic coding feature vector through a fully connected layer, more specifically, to automatically learn this text semantic feature through a (trained) weight coefficient matrix, a (trained) weight coefficient vector and a (trained) bias vector. And it is worth mentioning that the ordinary fully connected layer usually does not have a weight coefficient vector. In the technical solution of the present application, the purpose of providing a weight coefficient vector is to numerically represent the text semantic features of each vision ability word granular semantic coding feature vector, that is, each significance coefficient. Then, the sequence of the significance coefficients is normalized based on the softmax activation function to quantify the importance and overall relevance of the significance coefficients corresponding to each vision ability word granular semantic coding feature vector, that is, the full text semantic autocorrelation. That is to say, in the technical solution of the present application, the essence of the important content attention integration network is a weight probability distribution mechanism, that is, to assign greater weights to important content and reduce weights to other content. Such a mechanism is more focused on finding useful information in the input data that is significantly related to the current data.

Subsequently, the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector are passed through a vision ability-vision test requirement collaborative encoder based on the CLIP model to obtain a vision ability-vision test requirement collaborative semantic representation matrix. That is, the semantic information of the two is integrated through the vision ability-vision test requirement collaborative encoder based on the CLIP model to ensure their consistency in the semantic space and form a comprehensive representation that can fully reflect the patient's vision condition and test requirements. Further, the vision ability-vision test requirement is passed through a decoder-based stereoscopic electronic sight mark parameter recommender to obtain a recommendation result, and the recommendation result includes the color value and brightness value of the stereoscopic electronic sight mark.

In one embodiment of the present invention, a recommendation result of a stereoscopic electronic sight mark is determined based on the semantic fusion representation between the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector, including: passing the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector through a CLIP model-based vision ability-vision test requirement collaborative encoder to obtain a vision ability-vision test requirement collaborative semantic representation matrix; passing the vision ability-vision test requirement through a decoder-based stereoscopic electronic sight mark parameter recommender to obtain the recommendation result, and the recommendation result includes the color value and brightness value of the stereoscopic electronic sight mark.

Preferably, the visual ability-visual test requirement collaborative semantic representation matrix is passed through a decoder-based stereoscopic electronic sight mark parameter recommender to obtain a recommendation result, comprising the following steps:

Expanding the vision ability-vision test requirement collaborative semantic representation matrix to obtain a vision ability-vision test requirement collaborative semantic representation vector;

Calculate the point sum of the vision ability-vision test requirement collaborative semantic representation vector and the square root of its length and the reciprocal of the square root of its second norm to obtain a first vision ability-vision test requirement collaborative semantic representation intermediate vector;

Calculating an exponential function with a natural constant as the base of the intermediate vector of the first vision ability-vision test requirement collaborative semantic representation to obtain a second vision ability-vision test requirement collaborative semantic representation intermediate vector;

Calculate the dot product of the vision ability-vision test requirement collaborative semantic representation vector and its norm and weight hyperparameter to obtain a third vision ability-vision test requirement collaborative semantic representation intermediate vector;

Calculating the point sum of the second vision ability-vision test requirement collaborative semantic representation intermediate vector and the third vision ability-vision test requirement collaborative semantic representation intermediate vector to obtain an optimized vision ability-vision test requirement collaborative semantic representation vector;

Converting the optimized vision ability-vision test requirement collaborative semantic representation vector into an optimized vision ability-vision test requirement collaborative semantic representation matrix based on the expansion of the vision ability-vision test requirement collaborative semantic representation matrix;

The optimized vision ability-vision test requirement collaborative semantic representation matrix is passed through a decoder-based stereoscopic electronic sight mark parameter recommender to obtain a recommendation result.

Specifically, considering the differences in semantic features encoded by word embeddings caused by the source semantic differences between the text description of vision ability and the text description of vision test requirements, after the integration of important content attention, the semantic feature distributions of the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision test requirement semantic representation vector will be significantly misaligned, affecting the decoding regression constraints of the vision ability-vision test requirement collaborative semantic representation matrix obtained by bit-by-bit semantic association. In the above preferred example, the structured norm representation of the vision ability-vision test requirement collaborative semantic representation vector after the vision ability-vision test requirement collaborative semantic representation matrix is expanded is used as the local canonical coordinates for each eigenvalue of the vision ability-vision test requirement collaborative semantic representation matrix to determine the overall distribution of the characteristics of the vision ability-vision test requirement collaborative semantic representation matrix, and the rotation offset of the eigenvalue represents the predicted direction of the offset of each eigenvalue of the vision ability-vision test requirement collaborative semantic representation matrix as the center, and the bounding box of the eigenvalue distribution of the vision ability-vision test requirement collaborative semantic representation matrix is used to perform eigenvalue constraints to improve the constraint of the vision ability-vision test requirement collaborative semantic representation matrix under the overall decoding distribution regression, thereby improving the training speed of the model and the accuracy of the recommendation results obtained by the decoder-based stereoscopic electronic sight mark parameter recommender for the vision ability-vision test requirement.

In summary, the above scheme is adopted to capture the customer's vision and test requirements from the text description of the customer's vision ability and the text description of the vision test requirements through text semantic understanding and semantic analysis based on natural language processing technology, and then use the deep learning algorithm to enhance and integrate the key semantic information of the two to form a semantic collaborative correlation feature representation of the customer's vision and vision test requirements, and based on this, realize the intelligent recommendation of the color value and brightness value of the stereoscopic electronic sight mark. In this way, by using stereoscopic electronic sight marks and establishing an intelligent adjustment control scheme for stereoscopic electronic sight marks, the consumption of paper materials can be reduced, providing a more convenient and personalized inspection experience.

Fig. 2 is a block diagram of a stereoscopic vision detection control system according to an exemplary embodiment. As shown in Fig. 2, the system 200 includes:

A text description acquisition module 201 is used to acquire a text description of the vision ability and a text description of the vision test requirement of a customer object;

A semantic encoding module 202, used to perform semantic encoding on the vision ability text description and the vision test requirement text description to obtain a sequence of vision ability word granularity semantic encoding feature vectors and a sequence of vision test requirement word granularity semantic encoding feature vectors;

An important content attention integration module 203 is used to pass the sequence of the granular semantic encoding feature vectors of the vision ability words and the sequence of the granular semantic encoding feature vectors of the vision test requirement words through an important content attention integration network to obtain a semantically enhanced vision ability semantic representation vector and a semantically enhanced vision test requirement semantic representation vector;

The semantic fusion module 204 is used to determine the recommendation result of the stereoscopic electronic sight mark based on the semantic fusion representation between the semantic enhancement vision ability semantic representation vector and the semantic enhancement vision test requirement semantic representation vector.

In one embodiment of the present invention, the semantic encoding module includes: a word segmentation processing unit, which is used to perform word segmentation processing on the text description of vision ability and then pass it through a semantic encoder including a word embedding layer to obtain a sequence of semantic encoding feature vectors of the vision ability word granularity; a semantic encoding unit, which is used to perform word segmentation processing on the text description of vision test requirements and then pass it through the semantic encoder including a word embedding layer to obtain a sequence of semantic encoding feature vectors of the vision test requirement word granularity.

Referring to FIG3 below, a block diagram of an electronic device 600 suitable for implementing an embodiment of the present invention is shown. The terminal device in the embodiment of the present invention may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG3 is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

As shown in FIG3 , the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to the bus 604.

Typically, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609. The communication device 609 may allow the electronic device 600 to communicate wirelessly or wired with other devices to exchange data. Although FIG. 3 shows an electronic device 600 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present invention are executed.

It should be noted that the computer-readable medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperTextTransferProtocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

Computer program code for performing the operations of the present invention may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The modules involved in the embodiments of the present invention may be implemented by software or hardware. The name of a module does not limit the module itself in some cases. For example, a test parameter acquisition module may also be described as a "module for acquiring device test parameters corresponding to a target device".

The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Fig. 4 is an application scenario diagram of a stereoscopic vision detection control method according to an exemplary embodiment. As shown in Fig. 4, in this application scenario, first, a text description of the visual ability of the customer object (for example, C1 shown in Fig. 4) and a text description of the visual test requirements (for example, C2 shown in Fig. 4) are obtained; then, the obtained text description of the visual ability and the text description of the visual test requirements are input into a server (for example, S shown in Fig. 4) deployed with a stereoscopic vision detection control algorithm, wherein the server can process the text description of the visual ability and the text description of the visual test requirements based on the stereoscopic vision detection control algorithm to determine the recommendation result of the stereoscopic electronic sight mark.

The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) by each other.

In addition, although each operation is described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present invention. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

Although the subject matter has been described in language specific to structural features and/or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims (9)

1. A stereoscopic vision detection control method, characterized by comprising:

Acquiring a vision ability text description and a vision test requirement text description of a client object;

Semantic coding is carried out on the vision ability text description and the vision testing requirement text description respectively to obtain a sequence of vision ability word granularity semantic coding feature vectors and a sequence of vision testing requirement word granularity semantic coding feature vectors;

The sequences of the vision ability word granularity semantic coding feature vectors and the sequences of the vision test requirement word granularity semantic coding feature vectors are integrated through an important content attention integration network to obtain semantic enhancement vision ability semantic representation vectors and semantic enhancement vision test requirement semantic representation vectors;

Determining a recommendation result of the stereoscopic electronic optotype based on a semantic fusion representation between the semantic enhanced vision ability semantic representation vector and the semantic enhanced vision testing requirement semantic representation vector;

Wherein determining a recommendation of the stereoscopic electronic optotype based on the semantic fusion representation between the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision testing requirement semantic representation vector comprises:

Passing the semantic enhanced vision ability semantic representation vector and the semantic enhanced vision testing requirement semantic representation vector through a vision ability-vision testing requirement cooperative encoder based on a CLIP model to obtain a vision ability-vision testing requirement cooperative semantic representation matrix;

Passing the vision ability-vision testing requirement through a decoder-based stereoscopic electronic optotype parameter recommender to obtain the recommendation, the recommendation comprising a color value and a brightness value of the stereoscopic electronic optotype.

2. The stereoscopic vision detection control method according to claim 1, wherein semantically encoding the vision ability text description and the vision testing requirement text description to obtain a sequence of vision ability word granularity semantically encoded feature vectors and a sequence of vision testing requirement word granularity semantically encoded feature vectors, respectively, comprises:

After word segmentation is carried out on the vision ability text description, a semantic encoder containing a word embedding layer is used for obtaining a sequence of vision ability word granularity semantic coding feature vectors;

And after word segmentation processing is carried out on the vision testing requirement text description, a sequence of the vision testing requirement word granularity semantic coding feature vector is obtained through the semantic encoder comprising the word embedding layer.

3. The stereoscopic vision detection control method according to claim 2, wherein the sequence of vision ability word granularity semantic coding feature vectors is obtained by a semantic encoder including a word embedding layer after the segmentation processing is performed on the vision ability text description, comprising:

Word segmentation processing is carried out on the vision ability text description so as to convert the vision ability text description into a word sequence composed of a plurality of words;

mapping each word in the word sequence to a word vector using a word embedding layer of the semantic encoder including the word embedding layer to obtain a sequence of word vectors;

And performing global-based context semantic coding on the sequence of word vectors by using the semantic coder comprising the word embedding layer to obtain the sequence of vision ability word granularity semantic coding feature vectors.

4. A stereoscopic vision detection control method according to claim 3, wherein passing the sequence of vision ability word granularity semantic coding feature vectors and the sequence of vision testing requirement word granularity semantic coding feature vectors through an important content attention integration network to obtain a semantic enhanced vision ability semantic representation vector and a semantic enhanced vision testing requirement semantic representation vector comprises:

calculating the saliency coefficient of each vision ability word granularity semantic coding feature vector in the sequence of the vision ability word granularity semantic coding feature vectors to obtain a sequence of saliency coefficients;

Normalizing the sequence of saliency coefficients to obtain a sequence of autocorrelation saliency coefficients;

And calculating a vector-by-vector weighted sum of the sequence of the vision ability word granularity semantic coding feature vectors by taking the sequence of the autocorrelation significance coefficients as weights so as to obtain the semantic enhancement vision ability semantic representation vector.

5. The stereoscopic vision detection control method according to claim 4, wherein calculating the saliency coefficient of each vision ability word granularity semantic coding feature vector in the sequence of vision ability word granularity semantic coding feature vectors to obtain the sequence of saliency coefficients comprises:

processing all vision ability word granularity semantic coding feature vectors in the vision ability word granularity semantic coding feature vector sequence through a full-connection layer to obtain a vision ability word granularity semantic full-connection coding feature vector sequence;

Multiplying each vision ability word granularity semantic coding feature vector in the sequence of vision ability word granularity semantic full-connection coding feature vectors with a transpose vector of a corresponding weight coefficient vector to obtain the sequence of significance coefficients.

6. The stereoscopic vision detection control method according to claim 5, wherein the full connection layer uses Selu activation functions.

7. The stereoscopic vision detection control method according to claim 6, wherein normalizing the sequence of saliency coefficients to obtain a sequence of autocorrelation saliency coefficients comprises:

normalizing the sequence of saliency coefficients based on a softmax activation function to obtain a sequence of autocorrelation saliency coefficients.

8. A stereoscopic vision detection control system, comprising:

the text description acquisition module is used for acquiring the vision ability text description and the vision test requirement text description of the client object;

The semantic coding module is used for respectively carrying out semantic coding on the vision ability text description and the vision testing requirement text description to obtain a sequence of vision ability word granularity semantic coding feature vectors and a sequence of vision testing requirement word granularity semantic coding feature vectors;

the important content attention integration module is used for enabling the sequences of the vision ability word granularity semantic coding feature vectors and the sequences of the vision testing requirement word granularity semantic coding feature vectors to pass through an important content attention integration network to obtain semantic enhanced vision ability semantic representation vectors and semantic enhanced vision testing requirement semantic representation vectors;

The semantic fusion module is used for determining a recommendation result of the stereoscopic electronic optotype based on semantic fusion representation between the semantic enhanced vision ability semantic representation vector and the semantic enhanced vision testing requirement semantic representation vector;

Wherein determining a recommendation of the stereoscopic electronic optotype based on the semantic fusion representation between the semantically enhanced vision ability semantic representation vector and the semantically enhanced vision testing requirement semantic representation vector comprises:

Passing the semantic enhanced vision ability semantic representation vector and the semantic enhanced vision testing requirement semantic representation vector through a vision ability-vision testing requirement cooperative encoder based on a CLIP model to obtain a vision ability-vision testing requirement cooperative semantic representation matrix;

Passing the vision ability-vision testing requirement through a decoder-based stereoscopic electronic optotype parameter recommender to obtain the recommendation, the recommendation comprising a color value and a brightness value of the stereoscopic electronic optotype.

9. The stereoscopic vision detection control system according to claim 8, wherein the semantic encoding module comprises:

The word segmentation processing unit is used for obtaining a sequence of the vision ability word granularity semantic coding feature vector through a semantic encoder comprising a word embedding layer after word segmentation processing is carried out on the vision ability text description;

the semantic coding unit is used for obtaining the sequence of the visual testing requirement word granularity semantic coding feature vector through the semantic coder containing the word embedding layer after the visual testing requirement text description is subjected to word segmentation.