CN113986018A - Visually impaired assisted reading learning method, system and storage medium based on smart glasses - Google Patents

Abstract

The invention discloses a reading and learning method, system and storage medium for visually impaired based on smart glasses, comprising: receiving a key activation signal, sending a reading activation signal to a reading display panel; and the reading display panel requesting a terminal server according to the reading activation signal Return the specified reading text data and display it; perform projection scanning on the display surface of the reading display panel; determine whether the projected scanning frame completely covers the bounding box marker points corresponding to the reading display panel; if so, take a picture of the currently displayed text to obtain picture information , identify and extract to obtain the currently extracted text content information and the currently extracted text typesetting information; determine whether it is consistent with the text content information and text typesetting information stored in the terminal server correspondingly; if so, read and learn. The invention can intuitively display the text content in the terminal server on the reading display board in the way of simulating the typesetting of the book, so as to facilitate the visually impaired people to read and study.

Description

Visually impaired assisted reading learning method, system and storage medium based on smart glasses

technical field

The present invention relates to the application technology field of smart glasses, in particular to a method, system and storage medium for reading and learning aids for the visually impaired based on smart glasses.

Background technique

With the increasing innovation of information technology, traditional products are also developing in the direction of functional integration, intelligent execution and convenient operation. For example, glasses are a very common object in people's daily life, and in recent years, technological changes led by smart glasses have also occurred.

Specifically, smart glasses, also known as smart mirrors, refer to "like smart phones, with an independent operating system. Smart glasses can be installed by users with programs provided by software, games and other software service providers. Smart glasses can be controlled by voice or action. A general term for such glasses that can complete functions such as adding schedules, map navigation, interacting with friends, taking photos and videos, and making video calls with friends, and can achieve wireless network access through mobile communication networks. At present, most smart glasses Glasses are mostly focused on augmented reality applications.

As we all know, blind people or people with vision diseases generally have reading barriers and cannot read and learn like ordinary people, which will greatly affect the quality of life and limit self-improvement. Based on this, it is necessary to design a smart glasses system with better auxiliary reading function to meet practical application requirements.

SUMMARY OF THE INVENTION

Therefore, the present invention is to solve the problem in the prior art that the existing blind people or groups suffering from vision diseases generally have barriers in reading and cannot read and learn like ordinary normal people, which will affect to a large extent. Quality of life and problems limiting self-improvement.

The present invention proposes a reading and learning method for visually impaired based on smart glasses, which is applied to smart glasses. The smart glasses and the reading display board are interactively controlled, and the smart glasses, the reading display board and the terminal server interact with each other. There is data transmission, a first camera and a first voice announcer are arranged on the smart glasses, and a second voice announcer is arranged on the reading display board, wherein the method includes the following steps:

Step 1: when the smart glasses receive a button activation signal, send a reading activation signal to the reading display panel, and the reading activation signal at least includes target text position information;

Step 2: the reading display board requests the terminal server to return the specified reading text data according to the reading start signal, and displays it on the reading display board after the loading is successful;

Step 3: The smart glasses perform projection scanning on the display surface of the reading display panel through the set first camera, wherein a projection scanning frame corresponds to a projection scanning frame when performing projection scanning;

Step 4: judging whether the projection scanning frame completely covers the bounding box marking points corresponding to the reading display panel, wherein each of the bounding box marking points encloses a display area;

Step 5: If yes, take a picture of the currently displayed text in the display area in the reading display panel to obtain picture information, and identify and extract the text in the picture information to obtain the currently extracted text content information and Currently extracting text layout information;

Step 6: judging whether the currently extracted text content information is consistent with the text content information stored in the terminal server correspondingly, and whether the currently extracted text layout information is consistent with the corresponding stored text layout information in the terminal server;

Step 7: If they match, broadcast each character in the currently extracted character content information in sequence through the second voice broadcaster.

The smart glasses-based assisted reading learning method for visually impaired proposed by the present invention sends a reading start signal to the reading display panel when the smart glasses receive the button activation signal, and the reading display panel requests the terminal server to return the specified reading text data according to the reading activation information , and display it on the reading display board after loading; then, take pictures of the current displayed text on the reading display board through smart glasses to obtain picture information, extract the text content information and text layout information in the picture information through analysis, and then store and store it with The text content information and the text layout information in the terminal server are judged to be consistent, and if they are consistent, each character in the currently extracted text content information is broadcast and learned by the second voice broadcaster. The visually impaired reading and learning method based on smart glasses proposed by the present invention can intuitively display the text content stored in the terminal server on the reading display board according to the way of simulating book typesetting, which is convenient for visually impaired people to read and learn; , due to the interactive learning through touch, the effect of reading learning can be improved, and the practical application experience can be enhanced.

The method for reading and learning aided reading for the visually impaired based on smart glasses, wherein, in the first step, the method includes the following steps:

After the smart glasses receive the button activation signal, the smart glasses generate first voice prompt information and broadcast it through the first voice announcer, wherein the first voice prompt information includes text chapter list information;

In the first preset time after the first voice announcer completes broadcasting, determine whether a confirmation reading signal returned by the user is received, and the confirmation reading signal includes the target text position information;

If so, control the smart glasses to send the reading start signal to the reading display panel according to the received confirmation reading signal.

The smart glasses-based aided reading learning method for the visually impaired, wherein the target text position information includes a book title serial number, a chapter number, a target text starting position, and a target text length. In the second step, the The acquisition method of the specified reading text data includes the following steps:

According to the title sequence number, chapter number, target text starting position and target text length in the target text position information, search and determine the corresponding specified reading text data in the terminal server and return it to the reading display board ;

The method for displaying on the reading display panel after the loading is successful includes the following steps:

Search and determine the corresponding specified reading text data in the terminal server, wherein the specified reading text data includes the specified reading text typesetting information and the specified reading text length;

According to the maximum display text length of a single page of the reading display board, in combination with the specified reading text typesetting information, the text of the corresponding length is intercepted from the acquired specified reading text data, and according to the specified reading text typesetting information in the specified reading text typesetting information displayed on the reading display panel.

The smart glasses-based aided reading and learning method for the visually impaired, wherein the reading display board is square in shape, and a first bounding box mark point and a second bounding box mark point are arranged on the inner side of the reading display board , the third bounding box mark point and the fourth bounding box mark point, the first bounding box mark point, the second bounding box mark point, the third bounding box mark point and the fourth bounding box mark point enclosing the display area;

A plurality of display unit blocks of the same size and uniformly arranged are arranged in the display area, wherein each of the display unit blocks is touch-enabled, and only one character is displayed in each of the display unit blocks;

In the step 4, the method for judging whether the projection scanning frame completely covers the bounding frame marking points corresponding to the reading display panel includes the following steps:

Determine whether the first bounding box marker point, the second bounding box marker point, the third bounding box marker point and the marker signal corresponding to the fourth bounding box marker point can be simultaneously detected in the projected scan frame .

The described smart glasses-based assisted reading learning method for the visually impaired, wherein the method for intercepting and obtaining text of a corresponding length in the obtained designated reading text data in combination with the designated reading text typesetting information includes the following steps:

Splitting each paragraph in the specified reading text data to obtain paragraph text data, wherein each paragraph text data includes a paragraph text length, a paragraph start character and a paragraph end character;

Confirm the starting position of the paragraph in the reading display panel according to the obtained paragraph starter;

According to the number of display unit blocks in each row of the display area in the reading display panel, starting from the starting position of the paragraph, fill the paragraph text data into the corresponding display unit blocks in a row-by-row filling manner, And accumulatively count to get the length of the paragraph padding text corresponding to the current paragraph;

When it is determined that the length of the filled text of the paragraph is equal to the length of the paragraph text, the fill and display operation of the current paragraph is completed according to the obtained paragraph cutoff.

The described method for assisting reading and learning for the visually impaired based on smart glasses, wherein the method further comprises:

Splitting each paragraph in the specified reading text data to obtain paragraph text data, wherein each paragraph text data includes a paragraph text length, a paragraph start character and a paragraph end character;

When it is determined that the length of the paragraph text corresponding to the current paragraph is less than the number of display unit blocks in a single line in the display area of the reading display panel, the current paragraph is displayed as a single line in the display area as a single line.

The described method for assisting reading and learning for the visually impaired based on smart glasses, wherein the method further comprises the following steps:

When the second voice announcer starts broadcasting, it immediately timed to obtain the current learning time, wherein the second voice announcer is used to announce each character in the currently extracted text content information in sequence;

Accumulatively calculate the number of times that each display unit block in the display area is repeatedly clicked in the current learning time to obtain the corresponding unit block click times;

When judging that the number of clicks on the unit block exceeds the preset number of clicks, confirming that the word corresponding to the display unit block is a key marker word;

Acquire the number of key marked words in the display area of the reading display board during the current learning time, and calculate the reading learning difficulty value according to the current learning time.

Said kind of reading learning method for visually impaired based on smart glasses, wherein, the calculation formula corresponding to said reading learning difficulty value is:

表示计算得到的所述阅读学习难度值，

表示基础难度分值，

表示显示区域的重点标记词的数量，

表示阅读显示板中显示区域内的显示单元块的总数量，

表示第一计分系数，

表示第二计分系数，

表示当前学习时间，

表示标准学习时间。in,

represents the calculated reading learning difficulty value,

represents the basic difficulty score,

Indicates the number of key words in the display area,

Indicates the total number of display unit blocks in the display area of the reading display panel,

represents the first scoring coefficient,

represents the second scoring coefficient,

represents the current study time,

Indicates standard study time.

The present invention also proposes a reading and learning system for visually impaired based on smart glasses, including smart glasses, a reading display panel and a terminal server, the smart glasses and the reading display panel are interactively controlled, the smart glasses, the reading display panel There is data transmission between the board and the terminal server, a first camera and a first voice announcer are arranged on the smart glasses, and a second voice announcer is arranged on the reading display board, wherein,

The smart glasses are used for:

When receiving a key activation signal, send a reading activation signal to the reading display panel, where the reading activation signal at least includes target text position information;

The reading display panel is used for:

According to the reading start signal, request the terminal server to return the specified reading text data, and display it on the reading display panel after the loading is successful;

The smart glasses are also used for:

Perform projection scanning on the display surface of the reading display panel through the provided first camera, wherein there is a projection scanning frame corresponding to the projection scanning;

judging whether the projection scanning frame completely covers the bounding box marking points corresponding to the reading display panel, wherein each of the bounding box marking points is combined to form a display area;

If so, take a picture of the currently displayed text in the display area in the reading display panel to obtain picture information, and identify and extract the text in the picture information to obtain the currently extracted text content information and the currently extracted text Typesetting information;

The reading display panel is also used for:

Judging whether the currently extracted text content information matches the text content information stored correspondingly in the terminal server, and whether the currently extracted text typesetting information matches the correspondingly stored text typesetting information in the terminal server;

If they match, each character in the currently extracted text content information is broadcast in sequence through the second voice broadcaster.

The present invention also provides a storage medium on which a computer program is stored, wherein when the program is executed by a processor, the smart glasses-based assisted reading and learning method for the visually impaired can be implemented.

Additional aspects and advantages of the present invention will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned from the embodiments of the invention.

Description of drawings

The above and/or additional aspects and advantages of embodiments of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, wherein:

FIG. 1 is a flowchart of a method for assisted reading and learning for the visually impaired based on smart glasses according to a first embodiment of the present invention;

2 is a flowchart of a method for generating and sending a reading start signal in the first embodiment of the present invention;

FIG. 3 is a flowchart of a method for assisted reading and learning for the visually impaired based on smart glasses according to a second embodiment of the present invention;

4 is a flowchart of a method for assisted reading and learning for the visually impaired based on smart glasses according to a third embodiment of the present invention;

FIG. 5 is a flowchart of a method for assisted reading and learning for the visually impaired based on smart glasses according to a fourth embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Blind people or people with vision diseases generally have reading barriers and cannot learn to read like ordinary people, which will greatly affect the quality of life and limit self-improvement. Based on this, it is necessary to design a smart glasses system with better auxiliary reading function to meet practical application requirements.

Example 1

In order to solve this technical problem, please refer to FIG. 1, the present invention proposes a reading learning method for visually impaired based on smart glasses, which is applied to smart glasses, interactive control between smart glasses and a reading display board, the smart glasses, reading There is data transmission between the display board and the terminal server, a first camera and a first voice announcer are arranged on the smart glasses, and a second voice announcer is arranged on the reading display board, wherein the method includes the following steps:

S101, when the smart glasses receive a key activation signal, send a reading activation signal to the reading display panel, where the reading activation signal at least includes target text position information.

Specifically, please refer to Figure 2. In this step, the following specific steps are included:

S1011 , after the smart glasses receive the button activation signal, the smart glasses generate first voice prompt information and broadcast it through the first voice announcer, wherein the first voice prompt information includes text chapter list information.

Specifically, after the smart glasses receive the button activation signal, the smart glasses can obtain a list of text chapters corresponding to each book from the terminal server. For example, there are 5 books stored in the terminal server. At this time, the first voice prompt information will broadcast the names of the five books in sequence. After the user confirms the selection of one of the books by voice, the first voice announcer in the smart glasses will announce the chapter list corresponding to the selected book by voice. For example, One Hundred Years of Solitude, Chapter 1, Section 2.

S1012, within a first preset time after the first voice announcer completes broadcasting, determine whether a confirmation reading signal returned by the user is received, and the confirmation reading signal includes target text location information.

In this step, the confirmation reading signal returned by the user needs to be acquired within the first preset time after the first voice announcer completes broadcasting, otherwise the device will stand by again.

Specifically, the target text position information includes a book title serial number, a chapter number, a start position of the target text, and a length of the target text. For example, the title number of "One Hundred Years of Solitude" is No. 4, the chapter number of the target text to be read is Chapter 1, Section 2, the starting position of the target text is the first word of Chapter 1, Section 2, and the target text The length is 3000 words.

S1013, if yes, control the smart glasses to send a reading start signal to the reading display panel according to the received confirmation reading signal.

It can be understood that, according to the position information of the target text in the confirmation reading signal, the target text can be clearly confirmed. At this time, the smart glasses will send a confirmation reading signal to the reading display panel to activate the reading display panel.

S102, the reading display board requests the terminal server to return the specified reading text data according to the reading start signal, and displays it on the reading display board after the loading is successful.

In this step, the acquisition method of the specified reading text data includes the following steps:

According to the book title serial number, chapter number, target text starting position and target text length in the target text position information, find and determine the corresponding designated reading text data in the terminal server and return it to the reading display board. The specific acquisition method is as described in the above example, and is not repeated here.

Further, the method for displaying on the reading display panel after the loading is successful includes the following steps:

S1021 , searching and determining the corresponding designated reading text data in the terminal server, wherein the designated reading text data includes the typesetting information of the designated reading text and the designated reading text length.

S1022, according to the maximum display text length of a single page of the reading display board, in combination with the typesetting information of the specified reading text, intercept the text of the corresponding length from the acquired specified reading text data, and display it on the reading display board according to the typesetting information of the specified reading text.

It should be noted here that, in this embodiment, the shape of the reading display panel is square. A first bounding box marking point, a second bounding box marking point, a third bounding box marking point and a fourth bounding box marking point are arranged on the inner side of the reading display panel. Wherein, the first bounding box marking point, the second bounding box marking point, the third bounding box marking point and the fourth bounding box marking point form the display area.

In this embodiment, a plurality of display unit blocks of the same size and uniformly arranged are provided in the display area. Wherein, each display unit block is touchable, and only one text is displayed in each display unit block. It can be understood that the maximum display text length of a single page of the reading display panel is the total number of display unit blocks in the reading display panel (for example, in this embodiment, the reading display panel is set to 15 rows and 20 columns, that is, it contains 300 display unit blocks, that is, the maximum display text length of a single page is 300 characters).

Further, in this step, the method of intercepting and obtaining the text of the corresponding length in the obtained specified reading text data in combination with the typesetting information of the specified reading text includes the following steps:

S1022a: Split each paragraph in the specified reading text data to obtain paragraph text data, where each paragraph text data includes a paragraph text length, a paragraph start character, and a paragraph end character.

After locating the specified reading text data in the terminal server, it is processed and split in units of paragraphs. Specifically, a paragraph is processed independently through the paragraph start character, the paragraph end character, and the corresponding paragraph text length. For example, for "One Hundred Years of Solitude" whose title serial number is No. 4, the chapter number of the target text to be read is Chapter 1, Section 2, the starting position of the target text is the first word of Chapter 1, Section 2, and the target text The length of the text is 3000 words (this is the object that needs to be read as a whole).

In this step, since the maximum displayed text length on a single page is 300 characters, the first paragraph of text in Chapter 1, Section 2 of "One Hundred Years of Solitude" is processed first. For example, the number of characters in the first paragraph of text is 157. At this time, all the contents of the first paragraph are filled into the reading display board. In the process of importing fill, it is positioned by paragraph start and paragraph end.

S1022b, confirming the starting position of the paragraph on the reading display panel according to the obtained paragraph starting symbol.

It needs to be supplemented here that, by default, two display unit blocks are vacated before the paragraph start character.

S1022c, according to the number of display unit blocks in each row of the display area in the reading display panel, starting from the starting position of the paragraph, fill the paragraph text data into the corresponding display unit blocks in a row-by-row filling manner, and accumulatively count up to Get the length of the paragraph padding text corresponding to the current paragraph.

For example, since the reading display panel is arranged in 15 rows and 20 columns, that is, at most 20 characters are arranged in each row. In this way, after filling each row, it automatically switches to the next row. As can be seen from the above example, since the number of characters currently being processed is 157, all the characters in this paragraph can be filled into the reading display board in this way.

It needs to be supplemented here that if the number of characters in the text currently being processed is too many, that is, exceeds 300 characters, the maximum number of characters that can be accommodated in the display area of the reading display panel shall prevail. After the current learning is completed, the filling and display will continue to be performed according to the original arrangement and filling method.

S1022d, when it is determined that the length of the filling text of the paragraph is equal to the length of the paragraph text, complete the filling and displaying operation of the current paragraph according to the obtained paragraph terminator.

What needs to be added here is that for the text with less single-line characters stored in the terminal server, the processing method is as follows:

Splitting each paragraph in the specified reading text data to obtain paragraph text data, wherein each paragraph text data includes paragraph text length, paragraph start character and paragraph end character;

When it is determined that the length of the paragraph text corresponding to the current paragraph is less than the number of display unit blocks in a single line in the display area of the reading display panel, the current paragraph is used as a separate line for single-line display in the display area.

S103, the smart glasses perform projection scanning on the display surface of the reading display panel through the set first camera, wherein a projection scanning frame corresponds to the projection scanning.

It can be understood that since the smart glasses are provided with a first camera, the reading display board after the filling and display operation can be photographed by the first camera.

S104: Determine whether the projection scanning frame completely covers the bounding box marking points corresponding to the reading display panel, wherein each of the bounding box marking points encloses a display area.

As described above, the first bounding box marker, the second bounding box marker, the third bounding box marker, and the fourth bounding box marker are combined to form a display area.

In this step, the projection scan frame corresponding to the first camera of the smart glasses needs to simultaneously cover the first bounding box mark point, the second bounding box mark point, the third bounding box mark point and the fourth bounding box mark point into the projection within the scan frame. That is, the marking signals corresponding to the first bounding box marking point, the second bounding box marking point, the third bounding box marking point and the fourth bounding box marking point need to be simultaneously detected in the projected scanning frame.

S105, if yes, take a picture of the currently displayed text in the display area in the reading display panel to obtain picture information, and identify and extract the text in the picture information to obtain the currently extracted text content information and the current Extract text typography information.

S106: Determine whether the currently extracted text content information matches the text content information correspondingly stored in the terminal server, and whether the currently extracted text typesetting information matches the correspondingly stored text typesetting information in the terminal server.

It should be noted that, in this step, comparing the currently extracted text typesetting information with the corresponding stored text typesetting information in the terminal server does not mean that the position of each text needs to be exactly the same, but the currently extracted text typesetting information The information needs to meet the set layout rules. Specifically, the text originally located at the position of the paragraph start character and the paragraph end character, when displayed on the reading display board, is also located at the position of the paragraph start character and the paragraph end character, and the number of characters in this paragraph is not equal. Changes occur, and there is no problem of missing words.

S107 , if all are matched, broadcast each character in the currently extracted character content information in sequence through the second voice broadcaster.

It can be understood that when it is confirmed that both the currently extracted text content information and the currently extracted text typesetting information are correct, the display typesetting operation of the reading display panel is completed at this time. At this time, each character in the currently extracted text content information is broadcast in sequence through the second voice broadcaster. In this embodiment, the setting can intuitively display the text content stored in the terminal server on the reading display board in a manner of simulating the typesetting of a book, which is convenient for visually impaired people to read and learn.

The smart glasses-based auxiliary reading learning method for visually impaired provided by the present invention sends a reading start signal to the reading display panel when the smart glasses receive the button activation signal, and the reading display panel requests the terminal server to return the specified reading text data according to the reading activation information , and display it on the reading display board after loading; then, take pictures of the current displayed text on the reading display board through smart glasses to obtain picture information, extract the text content information and text layout information in the picture information through analysis, and then store and store it with The text content information and the text layout information in the terminal server are judged to be consistent, and if they are consistent, each character in the currently extracted text content information is broadcast and learned by the second voice broadcaster. The visually impaired reading and learning method based on smart glasses proposed by the present invention can intuitively display the text content stored in the terminal server on the reading display board according to the way of simulating book typesetting, which is convenient for visually impaired people to read and learn; , due to the interactive learning through touch, the effect of reading learning can be improved, and the practical application experience can be enhanced.

Embodiment 2

Referring to FIG. 3 , a second embodiment of the present invention proposes a reading and learning method for visually impaired based on smart glasses, which specifically includes the following steps:

S201 , when the smart glasses receive a key activation signal, send a reading activation signal to the reading display panel, where the reading activation signal at least includes target text position information.

S202, the reading display board requests the terminal server to return the specified reading text data according to the reading start signal, displays it on the reading display board after the loading is successful, and generates a screenshot request query instruction.

The above steps S201 and S202 have been described above. No further description is given here.

S203, after receiving the confirmation signal returned by the user for the screenshot request query instruction, take a screenshot of the current display interface of the reading display panel to obtain screenshot image information.

In order to reduce the inconvenience caused to a certain extent by taking pictures of the reading display panel through the camera of the smart glasses. In this embodiment, after receiving the confirmation signal for the screenshot request query instruction returned by the user, a screenshot is taken of the current display interface of the reading display panel to obtain screenshot image information.

S204: Recognize and extract the text in the screenshot image information to obtain currently extracted text content information and currently extracted text typesetting information.

S205, judging whether the currently extracted text content information matches the correspondingly stored text content information in the terminal server, and whether the currently extracted text typesetting information matches the correspondingly stored text typesetting information in the terminal server.

S206, if all are matched, broadcast each character in the currently extracted character content information in sequence through the second voice broadcaster.

It can be understood that in this embodiment, the problem of inconvenience caused to a certain extent by taking pictures of the reading display panel through the camera of the smart glasses is reduced, and the user experience is improved.

Embodiment 3

Referring to FIG. 4 , a third embodiment of the present invention proposes a method for assisted reading and learning for the visually impaired based on smart glasses, which specifically includes the following steps:

S301, when the second voice announcer starts broadcasting, time immediately to obtain the current learning time, wherein the second voice announcer is configured to announce each character in the currently extracted character content information in sequence.

S302, cumulatively calculate the number of times that each display unit block in the display area is repeatedly clicked respectively during the current learning time to obtain the number of corresponding unit block clicks.

In actual reading learning, each display unit block in the reading display panel is touchable. When the user has doubts about the text corresponding to one of the display unit blocks, the user can repeatedly click the corresponding display unit block for many times. As a supplement, after a certain display unit block is clicked, the reading display board will communicate with the terminal server to obtain the corresponding explanation about the difficult word, and broadcast it through the second voice announcer.

S303, when it is determined that the number of clicks on the unit block exceeds the preset number of clicks, confirm that the word corresponding to the display unit block is a key marked word.

It is understandable that, for example, the number of clicks of a unit block corresponding to a certain display unit block is 4 times, and the preset number of clicks is set to 3 times. At this time, the corresponding display unit block is listed as a key marker word.

S304: Acquire the number of key marked words in the display area of the reading display board during the current learning time, and calculate and obtain a reading learning difficulty value according to the current learning time.

In this step, the reading learning difficulty value is calculated by the following formula, and the specific corresponding calculation formula is:

表示计算得到的所述阅读学习难度值，

表示基础难度分值，

表示显示区域的重点标记词的数量，

表示阅读显示板中显示区域内的显示单元块的总数量，

表示第一计分系数，

表示第二计分系数，

表示当前学习时间，

表示标准学习时间。in,

represents the calculated reading learning difficulty value,

represents the basic difficulty score,

Indicates the number of key words in the display area,

Indicates the total number of display unit blocks in the display area of the reading display panel,

represents the first scoring coefficient,

represents the second scoring coefficient,

represents the current study time,

Indicates standard study time.

Referring to FIG. 5, a fourth embodiment of the present invention proposes a reading and learning system for visually impaired based on smart glasses, including smart glasses, a reading display board, and a terminal server. The smart glasses and the reading display board are interactively controlled, so There is data transmission between the smart glasses, the reading display board and the terminal server, a first camera and a first voice broadcaster are arranged on the smart glasses, and a second voice broadcaster is arranged on the reading display board device, of which,

The smart glasses are used for:

When receiving a key activation signal, send a reading activation signal to the reading display panel, where the reading activation signal at least includes target text position information;

The reading display panel is used for:

According to the reading start signal, request the terminal server to return the specified reading text data, and display it on the reading display panel after the loading is successful;

The smart glasses are also used for:

Perform projection scanning on the display surface of the reading display panel through the provided first camera, wherein there is a projection scanning frame corresponding to the projection scanning;

judging whether the projection scanning frame completely covers the bounding box marking points corresponding to the reading display panel, wherein each of the bounding box marking points is combined to form a display area;

If so, take a picture of the currently displayed text in the display area in the reading display panel to obtain picture information, and identify and extract the text in the picture information to obtain the currently extracted text content information and the currently extracted text Typesetting information;

The reading display panel is also used for:

Judging whether the currently extracted text content information matches the text content information stored correspondingly in the terminal server, and whether the currently extracted text typesetting information matches the correspondingly stored text typesetting information in the terminal server;

If they match, each character in the currently extracted text content information is broadcast in sequence through the second voice broadcaster.

The present invention also provides a storage medium on which a computer program is stored, wherein when the program is executed by a processor, the smart glasses-based assisted reading and learning method for the visually impaired can be implemented.

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus.

More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, ASICs with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, The scope of the invention is defined by the claims and their equivalents.

Claims (10)

1. A vision impairment auxiliary reading learning method based on intelligent glasses is applied to the intelligent glasses, interactive control is carried out between the intelligent glasses and a reading display panel, data transmission exists among the intelligent glasses, the reading display panel and a terminal server, a first camera and a first voice broadcast device are arranged on the intelligent glasses, and a second voice broadcast device is arranged on the reading display panel, and is characterized by comprising the following steps:

the method comprises the following steps: when the intelligent glasses receive a key enabling signal, sending a reading starting signal to the reading display panel, wherein the reading starting signal at least comprises target text position information;

step two: the reading display board requests to return appointed reading text data to the terminal server according to the reading starting signal, and the appointed reading text data is displayed on the reading display board after the loading is successful;

step three: the intelligent glasses perform projection scanning on the display surface of the reading display panel through the arranged first camera, wherein a projection scanning frame is correspondingly arranged during the projection scanning;

step four: judging whether the projection scanning frame completely covers the boundary frame mark points corresponding to the reading display board or not, wherein each boundary frame mark point is encircled to form a display area;

step five: if so, photographing the currently displayed characters in the display area in the reading display panel to acquire picture information, and identifying and extracting the characters in the picture information to acquire currently extracted character content information and currently extracted character typesetting information;

step six: judging whether the current extracted text content information is matched with the text content information correspondingly stored in the terminal server or not and whether the current extracted text typesetting information is matched with the text typesetting information correspondingly stored in the terminal server or not;

step seven: if all coincide, then through the second voice broadcast ware is reported according to the preface every characters in the current extraction text content information.

2. The vision impairment assisted reading learning method based on smart glasses as claimed in claim 1, wherein in the step one, the method comprises the following steps:

after the intelligent glasses receive the key starting signal, the intelligent glasses generate first voice prompt information and broadcast the first voice prompt information through the first voice broadcaster, wherein the first voice prompt information comprises text chapter list information;

judging whether a reading confirmation signal returned by a user is received or not within a first preset time after the first voice broadcast device finishes broadcasting, wherein the reading confirmation signal comprises the target text position information;

and if so, controlling the intelligent glasses to send the reading starting signal to the reading display board according to the received reading confirming signal.

3. The vision impairment assistant reading and learning method based on the intelligent glasses as claimed in claim 1, wherein the target text position information includes a book title serial number, a chapter number, a target text start position and a target text length, and in the second step, the method for obtaining the designated reading text data includes the following steps:

searching and determining corresponding appointed reading text data in the terminal server according to the book name serial number, the chapter number, the target text starting position and the target text length in the target text position information, and returning and loading the appointed reading text data to the reading display board;

the method for displaying on the reading display board after the loading is successful comprises the following steps:

searching and determining corresponding appointed reading text data in the terminal server, wherein the appointed reading text data comprises appointed reading text typesetting information and appointed reading text length;

and intercepting the text with the corresponding length in the acquired specified reading text data in combination with the specified reading text typesetting information according to the maximum single-page display text length of the reading display panel, and displaying the text on the reading display panel according to the specified reading text typesetting information.

4. The reading and learning method for assisting in reading the visual impairment based on the smart glasses as claimed in claim 3, wherein the reading display panel is square, and a first border marker point, a second border marker point, a third border marker point and a fourth border marker point are disposed on the inner side of the reading display panel, and the first border marker point, the second border marker point, the third border marker point and the fourth border marker point enclose the display area;

a plurality of display unit blocks which are equal in size and are uniformly arranged are arranged in the display area, wherein each display unit block is arranged in a touch manner, and only one character is displayed in each display unit block;

in the fourth step, the method for determining whether the projection scan frame completely covers the border frame mark point corresponding to the reading display panel includes the following steps:

and judging whether the marking signals corresponding to the first boundary frame marking point, the second boundary frame marking point, the third boundary frame marking point and the fourth boundary frame marking point can be detected in the projection scanning frame at the same time.

5. The vision impairment auxiliary reading learning method based on the intelligent glasses as claimed in claim 4, wherein the method for intercepting the text with the corresponding length from the obtained specified reading text data by combining the specified reading text typesetting information comprises the following steps:

splitting each paragraph in the specified reading text data to obtain paragraph text data, wherein each paragraph text data comprises a paragraph text length, a paragraph start character and a paragraph end character;

confirming a paragraph start position in the reading display board according to the obtained paragraph start character;

according to the number of display unit blocks of each line of a display area in the reading display panel, sequentially filling the paragraph text data into corresponding display unit blocks in a line-by-line filling mode from the beginning position of the paragraph, and accumulating and counting to obtain the paragraph filling character length corresponding to the current paragraph;

and when the paragraph filling character length is judged to be equal to the paragraph text length, completing the filling display operation of the current paragraph according to the obtained paragraph cut-off symbol.

6. The method for learning reading assistance with impaired vision based on smart glasses according to claim 5, further comprising:

splitting each paragraph in the specified reading text data to obtain paragraph text data, wherein each paragraph text data comprises a paragraph text length, a paragraph start character and a paragraph end character;

and when judging that the text length of the paragraph corresponding to the current paragraph is smaller than the number of display unit blocks in a single line in a display area of the reading display panel, displaying the current paragraph as a single line in the display area.

7. The vision impairment assisted reading learning method based on smart glasses as claimed in claim 4, wherein the method further comprises the following steps:

when the second voice broadcast device starts broadcasting, timing immediately to obtain the current learning time, wherein the second voice broadcast device is used for broadcasting each character in the currently extracted character content information in sequence for a user to learn;

accumulating and calculating the times of repeated clicks of each display unit block in the display area in the current learning time to obtain the corresponding times of clicks of the unit blocks;

when the clicking times of the unit block exceed the preset clicking times, confirming that the words corresponding to the display unit block are key marked words;

and acquiring the number of key marked words in a display area of the reading display board in the current learning time, and calculating to obtain a reading learning difficulty value according to the current learning time.

8. The vision impairment auxiliary reading learning method based on the intelligent glasses as claimed in claim 7, wherein the corresponding calculation formula of the reading learning difficulty value is:

wherein,

representing the calculated reading learning difficulty value,

a base difficulty score is indicated that represents the base difficulty score,

the number of highlight marker words representing the display area,

indicating the total number of display unit blocks in the display area in the reading display panel,

a first scoring coefficient is represented that is a function of,

a second scoring coefficient is represented that is a function of,

indicates the current learning time of the current time,

indicating the standard learning time.

9. A vision impairment auxiliary reading learning system based on intelligent glasses comprises the intelligent glasses, a reading display panel and a terminal server, wherein the intelligent glasses and the reading display panel are controlled in an interactive mode, data transmission exists among the intelligent glasses, the reading display panel and the terminal server, a first camera and a first voice announcer are arranged on the intelligent glasses, a second voice announcer is arranged on the reading display panel, and the vision impairment auxiliary reading learning system is characterized in that,

the smart glasses are configured to:

when a key enabling signal is received, sending a reading starting signal to the reading display panel, wherein the reading starting signal at least comprises target text position information;

the reading display board is used for:

requesting to return appointed reading text data to the terminal server according to the reading starting signal, and displaying on the reading display board after the loading is successful;

the smart glasses are further configured to:

the display surface of the reading display panel is subjected to projection scanning through the arranged first camera, wherein a projection scanning frame is correspondingly arranged during the projection scanning;

judging whether the projection scanning frame completely covers the boundary frame mark points corresponding to the reading display board or not, wherein each boundary frame mark point is encircled to form a display area;

if so, photographing the currently displayed characters in the display area in the reading display panel to acquire picture information, and identifying and extracting the characters in the picture information to acquire currently extracted character content information and currently extracted character typesetting information;

the reading display panel is further configured to:

judging whether the current extracted text content information is matched with the text content information correspondingly stored in the terminal server or not and whether the current extracted text typesetting information is matched with the text typesetting information correspondingly stored in the terminal server or not;

if all coincide, then through the second voice broadcast ware is reported according to the preface every characters in the current extraction text content information.

10. A storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method of reading and learning based on smart eyewear vision-aided as recited in any one of claims 1 to 8.

Images