CN114154464A

CN114154464A - Structure picture restoration method, structure picture restoration device, electronic equipment, medium and program product

Info

Publication number: CN114154464A
Application number: CN202111275846.2A
Authority: CN
Inventors: 韦涛; 谷枫; 饶旭东
Original assignee: Beijing Sogou Technology Development Co Ltd
Current assignee: Beijing Sogou Technology Development Co Ltd
Priority date: 2021-10-29
Filing date: 2021-10-29
Publication date: 2022-03-08

Abstract

The invention discloses a method, a device, electronic equipment, a medium and a program product for restoring a structural picture, wherein the method comprises the following steps: acquiring a transformation hierarchical structure picture; performing text recognition on the converted hierarchical structure picture to obtain a recognition text in the converted hierarchical structure picture; carrying out image transformation classification on the identification text to obtain a target transformation classification of the transformation hierarchy structure picture; and restoring the conversion hierarchical structure picture according to the target conversion classification to obtain a corresponding restoration hierarchical structure picture. By the scheme, the efficiency, the accuracy and the rapidness of the structure picture restoration can be improved.

Description

Structure picture restoration method, structure picture restoration device, electronic equipment, medium and program product

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for restoring a structure picture, an electronic device, a medium, and a program product.

Background

The hierarchical picture is a nested structural picture with clear hierarchy, and is often found in daily work and study life of users. The thinking diagram is an effective graphic thinking tool for expressing divergent thinking. To facilitate file transfer and user reading, the mind map typically exists in the form of an image. However, when the mind map is stored and transmitted in an image format as a file, a great defect exists in that only reference can be supported, but re-editing cannot be supported.

Therefore, it is necessary to provide an efficient and reliable mind map restoring scheme.

Disclosure of Invention

The embodiment of the invention provides a method and a device for restoring a structural picture, electronic equipment and a readable storage medium, and solves the technical problem that the thinking picture restoration after image transposition transformation cannot be correctly realized in the prior art.

In a first aspect, an embodiment of the present invention provides a method for restoring a structural picture, including:

acquiring a transformation hierarchical structure picture, wherein the transformation hierarchical structure picture is a hierarchical structure picture subjected to image transformation processing;

performing text recognition on the converted hierarchical structure picture to obtain a recognition text in the converted hierarchical structure picture;

performing image transposition classification on the identification text to obtain target transformation classification of the transformation hierarchy structure picture;

and restoring the conversion hierarchical structure picture according to the target conversion classification to obtain a corresponding restoration hierarchical structure picture.

Optionally, when the converted hierarchical structure picture is a mirror image hierarchical structure picture after image mirror image conversion, the identification text in the converted hierarchical structure picture is a mirror image text, and the target conversion classification is a target mirror image classification;

the reducing the converted hierarchical structure picture according to the target conversion classification to obtain a corresponding reduced hierarchical structure picture comprises:

performing corresponding mirror image turnover on the mirror image hierarchical structure picture according to the target mirror image classification to obtain a turnover hierarchical structure picture, wherein the turnover hierarchical structure picture comprises a turnover text corresponding to the mirror image text, a plurality of hierarchical nodes and logical connecting lines among the hierarchical nodes;

performing structural reconstruction on the plurality of hierarchical nodes in the turnover hierarchical structure picture and the logical connection lines among the hierarchical nodes to obtain a reconstructed hierarchical structure;

and performing correlation matching on the turning text and the reconstructed hierarchical structure to obtain the restored hierarchical structure picture.

Optionally, when the converted hierarchical structure picture is a transposed hierarchical structure picture after image transposition conversion, the identification text in the converted hierarchical structure picture is a transposed text, and the target conversion classification is a target transposition classification; the reducing the converted hierarchical structure picture according to the target conversion classification to obtain a corresponding reduced hierarchical structure picture comprises:

performing reverse transposition processing on the transposed hierarchical structure picture according to the target transposition classification to obtain a reverse transposition structure picture, wherein the reverse transposition structure picture comprises a reverse transposition text corresponding to the transposition text, a plurality of hierarchical nodes and logic connecting lines among the hierarchical nodes;

performing structural reconstruction on the multiple hierarchical nodes in the reverse inversion structure picture and the logical connection lines among the hierarchical nodes to obtain a reconstructed hierarchical structure;

and performing correlation matching on the reverse-placed text and the reconstructed hierarchical structure to obtain the restored hierarchical structure picture.

Optionally, the performing structural reconstruction on the multiple hierarchical nodes in the reverse inverse structural picture and the logical connection lines between the hierarchical nodes to obtain a reconstructed hierarchical structure includes:

carrying out node extraction and connection extraction on the reverse inversion structure picture to obtain a plurality of hierarchy nodes in the reverse inversion structure picture and logic connection lines among the hierarchy nodes;

and performing structural reconstruction on the plurality of hierarchical nodes and the logical connection lines among the hierarchical nodes to obtain the reconstructed hierarchical structure.

Optionally, the reversely transposed text includes at least one reversely transposed text line and a position of each reversely transposed text line, and the associating and matching the reversely transposed text and the reconstructed hierarchical structure to obtain the restored hierarchical structure picture includes:

and correspondingly adding each reverse transposed text line into the reconstruction hierarchical structure according to the position of each reverse transposed text line to obtain the picture of the reduction hierarchical structure.

Optionally, the method comprises:

receiving an editing instruction for the restored hierarchical structure picture, the editing instruction being used for editing target information in the restored hierarchical structure picture, the target information including at least one of: target text lines, target level nodes and target logical connecting lines;

and responding to the editing instruction, and correspondingly editing the target information in the restored hierarchical structure picture.

Optionally, the restored hierarchy picture includes an editable reconstructed hierarchy map, and the method further includes:

receiving an editing instruction for the reconstruction hierarchy structure chart, wherein the editing instruction is used for editing the reconstruction hierarchy structure in the reconstruction hierarchy structure chart;

and responding to the editing instruction, and correspondingly editing the reconstructed hierarchical structure diagram in the restored hierarchical structure picture.

Optionally, the transposed text includes at least one transposed text line, and the performing image transpose classification on the transposed text to obtain a target transpose classification of the transposed hierarchical structure picture includes:

performing image transposition classification on each transposed text line to obtain the text transposition classification of each transposed text line;

when the number of transposed text lines for a preset transpose classification exceeds a preset threshold, determining the preset transpose classification as a target transpose classification of the transposed hierarchical structure picture.

In a second aspect, an embodiment of the present invention provides a structure reduction apparatus, including:

the acquisition module is used for acquiring a transformation hierarchical structure picture, wherein the transformation hierarchical structure picture is a hierarchical structure picture subjected to image transformation processing;

the identification module is used for performing text identification on the converted hierarchical structure picture to obtain an identification text in the converted hierarchical structure picture;

the classification module is used for carrying out image transformation classification on the identification text to obtain target transformation classification of the transformation hierarchy structure picture;

and the restoration module is used for restoring the conversion hierarchical structure picture according to the target conversion classification to obtain a corresponding restoration hierarchical structure picture.

Optionally, when the converted hierarchical structure picture is a transposed hierarchical structure picture after image transposition conversion, the identification text in the converted hierarchical structure picture is a transposed text, and the target conversion classification is a target transposition classification; the reduction module is specifically configured to:

Optionally, the reduction module is specifically configured to:

Optionally, the reverse transposed text includes at least one reverse transposed text line and a position of each reverse transposed text line, and the restoring module is specifically configured to:

Optionally, the apparatus further comprises:

a receiving module, configured to receive an editing instruction for the restored hierarchical structure picture, where the editing instruction is used to edit target information in the restored hierarchical structure picture, and the target information includes at least one of the following: target text lines, target level nodes and target logical connecting lines;

and the processing module is used for responding to the editing instruction and correspondingly editing the target information in the restored hierarchical structure picture.

Optionally, the transposed text comprises at least one transposed text line, and the classification module is specifically configured to:

For the content that is not introduced or not described in the embodiment of the present application, reference may be made to the related descriptions in the foregoing method embodiments, and details are not described here again.

In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and one or more programs, where the one or more programs are stored in the memory and configured to be executed by one or more processors to execute operation instructions included in the one or more programs for performing the corresponding method for restoring the structural picture according to the first aspect.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps corresponding to the structural picture restoration method provided in the first aspect.

One or more technical solutions provided by the embodiments of the present invention at least achieve the following technical effects or advantages:

according to the scheme provided by the embodiment of the invention, a conversion hierarchical structure picture is obtained, text recognition is carried out on the conversion hierarchical structure picture to obtain a recognition text in the conversion hierarchical structure picture, then conversion classification is carried out on the recognition text to obtain target conversion classification of the conversion hierarchical structure picture, and finally reduction processing is carried out on the conversion hierarchical structure picture according to the target conversion classification to obtain a reduction hierarchical structure picture corresponding to the conversion hierarchical structure picture. According to the scheme, the target transformation classification of the whole transformation hierarchy structure picture can be identified based on the identification text in the transformation hierarchy structure picture, and then the transformation hierarchy structure picture is restored according to the target transformation classification to obtain the final editable restoration hierarchy structure picture, so that the quick restoration of the transformation hierarchy structure picture can be efficiently and accurately realized, and the efficiency, the accuracy and the quickness of the restoration of the structure picture are improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

Fig. 1 is a schematic flow chart of a method for restoring a structural picture according to an embodiment of the present invention.

Fig. 2 is a schematic flow chart of a method for restoring a structural picture according to an embodiment of the present invention.

Fig. 3 is a schematic diagram of a transposed hierarchical picture according to an embodiment of the present invention.

Fig. 4 is a schematic diagram of a reverse-inversion structural picture according to an embodiment of the present invention.

Fig. 5 is a schematic diagram of hierarchical structure information in a reverse-inversion-structure picture according to an embodiment of the present invention.

Fig. 6 is a schematic diagram of a reconstruction hierarchy structure according to an embodiment of the present invention.

Fig. 7 is a schematic structural diagram of a structural image reduction device according to an embodiment of the present invention.

Fig. 8 is a schematic structural diagram of another structural picture reduction apparatus according to an embodiment of the present invention.

Fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present invention.

Fig. 10 is a schematic structural diagram of a server according to an embodiment of the present invention.

Detailed Description

The invention provides a method and a device for restoring a structural picture, electronic equipment and a readable storage medium, which are used for solving the technical problem that thinking guide picture restoration cannot be realized in the prior art, and have the following general ideas:

acquiring a transformation hierarchical structure picture, wherein the transformation hierarchical structure picture is a hierarchical structure picture subjected to image transformation processing; performing text recognition on the converted hierarchical structure picture to obtain a recognition text in the converted hierarchical structure picture; carrying out image transformation classification on the identification text to obtain a target transformation classification of the transformation hierarchy structure picture; and restoring the conversion hierarchical structure picture according to the target conversion classification to obtain a corresponding restoration hierarchical structure picture.

According to the technical scheme, the target transformation classification of the whole transformation hierarchy structure picture can be identified based on the identification text in the transformation hierarchy structure picture, and then the transformation hierarchy structure picture is restored according to the target transformation classification to obtain the final reduction hierarchy structure picture supporting editability, so that the rapid restoration of the transformation hierarchy structure picture can be efficiently and accurately realized, and the efficiency, the accuracy and the quickness of the restoration of the structure picture are improved.

Fig. 1 is a schematic flow chart illustrating a method for restoring a structural picture according to an embodiment of the present invention. The method can be applied to terminal equipment, such as a smart phone and a tablet computer, can also be applied to a server which is established with data interaction with the terminal equipment, and can also be applied to a system which consists of the terminal equipment and the server, and the invention is not limited. The method as shown in fig. 1 comprises the following steps:

s101, obtaining a transformation hierarchical structure picture, wherein the transformation hierarchical structure picture is a hierarchical structure picture after image transformation processing.

The transformation of the hierarchical structure picture refers to a hierarchical structure picture after image transformation, for example, image transposition transformation or image mirroring transformation. In a possible implementation, the transform hierarchy picture may be a transposed hierarchy picture after an image transpose transform; alternatively, the converted hierarchical structure picture may be a mirror image hierarchical structure picture after image mirror conversion, or may also be a hierarchical structure picture after other image conversion processes, which is not limited in the present invention.

S102, performing text recognition on the converted hierarchical structure picture to obtain a recognition text in the converted hierarchical structure picture.

In one possible implementation, the transform hierarchy picture is a transposed hierarchy picture. The method can perform text recognition on the transposed hierarchical structure picture to obtain the transposed text in the transposed hierarchical structure picture.

In another possible implementation, the transformed hierarchical picture is a mirror hierarchical picture. The invention can perform text recognition on the image hierarchical structure picture to obtain the image text in the image hierarchical structure picture.

S103, carrying out image transformation and classification on the recognition texts to obtain target transformation and classification of the transformation hierarchy structure picture.

In one possible implementation, the identified text is transposed text in the transposed hierarchy of pictures. The method can perform image transposition classification on the transposed text to obtain the target transposition classification of the transposed hierarchical structure picture.

In another possible embodiment, the identification text is a mirror image text in the mirror image hierarchical picture. The invention can carry out image mirror image classification on the mirror image text to obtain the target mirror image classification of the mirror image hierarchical structure picture.

And S104, restoring the converted hierarchical structure picture according to the target conversion classification to obtain a corresponding restored hierarchical structure picture.

In one possible implementation, the target transform classification is a target transpose classification. The invention can restore the transposed hierarchical structure picture according to the target transposed classification to obtain the corresponding restored hierarchical structure picture.

In another possible embodiment, the object transformation classification is an object mirror classification. The invention can restore the mirror image hierarchical structure chart according to the target mirror image classification to obtain a corresponding restored hierarchical structure picture.

It should be noted that the present invention is similar to the specific embodiments of the restoration processing for the transposed hierarchically structured picture and the mirror-image hierarchically structured picture, and can be implemented by referring to each other correspondingly.

Please refer to fig. 2, which is a flowchart illustrating another method for restoring a structural picture according to an embodiment of the present application. The method as shown in fig. 2 comprises the following implementation steps:

s201, acquiring the transposed hierarchical structure picture.

The transposed hierarchical structure picture is a hierarchical structure picture after image transposing transformation, and specifically may refer to a hierarchical structure picture obtained by performing transposing/rotation processing of an original hierarchical structure picture at any angle, for example, performing image transposing operation at an angle of 90 °, 180 °, 270 °, or the like. The original hierarchically structured picture refers to a conventional hierarchically structured picture that has not been subjected to any image transformation processing. For example, please refer to fig. 3, which shows a schematic diagram of a possible transposed hierarchical picture. The transposed hierarchy picture shown in fig. 3 is a hierarchy picture after being subjected to 90 ° transposition processing.

The hierarchical structure picture related to the present invention refers to a structure picture that can be used for representing a hierarchy or a level relationship, and the structure picture can clearly represent the hierarchical relationship of each level, which may include but is not limited to a thinking diagram, a flow structure picture, a tree structure picture, or other customized structure pictures with a hierarchy or a hierarchical relationship, etc.

S202, performing text recognition on the transposed hierarchical structure picture to obtain a transposed text in the transposed hierarchical structure picture.

In an embodiment, the invention may perform text recognition on the transposed hierarchical picture by using a text recognition technology to obtain the transposed text in the transposed hierarchical picture. The text recognition techniques include, but are not limited to, Optical Character Recognition (OCR), geometric feature extraction techniques, or other techniques for text or word recognition, among others. The transposed text refers to a text after image transposition processing, and may include a custom text in any format, such as a bmp image format text, a jpg image format text, and the like.

In another embodiment, the invention can perform text recognition on the transposed hierarchical picture by using a pre-trained text recognition model to obtain the transposed text in the transposed hierarchical picture. The text recognition model is pre-trained for recognizing text information in the hierarchical structure picture, and may include, but is not limited to, a feedforward neural network model, a convolutional neural network model, a deep residual network model, a cyclic neural network model, a long-short term memory model, or other machine learning models.

The transposed text to which the present invention relates includes, but is not limited to, at least one line of transposed text and the position (also referred to as position coordinates) of each line of transposed text. The transposed text line may refer to the text content of a line, or may refer to the text content of a line and a text box owned by the text content, which is not limited in the present invention. Optionally, the following text is used as an example of text content to describe related content, but the invention is not limited thereto.

S203, performing image transposition classification on the transposed text to obtain a target transposition classification of the transposed hierarchical structure picture.

In an embodiment, the present invention may use a pre-constructed variable-length text transpose classifier to perform image transpose classification on the transposed text to determine/obtain a target transpose classification of the transposed hierarchical structure picture.

In another embodiment, the invention may use a pre-trained transposition classification model to perform image transposition classification on the transposed text to obtain a target transposition classification of the transposed hierarchical structure picture. The transposed classification model is a pre-trained model for text transposed classification, which may include, but is not limited to, a sequence-to-sequence (seq 2seq) model, a Bidirectional Encoding Representation (BERT) model, a Word-to-vector (Word2Vec) model, a text neural network model, or other models for text transposed classification, etc.

In a particular embodiment, the transposed text includes at least one line of transposed text. In the image transposing classification process, the invention can specifically classify each transposed text line in the transposed text to obtain the text transposing classification of each transposed text line. When the number of the transposed text lines corresponding to the preset transposed classification of the text transposed classification exceeds a preset threshold, the text transposed classification of most of the transposed text lines in the transposed text may be considered as the preset transposed classification, and the preset transposed classification may be determined as a target transposed classification of the transposed hierarchical structure picture. Otherwise, the process may end.

The preset threshold is a threshold set by the system in a self-defined manner, and may be set according to actual requirements of the system, or may be an empirical value set according to experience of a user, and the like, which is not limited in the present invention. The preset transpose classification, the text transpose classification, and the target transpose classification related to the present invention are all pre-configured classifications in the transpose classifier or the transpose classification model, which may include but are not limited to, for example, a normal transpose classification, a 90 ° transpose classification, a 180 ° transpose classification, a 270 ° transpose classification, or any other transpose classification of a user-defined angle, etc.

And S204, according to the target transposition classification, restoring the transposed hierarchical structure picture to obtain a corresponding restored hierarchical structure picture.

In an embodiment, the present invention may first perform reverse transposition processing corresponding to the target transposition classification on the transposed hierarchical structure picture according to the target transposition classification to obtain a corresponding reverse transposition structural picture, where the reverse transposition structural picture includes a reverse transposition text corresponding to the transposition text, a plurality of hierarchical nodes, and logical connection lines between the hierarchical nodes. The reverse transposed text is specifically a text obtained by performing corresponding reverse transposition processing on the transposed text in the transposed hierarchical structure picture according to the target transposition classification.

Specifically, the target transpose classification includes four transpose classifications, which specifically include a normal transpose classification, a 90 ° transpose classification, a 180 ° transpose classification, and a 270 ° transpose classification. When the target transposition classification is the normal transposition classification, the present invention does not need to perform inverse transposition processing on the transposed hierarchically structured picture according to the indication of the normal transposition classification, and at this time, the transposed hierarchically structured picture is regarded as an original hierarchically structured picture without image transformation, and the subsequent processes are continued.

When the target transpose classification is a 90 ° transpose classification, the present invention may perform corresponding 90 ° reverse transpose processing on the transposed hierarchical structure picture according to the indication of the 90 ° transpose classification, so as to obtain a corresponding reverse transposed hierarchical structure picture.

Accordingly, when the target transpose classification is a 180 ° transpose classification, the present invention may perform corresponding 180 ° reverse transpose processing on the transposed hierarchy picture according to the indication of the 180 ° transpose classification to obtain a corresponding reverse transposed hierarchy picture.

Accordingly, when the target transpose classification is a 270 ° transpose classification, the present invention may perform a corresponding 270 ° inverse transpose process on the transposed hierarchy picture according to the indication of the 270 ° transpose classification, so as to obtain a corresponding inverse transposed hierarchy picture.

For example, referring to the example of the transposed hierarchy picture shown in fig. 3, after performing corresponding 90 ° inverse transposition processing on the transposed hierarchy picture according to the target transposition classification (i.e. fig. 3 is the 90 ° transposition classification), the present invention can obtain the inverse-transposed hierarchy picture shown in fig. 4.

Then, after the reverse transposed structure picture is obtained, the invention can carry out structural reconstruction on the reverse transposed structure picture so as to obtain a corresponding reconstruction hierarchical structure. Specifically, the present invention may perform structure information extraction (specifically, may perform node extraction and link extraction) on the reverse transposed structure picture to obtain hierarchical structure information in the reverse transposed structure picture, where the hierarchical structure information includes a plurality of hierarchical nodes in the reverse transposed structure picture and logical links between the hierarchical nodes. And further performing structural reconstruction on the plurality of hierarchical nodes and the logical connection lines among the hierarchical nodes to obtain corresponding reconstructed hierarchical structures.

In a specific implementation, the invention may adopt a pre-trained guideline segmentation model to extract the logical links between the level nodes in the reverse transposed structure picture, so as to segment the logical links from the image background of the reverse transposed structure picture, thereby obtaining the logical links between the level nodes in the reverse transposed structure picture, where the number of the logical links is not limited, and is usually multiple. The guideline segmentation model includes, but is not limited to, a contour extraction model, a gaussian line extraction model, a multi-source image line extraction model, or other models for extracting image lines.

Correspondingly, the invention can also adopt a pre-trained key node detection model to detect and extract the level nodes in the reverse inversion structure picture to obtain the level nodes in the reverse inversion structure picture. The number of the hierarchical nodes is not limited, and generally, the number of the hierarchical nodes of the reverse transposed structure picture is plural. The key node detection model includes, but is not limited to, a machine translation Tensorflow model, a finite element ANSYS extraction model, a feed-forward neural network model, a convolutional neural network model, or other models for node extraction, etc.

For example, please refer to fig. 5, which illustrates a schematic diagram of the hierarchical structure information in a possible reverse-inversion structure picture. As shown in fig. 5, the hierarchical structure information includes each hierarchical node in the reverse transposed structure picture and a logical connection between the hierarchical nodes, for example, a first reference numeral in the figure represents a hierarchical node, and a second reference numeral in the figure represents a logical connection.

After the hierarchical nodes in the reverse transposed structure picture and the logical connecting lines between the hierarchical nodes are obtained, the invention can adopt a depth-first search algorithm based on the point-line information to carry out structural reconstruction on the hierarchical nodes and the logical connecting lines between the hierarchical nodes, so as to obtain a corresponding reconstructed hierarchical structure. Specifically, the depth-first search algorithm can be used for performing information fusion on the level information and the logical connection line between the level information, and judging whether connectivity exists between any two level nodes. If the connectivity is provided, connecting the two level nodes provided with the connectivity together through corresponding logic connecting lines; if the connectivity is not available, the subsequent process is continued, for example, whether the connectivity is available between the other two hierarchy nodes is judged. In this principle, a reconstruction of the hierarchy information between the nodes can be achieved to obtain the reconstructed hierarchy.

For example, please refer to fig. 6, which shows a schematic diagram of a possible reconstruction hierarchy. Specifically, the present invention performs structure reconstruction on hierarchical structure information (specifically, a plurality of hierarchical nodes and logical connections between the hierarchical nodes) in the reverse transposed structure picture shown in fig. 5 by using a depth-first search algorithm based on dotted line information, so as to obtain the reconstructed hierarchical structure shown in fig. 6.

Finally, the reverse transposition text and the reconstruction hierarchical structure are associated and matched, so that the restored hierarchical structure picture corresponding to the transposition hierarchical structure picture is obtained. In one embodiment, the reverse pinned text includes at least one reverse pinned text line and the position of each reverse pinned text line (i.e., text box position). Then, the present invention may correspondingly add each of the reverse transposed texts to a corresponding position of the reconstructed hierarchical structure, such as a corresponding hierarchical node, according to the position of each of the reverse transposed text lines, so as to obtain the restored hierarchical structure picture. It can be understood that when the reverse transposed text line includes the text content and the text box of the reverse transposed text line, the present invention specifically can add the text content corresponding to each reverse transposed text line to a corresponding position (such as a hierarchical node) of the reconstructed hierarchical structure according to the position of each reverse transposed text line, so that the linkage matching between the text content and the hierarchical node can be realized to obtain the final picture of the restored hierarchical structure.

In practical application, the reduced hierarchical structure picture may be stored in a file with a preset format, for example, a json file. The restored hierarchical structure picture can also be displayed to a user for viewing and editing in a visual structure tree form.

In an alternative embodiment, after obtaining the restored hierarchical structure picture, the present invention may employ some editing tools to perform information editing processing, such as modification processing, on the restored hierarchical structure picture. Specifically, the present invention may receive an editing instruction of a user for the restored hierarchical structure picture, where the editing instruction is used to request to edit target information in the restored hierarchical structure picture, and the target information includes at least one of the following: and restoring any target text line, any target level node, any target logic connecting line and the like in the hierarchical structure picture. The respective numbers of the target text lines, the target level nodes and the target logical connecting lines are not limited, and can be determined according to the actual needs of users.

Accordingly, after receiving the editing instruction, the present invention may perform corresponding editing processing on the target information in the restored hierarchical structure picture in response to the editing instruction, for example, modify a text line at a certain hierarchical node in the restored hierarchical structure picture, and so on.

By implementing the embodiment of the invention, a transposed hierarchical structure picture is obtained, text recognition is performed on the transposed hierarchical structure picture to obtain a transposed text in the transposed hierarchical structure picture, then image transposition classification is performed on the transposed text to obtain a target transposition classification of the transposed hierarchical structure picture, and finally reduction processing is performed on the transposed hierarchical structure picture according to the target transposition classification to obtain a reduced hierarchical structure picture corresponding to the transposed hierarchical structure picture. In the scheme, the target transposition classification of the whole transposition hierarchical structure picture can be identified based on the transposition text in the transposition hierarchical structure picture, and then the transposition hierarchical structure picture is restored according to the target transposition classification to obtain the final restored hierarchical structure picture, so that the quick restoration of the transposition hierarchical structure picture can be efficiently and accurately realized, the efficiency, the accuracy and the quickness of the restoration of the structural picture are improved, and the technical problems that the accuracy and the reliability of the restoration of the guide picture are reduced and the like caused by the correct restoration of the thought guide picture after the image transposition transformation cannot be realized in the prior art are solved.

Based on the same inventive concept, the embodiment of the present specification further provides a device and an electronic device corresponding to the structural picture restoration method. Fig. 7 is a schematic structural diagram of a structural wafer reduction apparatus according to an embodiment of the present invention. The apparatus 60 shown in fig. 7 comprises: a text recognition module 601, a transposition judgment module 602, a structure restoration module 603 and an information fusion module 604. Wherein:

the text recognition module 601 may also be referred to as an OCR recognition module for recognizing text in a hierarchically structured picture, e.g., the text recognition module 601 may be used in the present invention for recognizing transposed text in the transposed hierarchically structured picture, etc.

The transposition determining module 602 is configured to determine a target transposition classification of the transposed hierarchical structure picture, and specifically may determine/determine a transposed text in the transposed hierarchical structure picture according to the transposed text in the transposed hierarchical structure picture.

The structure restoring module 603 is configured to perform structure restoration or reconstruction on the hierarchical structure information in the transposed hierarchical structure picture according to the target transpose classification, and specifically may include a lead line segmentation unit 6031, a key node detection unit 6032, and a structure search unit 6033. Wherein:

the guideline segmentation unit 6031 is configured to extract guidelines (i.e., links) of the hierarchical picture to obtain logical links between the hierarchical nodes in the hierarchical picture. Specifically, for example, the guideline segmentation unit 6031 in the invention may be used to extract a connection of the reverse inversion structure picture.

The key node detecting unit 6032 is configured to perform node extraction on the hierarchical structure picture to obtain a plurality of hierarchical nodes in the hierarchical structure picture. Specifically, for example, the key node detecting unit 6032 in the present invention may be configured to perform node extraction on the reverse flip structure picture.

The structure searching unit 6033 may be configured to process the information extracted by the guideline dividing unit 6031 and the key node detecting unit 6032, for example, the structure searching unit 6033 may be configured to perform structure reconstruction on a plurality of level nodes in the reverse inverse structure picture and a logical connection between the level nodes, for example, connectivity determination between the nodes and hierarchy structure reconstruction between the nodes, so as to obtain a reconstructed level structure.

The information fusion module 604 is configured to perform association matching on the reverse transposed text corresponding to the transposed text and the reconstructed hierarchical structure, so as to obtain the restored hierarchical structure picture.

For the content that is not introduced or not described in the embodiments of the present invention, reference may be made to the related descriptions in the foregoing method embodiments, and details are not described here again.

Please refer to fig. 8, which is a schematic structural diagram of another structural picture restoration apparatus according to an embodiment of the present invention. The apparatus 70 shown in fig. 8 comprises: an acquisition module 701, an identification module 702, a classification module 703 and a restoration module 704. Wherein:

the obtaining module 701 is configured to obtain a conversion hierarchical structure picture, where the conversion hierarchical structure picture is a hierarchical structure picture after image conversion processing;

the identification module 702 is configured to perform text identification on the converted hierarchical structure picture to obtain an identification text in the converted hierarchical structure picture;

the classification module 703 is configured to perform image transformation classification on the identification text to obtain a target transformation classification of the transformation hierarchy structured picture;

the restoring module 704 is configured to restore the converted hierarchical structure picture according to the target conversion classification, so as to obtain a corresponding restored hierarchical structure picture.

Optionally, when the converted hierarchical structure picture is a transposed hierarchical structure picture after image transposition conversion, the identification text in the converted hierarchical structure picture is a transposed text, and the target conversion classification is a target transposition classification; the restoring module 704 is specifically configured to:

Optionally, the restoring module 704 is specifically configured to:

Optionally, the reverse transposed text includes at least one reverse transposed text line and a position of each reverse transposed text line, and the restoring module 704 is specifically configured to:

Optionally, the apparatus further comprises a receiving module 705 and a processing module 706, wherein:

the receiving module 705 is configured to receive an editing instruction for the restored hierarchical structure picture, where the editing instruction is used to edit target information in the restored hierarchical structure picture, and the target information includes at least one of the following: target text lines, target level nodes and target logical connecting lines;

the processing module 706 is configured to respond to the editing instruction, and perform corresponding editing processing on the target information in the restored hierarchical structure picture.

Optionally, the reduction hierarchy picture comprises an editable reconstruction hierarchy map,

the receiving module 705 is further configured to receive an editing instruction for the reconstructed hierarchical structure diagram, where the editing instruction is used to edit the reconstructed hierarchical structure in the reconstructed hierarchical structure diagram;

the processing module 706 is further configured to respond to the editing instruction, and perform corresponding editing processing on the reconstructed hierarchical structure diagram in the restored hierarchical structure picture.

Optionally, the transposed text comprises at least one transposed text line, and the classification module 703 is specifically configured to:

It should be noted that the recognition module 702 in the apparatus according to the present invention can be implemented by the text recognition module 601 in the apparatus shown in fig. 7 instead, in other words, the recognition module 702 and the text recognition module 601 are modules having the same function. The classification module 703 in the apparatus of the present invention may be implemented by using the transpose determination module 602 in the apparatus shown in fig. 7 instead, and the restoration module 704 in the apparatus of the present invention may be implemented by using the structure restoration module 603 and the information fusion module 604 in the apparatus shown in fig. 7 instead. With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Based on the same inventive concept, an embodiment of the present invention provides an electronic device 800, and fig. 9 is a block diagram illustrating the electronic device 800 according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 9, device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.

The processing component 802 generally controls overall operation of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing elements 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operation at the device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

Power component 806 provides power to the various components of device 800. Power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for device 800.

The multimedia component 808 includes a screen that provides an output interface between the device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data when the device 800 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the device 800. For example, the sensor assembly 814 may detect the open/closed state of the device 800, the relative positioning of the components, such as a display and keypad of the device 800, the sensor assembly 814 may also detect a change in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, orientation or acceleration/deceleration of the device 800, and a change in the temperature of the device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communications component 816 is configured to facilitate communications between device 800 and other devices in a wired or wireless manner. The device 800 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast associated information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communications component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 820 of the device 800 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

Fig. 10 is a schematic diagram of a server in some embodiments of the invention. The server 1900 may vary widely by configuration or performance and may include one or more Central Processing Units (CPUs) 1922 (e.g., one or more processors) and memory 1932, one or more storage media 1930 (e.g., one or more mass storage devices) storing applications 1942 or data 1944. Memory 1932 and storage medium 1930 can be, among other things, transient or persistent storage. The program stored in the storage medium 1930 may include one or more modules (not shown), each of which may include a series of instructions operating on a server. Still further, a central processor 1922 may be provided in communication with the storage medium 1930 to execute a series of instruction operations in the storage medium 1930 on the server 1900.

The server 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input-output interfaces 1958, one or more keyboards 1956, and/or one or more operating systems 1941, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

A non-transitory computer readable storage medium, wherein instructions in the storage medium, when executed by a processor of an apparatus (server or terminal), enable the apparatus to perform the structure picture restoration method of the foregoing embodiments.

A non-transitory computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor of an apparatus (server or terminal), the instructions cause the computer device to perform the description of the method for restoring the structural image in the embodiment corresponding to fig. 1, and therefore, the description will not be repeated here. In addition, the beneficial effects of the same method are not described in detail.

Further, it should be noted that: the embodiments of the present application also provide a technical detail that is not disclosed in the embodiments of the computer program product or the computer program referred to in the present application, please refer to the description of the embodiments of the method in the present application. A computer program product or computer program may comprise computer instructions, which may be stored in a computer readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor can execute the computer instruction, so that the computer device executes the description of the method for restoring the structural slice in the embodiment corresponding to fig. 1, which is described above, and therefore, the description of the method will not be repeated here. In addition, the beneficial effects of the same method are not described in detail.

One or more technical solutions provided by the embodiments of the present invention at least achieve the following technical effects or advantages: the method comprises the steps of obtaining a conversion hierarchical structure picture, carrying out text recognition on the conversion hierarchical structure picture to obtain a recognition text in the conversion hierarchical structure picture, carrying out conversion classification on the recognition text to obtain target conversion classification of the conversion hierarchical structure picture, and finally carrying out reduction processing on the conversion hierarchical structure picture according to the target conversion classification to obtain a reduction hierarchical structure picture corresponding to the conversion hierarchical structure picture. According to the scheme, the target transformation classification of the whole transformation hierarchy structure picture can be identified based on the identification text in the transformation hierarchy structure picture, and then the transformation hierarchy structure picture is restored according to the target transformation classification to obtain the final editable restoration hierarchy structure picture, so that the quick restoration of the transformation hierarchy structure picture can be efficiently and accurately realized, and the efficiency, the accuracy and the quickness of the restoration of the structure picture are improved.

Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is defined only by the appended claims, and is not intended to be limited by the foregoing description, and any modifications, equivalents, improvements, etc. within the spirit and scope of the present invention are intended to be included therein.

Claims

1. A method for restoring a structural picture, comprising:

carrying out image transformation classification on the identification text to obtain a target transformation classification of the transformation hierarchy structure picture;

2. The method according to claim 1, wherein when the transformed hierarchical structure picture is a mirror image hierarchical structure picture after image mirror transformation, the identification text in the transformed hierarchical structure picture is a mirror image text, and the target transformation is classified as a target mirror image classification;

3. The method according to claim 1, wherein when the transform hierarchy picture is a transposed hierarchy picture after an image transposition transform, the identification text in the transform hierarchy picture is a transposed text, and the target transform class is a target transposition class;

4. The method of claim 3, wherein the structurally reconstructing the plurality of level nodes and the logical connections between the level nodes in the inverse transposed structure picture to obtain a reconstructed level structure comprises:

5. The method of claim 3, wherein the reverse transposed text comprises at least one reverse transposed text line and a position of each reverse transposed text line, and the correlating and matching the reverse transposed text with the reconstructed hierarchical structure to obtain the restored hierarchical structure picture comprises:

6. The method of claim 5, further comprising:

receiving an editing instruction for the restored hierarchical structure picture, the editing instruction being used for modifying target information in the restored hierarchical structure picture, the target information including at least one of: target text lines, target level nodes and target logical connecting lines;

and responding to the editing instruction, and performing corresponding modification processing on target information in the restored hierarchical structure picture.

7. The method of claim 5, wherein the reduction hierarchy picture comprises an editable reconstructed hierarchy map, the method further comprising:

8. The method of claim 3, wherein the transposed text comprises at least one line of transposed text, and wherein image transformation classifying the identified text results in a target transformation classification for the transformed hierarchical picture comprising:

9. A structural picture reduction apparatus, comprising:

10. The apparatus according to claim 9, wherein when the transform hierarchy picture is a transposed hierarchy picture after an image transposition transform, the identification text in the transform hierarchy picture is a transposed text, and the target transform class is a target transposition class; the reduction module is specifically configured to:

11. The apparatus of claim 10, wherein the reduction module is specifically configured to:

12. The apparatus of claim 10, wherein the inverted transposed text comprises at least one inverted transposed text line and a position of each inverted transposed text line, and wherein the restoring module is specifically configured to:

13. An electronic device, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors to perform the corresponding operational instructions of the method according to any one of claims 1-8.

14. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps corresponding to the method according to any one of claims 1 to 8.

15. A computer program product, characterized in that it comprises computer instructions stored in a computer readable storage medium and adapted to be read and executed by a processor to cause a computer device having said processor to perform the method of any of claims 1-8.