CN111523169B - Decoration scheme generation method and device, electronic equipment and storage medium - Google Patents
Decoration scheme generation method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111523169B CN111523169B CN202010331237.3A CN202010331237A CN111523169B CN 111523169 B CN111523169 B CN 111523169B CN 202010331237 A CN202010331237 A CN 202010331237A CN 111523169 B CN111523169 B CN 111523169B
- Authority
- CN
- China
- Prior art keywords
- sequence data
- room
- data
- information
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005034 decoration Methods 0.000 title claims abstract description 89
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000000605 extraction Methods 0.000 claims abstract description 77
- 239000013598 vector Substances 0.000 claims description 56
- 238000012546 transfer Methods 0.000 claims description 14
- 230000008439 repair process Effects 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 13
- 230000011218 segmentation Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 239000003086 colorant Substances 0.000 claims description 2
- 238000012545 processing Methods 0.000 description 15
- 230000000694 effects Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 9
- 238000013461 design Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/13—Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Architecture (AREA)
- Civil Engineering (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Structural Engineering (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Processing Or Creating Images (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the invention discloses a decoration scheme generation method, a decoration scheme generation device, electronic equipment and a storage medium. The feature data of each room of the drawing to be repaired is obtained by inputting the acquired drawing to be repaired into a pre-trained feature extraction model, the feature data can be quickly determined by inputting the drawing to be repaired into the feature extraction model, the feature data is not required to be acquired depending on a display platform, the accuracy, efficiency and flexibility of feature data determination are improved, the feature data are converted into the middle sequence data of each room of the drawing to be repaired, the sequence data distance between the middle sequence data and at least one sample sequence data stored in advance is calculated respectively, the target sequence data is determined from the sample sequence data according to the sequence data distance, the layout information corresponding to the target sequence data is determined as the decoration scheme of the drawing to be repaired, and the decoration scheme of the drawing to be repaired can be accurately and quickly determined.
Description
Technical Field
The embodiment of the invention relates to computer technology, in particular to a decoration scheme generation method and device, electronic equipment and storage medium.
Background
With the improvement of the living standard of people, more and more people purchase commodity houses, and the decoration market demand of the commodity houses is also higher and higher. In addition, market clients have higher requirements on the decoration design of rooms, on one hand, the clients want to design according to own expectations, on the other hand, the clients want to quickly see the design effect diagram, and on the other hand, the clients want to save decoration cost. These increases in demand present significant challenges to designers of traditional finishing design companies.
Currently, some decoration design companies utilize deep countermeasure networks to functionally partition the acquired blank house pattern graph, and design a decoration scheme according to each functional partition. However, this approach has poor functional efficiency for the entire blank room and the determined finishing scheme is relatively dead.
It can be seen that the decoration scheme generated by the prior art method has poor effect and needs to be improved.
Disclosure of Invention
The embodiment of the invention provides a decoration scheme generation method, a device, electronic equipment and a storage medium, so as to achieve the effect of improving the design effect and flexibility of a decoration scheme.
In a first aspect, an embodiment of the present invention provides a decoration scheme generating method, including:
Inputting the acquired drawing to be repaired to a pre-trained feature extraction model to obtain feature data of each room of the drawing to be repaired;
converting the characteristic data into intermediate sequence data of each room of the to-be-assembled drawing, wherein the intermediate sequence data comprises tag word vectors and spatial characteristic information of each room;
respectively calculating sequence data distances between the intermediate sequence data and at least one piece of pre-stored sequence data;
and determining target sequence data from the sample sequence data according to the sequence data distance, and determining layout information corresponding to the target sequence data as a decoration scheme of the drawing to be decorated.
In a second aspect, an embodiment of the present invention further provides a decoration scheme generating device, where the decoration scheme generating device includes:
the feature data determining module is used for inputting the acquired drawing to be repaired into a feature extraction model trained in advance to obtain feature data of each room of the drawing to be repaired;
the middle sequence determining module is used for converting the characteristic data into middle sequence data of each room of the to-be-installed repair paper, wherein the middle sequence data comprises tag word vectors and spatial characteristic information of each room;
A sequence data distance calculation module, configured to calculate sequence data distances between the intermediate sequence data and at least one sample sequence data stored in advance, respectively;
and the decoration scheme determining module is used for determining target sequence data from the sample sequence data according to the sequence data distance and determining layout information corresponding to the target sequence data as a decoration scheme of the drawing to be decorated.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor, where the processor implements the decoration scheme generating method according to any one of the first aspects when executing the computer program.
In a fourth aspect, embodiments of the present invention also provide a storage medium containing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, implement the finishing scheme generation method of any one of the first aspects.
According to the technical scheme provided by the embodiment of the invention, the acquired drawing to be repaired is input into the feature extraction model trained in advance to obtain the feature data of each room of the drawing to be repaired, so that the feature data can be quickly determined only by inputting the drawing to be repaired into the feature extraction model, the feature data is not required to be acquired by a display platform, the accuracy, efficiency and flexibility of feature data determination are improved, the feature data are converted into the intermediate sequence data of each room of the drawing to be repaired, the sequence data distance between the intermediate sequence data and at least one sample sequence data stored in advance is calculated respectively, the target sequence data is determined from the sample sequence data according to the sequence data distance, the layout information corresponding to the target sequence data is determined as the fitment scheme of the drawing to be repaired, and the fitment scheme to be repaired can be accurately and quickly determined. The problem of the relatively poor decoration scheme effect of the decoration scheme who generates among the prior art is solved, the effect of improving the accuracy and the efficiency of the decoration scheme of confirming the drawing of waiting to adorn is realized.
Drawings
Fig. 1 is a flow chart of a decoration scheme generating method according to a first embodiment of the present invention;
fig. 2 is a flow chart of a decoration scheme generating method according to a second embodiment of the present invention;
FIG. 3 is a logic diagram of training and applying a feature extraction model according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a decoration scheme generating device according to a third embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Example 1
Fig. 1 is a schematic flow chart of decoration scheme generation provided in a first embodiment of the present invention, where the embodiment is applicable to a case of determining intermediate sequence data through a feature extraction model and determining a decoration scheme according to the intermediate sequence data and sample sequence data, the method may be performed by a decoration scheme generation device, where the device may be implemented by software and/or hardware and is generally integrated in a terminal or an electronic device. Referring specifically to fig. 1, the method may include the steps of:
S110, inputting the acquired drawing to be repaired into a pre-trained feature extraction model to obtain feature data of each room of the drawing to be repaired.
The to-be-assembled repair drawing can be a blank house type drawing, and the characteristic extraction model is used for extracting characteristic data of the blank house type drawing. Optionally, the feature data may include label information, position information, room wall information, door and window information, etc. of the blank room to be repaired corresponding to the blank room type map. The label information can be passenger dining room, owner's room, parent's room, boy's room, girl's room, workman's room, multi-functional room, kitchen, bathroom, life balcony, view balcony, water supply balcony etc. the position information can be the coordinate information in every room, room wall body information can include information such as width information and the height of wall body, door and window information can include information such as width information and the height of door and window.
Optionally, the feature extraction model may be obtained by training the initial network according to the sample decoration drawing and the sample feature data. In this embodiment, the feature extraction model may be an SSD network (Single Shot multiboxDetector, single-shot multi-target detector), and uses VGG-16 (Visual Geometry Group, visual geometry group-16) as a basic network model to perform feature extraction on the drawing to be decorated through multiple convolution layers, so as to obtain feature data of each room. The feature data can be determined by the feature extraction model only by inputting the to-be-assembled repair drawing into the feature extraction model, the feature data can be determined rapidly, the feature data is not required to be acquired by depending on the display platform, and the accuracy, efficiency and flexibility of feature data determination are improved.
And S120, converting the characteristic data into intermediate sequence data of each room of the drawing to be repaired.
The intermediate sequence data comprises tag word vectors and spatial characteristic information of each room. As described in the foregoing steps, the feature data includes tag information, position information, wall information of a room, door and window information, and the like, and the tag word vector in the step is vector data corresponding to the tag information, and the spatial feature information includes position information, wall information of a room, door and window information, and the like.
Optionally, the intermediate sequence data is determined in the following manner: inputting the tag information in the feature data into a word vector model trained in advance to obtain tag word vectors of all rooms of the drawing to be decorated; obtaining initial sequence data of walls in each room of the to-be-installed repair paper based on the tag word vector and second coordinate information and size information in the feature data; and splicing the initial sequence data according to a specific splicing order to obtain the intermediate sequence data of each room.
Optionally, the word vector model may be obtained by training sample tag information and sample tag word vectors, the second coordinate information may be coordinate information of walls of each room or coordinate information of doors and windows, the size information may be width and height of walls of each room or width and height of doors and windows of each room, and the specific splicing order may be clockwise. In this embodiment, the word vector model may select to use a 50-dimensional word vector to obtain a tag word vector of each room, sequentially splice the tag word vector, the second coordinate information and the size information of each room to obtain initial sequence data, and then start each room with a door, splice the initial sequence data of each room again in a clockwise direction to obtain intermediate sequence data of each room.
And S130, respectively calculating sequence data distances between the intermediate sequence data and at least one sample of pre-stored sequence data.
In this embodiment, sample feature data of each room of the room type diagrams of the plurality of sample blanks may be stored in advance, and the sample feature data may be input into the word vector model trained in advance, so as to obtain the sample sequence data. Alternatively, the sample feature data may include sample label information, sample coordinate information, sample size information, and the like of each room, and the sample sequence data includes a label word vector of each room of the sample blank house type graph.
Alternatively, the sequence data distance may be determined by: determining the transfer quantity between two sequences of the intermediate sequence data and the sample sequence data and the distance between the label vectors in the two sequences based on the spatial feature information of any sequence in the intermediate sequence data and the spatial feature information of any sequence in the sample sequence data; and determining a sequence data distance between the intermediate sequence data and the sample sequence data based on the transfer amount between the two sequences and the tag vector distance between the two sequences, wherein the sequence data distance is the minimum value of the sum of products of the transfer amount between the two sequences and the tag vector distance between the two sequences.
Specifically, the sequence data Distance may be calculated according to a WMD (Earth Mover's Distance) algorithm. The objective function of WMD algorithm is:
the constraint conditions are as follows:
wherein T is ij Representing the transfer amount between two sequences of the intermediate sequence data and the sample sequence data, c (i, j) represents the Euclidean distance of the ith value in the intermediate sequence data and the jth value in the sample sequence data, i.e. the label vector distance in the two sequences of the intermediate sequence data and the sample sequence data, d i And d' i The transfer amount of the tag vector and the transfer amount of the tag vector are represented, respectively, as probabilities of occurrence of the ith tag vector in the midamble data.
And S140, determining target sequence data from the sample sequence data according to the sequence data distance, and determining layout information corresponding to the target sequence data as a decoration scheme of the drawing to be decorated.
In this embodiment, the sample feature data and the sample layout information may be stored in advance according to the label information of each room of the sample blank room type map. It will be appreciated that the sequence data distance may be used to determine the similarity between the intermediate sequence data and at least one sample sequence data, the smaller the sequence data distance, the more closely the intermediate sequence data is to the sample sequence data, facilitating the determination of the target sequence data from the intermediate sequence data based on the sequence data distance. As described in the foregoing steps, the sample sequence data includes the tag word vector of each room of the sample blank house type graph, and thus the target sequence data also includes the tag word vector of each room of the sample blank house type graph. Optionally, a tag word vector in the target sequence data can be obtained, the tag word vector of the target sequence data is analyzed, arrangement information corresponding to the target sequence data is determined according to an analysis result, and the arrangement information is determined as a decoration scheme of the drawing to be decorated. Wherein the arrangement information is the sample layout information.
Alternatively, sample sequence data corresponding to a number of data distances smaller than a preset threshold may be determined as the target sequence data. The preset threshold may be a minimum value, and is used for screening target sequence data of the intermediate sequence data. In this embodiment, the sample sequence data may be further sequenced based on the calculated sequence data distance, and the sample sequence data corresponding to the sequence data distance at the preset sequencing position may be determined as the target sequence data. For example, the sample sequence data are arranged in order from small to large, and the sample sequence data corresponding to the 1-3 sequence data distances with the smallest values are used as the target sequence data. The method has the advantage that the decoration scheme of the drawing to be decorated can be quickly and accurately determined.
According to the technical scheme provided by the embodiment of the invention, the acquired drawing to be repaired is input into the feature extraction model trained in advance to obtain the feature data of each room of the drawing to be repaired, so that the feature data can be quickly determined only by inputting the drawing to be repaired into the feature extraction model, the feature data is not required to be acquired by a display platform, the accuracy, efficiency and flexibility of feature data determination are improved, the feature data are converted into the intermediate sequence data of each room of the drawing to be repaired, the sequence data distance between the intermediate sequence data and at least one sample sequence data stored in advance is calculated respectively, the target sequence data is determined from the sample sequence data according to the sequence data distance, the layout information corresponding to the target sequence data is determined as the fitment scheme of the drawing to be repaired, and the fitment scheme to be repaired can be accurately and quickly determined. The problem of the relatively poor decoration scheme effect of the decoration scheme who generates among the prior art is solved, the effect of improving the accuracy and the efficiency of the decoration scheme of confirming the drawing of waiting to adorn is realized.
Example two
Fig. 2 is a flow chart of a decoration scheme generating method according to a second embodiment of the present invention. The technical scheme of the embodiment is refined on the basis of the embodiment. Optionally, the feature extraction model includes a first feature extraction model and at least one second feature extraction model; correspondingly, the inputting the acquired drawing to be repaired to a pre-trained feature extraction model to obtain the feature data of each room of the drawing to be repaired comprises the following steps: inputting the drawing to be repaired to the first feature extraction model to obtain room type label information and first coordinate information of each room of the drawing to be repaired; inputting the room type label information, the first coordinate information and the to-be-installed repair drawing corresponding to the room type label information into a second feature extraction model corresponding to the room type label information to obtain feature data of each room of the to-be-installed repair drawing, wherein the feature data comprise label information, second coordinate information and size information of components in the room. The benefit of refinement is that it facilitates an understanding of the specific manner in which the determination of the feature data is made. For parts which are not described in detail in this method embodiment, reference is made to the above-described embodiments. Referring specifically to fig. 2, the method may include the steps of:
S210, inputting the drawing to be repaired into a first feature extraction model to obtain room type label information and first coordinate information of each room of the drawing to be repaired.
In this embodiment, the feature extraction model may include a first feature extraction model and a second feature extraction model. Optionally, the first feature extraction model may obtain first predicted feature information by inputting a sample decoration drawing into the initial model, calculating a first loss function of the first predicted feature information and the first sample feature information, and reversely adjusting network parameters of the initial model through the first loss function to obtain the first feature extraction model. Optionally, the first prediction feature information includes first prediction room type tag information and first prediction coordinate information, and the first sample feature information includes first sample room type tag information and first sample coordinate information, where the first sample coordinate information may be position information of each room of the sample decoration drawing. Further, after the first feature extraction model is obtained, the drawing to be repaired to be assembled can be input into the first feature extraction model, and room type label information and first coordinate information of each room of the drawing to be repaired are obtained.
Optionally, the expression of the first loss function is:
wherein L is conf(x,c) For category confidence error, L loc(x,l,g) For position error, N is the number of positive samples of the a priori frame,
is an indication parameter, when +>
And when the i priori frame is matched with the j true frame, the correctly marked category (group trunk) is p, c is a category confidence prediction value, l is a position prediction value of the corresponding boundary frame of the priori frame, and g is a position parameter of the group trunk.
As shown in fig. 3, which is a logic schematic diagram of feature extraction model training and application, it can be seen in conjunction with fig. 3 that, before training the first feature extraction model, a water-flooding filling treatment may also be performed on the sample decoration drawing. The sample decoration drawing of the feature extraction model can be subjected to region segmentation according to the spatial feature information of each room of the sample decoration drawing of the feature extraction model to obtain at least one segmentation region, pixel values corresponding to pixel points of the segmentation regions are traversed starting from set seed points of the segmentation regions, the segmentation regions are subjected to color filling according to a traversing result, and the initial model is trained according to the sample decoration drawing after the color filling to obtain the feature extraction model.
Specifically, the sample decoration drawing may be divided into at least one divided area according to spatial feature information such as position information and label information of each room of the sample decoration drawing, for example, divided into areas such as a host room, a parent room, a guest restaurant, a bathroom, a study room, and the like, then an arbitrary point is selected as a seed point in each divided area, and the seed point starts to traverse color values of pixel points of each divided area, so as to fill each divided area with different colors. The method has the advantages that training accuracy and detection accuracy of the first feature model can be improved, and information such as characters and marking lines of a sample decoration drawing can be filtered, so that noise influence is reduced.
Optionally, the normalization processing and the data enhancement processing may also be performed on the sample decoration drawing before the training of the first feature extraction model. Specifically, the sample decoration drawing can be subjected to size transformation processing, picture data are uniformly processed into pictures with 512 x 512 sizes, then the pictures are subjected to horizontal overturning and random sampling to complete data enhancement processing, so that the generalization capability of the model is improved, and the data are subjected to normalization processing, so that the convergence rate of the model can be better improved, and the training time of the first characteristic model is shortened.
S220, inputting the room type label information, the first coordinate information and the to-be-assembled drawing corresponding to the room type label information into a second feature extraction model corresponding to the room type label information, and obtaining feature data of each room of the to-be-assembled drawing.
Wherein the characteristic data includes tag information, second coordinate information, and size information of the components in the room. For example, the feature data includes tag information, coordinate information, and size information of walls of each room, and may include tag information, coordinate information, and size information of doors and windows of each room.
In this embodiment, the second feature extraction model may be obtained by training the initial network according to sample room type tag information, sample coordinate information, sample decoration drawing and feature data of each room of the sample decoration drawing, so as to obtain a second feature extraction model corresponding to each sample room type tag information. The expression of the loss function corresponding to the second feature extraction model is the same as the expression of the loss function of the first feature extraction model, and this step is not specifically explained.
It can be understood that, because the size information and the category information of the components in each room of each sample decoration drawing are different, in order to train the second feature extraction model in a targeted manner, the pre-selected frame proportion and the category of the second feature extraction model can be adjusted according to the size information and the category information of the components in each room of the sample decoration drawing, so that the efficiency and the accuracy of feature data extraction of the second feature extraction model can be improved.
According to the embodiment, the drawing to be decorated is input into the first feature extraction model to obtain the room type label information and the first coordinate information of each room of the drawing to be decorated, and then the feature data of each room of the drawing to be decorated is determined through the second feature extraction model corresponding to the room type label information, so that the accuracy of feature data determination can be improved, and the accuracy of decoration scheme determination is further improved.
And S230, converting the characteristic data into intermediate sequence data of each room of the drawing to be repaired.
S240, respectively calculating sequence data distances between the intermediate sequence data and at least one sample of pre-stored sequence data.
S250, determining target sequence data from sample sequence data according to the sequence data distance, and determining layout information corresponding to the target sequence data as a decoration scheme of a to-be-assembled repair paper.
According to the technical scheme provided by the embodiment of the invention, the first feature extraction model and at least one second feature extraction model are respectively trained in advance, and the feature information of the drawing to be repaired is extracted through the trained first feature extraction model and the second extracted feature model corresponding to the label information of each room type, so that the aim of improving the accuracy of the determination of the decoration scheme can be fulfilled. Before training the first feature extraction model, the sample decoration drawing is subjected to water filling processing, normalization processing and data enhancement processing, so that the training accuracy and detection accuracy of the first feature model can be improved, the convergence speed of the model can be improved, and the training time of the first feature model can be shortened.
Example III
Fig. 4 is a schematic structural diagram of a decoration scheme generating device according to a third embodiment of the present invention. As shown in fig. 4, the apparatus includes: the characteristic data determining module 31, the intermediate sequence determining module 32, the sequence data distance calculating module 33 and the decoration scheme determining module 34.
The feature data determining module 31 is configured to input the acquired drawing to be repaired to a feature extraction model trained in advance, so as to obtain feature data of each room of the drawing to be repaired;
a middle sequence determining module 32, configured to convert the feature data into middle sequence data of each room of the to-be-repaired drawing, where the middle sequence data includes a tag word vector and spatial feature information of each room;
a sequence data distance calculation module 33, configured to calculate sequence data distances between the intermediate sequence data and at least one sample sequence data stored in advance, respectively;
and the decoration scheme determining module 34 is configured to determine target sequence data from the sample sequence data according to the sequence data distance, and determine layout information corresponding to the target sequence data as a decoration scheme of the to-be-decorated drawing.
On the basis of the above embodiments, the feature extraction model includes a first feature extraction model and at least one second feature extraction model;
on the basis of the above embodiments, the feature data determining module 31 is further configured to input the drawing to be repaired to the first feature extraction model, so as to obtain room type label information and first coordinate information of each room of the drawing to be repaired;
inputting the room type label information, the first coordinate information and the to-be-installed repair drawing corresponding to the room type label information into a second feature extraction model corresponding to the room type label information to obtain feature data of each room of the to-be-installed repair drawing, wherein the feature data comprise label information, second coordinate information and size information of components in the room.
The intermediate sequence determining module 32 is further configured to input tag information in the feature data to a word vector model trained in advance, so as to obtain tag word vectors of each room of the drawing to be repaired;
acquiring initial sequence data of each room of the to-be-installed repair paper based on the tag word vector and second coordinate information and size information in the feature data;
And splicing the initial sequence data according to a specific splicing order to obtain the intermediate sequence data of each room.
Based on the above embodiments, the sequence data distance calculating module 33 is further configured to determine a transition amount between two sequences of the intermediate sequence data and the sample sequence data and a label vector distance between two sequences based on spatial feature information of any sequence in the intermediate sequence data and spatial feature information of any sequence in the sample sequence data;
and determining a sequence data distance between the intermediate sequence data and the sample sequence data based on the transfer amount between the two sequences and the label vector distance between the two sequences, wherein the sequence data distance is the minimum value of the product sum of the transfer amount between the two sequences and the label vector distance between the two sequences.
On the basis of the above embodiments, the decoration scheme determining module 34 is further configured to determine, as the target sequence data, sample sequence data corresponding to a sequence data distance less than a preset threshold;
or,
sorting the at least one sample sequence data based on the calculated sequence data distance;
and determining sample sequence data corresponding to the sequence data distance of the preset sequencing position as the target sequence data.
On the basis of the above embodiments, the device further includes: the tag word vector acquisition module and the analysis module; the tag word vector acquisition module is used for acquiring tag word vectors in the target sequence data;
the analysis module is used for analyzing the tag word vector of the target sequence data and determining arrangement information corresponding to the target sequence data according to an analysis result.
On the basis of the above embodiments, the device further includes: a preprocessing module; the device comprises a segmentation module and a filling module; the segmentation module is used for carrying out region segmentation on the sample decoration drawing according to the spatial feature information of each room of the sample decoration drawing of the feature extraction model to obtain at least one segmentation region;
a filling module, configured to start with a set seed point of each of the divided regions, traverse pixel values corresponding to pixel points of each of the divided regions, perform color filling on the divided regions according to the traversing result, and train an initial model according to a color-filled sample decoration drawing to obtain the feature extraction model
According to the technical scheme provided by the embodiment of the invention, the acquired drawing to be repaired is input into the feature extraction model trained in advance to obtain the feature data of each room of the drawing to be repaired, so that the feature data can be quickly determined only by inputting the drawing to be repaired into the feature extraction model, the feature data is not required to be acquired by a display platform, the accuracy, efficiency and flexibility of feature data determination are improved, the feature data are converted into the intermediate sequence data of each room of the drawing to be repaired, the sequence data distance between the intermediate sequence data and at least one sample sequence data stored in advance is calculated respectively, the target sequence data is determined from the sample sequence data according to the sequence data distance, the layout information corresponding to the target sequence data is determined as the fitment scheme of the drawing to be repaired, and the fitment scheme to be repaired can be accurately and quickly determined. The problem of the relatively poor decoration scheme effect of the decoration scheme who generates among the prior art is solved, the effect of improving the accuracy and the efficiency of the decoration scheme of confirming the drawing of waiting to adorn is realized.
Example IV
Fig. 5 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention. Fig. 5 illustrates a block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention. The electronic device 12 shown in fig. 5 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 5, the electronic device 12 is in the form of a general purpose computing device. Components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, a bus 18 that connects the various system components, including the system memory 28 and the processing units 16.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory. The electronic device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from or write to non-removable, nonvolatile magnetic media (not shown in FIG. 5, commonly referred to as a "hard disk drive"). Although not shown in fig. 5, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 18 through one or more data medium interfaces. The memory 28 may include at least one program product having a set of program modules (e.g., a feature data determination module 31, an intermediate sequence determination module 32, a sequence data distance 33 calculation module, and a decoration scheme determination module 34) configured to perform the functions of the various embodiments of the invention.
A program/utility 44 having a set of program modules 46 (e.g., the feature data determination module 31, the intermediate sequence determination module 32, the sequence data distance 33 calculation module, and the decoration scheme determination module 34) may be stored in, for example, the memory 28, such program modules 46 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 46 generally perform the functions and/or methods of the embodiments described herein.
The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the electronic device 12, and/or any devices (e.g., network card, modem, etc.) that enable the electronic device 12 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 22. Also, the electronic device 12 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 over the bus 18. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 12, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, for example, to implement a decoration scheme generating method provided by an embodiment of the present invention, the method including:
inputting the acquired drawing to be repaired to a pre-trained feature extraction model to obtain feature data of each room of the drawing to be repaired;
converting the characteristic data into intermediate sequence data of each room of the to-be-assembled drawing, wherein the intermediate sequence data comprises tag word vectors and spatial characteristic information of each room;
respectively calculating sequence data distances between the intermediate sequence data and at least one piece of pre-stored sequence data;
and determining target sequence data from the sample sequence data according to the sequence data distance, and determining layout information corresponding to the target sequence data as a decoration scheme of the drawing to be decorated.
The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, for example, implementing a decoration scheme generating method provided by an embodiment of the present invention.
Of course, those skilled in the art will understand that the processor may also implement the technical scheme of the decoration scheme generating method provided by any embodiment of the present invention.
Example five
The fifth embodiment of the present invention further provides a computer readable storage medium having a computer program stored thereon, the program when executed by a processor implementing a decoration scheme generating method as provided by the fifth embodiment of the present invention, the method comprising:
inputting the acquired drawing to be repaired to a pre-trained feature extraction model to obtain feature data of each room of the drawing to be repaired;
converting the characteristic data into intermediate sequence data of each room of the to-be-assembled drawing, wherein the intermediate sequence data comprises tag word vectors and spatial characteristic information of each room;
respectively calculating sequence data distances between the intermediate sequence data and at least one piece of pre-stored sequence data;
and determining target sequence data from the sample sequence data according to the sequence data distance, and determining layout information corresponding to the target sequence data as a decoration scheme of the drawing to be decorated.
Of course, the computer-readable storage medium provided by the embodiments of the present invention, on which the computer program stored, is not limited to the above-described method operations, but may also perform the related operations in a decoration scheme generating method provided by any of the embodiments of the present invention.
The computer storage media of embodiments of the invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, or device.
The computer readable signal medium may include, among other things, characteristic data, intermediate sequence data, sequence data distance, and target sequence data, in which computer readable program code is embodied. Such propagated characteristic data, intermediate sequence data, sequence data distance, and target sequence data. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
It should be noted that, in the embodiment of the patch computing device, each module included is only divided according to the functional logic, but not limited to the above-mentioned division, so long as the corresponding function can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present invention.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (7)
1. A decoration scheme generation method, characterized by comprising:
inputting the acquired drawing to be repaired to a pre-trained feature extraction model to obtain feature data of each room of the drawing to be repaired; wherein the feature extraction model comprises a first feature extraction model and at least one second feature extraction model; the characteristic data is obtained specifically by the following modes: inputting the drawing to be repaired to the first feature extraction model to obtain room type label information and first coordinate information of each room of the drawing to be repaired; inputting the room type label information, the first coordinate information and the to-be-installed drawing corresponding to the room type label information into a second feature extraction model corresponding to the room type label information to obtain feature data of each room of the to-be-installed drawing; the characteristic data comprises label information, second coordinate information and size information of the components in the room;
Converting the characteristic data into intermediate sequence data of each room of the to-be-assembled drawing, wherein the intermediate sequence data comprises tag word vectors and spatial characteristic information of each room; the intermediate sequence data is obtained specifically by the following way: inputting the label information in the characteristic data into a pre-trained word vector model to obtain label word vectors of each room of the drawing to be repaired; acquiring initial sequence data of each room of the to-be-installed repair paper based on the tag word vector and second coordinate information and size information in the feature data; splicing the initial sequence data according to a specific splicing order to obtain intermediate sequence data of each room;
respectively calculating sequence data distances between the intermediate sequence data and at least one piece of pre-stored sequence data; the sequence data distance is specifically obtained by the following way: determining the transfer quantity between two sequences of the intermediate sequence data and the sample sequence data and the distance between the label vectors in the two sequences based on the spatial feature information of any sequence in the intermediate sequence data and the spatial feature information of any sequence in the sample sequence data; determining a sequence data distance between the intermediate sequence data and the sample sequence data based on the transfer amount between the two sequences and the tag vector distance between the two sequences, wherein the sequence data distance is the minimum value of the product of the transfer amount between the two sequences and the tag vector distance between the two sequences;
And determining target sequence data from the sample sequence data according to the sequence data distance, and determining layout information corresponding to the target sequence data as a decoration scheme of the drawing to be decorated.
2. The method of claim 1, wherein said determining target sequence data from said sample sequence data based on said sequence data distance comprises:
determining sample sequence data corresponding to the sequence data distance smaller than a preset threshold value as the target sequence data;
or,
sorting the sample sequence data based on the calculated sequence data distance;
and determining sample sequence data corresponding to the sequence data distance of the preset sequencing position as the target sequence data.
3. The method according to claim 1, further comprising, before the determining the layout information corresponding to the target sequence data as the decoration scheme of the to-be-decorated drawing:
acquiring a tag word vector of the target sequence data;
and analyzing the tag word vector of the target sequence data, and determining arrangement information corresponding to the target sequence data according to an analysis result.
4. The method according to claim 1, wherein the method further comprises:
According to the spatial feature information of each room of the sample decoration drawing of the feature extraction model, carrying out region segmentation on the sample decoration drawing to obtain at least one segmentation region;
starting with the set seed points of each divided area, traversing the pixel values corresponding to the pixel points of each divided area, and filling the colors of the divided areas according to the traversing result;
training the initial model according to the sample decoration drawing filled with the color to obtain the feature extraction model.
5. A decoration scheme generating device, characterized by comprising:
the feature data determining module is used for inputting the acquired drawing to be repaired into a feature extraction model trained in advance to obtain feature data of each room of the drawing to be repaired; wherein the feature extraction model comprises a first feature extraction model and at least one second feature extraction model; the characteristic data is obtained specifically by the following modes: inputting the drawing to be repaired to the first feature extraction model to obtain room type label information and first coordinate information of each room of the drawing to be repaired; inputting the room type label information, the first coordinate information and the to-be-installed drawing corresponding to the room type label information into a second feature extraction model corresponding to the room type label information to obtain feature data of each room of the to-be-installed drawing; the characteristic data comprises label information, second coordinate information and size information of the components in the room;
The middle sequence determining module is used for converting the characteristic data into middle sequence data of each room of the to-be-installed repair paper, wherein the middle sequence data comprises tag word vectors and spatial characteristic information of each room; the intermediate sequence data is obtained specifically by the following way: inputting the label information in the characteristic data into a pre-trained word vector model to obtain label word vectors of each room of the drawing to be repaired; acquiring initial sequence data of each room of the to-be-installed repair paper based on the tag word vector and second coordinate information and size information in the feature data; splicing the initial sequence data according to a specific splicing order to obtain intermediate sequence data of each room;
a sequence data distance calculation module, configured to calculate sequence data distances between the intermediate sequence data and at least one sample sequence data stored in advance, respectively; the sequence data distance is specifically obtained by the following way: determining the transfer quantity between two sequences of the intermediate sequence data and the sample sequence data and the distance between the label vectors in the two sequences based on the spatial feature information of any sequence in the intermediate sequence data and the spatial feature information of any sequence in the sample sequence data; determining a sequence data distance between the intermediate sequence data and the sample sequence data based on the transfer amount between the two sequences and the tag vector distance between the two sequences, wherein the sequence data distance is the minimum value of the product of the transfer amount between the two sequences and the tag vector distance between the two sequences;
And the decoration scheme determining module is used for determining target sequence data from the sample sequence data according to the sequence data distance and determining layout information corresponding to the target sequence data as a decoration scheme of the drawing to be decorated.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the decoration scheme generation method of any one of claims 1-4 when the computer program is executed by the processor.
7. A storage medium containing computer executable instructions which when executed by a computer processor implement the decoration scheme generation method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010331237.3A CN111523169B (en) | 2020-04-24 | 2020-04-24 | Decoration scheme generation method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010331237.3A CN111523169B (en) | 2020-04-24 | 2020-04-24 | Decoration scheme generation method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111523169A CN111523169A (en) | 2020-08-11 |
| CN111523169B true CN111523169B (en) | 2023-06-13 |
Family
ID=71911046
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010331237.3A Active CN111523169B (en) | 2020-04-24 | 2020-04-24 | Decoration scheme generation method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111523169B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112651801B (en) * | 2020-12-23 | 2022-04-15 | 北京城市网邻信息技术有限公司 | Method and device for displaying house source information |
| CN114969877A (en) * | 2021-02-24 | 2022-08-30 | 广东博智林机器人有限公司 | Door and window arrangement method, device, equipment and storage medium |
| CN113486432B (en) * | 2021-07-15 | 2022-02-18 | 苏州苏明装饰股份有限公司 | Assembly decoration module combination method and system based on standardization |
| CN115796556A (en) * | 2023-02-01 | 2023-03-14 | 北京有竹居网络技术有限公司 | Decoration scheme determination method and device, electronic equipment and readable storage medium |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104778756B (en) * | 2015-04-10 | 2017-11-21 | 北京明兰网络科技有限公司 | Smart home Art Design system |
| CA3055597A1 (en) * | 2017-03-17 | 2018-09-20 | Magic Leap, Inc. | Room layout estimation methods and techniques |
| CN107832476B (en) * | 2017-12-01 | 2020-06-05 | 北京百度网讯科技有限公司 | Method, device, equipment and storage medium for understanding search sequence |
| US20190354850A1 (en) * | 2018-05-17 | 2019-11-21 | International Business Machines Corporation | Identifying transfer models for machine learning tasks |
| CN108984904B (en) * | 2018-07-17 | 2022-09-20 | 北京理工大学 | Home design method based on deep neural network |
| US11429824B2 (en) * | 2018-09-11 | 2022-08-30 | Intel Corporation | Method and system of deep supervision object detection for reducing resource usage |
| CN109933662B (en) * | 2019-02-15 | 2021-03-12 | 北京奇艺世纪科技有限公司 | Model training method, information generation method, device, electronic equipment and computer readable medium |
| CN110363853B (en) * | 2019-07-15 | 2020-09-01 | 贝壳找房(北京)科技有限公司 | Furniture placement scheme generation method, device and equipment and storage medium |
| KR20190106867A (en) * | 2019-08-27 | 2019-09-18 | 엘지전자 주식회사 | An artificial intelligence apparatus for guiding arrangement location of furniture and operating method thereof |
| CN110929319B (en) * | 2019-11-01 | 2023-04-07 | 深圳市彬讯科技有限公司 | Decoration design case information processing method and system |
| CN110826328A (en) * | 2019-11-06 | 2020-02-21 | 腾讯科技（深圳）有限公司 | Keyword extraction method and device, storage medium and computer equipment |
-
2020
- 2020-04-24 CN CN202010331237.3A patent/CN111523169B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111523169A (en) | 2020-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111523169B (en) | Decoration scheme generation method and device, electronic equipment and storage medium | |
| CN107293296B (en) | Voice recognition result correction method, device, equipment and storage medium | |
| EP3543960B1 (en) | Information processing device, information processing method, and program | |
| US10395147B2 (en) | Method and apparatus for improved segmentation and recognition of images | |
| CN110059734B (en) | Training method of target recognition classification model, object recognition method, object recognition device, robot and medium | |
| CN112016638B (en) | Method, device and equipment for identifying steel bar cluster and storage medium | |
| CN110929802A (en) | Information entropy-based subdivision identification model training and image identification method and device | |
| CN111311613B (en) | Image segmentation model training method, image segmentation method and device | |
| CN111242291A (en) | Neural network backdoor attack detection method and device and electronic equipment | |
| CN112288831A (en) | Scene image generation method and device based on generation countermeasure network | |
| CN111563429A (en) | Drawing verification method and device, electronic equipment and storage medium | |
| CN113378852A (en) | Key point detection method and device, electronic equipment and storage medium | |
| CN112580666A (en) | Image feature extraction method, training method, device, electronic equipment and medium | |
| CN114266860A (en) | Three-dimensional face model establishing method and device, electronic equipment and storage medium | |
| CN110647675A (en) | Method and device for recognition of stop point and training of prediction model and storage medium | |
| CN114220163B (en) | Human body posture estimation method and device, electronic equipment and storage medium | |
| CN108460335B (en) | Video fine-granularity identification method and device, computer equipment and storage medium | |
| CN114581732A (en) | Image processing and model training method, device, equipment and storage medium | |
| CN114549904A (en) | Visual processing and model training method, apparatus, storage medium, and program product | |
| CN114581794A (en) | Geographic digital twin information acquisition method and device, electronic equipment and storage medium | |
| CN113963211B (en) | Unsupervised domain adaptation training method and system for gesture recognition | |
| CN112541557B (en) | Training method and device for generating countermeasure network and electronic equipment | |
| CN111612890B (en) | Method and device for automatically generating three-dimensional model by two-dimensional house type graph and electronic equipment | |
| CN114973333A (en) | Human interaction detection method, human interaction detection device, human interaction detection equipment and storage medium | |
| CN114937187A (en) | Image optimization method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |