CN108983769B - Instant positioning and map construction optimization method and device - Google Patents
Instant positioning and map construction optimization method and device Download PDFInfo
- Publication number
- CN108983769B CN108983769B CN201810650981.2A CN201810650981A CN108983769B CN 108983769 B CN108983769 B CN 108983769B CN 201810650981 A CN201810650981 A CN 201810650981A CN 108983769 B CN108983769 B CN 108983769B
- Authority
- CN
- China
- Prior art keywords
- processed
- image
- stability
- map
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000005457 optimization Methods 0.000 title claims abstract description 29
- 238000010276 construction Methods 0.000 title abstract description 9
- 238000012545 processing Methods 0.000 claims abstract description 25
- 238000013507 mapping Methods 0.000 claims abstract description 10
- 230000006870 function Effects 0.000 claims description 13
- 238000010801 machine learning Methods 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 2
- 230000000873 masking effect Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 6
- 238000012937 correction Methods 0.000 description 5
- 239000000463 material Substances 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0251—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0223—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving speed control of the vehicle
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Automation & Control Theory (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Aviation & Aerospace Engineering (AREA)
- Theoretical Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an optimization method and device for instant positioning and map construction. The instant positioning and mapping optimization method comprises the following steps: acquiring the stability characteristic of an image to be processed; masking the similar stability characteristics to construct an initial map and location from the masked stability characteristics; tracking the image to be processed after initialization processing according to the stability characteristic; and optimizing the constructed map and the constructed position according to the stability characteristics. According to the technical scheme, the stability characteristics of the image to be processed are obtained, on one hand, the map and the position can be initialized according to the stability characteristics similar to shielding, on the other hand, the image to be processed can be tracked and optimized according to the stability characteristics, and the stability characteristics are obtained according to the attributes of the object and are relatively stable, so that the loss of the characteristics can be effectively prevented in the processes of initializing, tracking and optimizing the map and the position.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to an optimization method and device for instant positioning and map construction.
Background
The instant positioning and Mapping (SLAM), also called cml (coordinated Mapping and Mapping), refers to a robot creating a map in a completely unknown environment under the condition that its own position is uncertain, and simultaneously, autonomously positioning and navigating by using the map. Based on the positioning mode of the visual sensor, the method is a hot spot of domestic and foreign research in recent years, and is divided into monocular, binocular and multiocular positioning. Currently, research on the monocular SLAM method focuses on improving the speed and accuracy of positioning and composition in indoor or road scenes. In these scenarios, the features are mostly distinguishable, so it is easy to estimate the camera pose from matching features or pixels. However, in many scenarios, features are not easily distinguishable, which can lead to problems with scale drift and loss of tracking.
Conventionally, indirect methods (ORB-SLAM), direct methods (lsd-SLAM), and the like have been used to solve this problem. Among them, the indirect method works well in most scenarios and involves complete functionality including repositioning, but it relies heavily on feature extraction and feature matching. Direct methods directly use the actual sensor values, but the problems of scale drift and relocation are difficult to solve in large scenarios.
Disclosure of Invention
The invention provides an optimization method and device for instant positioning and map construction, and provides a beneficial choice for solving one or more technical problems in the background art.
As an aspect of the present invention, an embodiment of the present invention provides an optimization method for instant positioning and map building, including:
acquiring the stability characteristic of an image to be processed;
masking similar said stability characteristics to construct an initial map and location from said stability characteristics;
tracking the image to be processed after initialization processing according to the stability characteristic;
and optimizing the constructed map and the constructed position according to the stability characteristics.
With reference to the first aspect, this example of the invention in a first implementation of the first aspect, the stability feature includes at least one of: corner points, shape, size, color.
With reference to the first implementation manner of the first aspect, in a second implementation manner of the first aspect, an embodiment of the present invention masks similar stability features for performing initialization processing on a map and a location, where the initialization processing includes:
acquiring a sub-feature of the stability feature;
clustering the sub-features to obtain sub-features with similarity greater than a preset threshold;
and shielding the sub-features with the similarity larger than a preset threshold.
With reference to the first implementation manner of the first aspect, in a third implementation manner of the first aspect, an embodiment of the present invention masks similar stability features for performing initialization processing on a map and a location, where the initialization processing includes:
adopting a preset machine learning model to segment the similar stability characteristic region of the image to be processed so as to shield the similar stability characteristic region; wherein the similarity of the stability characteristics in the similar stability characteristic region is greater than a preset threshold.
With reference to the first aspect, in a fourth implementation manner of the first aspect, an embodiment of the present invention provides that, according to the stability feature, performing optimization processing on the constructed map and the constructed location includes:
and correcting the scale drift of the map and the position by adopting a corresponding correction method according to the tracking condition of the object indicated by the image to be processed.
With reference to the first aspect, in a first implementation manner of the fourth implementation manner of the first aspect, the embodiment of the present invention employs a corresponding correction method to correct the scale drift of the map and the position according to the tracking condition of the object indicated by the image to be processed, and includes:
if the object indicated by the image to be processed can be tracked, the scale drift of the map and the position is corrected by adopting a light beam adjustment method.
With reference to the fourth implementation manner of the first aspect, in a second implementation manner of the fourth implementation manner, according to a tracking situation of an object indicated by the image to be processed, the correcting the scale drift of the map and the position by using a corresponding correction method includes:
and if the object indicated by the image to be processed cannot be tracked, correcting the scale drift of the map and the position by a preset incremental function.
In a second aspect, an embodiment of the present invention provides an instant positioning and mapping apparatus, where the apparatus includes:
the acquisition module is configured to acquire the stability characteristics of the image to be processed;
a shielding module configured to shield similar stability features for map and location initialization processing;
the tracking module is configured to track the to-be-processed image after initialization processing according to the stability characteristics;
and the optimization module is configured to perform optimization processing on the constructed map and the constructed position according to the stability characteristics.
With reference to the second aspect, in a first implementation of the second aspect of the embodiments of the present invention, the shielding module includes:
an obtaining sub-module configured to obtain sub-features of the stability feature;
the clustering sub-module is configured to perform clustering processing on the sub-features to obtain sub-features of which the similarity is greater than a preset threshold;
and the first shielding submodule is configured to shield the sub-features of which the similarity is greater than a preset threshold.
With reference to the second aspect, in a second implementation manner of the second aspect of the embodiments of the present invention, the initialization module includes:
the second shielding submodule is configured to segment the similar stability characteristic region of the image to be processed by adopting a preset machine learning model so as to shield the similar stability characteristic region; wherein the similarity of the stability characteristics in the similar stability characteristic region is greater than a preset threshold.
By adopting the technical scheme, the invention has the following advantages: according to the technical scheme, the stability characteristics of the image to be processed are obtained, on one hand, the map and the position can be initialized according to the stability characteristics similar to shielding, on the other hand, the image to be processed can be tracked and optimized according to the stability characteristics, and the stability characteristics are obtained according to the attributes of the object and are relatively stable, so that the loss of the characteristics can be effectively prevented in the processes of initializing, tracking and optimizing the map and the position.
Drawings
In the drawings, like reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily to scale. It is appreciated that these drawings depict only some embodiments in accordance with the disclosure and are therefore not to be considered limiting of its scope.
Fig. 1 is a flowchart of an optimization method for instant positioning and map building according to a first embodiment of the present invention;
FIG. 2 is a flowchart of an instant positioning and mapping optimization method according to a second embodiment of the present invention;
fig. 3(1) is an image of a solar panel before a phase region feature area segmentation is performed on an image to be processed in the optimization method for real-time positioning and map construction according to the second embodiment of the present invention;
fig. 3(2) is an image obtained by shielding similar feature areas of a solar panel in the instant positioning and mapping optimization method according to the second embodiment of the present invention;
fig. 4 is a schematic diagram of the generation of scale drift in the optimization method of instant positioning and map construction according to the second embodiment of the present invention;
fig. 5 is a schematic diagram illustrating a principle of correcting the scale drift according to the attribute of the object in the optimization method of instant positioning and map building according to the second embodiment of the present invention;
fig. 6 is a schematic diagram of correcting the scale drift of the image to be processed in the optimization method of instant positioning and map building according to the second embodiment of the present invention;
fig. 7 is a schematic diagram of correcting the scale drift of the image to be processed in the optimization method of instant positioning and map building according to the second embodiment of the present invention;
fig. 8 is a schematic diagram of an instant positioning and mapping apparatus according to a third embodiment of the present invention.
Detailed Description
In the following, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.
In the description of the present invention, it is to be understood that the terms "central," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," "circumferential," and the like are used in the orientations and positional relationships indicated in the drawings for convenience in describing the invention and to simplify the description, and are not intended to indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and are not to be considered limiting of the invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; the connection can be mechanical connection, electrical connection or communication; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, "above" or "below" a first feature means that the first and second features are in direct contact, or that the first and second features are not in direct contact but are in contact with each other via another feature therebetween. Also, the first feature being "on," "square," and "over" the second feature includes the first feature being directly above and obliquely above the second feature, or merely indicating that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature includes the first feature being directly above and obliquely above the second feature, or simply meaning that the first feature is at a lesser level than the second feature.
The following disclosure provides many different embodiments or examples for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Of course, they are merely examples and are not intended to limit the present invention. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples, such repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. In addition, the present invention provides examples of various specific processes and materials, but one of ordinary skill in the art will recognize the application of other processes and/or the use of other materials.
At present, the indirect method is widely used, but the performance of the indirect method is greatly influenced by the field characteristics. For example, in a repetitive feature scenario, the indirect method is difficult to initialize and easily lost due to the effects of similar characteristics.
Example one
Fig. 1 is a schematic diagram of an optimization method for instant positioning and map building according to an embodiment of the present invention. The instant positioning and map construction optimization method provided by the embodiment of the invention comprises the following steps:
s101, acquiring stability characteristics of the image to be processed.
The image to be processed of the embodiment of the invention can be acquired by a camera installed on the intelligent equipment. Such as a smart robot.
In specific implementation, because the map and the position are three-dimensional spaces, and the image belongs to a two-dimensional image, the embodiment of the present invention generally obtains at least two images or image frames to be processed. First, the stability characteristics on the image to be processed need to be acquired.
Wherein the stability characteristics include at least one of: corner points, shape, size, color.
The stability characteristics are obtained from properties of the object. For example, the stability feature may be a corner point or a point on some special structure. The corner point is an extreme point, that is, a point with a special attribute in some aspect. Of course, the attribute of the corner point can be defined by the user according to the actual requirement (setting a specific entropy value for corner point detection). The corner point may be the intersection of two lines or a point located on two adjacent objects with different main directions. Some points on a particular structure are points on a particular object that identify attributes of the object, such as the outline or shape of a graphical object.
S102, shielding the similar stability characteristics to construct an initial map and a position according to the shielded stability characteristics.
In the process of initializing the map and the position, if two images to be processed have similar features, the feature matching and the calculation of the basic matrix or the homography matrix in the initialization process are influenced, so that the embodiment of the invention firstly shields the similar features to avoid repeated features. All features having similarity are not masked out, but at least one or more features are retained and other features similar to the one or more features are masked out.
As will be understood by those skilled in the art, initializing the map and the location is to build a three-dimensional map and locate an object according to corresponding feature points in at least two images to be processed.
The features on the image to be processed can be compared by different methods, for example, similar features can be determined by a clustering method, a method of segmenting similar regions of the image, and the like.
S103, tracking the image to be processed after initialization processing according to the stability characteristics.
Stability is provided because the stability characteristics are derived from the properties of the object. Therefore, the image to be processed can be tracked with reference to the stability characteristics of the object.
And S104, optimizing the constructed map and position according to the stability characteristics.
Similarly, the stability characteristic is obtained according to the property of the object, so that the stability is provided. Thus, the image to be processed can be optimized with reference to the stability characteristics of the object.
It should be noted that in the step of acquiring the stability characteristics of the image to be processed, the embodiment of the present invention actually acquires not only the stability characteristics of the image to be processed, but also the basic characteristics of the image to be processed while acquiring the stability characteristics. When masking similar features, both the base and stability features are masked and then used for map and location initialization. The embodiment of the invention is for simplicity in describing the technical scheme, and does not focus on shielding processing of basic features by pen ink. It will be appreciated by those skilled in the art that in acquiring the features and the mask features, all of the features, i.e., the base features and the stability features, are processed simultaneously.
According to the technical scheme, the stability characteristics of the image to be processed are obtained, on one hand, the map and the position can be initialized according to the stability characteristics similar to shielding, on the other hand, the image to be processed can be tracked and optimized according to the stability characteristics, and the stability characteristics are obtained according to the attributes of the object and are relatively stable, so that the loss of the characteristics can be effectively prevented in the processes of initializing, tracking and optimizing the map and the position.
Example two
On the basis of the first embodiment, the second embodiment of the present invention describes the implementation process of the present invention in more detail. Fig. 2 is a flowchart of an optimization method for instant positioning and map building according to a second embodiment of the present invention. The instant positioning and map construction optimization method provided by the embodiment of the invention comprises the following steps:
s201, acquiring stability characteristics of the image to be processed.
Step S201 of this embodiment corresponds to step S101 of the first embodiment.
S202, acquiring the sub-characteristics of the stability characteristics.
The sub-features involved in embodiments of the present invention are not subdivided for individual features, but rather all features are divided into different subsets. Since all features cannot be processed at once, the processing can be performed one by one after being divided into subsets, thereby increasing the processing speed.
S203, clustering the sub-features to obtain the sub-features with the similarity greater than a preset threshold.
Clustering is the process of dividing a collection of physical or abstract objects into classes composed of similar objects, called clustering. The cluster generated by clustering is a collection of a set of data objects that are similar to objects in the same cluster and distinct from objects in other clusters.
The embodiment of the invention carries out clustering processing on the sub-features, and can determine the similar sub-features.
In other embodiments of the invention, shielding similar said stability characteristics may also be achieved by:
adopting a preset machine learning model to segment the similar stability characteristic region of the image to be processed so as to shield the similar stability characteristic region; and the similarity of the characteristic points in the similar stability characteristic region is greater than a preset threshold value.
And S204, shielding the sub-features of which the similarity is greater than a preset threshold.
The embodiment of the invention can adopt a machine learning algorithm to train a preset machine learning model, so that the machine learning model can filter or detect similar characteristic regions in the image to be processed. As shown in fig. 3(1), an image of the solar panel before being divided is shown, and fig. 3(2) is shown after the similar feature region of the solar panel is masked.
Steps S202 to S204 of the present embodiment correspond to step S102 of the first embodiment.
S205, tracking the image to be processed according to the stability characteristics.
Step S204 of this embodiment corresponds to step S103 of the first embodiment.
In general, smart devices locate and track objects through image processing. In the tracking process, when the image to be processed is full of similar features, the tracking is easily lost. With the conventional technique, the feature point detected at time k is not necessarily detected at time k + 1. In order to prevent tracking loss, the embodiment of the invention can extract the same feature point at different times by using the attribute of the object. For example, the corner points of the object are taken as stability features. Therefore, when the object is tracked, the corner points of the object can be acquired at any time, and the tracked image is prevented from being lost.
The method for tracking the graph to be processed may refer to a tracking method in the conventional art, and is not described herein again.
And S206, optimizing the constructed map and position according to the stability characteristics.
Among them, the scale drift is an inherent problem of monocular SLAM. As shown in fig. 4, when the same object is observed at different times, the object may be considered as a different object at other viewing angles due to the occurrence of a dimensional shift. Moreover, once the scale drift occurs, errors will occur in the maps and locations that are subsequently built. As shown in fig. 5, when the camera photographs three rectangular objects, i.e., an object 1, an object 2, and an object 3, images formed by the cameras overlap due to the difference in the distance of the three rectangular objects. At this time, the currently constructed map may be subjected to scale correction according to the size features among the stability features of the three rectangular objects. Wherein the dimension is one of the properties of the object, and the dimension characteristic is thus obtainable from the property of the object.
When the scale drift of the map and the position is corrected, the correction can be respectively carried out in the following two cases.
If the object indicated by the image to be processed can be tracked, the scale drift of the map and the position is corrected by using a light beam adjustment method.
As shown in fig. 6, the same object is observed at times k, k + i, and k + j, respectively. At each time, the object may be mapped into three-dimensional coordinates. Because of the dimensional drift, the object varies in both position and size in three-dimensional coordinates, and therefore an energy function can be defined to describe this difference:
wherein r isiAnd rjIs the object observed at times i and j, function S (r)i，rj) Is shown at time riAnd time rjThe shape or size of the object in three-dimensional coordinates is different.
Since the estimation of the object in three-dimensional coordinates is related to the pose of the camera, the position and size differences of the object at different times can be minimized by optimizing the pose of the camera. The calculation formula is as follows:
wherein T represents the attitude of the camera, EerrRepresenting the position and size differences of the object at different times,
representing the set of keyframes that need to be optimized, R represents objects observed in the set of keyframes,
and if the object indicated by the image to be processed cannot be tracked, correcting the scale drift of the map and the position by a preset increasing function.
When the object cannot be tracked, the position and size of the object at different times cannot be obtained, and therefore, the scale estimation needs to be performed on each new key frame. As shown in fig. 7, an object is observed in a keyframe and its position in three-dimensional coordinates is obtained. Given that the difference between known real shapes can be defined as:
Dr(rk)＝αkD(rk)+v (3)
wherein alpha iskDenotes the modified scale parameter, v denotes error noise, k denotes key frame, D (r)k) Representing the shape estimate.
When a new key frame is inserted into the image to be processed, the camera pose of this frame can be obtained by the following incremental function:
wherein,
indicating the corrected camera pose, TkT-1 k-1Representing a transformation matrix, function, transforming coordinates in k-1 frames into k frames
Representing a similarity transformation function, usable for updating the pose, alpha, of the camerakRepresenting the random variables that need to be estimated in the k-th frame.
In the tracking and optimizing steps, the stability characteristics are referred for processing; therefore, on one hand, in the tracking process, the system can stably estimate the posture of the camera; on the other hand, in the optimization process, the scale drift in the map building process can be corrected according to the shape or the size of the object.
EXAMPLE III
The third embodiment of the invention provides an instant positioning and map building device. As shown in fig. 8, the instant positioning and mapping apparatus according to the embodiment of the present invention includes:
an obtaining module 81 configured to obtain a stability characteristic of an image to be processed;
a masking module 82 configured to mask similar said stability characteristics to construct an initial map and location from said stability characteristics;
a tracking module 83 configured to track the to-be-processed image after the initialization processing according to the stability feature;
and an optimization module 84 configured to perform optimization processing on the constructed map and position according to the stability characteristics.
Further, the shielding module includes:
an obtaining sub-module 821 configured to obtain sub-features of the stability feature;
a clustering submodule 822 configured to perform clustering processing on the sub-features to obtain sub-features of which the similarity is greater than a preset threshold;
the first shielding sub-module 823 is configured to shield the sub-features with the similarity greater than a preset threshold.
In another alternative embodiment of the present invention, the shielding module 82 may further include:
a second shielding submodule (not shown in the figure) configured to segment the similar stability feature region of the image to be processed by using a preset machine learning model so as to shield the similar stability feature region; wherein the similarity of the stability characteristics in the similar stability characteristic region is greater than a preset threshold.
The technical scheme has the following advantages: according to the technical scheme, the stability characteristics of the image to be processed are referred in the initialization, tracking and optimization processes, so that the characteristics are prevented from being lost in the tracking process.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive various changes or substitutions within the technical scope of the present invention, and these should be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (6)
1. An optimization method for instant positioning and map building, which is characterized by comprising the following steps:
s201: acquiring the stability characteristic of an image to be processed;
s202: acquiring a sub-feature of the stability feature;
s203: clustering the sub-features to obtain sub-features with similarity greater than a preset threshold;
s204: shielding the sub-features with the similarity larger than a preset threshold;
s205: tracking the image to be processed according to the stability characteristics;
s206: according to the stability characteristics, optimizing the constructed map and position:
when the scale drift of the map and the position is corrected, the following two conditions can be respectively processed: if the object indicated by the image to be processed can be tracked, correcting the scale drift of the map and the position by using a beam adjustment method, which comprises the following steps:
at times k, k + i and k + j, respectively, the same object is observed, defining an energy function to describe this difference:
wherein r isiAnd rjIs the object observed at times i and j, function S (r)i,rj) Is shown at time riAnd time rjThe shape or size difference of the object in the three-dimensional coordinates;
since the estimation of the object in three-dimensional coordinates is related to the pose of the camera, the position and size differences of the object at different times can be minimized by optimizing the pose of the camera, the calculation formula is as follows:
wherein T represents the attitude of the camera, EerrRepresenting the position and size differences of the object at different times,
representing the set of keyframes that need to be optimized, R represents objects observed in the set of keyframes,
if the object indicated by the image to be processed cannot be tracked, correcting the scale drift of the map and the position by a preset increasing function:
when the object cannot be tracked, the position and size of the object at different times cannot be obtained, for example, the difference between known real shapes can be defined as:
Dr(rk)＝αkD(rk)+v (3)
wherein alpha iskDenotes the modified scale parameter, v denotes error noise, k denotes key frame, D (r)k) Representing the shape estimate, when a new key frame is inserted into the image to be processed, the camera pose of this frame can be obtained by the following incremental function:
wherein,
indicating the corrected camera pose, TkT-1 k-1Representing a transformation matrix, function, transforming coordinates in k-1 frames into k frames
Representing a phaseLike a transformation function, and can be used to update the pose, alpha, of the camerakIndicating the random variables that need to be estimated in the k-th frame.
2. The method of claim 1, wherein the stability characteristics include at least one of: corner points, shape, size and color.
3. The method according to claim 1, wherein in the step 203, a preset machine learning model is used to segment a similar stability feature region of the image to be processed so as to mask the similar stability feature region.
4. An instant positioning and mapping apparatus, the apparatus comprising:
the acquisition module is configured to acquire the stability characteristics of the image to be processed;
a shielding module configured to shield similar stability features for map and location initialization processing;
the tracking module is configured to track the to-be-processed image after the initialization processing according to the stability characteristic;
and the optimization module is configured to perform optimization processing on the constructed map and the constructed position according to the stability characteristics.
5. The apparatus of claim 4, wherein the shielding module comprises:
an obtaining sub-module configured to obtain sub-features of the stability feature;
the clustering sub-module is configured to perform clustering processing on the sub-features to obtain sub-features of which the similarity is greater than a preset threshold;
and the first shielding submodule is configured to shield the sub-features of which the similarity is greater than a preset threshold.
6. The apparatus of claim 5, wherein the shielding module comprises:
the second shielding submodule is configured to segment the similar stability characteristic region of the image to be processed by adopting a preset machine learning model so as to shield the similar stability characteristic region; wherein the similarity of the stability characteristics in the similar stability characteristic region is greater than a preset threshold.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810650981.2A CN108983769B (en) | 2018-06-22 | 2018-06-22 | Instant positioning and map construction optimization method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810650981.2A CN108983769B (en) | 2018-06-22 | 2018-06-22 | Instant positioning and map construction optimization method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108983769A CN108983769A (en) | 2018-12-11 |
| CN108983769B true CN108983769B (en) | 2022-06-21 |
Family
ID=64538147
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810650981.2A Active CN108983769B (en) | 2018-06-22 | 2018-06-22 | Instant positioning and map construction optimization method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108983769B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109887003B (en) * | 2019-01-23 | 2021-11-19 | 亮风台（上海）信息科技有限公司 | Method and equipment for carrying out three-dimensional tracking initialization |
| CN109965797B (en) * | 2019-03-07 | 2021-08-24 | 深圳市愚公科技有限公司 | Floor sweeping robot map generation method, floor sweeping robot control method and terminal |
| CN112767818B (en) * | 2019-11-01 | 2022-09-27 | 北京初速度科技有限公司 | Map construction method and device |
| CN113220818B (en) * | 2021-05-27 | 2023-04-07 | 南昌智能新能源汽车研究院 | Automatic mapping and high-precision positioning method for parking lot |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103886107A (en) * | 2014-04-14 | 2014-06-25 | 苏州市华天雄信息科技有限公司 | Robot locating and map building system based on ceiling image information |
| CN103900583A (en) * | 2012-12-25 | 2014-07-02 | 联想(北京)有限公司 | Device and method used for real-time positioning and map building |
| CN104574421A (en) * | 2015-01-29 | 2015-04-29 | 北方工业大学 | Large-breadth small-overlapping-area high-precision multispectral image registration method and device |
| CN105825520A (en) * | 2015-01-08 | 2016-08-03 | 北京雷动云合智能技术有限公司 | Monocular SLAM (Simultaneous Localization and Mapping) method capable of creating large-scale map |
| CN106355197A (en) * | 2016-08-24 | 2017-01-25 | 广东宝乐机器人股份有限公司 | Navigation image matching filtering method based on K-means clustering algorithm |
| WO2018089703A1 (en) * | 2016-11-09 | 2018-05-17 | The Texas A&M University System | Method and system for accurate long term simultaneous localization and mapping with absolute orientation sensing |
-
2018
- 2018-06-22 CN CN201810650981.2A patent/CN108983769B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103900583A (en) * | 2012-12-25 | 2014-07-02 | 联想(北京)有限公司 | Device and method used for real-time positioning and map building |
| CN103886107A (en) * | 2014-04-14 | 2014-06-25 | 苏州市华天雄信息科技有限公司 | Robot locating and map building system based on ceiling image information |
| CN105825520A (en) * | 2015-01-08 | 2016-08-03 | 北京雷动云合智能技术有限公司 | Monocular SLAM (Simultaneous Localization and Mapping) method capable of creating large-scale map |
| CN104574421A (en) * | 2015-01-29 | 2015-04-29 | 北方工业大学 | Large-breadth small-overlapping-area high-precision multispectral image registration method and device |
| CN106355197A (en) * | 2016-08-24 | 2017-01-25 | 广东宝乐机器人股份有限公司 | Navigation image matching filtering method based on K-means clustering algorithm |
| WO2018089703A1 (en) * | 2016-11-09 | 2018-05-17 | The Texas A&M University System | Method and system for accurate long term simultaneous localization and mapping with absolute orientation sensing |
Non-Patent Citations (1)
| Title |
|---|
| 室内机器人的同步定位与建图方法的研究;刘圣龙;《中国优秀硕士学位论文全文数据库 信息科技辑》;20180315(第03期);第34-38页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108983769A (en) | 2018-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021233029A1 (en) | Simultaneous localization and mapping method, device, system and storage medium | |
| CN114862949B (en) | Structured scene visual SLAM method based on dot-line surface characteristics | |
| CN111462200B (en) | Cross-video pedestrian positioning and tracking method, system and equipment | |
| CN110097553B (en) | Semantic mapping system based on instant positioning mapping and three-dimensional semantic segmentation | |
| CN108983769B (en) | Instant positioning and map construction optimization method and device | |
| CN110807809B (en) | Light-weight monocular vision positioning method based on point-line characteristics and depth filter | |
| Saeedi et al. | Vision-based 3-D trajectory tracking for unknown environments | |
| CN108090958B (en) | Robot synchronous positioning and map building method and system | |
| CN108564616B (en) | Fast robust RGB-D indoor three-dimensional scene reconstruction method | |
| CN103136525B (en) | High-precision positioning method for special-shaped extended target by utilizing generalized Hough transformation | |
| US10366501B2 (en) | Method and apparatus for performing background image registration | |
| CN109815847B (en) | Visual SLAM method based on semantic constraint | |
| CN105096317A (en) | Fully automatic calibration method for high performance camera under complicated background | |
| CN108537844B (en) | Visual SLAM loop detection method fusing geometric information | |
| CN104036524A (en) | Fast target tracking method with improved SIFT algorithm | |
| CN112083403B (en) | Positioning tracking error correction method and system for virtual scene | |
| CN108257155B (en) | Extended target stable tracking point extraction method based on local and global coupling | |
| Cvišić et al. | Recalibrating the KITTI dataset camera setup for improved odometry accuracy | |
| CN112950696A (en) | Navigation map generation method and generation device and electronic equipment | |
| Fiala et al. | Robot navigation using panoramic tracking | |
| CN116879870B (en) | Dynamic obstacle removing method suitable for low-wire-harness 3D laser radar | |
| Long et al. | A real-time dense stereo matching method for critical environment sensing in autonomous driving | |
| CN112598709B (en) | Pedestrian movement speed intelligent sensing method based on video stream | |
| JP6922348B2 (en) | Information processing equipment, methods, and programs | |
| Ou et al. | Targetless Lidar-camera Calibration via Cross-modality Structure Consistency |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |