[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2020024845A1 - 定位方法和装置 - Google Patents

定位方法和装置 Download PDF

Info

Publication number
WO2020024845A1
WO2020024845A1 PCT/CN2019/097248 CN2019097248W WO2020024845A1 WO 2020024845 A1 WO2020024845 A1 WO 2020024845A1 CN 2019097248 W CN2019097248 W CN 2019097248W WO 2020024845 A1 WO2020024845 A1 WO 2020024845A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
coordinate system
coordinates
world coordinate
image
Prior art date
Application number
PCT/CN2019/097248
Other languages
English (en)
French (fr)
Inventor
李月
许景涛
刘童
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to US16/635,036 priority Critical patent/US11100670B2/en
Publication of WO2020024845A1 publication Critical patent/WO2020024845A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Definitions

  • the present disclosure relates to the field of positioning technology, and in particular, to a positioning method, apparatus, and device, and a computer-readable storage medium.
  • the methods used for indoor positioning mainly include vision-based positioning methods, and vision-based positioning methods mainly include SLAM (simultaneous localization and mapping) positioning methods and deep learning-based positioning methods.
  • vision-based positioning methods mainly include SLAM (simultaneous localization and mapping) positioning methods and deep learning-based positioning methods.
  • the vision odometer-based positioning method is a SLAM-based positioning method.
  • the initial coordinates of the camera in the world coordinate system need to be determined in advance, and then positioning is performed by frame matching based on the initial coordinates and a continuous image sequence.
  • the visual odometer-based positioning method also needs to eliminate the cumulative error every interval of a preset time.
  • the scene-based end-to-end positioning method is a deep learning-based positioning method.
  • the scene-based positioning method directly obtains the three-dimensional pose of the camera in the world coordinate system based on the scene of the image captured by the camera.
  • a positioning method including:
  • real-time coordinates of the camera in the world coordinate system are determined through inter-frame matching.
  • the method further includes:
  • obtaining the initial coordinates of the camera in the world coordinate system based on the first coordinates includes:
  • the first coordinate is used as an initial coordinate of the camera in a world coordinate system.
  • the method further includes:
  • obtaining the initial coordinates of the camera in the world coordinate system based on the first coordinates includes:
  • the second coordinate is used as an initial coordinate of the camera in a world coordinate system.
  • the obtaining the second coordinate of the camera in the world coordinate system based on the marker and the first coordinate includes:
  • a second coordinate of the camera in a world coordinate system is determined based on the rotation matrix and the translation matrix.
  • the detecting whether a marker is included in the image includes:
  • Whether a marker is included in the image is detected based on a single-shot multi-frame detector algorithm.
  • the method further includes:
  • the real-time coordinates of the camera in the world coordinate system are determined through frame matching.
  • the updating the initial coordinates of the camera in the world coordinate system includes:
  • the initial coordinates of the camera in the world coordinate system are obtained.
  • the deep learning model includes a deep learning model based on a PoseNet network.
  • a positioning device including:
  • An image acquirer configured to acquire an image captured by a camera
  • An initial coordinate determiner coupled to the image acquirer and configured to obtain a first coordinate of the camera in a world coordinate system from the image based on a deep learning model, and obtain the first coordinate based on the first coordinate The camera's initial coordinates in the world coordinate system;
  • the real-time position determiner is coupled with the image acquirer and the initial coordinate determiner, and is configured to determine the camera in the world coordinate system through frame matching based on the initial coordinates of the camera in the world coordinate system. Real-time coordinates.
  • the initial coordinate determiner is further configured to detect whether a landmark is included in the image, wherein the landmark has at least 3 vertices;
  • the first coordinate is used as an initial coordinate of the camera in a world coordinate system.
  • the initial coordinate determiner is further configured to: detect whether the image includes a marker, wherein the marker has at least 3 vertices;
  • the second coordinate is used as an initial coordinate of the camera in a world coordinate system.
  • the initial coordinate determiner is further configured to:
  • a second coordinate of the camera in a world coordinate system is determined based on the rotation matrix and the translation matrix.
  • the initial coordinate determiner is further configured to detect whether a marker is included in the image based on a single-shot multi-frame detector algorithm.
  • the initial coordinate determiner is further configured to:
  • the real-time coordinates of the camera in the world coordinate system are determined through inter-frame matching.
  • the initial coordinate determiner is further configured to:
  • An initial coordinate of the camera in a world coordinate system is obtained based on at least one of a scene and a landmark included in an image acquired by the image acquirer in real time.
  • the deep learning model includes a deep learning model based on a PoseNet network.
  • a positioning device including:
  • One or more processors are One or more processors.
  • a memory coupled to the one or more processors and storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors Perform the method.
  • a computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by one or more processors, cause the one or more Processors execute the method.
  • FIG. 1 shows a schematic diagram of a positioning device according to an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram showing a flow of a positioning method according to an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram showing a flow of a positioning method according to an embodiment of the present disclosure.
  • FIG. 4 shows a schematic diagram of a positioning device according to an embodiment of the present disclosure.
  • Visual odometer-based positioning methods are virtually unaffected by environmental changes.
  • the positioning method based on the visual odometer needs to obtain the initial coordinates of the camera in the world coordinate system, and eliminate the cumulative error every interval.
  • the acquisition of the camera's initial coordinates in the world coordinate system and the elimination of accumulated errors require the use of a hardware device such as a WIFI module or Bluetooth, which adds additional hardware costs.
  • the scene-based end-to-end positioning method is robust to environmental changes, but its positioning accuracy is relatively low.
  • One of the objectives of the embodiments of the present disclosure is to provide a positioning method that satisfies the required positioning accuracy.
  • FIG. 1 shows a schematic diagram of a positioning device 110 according to an embodiment of the present disclosure.
  • the device 110 and the camera 120 may be coupled to each other. They communicate via wireless technologies such as WIFI, Bluetooth, and so on.
  • wireless technologies such as WIFI, Bluetooth, and so on.
  • the positioning device 110 and the camera 120 are shown as separate devices in FIG. 1, according to an embodiment of the present disclosure, the positioning device 110 and the camera 120 may be integrated together.
  • the camera 120 is configured to capture an image and send the captured image to the positioning device 110.
  • the camera 120 may capture and send images in real time or at preset time intervals.
  • the camera 120 may be any device capable of capturing an image. In a practical application, for example, the camera 120 may be fixed on a shopping cart and move with the shopping cart to capture multiple images in a real scene.
  • the location of the camera 120 is essentially the location of the shopping cart or the user.
  • the positioning device 110 is coupled to the camera 120 and is configured to acquire or receive images from the camera 120.
  • the positioning device 110 may be any positioning device such as a mobile phone, a computer, a server, and a netbook.
  • the positioning device 110 may include an image acquirer 101, an existence determiner 102, an initial coordinate determiner 103, and a real-time position determiner 104.
  • the image acquirer 101 may be configured to acquire or receive an image in real time from the camera 120, for example, and send the acquired or received image to the presence determiner 102 and / or the initial coordinate determiner 103 and / or the real-time position determiner 104.
  • the presence determiner 102 may be coupled with the image acquirer 101, and may be configured to determine, for example, whether the initial coordinates of the camera 120 in the world coordinate system are included in the memory based on the image captured by the camera 120, and send the determined result Gives the initial coordinate determiner 103.
  • the initial coordinate determiner 103 may be coupled with the image acquirer 101 and the presence determiner 102.
  • the initial coordinate determiner 103 may obtain or receive an image from the image acquirer 101 and obtain or receive a result of the determination from the existence determiner 102.
  • the initial coordinate determiner 103 may be configured to obtain the current coordinates of the camera 120 in the world coordinate system from the acquired current image based on the deep learning model in response to determining that the initial coordinates of the camera in the world coordinate system are not included in the memory, The initial coordinates of the camera 120 in the world coordinate system are obtained based on the current coordinates.
  • the current coordinates may be used as the initial coordinates of the camera 120 in the world coordinate system, and may also be used as the current position of the camera 120.
  • the deep learning model may include a deep learning model based on a PoseNet network.
  • the input of the PoseNet network can be RGB images, and the output can be the 3D pose of the camera.
  • a real scene can be continuously collected for learning as needed.
  • the initial coordinate determiner 103 may be configured to save the initial coordinates of the camera 120 in the world coordinate system to a memory and the initial coordinates Send to real-time position determiner 104.
  • the real-time position determiner 104 may be coupled with the image acquirer 101 and the initial coordinate determiner 103.
  • the real-time position determiner 104 may obtain or receive an image from the image acquirer 101 and obtain or receive the saved initial coordinates from the initial coordinate determiner 103.
  • the real-time position determiner 104 may determine the real-time coordinates of the camera in the world coordinate system through inter-frame matching based on the current initial coordinates of the camera in the world coordinate system obtained or received from the initial coordinate determiner 103.
  • the real-time position determiner 104 may be configured to continuously receive an image sequence from the image acquirer 101, and determine that the camera is under world coordinates through inter-frame matching based on the initial coordinates obtained or received from the initial coordinate determiner 103.
  • the initial coordinates of the camera and the coordinates obtained in real time may be fused by Kalman filtering, and the result of the fusion may be better than the initial coordinates and the coordinates obtained in real time.
  • the implementation of obtaining the coordinates of the camera in the world coordinate system from the image based on the deep learning model and determining the real-time coordinates of the camera in the world coordinate system through frame matching based on the initial coordinates of the camera in the world coordinate system may be implemented. It is known from the related art that it will not be described in detail here.
  • the positioning method according to an embodiment of the present disclosure is completely based on vision without requiring any additional hardware devices such as WiFi and Bluetooth, thereby saving hardware costs.
  • a portion according to an embodiment of the present disclosure is based on a visual odometer, and thus a relatively high positioning accuracy can be satisfied.
  • the initial coordinate determiner 103 may be further configured to detect whether the image includes a landmark, wherein the landmark has at least three vertices. In such a case, the initial coordinate determiner 103 may be further configured to, in response to the landmark not being included in the image, use the current coordinates of the camera obtained from the scene of the image in the world coordinate system as Initial coordinates of the camera in the world coordinate system; and in response to detecting that the image includes the landmark, obtaining the camera's world coordinates based on the landmark and the current coordinates of the camera obtained from the scene of the image in the world coordinate system Another coordinate in the system; and using the other coordinate as an initial coordinate of the camera in the world coordinate system.
  • obtaining another coordinate of the camera in the world coordinate system based on the landmark and the current coordinates of the camera obtained from the scene of the image in the world coordinate system may include: obtaining the vertex of the landmark on the camera based on the image Coordinates in the coordinate system; based on the current coordinates of the camera in the world coordinate system obtained from the scene of the image, the landmarks included in the image are determined from the coordinates of the vertices of multiple landmarks stored in the world coordinate system in advance The coordinates of the vertex of the object in the world coordinate system; based on the coordinates of the vertex of the marker included in the image in the camera coordinate system and the coordinates of the vertex of the marker included in the image in the world coordinate system The coordinates obtain a rotation matrix and a translation matrix of the camera coordinate system relative to the world coordinate system; and determine the coordinates of the camera in the world coordinate system based on the rotation matrix and the translation matrix.
  • the markers can be arranged around the mall at certain space intervals so that the camera can capture the markers from time to time during the journey.
  • the marker may be any shape having at least three vertices, such as a triangle, rectangle, trapezoid, diamond, pentagon, and so on. Because the markers are arranged in advance, the coordinates of each vertex of the marker in the world coordinate system can be determined in advance, and these coordinates are stored in the memory to determine the real-time position of the camera in the world coordinate system.
  • R is the rotation matrix of the camera relative to the world coordinate system
  • ⁇ is the yaw angle of the camera rotating around the Z axis of the world coordinate system
  • ⁇ is the pitch angle of the camera rotating around the Y axis of the world coordinate system
  • ⁇ is The roll angle of the camera around the X axis of the world coordinate system
  • t is the translation of the camera relative to the world coordinate system
  • ⁇ Z is the coordinate of the camera on the Z axis of the world coordinate system
  • ⁇ X is on the X axis of the world coordinate system
  • Y is the coordinate on the Y axis of the world coordinate system.
  • SSD Single Shot MultiBox Detector
  • the background is removed from the image, the detected marker is cropped, and line features are extracted to obtain each vertex of the marker, so that the pixel coordinates of each vertex of the marker can be obtained.
  • the corresponding depth data of each vertex can be directly obtained by the camera (such as a binocular camera or a depth camera) itself (internal parameters of the camera), and then the coordinates of each vertex of the detected landmark in the camera coordinates are determined according to the pinhole camera model.
  • the coordinates Pc (Xc i , Yc i , Zc i ), i ⁇ 3 and so on.
  • a coordinate set of each vertex of the marker in the camera coordinate system can be obtained (wherein the marker includes at least three vertices, the coordinate set includes at least 3 coordinates).
  • each of the vertices of the detected marker coordinates corresponding to the world coordinate system (P (X i, Y i, Z i), i ⁇ 3) previously stored in the memory, so they Is known.
  • the heights ⁇ Z, ⁇ elevation angles, and roll angles ⁇ of the cameras are also known. In this case, only the three parameters of the yaw angle ⁇ , ⁇ X and ⁇ Y in the rotation matrix R and the translation matrix T are unknown.
  • the coordinates of each vertex of the marker in the camera coordinate system and its corresponding coordinates in the world coordinate system have been obtained, and the number of vertices is at least three, so the bias can be obtained according to formula (1) Heading angles ⁇ , ⁇ X and ⁇ Y.
  • a rotation matrix R and a translation matrix T can be obtained, thereby obtaining the current coordinates of the camera in the world coordinate system.
  • the coordinates of a plurality of markers and their corresponding vertices are stored in the memory in advance. Therefore, when a marker is detected from the current image, it is necessary to find the detected marker from a plurality of markers stored in the memory, and then find the coordinates of its corresponding vertex in the world coordinate system.
  • the current coordinates (Xs, Ys, Zs) of the camera in the world coordinate system may be obtained from the current image based on the deep learning model (such as the PoseNet network) by scene-based positioning. Then based on the coordinates (Xs, Ys, Zs) of the camera in the world coordinate system, a marker closest to the coordinates is searched from the memory.
  • the distribution positions of the markers are relatively scattered, such as arranging one at several tens of meters, at the camera position where the marker is detected, the closest marker to the camera is unique (that is, no marker is detected and there are two equal Markers of distance), and then find the coordinates of each vertex of the marker in the world coordinate system stored in advance. Since there is a certain error based on scene positioning (assuming the maximum positioning error is d), the coordinate range of the camera obtained based on scene positioning may be (Xs ⁇ d, Ys ⁇ d, Zs ⁇ d).
  • positioning is performed based on the marker to determine the initial position of the camera in the world coordinate system. This can further improve the accuracy of positioning.
  • the initial coordinate determiner 103 may be further configured to update the initial coordinates of the camera 120 in the world coordinate system at preset time intervals.
  • the real-time position determiner 104 may be configured to determine the real-time coordinates of the camera in the world coordinate system through inter-frame matching based on the initial coordinates of the updated camera in the world coordinate system.
  • updating the initial coordinates of the camera in the world coordinate system may include the initial coordinate determiner 103 re-obtaining the camera based on at least one of a scene and a landmark included in the image obtained in real time from the image acquirer 101 The initial coordinates in the world coordinate system.
  • a timer may be set for the initial coordinates.
  • the timer (preset time interval) may be, for example, 30 seconds, or any other suitable time.
  • the initial coordinates of the stored camera in the world coordinate system are kept unchanged, and continue to be obtained from the image sequence through inter-frame matching based on the stored initial coordinates.
  • the real-time coordinates of the camera in the world coordinate system When the initial coordinates have passed the time period set by the timer, the acquisition of the initial coordinates of the camera in the world coordinate system according to the embodiment of the present disclosure may be triggered again.
  • acquisition of a current image, detection of a marker in a captured current image may be triggered, and then a camera is determined based on a scene of the current image or a marker of the current image as described above.
  • the current coordinates in the world coordinate system, and the current coordinates of the previously stored camera in the world coordinate system are updated or replaced with the current coordinates of the determined camera in the world coordinate system, so that the determined camera is in the world coordinate system.
  • the current coordinates are used as the new initial coordinates of the camera in the world coordinate system, and then the real-time coordinates of the camera in the world coordinate system are determined through frame matching based on the updated initial coordinates of the camera in the world coordinate system. While using the determined current coordinates of the camera in the world coordinate system as the new initial coordinates of the camera in the world coordinate system, a timer is set again, and so on.
  • the stored camera's initial coordinates in the world coordinate system are updated at preset time intervals, the accumulated positioning error can be eliminated, thereby further improving the positioning accuracy.
  • FIG. 2 is a schematic diagram illustrating a flow of a positioning method 200 according to an embodiment of the present disclosure.
  • the method 200 may be performed, for example, by a positioning device 110 as shown in FIG. 1.
  • an image captured by the camera may be acquired, for example, a current image captured by the camera 120 is received from the camera 120.
  • a first coordinate (ie, current coordinate) of the camera in the world coordinate system is obtained from the image (current image) based on the deep learning model.
  • the deep learning model includes a deep learning model based on a PoseNet network.
  • the initial coordinates of the camera in the world coordinate system are obtained based on the first coordinates (current coordinates) of the camera in the world coordinate system.
  • the first coordinate (current coordinate) of the camera in the world coordinate system can be used as the initial coordinate of the camera in the world coordinate system.
  • step 204 the current (real-time) coordinates of the camera in the world coordinate system are determined through inter-frame matching based on the initial coordinates of the camera in the world coordinate system. Since then, the image has been repeatedly received, and the current (real-time) coordinates of the camera in the world coordinate system are determined by inter-frame matching based on the stored camera's initial coordinates in the world coordinate system.
  • the coordinates of the camera in the world coordinate system obtained through inter-frame matching can be obtained from the visual odometer-based positioning method, which will not be described in detail here.
  • the method 200 in FIG. 2 may further include a step of determining whether the initial coordinates of the camera in the world coordinate system are included in the memory. In response to determining that the initial coordinates of the camera in the world coordinate system are already included in the memory, proceed directly to step 203.
  • FIG. 3 illustrates a flowchart of a positioning method 300 according to an embodiment of the present disclosure.
  • the method 300 may be performed, for example, by a positioning device 110 as shown in FIG. 1.
  • an image captured by the camera may be acquired, for example, a current image captured by the camera 120 is received from the camera 120.
  • a first coordinate (ie, current coordinate) of the camera in the world coordinate system is obtained from the image (current image) based on the deep learning model.
  • the deep learning model includes a deep learning model based on a PoseNet network.
  • step 303 it can be detected whether the acquired image includes a marker.
  • the marker has at least 3 vertices.
  • step 304 is performed in response to detecting that the image includes the marker.
  • the second coordinate of the camera in the world coordinate system is obtained based on the marker and the first coordinate (current coordinate) of the camera in the world coordinate system determined in step 302, and the camera is in the world coordinate system.
  • the initial coordinates In one embodiment, coordinates of a vertex of the landmark in a camera coordinate system are obtained based on the image. The coordinates of the vertices of the landmarks included in the image in the world coordinate system are determined based on the coordinates of the vertices of the plurality of landmarks stored in advance in the world coordinate system.
  • a second coordinate of the camera in a world coordinate system is determined based on the rotation matrix and the translation matrix.
  • step 305 in response to detecting that the image does not include the marker, step 305 is performed.
  • the first coordinate (current coordinate) of the camera in the world coordinate system obtained from the acquired image is directly used as the initial coordinate of the camera in the world coordinate system.
  • step 306 based on the initial coordinates of the camera in the world coordinate system, real-time coordinates of the camera in the world coordinate system are determined through inter-frame matching.
  • the method 300 may further include, in response to determining that the initial coordinates of the camera in the world coordinate system are included in the memory, determining whether the initial coordinates have passed a preset period of time. Further, in response to determining that the initial coordinates have not experienced the preset time period, the real-time coordinates of the camera are determined from the camera based on the current initial coordinates through inter-frame matching. In response to determining that the initial coordinates have passed the preset time period, step 301 is performed.
  • the initial coordinates of the camera in the world coordinate system are updated based on the scene of the image or the landmarks of the image by determining whether a landmark is included in the image, so that based on the updated initial coordinates of the camera in the world coordinate system, Inter-frame matching determines the real-time coordinates of the camera in the world coordinate system.
  • the initial coordinates of the camera in the world coordinate system are continuously updated at preset time intervals, and the real-time coordinates of the camera in the world coordinate system are determined through frame matching based on the updated initial coordinates of the camera in the world coordinate system. Updating the initial coordinates of the camera in the world coordinate system may include executing method 300 once.
  • FIG. 4 shows a schematic diagram of a positioning device 400 according to an embodiment of the present disclosure.
  • the positioning device 400 may be used to implement the method of FIG. 2 or FIG. 3.
  • the positioning device 400 is only one example of a suitable positioning device and is not intended to suggest any limitation as to the scope of use or functionality of the presently disclosed subject matter.
  • the components of the positioning device 400 may include, but are not limited to, a processor 401, a memory 402, and a bus 403 that couples various system components including the memory to the processor 401.
  • the bus 403 may be any one of several types of bus structures including a memory bus or a memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
  • bus architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and also known as mezzanine bus Peripheral Component Interconnect (PCI) bus.
  • the positioning device 400 may include a variety of computer-readable media.
  • Computer-readable media can be any available media that can be accessed by the positioning device 400 and includes both volatile and nonvolatile media, removable and non-removable media.
  • Computer-readable media may include computer-readable storage media and communication media.
  • Computer-readable storage media includes volatile and nonvolatile and removable and non-volatile implementations using any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data Removable media.
  • Computer-readable storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, cartridges Tape, magnetic tape, magnetic disk storage, or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by the positioning device 400.
  • Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and includes any information delivery media.
  • modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner that information is encoded in the signal.
  • communication media includes: wired media such as a wired network or direct-wired connection; and wireless media such as RF and other wireless media. Combinations of any of the above are also included within the scope of computer-readable media.
  • the memory 402 may include computer storage media in the form of volatile and / or non-volatile memory such as ROM and RAM.
  • a basic input / output system (BIOS) containing basic routines such as helping to transfer information between elements within the positioning device 400 during startup is typically stored in ROM.
  • the RAM typically contains data and / or program modules that are immediately accessible to the processor 401 and / or are currently being manipulated by the processor 401.
  • the data that can be stored in the memory 402 shown in FIG. 4 may include a BIOS, an operating system, an application program, other program modules, and program data.
  • the positioning device 400 may also include other removable / non-removable, volatile / nonvolatile computer storage media.
  • removable / non-removable, volatile / non-volatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, cartridges, flash cards, DVDs, digital video tapes, solid state RAM, Solid-state ROM, etc.
  • the computer storage medium discussed above provides the positioning device 400 with storage for computer-executable instructions, data structures, program modules, and other data.
  • the user can input commands and information into the pointing device 400 through input devices such as a keyboard and a pointing device commonly referred to as a mouse, trackball, or touchpad.
  • Other input devices may include a microphone, a joystick, a game pad, a satellite dish, a scanner, and the like.
  • I / O user input-output
  • a monitor or other type of display device may be connected to the bus 403 via a user input-output (I / O) interface 14 such as a video interface.
  • the positioning device 400 may also be connected to other peripheral output devices such as speakers and printers through a user input-output (I / O) interface 14.
  • the positioning device 400 may be connected to one or more remote computers via the network interface 15.
  • the remote computer may be a personal computer, server, router, network PC, peer-to-peer device, or other common network node, and typically includes many or all of the elements described above with respect to the positioning device 400.
  • An embodiment of the present disclosure also provides a computer-readable storage medium on which computer-executable instructions are stored.
  • the computer instructions are executed on one or more processors, the one or more processors execute the instructions according to the present invention.
  • the methods and functions of the embodiments are disclosed.
  • the computer-readable medium may include any one of the computer-readable media described above.
  • An embodiment of the present disclosure also provides a computer program product.
  • the instructions in the computer program product are executed by a processor, the method according to the embodiment of the present disclosure can be implemented.
  • first and second are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as “first” and “second” may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality” is at least two, for example, two, three, etc., unless it is specifically and specifically defined otherwise.
  • Any process or method description in a flowchart or otherwise described herein can be understood as representing a module, fragment, or portion of code that includes one or more executable instructions for implementing steps of a custom logic function or process
  • the scope of the embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed (including in a substantially simultaneous manner or in the reverse order according to the functions involved), which should be performed by It is understood by those skilled in the art to which the embodiments of the present disclosure belong.
  • Logic and / or steps represented in a flowchart or otherwise described herein, for example, a sequenced list of executable instructions that may be considered to implement a logical function, may be embodied in any computer-readable medium, For use by, or in combination with, an instruction execution system, device, or device (such as a computer-based system, a system that includes a processor, or another system that can fetch and execute instructions from an instruction execution system, device, or device) Or equipment.
  • a "computer-readable medium” may be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.
  • Computer-readable media may include, for example, the following: electrical connections (electronic devices) with one or more wirings, portable computer disk cartridges (magnetic devices), random access memories (Random Access Memory), Read-only memory (Read Only Memory), erasable and editable read-only memory (Erasable, Programmable, Read Only Memory) or flash memory, fiber optic devices, and portable optical disc read-only memory (Compact Disc Read Only Memory).
  • the computer-readable medium may even be paper or other suitable medium on which the program can be printed, because, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or other suitable Processing to obtain the program electronically and then store it in computer memory.
  • portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.
  • multiple steps or methods may be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if it is implemented in hardware, it may be implemented in any one of the following technologies or a combination thereof known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a suitable combination Application-specific integrated circuits for logic gate circuits, Programmable Gate Arrays, Field Programmable Gate Arrays, etc.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing module, or each unit may exist separately physically, or two or more units may be integrated into one module.
  • the above integrated modules may be implemented in the form of hardware or software functional modules. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

本公开实施例提供了一种定位方法和装置。该方法可以包括：获取相机所捕获的图像；基于深度学习模型从所述图像获得相机在世界坐标系下的第一坐标，基于所述第一坐标获得相机在世界坐标系下的初始坐标；以及基于相机在世界坐标系下的初始坐标通过帧间匹配确定相机在世界坐标系下的实时坐标。

Description

定位方法和装置
相关申请的交叉引用
本申请要求2018年8月1日递交的中国申请CN201810862721.1的优先权，其全部公开内容通过引用合并于此。
技术领域
本公开涉及定位技术领域，尤其涉及一种定位方法、装置和设备以及计算机可读存储介质。
背景技术
目前，用于室内定位的方法主要包括基于视觉的定位方法，而基于视觉的定位方法又主要包括基于SLAM(simultaneous localization and mapping)的定位方法以及基于深度学习的定位方法。
基于视觉里程计的定位方法是一种基于SLAM的定位方法。在基于视觉里程计的定位方法中，需要预先确定相机在世界坐标系下的初始坐标，然后再基于该初始坐标以及连续的图像序列通过帧间匹配进行定位。此外，该基于视觉里程计的定位方法还需要每间隔一段预设时间就进行累计误差的消除。
基于场景的端到端定位方法是一种基于深度学习的定位方法。该基于场景的定位方法基于相机所拍摄的图像的场景直接得到相机在世界坐标系下的三维相姿。
发明内容
根据本公开的一个方面，提供了一种定位方法，包括：
获取相机所捕获的图像；
基于深度学习模型从所述图像获得所述相机在世界坐标系下的第一坐标，
基于所述第一坐标获得所述相机在世界坐标系下的初始坐标；以及
基于所述相机在世界坐标系下的初始坐标，通过帧间匹配确定所述相机在世界坐标系下 的实时坐标。
在实施例中，所述方法还包括：
检测所述图像中是否包括标志物，其中所述标志物具有至少3个顶点，
其中，所述基于所述第一坐标获得所述相机在世界坐标系下的初始坐标包括：
响应于所述图像中不包括所述标志物，将所述第一坐标用作所述相机在世界坐标系下的初始坐标。
在实施例中，所述方法还包括：
检测所述图像是否包括标志物，其中所述标志物具有至少3个顶点；
其中，所述基于所述第一坐标获得所述相机在世界坐标系下的初始坐标包括：
响应于所述图像中包括所述标志物，基于所述标志物和所述第一坐标获得所述相机在世界坐标系下的第二坐标；以及
将所述第二坐标用作所述相机在世界坐标系下的初始坐标。
在实施例中，所述基于所述标志物和所述第一坐标获得相机在世界坐标系下的第二坐标包括：
基于所述图像获得所述标志物的顶点在相机坐标系下的坐标；
基于所述第一坐标从预先存储的多个标志物的顶点在世界坐标系下的坐标，确定所述图像中包括的所述标志物的顶点在世界坐标系下的坐标；
基于所述图像中包括的所述标志物的顶点在相机坐标系下的坐标以及所述图像中包括的所述标志物的顶点在世界坐标系下的坐标，获得相机坐标系相对于世界坐标系的旋转矩阵和平移矩阵；以及
基于所述旋转矩阵和所述平移矩阵确定所述相机在世界坐标系下的第二坐标。
在实施例中，所述检测所述图像中是否包括标志物包括：
基于单次多框检测器算法检测所述图像中是否包括标志物。
在实施例中，所述方法还包括，
以预设的时间间隔更新所述相机在世界坐标系下的初始坐标；以及
基于更新的相机在世界坐标系下的初始坐标，通过帧间匹配确定相机在世界坐标系下的 实时坐标。
在实施例中，所述更新所述相机在世界坐标系下的初始坐标包括：
基于实时获取的图像中包括的场景和标志物中的至少一个，获得相机在世界坐标系下的初始坐标。
在实施例中，所述深度学习模型包括基于PoseNet网络的深度学习模型。
本公开的另一方面，提供了一种定位装置，包括：
图像获取器，被配置成获取相机所捕获的图像；
初始坐标确定器，与所述图像获取器相耦接且被配置成：基于深度学习模型从所述图像获得所述相机在世界坐标系下的第一坐标，基于所述第一坐标获得所述相机在世界坐标系下的初始坐标；以及
实时位置确定器，与图像获取器和所述初始坐标确定器相耦接，且被配置成：基于所述相机在世界坐标系下的初始坐标，通过帧间匹配确定所述相机在世界坐标系下的实时坐标。
在实施例中，所述初始坐标确定器还被配置成：检测所述图像中是否包括标志物，其中所述标志物具有至少3个顶点；以及
响应于所述图像中不包括所述标志物，将所述第一坐标用作所述相机在世界坐标系下的初始坐标。
在实施例中，所述初始坐标确定器还被配置成：检测所述图像是否包括标志物，其中所述标志物具有至少3个顶点；
响应于所述图像中包括所述标志物，基于所述标志物和所述第一坐标获得所述相机在世界坐标系下的第二坐标，以及
将所述第二坐标用作所述相机在世界坐标系下的初始坐标。
在实施例中，所述初始坐标确定器还被配置成：
基于所述图像获得所述标志物的顶点在相机坐标系下的坐标；
基于所述第一坐标从预先存储的多个标志物的顶点在世界坐标系下的坐标，确定所述图像中包括的所述标志物的顶点在世界坐标系下的坐标；
基于所述图像中包括的所述标志物的顶点在相机坐标系下的坐标以及所述图像中包括的 所述标志物的顶点在世界坐标系下的坐标，获得相机坐标系相对于世界坐标系的旋转矩阵和平移矩阵；以及
基于所述旋转矩阵和所述平移矩阵确定所述相机在世界坐标系下的第二坐标。
在实施例中，所述初始坐标确定器还被配置成：基于单次多框检测器算法检测所述图像中是否包括标志物。
在实施例中，所述初始坐标确定器还被配置成：
以预设的时间间隔更新所述相机在世界坐标系下的初始坐标；以及
基于更新的相机在世界坐标系下的初始坐标，通过帧间匹配确定相机在世界坐标系下的实时坐标。
在实施例中，所述初始坐标确定器还被配置成：
基于由图像获取器实时获取的图像中包括的场景和标志物中的至少一个，获得所述相机在世界坐标系下的初始坐标。
在实施例中，所述深度学习模型包括基于PoseNet网络的深度学习模型。
根据本公开的再一方面，提供了一种定位设备，包括：
一个或多个处理器；以及
存储器，与所述一个或多个处理器相耦接，并存储有计算机可执行指令，所述计算机可执行指令在由所述一个或多个处理器执行时使得所述一个或多个处理器执行所述方法。
根据本公开的再一方面，提供了一种计算机可读存储介质，其上存储了计算机可执行指令，所述计算机可执行指令在由一个或多个处理器执行时，使所述一个或多个处理器执行所述方法。
附图说明
为了更清楚地说明本公开实施例，下面通过参考附图详细描述本公开的实施例。本公开实施例旨在用于解释本公开，而不应理解为对本公开的限制。在附图中，
图1示出了根据本公开的实施例的定位装置的示意图；
图2示出了根据本公开的实施例的定位方法的流程的示意图；
图3示出了根据本公开的实施例的的定位方法的流程的示意图；以及
图4示出了根据本公开的实施例的定位设备的示意图。
贯穿附图，相同或类似的标号表示相同或类似的元件。
具体实施方式
基于视觉里程计的定位方法几乎不受环境变化的影响。但是，基于视觉里程计的定位方法需要获取相机在世界坐标系下的初始坐标，并且每间隔一段时间就进行累积误差的消除。然而，相机在世界坐标系下的初始坐标的获取以及累积误差的消除都需要借助于WIFI模块或者蓝牙之类的硬件设备，这增加了额外的硬件成本。基于场景的端到端定位方法对环境变化具有一定的鲁棒性，但是它的定位精度相对较低。
本公开的实施例的目的之一在于提供一种满足需要的定位精度的定位方法。
以下将结合附图对根据本公开的实施例进行说明。需要注意的是，尽管以下描述基于环境变化较大的商场进行说明，但是根据本公开的实施例可以应用于环境变化较大的任何其他现实应用情形。
图1示出了根据本公开的实施例的定位装置110的示意图。该装置110可以与相机120彼此耦接。它们通过诸如WIFI、蓝牙等无线技术进行通信。尽管在图1中，定位装置110和相机120被示为分离的设备，但是根据本公开的实施例不限于此，定位装置110和相机120可以被集成在一起。
相机120被配置成捕获图像并将所捕获的图像发送给定位装置110。相机120可以实时地，或者以预设的时间间隔捕获和发送图像。相机120可以是任何一种能够捕获到图像的设备。在现实的应用中，例如相机120可以被固定在购物车上，并随着购物车移动从而捕获到现实场景中的多个图像。相机120的位置实质上就是购物车或者用户的位置。
定位装置110与相机120相耦接并且被配置成从相机120获取或接收图像。定位装置110可以是移动电话、计算机、服务器、上网本等任何定位设备。
定位装置110可以包括图像获取器101、存在性确定器102、初始坐标确定器103以及实时位置确定器104。
图像获取器101可以被配置成例如从相机120实时地获取或接收图像，并将所获取或所 接收的图像发送至存在性确定器102和/或初始坐标确定器103和/或实时位置确定器104。
存在性确定器102可以与图像获取器101相耦接，并且可以被配置成基于相机120所捕获的图像确定例如存储器中是否包括相机120在世界坐标系下的初始坐标，并将确定的结果发送给初始坐标确定器103。
初始坐标确定器103可以与图像获取器101和存在性确定器102相耦接。初始坐标确定器103可以从图像获取器101获得或接收图像，以及从存在性确定器102获得或接收所述确定的结果。初始坐标确定器103可以被配置成，例如响应于确定存储器中不包括相机在世界坐标系下的初始坐标，基于深度学习模型从所获取的当前图像获得相机120在世界坐标系下的当前坐标，并基于所述当前坐标获得相机120在世界坐标系下的初始坐标。在一个实施例中，所述当前坐标就可以用作相机120在世界坐标系下的初始坐标，也可以用作相机120的当前位置。所述深度学习模型可以包括基于PoseNet网络的深度学习模型。PoseNet网络的输入可以是RGB图像，输出可以是相机的三维位姿。为了提高所述深度学习模型的准确度，可以根据需要不断地采集现实场景进行学习。
在初始坐标确定器103确定了相机120在世界坐标系下的初始坐标之后，初始坐标确定器103可以被配置成保存相机120在世界坐标系下的该初始坐标保存到存储器，以及将该初始坐标发送给实时位置确定器104。
实时位置确定器104可以与图像获取器101和初始坐标确定器103相耦接。实时位置确定器104可以从图像获取器101获得或接收图像，以及从初始坐标确定器103获得或接收所保存的初始坐标。在一个实施例中，实时位置确定器104可以基于从初始坐标确定器103获得或接收的相机在世界坐标系下的当前初始坐标通过帧间匹配确定相机在世界坐标系下的实时坐标。具体地，实时位置确定器104可以被配置成持续地从图像获取器101接收图像序列，并且基于从初始坐标确定器103获得或接收的所述初始坐标通过帧间匹配来确定相机在世界坐标下的实时坐标。在此过程中，可以将相机的所述初始坐标和实时获得的坐标进行卡尔曼滤波融合，融合后的坐标结果可优于所述初始坐标和实时获得的坐标。需要注意的是，基于深度学习模型从图像获得相机在世界坐标系下的坐标以及基于相机在世界坐标系下的该初始坐标通过帧间匹配确定相机在世界坐标系下的实时坐标的实施方式可从相关技术中获 知，在此不再详述。
根据本公开的实施例的定位方法完全基于视觉而不需要诸如WiFi和蓝牙之类的任何额外的硬件设备，从而节约了硬件成本。此外，根据本公开的实施例的部分是基于视觉里程计的，因而可以满足相对高的定位精度。
在一个实施例中，所述初始坐标确定器103还可以被配置成：检测所述图像是否包括标志物，其中，所述标志物具有至少三个顶点。在这样的情况下，所述初始坐标确定器103还可以被配置为，响应于所述图像中不包括所述标志物，将从图像的场景获得的相机在世界坐标系下的当前坐标用作相机在世界坐标系下的初始坐标；以及响应于检测到所述图像包括所述标志物，基于所述标志物和从图像的场景获得的相机在世界坐标系下的当前坐标获得相机在世界坐标系下的另一坐标；以及将所述另一坐标用作相机在世界坐标系下的初始坐标。
在一个实施例中，基于所述标志物和从图像的场景获得的相机在世界坐标系下的当前坐标获得相机在世界坐标系下的另一坐标可以包括：基于图像获得标志物的顶点在相机坐标系下的坐标；基于从图像的场景获得的相机在世界坐标系下的当前坐标从预先存储的多个标志物的顶点在世界坐标系下的坐标确定出所述图像中包括的所述标志物的顶点在世界坐标系下的坐标；基于所述图像中包括的所述标志物的顶点在相机坐标系下的坐标以及所述图像中包括的所述标志物的顶点在世界坐标系下的坐标获得相机坐标系相对于世界坐标系的旋转矩阵和平移矩阵；以及基于所述旋转矩阵和平移矩阵确定相机在世界坐标系下的坐标。
在实践中，标志物可以以一定的空间间隔布置在商场的各处，以便相机在行进中能够不定时地拍摄到所述标志物。在一个实施例中，标志物可以是具有至少三个顶点的任何形状，例如三角形、矩形、梯形、菱形、五边形等等。因为标志物是预先布置的，所以可以预先确定标志物的各个顶点在世界坐标系下的坐标，并将这些坐标存储在存储器中以便确定相机在世界坐标系下的实时位置。
为了说明如何基于标志物获得相机在世界坐标系下的坐标，假设一个物体在像素平面坐标系下的坐标为(u，v)，在相机坐标系中的坐标为(Xc，Yc，Zc)、在世界坐标系中的坐标为(X，Y，Z)，则相机坐标系与世界坐标系的转换关系为：
Figure PCTCN2019097248-appb-000001
其中，R为旋转矩阵，R＝R z(α)R y(β)R x(γ)，
Figure PCTCN2019097248-appb-000002
Figure PCTCN2019097248-appb-000003
t为三维平移矩阵，t＝[ΔX，ΔY，ΔZ]；以及0＝(0，0，0)。具体地，R为相机相对于世界坐标系的旋转矩阵，α为相机绕着世界坐标系的Z轴旋转的偏航角，β为相机绕着世界坐标系的Y轴旋转的俯仰角，γ为相机绕着世界坐标系的X轴旋转的横滚角，t为相机相对于世界坐标系的平移，ΔZ是相机在世界坐标系的Z轴上的坐标，ΔX是在世界坐标系的X轴上的坐标，ΔY是在世界坐标系的Y轴上的坐标。
在根据本公开的一个实施例中，例如，可以基于单次多框检测器(SSD：Single Shot MultiBox Detector)深度学习算法来检测图像中是否包括标志物。这种算法的输入是RGB图像，而输出则是检测出的标志物的检测框。可以通过不断地模型训练而提高这种检测的精度。基于SSD的目标检测方法可以较快地检测到标志物。
在检测到图像中包括标志物时，从图像中去掉背景，裁剪出检测到的标志物，对其进行线特征提取，得到标志物的各个顶点，从而可以求出标志物的各个顶点的像素坐标(ui，vi)，i≥3。进一步地，可以通过相机(例如双目相机或者深度相机)本身(相机的内部参数)直接得到各个顶点的对应深度数据，进而根据针孔相机模型确定所检测到的标志物的各个顶点在相机坐标系下的坐标Pc＝(Xc i，Yc i，Zc i)，i≥3等等。由此，可以得到标志物的各个顶点在相机坐标系下的坐标集合(其中，由于标志物至少包括三个顶点，因此，该坐标集合至少包括3个坐标)。如前所述，所检测到的标志物的各个顶点在世界坐标系下的对应坐标 (P＝(X i，Y i，Z i)，i≥3)预先被存储在了存储器中，因此它们是已知的。此外，由于相机被固定于购物车上，所以相机的高度ΔZ、β俯仰角和横滚角γ也是已知的。在这种情况下，旋转矩阵R和平移矩阵T中仅仅偏航角α，ΔX和ΔY这三个参数是未知的。如前所述，已经获得了标志物的各个顶点在相机坐标系下的坐标以及其在世界坐标系下的对应坐标，并且顶点的数目至少为3个，所以可以根据公式(1)求出偏航角α，ΔX和ΔY。在这种情况下，可以得到旋转矩阵R和平移矩阵T，从而获得相机在世界坐标系下的当前坐标。
如前所述，在存储器中预先存储了多个标志物及其对应的顶点的坐标。因此，在从当前图像中检测出标志物时，需要从存储器中存储的多个标志物中找到所检测出的标志物，进而找出其对应的顶点在世界坐标系下的坐标。在一个实施例中，可以首先基于深度学习模型(例如PoseNet网络)通过基于场景定位而从当前的图像获得相机在世界坐标系下的当前坐标(Xs，Ys，Zs)。然后基于相机在世界坐标系下的该坐标(Xs，Ys，Zs)从存储器中搜索距离该坐标最近的标志物。由于标志物的分布位置比较分散，如几十米布置一个，在检测出标志物的相机位置处，距离该相机最近的标志物为唯一的(即不存在检测出标志物、并且有两个同等距离的标志物)，进而找出预先存储的该标志物的各个顶点在世界坐标系下的坐标。由于基于场景定位存在一定的误差(假设最大定位误差是d)，所以基于场景定位获得的相机的坐标范围可能是(Xs±d，Ys±d，Zs±d)。在这种情况下，基于相机的坐标范围(Xs±d，Ys±d，Zs±d)从存储器中搜索当前的图像中的标志物，进而找出预先存储的该标志物的各个顶点在世界坐标系下的坐标。
根据本公开的实施例，基于标志物进行定位进而确定相机在世界坐标系下的初始位置。这可以进一步提高定位的精度。
在一个实施例中，所述初始坐标确定器103还可以被配置成：以预设的时间间隔更新相机120在世界坐标系下的初始坐标。在这种情况下，实时位置确定器104可以被配置为，基于更新的相机在世界坐标系下的初始坐标，通过帧间匹配确定相机在世界坐标系下的实时坐标。根据本公开的实施例，更新所述相机在世界坐标系下的初始坐标可以包括初始坐标确定器103重新基于从图像获取器101实时获得的图像中包括的场景和标志物中的至少一个获得相机在世界坐标系下的初始坐标。具体地，在根据本公开的实施例中，在获得相机在世界坐 标系下的当前初始坐标的同时，可以为该初始坐标设置一个定时器。该定时器(预设的时间间隔)可以例如是30秒，或者任何其他适当的时间。当该初始坐标没有经历定时器所设定的时间段时，保持所存储的相机在世界坐标系下的该初始坐标不变，并且继续基于所存储的该初始坐标从图像序列通过帧间匹配获得相机在世界坐标系下的实时坐标。当该初始坐标经历了定时器所设定的时间段时，可以再一次触发根据本公开实施例的对于相机在世界坐标系下的初始坐标的获取。在一个实施例中，可以触发对于当前图像的获取、对于所捕获的当前图像中的标志物的检测，进而如前所述那样基于所述当前图像的场景或所述当前图像的标志物确定相机在世界坐标系下的当前坐标，并用所确定的相机在世界坐标系下的当前坐标更新或者替换先前存储的相机在世界坐标系下的初始坐标，以便将所确定的相机在世界坐标系下的当前坐标用作相机在世界坐标系下的新的初始坐标，进而继续基于所更新的相机在世界坐标系下的初始坐标通过帧间匹配确定相机在世界坐标系下的实时坐标。在将所确定的相机在世界坐标系下的当前坐标用作相机在世界坐标系下的新的初始坐标的同时，再一次设置定时器，以此类推。
根据本公开实施例，由于以预设的时间间隔更新所存储的相机在世界坐标系下的初始坐标，所以可以消除累积定位误差，从而进一步提高定位精度。
图2示出了根据本公开的实施例的定位方法200的流程的示意图。该方法200例如可以由如图1中所示的定位装置110执行。
如图2中所示，在步骤201中，可以获取相机所捕获的图像，例如，从相机120接收相机120所捕获的当前图像。
在步骤202中，基于深度学习模型从所述图像(当前图像)获得相机在世界坐标系下的第一坐标(即当前坐标)。在一个实施例中，所述深度学习模型包括基于PoseNet网络的深度学习模型。
在步骤203中，基于所述相机在世界坐标系下的第一坐标(当前坐标)获得相机在世界坐标系下的初始坐标。在一个实施例中，所述相机在世界坐标系下的第一坐标(当前坐标)就可以用作相机在世界坐标系下的初始坐标。
在步骤204中，基于相机在世界坐标系下的初始坐标通过帧间匹配确定相机在世界坐标 系下的当前(实时)坐标。自此以后，反复地接收图像，并基于所存储的相机在世界坐标系下的初始坐标通过帧间匹配确定相机在世界坐标系下的当前(实时)坐标。通过帧间匹配获得相机在世界坐标系下的坐标可以从基于视觉里程计定位方法中获知，在此不再详述。
在实施例中，在步骤202之前，图2中的方法200还可以包括确定在存储器中是否包括相机在世界坐标系下的初始坐标的步骤。响应于确定存储器中已经包括相机在世界坐标系下的初始坐标，直接前进到步骤203。
图3示出了根据本公开的实施例的定位方法300的流程的示意图。该方法300例如可以由如图1中所示的定位装置110执行。
如图3中所示，在步骤301中，可以获取相机所捕获的图像，例如，从相机120接收相机120所捕获的当前图像。
在步骤302中，基于深度学习模型从所述图像(当前图像)获得相机在世界坐标系下的第一坐标(即当前坐标)。在一个实施例中，所述深度学习模型包括基于PoseNet网络的深度学习模型。
在步骤303中，可以检测所获取的图像是否包括标志物。在一个实施例中，所述标志物具有至少3个顶点。在一个实施例中，响应于检测到所述图像包括所述标志物，执行步骤304。
在步骤304中，基于所述标志物以及在步骤302中确定的相机在世界坐标系下的第一坐标(当前坐标)获得相机在世界坐标系下的第二坐标，作为相机在世界坐标系下的初始坐标。在一个实施例中，基于所述图像获得所述标志物的顶点在相机坐标系下的坐标。基于所述第一坐标从预先存储的多个标志物的顶点在世界坐标系下的坐标，确定所述图像中包括的所述标志物的顶点在世界坐标系下的坐标。基于所述图像中包括的所述标志物的顶点在相机坐标系下的坐标以及所述图像中包括的所述标志物的顶点在世界坐标系下的坐标，获得相机坐标系相对于世界坐标系的旋转矩阵和平移矩阵。基于所述旋转矩阵和所述平移矩阵确定所述相机在世界坐标系下的第二坐标。
在一个实施例中，响应于检测到所述图像不包括所述标志物，执行步骤305。在步骤305中将从所获取的图像获得的相机在世界坐标系下的第一坐标(当前坐标)直接作为相机 在世界坐标系下的初始坐标。
在步骤306中，基于相机在世界坐标系下的初始坐标，通过帧间匹配确定所述相机在世界坐标系下的实时坐标。在实施例中，方法300还可以包括响应于确定存储器中包括相机在世界坐标系下的初始坐标，确定该初始坐标是否经历了预设的时间段。进一步地，响应于确定该初始坐标没有经历所述预设的时间段，基于当前的初始坐标通过帧间匹配从确定相机实时坐标。响应于确定该初始坐标经历了所述预设的时间段，执行步骤301。也就是，通过确定图像中是否包括标志物而基于所述图像的场景或所述图像的标志物更新相机在世界坐标系下的初始坐标，从而基于更新的相机在世界坐标系下的初始坐标通过帧间匹配确定相机在世界坐标系下的实时坐标。以预设的时间间隔不断地更新相机在世界坐标系下的初始坐标，并且基于更新的相机在世界坐标系下的初始坐标通过帧间匹配确定相机在世界坐标系下的实时坐标。更新所述相机在世界坐标系下的初始坐标可以包括执行一次方法300。
图4示出了根据本公开的实施例的定位设备400的示意图。
参考图4，定位设备400可以用于实施图2或图3的方法。定位设备400仅仅是合适的定位设备的一个示例，并且不旨在关于当前所公开的主题的用途或者功能性的范围建议任何限制。
定位设备400的部件可以包括但不限于处理器401、存储器402和将包括存储器的各种系统部件耦接到处理器401的总线403。总线403可以是使用各种各样总线架构中的任一种总线架构的包括存储器总线或者存储器控制器、外围总线和本地总线的几种类型的总线结构中的任一种总线结构。作为示例而非限制，这样的架构包括工业标准架构(ISA)总线、微通道架构(MCA)总线、增强型ISA(EISA)总线、视频电子标准协会(VESA)本地总线和也被称为夹层总线的外围部件互连(PCI)总线。
定位设备400可以包括各种各样的计算机可读介质。计算机可读介质可以是可以被定位设备400访问的任何可用介质，并且包括易失性和非易失性介质、可移除和非可移除介质两者。作为示例而非限制，计算机可读介质可以包括计算机可读存储介质和通信介质。计算机可读存储介质包括使用任何用于存储诸如是计算机可读指令、数据结构、程序模块或者其他数据之类的信息的方法或者技术实施的易失性和非易失性以及可移除和非可移除介质。计算 机可读存储介质包括但不限于：随机存取存储器(RAM)、只读存储器(ROM)、EEPROM、闪存或者其他存储器技术、CD-ROM、数字多功能盘(DVD)或者其他光盘存储器、盒式磁带、磁带、磁盘存储装置或者其他磁存储设备、或者任何其他可以用于存储期望的信息并且可以被定位设备400访问的介质。通信介质通常将计算机可读指令、数据结构、程序模块或者其他数据体现在诸如是载波或者其他传输机制之类的已调制数据信号中，并且包括任何信息递送介质。术语“已调制数据信号”表示使其特性中的一个或多个特性以使得将信息编码在该信号中的方式被设置或者改变的信号。作为示例而非限制，通信介质包括：诸如是有线网络或者直接有线连接之类的有线介质；以及诸如是RF和其他无线介质之类的无线介质。以上各项中的任何项的组合也被包括在计算机可读介质的范围内。
存储器402可以包括采用诸如ROM和RAM之类的易失性和/或非易失性存储器的形式的计算机存储介质。包含诸如在启动期间帮助在定位设备400内的元件之间传输信息的基本例程的基本输入/输出系统(BIOS)通常被存储在ROM中。RAM通常包含对于处理器401来说可立即访问和/或当前正被处理器401操作的数据和/或程序模块。作为示例而非限制，图4所示的存储器402中可以存储的数据可以包括BIOS、操作系统、应用程序、其他程序模块和程序数据。
定位设备400还可以包括其他可移除/非可移除、易失性/非易失性计算机存储介质。
可以在示例性操作环境中使用的其他可移除/非可移除、易失性/非易失性计算机存储介质包括但不限于盒式磁带、闪存卡、DVD、数字视频磁带、固态RAM、固态ROM等。
上面所讨论的计算机存储介质为定位设备400提供对计算机可执行指令、数据结构、程序模块和其他数据的存储。
用户可以通过诸如键盘和通常被称为鼠标、轨迹球或者触摸板的指向设备之类的输入设备向定位设备400输进命令和信息。其他输入设备可以包括麦克风、操纵杆、游戏板、碟形卫星天线、扫描仪等。这些和其他输入设备通常通过耦接到总线403的用户输入输出(I/O)接口14连接到处理器401。监视器或者其他类型的显示设备可以经由诸如是视频接口之类的用户输入输出(I/O)接口14连接到总线403。除监视器之外，定位设备400还可以通过用户输入输出(I/O)接口14连接到其他外围输出设备，诸如是扬声器和打印机。
定位设备400可以通过网络接口15与一个或多个远程计算机连接。远程计算机可以是个人计算机、服务器、路由器、网络PC、对等设备或者其他常见网络节点，并且通常包括上面关于定位设备400所描述元件中的许多或者全部单元。
本公开实施例还提供一种计算机可读存储介质，其上存储有计算机可执行指令，所述计算机指令在一个或多个处理器上运行时，使得所述一个或多个处理器执行根据本公开实施例的方法和功能。所述计算机可读介质可以包括以上所述的任何一种计算机可读介质。
本公开实施例还提供一种计算机程序产品，当上述计算机程序产品中的指令由处理器执行时，可以实现根据本公开实施例的方法。
在本说明书的描述中，术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点被包含于本公开的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本公开的描述中，“多个”的含义是至少两个，例如两个、三个等，除非另有明确具体的限定。
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本公开的实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序(包括根据所涉及的功能按基本同时的方式或按相反的顺序)来执行功能，这应被本公开的实施例所属技术领域的技术人员所理解。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系 统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。就本说明书而言，“计算机可读介质”可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读介质的更具体的示例可以例如包括以下各项：具有一个或多个布线的电连接部(电子装置)、便携式计算机盘盒(磁装置)、随机存取存储器(Random Access Memory)、只读存储器(Read Only Memory)，可擦除可编辑只读存储器(Erasable Programmable Read Only Memory)或闪速存储器、光纤装置、以及便携式光盘只读存储器(Compact Disc Read Only Memory)。另外，计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。
应当理解，本公开的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。例如，如果用硬件来实现，则可用本领域公知的下列技术中的任一项或它们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路、具有合适的组合逻辑门电路的专用集成电路、可编程门阵列(Programmable Gate Array)、现场可编程门阵列(Field Programmable Gate Array)等。
本技术领域的普通技术人员可以理解上述实施例方法的全部或部分步骤可以通过程序指令相关的硬件完成，所述程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括执行方法实施例的步骤之一或其组合。
此外，在本公开各个实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。
尽管上面已经示出和描述了本公开的实施例，但是可以理解的是，上述实施例是示例性的，不应理解为对本公开的限制，本领域的普通技术人员在本公开的范围内可以对上述实施 例进行变化、修改和替换。

Claims (18)

  1. 一种定位方法，包括：
    获取相机所捕获的图像；
    基于深度学习模型从所述图像获得所述相机在世界坐标系下的第一坐标，
    基于所述第一坐标获得所述相机在世界坐标系下的初始坐标；以及
    基于所述相机在世界坐标系下的初始坐标，通过帧间匹配确定所述相机在世界坐标系下的实时坐标。
  2. 根据权利要求1所述的方法，还包括：
    检测所述图像中是否包括标志物，其中所述标志物具有至少3个顶点，
    其中，所述基于所述第一坐标获得所述相机在世界坐标系下的初始坐标包括：
    响应于所述图像中不包括所述标志物，将所述第一坐标用作所述相机在世界坐标系下的初始坐标。
  3. 根据权利要求1所述的方法，还包括：
    检测所述图像是否包括标志物，其中所述标志物具有至少3个顶点；
    其中，所述基于所述第一坐标获得所述相机在世界坐标系下的初始坐标包括：
    响应于所述图像中包括所述标志物，基于所述标志物和所述第一坐标获得所述相机在世界坐标系下的第二坐标；以及
    将所述第二坐标用作所述相机在世界坐标系下的初始坐标。
  4. 根据权利要求3所述的方法，其中，所述基于所述标志物和所述第一坐标获得相机在世界坐标系下的第二坐标包括：
    基于所述图像获得所述标志物的顶点在相机坐标系下的坐标；
    基于所述第一坐标从预先存储的多个标志物的顶点在世界坐标系下的坐标，确定所述图像中包括的所述标志物的顶点在世界坐标系下的坐标；
    基于所述图像中包括的所述标志物的顶点在相机坐标系下的坐标以及所述图像中包括的所述标志物的顶点在世界坐标系下的坐标，获得相机坐标系相对于世界坐标系的旋转矩阵和 平移矩阵；以及
    基于所述旋转矩阵和所述平移矩阵确定所述相机在世界坐标系下的第二坐标。
  5. 根据权利要求2或3所述的方法，其中，所述检测所述图像中是否包括标志物包括：
    基于单次多框检测器算法检测所述图像中是否包括标志物。
  6. 根据权利要求2或3所述的方法，还包括，
    以预设的时间间隔更新所述相机在世界坐标系下的初始坐标；以及
    基于更新的相机在世界坐标系下的初始坐标，通过帧间匹配确定相机在世界坐标系下的实时坐标。
  7. 根据权利要求6所述的方法，其中所述更新所述相机在世界坐标系下的初始坐标包括：
    基于实时获取的图像中包括的场景和标志物中的至少一个，获得相机在世界坐标系下的初始坐标。
  8. 根据权利要求1所述的方法，其中，所述深度学习模型包括基于PoseNet网络的深度学习模型。
  9. 一种基于定位装置，包括：
    图像获取器，被配置成获取相机所捕获的图像；
    初始坐标确定器，与所述图像获取器相耦接且被配置成：基于深度学习模型从所述图像获得所述相机在世界坐标系下的第一坐标，基于所述第一坐标获得所述相机在世界坐标系下的初始坐标：以及
    实时位置确定器，与图像获取器和所述初始坐标确定器相耦接，且被配置成：基于所述相机在世界坐标系下的初始坐标，通过帧间匹配确定所述相机在世界坐标系下的实时坐标。
  10. 根据权利要求9所述的装置，其中，所述初始坐标确定器还被配置成：检测所述图像中是否包括标志物，其中所述标志物具有至少3个顶点；以及
    响应于所述图像中不包括所述标志物，将所述第一坐标用作所述相机在世界坐标系下的初始坐标。
  11. 根据权利要求9所述的装置，其中，所述初始坐标确定器还被配置成：检测所述图像是否包括标志物，其中所述标志物具有至少3个顶点；
    响应于所述图像中包括所述标志物，基于所述标志物和所述第一坐标获得所述相机在世界坐标系下的第二坐标，以及
    将所述第二坐标用作所述相机在世界坐标系下的初始坐标。
  12. 根据权利要求11所述的装置，其中，所述初始坐标确定器还被配置成：
    基于所述图像获得所述标志物的顶点在相机坐标系下的坐标；
    基于所述第一坐标从预先存储的多个标志物的顶点在世界坐标系下的坐标，确定所述图像中包括的所述标志物的顶点在世界坐标系下的坐标；
    基于所述图像中包括的所述标志物的顶点在相机坐标系下的坐标以及所述图像中包括的所述标志物的顶点在世界坐标系下的坐标，获得相机坐标系相对于世界坐标系的旋转矩阵和平移矩阵；以及
    基于所述旋转矩阵和所述平移矩阵确定所述相机在世界坐标系下的第二坐标。
  13. 根据权利要求10或11所述的装置，其中，所述初始坐标确定器还被配置成：基于单次多框检测器算法检测所述图像中是否包括标志物。
  14. 根据权利要求10或11所述的装置，其中，所述初始坐标确定器还被配置成：
    以预设的时间间隔更新所述相机在世界坐标系下的初始坐标；以及
    基于更新的相机在世界坐标系下的初始坐标，通过帧间匹配确定相机在世界坐标系下的实时坐标。
  15. 根据权利要求14所述的装置，其中所述初始坐标确定器还被配置成：
    基于由图像获取器实时获取的图像中包括的场景和标志物中的至少一个，获得所述相机在世界坐标系下的初始坐标。
  16. 根据权利要求9所述的装置，其中，所述深度学习模型包括基于PoseNet网络的深度学习模型。
  17. 一种定位设备，包括：
    一个或多个处理器；以及
    存储器，与所述一个或多个处理器相耦接，并存储有计算机可执行指令，所述计算机可执行指令在由所述一个或多个处理器执行时使得所述一个或多个处理器执行如权利要求1-8中任一项所述的方法。
  18. 一种计算机可读存储介质，其上存储了计算机可执行指令，所述计算机可执行指令在由一个或多个处理器执行时，使所述一个或多个处理器执行如权利要求1-8中任一项所述的方法。
PCT/CN2019/097248 2018-08-01 2019-07-23 定位方法和装置 WO2020024845A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/635,036 US11100670B2 (en) 2018-08-01 2019-07-23 Positioning method, positioning device and nonvolatile computer-readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810862721.1A CN110163914B (zh) 2018-08-01 2018-08-01 基于视觉的定位
CN201810862721.1 2018-08-01

Publications (1)

Publication Number Publication Date
WO2020024845A1 true WO2020024845A1 (zh) 2020-02-06

Family

ID=67645162

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/097248 WO2020024845A1 (zh) 2018-08-01 2019-07-23 定位方法和装置

Country Status (3)

Country Link
US (1) US11100670B2 (zh)
CN (1) CN110163914B (zh)
WO (1) WO2020024845A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113283496A (zh) * 2021-05-21 2021-08-20 国网宁夏电力有限公司中卫供电公司 一种应用于配网不停电作业危险动作智能识别的方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111445531B (zh) * 2020-03-24 2022-08-30 云南电网有限责任公司楚雄供电局 一种多目相机导航方法、装置、设备及存储介质
CN113129378A (zh) * 2021-04-28 2021-07-16 北京市商汤科技开发有限公司 一种定位方法、装置、电子设备及存储介质
CN114049400A (zh) * 2021-10-29 2022-02-15 广东嘉腾机器人自动化有限公司 圆形目标物的定位方法及其设备、agv、介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140300637A1 (en) * 2013-04-05 2014-10-09 Nokia Corporation Method and apparatus for determining camera location information and/or camera pose information according to a global coordinate system
CN106485753A (zh) * 2016-09-09 2017-03-08 奇瑞汽车股份有限公司 用于无人驾驶汽车的摄像机标定的方法和装置
CN106780699A (zh) * 2017-01-09 2017-05-31 东南大学 一种基于sins/gps和里程计辅助的视觉slam方法
CN107438752A (zh) * 2016-12-23 2017-12-05 深圳前海达闼云端智能科技有限公司 定位方法、终端和服务器

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9031809B1 (en) * 2010-07-14 2015-05-12 Sri International Method and apparatus for generating three-dimensional pose using multi-modal sensor fusion
CN102506830B (zh) * 2011-11-21 2014-03-12 奇瑞汽车股份有限公司 视觉定位方法及装置
CN104881860B (zh) * 2014-02-28 2019-01-08 国际商业机器公司 基于照片进行定位的方法和装置
ES2584554B2 (es) * 2015-11-30 2017-06-13 Defensya Ingeniería Internacional, S.L. Sistema de detección de punta de la pértiga y boca de receptáculo, automatización progresiva del repostaje aéreo con botalón y procedimiento de repostaje
CN106780609B (zh) * 2016-11-28 2019-06-11 中国电子科技集团公司第三研究所 视觉定位方法和视觉定位装置
CN107742311B (zh) * 2017-09-29 2020-02-18 北京易达图灵科技有限公司 一种视觉定位的方法及装置
CN107833236B (zh) * 2017-10-31 2020-06-26 中国科学院电子学研究所 一种动态环境下结合语义的视觉定位系统和方法
US11487351B2 (en) * 2018-11-23 2022-11-01 International Business Machines Corporation Intelligent directing system in an internet of things (IoT) computing environment
US10885666B2 (en) * 2019-02-06 2021-01-05 Ford Global Technologies, Llc Hybrid metric-topological camera-based localization
US12094175B2 (en) * 2020-11-24 2024-09-17 Intel Corporation Real-time calibration of wide-baseline outside-in multi-camera systems

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140300637A1 (en) * 2013-04-05 2014-10-09 Nokia Corporation Method and apparatus for determining camera location information and/or camera pose information according to a global coordinate system
CN106485753A (zh) * 2016-09-09 2017-03-08 奇瑞汽车股份有限公司 用于无人驾驶汽车的摄像机标定的方法和装置
CN107438752A (zh) * 2016-12-23 2017-12-05 深圳前海达闼云端智能科技有限公司 定位方法、终端和服务器
CN106780699A (zh) * 2017-01-09 2017-05-31 东南大学 一种基于sins/gps和里程计辅助的视觉slam方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113283496A (zh) * 2021-05-21 2021-08-20 国网宁夏电力有限公司中卫供电公司 一种应用于配网不停电作业危险动作智能识别的方法
CN113283496B (zh) * 2021-05-21 2022-07-29 国网宁夏电力有限公司中卫供电公司 一种应用于配网不停电作业危险动作智能识别的方法

Also Published As

Publication number Publication date
US20200380717A1 (en) 2020-12-03
CN110163914A (zh) 2019-08-23
US11100670B2 (en) 2021-08-24
CN110163914B (zh) 2021-05-25

Similar Documents

Publication Publication Date Title
JP5950973B2 (ja) フレームを選択する方法、装置、及びシステム
US10810734B2 (en) Computer aided rebar measurement and inspection system
WO2020024845A1 (zh) 定位方法和装置
WO2021136386A1 (zh) 数据处理方法、终端和服务器
US7554575B2 (en) Fast imaging system calibration
CN103026385B (zh) 使用模板切换和特征适配提供对象跟踪的方法、装置和计算机程序产品
CN110275516A (zh) 通过移除动态交通参与者的稳健的即时定位和地图构建
US20150103183A1 (en) Method and apparatus for device orientation tracking using a visual gyroscope
CN110648363B (zh) 相机姿态确定方法、装置、存储介质及电子设备
CN117940956A (zh) 关键点检测和特征描述符计算
US10878577B2 (en) Method, system and apparatus for segmenting an image of a scene
JP6591594B2 (ja) 情報提供システム、サーバ装置、及び情報提供方法
CN107343141A (zh) 对焦方法、装置和计算机设备
CN109754034A (zh) 一种基于二维码的终端设备定位方法及装置
US20210133995A1 (en) Electronic devices, methods, and computer program products for controlling 3d modeling operations based on pose metrics
CN110706257B (zh) 有效特征点对的识别方法、相机状态的确定方法及装置
CN108777784B (zh) 深度获取方法和装置、电子装置、计算机设备和存储介质
JP7003617B2 (ja) 推定装置、推定方法、及び推定プログラム
JP2014102805A (ja) 情報処理装置、情報処理方法及びプログラム
JP2006113832A (ja) ステレオ画像処理装置およびプログラム
WO2016141984A1 (en) Image processing device and method for geometric calibration of images
WO2022205841A1 (zh) 机器人导航方法、装置、终端设备及计算机可读存储介质
JP2008135996A (ja) 情報処理方法、情報処理装置
US20220030206A1 (en) Information processing apparatus, information processing method, program, and projection system
CN112766061A (zh) 一种多模态无监督的行人像素级语义标注方法和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19843938

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19843938

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 15.10.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19843938

Country of ref document: EP

Kind code of ref document: A1