[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN112733641A - Object size measuring method, device, equipment and storage medium - Google Patents

Object size measuring method, device, equipment and storage medium Download PDF

Info

Publication number
CN112733641A
CN112733641A CN202011603309.1A CN202011603309A CN112733641A CN 112733641 A CN112733641 A CN 112733641A CN 202011603309 A CN202011603309 A CN 202011603309A CN 112733641 A CN112733641 A CN 112733641A
Authority
CN
China
Prior art keywords
point cloud
images
image
coordinates
scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011603309.1A
Other languages
Chinese (zh)
Other versions
CN112733641B (en
Inventor
乔亮
龙力
李鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Yishi Huolala Technology Co Ltd
Original Assignee
Shenzhen Yishi Huolala Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Yishi Huolala Technology Co Ltd filed Critical Shenzhen Yishi Huolala Technology Co Ltd
Priority to CN202011603309.1A priority Critical patent/CN112733641B/en
Publication of CN112733641A publication Critical patent/CN112733641A/en
Application granted granted Critical
Publication of CN112733641B publication Critical patent/CN112733641B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/08—Projecting images onto non-planar surfaces, e.g. geodetic screens
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/757—Matching configurations of points or features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The embodiment of the application provides a method and a device for measuring the size of an object, computer equipment and a storage medium, wherein the method comprises the steps of obtaining a plurality of frames of images of the object to be measured and the posture and point cloud of each frame of image; updating and expanding the point clouds of the images according to the posture change among the multiple frames of images, and fusing the same point clouds among the multiple frames of images; carrying out three-dimensional reconstruction on the scene of the object to be detected, and calculating the coordinates of the point cloud after reconstruction; and selecting a point cloud range, and measuring the size of the object to be measured according to the point cloud coordinates in the range. In the embodiment, the point cloud of the image is updated and expanded, the same point cloud among multiple frames of images is fused, the point cloud of the image is updated and expanded, the obtained three-dimensional point cloud information is covered comprehensively, accurate data are provided for carrying out three-dimensional reconstruction on the scene of the image, coordinates in the scene can be updated continuously, errors caused by a single visual angle can be avoided, and the measurement result is more accurate when the point cloud range is selected for measuring the object to be measured.

Description

Object size measuring method, device, equipment and storage medium
Technical Field
The application relates to the technical field of intelligent measurement objects, in particular to a method, a device, equipment and a storage medium for measuring body dimensions.
Background
In daily life, we are often faced with the need to measure the dimensions of some objects, but measuring tools are not necessarily present around. With the provision of the scientific and technological level and the living level, various intelligent terminals are integrated into the daily life of people, and the geometric dimension of an object to be detected can be quickly obtained through the visual detection of the intelligent terminals and the like. For example, in some scenes, there are some requirements for identifying the types and sizes of objects, so that the matched vehicle type or packing box can be accurately selected in moving and transporting.
Most of the prior art intelligent measurement size schemes are calculated based on the distance between two points in an AR (Augmented Reality) scene, that is, two anchor points are selected in the AR constructed scene to calculate the length between two points of an object, or at least 2 rectangular vertexes are selected to calculate the area of the object, or at least 4 cubic vertexes are selected to calculate the volume of the object.
However, since the mobile terminal AR greatly simplifies the image and scene information in order to calculate the pose of the camera in real time, when an object image is acquired, each acquired image only corresponds to hundreds of point clouds, the distribution is sparse, and the shape of the object cannot be accurately reflected. Moreover, the movable end AR can only recognize a large horizontal or vertical plane, and the planar range is not completely consistent with the planar range on the object, resulting in incomplete reaction shape of the object and further deviation. And tends to result in the selected point not necessarily being on the surface of the object when the range measurement is taken, further affecting the measurement result.
Disclosure of Invention
The embodiment of the application aims to provide an object size measuring method, device, equipment and storage medium, and aims to solve the technical problems that in the prior art, the point cloud of an image of an object to be measured is inaccurate and sparse, the shape of the object is inaccurate, and the measuring result is influenced.
In order to solve the above technical problem, an embodiment of the present application provides an object dimension measuring method, including the following steps:
acquiring a plurality of frames of images of an object to be detected and the posture and point cloud of each frame of image;
updating and expanding the point clouds of the images according to the posture change among the multiple frames of images, and fusing the same point clouds among the multiple frames of images;
carrying out three-dimensional reconstruction on the scene of the object to be detected, and calculating the coordinates of the point cloud after reconstruction;
and selecting a point cloud range, and measuring the size of the object to be measured according to the point cloud coordinates in the range.
Further, the updating and expanding of the point clouds of the images according to the posture change among the multiple frames of images and the fusion of the same point clouds among the multiple frames of images comprise
Acquiring the attitude of each frame of image at the acquisition moment, and estimating the relative position of the object to be detected at different moments;
tracking the point cloud position change of the image according to the relative position change of the object to be detected, and recording the number, the coordinates and the corresponding image characteristics of the point cloud;
and fusing the same point clouds among the multiple frames of images according to the numbers, the coordinates and the corresponding image characteristics.
Further, the algorithm for fusing the same point clouds among the multiple frames of images comprises a feature matching algorithm or a point cloud ICP algorithm.
Further, the three-dimensional reconstruction is carried out on the scene of the image, and the calculation of the coordinates of the point cloud after reconstruction comprises
Selecting at least two key frames, and reconstructing the scene of the object image to be detected according to the image and the point cloud of the key frames and the posture of the image acquisition moment;
and calculating the coordinates of the point cloud in the reconstructed new scene.
Further, the selecting the point cloud range to measure the object to be measured comprises
Selecting a range in the image;
projecting rays on the scene in the selected range to form a light cone;
and calculating the size of the object to be measured according to the point cloud coordinates in the light cone.
Further, the selecting the point cloud range to measure the object to be measured further comprises
After the scene of the image is subjected to three-dimensional reconstruction, the point cloud is projected, and point cloud coordinates in a range are selected to calculate the size of the object to be measured.
Further, when the size of the object to be measured is measured, a three-dimensional identification is formed on the object to be measured.
In order to solve the above technical problem, an embodiment of the present application further provides an object dimension measuring apparatus, including:
the image acquisition module is used for acquiring multi-frame images of the object to be detected and the attitude and point cloud of each frame of the images;
the point cloud fusion module is used for updating and expanding the point cloud of the image according to the change of the postures of the images of multiple frames and fusing the same point cloud among the images of the multiple frames;
the three-dimensional reconstruction module is used for performing three-dimensional reconstruction on the scene of the object to be detected and calculating the coordinates of the point cloud after reconstruction;
and the size calculation module is used for selecting a point cloud range and measuring the size of the object to be measured according to the point cloud coordinates in the range.
In order to solve the above technical problem, an embodiment of the present application further provides a computer device, which adopts the following technical solutions:
a computer device comprising a memory, a processor and a network interface, the memory having stored therein a computer program, the processor implementing the steps of the object dimension measuring method described above when executing the computer program.
In order to solve the above technical problem, an embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solutions:
a computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the object dimension measuring method described above.
Compared with the prior art, the embodiment of the application mainly has the following beneficial effects:
the embodiment of the application provides an object size measurement method, through obtaining the multiframe image and every frame of the object that awaits measuring the gesture and the point cloud of image, and according to the multiframe the change of image gesture is right the point cloud of image updates the extension, through with the same point cloud fusion between the multiframe image to update the extension to the point cloud of image, the three-dimensional point cloud information that acquires covers comprehensively, provides accurate data right the scene of image carries out three-dimensional reconstruction, and calculates the coordinate of rebuilding back point cloud, and the coordinate in the scene of the object that awaits measuring can be constantly updated, can avoid the error that single visual angle caused, so that it is right to select the point cloud scope the object that awaits measuring when measuring measurement. Therefore, the object size measuring device of the invention can conveniently and quickly measure the size of the object, and has high measuring reliability and accuracy.
Drawings
In order to more clearly illustrate the solution of the present application, the drawings needed for describing the embodiments of the present application will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
FIG. 1 is a diagram of an exemplary system architecture in which embodiments of the present invention may be employed;
FIG. 2 is a schematic flow chart of the object dimension measuring method of the present application;
FIG. 3 is a schematic view of a sample of the subject dimensional measurement method of application;
FIG. 4 is a schematic structural view of one embodiment of an object dimension measuring device of the present application;
FIG. 5 is a schematic block diagram of one embodiment of a computer device according to the present application.
Reference numerals:
100 system architecture 101、102、103 Terminal device
104 Network 105 Server
4 Object size measuring device 401 Image acquisition module
402 Point cloud fusion module 403 Three-dimensional reconstruction module
404 Size calculation module
5 Computer equipment 51 Memory device
52 Processor with a memory having a plurality of memory cells 53 Network interface
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof, in the description and claims of this application and the description of the above figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or in the above-described drawings are used for distinguishing between different objects and not for describing a particular order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user can use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like to operate the services and applications of the object dimension measuring method. The terminal devices 101, 102, 103 may have various communication client applications installed thereon, such as a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, and the like.
The terminal devices 101, 102, 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to user devices, network devices, or devices formed by integrating user devices and network devices through a network. The user equipment includes, but is not limited to, any mobile electronic product, such as a smart phone, a tablet computer, and the like, which can perform human-computer interaction with a user through a touch panel, and the mobile electronic product may employ any operating system, such as an android operating system, an iOS operating system, and the like. The network device includes an electronic device capable of automatically performing numerical calculation and information processing according to preset or stored instructions, and the hardware includes but is not limited to a microprocessor, an Application Specific Integrated Circuit (ASIC), a programmable gate array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like. The network device comprises but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud formed by a plurality of servers; here, the Cloud is composed of a large number of computers or web servers based on Cloud Computing (Cloud Computing), which is a kind of distributed Computing, one virtual supercomputer consisting of a collection of loosely coupled computers. Including, but not limited to, the internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless Ad Hoc network (Ad Hoc network), etc. Of course, those skilled in the art should understand that the above terminal device is only an example, and other existing or future terminal devices may be applicable to the present application, and are included in the scope of the present application and are incorporated herein by reference.
The server 105 may be a server, a server cluster composed of several servers, or a cloud computing service center. It may also be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that the object size measuring method provided in the embodiments of the present application is generally executed by a terminal device, and accordingly, the object size measuring apparatus is generally disposed in the terminal device.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With the increasing functionality of terminal mobile devices (e.g., mobile phones, tablet computers, etc.), users will carry the mobile devices with them every day, but will not necessarily carry tools such as measuring scales with them. For example, sometimes, if an object is placed in a certain place, but the placing space is not enough, the specific length or volume of the object needs to be known. Or sometimes it is also desirable to know what the person's height is in order to fill in information, etc. However, the user does not have a tool for measuring the length of the object, and when the user does not have a measuring scale or other tools for measuring the length and other dimensions of the object or the size of the object is too large to be directly measured by the measuring scale, a method for easily and conveniently measuring the size information of the object by using the portable mobile terminal device is needed, so that the user can conveniently and quickly know the size of the object to be known.
In view of the problems in the background art, the following technical solutions are proposed in the present application. Referring to FIG. 2, a flow chart of one embodiment of a method of measuring dimensions of an object according to the present application is shown. The object size measuring method comprises the following steps:
101: acquiring a plurality of frames of images of the object to be detected and the posture and point cloud of each frame of image.
In the embodiment of the invention, the image of the object to be detected can be acquired through a mobile terminal, such as a mobile phone camera, a tablet personal computer and other equipment with a camera shooting function.
In the embodiment of the present invention, the point cloud is a set of point data on the surface of the object to be measured, and the point cloud data is represented in the form of three-dimensional coordinates of world coordinates X, Y, and Z, and in addition, the point cloud data may also represent RGB color, gray value, depth, segmentation result, and the like of one point in addition to the geometric location information represented by (X, Y, Z). Specifically, the point cloud is obtained by acquiring an image of an object to be measured, video image data and point cloud data in the image by using sensors such as an accelerometer, a gyroscope and a camera on mobile equipment, then estimating the relative position of the moving target at different times by using algorithms such as VIO (Visual Inertial odometer) or SLAM (Simultaneous localization and mapping, even if positioning and mapping are performed instantly) and processing the accumulated error of the estimation result of the Visual odometer, and performing three-dimensional coordinate construction on the object to be measured.
In this embodiment, when the image of the object to be detected is obtained in step 101, it is necessary to slowly move a mobile device such as a mobile phone, and obtain the image of the object to be detected as a continuously obtained offset image, and then acquire information of a large number of points on the surface of the target object through a VIO algorithm, as shown in fig. 3, the hashed points in the image are point clouds of the object to be detected, and store coordinates of the acquired point clouds.
Further, the mobile terminal equipment acquires multi-frame images of the object to be detected, acquires the posture of each frame image at the acquisition moment and corresponding sparse point cloud, and records and stores the number, three-dimensional coordinates and corresponding image characteristics of the point cloud.
102: and updating and expanding the point clouds of the images according to the posture change among the multiple frames of images, and fusing the same point clouds among the multiple frames of images.
In step 101, mobile devices such as a mobile phone need to be slowly moved, the object to be measured is constantly observed at different angles or poses, and the point cloud is also continuously updated and expanded, so that observation results of different visions are accumulated.
The embodiment of the invention repeatedly positions and collects point clouds from a plurality of visual angles, updates and expands the point clouds of the images, transforms multi-frame point clouds to the same space coordinate system by using a point cloud splicing technology, and fuses the same point clouds among the multi-frame images so as to restore the whole scene or object. In the embodiment, the relative position of the object to be detected at different moments is estimated by acquiring the posture of each frame of image at the acquisition moment, the point cloud position change of the image is tracked according to the relative position change of the object to be detected, the point cloud information in each frame of image is recorded, and the number, the coordinates and the corresponding image characteristics of the point cloud are recorded; and fusing the same point clouds among the multiple frames of images according to the numbers, the coordinates and the corresponding image characteristics. The same point clouds among multiple frames of images are fused by updating and expanding the point clouds of the images, the three-dimensional point cloud information acquired by the embodiment is comprehensively covered, accurate data are provided for the subsequent point cloud data processing of three-dimensional reconstruction, and the data processing work is greatly facilitated.
Specifically, in this embodiment, an ICP (Iterative Closest Point) algorithm is used to fuse the overlapped Point clouds, and Point clouds in different visual environments are converted by a certain rotation and translation to unify Point cloud data in different coordinate systems into the same reference coordinate system.
Further, fusing the same point clouds among the images comprises the following processes:
(1) taking a point set P in a frame of image point cloud Pi∈P；
(2) From a corresponding point set Q in another frame of image point cloud QiBelongs to Q, so that | | | Qi-pi||＝min；
(3) Calculating a rotation matrix R and a translation matrix t to minimize an error function;
(4) carrying out rotation and translation transformation on pi by using the rotation matrix R and the translation matrix t obtained in the previous step to obtain a new corresponding point set pi’＝{pi’＝Rpi+t，pi∈P}；
(5) Calculating pi' and corresponding point set qiAverage distance of (d):
Figure BDA0002869862090000081
(6) if d is less than a given threshold or greater than a preset maximum number of iterations, the iterative computation is stopped. Otherwise, returning to the step (2) until the convergence condition is met.
In this embodiment, the error function E (R, t) in step (3) is:
Figure BDA0002869862090000082
where n is the number of nearest neighbor point pairs, piAnd (3) taking one point in the target point cloud P, qi as the closest point corresponding to pi in the source point cloud Q, R as a rotation matrix, and t as a translation vector.
In other embodiments, other algorithms may be used to fuse the point cloud, such as a feature matching method, a TSDF (spatial voxel based fusion method) algorithm, or a point cloud fusion method based on a clustering concept, etc.
103: and carrying out three-dimensional reconstruction on the scene of the object to be detected, and calculating the coordinates of the point cloud after reconstruction.
In the embodiment of the invention, the image of the object to be detected acquired in step 101 is a two-dimensional image, the same points among multiple frames of images are fused in step 102, the three-dimensional point clouds among the multiple frames of images form a corresponding relation, and the characteristics of the multiple frames of images correspond one to form stereo matching. And based on the point cloud information fused and matched among the multiple frames, carrying out three-dimensional scene reconstruction on the object to be detected, recovering the geometric structure of the object to be detected, and obtaining a three-dimensional model of the object to be detected. Furthermore, through the three-dimensional reconstruction of multiple visual angles, the coordinates of the point cloud in the object scene to be detected can be continuously updated along with the change of the posture, and errors caused by a single visual angle can be avoided.
In the embodiment of the invention, a scene is three-dimensionally reconstructed through sfm (structure from motion) and other algorithms, and new three-dimensional coordinates are sampled from the reconstructed scene according to the stored coordinates of the point cloud or image features. Specifically, in this embodiment, a first frame of image of the object to be measured is obtained, point cloud information of the object to be measured is recorded, the frame serving as the frame for recording the point cloud is recorded as a first key frame, along with step 101, the terminal device is moved greatly, another frame of image is selected as a second key frame, new key frames are continuously updated, when more than two key frames are obtained, a three-dimensional reconstruction is performed on a scene of the object to be measured according to the image, the feature point, the point cloud of the key frame and the pose of the mobile terminal, and coordinates of the point cloud after the reconstruction are calculated.
More specifically, the sfm algorithm describes and extracts feature points and point clouds of the image by ORB (organized FAST and Rotated bright BRIEF, algorithm for FAST feature point extraction and description) by extracting point clouds and feature points of all key frames. Matching any two key frames, constraining the point clouds fused in the step 102 by using a homography matrix (homography matrix) or a fundamental matrix (fundamental matrix), and rejecting misfused point clouds (for example, rejecting mismatching points by using a RANSAC method). Projecting the correct camera attitude (R, t) and point cloud to a zero system according to the error function in the step 102, wherein the zero system is a camera coordinate system corresponding to the first key frame; and then, optimizing by using a Beam Adjustment (BA), projecting all three-dimensional points in a zero system onto each key frame according to the camera posture (R, t) of each key frame, calculating the projection error of each three-dimensional point, adjusting the camera posture (R, t) of each key frame, and performing multiple iterations to reduce the projection error to the minimum.
104: and selecting a point cloud range, and measuring the size of the object to be measured according to the point cloud coordinates in the range.
In the prior art, a ray is generally emitted into a scene from the position of an AR camera along the center of the screen or the direction of a click point of a user clicking the screen, an intersection point is calculated when the ray intersects with an estimated plane in the scene, the coordinates of a point in a current frame point cloud are calculated when the ray intersects with the point or the distance of the point is within a certain range, and a point closest to the camera is selected from the intersection results as a point selected by the user. This approach only considers the situation at the current viewing angle, and the selected point is not necessarily on the object surface, which is generally the case when the viewing angle is changed. Namely, the accuracy of the measurement method in the prior art depends on the accuracy of two point selection, the measurement result has high contingency, once the point is selected, the point cannot be updated and improved in the subsequent AR session, and the constraint update of multiple visual angles is not supported.
In the embodiment, when the point cloud is selected, a range is determined from the image of the object to be detected, the selected range can be rectangular, circular or other shapes, rays are projected into a scene from the position of the mobile terminal camera through the determined screen range to form a selected light cone, and the point cloud located in the light cone is selected. Further, the point cloud of the same ray is selected as the point cloud closest to the camera.
Three-dimensional coordinates of each point of the selected point cloud are acquired, and the size of the object is calculated according to the point clouds. Specifically, the height of the object to be measured is calculated by subtracting the minimum value from the maximum value of the y coordinates of the point clouds, and the width and the height of the object to be measured are determined by projecting the x and z coordinates of the point clouds to the main axis direction and selecting the maximum and minimum projection values. The main axis direction of the coordinates is obtained by decomposing the eigenvalues of the x and z coordinates, so that the three-dimensional bounding box can be determined according to the actual aspect ratio and the placement of the object, rather than the maximum and minimum x, y, and z in the world coordinate system.
In yet another embodiment, selecting the point cloud is back projecting the sparse point cloud onto a screen and then selecting the point cloud that lies within the range. The selected point clouds are displayed on the screen in different colors.
It should be noted that, after the object range is determined through the above steps, the range may be converted into a three-dimensional cube identifier, and the three-dimensional cube identifier may be drawn on a screen. As shown in fig. 3, the range of the object to be measured is a range of a cube in the drawing. With the continuous selection of the user, the three-dimensional surrounding frame is continuously updated, so that the measurement progress and the measurement precision can be mastered by the user in real time.
The embodiment of the application provides an object size measurement method, through right the point cloud of image updates the extension, through inciting somebody to action the same point cloud fuses between the multiframe image to the point cloud to the image updates the extension, and the three-dimensional point cloud information that obtains covers comprehensively, provides accurate data right the scene of image carries out three-dimensional reconstruction, and the coordinate in the scene can constantly be updated, can avoid the error that single visual angle caused, so that it is right to select the point cloud scope the measuring result is more accurate when the object that awaits measuring is measured.
In order to solve the above technical problem, as shown in fig. 4, an object dimension measuring device 4 is further provided in the embodiment of the present application.
An object dimension measuring apparatus 4, comprising:
the image acquisition module 401 is configured to acquire a plurality of frames of images of an object to be detected and a posture and a point cloud of each frame of the images;
a point cloud fusion module 402, configured to update and expand the point cloud of the image according to the change of the image pose, and fuse the same point cloud among the multiple frames of images;
a three-dimensional reconstruction module 403, configured to perform three-dimensional reconstruction on the scene of the image, and calculate coordinates of a point cloud after reconstruction;
and the size calculation module 404 is configured to select a point cloud range to measure the object to be measured.
The embodiment of the application acquires multiframe images and each frame of an object to be measured through the image acquisition module 401 the attitude and the point cloud of the image, the point cloud fusion module 402 is used for covering the acquired three-dimensional point cloud information comprehensively according to the change of the image attitude, right the point cloud updating and expanding of the image, and the same point cloud fusion between the multiframe images, and through the point cloud updating and expanding of the image, the acquired three-dimensional point cloud information is covered comprehensively, so that the three-dimensional reconstruction module 403 is provided, the scene of the image is subjected to three-dimensional reconstruction, the coordinate in the scene is continuously updated, the error caused by a single visual angle can be avoided, and the point cloud range is selected by the size calculation module 404 to be more accurate in the. The object size measuring device can measure the size of the object more conveniently and quickly, and has high measuring reliability and accuracy.
The object size measuring device 4 further comprises a display module, the display module is connected with the adjusting module, and the display module is used for displaying data information and an operation page of the object size measuring device.
The object size measuring device 4 may further include an input module, the input module is connected to the display module, the input module may include a key, and may be configured to input information such as an account number, a password, and a name of a user id, the object size measurement process operation page may be displayed on the display module in the object size measuring device, and the display module may further display other information of the user, and store the information, so that the user can view the information at any time.
It should be noted that the apparatus of this embodiment and the method embodiment belong to the same concept, and specific implementation processes thereof are detailed in the method embodiment, and technical features in the method embodiment are all correspondingly applicable in this embodiment, which is not described herein again.
In order to solve the above technical problem, the embodiment of the present application further provides a computer device 5. Referring to fig. 5, (Application Specific Integrated Circuit, ASIC), Programmable Gate Array (FPGA), Digital Signal Processor (DSP), embedded device, etc.
The computer device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The computer equipment can carry out man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch panel or voice control equipment and the like.
The memory 51 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 51 may be an internal storage unit of the computer device 5, such as a hard disk or a memory of the computer device 5. In other embodiments, the memory 51 may also be an external storage device of the computer device 5, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the computer device 5. Of course, the memory 51 may also comprise both an internal storage unit of the computer device 5 and an external storage device thereof. In this embodiment, the memory 51 is generally used for storing an operating fund system installed in the computer device 5 and various types of application software, such as program codes of an object dimension measuring method. Further, the memory 51 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 52 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 52 is typically used to control the overall operation of the computer device 5. In this embodiment, the processor 52 is configured to execute the program code stored in the memory 51 or process data, for example, execute the program code of the object dimension measuring method.
The network interface 53 may comprise a wireless network interface or a wired network interface, and the network interface 53 is generally used for establishing communication connections between the computer device 5 and other electronic devices.
The present application provides yet another embodiment, which provides a computer-readable storage medium storing an object dimension measurement program executable by at least one processor to cause the at least one processor to perform the steps of the object dimension measurement method as described above.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.
It is to be understood that the above-described embodiments are merely illustrative of some, but not restrictive, of the broad invention, and that the appended drawings illustrate preferred embodiments of the invention and do not limit the scope of the invention. This application is capable of embodiments in many different forms and is provided for the purpose of enabling a thorough understanding of the disclosure of the application. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to one skilled in the art that the present application may be practiced without modification or with equivalents of some of the features described in the foregoing embodiments. All equivalent structures made by using the contents of the specification and the drawings of the present application are directly or indirectly applied to other related technical fields and are within the protection scope of the present application.

Claims (10)

1. A method of measuring the dimensions of an object, comprising the steps of:
acquiring a plurality of frames of images of an object to be detected and the posture and point cloud of each frame of image;
updating and expanding the point clouds of the images according to the posture change among the multiple frames of images, and fusing the same point clouds among the multiple frames of images;
carrying out three-dimensional reconstruction on the scene of the object to be detected, and calculating the coordinates of the point cloud after reconstruction;
and selecting a point cloud range, and measuring the size of the object to be measured according to the point cloud coordinates in the range.
2. The method for measuring the size of an object according to claim 1, wherein the updating and expanding of the point clouds of the images according to the posture change among the multiple frames of images and the fusion of the same point clouds among the multiple frames of images comprise
Acquiring the attitude of each frame of image at the acquisition moment, and estimating the relative position of the object to be detected at different moments;
tracking the point cloud position change of the image according to the relative position change of the object to be detected, and recording the number, the coordinates and the corresponding image characteristics of the point cloud;
and fusing the same point clouds among the multiple frames of images according to the numbers, the coordinates and the corresponding image characteristics.
3. The object dimension measuring method according to claim 2, wherein the algorithm for fusing the same point cloud among the plurality of frames of images includes a feature matching algorithm or a point cloud ICP algorithm.
4. The object dimension measuring method of claim 1, wherein three-dimensionally reconstructing a scene of the image and calculating coordinates of a point cloud after reconstruction comprises
Selecting at least two key frames, and reconstructing the scene of the object image to be detected according to the image and the point cloud of the key frames and the posture of the image acquisition moment;
and calculating the coordinates of the point cloud in the reconstructed new scene.
5. The method as claimed in claim 4, wherein the selecting the point cloud range to measure the object comprises
Selecting a range in the image;
projecting rays on the scene in the selected range to form a light cone;
and calculating the size of the object to be measured according to the point cloud coordinates in the light cone.
6. The method as claimed in claim 4, wherein the selecting a point cloud range for measuring the object to be measured further comprises
After the scene of the image is subjected to three-dimensional reconstruction, the point cloud is projected, and point cloud coordinates in a range are selected to calculate the size of the object to be measured.
7. The object dimension measuring method according to claim 5 or 6, characterized in that a three-dimensional solid mark is formed for the object to be measured at the time of measuring the dimension of the object to be measured.
8. An object dimension measuring apparatus, comprising:
the image acquisition module is used for acquiring multi-frame images of the object to be detected and the attitude and point cloud of each frame of the images;
the point cloud fusion module is used for updating and expanding the point cloud of the images according to the posture change among the multi-frame images and fusing the same point cloud among the multi-frame images;
the three-dimensional reconstruction module is used for performing three-dimensional reconstruction on the scene of the object to be detected and calculating the coordinates of the point cloud after reconstruction;
and the size calculation module is used for selecting a point cloud range and measuring the size of the object to be measured according to the point cloud coordinates in the range.
9. A computer device, characterized by comprising a memory in which a computer program is stored, a processor and a network interface, the processor implementing the steps of the object dimension measuring method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, carries out the steps of the object dimension measuring method according to any one of claims 1 to 7.
CN202011603309.1A 2020-12-29 2020-12-29 Object size measuring method, device, equipment and storage medium Active CN112733641B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011603309.1A CN112733641B (en) 2020-12-29 2020-12-29 Object size measuring method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011603309.1A CN112733641B (en) 2020-12-29 2020-12-29 Object size measuring method, device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN112733641A true CN112733641A (en) 2021-04-30
CN112733641B CN112733641B (en) 2024-07-05

Family

ID=75610579

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011603309.1A Active CN112733641B (en) 2020-12-29 2020-12-29 Object size measuring method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112733641B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114111568A (en) * 2021-09-30 2022-03-01 深圳市速腾聚创科技有限公司 Method and device for determining appearance size of dynamic target, medium and electronic equipment
CN114630096A (en) * 2022-01-05 2022-06-14 深圳技术大学 Method, device and equipment for densifying point cloud of TOF camera and readable storage medium
CN114898354A (en) * 2022-03-24 2022-08-12 中德（珠海）人工智能研究院有限公司 Measuring method and device based on three-dimensional model, server and readable storage medium

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150009214A1 (en) * 2013-07-08 2015-01-08 Vangogh Imaging, Inc. Real-time 3d computer vision processing engine for object recognition, reconstruction, and analysis
CN107504917A (en) * 2017-08-17 2017-12-22 深圳市异方科技有限公司 A kind of three-dimensional dimension measuring method and device
CN109003325A (en) * 2018-06-01 2018-12-14 网易（杭州）网络有限公司 A kind of method of three-dimensional reconstruction, medium, device and calculate equipment
JP2019074532A (en) * 2017-10-17 2019-05-16 有限会社ネットライズ Method for giving real dimensions to slam data and position measurement using the same
CN109916302A (en) * 2019-03-27 2019-06-21 青岛小鸟看看科技有限公司 A kind of volume measuring method and system of freight container
CN110276317A (en) * 2019-06-26 2019-09-24 Oppo广东移动通信有限公司 A kind of dimension of object detection method, dimension of object detection device and mobile terminal
CN111637850A (en) * 2020-05-29 2020-09-08 南京航空航天大学 Self-splicing surface point cloud measuring method without active visual marker
CN111709987A (en) * 2020-06-11 2020-09-25 上海东普信息科技有限公司 Package volume measuring method, device, equipment and storage medium
CN111915741A (en) * 2020-08-13 2020-11-10 广东申义实业投资有限公司 VR generater based on three-dimensional reconstruction

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150009214A1 (en) * 2013-07-08 2015-01-08 Vangogh Imaging, Inc. Real-time 3d computer vision processing engine for object recognition, reconstruction, and analysis
CN107504917A (en) * 2017-08-17 2017-12-22 深圳市异方科技有限公司 A kind of three-dimensional dimension measuring method and device
JP2019074532A (en) * 2017-10-17 2019-05-16 有限会社ネットライズ Method for giving real dimensions to slam data and position measurement using the same
CN109003325A (en) * 2018-06-01 2018-12-14 网易（杭州）网络有限公司 A kind of method of three-dimensional reconstruction, medium, device and calculate equipment
CN109916302A (en) * 2019-03-27 2019-06-21 青岛小鸟看看科技有限公司 A kind of volume measuring method and system of freight container
CN110276317A (en) * 2019-06-26 2019-09-24 Oppo广东移动通信有限公司 A kind of dimension of object detection method, dimension of object detection device and mobile terminal
CN111637850A (en) * 2020-05-29 2020-09-08 南京航空航天大学 Self-splicing surface point cloud measuring method without active visual marker
CN111709987A (en) * 2020-06-11 2020-09-25 上海东普信息科技有限公司 Package volume measuring method, device, equipment and storage medium
CN111915741A (en) * 2020-08-13 2020-11-10 广东申义实业投资有限公司 VR generater based on three-dimensional reconstruction

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114111568A (en) * 2021-09-30 2022-03-01 深圳市速腾聚创科技有限公司 Method and device for determining appearance size of dynamic target, medium and electronic equipment
CN114630096A (en) * 2022-01-05 2022-06-14 深圳技术大学 Method, device and equipment for densifying point cloud of TOF camera and readable storage medium
CN114630096B (en) * 2022-01-05 2023-10-27 深圳技术大学 Method, device and equipment for densification of TOF camera point cloud and readable storage medium
CN114898354A (en) * 2022-03-24 2022-08-12 中德（珠海）人工智能研究院有限公司 Measuring method and device based on three-dimensional model, server and readable storage medium

Also Published As

Publication number Publication date
CN112733641B (en) 2024-07-05

Similar Documents

Publication Publication Date Title
CN110335316B (en) Depth information-based pose determination method, device, medium and electronic equipment
CN111046744B (en) Method and device for detecting attention area, readable storage medium and terminal equipment
CN110322500B (en) Optimization method and device for instant positioning and map construction, medium and electronic equipment
CN110135455B (en) Image matching method, device and computer readable storage medium
CN109887003B (en) Method and equipment for carrying out three-dimensional tracking initialization
US9189853B1 (en) Automatic pose estimation from uncalibrated unordered spherical panoramas
CA2887763C (en) Systems and methods for relating images to each other by determining transforms without using image acquisition metadata
CN109683699B (en) Method and device for realizing augmented reality based on deep learning and mobile terminal
JP6057298B2 (en) Rapid 3D modeling
CN110111388B (en) Three-dimensional object pose parameter estimation method and visual equipment
US11842514B1 (en) Determining a pose of an object from rgb-d images
CN110986969B (en) Map fusion method and device, equipment and storage medium
CN111459269B (en) Augmented reality display method, system and computer readable storage medium
CN109191554B (en) Super-resolution image reconstruction method, device, terminal and storage medium
CN113689578B (en) Human body data set generation method and device
CN112733641B (en) Object size measuring method, device, equipment and storage medium
CN110109535A (en) Augmented reality generation method and device
CN110349212B (en) Optimization method and device for instant positioning and map construction, medium and electronic equipment
CN109613974B (en) AR home experience method in large scene
CN117132649A (en) Ship video positioning method and device for artificial intelligent Beidou satellite navigation fusion
Díaz et al. An algorithm to estimate building heights from Google street-view imagery using single view metrology across a representational state transfer system
Wientapper et al. Composing the feature map retrieval process for robust and ready-to-use monocular tracking
CN113673288B (en) Idle parking space detection method and device, computer equipment and storage medium
CN115994944A (en) Three-dimensional key point prediction method, training method and related equipment
CN112085842B (en) Depth value determining method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant