WO2017143745A1

WO2017143745A1 - Method and apparatus for determining movement information of to-be-detected object

Info

Publication number: WO2017143745A1
Application number: PCT/CN2016/096379
Authority: WO
Inventors: 陆真国; 王金亮
Original assignee: 上海乐相科技有限公司
Priority date: 2016-02-22
Filing date: 2016-08-23
Publication date: 2017-08-31
Also published as: CN105631901A

Abstract

Disclosed are a method and an apparatus for determining movement information of a to-be-detected object. In embodiments of the present invention, the N^th frame of image of a to-be-detected object acquired by a photography device is obtained; corresponding relations between mark point phantoms and physical mark points are determined according to mark point phantoms in the N^th frame of image; and movement information of the to-be-detected object at a moment corresponding to the N^th frame of image is determined according to position information of the physical mark points and position information of the mark point phantoms. In the embodiments of the present invention, corresponding relations between mark point phantoms and physical mark points are determined, and movement information of a to-be-detected object is determined according to position information of the mark point phantoms and position information of the physical mark points. Compared with the method in the prior art in which a rotation attitude is obtained using a sensor such as a gyroscope, the embodiments of the present invention can effectively determine the translation of the to-be-detected object, so as to more accurately and quickly sense the movement state of the to-be-detected object, thereby achieving high instantaneity and significantly improving actual experience of a user.

Description

Method and device for determining motion information of object to be tested

This application claims priority to the Chinese Patent Application of the People's Republic of China Intellectual Property Office, Application No. 201610096765.9, entitled "A Method and Apparatus for Determining Motion Information of Objects to Be Tested", on February 22, 2016, The entire contents are incorporated herein by reference.

Technical field

The present invention relates to the field of virtual reality technologies, and in particular, to a method and apparatus for determining motion information of an object to be tested.

Background technique

A virtual reality helmet refers to a helmet that guides a user to create a feeling in a virtual environment by using a helmet display to close a person's visual and auditory sense to the outside world. With the continuous development of electronic technology, virtual reality helmets have allowed users to control virtual images according to their viewpoints and locations in the virtual environment through a variety of advanced sensing methods, specifically, in the process of users using virtual reality helmets. By presenting the motion state of the user's head, the user is presented with different scenes. An important experience of the virtual reality helmet is immersion. Therefore, whether the motion state of the user's head can be accurately and quickly perceived is an important indicator that affects the performance of the virtual reality helmet.

At present, because ordinary accelerometers cannot accurately obtain spatial translation vectors, most virtual reality helmets can only obtain rotational attitudes through sensors such as gyroscopes. However, in this way, in the process of users using virtual reality helmets, there is a lack of user heads. The perception of the translational motion (that is, the translational motion of the virtual reality helmet) leads to a deviation in the perception of the motion state of the virtual reality helmet, which greatly affects the actual experience of the user.

In summary, there is a need for a method for accurately and quickly sensing the state of motion of a virtual reality helmet.

Summary of the invention

Embodiments of the present invention provide a method and apparatus for determining motion information of an object to be tested, so as to achieve accurate and rapid perception of a virtual reality helmet motion state.

A method for determining motion information of an object to be tested according to an embodiment of the present invention includes:

Obtaining an Nth frame image of the object to be tested collected by the camera device, where the image of the Nth frame includes a point image of each physical marker point on the first side of the object to be tested;

Determining, according to the point image in the image of the Nth frame, a correspondence between the point image and the physical point;

Acquiring position information of each physical mark point of the first side of the object to be tested in a preset world coordinate system and position information of each point image of the Nth frame image in a preset image coordinate system;

Determining, according to the correspondence between the mark point image and the physical mark point, and the position information of each of the physical mark points and the respective mark point images, determining, at the time corresponding to the image of the Nth frame, the object to be tested Sports information.

Preferably, the determining, according to the point image in the image of the Nth frame, the correspondence between the point image and the physical point, including:

Determining, by an envelope method, a reference mark point image in the image of the Nth frame; the reference mark point image is one of the mark point images in the image of the Nth frame;

Determining, according to a positional relationship between the point image in the image of the Nth frame and the image of the reference point image, a number of each point image in the image of the Nth frame;

Determining a physical marker point corresponding to the number of the marker point image as a physical marker point corresponding to the marker point image, and obtaining a correspondence relationship between the marker point image and the physical marker point; The number of the mark point image is obtained based on the same numbering rule; the physical mark points of the first side of the object to be tested are distributed in a convex polygon array.

Preferably, determining, according to the positional relationship between the point image and the reference point image in the image of the Nth frame, the number of each point image in the image of the Nth frame, including:

Determining, according to a positional relationship between the mark point image and the reference mark point image in the image of the Nth frame, a mark point image of the first layer and an order of the mark point image of the first layer; the reference mark point image a mark point image of the first layer;

And other than the mark image of the first layer to the M-1th layer in the image of the Nth frame Determining a positional relationship between the marker point image and the reference marker point image, determining an image of the marker point image of the Mth layer and the marker point image of the Mth layer; M is an integer greater than or equal to 2;

Determining the number of the point image of the first layer to the Mth layer according to the order of the image of the mark points of the first layer to the Mth layer.

Preferably, after acquiring the image of the Nth frame of the object to be tested collected by the camera, determining the correspondence between the point image and the physical point according to the point image in the image of the Nth frame Previously, it also included:

The marker point image is determined based on at least a pixel value of each pixel of the Nth frame image, a number of pixel points on each contour, and a number of pixel points included in the contour.

Preferably, the image of the marker point is determined according to at least a pixel value of each pixel of the image of the Nth frame, a number of pixels on each contour, and a number of pixels included in the contour, including:

Obtaining, according to the pixel value of each pixel of the image of the Nth frame, a first candidate point image; the pixel value of the pixel of the first candidate point image is greater than or equal to a first threshold;

Obtaining a second candidate point image according to the number of pixels on the contour of each of the first candidate point images; the number of pixels on the contour of the second candidate point image is greater than or equal to a second threshold and less than or equal to a third threshold;

Obtaining, according to the number of pixels included in the contour of each of the second candidate point images, a third candidate point image; the number of pixels included in the contour of the third candidate point image Greater than or equal to the fourth threshold;

Determining an ellipse parameter of the third candidate point image, and determining a third candidate point image that matches the ellipse parameter to the preset parameter range as the point image.

Preferably, the object to be tested is determined in the Nth frame according to the correspondence between the point image and the physical point, and the position information of each physical point and the image of each point The motion information of the image corresponding to the moment, including:

Determining the camera device by using a PnP algorithm according to the correspondence between the point image and the physical point, and the position information of each of the physical point and the image of each point The amount of rotation and the amount of translation for the object to be tested;

Obtaining motion information of the object to be tested at a time corresponding to the image of the Nth frame according to a rotation amount and a translation amount of the imaging device with respect to the object to be tested; and the image of the object to be tested is in the image of the Nth frame The motion information corresponding to the time is the amount of rotation and the amount of translation of the object to be tested relative to the camera at the time corresponding to the image of the Nth frame.

Preferably, after determining the rotation amount and the translation amount of the imaging device relative to the object to be tested by using the PnP algorithm, determining the motion information of the object to be tested at the time corresponding to the image of the Nth frame, Also includes:

The amount of rotation and the amount of translation of the camera device relative to the object to be tested are optimized using an LM algorithm.

Preferably, the physical mark point is an infrared point; the mark point image is an infrared point image;

After determining the motion information of the object to be tested at the time corresponding to the image of the Nth frame, the method further includes:

Determining, when the motion information of the object to be tested is in the range of the preset motion amount, the infrared point of the first side of the object to be tested is turned off, and the second side of the object to be tested is opened. An infrared point; the second side is predicted according to motion information of the object to be tested at a time corresponding to the image of the Nth frame;

Acquiring an image of the (N+1)th frame acquired by the camera device;

Determining, by the N+1 frame image, an infrared point image of each infrared point of the second side of the object to be tested, and if yes, determining, according to the (N+1)th frame image, the object to be tested is The N+1 frame image corresponds to the motion information of the time; if not, the infrared point of the second side of the object to be tested is turned off, the infrared point of the third side of the object to be tested is turned on, and the image capturing device is acquired. The N+2th frame image; the third side is obtained according to a preset cyclic sequence.

An embodiment of the present invention provides an apparatus for determining motion information of an object to be tested, including:

a first acquiring module, configured to acquire an image of an Nth frame of the object to be tested collected by the camera, where the image of the Nth frame includes a point image of each physical point of the first side of the object to be tested;

a determining module, configured to determine the mark point image according to the mark point image in the image of the Nth frame Corresponding relationship with the physical marker points;

a second acquiring module, configured to acquire position information of each physical point of the first side of the object to be tested in a preset world coordinate system, and image points of the image of the Nth frame in preset image coordinates Location information in the system;

a processing module, configured to determine, according to a correspondence between the mark point image and the physical mark point, and location information of each of the physical mark points and the image of each mark point, the object to be tested is at the Nth The frame image corresponds to the motion information at the time.

Preferably, the determining module is specifically configured to:

Determining a mark point image of the Mth layer according to a positional relationship between the mark point image other than the mark point image of the first layer to the M-1th layer and the reference mark point image in the image of the Nth frame Sorting of the image of the mark point of the Mth layer; M is an integer greater than or equal to 2;

Preferably, the determining module is further configured to:

At least a pixel value of each pixel of the image of the Nth frame, a pixel point of each contour The number of pixels and the number of pixels included in the contour determine the image of the marker point.

Preferably, the determining module is specifically configured to:

Preferably, the processing module is specifically configured to:

Determining, by the PnP algorithm, the camera device relative to the object to be tested, according to the correspondence between the point image and the physical point, and the position information of each of the physical point and the image of each point The amount of rotation and the amount of translation;

Preferably, the processing module is further configured to:

The processing module is further configured to:

Determining, in the case that the motion information of the object to be tested is in the range of the preset motion amount, the infrared point of the first side of the object to be tested is turned off, and the object to be tested is turned on. An infrared point of the second side; the second side is predicted according to motion information of the object to be tested at a time corresponding to the image of the Nth frame;

Acquiring an image of the (N+1)th frame acquired by the camera device;

In the above embodiment of the present invention, the image of the Nth frame of the object to be tested collected by the camera device is acquired, and the image of the Nth frame includes the image of the point of each physical mark on the first side of the object to be tested; The mark point image determines the correspondence relationship between the mark point image and the physical mark point; acquires the position information of each physical mark point of the first side of the object to be tested in the preset world coordinate system and the image of each mark point of the Nth frame image The position information in the preset image coordinate system; determining the object to be tested at the corresponding moment of the image of the Nth frame according to the correspondence between the point image and the physical point, and the position information of each physical point and each point image Sports information. In the embodiment of the present invention, by determining the correspondence between the point image and the physical point, and based on the position information of the point image and the position information of the physical point, the motion information of the object to be tested is determined, which is compared with the prior art. The method for obtaining a rotational posture by using a sensor such as a gyroscope can effectively determine the amount of translation of the object to be tested, thereby more accurately and quickly sensing the motion state of the object to be tested, and has high real-time performance, which can significantly improve the actual user. Experience.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention, Those skilled in the art can also obtain other drawings based on these drawings without paying for inventive labor.

1 is a schematic structural diagram of a system applicable to an embodiment of the present invention;

2 is a schematic view showing the arrangement of infrared lamps on each side of the virtual reality helmet;

FIG. 3 is a schematic flowchart of determining motion information of an object to be tested according to an embodiment of the present disclosure;

4 is a schematic flowchart of preprocessing an image according to an embodiment of the present invention;

FIG. 5 is a schematic flowchart of determining a correspondence between a mark point image and a physical mark point according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a marker point image number according to an embodiment of the present invention; FIG.

7 is a schematic diagram of a process of performing prediction processing according to motion information of an object to be tested;

FIG. 8 is a schematic structural diagram of an apparatus for determining motion information of an object to be tested according to an embodiment of the present disclosure;

FIG. 9 is a schematic structural diagram of an apparatus for determining motion information of an object to be tested according to an embodiment of the present invention.

detailed description

The present invention will be further described in detail with reference to the accompanying drawings, in which FIG. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

The method for determining the motion information of the object to be tested provided in the embodiment of the present invention is applicable to various application scenarios. FIG. 1 exemplarily shows a system architecture diagram applicable to the embodiment of the present invention.

As shown in FIG. 1, the system architecture includes a server 101, an imaging device 102, and an object to be tested 103. The server 101 and the camera 102 can perform wired or wireless communication, that is, the information can be transmitted between the server 101 and the camera 102 by wired transmission or wireless transmission. For example, the camera 102 can be transmitted by wire or wirelessly. The method sends the captured image to the server 101; the server 101 and the object to be tested 103 can also perform wired or wireless communication. For example, the server 101 sends the scene rendering data to the object to be tested 103 by wired transmission or wireless transmission. .

In the embodiment of the present invention, the server 101 may be a PC host with data processing capability.

The object to be tested 103 may be a virtual reality helmet including first to sixth sides (assuming that the normal use condition of the virtual reality helmet is worn on the user's head, based on which the first to sixth helmets may be determined The side faces are front, back, top, bottom, left, and right with respect to the camera. Infrared lights (also called infrared dots) are arranged on each side, wherein the infrared lamps on each side are arranged according to a preset layout rule. Specifically, in order to facilitate the subsequent operation process, the routing rule in the embodiment of the present invention may be that the infrared lamps on each side are arranged in a manner of a convex polygon array, and may be based on the above-mentioned layout rules for different sides. Next, consider different layouts, for example, the left and right sides, the upper and lower sides can be the same layout. As shown in FIG. 2, it is a schematic diagram of the arrangement of infrared lamps on each side of the virtual reality helmet. It should be noted that FIG. 2 is only an exemplary representation of the arrangement of the infrared lamps, and there may be a difference in the proportion of the specific objects.

The camera device 102 can be an infrared camera, which is mainly used for capturing state information of the infrared lamp disposed on the virtual reality helmet, and transmits the captured image to the server 101, so that the server determines the motion information of the virtual reality helmet through correlation calculation (rotation) Matrix R and translation vector T).

Specifically, in the embodiment of the present invention, the virtual reality helmet and the infrared camera are connected to the PC host through a USB data line, and both USB2.0 and USB3.0, preferably USB3.0. The virtual reality helmet is also connected to the PC host through the HDMI interface to facilitate capturing scene rendering data.

Based on the system architecture shown in FIG. 1 , FIG. 3 is a schematic flowchart of determining motion information of an object to be tested according to an embodiment of the present invention.

Step 301: Acquire an image of an Nth frame of the object to be tested collected by the camera, where the image of the Nth frame includes a point image of each physical mark on the first side of the object to be tested;

Step 302: Determine, according to the point image in the image of the Nth frame, a correspondence between the point image and the physical point;

Step 303: Acquire location information of each physical marker point of the first side of the object to be tested in a preset world coordinate system, and position of each marker image of the Nth frame image in a preset image coordinate system. information;

Step 304, according to the correspondence between the mark point image and the physical mark point, and the And determining, by the physical location points and the location information of the respective marker point images, motion information of the object to be tested at a time corresponding to the image of the Nth frame.

In the embodiment of the present invention, by determining the correspondence between the point image and the physical point, and based on the position information of the point image and the position information of the physical point, the motion information of the object to be tested is determined, which is compared with the prior art. The method for obtaining a rotational posture by using a sensor such as a gyroscope can effectively determine the amount of translation of the object to be tested, thereby more accurately and quickly sensing the motion state of the object to be tested, and has high real-time performance, which can significantly improve the actual user. Experience.

The physical marker point in the embodiment of the present invention may be an infrared point, and the marker point image is an infrared point image.

In the embodiment of the present invention, in the optical tracking process of the motion information (space shift amount) of the virtual reality helmet, only the side of the six sides of the helmet that can be completely photographed by the camera is performed on the upper, the lower, the left, the right, the front and the back. deal with. Therefore, the following is a detailed description of the process of determining the motion information of the virtual reality helmet by a single side.

In the embodiment of the present invention, since the image capturing device may interfere with the ambient light and other factors when capturing the image, after acquiring the image of the Nth frame of the object to be tested collected by the camera device, the image of the Nth frame should be preprocessed. And determining, according to at least the pixel value of each pixel of the image of the Nth frame, the number of pixels on each contour, and the number of pixels included in the contour, the interference image is excluded, and the marker image is determined.

Specifically, the first candidate point image is obtained according to the pixel value of the pixel of each image; wherein the pixel value of the pixel of the first candidate point image is greater than or equal to the first threshold; according to each first candidate mark Obtaining a second candidate point image by the number of pixels on the contour of the image; the number of pixels on the contour of the second candidate point image is greater than or equal to a second threshold and less than or equal to a third threshold; The number of pixels included in the contour of each second candidate point image obtains a third candidate point image; the number of pixels included in the contour of the third candidate point image is greater than or equal to a fourth threshold Determining an ellipse parameter of the third candidate point image, and determining a third candidate point image that matches the ellipse parameter to the preset parameter range as the point image. The first threshold, the second threshold, the third threshold, the fourth threshold, and the preset parameter range may all be set by experience by those skilled in the art.

FIG. 4 is a schematic flowchart of pre-processing an image according to an embodiment of the present invention, including steps 401 to 408, which are specifically described below in conjunction with FIG. 4.

Step 401: Acquire an image of an Nth frame.

Step 402: Binarization processing, to obtain a first candidate point image; specifically: determining a maximum pixel value max of each pixel in the image of the Nth frame, using a*max as a binarization threshold (first threshold) Each pixel is traversed. If the pixel value of the pixel is less than a*max, the pixel value is set to 0. If the pixel value of the pixel is greater than or equal to a*max, the pixel value is set to 255. Where a is a weight, the value of a can be set by a person skilled in the art according to experience, for example, can be set to 0.9;

Step 403: Obtain a number of pixel points on a contour of each first candidate point image, where the contour is a position of a pixel point whose pixel value changes from 0 to 255 or changes from 255 to 0. Specifically, In the process of determining each contour, 8 neighborhood pixels can be obtained (except for the edge region of the image, each pixel is adjacent to 8 pixels, and the first pixel on the contour is determined. A method of traversing 8 pixel points adjacent to the pixel to quickly acquire a second pixel on the contour, and so on, so as to more quickly determine the number of pixels on the contour;

Step 404: For each first candidate point image, delete an image whose number of pixels on the contour is smaller than a second threshold or greater than a third threshold, to obtain a second candidate point image;

Step 405: Delete, for each second candidate point image, an image whose number of pixels included in the contour is less than a fourth threshold, to obtain a third candidate point image;

Step 406, based on a preset image coordinate system (including an x-axis and a y-axis), fitting an ellipse parameter (including an ellipse center, a long and short axis, a tilt angle, and the like) of the third candidate point image by a fitting algorithm;

Step 407: For each third candidate point image, delete an image whose ellipse parameter does not meet the preset parameter range, and determine a third candidate point image whose ellipse parameter meets the preset parameter range as the point image.

Step 408: Output an ellipse parameter of the obtained point image, and determine position information of the point according to the ellipse parameter.

In the embodiment of the present invention, the image of the Nth frame is preprocessed by the foregoing process, thereby quickly and accurately eliminating interference factors in the environment, determining the image of the marker point, and determining the image and label of the marker point for subsequent determination. The correspondence between the points has laid a good foundation.

Since the arrangement of the infrared lamps on each side of the virtual reality helmet is not completely consistent, in the embodiment of the present invention, only one side (the first side, ie, the front side) is processed, and the other sides are similar. In step 302, the correspondence between the point image in the image of the Nth frame and the physical point of the first side is determined by determining an image of the reference point in the image of the Nth frame based on the envelope method; The reference mark point image is one of the mark point images in the Nth frame image; the number of the mark point image in the Nth frame image is determined according to the positional relationship between the mark point image and the reference mark point image in the Nth frame image; The physical marker point with the same number as the marker point image is determined as the physical marker point corresponding to the marker point image, and the correspondence relationship between the marker point image and the physical marker point is obtained; the number of the physical marker point and the number of the marker point image are based on the same Numbering rules are obtained.

Further, in the embodiment of the present invention, when determining the number of each point image in the image of the Nth frame according to the positional relationship between the point image in the image of the Nth frame and the image of the reference point image, Based on the envelope method, layer by layer, from the outermost layer to the innermost layer, specifically: determining the marking point of the first layer according to the positional relationship between the marked point image and the reference point image in the image of the Nth frame The image and the sorting of the image of the first layer of the marker point; the reference marker image is the marker image of the first layer; and the marker points other than the marker image of the first layer to the M-1 layer according to the image of the Nth frame The positional relationship between the image and the reference point image determines the order of the image of the mark of the Mth layer and the image of the mark of the Mth layer; M is an integer greater than or equal to 2; according to the image of the mark of the first layer to the Mth layer Sort to determine the number of the marker image of the first layer to the third layer.

The specific value of M can be set by the person skilled in the art according to the experience and the arrangement of the infrared light on the side. In general, the value of M can be set to 3.

FIG. 5 is a schematic flowchart of determining a correspondence between a point image and a physical point according to an embodiment of the present invention, which includes steps 501 to 504, which are specifically described below with reference to FIG. 5.

Step 501: Determine, according to the convex hull algorithm, the reference mark point image and the mark image of the first layer (the outermost layer), and according to the distance between each mark point image of the first layer and the reference mark point, according to the small to The order of the reference point images is specifically as follows: the point images of the image of the Nth frame are arranged in ascending order according to the y-axis coordinate, and if the same y-axis coordinate corresponds to multiple Marking the point image, the plurality of point images are arranged in ascending order according to the x-axis coordinate, and the point image with the largest y-axis coordinate and the smallest x-axis coordinate (ie, the point image in the lower left corner) is determined as the reference point image;

Step 502, determining four image points of the second layer (secondary outer layer), according to the distance between the four point image and the reference point image, arranged in order from small to large;

Step 503, determining three image points of the third layer (the innermost layer), according to the distance between the three point image and the reference point image, arranged in order from small to large;

Step 504, according to the first layer to the third layer obtained above, sequentially numbered in order from outer to inner, and the first layer number is 1-8, the second layer number is 9-12, and the third layer number is 13-15, as shown in FIG. 6, is a schematic diagram of the number of points of the marker point.

Since the first side physical mark point is obtained based on the same numbering rule as described above, if the number of the mark point image determined by the above method is normal, the physical mark point having the same number as the mark point image can be determined to correspond. Physical point points, the correspondence between the point image and the physical point is obtained.

In the embodiment of the present invention, the convex bundle algorithm is used to determine the correspondence between the point image and the physical marker point, so that the determination of the correspondence relationship is more accurate and fast, and lays a good foundation for subsequently determining the motion information of the object to be tested.

In step 304, based on the correspondence between the point image and the physical point obtained in step 302, and the position information of each physical point and each point image obtained in step 303, the PnP algorithm is used to determine the camera relative to the test. The amount of rotation and the amount of translation of the object, and the LM algorithm is used to optimize the amount of rotation and the amount of translation of the camera relative to the object to be tested; according to the amount of rotation and the amount of translation of the camera device relative to the object to be measured, The motion information of the object to be tested at the time corresponding to the image of the Nth frame; the motion information of the object to be tested at the time corresponding to the image of the Nth frame is the amount of rotation and the amount of translation of the object to be tested relative to the camera at the corresponding time of the image of the Nth frame.

FIG. 7 is a schematic diagram of a process of performing prediction processing according to motion information of an object to be tested. The embodiment of the present invention further determines the prediction processing procedure shown in FIG. 7 after the object to be measured is rotated and translated by the camera at the corresponding time of the image of the Nth frame, specifically:

Step 701: Determine, by using a PnP algorithm and an LM algorithm, a rotation amount and a translation amount of the object to be tested relative to the imaging device at a corresponding time of the image of the Nth frame;

Step 702: determining that the motion information of the object to be tested at the time corresponding to the image of the Nth frame conforms to the preset motion amount range, specifically: whether the rotation amount is within a preset rotation amount range, and whether the translation amount is within a preset translation amount range; if yes, Then, step 703 is performed; if not, step 701 is performed; wherein the preset rotation amount range and the preset translation amount range can be obtained by a person skilled in the art according to experience or according to a large number of experiments;

Step 703: predict a second side according to the motion information of the object to be tested at the corresponding time of the image of the Nth frame, close the infrared point of the first side of the object to be tested, and open the infrared point of the second side of the object to be tested;

Step 704: Acquire an image of an N+1th frame acquired by the camera.

Step 705, determining whether the infrared point image of each infrared point of the second side of the object to be tested is included in the image of the N+1th frame, and if yes, executing step 701, determining, according to the image of the (N+1)th frame Determining the motion information of the object to be measured at the time corresponding to the (N+1)th frame image; if not, executing step 706;

Step 706, the infrared point of the second side of the object to be tested is turned off, the infrared point of the third side of the object to be tested is turned on, and the image of the N+2 frame acquired by the camera device is acquired; wherein the third side is It is obtained according to the preset cycle order. For example, the preset cycle order can be front, left, right, top, bottom, and back.

Step 707: Determine whether the infrared point image of each infrared point of the third side of the object to be tested is included in the image of the N+2 frame, and if yes, execute step 701, and determine the image according to the image of the N+2 frame. The motion information of the object to be tested at the time corresponding to the (N+1)th frame image; if not, step 706 is executed to cycle the infrared point of the next side according to a preset cycle sequence.

For the above method flow, the embodiment of the present invention further provides an apparatus for determining motion information of an object to be tested, and the specific content of the apparatus may be implemented by referring to the foregoing method.

FIG. 8 is a schematic structural diagram of an apparatus for determining motion information of an object to be tested according to an embodiment of the present invention.

The first obtaining module 801 is configured to acquire an image of the Nth frame of the object to be tested collected by the camera, The image of the Nth frame includes a mark point image of each physical mark point of the first side of the object to be tested;

a determining module 802, configured to determine, according to the point image in the image of the Nth frame, a correspondence between the point image and the physical point;

a second acquiring module 803, configured to acquire position information of each physical mark point of the first side of the object to be tested in a preset world coordinate system, and image of each point of the Nth frame image in a preset image Position information in the coordinate system;

The processing module 804 is configured to determine, according to the correspondence between the mark point image and the physical mark point, and the position information of each physical mark point and the image of each mark point, the object to be tested is in the first The N frame image corresponds to the motion information of the time.

Preferably, the determining module 802 is specifically configured to:

Determining the first layer to the Mth according to the ordering of the image points of the first layer to the Mth layer The number of the point image of the layer.

Preferably, the determining module 802 is further configured to:

Preferably, the determining module 802 is specifically configured to:

Preferably, the processing module 804 is specifically configured to:

Preferably, the processing module 804 is further configured to:

The processing module 804 is further configured to:

Acquiring an image of the (N+1)th frame acquired by the camera device;

As can be seen from the above:

In the embodiment of the present invention, the image of the Nth frame of the object to be tested collected by the camera device is acquired, and the image of the Nth frame includes the image of the point of each physical mark on the first side of the object to be tested; Marking the point image to determine the correspondence between the point image and the physical point; obtaining the position information of each physical point of the first side of the object to be tested in a preset world coordinate system and the image of each point of the Nth frame image Position information in a preset image coordinate system; determining the motion of the object to be tested at the corresponding moment of the image of the Nth frame according to the correspondence between the point image and the physical point, and the position information of each physical point and each point image information. In the embodiment of the present invention, by determining the correspondence between the point image and the physical point, and based on the position information of the point image and the position information of the physical point, the motion information of the object to be tested is determined, which is compared with the prior art. The method for obtaining a rotational posture by using a sensor such as a gyroscope can effectively determine the amount of translation of the object to be tested, thereby more accurately and quickly sensing the motion state of the object to be tested, and has high real-time performance, which can significantly improve the actual user. Experience.

Based on the same technical concept, the embodiment of the present application provides another apparatus for determining motion information of an object to be tested. The device for determining the motion information of the object to be tested provided by the embodiment of the present application is as shown in FIG. 9. The device for determining the motion information of the object to be tested includes: a communication interface 901, a processor 902, and a memory. 903 and bus system 904;

The memory 903 is used to store a program. In particular, the program can include program code, the program code including computer operating instructions. The memory 903 may be a random access memory (RAM) or a non-volatile memory such as at least one disk storage. Only one memory is shown in the figure, of course, the memory can also be set to a plurality as needed. Memory 903 can also be a memory in processor 902.

The memory 903 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof:

Operation instructions: include various operation instructions for implementing various operations.

Operating system: Includes a variety of system programs for implementing various basic services and handling hardware-based tasks.

The processor 902 controls the operation of the device that determines the motion information of the object to be tested, and the processor 902 may also be referred to as a CPU (Central Processing Unit). In a specific application, the components of the device for determining the motion information of the object to be tested are coupled together by a bus system 904. The bus system 904 may include a power bus, a control bus, a status signal bus, and the like in addition to the data bus. However, for clarity of description, various buses are labeled as bus system 904 in the figure. For ease of representation, only the schematic drawing is shown in FIG.

The method disclosed in the foregoing embodiment of the present application may be applied to the processor 902 or implemented by the processor 902. Processor 902 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 902 or an instruction in a form of software. The processor 902 described above may be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or discrete hardware. Component. The methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general purpose processor may be a microprocessor or the processor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly implemented by the hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor. The software module can be located in a random access memory. Flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers, etc. are well-known storage media in the field. The storage medium is located in the memory 903, and the processor 902 reads the information in the memory 903 and performs the following steps in conjunction with its hardware:

Obtaining an Nth frame image of the object to be tested collected by the camera device, where the Nth frame image includes a mark point image of each physical mark point of the first side of the object to be tested; according to the mark in the image of the Nth frame a point image, determining a correspondence between the point image and the physical point; acquiring position information of each physical point of the first side of the object to be tested in a preset world coordinate system, and the Nth frame Position information of each point image of the image in a preset image coordinate system; according to the correspondence between the point image and the physical point, and the position of each physical point and the image of each point And determining information about the motion of the object to be tested at the time corresponding to the image of the Nth frame.

Preferably, the processor 902 is specifically configured to:

Preferably, the processor 902 is further configured to:

Preferably, the processor 902 is specifically configured to:

Preferably, the processor 902 is further configured to:

The processor 902 is further configured to:

Acquiring an image of the (N+1)th frame acquired by the camera device;

As can be seen from the above:

Those skilled in the art will appreciate that embodiments of the present invention may be provided as a method, or a computer Program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

While the preferred embodiment of the invention has been described, it will be understood that Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and the modifications and

It is apparent that those skilled in the art can make various modifications and variations to the invention without departing from the spirit and scope of the invention. Thus, it is intended that the present invention cover the modifications and modifications of the invention

Claims

A method for determining motion information of an object to be tested, characterized in that it comprises:

Obtaining an Nth frame image of the object to be tested collected by the camera device, where the image of the Nth frame includes a point image of each physical marker point on the first side of the object to be tested;

Determining, according to the point image in the image of the Nth frame, a correspondence between the point image and the physical point;

Acquiring position information of each physical mark point of the first side of the object to be tested in a preset world coordinate system and position information of each point image of the Nth frame image in a preset image coordinate system;

Determining, according to the correspondence between the mark point image and the physical mark point, and the position information of each of the physical mark points and the respective mark point images, determining, at the time corresponding to the image of the Nth frame, the object to be tested Sports information.
The method according to claim 1, wherein the determining the correspondence between the point image and the physical point according to the point image in the image of the Nth frame comprises:

Determining, by an envelope method, a reference mark point image in the image of the Nth frame; the reference mark point image is one of the mark point images in the image of the Nth frame;

Determining, according to a positional relationship between the point image in the image of the Nth frame and the image of the reference point image, a number of each point image in the image of the Nth frame;

Determining a physical marker point corresponding to the number of the marker point image as a physical marker point corresponding to the marker point image, and obtaining a correspondence relationship between the marker point image and the physical marker point; The number of the mark point image is obtained based on the same numbering rule; the physical mark points of the first side of the object to be tested are distributed in a convex polygon array.
The method according to claim 2, wherein determining the image of each of the points in the image of the Nth frame based on the positional relationship between the image of the point in the image of the Nth frame and the image of the reference point image Number, including:

According to the positional relationship between the point image in the image of the Nth frame and the image of the reference point point, Determining a point image of the first layer and an order of the image of the point of the first layer; the reference point image is a point image of the first layer;

Determining a mark point image of the Mth layer according to a positional relationship between the mark point image other than the mark point image of the first layer to the M-1th layer and the reference mark point image in the image of the Nth frame Sorting of the image of the mark point of the Mth layer; M is an integer greater than or equal to 2;

Determining the number of the point image of the first layer to the Mth layer according to the order of the image of the mark points of the first layer to the Mth layer.
The method according to claim 1, wherein after obtaining the image of the Nth frame of the object to be tested acquired by the camera device, determining the image of the marker point based on the image of the marker point in the image of the Nth frame Before the correspondence with the physical mark points, the method further includes:

The marker point image is determined based on at least a pixel value of each pixel of the Nth frame image, a number of pixel points on each contour, and a number of pixel points included in the contour.
The method according to claim 4, wherein at least the pixel value of each pixel of the Nth frame image, the number of pixel points on each contour, and the number of pixel points included in the contour are determined. Out of the marker image, including:

Obtaining, according to the pixel value of each pixel of the image of the Nth frame, a first candidate point image; the pixel value of the pixel of the first candidate point image is greater than or equal to a first threshold;

Obtaining a second candidate point image according to the number of pixels on the contour of each of the first candidate point images; the number of pixels on the contour of the second candidate point image is greater than or equal to a second threshold and less than or equal to a third threshold;

Obtaining, according to the number of pixels included in the contour of each of the second candidate point images, a third candidate point image; the number of pixels included in the contour of the third candidate point image Greater than or equal to the fourth threshold;

Determining an ellipse parameter of the third candidate point image, and determining a third candidate point image that matches the ellipse parameter to the preset parameter range as the point image.
The method according to claim 1, wherein: according to the correspondence between the point image and the physical point, and the respective physical point and the image of each point Position information, determining motion information of the object to be tested at a time corresponding to the image of the Nth frame, including:

Determining, by the PnP algorithm, the camera device relative to the object to be tested, according to the correspondence between the point image and the physical point, and the position information of each of the physical point and the image of each point The amount of rotation and the amount of translation;

Obtaining motion information of the object to be tested at a time corresponding to the image of the Nth frame according to a rotation amount and a translation amount of the imaging device with respect to the object to be tested; and the image of the object to be tested is in the image of the Nth frame The motion information corresponding to the time is the amount of rotation and the amount of translation of the object to be tested relative to the camera at the time corresponding to the image of the Nth frame.
The method according to claim 6, wherein after determining the amount of rotation and the amount of translation of the camera device relative to the object to be tested by using a PnP algorithm, determining that the object to be tested is at the Nth Before the frame image corresponds to the motion information of the moment, it also includes:

The amount of rotation and the amount of translation of the camera device relative to the object to be tested are optimized using an LM algorithm.
The method according to claim 1, wherein the physical marker point is an infrared point; and the marker point image is an infrared point image;

After determining the motion information of the object to be tested at the time corresponding to the image of the Nth frame, the method further includes:

Determining, when the motion information of the object to be tested is in the range of the preset motion amount, the infrared point of the first side of the object to be tested is turned off, and the second side of the object to be tested is opened. An infrared point; the second side is predicted according to motion information of the object to be tested at a time corresponding to the image of the Nth frame;

Acquiring an image of the (N+1)th frame acquired by the camera device;

Determining, by the N+1 frame image, an infrared point image of each infrared point of the second side of the object to be tested, and if yes, determining, according to the (N+1)th frame image, the object to be tested is The N+1 frame image corresponds to the motion information of the time; if not, the infrared point of the second side of the object to be tested is turned off, the infrared point of the third side of the object to be tested is turned on, and the image capturing device is acquired. The N+2th frame image; the third side is obtained according to a preset cyclic sequence.
An apparatus for determining motion information of an object to be tested, comprising:

a first acquiring module, configured to acquire an image of an Nth frame of the object to be tested collected by the camera, where the image of the Nth frame includes a point image of each physical point of the first side of the object to be tested;

a determining module, configured to determine, according to the point image in the image of the Nth frame, a correspondence between the point image and the physical point;

a second acquiring module, configured to acquire position information of each physical point of the first side of the object to be tested in a preset world coordinate system, and image points of the image of the Nth frame in preset image coordinates Location information in the system;

a processing module, configured to determine, according to a correspondence between the mark point image and the physical mark point, and location information of each of the physical mark points and the image of each mark point, the object to be tested is at the Nth The frame image corresponds to the motion information at the time.
The device according to claim 9, wherein the determining module is specifically configured to:

Determining, by an envelope method, a reference mark point image in the image of the Nth frame; the reference mark point image is one of the mark point images in the image of the Nth frame;

Determining, according to a positional relationship between the point image in the image of the Nth frame and the image of the reference point image, a number of each point image in the image of the Nth frame;

Determining a physical marker point corresponding to the number of the marker point image as a physical marker point corresponding to the marker point image, and obtaining a correspondence relationship between the marker point image and the physical marker point; The number of the mark point image is obtained based on the same numbering rule; the physical mark points of the first side of the object to be tested are distributed in a convex polygon array.
The device according to claim 10, wherein the determining module is specifically configured to:

Determining, according to a positional relationship between the mark point image and the reference mark point image in the image of the Nth frame, a mark point image of the first layer and an order of the mark point image of the first layer; the reference mark point image a mark point image of the first layer;

Determining a mark point image of the Mth layer according to a positional relationship between the mark point image other than the mark point image of the first layer to the M-1th layer and the reference mark point image in the image of the Nth frame Sorting of the image of the mark point of the Mth layer; M is an integer greater than or equal to 2;

Determining the number of the point image of the first layer to the Mth layer according to the order of the image of the mark points of the first layer to the Mth layer.
The apparatus of claim 9, wherein the determining module is further configured to:

The marker point image is determined based on at least a pixel value of each pixel of the Nth frame image, a number of pixel points on each contour, and a number of pixel points included in the contour.
The device according to claim 12, wherein the determining module is specifically configured to:

Obtaining, according to the pixel value of each pixel of the image of the Nth frame, a first candidate point image; the pixel value of the pixel of the first candidate point image is greater than or equal to a first threshold;

Obtaining a second candidate point image according to the number of pixels on the contour of each of the first candidate point images; the number of pixels on the contour of the second candidate point image is greater than or equal to a second threshold and less than or equal to a third threshold;

Obtaining, according to the number of pixels included in the contour of each of the second candidate point images, a third candidate point image; the number of pixels included in the contour of the third candidate point image Greater than or equal to the fourth threshold;

Determining an ellipse parameter of the third candidate point image, and determining a third candidate point image that matches the ellipse parameter to the preset parameter range as the point image.
The device according to claim 9, wherein the processing module is specifically configured to:

Determining, by the PnP algorithm, the camera device relative to the object to be tested, according to the correspondence between the point image and the physical point, and the position information of each of the physical point and the image of each point The amount of rotation and the amount of translation;

Obtaining motion information of the object to be tested at a time corresponding to the image of the Nth frame according to a rotation amount and a translation amount of the imaging device with respect to the object to be tested; and the image of the object to be tested is in the image of the Nth frame The motion information corresponding to the time is the amount of rotation and the amount of translation of the object to be tested relative to the camera at the time corresponding to the image of the Nth frame.
The device of claim 14, wherein the processing module is further configured to:

The amount of rotation and the amount of translation of the camera device relative to the object to be tested are optimized using an LM algorithm.
The device according to claim 9, wherein the physical mark point is an infrared point; and the mark point image is an infrared point image;

The processing module is further configured to:

Determining, when the motion information of the object to be tested is in the range of the preset motion amount, the infrared point of the first side of the object to be tested is turned off, and the second side of the object to be tested is opened. An infrared point; the second side is predicted according to motion information of the object to be tested at a time corresponding to the image of the Nth frame;

Acquiring an image of the (N+1)th frame acquired by the camera device;

Determining, by the N+1 frame image, an infrared point image of each infrared point of the second side of the object to be tested, and if yes, determining, according to the (N+1)th frame image, the object to be tested is The N+1 frame image corresponds to the motion information of the time; if not, the infrared point of the second side of the object to be tested is turned off, the infrared point of the third side of the object to be tested is turned on, and the image capturing device is acquired. The N+2th frame image; the third side is obtained according to a preset cyclic sequence.