CN104375626B - Show processing method, the device and system of sign - Google Patents

Abstract

The embodiment of the present invention provides a kind of processing method, device and system for showing sign, and this method includes：Obtaining includes the mobile terminal and the human hand and the current scene image in imaging region of human hand gripping；According to the current scene image, determine that the mobile terminal points to the direction indication in the imaging region；According to the direction indication and scope of the imaging region in the current scene image, the relative position being located between the direction indication and the imaging region in the imaging region is determined；The display signal provided according to the mobile terminal, sign is shown in the relative position；This method is easy to user to make instruction on imaging region using any mobile terminal at any time, and versatility is preferable, and simple operation, and the work and study to user offer convenience.

Description

Processing method, device and system for displaying indication marks

technical field

Embodiments of the present invention relate to the field of electronic information technology, and in particular, to a processing method, device and system for displaying indication marks.

Background technique

In today's conference scenes, projectors and projection screens are the most commonly used means of information display. When participants explain and discuss based on the information displayed on the projection screen, they often display laser dots on the projection screen with a laser pointer. Indicate the projected content. However, in some conference scenarios, laser pointers are often neglected or limited in number, which causes a lot of inconvenience. Therefore, if mobile terminals that people can carry on a daily basis, such as mobile phones and PDAs (Personal Digital Assistants, personal digital assistants), have similar laser pointers The indication function will greatly facilitate people's work.

In order to make the above-mentioned mobile terminal have a pointer function similar to that of a laser pointer, in the prior art, a laser diode is usually built into the mobile terminal, and a corresponding driving circuit is configured, so that the laser diode emits laser light, and an indication effect appears on the projection screen. .

However, when realizing the above scheme, it is necessary to modify the existing hardware structure and software program of the mobile terminal, and currently different types of mobile terminals have different hardware structures and software programs, so different modification schemes need to be designed for different types of mobile terminals , consumes manpower and material resources, and has poor versatility.

Contents of the invention

Embodiments of the present invention provide a processing method, device, and system for displaying an indication mark, which are used to indicate an imaged area by using a mobile terminal.

In a first aspect, an embodiment of the present invention provides a processing method for displaying an indicator, including:

Acquiring the current scene image including the mobile terminal held by the human hand and the human hand and the imaging area;

According to the current scene image, determine the indication direction of the mobile terminal pointing to the imaging area;

According to the indication direction and the range of the imaging area in the current scene image, determine the associated position between the indication direction and the imaging area in the imaging area;

According to the display signal provided by the mobile terminal, an indication mark is displayed at the associated position.

With reference to the first aspect, in the first implementation manner, the determining the indication direction of the mobile terminal pointing to the imaging area according to the current scene image includes:

Determining the position of the human hand in the current scene image according to the preset human hand recognition feature;

Based on the position of the human hand, an edge detection operator is used to determine the characteristic plane of the mobile terminal held by the human hand, the characteristic plane is a geometric plane figure with a central axis of the mobile terminal, or the characteristic plane The plane is a geometric plane figure with a central axis displayed by the mobile terminal;

According to the characteristic plane of the mobile terminal, the central axis of the characteristic plane of the mobile terminal is determined as the indicating direction.

With reference to the first aspect or the first implementation of the first aspect, in the second implementation, before displaying the indicator at the associated position according to the display signal provided by the mobile terminal, the display method further includes:

The display signal sent by the mobile terminal is received.

With reference to the second embodiment of the first aspect, in the third embodiment, according to the characteristic plane of the mobile terminal, determining the central axis of the characteristic plane of the mobile terminal as the indication direction includes:

Using a straight line detection algorithm to determine the two-dimensional area equation of the geometric plane figure with the central axis;

According to the two-dimensional area equation, obtain the three-dimensional area equation of the geometric plane figure with the central axis in the standard three-dimensional coordinate system;

According to the three-dimensional area equation of the geometric plane figure with the central axis in the standard three-dimensional coordinate system, determine the three-dimensional line equation of the central axis of the geometric plane figure with the central axis in the standard three-dimensional coordinate system as the indication direction.

In combination with the third embodiment of the first aspect, in the fourth embodiment, the central axis of the geometric plane figure with the central axis is determined according to the three-dimensional area equation of the geometric plane figure with the central axis in the standard three-dimensional coordinate system The three-dimensional line equation under the standard three-dimensional coordinate system is used as the indicated direction, including:

When the geometric plane figure with the central axis has at least two central axes; then obtain the current indication direction of the built-in electronic compass of the mobile terminal;

Select the first central axis parallel to the current indication direction of the electronic compass among at least two central axes of the geometric plane figure with the central axis, and according to the geometric plane figure with the central axis in a standard three-dimensional coordinate system The three-dimensional area equation below, and determine the three-dimensional line equation of the first central axis in the standard three-dimensional coordinate system as the indicating direction.

With reference to the third or fourth implementation manner of the first aspect, in a fifth implementation manner, the indication direction and the imaging area are determined according to the range of the indication direction and the imaging area in the current scene image The associated positions between the regions in the imaging region include:

According to the three-dimensional line equation of the central axis of the geometric plane figure with the central axis in the standard three-dimensional coordinate system and the three-dimensional area equation of the image-presenting area in the standard three-dimensional coordinate system, the calculation is performed to obtain the three-dimensional area equation in the standard three-dimensional coordinate system. Intersection coordinates in a standard three-dimensional coordinate system, determining the intersection coordinates as the associated position.

With reference to any one of the third to fifth implementations of the first aspect, in the sixth implementation, before determining the pointing direction of the mobile terminal according to the current scene image, the control method further includes :

Obtain the calibration parameter information according to the preset sampling detection point;

determining the standard three-dimensional coordinate system according to the calibration parameter information;

A three-dimensional area equation of the imaging area in the standard three-dimensional coordinate system is determined according to the calibration parameter information and the standard three-dimensional coordinate system.

In a second aspect, an embodiment of the present invention provides a processing device for displaying indication marks, including:

An acquisition module, configured to acquire a current scene image including a mobile terminal held by a human hand and the human hand and the imaging area;

A first determining module, configured to determine an indication direction of the mobile terminal pointing to the imaging area according to the current scene image;

The second determination module is configured to determine the associated position between the indication direction and the imaging area in the imaging area according to the indication direction and the range of the imaging area in the current scene image;

A control module, configured to display an indication mark at the associated position according to the display signal provided by the mobile terminal.

With reference to the second aspect, in the first implementation manner, the first determination module includes:

A human hand recognition unit, configured to determine the position of the human hand in the current scene image according to preset human hand recognition features;

A characteristic plane determining unit, configured to determine a characteristic plane of the mobile terminal held by the human hand by using an edge detection operator based on the position of the human hand, the characteristic plane being a geometrical axis of the mobile terminal with a central axis A plane figure, or the feature plane is a geometric plane figure with a central axis displayed on the mobile terminal;

The pointing direction determining unit is configured to determine the central axis of the characteristic plane of the mobile terminal as the pointing direction according to the characteristic plane of the mobile terminal.

With reference to the second aspect or the first implementation manner of the second aspect, in the second implementation manner, the processing device further includes:

The receiving module is configured to receive the display signal sent by the mobile terminal.

With reference to the second implementation manner of the second aspect, in a third implementation manner, the pointing direction determining unit includes:

The first equation determination subunit is used to determine the two-dimensional area equation of the geometric plane figure with the central axis by using a straight line detection algorithm;

The second equation determination subunit is used to obtain the three-dimensional area equation of the geometric plane figure with the central axis in the standard three-dimensional coordinate system according to the two-dimensional area equation;

Indicate direction determination subunit, used to determine the central axis of the geometric plane figure with central axis in the standard three-dimensional coordinate system according to the three-dimensional area equation of the geometric plane figure with central axis in the standard three-dimensional coordinate system The three-dimensional straight line equation serves as the indicated direction.

With reference to the third embodiment of the second aspect, in the fourth embodiment, when the geometric planar figure with a central axis has at least two central axes; The current indication direction of the compass; select the first central axis parallel to the current indication direction of the electronic compass among at least two central axes of the geometric plane figure with the central axis, and according to the characteristic plane in the standard three-dimensional coordinates The three-dimensional area equation in the standard three-dimensional coordinate system is used to determine the three-dimensional line equation of the first central axis in the standard three-dimensional coordinate system as the indicating direction.

With reference to the third or fourth implementation of the second aspect, in the fifth implementation, the second determination module is specifically configured to, according to the central axis of the geometric planar figure with a central axis in the standard three-dimensional coordinate system The three-dimensional straight line equation below is operated with the three-dimensional area equation of the imaging area in the standard three-dimensional coordinate system to obtain the intersection coordinates in the standard three-dimensional coordinate system, and determine the intersection coordinates as the associated position.

With reference to any one of the third to fourth implementations of the second aspect, in the sixth implementation, the processing device further includes:

The third determination module is used to obtain calibration parameter information according to preset sampling detection points before determining the indication direction of the mobile terminal according to the current scene image; determine the standard three-dimensional coordinate system according to the calibration parameter information; A three-dimensional area equation of the imaging area in the standard three-dimensional coordinate system is determined according to the calibration parameter information and the standard three-dimensional coordinate system.

In a third aspect, an embodiment of the present invention provides a processing system for displaying indication marks, including a display device, a mobile terminal, and a control device; the control device includes any one of the above-mentioned second aspect to the sixth embodiment of the second aspect The above-mentioned processing device for displaying indication signs; the display device has an image display area for displaying information; the mobile terminal and the display device can respectively perform data transmission with the control device through a communication interface.

The indication control method, device and system provided by the embodiments of the present invention determine the indication direction of the mobile terminal and the associated position between the indication direction and the image display area of the information through the collected current scene image, so as to realize The indication mark displayed on the image area is convenient for the user to use any mobile terminal to make instructions on the image area of the information at any time, that is, the method has good versatility and is easy to operate, which brings convenience to the user's work and study.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 is a flow chart of Embodiment 1 of the processing method for displaying indication marks in the present invention;

FIG. 2 is a flow chart of Embodiment 2 of the processing method for displaying indication marks in the present invention;

FIG. 3 is a schematic diagram of an application scenario in Embodiment 2 of the present invention;

FIG. 4 is a schematic diagram of a mobile terminal in Embodiment 2 of the present invention;

FIG. 5 is a schematic diagram of another mobile terminal in Embodiment 2 of the present invention;

FIG. 6 is a flow chart of Embodiment 3 of the processing method for displaying indication signs according to the present invention;

FIG. 7 is a schematic diagram of an application scenario in Embodiment 3 of the present invention;

FIG. 8 is a structural diagram of Embodiment 1 of a processing device for displaying indication marks in the present invention;

FIG. 9 is a structural diagram of Embodiment 2 of a processing device for displaying indication signs in the present invention;

FIG. 10 is a schematic diagram of Embodiment 1 of a processing system for displaying indication marks according to the present invention.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

FIG. 1 is a flow chart of Embodiment 1 of a processing method for displaying an indicator in the present invention. The method of this embodiment can be applied to situations similar to projection display. When projecting and displaying, a projector and a projection screen are generally provided. The area where information is displayed on the projection screen is used as an image display area, and the projector is generally connected to a control device with control functions. For example, the control device is usually a computer. It is also necessary to set up an on-site image capture device and a mobile terminal. The on-site image capture device is, for example, a camera, which may be an independent camera, and the image captured by it is provided to the control device, or the control device may be integrated with an image capture function. The mobile terminal is typically a mobile phone, and the user uses the mobile phone to make a pointing action, and then the control device can display the pointing logo in the image display area according to the pointing action made by the user using the mobile phone.

The method in this embodiment can be specifically implemented by a processing device for displaying an indicator, which can be implemented by hardware and/or software, and is preferably set in the above-mentioned control device to realize the display of the indicator. As shown in Figure 1, the method includes:

S101. Acquire a current scene image including a mobile terminal held by a human hand, the human hand, and an imaging area.

Taking the projection display above as an example, the user holds the mobile terminal with his hand and makes an instruction (for example, if the user, as an interpreter, needs to explain a part of the machine currently displayed on the projection screen to the participants, he will hold the mobile terminal with his hand. hold the mobile terminal to point to the part, that is, make a pointing action, expecting to display the pointing mark at the part), at this time, the on-site image capture device is controlled by the processing device that displays the pointing mark, and the acquired current scene image includes human hands The held mobile terminal and the human hand, and the projection screen as the imaging area.

S102. According to the current scene image, determine the indication direction of the mobile terminal pointing to the imaging area;

The on-site image capture device provides the current scene image to the processing device that displays the indication mark, and the processing device performs image processing and analysis according to the collected current scene image to determine the direction in which the mobile terminal points to the image area; when determining the above direction, The processing device can know the current location of the mobile terminal through image analysis, and perform feature analysis, specifically, use a preset coordinate system to determine the current location of the mobile terminal, and carry out calculation processing in this coordinate system to determine the characteristics of the mobile terminal. the direction indicated. For example, the mobile terminal is a mobile phone, and the imaging area used to display information is a projection screen. The projection screen currently displays a mechanical picture, and there is a certain distance between the explainer and the projection screen. In the process of explaining the figure, it is necessary to give instructions to each part in the figure in accordance with the content of the explanation. After visual inspection, the explainer can hold the mobile phone with his hand and point to the direction of the part that needs to be explained. The image processing and analysis of the current scene image can determine the current indication direction of the mobile phone, that is, determine the direction currently indicated by the user through the mobile phone.

S103. According to the range of the indication direction and the imaging area in the current scene image, determine the associated position between the indication direction and the imaging area in the imaging area;

The aforementioned imaging area for displaying information can be various areas that can be used for displaying information. For example, in a conference room equipped with a projector and a projection screen, the above-mentioned imaging area is a projection screen, and the content displayed on the projection screen is the It is obtained by projecting the computer display screen facing the projection screen; if in a studio equipped with an electronic screen, the above-mentioned imaging area is the electronic screen, such as an LED (Light Emitting Diode, light emitting diode) display screen. The processing device can know the range of the imaging area through preset, manual adjustment or automatic recognition, and the range can be defined by coordinates.

The above-mentioned imaging area and the mobile terminal have an associated relationship in spatial position, such as the distance and angle between the mobile terminal and the imaging area, and the associated position between the mobile terminal and the imaging area can be determined according to the association. For example, the mobile terminal is usually a regular geometric shape, then the above-mentioned processing device can use the orientation of the central axis of one side of the mobile terminal (such as the side of the mobile phone where the display screen of the mobile phone is located) as the above-mentioned indicating direction, and the central axis must be in the same shape as If there is an intersection point between the planes where the image area is located, the intersection point is the associated position.

S104. According to the display signal provided by the mobile terminal, display the indication mark at the associated position.

The indicator is formed by the processing device according to the display signal provided by the mobile terminal. For example, if a person holds the mobile terminal and points to the image display area, in order to display the indicator at a certain position on the image display area, the processing device combines the mobile terminal with the The relative position of the imaging area, and the indicator mark is displayed on the imaging area. For example, the imaging area of the information is the projection screen, and the interpreter currently wishes to display an indicator mark, such as a red dot, at the center of the projection screen, then the interpreter will use the mobile terminal held by the human hand to show the position of the center point of the projection screen. After the instruction action is made, the processing device determines the accurate associated position through S101～S103 (due to the observation error of the human eye, the associated position may coincide with the center point of the projection screen on the projection screen, or be between the center point of the projection screen There is a certain distance difference between them), a red dot will be displayed at the associated position, that is, the indicator mark will be realized. If there is a distance difference between the indicator mark and the center point expected by the interpreter, the interpreter will hold and move it by hand The terminal performs movement adjustment, so that the processing device executes S101 to S104 again, and displays an indication mark at the central point.

In the processing method for displaying indication signs provided in this embodiment, the indication direction of the mobile terminal and the associated position between the indication direction and the image area of the information are determined by collecting the current scene image and performing image analysis, so as to realize the The terminal displays an indicator mark on the image display area, which is convenient for users to use any mobile terminal to make instructions on the image display area of information at any time, that is, this method has good versatility and is easy to operate, which brings convenience to users' work and study.

FIG. 2 is a flow chart of Embodiment 2 of the processing method for displaying indication marks according to the present invention. As shown in Figure 2, this embodiment further optimizes the process of image processing based on the above embodiments, the method includes:

S201. Determine a standard three-dimensional coordinate system.

FIG. 3 is a schematic diagram of an application scenario in Embodiment 2 of the present invention. As shown in FIG. 3 , the image display area for displaying information is an electronic display screen 20 , and the on-site image capture device 50 is a stereoscopic vision device, which uses dual cameras to capture on-site images. The processing device for displaying the indicator can control the display of the electronic display screen, and can also receive the image acquired by the stereoscopic vision device and perform image processing. Since the processing device analyzes the orientation of the mobile terminal 30 according to the scene image collected by the on-site image capture device, it is necessary to determine the standard three-dimensional coordinate system O _c -X _c Y _c Z _c under the perspective of the on-site image capture device and the coordinates of the processed image. Relevant parameters, the specific determination method is as follows:

S1a. Obtain calibration parameter information according to preset sampling detection points;

In practical applications, Zhang Zhengyou’s calibration method can be used to obtain calibration parameter information; specifically, a sampling detection point is preset in the scene, such as placing a black and white chessboard with a known fixed size in the scene, and using the on-site image capture device Capture the scene image including the sampling detection point; obtain calibration parameter information by performing image analysis and processing on the image captured by the on-site image capture device. The calibration parameter information includes internal parameters and structural parameters; the internal parameters include the on-site image capture device. The focal length of the camera, the distortion parameters of the imaging, etc. The structural parameters include the three-dimensional translation matrix and the three-dimensional rotation matrix between the two cameras. The translation matrix is The rotation matrix is The values of the elements t _x , _ty , _tz in the translation matrix and the elements r ₁ to r ₉ in the rotation matrix are obtained by analyzing and processing the captured image above.

S1b. Determine the standard three-dimensional coordinate system according to the calibration parameter information;

Specifically, the above structural parameters can be used to determine a standard three-dimensional coordinate system.

S1c. Determine the three-dimensional area equation of the imaging area in the standard three-dimensional coordinate system according to the calibration parameter information and the standard three-dimensional coordinate system.

The above-mentioned imaging area, such as an electronic display screen, also has a fixed position in the application scene. Therefore, after the standard three-dimensional coordinate system is determined, the three-dimensional area equation of the imaging area in the standard three-dimensional coordinate system can be determined. Specifically, when the on-site image capture equipment shoots the electronic display screen, the processing device performs distortion correction on the captured image according to the above internal parameters, and then performs image processing on the corrected image to obtain the three-dimensional image of the electronic display screen in the standard three-dimensional coordinate system. area equation.

It should be noted that the above S201 will only be executed when the processing method for displaying indication signs of the present invention is implemented for the first time in the application scene, and when the standard three-dimensional coordinate system, that is, the relevant parameters are determined, and the processing method for displaying indication signs is implemented again , directly start execution from S202.

S202. Acquire the current scene image including the mobile terminal held by the human hand, the human hand, and the imaging area.

S203. Determine a feature plane of the mobile terminal held by a human hand according to the current scene image.

Specifically, the processing device for displaying the indicator recognizes the position of the human hand in the image according to the preset human hand recognition features, such as the human skin color feature and the outline of the hand in the collected current scene image; or, the on-site image capture device Integrating the function of the infrared thermal imager, it senses the temperature of the human body and images the human hand in the image, thereby determining the position of the human hand in the image, and then identifying the mobile terminal held by the human hand.

Based on the position of the human hand, an edge detection operator may be used to determine the feature plane of the mobile terminal held by the human hand. Specifically, FIG. 4 is a schematic diagram of a mobile terminal in Embodiment 2 of the present invention. As shown in Figure 4, the mobile terminal is a common mobile phone, and the space occupied by the mobile phone overlaps with the space occupied by the human hand, so the human hand can be used as the detection reference point to detect the mobile phone within a certain detection range; for example After determining the position of the human hand, find the step change of the image near the finger area of the human hand, that is, use the edge detection operator (such as Canny edge operator, gradient operator, Marr operator, etc.) to detect the outline of the mobile phone; according to the mobile phone outline to further determine the direction indicated by the mobile phone.

Usually the mobile phone has a rectangular cubic structure, and the thickness is relatively small compared to the length and width of the front side or the rear side of the mobile phone, so the mobile phone can be regarded as a rectangular flat plate during image processing, and the above-mentioned detection of the outline of the mobile phone That is to say, for a mobile phone, the above-mentioned characteristic plane of the mobile terminal is a geometric plane figure with a central axis of the mobile terminal, that is, a rectangular side of the mobile phone.

More flexibly, FIG. 5 is a schematic diagram of another mobile terminal in Embodiment 2 of the present invention. As shown in Figure 5, the above-mentioned characteristic plane is a geometric plane figure with a central axis displayed on the mobile terminal; An application program can display a geometric plane figure with a central axis on the display screen of the mobile phone, such as a rectangle, and an edge detection operator can be used to detect the outline of the rectangular image displayed on the display screen of the mobile phone.

To sum up, the geometric figure with the central axis on the mobile terminal can be flexibly selected as the characteristic plane of the mobile terminal.

S204. According to the characteristic plane of the mobile terminal, determine a central axis of the characteristic plane of the mobile terminal as an indication direction.

Taking the feature plane as an example of a geometric plane figure with a central axis of a mobile terminal as shown in Figure 4, a straight line detection algorithm is used to detect the straight line equations L1, L2 and L3 of the outline of the above-mentioned mobile phone; the above-mentioned straight line detection algorithm can be Random Hough (Hough) transform and other straight line detection algorithms based on hough transform improvement; specifically take hough transform as an example, its principle is to use the point-line duality of image space and Hough parameter space to transform the detection algorithm in image space The problem is converted to parameter space to detect straight lines in images. What needs to be explained here is that the current scene image collected in S202 is a plane image, so L1, L2 and L3 at this time are two-dimensional straight line equations in this plane image, and L1, L2 and L3 together represent this feature The two-dimensional area equation of the plane; then select multiple feature points on each two-dimensional straight line for stereo feature matching, and obtain the three-dimensional coordinates of each feature point in the above-mentioned standard three-dimensional coordinate system, and finally according to each two-dimensional straight line The three-dimensional coordinates of multiple selected feature points are fitted by the least squares method to obtain the three-dimensional area equation of the feature plane in the standard three-dimensional coordinate system, that is, according to the two-dimensional area equation, the three-dimensional area of the feature plane in the standard three-dimensional coordinate system is obtained equation.

Specifically, in this embodiment, the on-site image capture device adopts dual cameras, and the same point in the scene is different in the two images captured by the two cameras; therefore, five feature points are selected on each of the above-mentioned two-dimensional straight lines (The number of feature points can also be set according to the accuracy requirements, and the specific number is not limited in this embodiment). Each of the five feature points also has a difference in the two images captured by the two cameras. The area matching method in the two images corresponded one by one to the five feature points in the two images, combined with the three-dimensional rotation matrix and three-dimensional translation matrix in the above structural parameters to determine the three-dimensional coordinates of the feature points on each line in the standard three-dimensional coordinate system , the specific formula for determining the three-dimensional coordinates (x, y, z) is as follows:

x=zX _l /f _l ;

y=zY _l /f _l ;

or,

Among them, f _l is the effective focal length of the left camera in the binocular camera, _{fr is the effective focal length of the right camera in the binocular camera; (X r ,} Y _r ) is the two-dimensional coordinates of the feature point in the right camera; (X _l , Y _l ) is the two-dimensional coordinates of the feature points in the left camera; , the positions of feature points A, B, C, D, E in the image captured by one camera are A1, B1, C1, D1, E1 respectively; while the positions in the image captured by another camera are A2, B2, C2, D2, E2; A1 and A2, B1 and B2, C1 and C2, D1 and D2, E1 and E2 are one-to-one corresponded using the grayscale-based area matching method, and use the above to determine the three-dimensional coordinates (x, y, z ) to determine the three-dimensional coordinates of A, B, C, D, and E in the three-dimensional coordinate system. After determining the three-dimensional coordinates of each feature point on each two-dimensional straight line in the standard three-dimensional coordinate system, according to the three-dimensional coordinates of multiple feature points on the same two-dimensional straight line in the three-dimensional coordinate system, the least square method is used to estimate The three-dimensional straight line equation of the straight line in the standard three-dimensional coordinate system is obtained; that is, the three-dimensional straight line equations L1', L2' and L3' of the above-mentioned L1, L2 and L3 in the standard three-dimensional coordinate system are obtained, so that the three-dimensional straight line equation L1 ', L2' and L3' can jointly represent the three-dimensional area equation of the geometric figure with the central axis in the standard three-dimensional coordinate system.

According to the above-mentioned three-dimensional area equations of the geometric figure with the central axis in the standard three-dimensional coordinate system, that is, the three-dimensional straight line equations L1', L2' and L3', it is determined that the central axis of the geometric figure with the central axis is in the standard three-dimensional coordinate system The three-dimensional line equation Lm is used as the indicating direction of the above-mentioned mobile terminal.

It should be noted that, in geometry, there is only one central axis for isosceles triangles and other figures, but for rectangles, there are two central axes. Therefore, when the above-mentioned geometric figures with central axes have at least two central axes axis, the above-mentioned processing device also needs to obtain the current indication direction of the built-in electronic compass of the mobile terminal, so that according to the current indication direction of the electronic compass, at least two central axes of the characteristic plane of the mobile terminal are selected to be parallel to the current indication direction of the electronic compass and according to the three-dimensional area equation of the feature plane in the standard three-dimensional coordinate system, determine the three-dimensional line equation of the first central axis in the standard three-dimensional coordinate system as the indicating direction.

The method of determining the three-dimensional line equation Ln of the central axis of the characteristic plane in the standard three-dimensional coordinate system based on the characteristic plane shown in FIG.

S205. Determine the associated position between the pointing direction and the image display area in the image display area according to the pointing direction and the range of the image display area of the information.

Since the three-dimensional area equation of the imaging area in the standard three-dimensional coordinate system is known in S201; then according to the three-dimensional line equation (Lm or Ln) of the central axis of the feature plane in the standard three-dimensional coordinate system and the imaging area in the standard three-dimensional coordinate system The three-dimensional area equation in the system is calculated to obtain the coordinates of the intersection point in the standard three-dimensional coordinate system, and the coordinates of the intersection point are determined as the associated position, that is, the three-dimensional line equation of the central axis of the simultaneous feature plane in the standard three-dimensional coordinate system and the The three-dimensional area equation of the image area in the standard three-dimensional coordinate system is solved, and the obtained solution value is the three-dimensional coordinate of the intersection point.

S206. Judging whether the display signal sent by the mobile terminal is received; if yes, execute S207; if not, stop.

When there are many mobile terminals in the scene (for example, each participant holds a mobile phone), the above-mentioned device can determine the associated position between the indication direction of the mobile terminal and the image area corresponding to each mobile terminal, but if each mobile terminal If the indication mark of the mobile terminal is displayed, the indication confusion is likely to appear on the imaging area, so the processing device needs to receive the display signal sent by the mobile terminal before controlling the indication mark to be displayed at the associated position; After the display signal is sent, the indicator will be displayed at the associated position corresponding to the mobile terminal; if the above-mentioned display message is not received, the indicator will not be displayed; but it is understandable that the processing device will keep track of the indication of the mobile terminal direction, and determine in real time the associated position between the current pointing direction of the mobile terminal and the image area, so that after receiving the display signal sent by the mobile terminal, the pointing logo can be displayed at the associated position immediately.

Taking the conference scene as an example, there are 5 participants, each of whom carries a mobile phone, and only one of the 5 people is the information explainer, and only the indicator based on the explainer’s mobile phone needs to be displayed. Therefore, the explainer Open the application program adapted to the processing method of this embodiment loaded on the mobile phone, and a "display" button will appear on the display screen of the mobile phone. Clicking this button will trigger the mobile phone to send a signal wave to the processing device. The display signal sent by the processing device is flexible, and the signal wave can also be triggered by pressing the physical button on the mobile phone; when the processing device controls the display of the indication sign, that is, the relationship between the direction indicated by the explainer's mobile phone and the projection screen A red dot is displayed at the position (the red dot is only used to illustrate the method of this embodiment, and the specific display shape or color is not limited in this embodiment), if the explainer finds that the red dot is not at the desired position , keep pressing the "Display" button and move the mobile phone, and the processing device obtains the associated position between the indicated direction of the mobile phone and the imaging area in the current scene in real time, so that the red dot is displayed at the current associated position.

It can also be understood that if only the explainer holds a mobile phone in the meeting scene, the processing device will send out a detection signal that the mobile phone can receive in the meeting scene after detecting that there is only one mobile phone on the scene (for example, the processing device sends a certain communication signal to wave, then the processing device can determine the number of mobile phones in the meeting scene according to the received feedback message sent by the mobile phone; or, the mobile terminal can control the indicator light on the mobile terminal to flash by opening the application program loaded on the mobile terminal, and send The processing device indicates that the mobile terminal will point to the imaging area. It is understandable that the processing device will not track the mobile terminal in real time for other mobile terminals that do not open the application program to make the indicator light on the mobile terminal flash. The indication direction of the mobile terminal.), then the indication logo will be displayed at the associated position between the indication direction of the mobile phone and the image area by default, and there is no need to execute S206 to determine whether the display message of the indication logo sent by the mobile terminal is received, that is, execute S207 may be directly executed after S205.

S207. According to the display signal provided by the mobile terminal, display the indication mark at the associated position.

According to the three-dimensional coordinates of the intersection determined in S205, the electronic display screen is controlled to display the indication mark at the corresponding position.

In the indication control method provided in this embodiment, the indication direction of the mobile terminal and the associated position between the indication direction and the image display area of the information are determined through the collected current scene image, so as to realize the identification of the display area in the image display area through the mobile terminal. The indicator mark is displayed on the screen, which is convenient for the user to use any mobile terminal to make instructions on the imaging area at any time, that is, the method has good versatility and is easy to operate, which brings convenience to the user's work and study.

FIG. 6 is a flow chart of Embodiment 3 of the processing method for displaying indication signs according to the present invention. As shown in Figure 6, on the basis of the first embodiment above, this embodiment can also perform another optimization of the image processing process, the method includes:

S301. Determine a standard three-dimensional coordinate system.

Specifically, FIG. 7 is a schematic diagram of an application scenario in Embodiment 3 of the present invention. As shown in Figure 7, the display of information is specifically realized by a projection screen 21 and a computer, wherein the content on the projection screen is obtained by projecting the computer screen 22 of a computer, and the on-site image capture device 50 is a stereoscopic vision device , wherein the processing device for displaying the indication mark is responsible for processing the image acquired by the stereoscopic vision device, and the display of the content on the projection screen is controlled by a computer, but the processing device for displaying the indication mark can control the display screen of the computer to display.

Similar to S201, it is necessary to determine the standard three-dimensional coordinate system under the perspective of the on-site image capture device and the relevant parameters of the processed image. The specific determination method is as follows:

S3a. Obtain calibration parameter information according to preset sampling detection points;

The method for obtaining the calibration parameter information in this embodiment is similar to step S1a in S201 of the second embodiment, but in this embodiment, the calibration parameter information includes not only internal parameters and structural parameters, but also information used for coordinate system conversion. Specifically, according to Zhang Zhengyou’s calibration method, the standard three-dimensional coordinate system O _c -X _c Y _c Z _c based on the perspective of the on-site image capture device and the relative three-dimensional coordinate system O _ps -X _ps based on the perspective of the projection screen are determined The conversion parameters between Y _ps Z _ps , and the conversion parameters between the two-dimensional coordinate system O _s -X _s Y _s of the computer display screen and the relative three-dimensional coordinate system O _ps -X _ps Y _ps Z _ps of the projection screen.

S3b. Determine a standard three-dimensional coordinate system according to the calibration parameter information;

Step S3b in this embodiment is similar to step S1b in S201 in Embodiment 2, and will not be repeated here.

S3c. Determine a three-dimensional area equation of the imaging area in the standard three-dimensional coordinate system according to the calibration parameter information and the standard three-dimensional coordinate system.

Step S3c in this embodiment is similar to step S1c in S201 in Embodiment 2, and will not be repeated here.

S3d. Determine the conversion relationship between the coordinate systems according to the above conversion parameters.

Specifically, determine the conversion relationship between the above-mentioned standard three-dimensional coordinate system and the relative three-dimensional coordinate system , where X _c , Y _c and Z _c represent the X axis, Y axis and Z axis in the standard coordinate system, while X _ps , Y _ps and Z _ps represent the X axis, Y axis in the above relative coordinate system Axis and Z axis, R is a 3×3 rotation matrix, T is a 3×1 translation matrix, wherein the assignment of elements in the matrix is determined according to the measured values on site, that is, determined by step S3a; further, determine the two-dimensional of the computer screen The conversion relationship between the coordinate system and the relative three-dimensional coordinate system of the projection screen Wherein, X _S and Y _S are the X axis and Y axis in the two-dimensional coordinate system of the computer screen, and the assignment _of the conversion parameters A1 to _A8 is determined in step S3a.

It should be noted that the above S301 will be executed only when the processing method of the present invention is implemented for the first time in the application scene, and when the relevant parameters of the standard three-dimensional coordinate system are determined, and the processing method of displaying the indication mark is implemented again, directly by S302 Begin execution.

S302. Obtain the current scene image including the mobile terminal held by the human hand, the human hand, and the imaging area

S303. Determine a feature plane of the mobile terminal according to the current scene image.

S304. According to the characteristic plane of the mobile terminal, determine the central axis of the characteristic plane of the mobile terminal held by the human hand as the indicating direction.

S305. According to the pointing direction and the range of the image display area of the information, determine the associated position between the pointing direction and the image display area in the image display area.

S306. Judging whether the display signal sent by the mobile terminal is received; if yes, execute S307; if not, stop.

S302-S306 in this embodiment are similar to Embodiment 2, and reference may be made to S202-S206, which will not be repeated here.

S307. According to the display signal provided by the mobile terminal, display the indication mark at the associated position.

The associated position determined in S305 is determined under the standard three-dimensional coordinate system, that is, the above-mentioned processing device determines the intersection point between the central axis of the characteristic plane of the mobile terminal and the projection screen under the standard three-dimensional coordinate system, but due to the The display is obtained by projecting the content on the computer display screen, so displaying the indication mark at the associated position on the projection screen requires the computer display screen to display the indication mark at the place corresponding to the associated position; therefore, the processing device for displaying the indication mark When displaying the indication mark at the associated position, the following steps shall also be included:

S4a, converting the associated position into a coordinate system.

Specifically, the associated position in the standard three-dimensional coordinate system is determined in S305, that is, the coordinates of the intersection point between the central axis of the characteristic plane of the mobile terminal and the projection screen are (X _c1 , Y _c1 , Z _c1 ); according to Transform (X _c1 , Y _c1 , Z _c1 ) to obtain the coordinates (X _ps1 , Y _ps1 , Z _ps1 ) of the associated position in the relative coordinate system; since the computer screen adopts a two-dimensional coordinate system, according to the computer screen The conversion relationship between the two-dimensional coordinate system and the relative three-dimensional coordinate system of the projection screen Convert (X _ps1 , Y _ps1 ) to obtain (X _s1 , Y _s1 ).

S4b. Send a display control message to the display device.

When the above-mentioned processing device sends a control display message to the computer in the display device, it carries a position indication message, that is, coordinates (X _s1 , Y _s1 ). For example, the computer determines (X _s1 , Y _s1 ) according to (X _s1 , Y s1 ). The position indicated by _s1 ) is the center point of the computer display screen, then the computer controls the computer screen to display a red dot at the center point, so that the red dot at the center point of the computer screen is projected and displayed on the projection screen at the same position as S305 The determined associated positions match to realize the display of the indication mark.

More flexibly, the processing device for displaying the indicator may not execute S4a, but directly execute S4b; that is, the coordinates of the intersection point (X _c1 , Y _c1 , Z _c1 ) between the central axis of the characteristic plane of the mobile terminal and the projection screen are directly sent to the computer, Then the computer converts (X _c1 , Y _c1 , Z _c1 ) into (X _s1 , Y _s1 ) according to the conversion formulas of each coordinate system, and displays the indicator at the corresponding position on the computer screen according to (X _s1 , Y _s1 ) , then the indicator will also be projected to the associated position determined in S304 on the projection screen.

In the indication control method provided in this embodiment, the indication direction of the mobile terminal and the associated position between the indication direction and the image display area of the information are determined through the collected current scene image, so as to achieve the mobile terminal on the image display area. Displaying the indication mark is convenient for the user to use any mobile terminal to make an indication on the image display area of the information at any time, that is, the method has good versatility and is easy to operate, which brings convenience to the user's work and study.

FIG. 8 is a structural diagram of Embodiment 1 of a processing device for displaying indication marks according to the present invention. As shown in Figure 8, the device includes:

The acquisition module 10 is used to acquire the current scene image including the mobile terminal held by the human hand and the human hand and the imaging area;

The first determination module 11 is configured to determine an indication direction of the mobile terminal pointing to the imaging area according to the current scene image;

The second determination module 12 is configured to determine the associated position between the indication direction and the imaging area in the imaging area according to the indication direction and the range of the imaging area in the current scene image ;

The control module 13 is configured to display an indication mark at the associated position according to the display signal provided by the mobile terminal.

The processing device for displaying indication signs provided in this embodiment determines the indication direction of the mobile terminal and the associated position between the indication direction and the image display area of the information through the collected current scene image, so as to realize the display of information through the mobile terminal. The indication mark is displayed on the area, which is convenient for the user to use any mobile terminal to make instructions on the information display area at any time, that is, the device has good versatility and is easy to operate, which brings convenience to the user's work and study.

FIG. 9 is a structural diagram of Embodiment 2 of a processing device for displaying indication marks according to the present invention. As shown in FIG. 9, the above-mentioned first determination module 11 specifically includes:

The human hand recognition unit 111 is configured to determine the position of the human hand in the current scene image according to the preset human hand recognition feature;

The characteristic plane determination unit 112 is configured to determine a characteristic plane of the mobile terminal held by the human hand by using an edge detection operator based on the position of the human hand, the characteristic plane being a central axis of the mobile terminal. A geometric plane figure, or the feature plane is a geometric plane figure with a central axis controlled and displayed by the mobile terminal;

The pointing direction determining unit 113 is further configured to determine the central axis of the characteristic plane of the mobile terminal as the pointing direction according to the characteristic plane of the mobile terminal.

Further, the above-mentioned processing device also includes:

The receiving module 14 is configured to receive a display signal sent by the mobile terminal.

The above-mentioned pointing direction determining unit 113 specifically includes: a first equation determining subunit 113a, which is specifically used to determine the two-dimensional area equation of the geometric plane figure with a central axis by using a straight line detection algorithm;

The second equation determination subunit 113b is also used to obtain the three-dimensional area equation of the geometric plane figure with the central axis in the standard three-dimensional coordinate system according to the two-dimensional area equation;

The indication direction determination subunit 113c is also used to determine the central axis of the geometric plane figure with the central axis in the standard three-dimensional coordinate system according to the three-dimensional area equation of the geometric plane figure with the central axis in the standard three-dimensional coordinate system The following three-dimensional straight line equation serves as the indicated direction.

Further, when the above-mentioned geometric plane figure with a central axis has at least two central axes; the above-mentioned pointing direction determination subunit 113c is also used to obtain the current pointing direction of the built-in electronic compass of the mobile terminal;

Select the first central axis parallel to the current indication direction of the electronic compass among at least two central axes of the above-mentioned geometric plane figure with the central axis, and according to the three-dimensional area equation of the characteristic plane in the standard three-dimensional coordinate system, A three-dimensional line equation of the first central axis in the standard three-dimensional coordinate system is determined as the indicating direction.

Further, the above-mentioned second determination module 12 is specifically configured to use the three-dimensional straight line equation of the central axis of the geometric plane figure with the central axis in the standard three-dimensional coordinate system and the image forming area in the standard three-dimensional coordinate system The three-dimensional area equation in the calculation is performed to obtain the coordinates of the intersection point in the standard three-dimensional coordinate system, and the coordinates of the intersection point are determined as the associated position.

Flexibly, corresponding to the third embodiment of the above-mentioned method, the above-mentioned processing device further includes a sending module 15, configured to send a control display message to the display device.

Further, the above-mentioned processing device also includes:

The third determination module 16, before determining the pointing direction of the mobile terminal according to the current scene image, obtains the calibration parameter information according to the preset sampling detection point; determines the standard three-dimensional coordinate system according to the calibration parameter information; The calibration parameter information and the standard three-dimensional coordinate system determine a three-dimensional area equation of the imaging region in the standard three-dimensional coordinate system.

Each module in each of the foregoing apparatus embodiments corresponds to each of the foregoing method embodiments, and details are not repeated here.

The processing device for displaying indication signs provided in this embodiment determines the indication direction of the mobile terminal and the associated position between the indication direction and the image display area of the information through the collected current scene image, so as to realize the display of information through the mobile terminal. The indication mark is displayed on the area, which is convenient for the user to use any mobile terminal to make instructions on the information display area at any time, that is, the device has good versatility and is easy to operate, which brings convenience to the user's work and study.

Further, FIG. 10 is a schematic diagram of Embodiment 1 of a processing system for displaying indication marks according to the present invention. As shown in Figure 10, the system includes a display device 60, a mobile terminal 30, and a control device 40, wherein the display device 60 has an image-presenting area for displaying information, which may specifically be composed of the above-mentioned projection screen 21 and the computer 22; It is the above-mentioned independent electronic display screen 20, and the content displayed on the electronic display screen 20 is controlled and displayed by the control device 40; the control device 40 includes the above-mentioned processing device for displaying indication marks. Indicates the identified processing device. In addition, the control device 40 can integrate the functions of the above-mentioned on-site image capture device 50, and can also receive the on-site scene image sent by the independent on-site image capture device 50; If there is an associated position between the direction and the image area, the control device 40 controls the display device 20 to display an indication mark at the associated position.

Flexible, after the control device 40 determines the associated location, before controlling the display device 20 to display the indicator at the associated location, it needs to judge whether it has received the display message indicating the identifier sent by the mobile terminal 30, and if so, then control the display device 20 to display the indicator at the associated location. Otherwise, do not display; when there are multiple mobile terminals in the scene, the display control of the indicator is performed by judging whether the display message of the indicator sent by a certain mobile terminal 30 is received, so as to avoid the occurrence of information The indicator mark appears on the image area to show confusion.

The indication control system provided in this embodiment determines the indication direction of the mobile terminal and the associated position between the indication direction and the image display area of the information through the collected current scene image, and realizes the presentation of the information through the mobile terminal. The indication mark of the image area is convenient for the user to use any mobile terminal to make instructions on the image area of the information at any time, that is, the system has good versatility and is easy to operate, which brings convenience to the user's work and study.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by program instructions and related hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (13)

1. A processing method for displaying an indication mark, characterized in that, comprising: Acquiring the current scene image including the mobile terminal held by the human hand and the human hand and the imaging area; According to the current scene image, determine the indication direction of the mobile terminal pointing to the imaging area; According to the indication direction and the range of the imaging area in the current scene image, determine the associated position between the indication direction and the imaging area in the imaging area; According to the display signal provided by the mobile terminal, display an indicator at the associated position; The determining the indication direction of the mobile terminal pointing to the imaging area according to the current scene image includes: Determining the position of the human hand in the current scene image according to the preset human hand recognition feature; Based on the position of the human hand, an edge detection operator is used to determine the characteristic plane of the mobile terminal held by the human hand, the characteristic plane is a geometric plane figure with a central axis of the mobile terminal, or the characteristic plane The plane is a geometric plane figure with a central axis displayed by the mobile terminal; According to the characteristic plane of the mobile terminal, the central axis of the characteristic plane of the mobile terminal is determined as the indicating direction. 2. The processing method according to claim 1, characterized in that, before displaying the indicator at the associated position according to the display signal provided by the mobile terminal, the processing method further comprises: The display signal sent by the mobile terminal is received. 3. The processing method according to claim 1, wherein, according to the characteristic plane of the mobile terminal, determining the central axis of the characteristic plane of the mobile terminal as the indicating direction comprises: Using a straight line detection algorithm to determine the two-dimensional area equation of the geometric plane figure with the central axis; According to the two-dimensional area equation, obtain the three-dimensional area equation of the geometric plane figure with the central axis in the standard three-dimensional coordinate system; According to the three-dimensional area equation of the geometric plane figure with the central axis in the standard three-dimensional coordinate system, determine the three-dimensional line equation of the central axis of the geometric plane figure with the central axis in the standard three-dimensional coordinate system as the indication direction. 4. The processing method according to claim 3, characterized in that, according to the three-dimensional area equation of the geometric plane figure with the central axis under the standard three-dimensional coordinate system, the central axis of the geometric plane figure with the central axis is determined The three-dimensional line equation under the standard three-dimensional coordinate system is used as the indicated direction, including: When the geometric plane figure with the central axis has at least two central axes; then obtain the current indication direction of the built-in electronic compass of the mobile terminal; Select the first central axis parallel to the current indication direction of the electronic compass among at least two central axes of the geometric plane figure with the central axis, and according to the geometric plane figure with the central axis in a standard three-dimensional coordinate system The three-dimensional area equation below, and determine the three-dimensional line equation of the first central axis in the standard three-dimensional coordinate system as the indicating direction. 5. The processing method according to claim 3 or 4, characterized in that, according to the range of the indication direction and the imaging area in the current scene image, the indication direction and the imaging area are determined Associated positions located in the imaging area between, including: According to the three-dimensional line equation of the central axis of the geometric plane figure with the central axis in the standard three-dimensional coordinate system and the three-dimensional area equation of the image-presenting area in the standard three-dimensional coordinate system, the calculation is performed to obtain the three-dimensional area equation in the standard three-dimensional coordinate system. Intersection coordinates in a standard three-dimensional coordinate system, determining the intersection coordinates as the associated position. 6. The processing method according to claim 3, further comprising: before determining the indication direction of the mobile terminal according to the current scene image: Obtain the calibration parameter information according to the preset sampling detection point; determining the standard three-dimensional coordinate system according to the calibration parameter information; A three-dimensional area equation of the imaging area in the standard three-dimensional coordinate system is determined according to the calibration parameter information and the standard three-dimensional coordinate system. 7. A processing device for displaying indication marks, characterized in that it comprises: An acquisition module, configured to acquire a current scene image including a mobile terminal held by a human hand and the human hand and the imaging area; A first determining module, configured to determine an indication direction of the mobile terminal pointing to the imaging area according to the current scene image; The second determination module is configured to determine the associated position between the indication direction and the imaging area in the imaging area according to the indication direction and the range of the imaging area in the current scene image; A control module, configured to display an indication mark at the associated position according to the display signal provided by the mobile terminal; The first determination module includes: A human hand recognition unit, configured to determine the position of the human hand in the current scene image according to preset human hand recognition features; A characteristic plane determining unit, configured to determine a characteristic plane of the mobile terminal held by the human hand by using an edge detection operator based on the position of the human hand, the characteristic plane being a geometrical axis of the mobile terminal with a central axis A plane figure, or the feature plane is a geometric plane figure with a central axis displayed on the mobile terminal; The pointing direction determining unit is configured to determine the central axis of the characteristic plane of the mobile terminal as the pointing direction according to the characteristic plane of the mobile terminal. 8. The processing device according to claim 7, wherein the processing device further comprises: The receiving module is configured to receive the display signal sent by the mobile terminal. 9. The processing device according to claim 7, wherein the pointing direction determining unit comprises: The first equation determination subunit is used to determine the two-dimensional area equation of the geometric plane figure with the central axis by using a straight line detection algorithm; The second equation determination subunit is used to obtain the three-dimensional area equation of the geometric plane figure with the central axis in the standard three-dimensional coordinate system according to the two-dimensional area equation; Indicate direction determination subunit, used to determine the central axis of the geometric plane figure with central axis in the standard three-dimensional coordinate system according to the three-dimensional area equation of the geometric plane figure with central axis in the standard three-dimensional coordinate system The three-dimensional straight line equation serves as the indicated direction. 10. The processing device according to claim 9, wherein when the geometric plane figure with a central axis has at least two central axes; The current indication direction of the compass; select the first central axis parallel to the current indication direction of the electronic compass among at least two central axes of the geometric plane figure with the central axis, and according to the characteristic plane in the standard three-dimensional coordinates The three-dimensional area equation in the standard three-dimensional coordinate system is used to determine the three-dimensional line equation of the first central axis in the standard three-dimensional coordinate system as the indicating direction. 11. The processing device according to claim 9 or 10, characterized in that, the second determination module is specifically configured to, according to the central axis of the geometric planar figure with a central axis in the standard three-dimensional coordinate system The three-dimensional line equation of the image forming area is calculated with the three-dimensional area equation in the standard three-dimensional coordinate system to obtain the intersection coordinates in the standard three-dimensional coordinate system, and the intersection coordinates are determined as the associated position. 12. The processing device according to claim 9, wherein the processing device further comprises: The third determination module is used to obtain calibration parameter information according to preset sampling detection points before determining the indication direction of the mobile terminal according to the current scene image; determine the standard three-dimensional coordinate system according to the calibration parameter information; A three-dimensional area equation of the imaging area in the standard three-dimensional coordinate system is determined according to the calibration parameter information and the standard three-dimensional coordinate system. 13. A processing system for displaying indication marks, characterized in that it includes a display device, a mobile terminal, and a control device; the control device includes the processing device for displaying indication marks according to any one of claims 7 to 12; the The display device has an image display area for displaying information; the mobile terminal and the display device can respectively perform data transmission with the control device through a communication interface.