WO2021138856A1 - Camera control method, device, and computer readable storage medium - Google Patents

Abstract

The present application provides a camera control method, comprising: obtaining a pixel mapping relationship between the current camera and a target camera; determining a target position of a target object in a picture of the current camera; and determining a switching angle of a center point of the target camera according to the pixel mapping relationship and the target position. Therefore, the accuracy of camera control is improved. The present application further provides a device and a computer readable storage medium.

Description

Camera control method, equipment and computer readable storage medium Technical field

This application relates to the field of device technology, and in particular to a camera control method, device, and computer-readable storage medium.

Background technique

In a multi-camera system composed of multiple cameras, for example, multi-camera mobile phones, multi-camera surveillance cameras, etc., when a user finds an object of interest in a camera screen, he needs to quickly switch to another camera for the object When performing observation and shooting, for example, when you find an object of interest on a wide-angle camera, you need to switch to a telephoto camera or a laser camera if you want to see the details of the object. Generally, you need to manually adjust the camera position first, so that the object is on the wide-angle camera screen. Perform centering calibration inside, and then perform corresponding operations such as zooming on the object through the telephoto camera, or perform corresponding operations such as distance measurement on the object through the laser camera. Because the user cannot accurately place the object in the center of the field of view on the wide-angle camera by manual operation, and requires the user to manually adjust the position of the camera repeatedly to complete the specific observation or measurement of the object, not only the operation process is very cumbersome, but the accuracy of the calibration object is relatively high. low.

Summary of the invention

The embodiments of the present application provide a camera control method, device, and computer-readable storage medium, which can improve the convenience of coordinated control of multiple cameras and improve the accuracy of controlling the target object at the center of the screen.

In the first aspect, an embodiment of the present application provides a camera control method, including:

Obtain the pixel mapping relationship between the current camera and the target camera;

Determining the target position of the target object in the current camera frame;

The switching angle of the center point of the target camera is determined according to the pixel mapping relationship and the target position.

In the second aspect, an embodiment of the present application also provides a camera control method, including:

Obtain the pixel mapping relationship between the current camera and the target camera in the multi-camera system;

Acquiring, according to the pixel mapping relationship, the mapping position of the center of the frame of the target camera in the current camera frame;

Determining the target position of the target object in the current camera frame;

Determining the deflection angle of the pan/tilt equipped with the target camera according to the mapping position and the target position;

The deflection of the pan/tilt is controlled according to the deflection angle, so that the target object is located at the center of the frame of the target camera after the deflection.

In the third aspect, an embodiment of the present application also provides a camera control method, including:

Control the current camera in the multi-camera system to capture a picture and display the picture;

Receiving a selection instruction input by the user based on the displayed screen;

Determining the target position of the target object in the current camera frame according to the selection instruction;

Acquiring the mapping position of the frame center of the target camera in the current camera frame in the multi-camera system;

Determining the deflection angle of the target camera according to the mapping position and the target position;

The target camera is controlled to deflect according to the deflection angle, so that the target object is located at the center of the frame of the target camera after the deflection.

In a fourth aspect, an embodiment of the present application also provides a multi-camera system, and the multi-camera system includes:

Multiple cameras

Memory, used to store computer programs;

The processor is used to call the computer program in the memory for executing:

Obtain the pixel mapping relationship between the current camera and the target camera;

Determining the target position of the target object in the current camera frame;

The switching angle of the center point of the target camera is determined according to the pixel mapping relationship and the target position.

In a fifth aspect, an embodiment of the present application also provides a pan-tilt camera, and the pan-tilt camera includes:

Multiple cameras

PTZ for carrying the multiple cameras;

Memory, used to store computer programs;

The processor is used to call the computer program in the memory for executing:

Acquiring a pixel mapping relationship between the current camera and the target camera in the plurality of cameras;

Acquiring, according to the pixel mapping relationship, the mapping position of the center of the frame of the target camera in the current camera frame;

Determining the target position of the target object in the current camera frame;

Determining the deflection angle of the pan/tilt equipped with the target camera according to the mapping position and the target position;

The deflection of the pan/tilt is controlled according to the deflection angle, so that the target object is located at the center of the frame of the target camera after the deflection.

In a sixth aspect, an embodiment of the present application also provides a movable platform, including:

PTZ, installed on the body of the movable platform;

Multiple cameras mounted on the pan/tilt;

Memory, used to store computer programs;

The processor is configured to call a computer program in the memory to execute any camera control method provided in the embodiments of the present application.

In a seventh aspect, the embodiments of the present application also provide a computer-readable storage medium, where the computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute any of the provided by the embodiments of the present application. A camera control method.

The embodiment of the application can obtain the pixel mapping relationship between the current camera and the target camera, and determine the target position of the target object in the current camera frame, and then determine the switching angle of the center point of the target camera according to the pixel mapping relationship and the target position, so as Subsequently, the target camera can be controlled to switch the center point according to the switching angle, so that the target object is located at the center of the screen of the target camera after the switch. The solution determines the switching angle of the target camera through the pixel mapping relationship between different cameras and the target position, so that the target object can be located in the center of the target camera screen after switching based on the switching angle, which improves the convenience and efficiency of coordinated control of multiple cameras , And improve the accuracy of controlling the target object in the center of the target camera frame, that is, improving the accuracy and convenience of controlling the camera.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic diagram of the structure of a pan-tilt camera provided by an embodiment of the present application;

FIG. 2 is another schematic diagram of the structure of a pan-tilt camera provided by an embodiment of the present application;

FIG. 3 is another schematic diagram of the structure of a pan-tilt camera provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of the interaction between the aircraft, the remote control device, and the display device provided by an embodiment of the present application;

FIG. 5 is a schematic flowchart of a camera control method provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of selecting a target object based on a selection instruction provided by an embodiment of the present application;

FIG. 7 is another schematic diagram of selecting a target object based on a selection instruction provided by an embodiment of the present application;

FIG. 8 is another schematic diagram of selecting a target object based on a selection instruction provided by an embodiment of the present application;

9 is a schematic diagram of determining the offset of the target object to the center of the screen provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of determining a switching angle based on an offset provided by an embodiment of the present application;

FIG. 11 is a schematic diagram of controlling a target camera to switch according to a switching angle according to an embodiment of the present application;

FIG. 12 is another schematic flowchart of a camera control method provided by an embodiment of the present application;

FIG. 13 is another schematic flowchart of a camera control method provided by an embodiment of the present application;

FIG. 14 is a schematic structural diagram of a multi-camera system provided by an embodiment of the present application;

15 is a schematic diagram of the structure of a pan-tilt camera provided by an embodiment of the present application;

FIG. 16 is a schematic structural diagram of a movable platform provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The flowchart shown in the drawings is only an example, and does not necessarily include all contents and operations/steps, nor does it have to be executed in the described order. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to actual conditions.

Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

The embodiments of the present application provide a camera control method, device, and computer-readable storage medium, which are used to perform coordinated control of multiple cameras in a multi-camera system, and improve the accuracy and convenience of camera control.

Among them, the device may include a multi-camera system, a pan-tilt camera, a removable platform, and a computer-readable storage medium, etc., which can be used to obtain the pixel mapping relationship between the current camera and the target camera, and to obtain the target camera according to the pixel mapping relationship The center of the screen is the mapped position in the current camera screen, and then the target position of the target object in the current camera screen can be determined, and the switching angle of the target camera can be determined according to the mapped position and target position. At this time, the target camera can be controlled to switch according to the switching angle , So that the target object is located in the center of the screen of the target camera after switching, which improves the accuracy and convenience of camera control.

Among them, the multi-camera system can include multiple cameras, and the type of each camera, the number of cameras, the setting position of the camera, the relative geometric position relationship between each camera, etc. can be flexibly set according to actual needs, for example, a multi-camera system It can include ultra-wide-angle cameras, wide-angle cameras, telephoto cameras (ie zoom cameras), infrared cameras, lidar cameras, far-infrared cameras, ultraviolet cameras, laser ranging cameras, and TOF (Time of Flight) depth Camera (referred to as TOF depth camera) and so on.

A pan-tilt camera may include multiple cameras and one or more pan-tilts, etc. One pan-tilt may be used to carry multiple cameras, or multiple pan-tilts may be used to carry multiple cameras. Taking a pan/tilt as an example, as shown in FIG. 1, the pan/tilt camera may include a telephoto camera 101a, a wide-angle camera 101b, an infrared camera 101c, and a lidar camera 101d. For another example, as shown in FIG. 2, the pan-tilt camera may include a lidar camera 102a, a laser ranging camera 102b, and a TOF depth camera 102c. For another example, as shown in FIG. 3, the pan/tilt camera may include a lidar camera 103a, a wide-angle camera 103b, and a telephoto camera 103c. Among them, the gimbal of the gimbal camera can be a three-axis gimbal or a two-axis gimbal, etc., for example, the gimbal can rotate in the pitch direction (that is, rotate around the Y axis), and rotate in the yaw direction (that is, Rotate around the X axis), and rotate in the roll direction (ie, rotate around the Z axis); or, the pan/tilt head can rotate in the pitch direction and in the yaw direction.

The movable platform may include one or more pan-tilts and multiple cameras, etc., one or more pan-tilts may be installed on the body of the movable platform, and the multiple cameras may be mounted on the one or more pan-tilts Above, the type of movable platform can be flexibly set according to actual needs. For example, the movable platform may be a mobile terminal, an aircraft, a robot or an unmanned vehicle, etc. The aircraft may include a UAV, which may include a rotary-wing UAV (such as a quadrotor UAV, a hexarotor UAV). Aircraft, or eight-rotor drones, etc.), fixed-wing drones, or a combination of rotary-wing and fixed-wing drones, which are not limited here.

As shown in Figure 4, taking the mobile platform as an aircraft as an example, the aircraft is equipped with a multi-camera system containing multiple cameras. The aircraft is also connected with remote control devices and display devices. Among them, the remote control devices may include one or more A remote controller is used to control the flight of the aircraft or perform corresponding actions, and obtain corresponding motion information from the aircraft. The motion information can include flight direction, flight attitude, flight altitude, flight speed and position information, etc., and will obtain The exercise information is sent to the display device, and the display device is used for analysis and display. The remote control device can also be used to control one or more of the multiple cameras on the aircraft to perform operations such as shooting or measuring.

The display device may include a notebook computer, a tablet computer, or a mobile phone, etc., and may be used to receive images (ie, pictures) sent by one or more of the multiple cameras on the aircraft, and display the images on the display screen. The display device may also receive a control instruction input by the user, and perform corresponding control on multiple cameras on the aircraft based on the control instruction. For example, the display device can receive shooting instructions or ranging instructions, and send the shooting instructions or ranging instructions to the aircraft. The aircraft can control the target camera to shoot the captured images according to the shooting instructions, or control the target camera to capture images according to the ranging instructions. The target object performs ranging and so on.

It should be noted that the device structures in FIGS. 1 to 4 do not constitute a limitation on the application scenario of the camera control method.

Please refer to FIG. 5. FIG. 5 is a schematic flowchart of a camera control method provided by an embodiment of the present application. The camera control method can be applied to a multi-camera system capable of controlling the shooting direction of the camera, especially the shooting direction of the target camera, for coordinated control of multiple cameras, so as to improve the accuracy, efficiency and convenience of camera control.

As shown in FIG. 5, the camera control method may include steps S101 to S103, etc., which may be specifically as follows:

S101: Acquire a pixel mapping relationship between the current camera and the target camera in the multi-camera system.

Among them, the multi-camera system can be a multi-camera system containing multiple different types of cameras, and the pixel mapping relationship can be between two or more different cameras (for example, between an infrared camera and a wide-angle camera, between a wide-angle camera and a telephoto camera, Between infrared camera and telephoto camera, etc.) the corresponding relationship between image pixels is collected. This pixel mapping relationship can be called a homography matrix or conversion relationship between different camera pixels. The conversion relationship can include a rotation relationship and a translation relationship. Etc., the pixel mapping relationship can be represented by a mapping matrix.

You can select any camera from the multi-camera system as the current camera, and select one or more cameras as the target camera. The current camera can be used to capture the current frame to find the target object from the current frame and switch the target object to The center image of one or more target cameras is displayed. The target camera can collect the target based on the user's selection of the target object in the current camera, or based on the determined target object type (such as license plate, person, dog, or flower, etc.) The object is in the center of the picture. Among them, the target object can be flexibly selected or set according to actual needs, such as any object such as people, animals, plants, vehicles, buildings, or stars, or even non-solid, non-three-dimensional targets, such as flames, clouds, or water bodies, etc. As long as the target can be imaged via one or more pixels of the camera's photosensitive element. There can be multiple examples of a multi-camera system. For example, a wide-angle camera can be selected as the current camera, and a telephoto camera can be selected as the target camera; or, a wide-angle camera can be selected as the current camera, and a laser rangefinder camera can be selected as the target camera; and so on.

After determining the current camera and the target camera, the pixel mapping relationship between the current camera and the target camera can be obtained. For example, the camera geometric parameter calibration method can be used to calibrate the internal and external parameters of the camera geometry to obtain the conversion relationship between each camera. Thus, the pixel mapping relationship between the current camera and the target camera can be obtained.

In some embodiments, obtaining the pixel mapping relationship between the current camera and the target camera may include: obtaining the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera; according to the internal parameter matrix of the current camera, and the target camera's internal parameter matrix. The internal parameter matrix and the external parameter matrix determine the conversion matrix between the current camera and the target camera, and obtain the pixel mapping relationship between the current camera and the target camera.

In order to improve the accuracy and efficiency of obtaining the pixel mapping relationship, the pixel mapping relationship between the current camera and the target camera can be obtained through the internal and external parameter matrices (ie, the internal parameter matrix and the external parameter matrix) of the current camera and the target camera.

Specifically, the internal parameter matrix of the current camera and the internal parameter matrix and the external parameter matrix of the target camera can be obtained. In some embodiments, obtaining the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera may include: obtaining the positional relationship between the current camera and the target camera in the multi-camera system; obtaining the internal parameter of the current camera according to the positional relationship, And obtain the internal parameters and external parameters of the target camera; construct the internal parameter matrix of the current camera according to the internal parameters of the current camera; construct the internal parameter matrix of the target camera according to the internal parameters of the target camera, and construct the external parameter matrix of the target camera according to the external parameters of the target camera.

In order to improve the accuracy and flexibility of obtaining the internal and external parameter matrix, the internal and external parameter matrix can be constructed based on the position relationship between the current camera and the target camera, and the internal and external parameters (ie, internal and external parameters). The position relationship may be the geometric position relationship between the current camera and the target camera in the multi-camera system. In some embodiments, the relative positions of the current camera and the target camera in the multi-camera system are fixed, which can reduce the complexity of the multi-camera system and reduce the computing resource consumption of the multi-camera system; in some embodiments, the current camera and the target camera The relative position of the camera in the multi-camera system is variable. For example, after the target object is determined in the current camera, the current camera and the target camera move separately to capture images with the target object in the center. This method can ensure a better user experience.

First, the position relationship between the current camera and the target camera in the multi-camera system can be obtained. For example, the position relationship between the current camera and the target camera in the multi-camera system recorded when the current camera and the target camera are installed can be obtained from a database. Or, in some embodiments, acquiring the positional relationship between the current camera and the target camera in the multi-camera system may include: acquiring a calibration image, extracting feature points in the calibration image; and determining the location of the feature points through the current camera and the target camera respectively; According to the position of the feature point, the position relationship between the current camera and the target camera is established in the multi-camera system, so that both the current camera and the target camera can collect the feature points.

In order to improve the accuracy of obtaining the position relationship, the position relationship between the current camera and the target camera can be determined through feature point calibration. For example, a calibration image can be obtained, and the calibration image can be flexibly set according to actual needs. For example, the calibration image can be an image of a collected calibration plate, and the calibration image can contain feature points. The location, size, and type of the feature points can be It can be set flexibly according to actual needs. For example, the feature point can be the vertex of the grid in the calibration image or the contour of the object, and the feature point can be extracted from the calibration image. Then, the location of the feature point is determined by the current camera and the target camera respectively, and the position relationship between the current camera and the target camera is established in the multi-camera system according to the location of the feature point, so that both the current camera and the target camera can collect the feature points .

Then, after obtaining the positional relationship between the current camera and the target camera, the internal parameters of the current camera and the internal and external parameters of the target camera can be obtained according to the positional relationship. Among them, the internal parameters can include the focal length (fx, fy), the optical center position (cx, cy), and the distortion parameter distortion_coeff, etc., so as to achieve accuracy through the geometric parameter calibration method (for example, a variant method based on Zhang Zhengyou’s checkerboard calibration algorithm) Get the internal parameters of the current camera, the internal parameters and external parameters of the target camera, etc.

Secondly, construct the internal parameter matrix of the current camera according to the internal parameters of the current camera, and construct the internal parameter of the target camera according to the internal parameters of the target camera. For example, if the current camera is a wide-angle camera, the internal parameter matrix of the wide-angle camera is:

Among them, K _wide represents the internal parameter matrix of the _wide -angle camera, fx wide represents the focal length of the wide-angle camera in the X-axis direction, fy _wide represents the focal length of the wide-angle camera in the Y-axis direction, and cx _wide represents the optical center of the wide-angle camera in the X-axis direction. Position, cy _wide represents the position of the optical center of the wide-angle camera in the Y-axis direction.

If the target camera is a zoom camera (ie a telephoto camera), the internal parameter matrix of the zoom camera at 1x focal length is:

表示变焦相机在1倍焦距下的内参矩阵，

表示变焦相机在X轴方向上的焦距，

表示变焦相机在Y轴方向上的焦距，

表示变焦相机在X轴方向上的光心位置，

表示变焦相机在Y轴方向上的光心位置。 among them,

Represents the internal parameter matrix of the zoom camera at 1x focal length,

Indicates the focal length of the zoom camera in the X-axis direction,

Indicates the focal length of the zoom camera in the Y-axis direction,

Indicates the position of the optical center of the zoom camera in the X-axis direction,

Indicates the position of the optical center of the zoom camera in the Y-axis direction.

If the target camera is an infrared camera, the internal parameter matrix of the infrared camera is:

Among them, _Kir represents the internal parameter matrix of the infrared camera, fx _ir represents the focal length of the infrared camera in the X-axis direction, fy _ir represents the focal length of the infrared camera in the Y-axis direction, and cx _ir represents the optical center of the infrared camera in the X-axis direction Position, cy _ir represents the position of the optical center of the infrared camera in the Y-axis direction.

It should be noted that if the distortion of the current camera and the target camera is very small, it is not necessary to use the distortion parameters to construct the internal parameter matrix.

And, the external parameter matrix of the target camera can be constructed according to the external parameters of the target camera, and the specific value of the external parameter can be flexibly set according to actual needs. For example, the current camera can be calculated according to the external parameters of the current camera and the external parameters of the target camera. The relationship between the rotation R and the translation T between the camera and the target camera, and the external parameter matrix of the target camera is constructed according to the relationship between the rotation R and the translation T. R represents the conversion from the current camera coordinate system to the target camera coordinate system. R and T can be both It is a 3*3 matrix or other types of matrices.

For example, if the current camera is a wide-angle camera and the target camera is a zoom camera, the external parameter matrices of the wide-angle camera and the zoom camera at 1x focal length are: R _{fpv_zoom} and T _{fpv_zoom} ; for another example, if the current camera is a wide-angle camera, the target camera is an infrared Camera, the external parameter matrix between the wide-angle camera and the infrared camera is: R _{fpv_ir} and T _{fpv_ir} . Examples of the external parameter matrix may include rotation angle parameters in three directions, or translation distance parameters in three directions; or a 3*3 matrix to represent the transformation in the rotation direction.

It should be noted that for zoom cameras, since the focal length of the zoom camera is variable, when the focal length of the zoom camera changes, the internal and external parameters will also change accordingly. It is necessary to re-determine the internal parameter matrix and external parameters of the zoom camera. Parameter matrix.

In some embodiments, constructing the internal parameter matrix of the target camera according to the internal parameters of the target camera, and constructing the external parameter matrix of the target camera according to the external parameters of the target camera may include: constructing the target camera according to the internal and external parameters of the target camera at a preset focal length Obtain the current magnification of the target camera; determine the internal parameter matrix and the external parameter matrix of the target camera according to the current magnification of the target camera, the preset internal parameter matrix and the preset external parameter matrix.

Taking the target camera as a zoom camera as an example, in order to improve the accuracy and convenience of obtaining the internal parameter matrix and the external parameter matrix, you can only calibrate the internal parameter matrix and the external parameter matrix of the zoom camera at one or several magnifications. The internal parameter matrix and external parameter matrix of can be obtained by equivalent conversion (equal conversion with magnification). At this time, the internal and external parameters corresponding to the target camera under the preset multiple focal length can be obtained, and the preset multiple focal length can be flexibly set according to actual needs. For example, the preset multiple focal length can be 1x focal length. According to the internal parameters corresponding to the target camera at the preset focal length, construct the preset internal parameter matrix of the target camera at the preset focal length, and according to the external parameters corresponding to the target camera at the preset focal length, construct the target camera at the preset focal length. The preset external parameter matrix under the focal length.

Then, the current magnification of the target camera can be obtained. In some embodiments, acquiring the current magnification of the target camera may include: acquiring the current actual focal length of the target camera, and determining the current magnification of the target camera according to the actual focal length; or, detecting the physical distance between the target camera and the reference object, based on the physical distance Set the current magnification of the target camera.

In order to improve the accuracy and flexibility of the current magnification acquisition of the target camera, when the target camera is a zoom camera, the zoom camera can zoom, and every time the focal length is changed, the change of the focal length will cause the internal parameters of the zoom camera to change, so the zoom camera can be obtained The current actual focal length, according to the corresponding relationship between the focal length and the magnification, determines the magnification corresponding to the actual focal length, and obtains the current magnification of the zoom camera.

Or, you can select a reference object, which can be flexibly set according to actual needs. For example, the reference object can be an object in the picture collected by the target camera, and then the target can be detected by lidar or laser rangefinder. The physical distance between the camera and the reference object (that is, the actual distance, which can be referred to as the object distance for short), the current magnification of the target camera is automatically set according to the correspondence between the distance and the magnification, and the detected physical distance.

It should be noted that the physical distance between the target camera and the reference object can be set to infinity according to actual needs, or can be set flexibly by the user based on the application scenario.

After the current magnification of the target camera is obtained, the internal parameter matrix and the external parameter matrix of the target camera can be determined according to the current magnification of the target camera, the preset internal parameter matrix and the preset external parameter matrix.

In some embodiments, determining the internal parameter matrix of the target camera according to the current magnification of the target camera and the preset internal parameter matrix may include: obtaining the width and height of the image currently collected by the target camera; and preset internal parameter according to the current magnification of the target camera The matrix, width and height determine the internal parameter matrix of the target camera.

In order to improve the accuracy of obtaining the internal parameter matrix of the target camera, the width and height of the image currently collected by the target camera can be obtained, and the internal parameter matrix of the target camera can be determined according to the current magnification, preset internal parameter matrix, width and height of the target camera.

That is, after determining the current focal length of the target camera (here, the target camera is a zoom camera), its magnification can be calculated, and then the current internal parameter matrix of the target camera can be calculated as follows:

表示目标相机在当前的倍率下对应的内参矩阵，ratio表示倍率，

表示目标相机在1倍焦距下X轴方向上的焦距，

表示目标相机在1倍焦距下Y轴方向上的焦距，

表示目标相机在1倍焦距下X轴方向上的光心位置，

表示目标相机在1倍焦距下Y轴方向上的光心位置，W表示目标相机当前采集到的图像的宽度，H表示目标相机当前采集到的图像的高度。 among them,

Represents the internal parameter matrix corresponding to the target camera at the current magnification, ratio represents the magnification,

Indicates the focal length of the target camera in the X-axis direction at 1x focal length,

Indicates the focal length of the target camera in the Y-axis direction at 1x focal length,

Indicates the optical center position of the target camera in the X-axis direction at 1x focal length,

It represents the optical center position of the target camera in the Y-axis direction at 1x focal length, W represents the width of the image currently captured by the target camera, and H represents the height of the image currently captured by the target camera.

After the internal and external parameter matrices of the current camera and the target camera are obtained, the conversion matrix between the current camera and the target camera can be determined according to the internal parameter matrix of the current camera and the internal parameter matrix and the external parameter matrix of the target camera to obtain the current camera and the target camera The pixel mapping relationship between. For example, if the current camera is a wide-angle camera and the target camera is a zoom camera, the pixel mapping relationship Homography _{zoom_fpv} from the center point on the zoom camera to the wide-angle camera is:

Homography _{zoom_fpv} = K _wide *R*inv(K)

表示变焦相机的外参矩阵，

表示变焦相机的在倍率ratio下的内参矩阵，inv为矩阵求逆运算，当目标相机为其他相机时，R表示其他相机的外参矩阵，K表示的内参矩阵。 Among them, K _wide represents the internal parameter matrix of the wide-angle camera,

Represents the external parameter matrix of the zoom camera,

Represents the internal parameter matrix of the zoom camera under the magnification ratio, inv is the matrix inversion operation, when the target camera is another camera, R represents the external parameter matrix of other cameras, and K represents the internal parameter matrix.

It should be noted that, when the multi-camera system includes other cameras in addition to the current camera and the target camera, the pixel mapping relationship between any two cameras can be obtained in the foregoing manner.

S102: Determine the target position of the target object in the current camera frame.

Among them, the target object can be a complete object, or a point, etc., for example, the target object can be a license plate, a vehicle, a person, a tree, or a dog, etc. The target object can be flexibly set according to actual needs. The specific content is here There is no limit. The method for determining the target object can be flexibly set according to actual needs. For example, it can receive a selection instruction input by the user to determine the target object, or automatically recognize the object in the current camera screen and select one of the objects as the target object; and so on.

In some embodiments, determining the target position of the target object in the current camera frame may include: receiving a selection instruction input by the user in the frame captured by the current camera; and determining the target position of the target object in the current camera frame according to the selection instruction. .

In order to improve the flexibility of determining the target position of the target object, the screen captured by the current camera can be displayed on the display screen for the user to view. At this time, the selection instruction input by the user can be received in the screen captured by the current camera. In order to determine the target position of the target object in the current camera frame according to the selection instruction. For example, as shown in Figure 6, if there is a vehicle in the current screen captured by the camera, the user's finger clicking or pressing the license plate on the vehicle can be received (that is, a selection instruction is generated), and the user's finger clicking or pressing operation can be detected at this time The area where the license plate is located is detected and the objects in the area are detected, and the user selects the area where the license plate is located, and then the area where the license plate is located can be extracted as the target location of the target object.

For another example, as shown in Figure 7, after the current camera captures the picture, all objects in the picture can be identified one by one through the preset recognition model, a list of recognized objects is generated, and the list of objects is displayed. For example, you can Recognize vehicles, license plates, water, trees, etc. At this time, the user can receive the operation of clicking or pressing the position of the license plate in the object list (that is, generating a selection instruction), and then, based on the position of the user clicking or pressing the operation, it is determined that the user has selected the license plate, and the area where the license plate is located in the screen can be extracted as The target position of the target object.

For another example, as shown in Figure 8, a text input box can be displayed for the user to input the target object to be detected. At this time, the "license plate" input by the user in the text input box (that is, a selection instruction is generated) can be received, and then based on the user input The "license plate" confirms that the user selects the license plate. The license plate in the picture can be recognized through the preset recognition model, and the area where the license plate is located in the picture is extracted as the target location of the target object.

Among them, the preset recognition model can be flexibly set according to actual needs. For example, the recognition model can be a target detection algorithm SSD or YOLO, and the recognition model can also be a convolutional neural network R-CNN or Faster R-CNN. The preset recognition model is a trained recognition model. For example, multiple sample images containing different types of objects can be obtained, and the recognition model can be trained based on the multiple sample images to obtain the trained recognition model.

In some embodiments, determining the target object in the current camera frame may include: extracting features of the current camera frame to obtain target feature information; identifying the target object in the current camera frame according to the target feature information, and determining the target position of the target object .

In order to improve the automation and convenience of determining the target position of the target object, the target object can be automatically detected. For example, perform feature extraction on the current camera image to obtain target feature information. The target feature information can be flexibly set according to actual needs. For example, the target feature information can be the feature information of the license plate or the feature information of a person, etc., according to the target feature information Can identify the target object in the current camera screen, and determine the target position of the target object. For example, for an aircraft equipped with a multi-camera system, after the aircraft takes off, the multi-camera system can automatically perform a screen inspection. When the current camera (such as a wide-angle camera) detects a vehicle's illegal parking image, it can automatically identify the violation in the screen. The location of the license plate of the parked vehicle, so that the license plate can be sent to the target camera (for example, a telephoto camera) for centered display in the future.

S103: Determine the switching angle of the center point of the target camera according to the pixel mapping relationship and the target position.

Wherein, the switching angle may include the switching angle in the yaw direction (that is, the switching angle in the X-axis direction), the switching angle in the pitch direction of the pitch direction (that is, the switching angle in the Y-axis direction), and the roll direction in the roll direction. The switching angle (ie, the switching angle in the Z-axis direction) and so on.

In some embodiments, the camera control method further includes: obtaining the mapped position of the center of the frame of the target camera in the current camera frame according to the pixel mapping relationship.

Determining the switching angle of the center point of the target camera according to the pixel mapping relationship and the target position may include: determining the switching angle of the center point of the target camera according to the mapping position and the target position.

That is, after obtaining the mapped position of the center of the target camera's frame in the current camera frame and the target position of the target object in the current camera frame, the switching angle of the target camera's center point can be determined according to the mapped position and target position.

Specifically, after the pixel mapping relationship between the current camera and the target camera is obtained, the mapping position of the center of the target camera's frame in the current camera frame can be obtained. For example, if the current camera is a wide-angle camera and the target camera is a zoom camera, then The mapping position of the corresponding point of the zoom camera's frame center in the wide-angle camera frame is (dis_1 _x , dis_1 _y ), and the calculation method can be as follows:

表示变焦相机在1倍焦距下X轴方向上的光心位置，

表示变焦相机在1倍焦距下Y轴方向上的光心位置，W表示变焦相机当前采集到的图像的宽度，H表示变焦相机当前采集到的图像的高度。 Among them, Homography _{zoom_fpv} represents the pixel mapping relationship between the zoom camera and the wide-angle camera, and ratio represents the magnification of the zoom camera.

Indicates the position of the optical center of the zoom camera in the X-axis direction at 1x focal length,

It represents the optical center position of the zoom camera in the Y-axis direction at 1x focal length, W represents the width of the image currently captured by the zoom camera, and H represents the height of the image currently captured by the zoom camera.

In some embodiments, according to the pixel mapping relationship, obtaining the mapping position of the frame center of the target camera in the current camera frame may include: obtaining the position deviation between the frame center of the target camera and the frame center of the current camera according to the pixel mapping relationship. ; Determine the mapping position of the target camera's frame center in the current camera frame according to the position deviation.

In order to improve the accuracy of obtaining the mapping position, the mapping position can be determined according to the position deviation of the image center of different cameras. For example, according to the pixel mapping relationship between the current camera and the target camera, the image center of the target camera can be mapped to the current camera. The position deviation between the center of the screen of the target camera can then be used to determine the mapping position of the center of the target camera in the current camera screen based on the position deviation.

In some embodiments, determining the mapping position of the center of the target camera's frame in the current camera frame according to the position deviation may include: determining a reference object, and obtaining the distance from the reference object to the center of the target camera; determining the frame of the target camera according to the position deviation and distance The center of the mapping position in the current camera frame.

In order to improve the accuracy and reliability of the mapping position of the target camera's image center in the current camera image, for example, calculate the corresponding position of the zoom camera's image center on the wide-angle camera (that is, the mapping position), or the image of the wide-angle camera The corresponding position of the center on the telephoto camera, etc., to accurately calculate this corresponding position requires knowing the distance of the reference object relative to the target camera. The reference object can be flexibly set according to actual needs. For example, the reference object can be collected by the target camera. The distance between the center of the target camera and the reference object can be detected by lidar or laser rangefinder. In the simplified calculation, the distance can be assumed to be infinite, or the user can use it based on the application scenario. Flexible settings. According to the pixel mapping relationship between the current camera and the target camera, after obtaining the positional deviation between the center of the target camera and the center of the current camera, the distance between the center of the target camera and the reference object can be obtained according to the positional deviation Determine the mapping position of the target camera's frame center in the current camera frame.

In some embodiments, determining the switching angle of the center point of the target camera according to the mapping position and the target position may include: determining the offset of the target object to the center of the current camera screen according to the mapping position and the target position; and determining the target according to the offset. The switching angle of the camera center point.

In order to improve the accuracy of acquiring the switching angle, the switching angle may be determined based on the offset of the target object to the center of the screen of the current camera. For example, as shown in Figure 9, according to the mapping position (dis_1x, dis_1y) of the target camera's frame center A in the current camera frame, and the target position B of the target object in the current camera frame, determine the frame of the target object to the current camera The center offset (dis_x, dis_y), where the offset may include the offset dis_x in the X-axis direction and the offset dis_y in the Y-axis direction. The calculation formula for the offset may be as follows:

dis_x=(tar_point_x-center_point_x+dis_1x)*UNIT_CELL_SIZE

dis_y=(tar_point_y-center_point_y+dis_1y)*UNIT_CELL_SIZE

Among them, dis_x represents the offset of the target object in the X-axis direction from the center of the current camera's screen, dis_y represents the offset of the target object in the Y-axis direction from the center of the current camera's screen, and tar_point_x represents the target position B of the target object. The pixel coordinate value in the X-axis direction, center_point_x represents the coordinate value of the screen center A in the X-axis direction, dis_1x represents the mapping position of the target camera's screen center in the X-axis direction in the current camera screen, and UNIT_CELL_SIZE represents the size of a pixel (I.e. side length), tar_point_y represents the pixel coordinate value of the target position B of the target object in the Y-axis direction, center_point_y represents the coordinate value of the screen center A in the Y-axis direction, dis_1y indicates that the screen center of the target camera is in the current camera screen The mapping position in the Y-axis direction.

After the offset is obtained, the offset needs to be converted into a switching angle, and the switching angle of the center point of the target camera can be determined according to the offset. In some embodiments, the offset includes a first offset and a second offset, the switching angle includes the first switching angle and the second switching angle, and determining the switching angle of the center point of the target camera according to the offset may include: Determine the first switching angle of the center point of the target camera according to the first offset of the target object in the first direction to the center of the current camera's screen; according to the second offset of the target object in the second direction to the center of the current camera's screen The amount of shift determines the second switching angle of the center point of the target camera.

Among them, the first direction and the second direction can be flexibly set according to actual needs. For example, the first direction can be the direction on the X axis, and the second direction can be the direction on the Y axis, which is determined in order to improve the switching angle. Accuracy, after obtaining the offset (dis_x, dis_y) of the target object to the center of the current camera screen, the first offset is dis_x and the second offset is dis_y, that is, the target object is at X The first offset in the axis direction from the center of the current camera's screen is dis_x, and the second offset from the target object in the Y axis direction to the center of the current camera's screen is dis_y. At this time, as shown in Figure 10, The first switching angle angle_x of the center point of the target camera (ie a_x in Figure 10) can be determined according to the first offset dis_x of the target object in the X-axis direction to the center of the screen of the current camera, and according to the target object in the Y-axis The second offset dis_y in the direction to the center of the screen of the current camera determines the second switching angle angle_y of the center point of the target camera. The calculation formula for the switching angle can be as follows:

angle_x=atan(dis_x/current_focal_length)

angle_y=atan(dis_y/current_focal_length)

Among them, current_focal_length (that is, c_f_l in FIG. 10) represents the focal length of the target camera, and atan represents the arctangent function.

When a mechanism such as a pan/tilt is used to adjust the direction of the target camera, the conversion of angle_x and angle_y to the rotation degrees of each axis is as follows:

angle_pitch=angle_y

angle_yaw=angle_x

Among them, angle_pitch represents the angle converted in degrees in the direction of the pitch axis, and angle_yaw represents the angle converted in degrees in the direction of the yaw axis.

In summary, the overall formula for switching angle can be obtained as follows:

angle_pitch=atan(((tar_point_x-1/2+dis_1x)*UNIT_CELL_SIZE)/current_focal_length)

angle_yaw=atan(((tar_point_y-1/2+dis_1y)*UNIT_CELL_SIZE)/current_focal_length)

Among them, angle_pitch is the second switching angle angle_y, and angle_yaw is the first switching angle angle_x. The meanings of other parameters are the same as those described above, and will not be repeated here.

In some embodiments, after determining the switching angle of the center point of the target camera according to the pixel mapping relationship and the target position, the camera control method may further include: controlling the target camera to switch the center point according to the switching angle, so that the target object is located at the switched target. The center of the camera's frame.

After determining the switching angle of the center point of the target camera, the target camera can be controlled to switch the center point according to the switching angle, so that the target object is located at the center of the screen of the target camera after the switching.

For example, as shown in Figure 11, when the selected target object is a license plate, if the current camera is a wide-angle camera and the target camera is a telephoto camera, a car is displayed in the picture collected by the wide-angle camera. How much is the license plate, you can receive the location where the user clicks on the license plate, and then determine the telephoto camera center based on the mapping position of the telephoto camera's screen center in the wide-angle camera screen and the target location of the license plate in the wide-angle camera screen according to the above method Point switching angle, switch to the telephoto camera based on the switching angle, and control the telephoto camera to switch the center point, so that the license plate is located in the center of the screen of the target camera after the switch, so that the license plate can be clearly viewed through the telephoto camera at the preset magnification Guangdong B XXX. At the same time, when the target camera includes a specific type of camera, other information of the target object can be correspondingly collected, calculated, or displayed, such as the distance of the target object relative to the target camera, or the temperature of the target object, or the height information of the target object.

In some embodiments, controlling the target camera to switch the center point according to the switching angle so that the target object is located at the center of the screen of the target camera after the switching may include: controlling the target camera to switch the first switching angle in the first direction, and in the first direction. The second switching angle is switched in the two directions to obtain the switched target camera; the control target object is located at the center of the screen of the switched target camera.

In order to improve the accuracy of switching the target camera, when the switching angle includes the first switching angle and the second switching angle, the target camera can be controlled to switch the first switching angle in the first direction and the second switching angle in the second direction. , Get the target camera after switching. The first direction and the second direction can be flexibly set according to actual needs. For example, the first direction can be a direction on the X axis, and the second direction can be a direction on the Y axis. At this time, the target object can be controlled to be located in the center of the screen of the target camera after the switch, and the multi-camera system can be quickly and accurately controlled to switch to the desired screen.

In some embodiments, after controlling the target camera to switch the center point according to the switching angle, the camera control method may further include: receiving a recording instruction, a shooting instruction, a distance measurement instruction, or a temperature measurement instruction; Record the picture with the target object in the center, or control the target camera to shoot the captured picture with the target object in the center according to the shooting instruction, or control the target camera to measure the distance of the target object according to the distance measurement instruction, or control according to the temperature measurement instruction The target camera measures the temperature of the target object.

In order to improve the convenience of controlling the camera system, in a multi-camera system, the user finds a person or object of interest in a certain camera screen, and needs to quickly switch to another camera to observe and shoot the person or object For example, if you find a person or object of interest in a wide-angle camera, you need to switch to a telephoto camera if you want to see the details of the person or object; if you find an object that needs temperature measurement in the wide-angle camera, you need to switch to an infrared camera to perform For temperature measurement, an object that needs to be ranged is found on the wide-angle camera, and it needs to be switched to the laser rangefinder camera to range the object; and so on. It can automatically switch the angle of the camera, center the person or object in the switched camera, and then perform the corresponding zoom, temperature measurement, or distance measurement, etc., to achieve accurate in the switched camera (such as a wide-angle camera) Place the person or object in the center of the field of view, without having to repeatedly adjust the position of the camera, you can complete the specific observation and measurement of the person or object. Specifically, the recording instruction input by the user can be received, and the target camera can be controlled according to the recording instruction to record the captured frame of the target object in the center; or the shooting instruction input by the user can be received, and the target camera can be controlled to capture the captured image according to the shooting instruction. Take a picture with the target object in the center, or receive the distance measurement instruction input by the user, and control the target camera according to the distance measurement instruction to measure the distance of the target object; or receive the temperature measurement instruction input by the user and control the target camera according to the temperature measurement instruction Measure the temperature of the target object; or, receive a focus instruction input by the user, and control the target camera to focus on the target object according to the focus instruction; or receive a zoom instruction input by the user, and control the target camera to zoom the target object according to the zoom instruction. Therefore, the coordinated operation between multiple cameras can be quickly completed, the linkage of multiple cameras can be realized, and the efficiency of camera control can be improved.

The embodiment of the application can obtain the pixel mapping relationship between the current camera and the target camera, and determine the target position of the target object in the current camera frame, and then determine the switching angle of the center point of the target camera according to the pixel mapping relationship and the target position, so as Subsequently, the target camera can be controlled to switch the center point according to the switching angle, so that the target object is located at the center of the screen of the target camera after the switch. The solution determines the switching angle of the target camera through the pixel mapping relationship between different cameras and the target position, so that the target object can be located in the center of the target camera screen after switching based on the switching angle, which improves the convenience of coordinated control of multiple cameras, and The accuracy of controlling the target object at the center of the screen is improved, that is, the accuracy and convenience of controlling the camera are improved.

According to the camera control method described in the foregoing embodiment, the following will further describe in detail.

Please refer to FIG. 12, which is a schematic diagram of another process of a camera control method provided by an embodiment of this application. The camera control method can be applied to a multi-camera system (that is, a multi-camera system with a pan-tilt) that uses a pan-tilt to control the shooting direction of the camera, particularly to control the shooting direction of the target camera, which may include one or more pan-tilts; for example, using One PTZ controls the shooting direction of the current camera, while using multiple PTZ to adjust the shooting direction of multiple target cameras; or using a robotic arm to control the shooting direction of the current camera, while using a PTZ to control the shooting direction of the target camera, etc. The above-mentioned multi-camera system is used for coordinated control of multiple cameras to improve the accuracy, efficiency and convenience of camera control. As shown in FIG. 12, the process of the camera control method may be as follows:

S201: Acquire a pixel mapping relationship between the current camera and the target camera in the multi-camera system.

S202: According to the pixel mapping relationship, obtain the mapping position of the frame center of the target camera in the current camera frame.

S203: Determine the target position of the target object in the current camera frame.

S204: Determine the deflection angle of the pan/tilt equipped with the target camera according to the mapping position and the target position.

S205: Control the deflection of the pan/tilt according to the deflection angle, so that the target object is located at the center of the frame of the target camera after the deflection.

At this time, the deflection angle of the pan/tilt includes angle_pitch and angle_yaw described above.

In some embodiments, according to the pixel mapping relationship, obtaining the mapping position of the frame center of the target camera in the current camera frame may include: obtaining the position deviation between the frame center of the target camera and the frame center of the current camera according to the pixel mapping relationship. ; Determine the mapping position of the target camera's frame center in the current camera frame according to the position deviation.

In some embodiments, determining the mapping position of the center of the target camera's frame in the current camera frame according to the position deviation may include: determining a reference object, and obtaining the distance from the reference object to the center of the target camera; determining the frame of the target camera according to the position deviation and distance The center of the mapping position in the current camera frame.

In some embodiments, determining the target position of the target object in the current camera frame may include: receiving a selection instruction input by the user in the frame captured by the current camera; and determining the target position of the target object in the current camera frame according to the selection instruction. .

In some embodiments, determining the target object in the current camera frame may include: extracting features of the current camera frame to obtain target feature information; identifying the target object in the current camera frame according to the target feature information, and determining the target position of the target object .

In some embodiments, obtaining the pixel mapping relationship between the current camera and the target camera may include: obtaining the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera; according to the internal parameter matrix of the current camera, and the target camera's internal parameter matrix. The internal parameter matrix and the external parameter matrix determine the conversion matrix between the current camera and the target camera, and obtain the pixel mapping relationship between the current camera and the target camera.

In some embodiments, obtaining the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera may include: obtaining the positional relationship between the current camera and the target camera in the multi-camera system; obtaining the internal parameter of the current camera according to the positional relationship, And obtain the internal parameters and external parameters of the target camera; construct the internal parameter matrix of the current camera according to the internal parameters of the current camera; construct the internal parameter matrix of the target camera according to the internal parameters of the target camera, and construct the external parameter matrix of the target camera according to the external parameters of the target camera.

In some embodiments, acquiring the positional relationship between the current camera and the target camera in the multi-camera system may include: acquiring a calibration image, extracting feature points in the calibration image; determining the location of the feature points through the current camera and the target camera respectively; according to the features For the position of the point, the positional relationship between the current camera and the target camera is established in the multi-camera system, so that both the current camera and the target camera can collect feature points.

In some embodiments, constructing the internal parameter matrix of the target camera according to the internal parameters of the target camera, and constructing the external parameter matrix of the target camera according to the external parameters of the target camera may include: constructing the target camera according to the internal and external parameters of the target camera at a preset focal length Obtain the current magnification of the target camera; determine the internal parameter matrix and the external parameter matrix of the target camera according to the current magnification of the target camera, the preset internal parameter matrix and the preset external parameter matrix.

In some embodiments, acquiring the current magnification of the target camera may include: acquiring the current actual focal length of the target camera, and determining the current magnification of the target camera according to the actual focal length; or, detecting the physical distance between the target camera and the reference object, based on the physical distance Set the current magnification of the target camera.

In some embodiments, determining the internal parameter matrix of the target camera according to the current magnification of the target camera and the preset internal parameter matrix may include: obtaining the width and height of the image currently collected by the target camera; and preset internal parameter according to the current magnification of the target camera The matrix, width and height determine the internal parameter matrix of the target camera.

In some embodiments, determining the deflection angle of the pan/tilt equipped with the target camera according to the mapping position and the target position may include: determining the offset of the target object to the center of the current camera's screen according to the mapping position and the target position; and according to the offset Determine the deflection angle of the gimbal with the target camera.

In some embodiments, the offset includes a first offset and a second offset, the yaw angle includes a pitch yaw angle and a yaw yaw angle, and determining the yaw angle of the pan/tilt equipped with the target camera according to the offset may include : According to the first offset of the target object in the first direction to the center of the screen of the current camera, determine the pitch and deflection angle of the pan/tilt equipped with the target camera; according to the first offset of the target object in the second direction to the center of the screen of the current camera The second offset is to determine the yaw deflection angle of the gimbal equipped with the target camera.

In some embodiments, controlling the yaw of the gimbal according to the yaw angle so that the target object is located in the center of the image of the target camera after the deflection may include: controlling the gimbal to yaw the pitch yaw angle in the pitch direction and yaw the yaw direction. The heading deflection angle is used to obtain the deflected target camera; the target object is controlled to be located at the center of the frame of the deflected target camera.

Wherein, the deflection angle can be understood according to the above-mentioned switching angle, and the calculation method of the deflection angle is similar to the above-mentioned switching angle calculation method, and will not be repeated here.

In some embodiments, after controlling the deflection of the pan/tilt according to the deflection angle, the camera control method may further include: receiving a recording instruction, a shooting instruction, a distance measurement instruction, or a temperature measurement instruction; and controlling the target camera to locate the collected target object according to the recording instruction. Record the center image, or control the target camera according to the shooting instruction to shoot the captured image of the target object in the center, or control the target camera to measure the distance of the target object according to the distance measurement instruction, or control the target camera to measure the distance according to the temperature measurement instruction The temperature of the target object is measured.

In some embodiments, the relative positions of the current camera and the target camera in the multi-camera system are fixed.

In some embodiments, the current camera is also mounted on the pan-tilt.

In the foregoing embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in an embodiment, please refer to the detailed description of the camera control method above, which will not be repeated here.

The embodiment of the application can obtain the pixel mapping relationship between the current camera and the target camera in the multi-camera system, and according to the pixel mapping relationship, obtain the mapping position of the frame center of the target camera in the current camera frame, and then determine the current camera frame The target position of the target object is determined according to the mapping position and the target position to determine the deflection angle of the pan/tilt equipped with the target camera. At this time, the deflection of the pan/tilt can be controlled according to the deflection angle, so that the target object is located at the center of the screen of the deflected target camera. This solution determines the mapping position through the pixel mapping relationship between different cameras, and determines the deflection angle of the PTZ equipped with the target camera based on the mapping position and the target position, so that the target object can be located in the center of the target camera's screen after the deflection. The convenience and efficiency of multi-camera coordinated control, and the accuracy of controlling the target object in the center of the target camera screen is improved.

According to the camera control method described in the foregoing embodiment, the following will further describe in detail.

Please refer to FIG. 13, which is a schematic diagram of another flow of a camera control method provided by an embodiment of the application. The camera control method can be applied to a multi-camera system, a pan-tilt camera, or a movable platform, etc., as shown in FIG. 13, The process of the camera control method can be as follows:

S301: Control the current camera in the multi-camera system to capture images and display the images.

S302: Based on the displayed screen, receive a selection instruction input by the user.

S303: Determine the target position of the target object in the current camera frame according to the selection instruction.

S304: Obtain a mapping position of the center of the target camera frame in the current camera frame in the multi-camera system.

S305: Determine the deflection angle of the target camera according to the mapping position and the target position.

S306. Control the target camera to deflect according to the deflection angle, so that the target object is located at the center of the frame of the target camera after the deflection.

In some embodiments, based on the displayed screen, receiving the selection instruction input by the user may include: receiving a click operation input by the user in the displayed screen; determining the area of the click operation in the screen, and generating the selection instruction according to the area.

In some embodiments, based on the displayed screen, receiving a selection instruction input by the user may include: extracting features of the screen to obtain target feature information; determining objects in the screen according to the target feature information to obtain a collection of objects; The object identification list of the object set is displayed inside; the selection instruction input by the user in the object identification list is received.

In some embodiments, after controlling the target camera to deflect according to the deflection angle, the camera control method may further include: receiving a recording instruction, a shooting instruction, a distance measurement instruction, or a temperature measurement instruction; Recording, or control the target camera to shoot the captured image according to the shooting instruction, or control the target camera to measure the distance of the target object according to the distance measurement instruction, or control the target camera to measure the temperature of the target object according to the temperature measurement instruction.

In some embodiments, obtaining the mapping position of the center of the target camera in the multi-camera system in the current camera frame may include: obtaining the pixel mapping relationship between the current camera and the target camera in the multi-camera system; and obtaining the pixel mapping relationship according to the pixel mapping relationship. The mapping position of the target camera's frame center in the current camera frame.

In some embodiments, according to the pixel mapping relationship, obtaining the mapping position of the frame center of the target camera in the current camera frame may include: obtaining the position deviation between the frame center of the target camera and the frame center of the current camera according to the pixel mapping relationship. ; Determine the mapping position of the target camera's frame center in the current camera frame according to the position deviation.

In some embodiments, determining the mapping position of the center of the target camera's frame in the current camera frame according to the position deviation may include: determining a reference object, and obtaining the distance from the reference object to the center of the target camera; determining the frame of the target camera according to the position deviation and distance The center of the mapping position in the current camera frame.

In some embodiments, obtaining the pixel mapping relationship between the current camera and the target camera may include: obtaining the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera; according to the internal parameter matrix of the current camera, and the target camera's internal parameter matrix. The internal parameter matrix and the external parameter matrix determine the conversion matrix between the current camera and the target camera, and obtain the pixel mapping relationship between the current camera and the target camera.

In some embodiments, obtaining the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera may include: obtaining the positional relationship between the current camera and the target camera in the multi-camera system; obtaining the internal parameter of the current camera according to the positional relationship, And obtain the internal parameters and external parameters of the target camera; construct the internal parameter matrix of the current camera according to the internal parameters of the current camera; construct the internal parameter matrix of the target camera according to the internal parameters of the target camera, and construct the external parameter matrix of the target camera according to the external parameters of the target camera.

In some embodiments, acquiring the positional relationship between the current camera and the target camera in the multi-camera system may include: acquiring a calibration image, extracting feature points in the calibration image; determining the location of the feature points through the current camera and the target camera respectively; according to the features For the position of the point, the positional relationship between the current camera and the target camera is established in the multi-camera system, so that both the current camera and the target camera can collect feature points.

In some embodiments, constructing the internal parameter matrix of the target camera according to the internal parameters of the target camera, and constructing the external parameter matrix of the target camera according to the external parameters of the target camera may include: constructing the target camera according to the internal and external parameters of the target camera at a preset focal length Obtain the current magnification of the target camera; determine the internal parameter matrix and the external parameter matrix of the target camera according to the current magnification of the target camera, the preset internal parameter matrix and the preset external parameter matrix.

In some embodiments, acquiring the current magnification of the target camera may include: acquiring the current actual focal length of the target camera, and determining the current magnification of the target camera according to the actual focal length; or, detecting the physical distance between the target camera and the reference object, based on the physical distance Set the current magnification of the target camera.

In some embodiments, determining the internal parameter matrix of the target camera according to the current magnification of the target camera and the preset internal parameter matrix may include: obtaining the width and height of the image currently collected by the target camera; and preset internal parameter according to the current magnification of the target camera The matrix, width and height determine the internal parameter matrix of the target camera.

In some embodiments, determining the deflection angle of the target camera according to the mapping position and the target position may include: determining the offset of the target object to the center of the screen of the current camera according to the mapping position and the target position; and determining the offset of the target camera according to the offset. Deflection angle.

In some embodiments, the offset includes a first offset and a second offset, the deflection angle includes the first deflection angle and the second deflection angle, and determining the deflection angle of the target camera according to the offset may include: The first offset of the object in the first direction to the center of the current camera's screen determines the first deflection angle of the target camera; determines the second offset of the target object in the second direction to the center of the current camera's screen The second deflection angle of the target camera.

Wherein, the deflection angle can be understood according to the above-mentioned switching angle, and the calculation method of the deflection angle is similar to the above-mentioned switching angle calculation method, and will not be repeated here.

In some embodiments, controlling the target camera to deflect according to the deflection angle so that the target object is located in the center of the deflected target camera frame may include: controlling the target camera to deflect in a first direction by a first deflection angle, and in a second direction Deflection upward to the second deflection angle to obtain the deflected target camera; control the target object to be located at the center of the frame of the deflected target camera.

In the foregoing embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in an embodiment, please refer to the detailed description of the camera control method above, which will not be repeated here.

The embodiment of the present application can control the current camera in the multi-camera system to collect a picture, display the picture, and then receive a selection instruction input by the user based on the displayed picture. Secondly, according to the selection instruction, the target position of the target object in the current camera frame is determined, and the mapping position of the center of the target camera frame in the current camera frame in the multi-camera system is obtained. At this time, the deflection angle of the target camera can be determined according to the mapping position and the target position, and the target camera can be controlled to deflect according to the deflection angle, so that the target object is located at the center of the screen of the target camera after the deflection. The solution determines the deflection angle of the target camera by mapping the position and the target position, so that the target object can be located in the center of the target camera's screen after the deflection, which improves the convenience and efficiency of the coordinated control of multiple cameras, and improves the control of the target object at the target The accuracy of the center of the camera frame.

Please refer to FIG. 14, which is a schematic block diagram of a multi-camera system according to an embodiment of the present application. The multi-camera system 11 may include a processor 111 and a memory 112, and the processor 111 and the memory 112 are connected by a bus, such as an I2C (Inter-integrated Circuit) bus.

Specifically, the processor 111 may be a micro-controller unit (Micro-controller Unit, MCU), a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.

Specifically, the memory 112 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk, etc., and may be used to store computer programs.

The multi-camera system 11 may also include multiple cameras, and the type of each camera can be flexibly set according to actual needs.

The processor 111 is configured to call a computer program stored in the memory 112, and implement the camera control method provided in the embodiment of the present application when the computer program is executed. For example, the following steps may be performed:

Obtain the pixel mapping relationship between the current camera and the target camera, determine the target position of the target object in the current camera frame, and determine the switching angle of the center point of the target camera according to the pixel mapping relationship and the target position.

In the foregoing embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in an embodiment, please refer to the detailed description of the camera control method above, which will not be repeated here.

Please refer to FIG. 15. FIG. 15 is a schematic block diagram of a pan-tilt camera according to an embodiment of the present application. The pan/tilt camera 12 may include a processor 121 and a memory 122, and the processor 121 and the memory 122 are connected by a bus, such as an I2C (Inter-integrated Circuit) bus.

Specifically, the processor 121 may be a micro-controller unit (MCU), a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.

Specifically, the memory 122 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk, etc., and may be used to store computer programs.

The pan-tilt camera 12 may also include a pan-tilt and multiple cameras. The type of each camera can be flexibly set according to actual needs. The pan-tilt is used to carry multiple cameras.

The processor 121 is configured to call a computer program stored in the memory 122, and implement the camera control method provided in the embodiment of the present application when the computer program is executed. For example, the following steps may be performed:

Obtain the pixel mapping relationship between the current camera and the target camera among multiple cameras; obtain the mapping position of the center of the target camera's frame in the current camera frame according to the pixel mapping relationship; determine the target position of the target object in the current camera frame; according to The mapping position and the target position determine the deflection angle of the PTZ equipped with the target camera; the deflection of the PTZ is controlled according to the deflection angle, so that the target object is located at the center of the screen of the deflection target camera.

In the foregoing embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in an embodiment, please refer to the detailed description of the camera control method above, which will not be repeated here.

Please refer to FIG. 16, which is a schematic block diagram of a movable platform provided by an embodiment of the present application. The mobile platform 13 may include a processor 131 and a memory 132, and the processor 131 and the memory 132 are connected by a bus, such as an I2C (Inter-integrated Circuit) bus.

Wherein, the movable platform 13 may include a mobile terminal, a drone, a robot, or an unmanned vehicle, etc.

Specifically, the processor 131 may be a micro-controller unit (MCU), a central processing unit (CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.

Specifically, the memory 132 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk, etc., and may be used to store computer programs.

The movable platform 13 may also include a pan-tilt and multiple cameras. The type of each camera can be flexibly set according to actual needs. The pan-tilt is installed on the body of the movable platform, and multiple cameras are mounted on the pan-tilt.

The processor 131 is configured to call a computer program stored in the memory 132, and implement the camera control method provided in the embodiment of the present application when the computer program is executed. For example, the following steps may be performed:

Obtain the pixel mapping relationship between the current camera and the target camera among multiple cameras; obtain the mapping position of the center of the target camera's frame in the current camera frame according to the pixel mapping relationship; determine the target position of the target object in the current camera frame; according to The mapping position and the target position determine the deflection angle of the PTZ equipped with the target camera; the deflection of the PTZ is controlled according to the deflection angle, so that the target object is located in the center of the screen of the deflection target camera.

In the foregoing embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in an embodiment, please refer to the detailed description of the camera control method above, which will not be repeated here.

The embodiments of the present application also provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the processor executes the program instructions to implement the present application The camera control method provided by the embodiment.

The computer-readable storage medium may be the multi-camera system, pan-tilt camera, or internal storage unit of the movable platform described in any of the foregoing embodiments, such as the hard disk or memory of the pan-tilt camera. The computer-readable storage medium can also be an external storage device of the pan/tilt camera, such as a plug-in hard disk equipped on the pan/tilt camera, a Smart Media Card (SMC), a Secure Digital (SD) card, and a flash memory. Card (Flash Card), etc.

Because the computer program stored in the computer-readable storage medium can execute any camera control method provided in the embodiments of the present application, it can implement what can be achieved by any camera control method provided in the embodiments of the present application. For the beneficial effects of, refer to the previous embodiment for details, and will not be repeated here.

It should be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in the specification of this application and the appended claims, unless the context clearly indicates other circumstances, the singular forms "a", "an" and "the" are intended to include plural forms.

It should also be understood that the term "and/or" used in the specification and appended claims of this application refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, It also includes other elements that are not explicitly listed, or elements inherent to the process, method, article, or system. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or system that includes the element.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims (56)

A camera control method, characterized in that it comprises: Obtain the pixel mapping relationship between the current camera and the target camera; Determining the target position of the target object in the current camera frame; The switching angle of the center point of the target camera is determined according to the pixel mapping relationship and the target position. The camera control method according to claim 1, wherein the method further comprises: Acquiring, according to the pixel mapping relationship, the mapping position of the center of the frame of the target camera in the current camera frame; The determining the switching angle of the center point of the target camera according to the pixel mapping relationship and the target position includes: The switching angle of the center point of the target camera is determined according to the mapping position and the target position. The camera control method according to claim 2, wherein the acquiring, according to the pixel mapping relationship, the mapping position of the center of the target camera's frame in the current camera frame comprises: Acquiring, according to the pixel mapping relationship, a positional deviation between the center of the frame of the target camera and the center of the frame of the current camera; The mapping position of the center of the frame of the target camera in the current camera frame is determined according to the position deviation. The camera control method according to claim 3, wherein the determining, according to the position deviation, the mapping position of the frame center of the target camera in the current camera frame comprises: Determine a reference object, and obtain the distance from the reference object to the center of the target camera; The mapping position of the center of the frame of the target camera in the current camera frame is determined according to the position deviation and the distance. The camera control method according to claim 1, wherein the determining the target position of the target object in the current camera frame comprises: Receiving a selection instruction input by the user in the picture captured by the current camera; The target position of the target object in the current camera frame is determined according to the selection instruction. The camera control method according to claim 1, wherein the determining the target object in the current camera frame comprises: Performing feature extraction on the current camera picture to obtain target feature information; Identify the target object in the current camera frame according to the target feature information, and determine the target position of the target object. The camera control method according to claim 1, wherein said obtaining the pixel mapping relationship between the current camera and the target camera comprises: Acquiring the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera; According to the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera, the conversion matrix between the current camera and the target camera is determined, and the conversion matrix between the current camera and the target camera is obtained. Pixel mapping relationship. 7. The camera control method according to claim 7, wherein said acquiring the internal parameter matrix of the current camera and the internal parameter matrix and the external parameter matrix of the target camera comprises: Acquiring the positional relationship between the current camera and the target camera in the multi-camera system; Obtaining the internal parameters of the current camera according to the position relationship, and obtaining the internal parameters and external parameters of the target camera; Constructing the internal parameter matrix of the current camera according to the internal parameters of the current camera; The internal parameter matrix of the target camera is constructed according to the internal parameters of the target camera, and the external parameter matrix of the target camera is constructed according to the external parameters of the target camera. The camera control method according to claim 8, wherein said acquiring the positional relationship between the current camera and the target camera in the multi-camera system comprises: Acquiring a calibration image, and extracting feature points in the calibration image; Determining the position of the feature point by the current camera and the target camera respectively; According to the position of the feature point, establish a positional relationship between the current camera and the target camera in the multi-camera system, so that both the current camera and the target camera can collect the feature point . The camera control method according to claim 8, wherein the internal parameter matrix of the target camera is constructed according to the internal parameters of the target camera, and the external parameter of the target camera is constructed according to the external parameters of the target camera The matrix includes: Constructing a preset internal parameter matrix and a preset external parameter matrix of the target camera according to the internal and external parameters of the target camera at a preset focal length; Acquiring the current magnification of the target camera; Determine the internal parameter matrix and the external parameter matrix of the target camera according to the current magnification of the target camera, the preset internal parameter matrix and the preset external parameter matrix. The camera control method according to claim 10, wherein said obtaining the current magnification of the target camera comprises: Obtain the current actual focal length of the target camera, and determine the current magnification of the target camera according to the actual focal length; or, The physical distance between the target camera and the reference object is detected, and the current magnification of the target camera is set based on the physical distance. The camera control method according to claim 10, wherein the determining the internal parameter matrix of the target camera according to the current magnification of the target camera and the preset internal parameter matrix comprises: Acquiring the width and height of the image currently collected by the target camera; Determine the internal parameter matrix of the target camera according to the current magnification of the target camera, the preset internal parameter matrix, the width and the height. The camera control method according to any one of claims 1 to 12, wherein the determining the switching angle of the center point of the target camera according to the mapping position and the target position comprises: Determining the offset of the target object to the center of the frame of the current camera according to the mapping position and the target position; The switching angle of the center point of the target camera is determined according to the offset. The camera control method according to claim 13, wherein the offset includes a first offset and a second offset, the switching angle includes a first switching angle and a second switching angle, and the Determining the switching angle of the center point of the target camera according to the offset includes: Determine the first switching angle of the center point of the target camera according to the first offset of the target object in the first direction to the center of the screen of the current camera; Determine the second switching angle of the center point of the target camera according to the second offset of the target object in the second direction from the center of the frame of the current camera. The camera control method according to claim 13, wherein after the switching angle of the center point of the target camera is determined according to the pixel mapping relationship and the target position, the method further comprises: The target camera is controlled to switch the center point according to the switching angle, so that the target object is located at the center of the screen of the target camera after the switching. The camera control method according to claim 15, wherein the controlling the target camera to switch the center point according to the switching angle so that the target object is located in the center of the screen of the target camera after the switching comprises : Controlling the target camera to switch the first switching angle in a first direction and switching the second switching angle in a second direction to obtain the target camera after switching; The target object is controlled to be located at the center of the screen of the target camera after switching. The camera control method according to claim 15, wherein after the control of the target camera to switch the center point according to the switching angle, the method further comprises: Receive recording instructions, shooting instructions, distance measurement instructions or temperature measurement instructions; Control the target camera according to the recording instruction to record the captured frame with the target object in the center, or control the target camera according to the shooting instruction to capture the captured frame with the target object in the center , Or control the target camera to measure the temperature of the target object according to the distance measurement instruction, or control the target camera to measure the temperature of the target object according to the temperature measurement instruction. The camera control method according to any one of claims 1 to 12, wherein the relative position of the current camera and the target camera is fixed. A camera control method, characterized in that it comprises: Obtain the pixel mapping relationship between the current camera and the target camera in the multi-camera system; Acquiring, according to the pixel mapping relationship, the mapping position of the center of the frame of the target camera in the current camera frame; Determining the target position of the target object in the current camera frame; Determining the deflection angle of the pan/tilt equipped with the target camera according to the mapping position and the target position; The deflection of the pan/tilt is controlled according to the deflection angle, so that the target object is located at the center of the frame of the target camera after the deflection. The camera control method according to claim 19, wherein the acquiring, according to the pixel mapping relationship, the mapping position of the frame center of the target camera in the current camera frame comprises: Acquiring, according to the pixel mapping relationship, a positional deviation between the center of the frame of the target camera and the center of the frame of the current camera; The mapping position of the center of the frame of the target camera in the current camera frame is determined according to the position deviation. 22. The camera control method according to claim 20, wherein the determining, according to the position deviation, the mapping position of the frame center of the target camera in the current camera frame comprises: Determine a reference object, and obtain the distance from the reference object to the center of the target camera; The mapping position of the center of the frame of the target camera in the current camera frame is determined according to the position deviation and the distance. The camera control method according to claim 19, wherein the determining the target position of the target object in the current camera frame comprises: Receiving a selection instruction input by the user in the picture captured by the current camera; The target position of the target object in the current camera frame is determined according to the selection instruction. The camera control method according to claim 19, wherein the determining the target object in the current camera frame comprises: Performing feature extraction on the current camera picture to obtain target feature information; Identify the target object in the current camera frame according to the target feature information, and determine the target position of the target object. The camera control method according to claim 19, wherein said obtaining the pixel mapping relationship between the current camera and the target camera comprises: Acquiring the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera; According to the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera, the conversion matrix between the current camera and the target camera is determined, and the conversion matrix between the current camera and the target camera is obtained. Pixel mapping relationship. The camera control method according to claim 24, wherein said acquiring the internal parameter matrix of the current camera and the internal parameter matrix and the external parameter matrix of the target camera comprises: Acquiring the positional relationship between the current camera and the target camera in the multi-camera system; Obtaining the internal parameters of the current camera according to the position relationship, and obtaining the internal parameters and external parameters of the target camera; Constructing the internal parameter matrix of the current camera according to the internal parameters of the current camera; The internal parameter matrix of the target camera is constructed according to the internal parameters of the target camera, and the external parameter matrix of the target camera is constructed according to the external parameters of the target camera. The camera control method according to claim 25, wherein said acquiring the positional relationship between the current camera and the target camera in the multi-camera system comprises: Acquiring a calibration image, and extracting feature points in the calibration image; Determining the position of the feature point by the current camera and the target camera respectively; According to the position of the feature point, establish a positional relationship between the current camera and the target camera in the multi-camera system, so that both the current camera and the target camera can collect the feature point . The camera control method according to claim 25, wherein the internal parameter matrix of the target camera is constructed according to the internal parameters of the target camera, and the external parameter of the target camera is constructed according to the external parameters of the target camera The matrix includes: Constructing a preset internal parameter matrix and a preset external parameter matrix of the target camera according to the internal and external parameters of the target camera at a preset focal length; Acquiring the current magnification of the target camera; Determine the internal parameter matrix and the external parameter matrix of the target camera according to the current magnification of the target camera, the preset internal parameter matrix and the preset external parameter matrix. The camera control method according to claim 27, wherein said obtaining the current magnification of the target camera comprises: Obtain the current actual focal length of the target camera, and determine the current magnification of the target camera according to the actual focal length; or, The physical distance between the target camera and the reference object is detected, and the current magnification of the target camera is set based on the physical distance. The camera control method according to claim 27, wherein the determining the internal parameter matrix of the target camera according to the current magnification of the target camera and the preset internal parameter matrix comprises: Acquiring the width and height of the image currently collected by the target camera; Determine the internal parameter matrix of the target camera according to the current magnification of the target camera, the preset internal parameter matrix, the width and the height. The camera control method according to any one of claims 19 to 29, wherein the determining the deflection angle of the pan/tilt on which the target camera is mounted according to the mapping position and the target position comprises: Determining the offset of the target object to the center of the frame of the current camera according to the mapping position and the target position; The deflection angle of the pan-tilt on which the target camera is mounted is determined according to the offset. The camera control method according to claim 30, wherein the offset includes a first offset and a second offset, the yaw angle includes a pitch yaw angle and a yaw yaw angle, and the The offset determining the deflection angle of the pan/tilt equipped with the target camera includes: Determine the pitch and yaw angle of the pan/tilt on which the target camera is mounted according to the first offset from the target object in the first direction to the center of the screen of the current camera; According to the second offset of the target object in the second direction from the center of the frame of the current camera, determine the yaw deflection angle of the pan/tilt on which the target camera is mounted. The camera control method of claim 31, wherein the controlling the deflection of the pan/tilt according to the deflection angle so that the target object is located at the center of the frame of the target camera after the deflection comprises: Controlling the gimbal to deflect the pitch yaw angle in the pitch direction and the yaw yaw angle in the yaw direction to obtain the target camera after being deflected; Control the target object to be located at the center of the frame of the target camera after the deflection. The camera control method according to any one of claims 19 to 29, characterized in that, after controlling the deflection of the pan/tilt according to the deflection angle, the method further comprises: Receive recording instructions, shooting instructions, distance measurement instructions or temperature measurement instructions; Control the target camera according to the recording instruction to record the captured frame with the target object in the center, or control the target camera according to the shooting instruction to capture the captured frame with the target object in the center , Or control the target camera to measure the temperature of the target object according to the distance measurement instruction, or control the target camera to measure the temperature of the target object according to the temperature measurement instruction. The camera control method according to any one of claims 19 to 29, wherein the relative position of the current camera and the target camera in the multi-camera system is fixed. The camera control method according to any one of claims 19 to 29, wherein the current camera is mounted on the pan/tilt. A camera control method, characterized in that it comprises: Control the current camera in the multi-camera system to capture a picture and display the picture; Receiving a selection instruction input by the user based on the displayed screen; Determining the target position of the target object in the current camera frame according to the selection instruction; Acquiring the mapping position of the frame center of the target camera in the current camera frame in the multi-camera system; Determining the deflection angle of the target camera according to the mapping position and the target position; The target camera is controlled to deflect according to the deflection angle, so that the target object is located at the center of the frame of the target camera after the deflection. The camera control method of claim 36, wherein, based on the displayed picture, receiving a selection instruction input by a user comprises: Receiving a click operation input by the user in the displayed screen; Determine the area within the screen of the click operation, and generate a selection instruction according to the area. The camera control method of claim 36, wherein, based on the displayed picture, receiving a selection instruction input by a user comprises: Performing feature extraction on the screen to obtain target feature information; Determine objects existing in the screen according to the target feature information to obtain a set of objects; Displaying a list of object identifiers of the object set in the displayed screen; Receive a selection instruction entered by the user in the object identification list. The camera control method according to claim 36, wherein after controlling the target camera to deflect according to the deflection angle, the method further comprises: Receive recording instructions, shooting instructions, distance measurement instructions or temperature measurement instructions; Control the target camera to record the captured image according to the recording instruction, or control the target camera to photograph the captured image according to the shooting instruction, or control the target camera pair according to the ranging instruction The target object performs distance measurement, or the target camera is controlled to measure the temperature of the target object according to the temperature measurement instruction. 36. The camera control method according to claim 36, wherein said acquiring the mapping position of the frame center of the target camera in the multi-camera system in the current camera frame comprises: Obtain the pixel mapping relationship between the current camera and the target camera in the multi-camera system; According to the pixel mapping relationship, the mapping position of the center of the frame of the target camera in the current camera frame is acquired. 40. The camera control method according to claim 40, wherein the acquiring, according to the pixel mapping relationship, the mapping position of the frame center of the target camera in the current camera frame comprises: Acquiring, according to the pixel mapping relationship, a positional deviation between the center of the frame of the target camera and the center of the frame of the current camera; The mapping position of the center of the frame of the target camera in the current camera frame is determined according to the position deviation. The camera control method according to claim 41, wherein the determining the mapping position of the frame center of the target camera in the current camera frame according to the position deviation comprises: Determine a reference object, and obtain the distance from the reference object to the center of the target camera; The mapping position of the center of the frame of the target camera in the current camera frame is determined according to the position deviation and the distance. The camera control method according to claim 40, wherein said obtaining the pixel mapping relationship between the current camera and the target camera comprises: Acquiring the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera; According to the internal parameter matrix of the current camera, and the internal parameter matrix and the external parameter matrix of the target camera, the conversion matrix between the current camera and the target camera is determined, and the conversion matrix between the current camera and the target camera is obtained. Pixel mapping relationship. The camera control method according to claim 43, wherein said acquiring the internal parameter matrix of the current camera and the internal parameter matrix and the external parameter matrix of the target camera comprises: Acquiring the positional relationship between the current camera and the target camera in the multi-camera system; Obtaining the internal parameters of the current camera according to the position relationship, and obtaining the internal parameters and external parameters of the target camera; Constructing the internal parameter matrix of the current camera according to the internal parameters of the current camera; The internal parameter matrix of the target camera is constructed according to the internal parameters of the target camera, and the external parameter matrix of the target camera is constructed according to the external parameters of the target camera. The camera control method according to claim 44, wherein said acquiring the positional relationship between the current camera and the target camera in the multi-camera system comprises: Acquiring a calibration image, and extracting feature points in the calibration image; Determining the position of the feature point by the current camera and the target camera respectively; According to the position of the feature point, establish a positional relationship between the current camera and the target camera in the multi-camera system, so that both the current camera and the target camera can collect the feature point . The camera control method according to claim 44, wherein the internal parameter matrix of the target camera is constructed according to the internal parameters of the target camera, and the external parameter of the target camera is constructed according to the external parameters of the target camera The matrix includes: Constructing a preset internal parameter matrix and a preset external parameter matrix of the target camera according to the internal and external parameters of the target camera at a preset focal length; Acquiring the current magnification of the target camera; Determine the internal parameter matrix and the external parameter matrix of the target camera according to the current magnification of the target camera, the preset internal parameter matrix and the preset external parameter matrix. The camera control method according to claim 46, wherein said obtaining the current magnification of the target camera comprises: Obtain the current actual focal length of the target camera, and determine the current magnification of the target camera according to the actual focal length; or, The physical distance between the target camera and the reference object is detected, and the current magnification of the target camera is set based on the physical distance. The camera control method according to claim 46, wherein the determining the internal parameter matrix of the target camera according to the current magnification of the target camera and the preset internal parameter matrix comprises: Acquiring the width and height of the image currently collected by the target camera; Determine the internal parameter matrix of the target camera according to the current magnification of the target camera, the preset internal parameter matrix, the width and the height. The camera control method according to any one of claims 36 to 48, wherein the determining the deflection angle of the target camera according to the mapping position and the target position comprises: Determining the offset of the target object to the center of the frame of the current camera according to the mapping position and the target position; The deflection angle of the target camera is determined according to the offset. The camera control method according to claim 49, wherein the offset includes a first offset and a second offset, the deflection angle includes a first deflection angle and a second deflection angle, and the Determining the deflection angle of the target camera according to the offset includes: Determine the first deflection angle of the target camera according to the first offset of the target object in the first direction to the center of the screen of the current camera; The second deflection angle of the target camera is determined according to the second offset of the target object in the second direction to the center of the frame of the current camera. The camera control method according to claim 50, wherein the controlling the target camera to deflect according to the deflection angle so that the target object is located at the center of the frame of the target camera after the deflection comprises: Controlling the target camera to deflect the first deflection angle in a first direction and deflect the second deflection angle in a second direction to obtain the deflected target camera; Control the target object to be located at the center of the frame of the target camera after the deflection. A multi-camera system, characterized in that, the multi-camera system includes: Multiple cameras Memory, used to store computer programs; The processor is used to call the computer program in the memory for executing: Obtain the pixel mapping relationship between the current camera and the target camera; Determining the target position of the target object in the current camera frame; The switching angle of the center point of the target camera is determined according to the pixel mapping relationship and the target position. A pan-tilt camera, characterized in that the pan-tilt camera includes: Multiple cameras PTZ for carrying the multiple cameras; Memory, used to store computer programs; The processor is used to call the computer program in the memory for executing: Acquiring a pixel mapping relationship between the current camera and the target camera in the plurality of cameras; Acquiring, according to the pixel mapping relationship, the mapping position of the center of the frame of the target camera in the current camera frame; Determining the target position of the target object in the current camera frame; Determining the deflection angle of the pan/tilt equipped with the target camera according to the mapping position and the target position; The deflection of the pan/tilt is controlled according to the deflection angle, so that the target object is located at the center of the frame of the target camera after the deflection. A movable platform, characterized in that it comprises: PTZ, installed on the body of the movable platform; Multiple cameras mounted on the pan/tilt; Memory, used to store computer programs; The processor is configured to call a computer program in the memory to execute the camera control method according to any one of claims 19 to 35, or execute the camera control method according to any one of claims 36 to 51. The movable platform according to claim 54, wherein the movable platform is a mobile terminal, a drone, a robot, or an unmanned vehicle. A computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute the camera control method according to any one of claims 1 to 18, Either the camera control method according to any one of claims 19 to 35 is executed, or the camera control method according to any one of claims 36 to 51 is executed.

Images