WO2020107375A1

WO2020107375A1 - Method and apparatus for image processing, and device and storage medium

Info

Publication number: WO2020107375A1
Application number: PCT/CN2018/118425
Authority: WO
Inventors: 朱涛
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2018-11-30
Filing date: 2018-11-30
Publication date: 2020-06-04
Also published as: CN111386710A

Abstract

Embodiments of the present invention provide a method and apparatus for image processing, and a device and a storage medium. In the embodiments of the present invention, by means of a ground control end, both an image captured by the photography device of an unmanned aerial vehicle and an image captured by a ground photography device communicationally connected to the ground control end are obtained, and the image captured by the ground photography device is superimposed into the image captured by the photography device of the unmanned aerial vehicle, or the image captured by the photography device of the unmanned aerial vehicle is superimposed into the image captured by the ground photography device, so that the image after the superposition includes both the image captured by the photography device of the unmanned aerial vehicle and the image captured by the ground photography device, the image captured by the ground photography device may include an image of a person such as an operator, a narrator, and a host, and may also include an image of a scene surrounding the ground control end, so that a user can see both the photography environment of the unmanned aerial vehicle and the photography environment of the ground photography device in one image, thereby improving the presence of the environment surrounding the ground control end.

Description

Image processing method, device, equipment and storage medium

Technical field

Embodiments of the present invention relate to the field of drones, and in particular, to an image processing method, device, equipment, and storage medium.

Background technique

In the prior art, a shooting device mounted on a movable platform such as a drone can be used for aerial photography. The drone can send the aerial image information of the shooting device to a ground control terminal such as a remote control.

In some application scenarios, the user hopes to see the shooting picture of the drone and the picture around the ground control terminal. For example, the drone is taking pictures of news events, and the host reports the news events around the ground control terminal. The user wants to watch the news event and the host at the same time. However, when the drone is flying, the ground control terminal may be far away from the drone, resulting in that the image information of the aerial photography of the shooting device may not have a picture around the ground control terminal, and the user cannot see The shooting pictures of the man-machine and the pictures around the ground control end reduce the presence of the environment around the ground control end.

Summary of the invention

Embodiments of the present invention provide an image processing method, device, equipment, and storage medium to improve the presence of the environment around the ground control terminal.

A first aspect of an embodiment of the present invention is to provide an image processing method, which is applied to a ground control terminal and includes:

Obtaining a first image and a second image, wherein the first image is one of a captured image of a drone camera and a captured image of a ground camera connected to the ground control terminal, and the second image is Another image different from the first image in the captured image of the shooting device of the drone and the captured image of the ground shooting device communicatively connected to the ground control terminal;

The second image is superimposed on the first image to obtain a third image.

A second aspect of an embodiment of the present invention is to provide a ground control terminal, including: a memory, a processor, and a first communication interface;

The first communication interface is used for communication connection with the ground shooting device;

The memory is used to store program codes;

The processor calls the program code, and when the program code is executed, it is used to perform the following operations:

Acquiring a first image and a second image, wherein the first image is one of a captured image of a drone camera and a captured image of the ground camera connected to the ground control terminal, The second image is another image different from the first image in the captured image of the shooting device of the drone and the captured image of the ground shooting device communicatively connected to the ground control terminal;

The second image is superimposed on the first image to obtain a third image.

A third aspect of the embodiments of the present invention is to provide a computer-readable storage medium on which a computer program is stored, which is executed by a processor to implement the method of the first aspect.

The image processing method, device, equipment and storage medium provided in this embodiment simultaneously acquire the captured image of the drone's shooting device and the ground shooting device's captured image in communication with the ground control terminal through the ground control terminal The shooting image of the shooting device is superimposed on the shooting image of the drone shooting device, or the shooting image of the drone shooting device is superimposed on the shooting image of the ground shooting device, so that the superimposed image also includes the drone The captured image of the shooting device and the captured image of the ground shooting device. The captured image of the ground shooting device can include images of characters such as pilots, commentators, moderators, etc., and can also include scene images around the ground control terminal, so that users can see It is possible to see the shooting environment of the drone and the shooting environment of the ground shooting device at the same time in one image, thereby improving the presence of the surrounding environment of the ground control end.

BRIEF DESCRIPTION

In order to more clearly explain the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without paying any creative labor, other drawings can also be obtained based on these drawings.

FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present invention;

2 is a flowchart of an image processing method provided by an embodiment of the present invention;

3 is a schematic diagram of an image processing provided by an embodiment of the present invention;

4 is a schematic diagram of another image processing provided by an embodiment of the present invention;

5 is a schematic diagram of still another image processing provided by an embodiment of the present invention;

6 is a schematic diagram of still another image processing provided by an embodiment of the present invention;

7 is a schematic diagram of still another image processing provided by an embodiment of the present invention;

8 is a schematic diagram of still another image processing provided by an embodiment of the present invention;

9 is a schematic diagram of still another image processing provided by an embodiment of the present invention;

FIG. 10 is a structural diagram of a ground control terminal provided by an embodiment of the present invention.

Reference mark:

10: UAV; 11: shooting device; 12: PTZ;

13: communication interface; 14: ground control terminal; 15: ground shooting device;

16: server; 31: first image; 32: second image;

33: image; 34: third image; 41: image;

42: third image; 61: image; 71: image;

100: ground control terminal; 101: memory; 102: processor;

103: first communication interface; 104: second communication interface.

detailed description

The technical solutions in the embodiments of the present invention will be described clearly below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.

It should be noted that when a component is said to be "fixed" to another component, it can be directly on another component or it can also exist in a centered component. When a component is considered to be "connected" to another component, it can be directly connected to another component or there can be centered components at the same time.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention. The terminology used in the description of the present invention herein is for the purpose of describing specific embodiments, and is not intended to limit the present invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

The following describes some embodiments of the present invention in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and the features in the embodiments can be combined with each other.

An embodiment of the present invention provides an image processing method. The image processing method described in this embodiment can be applied to the ground control terminal of an unmanned aerial vehicle. The ground control terminal can be a remote controller, a mobile phone, a tablet computer, and other devices. As shown in FIG. 1, the drone 10 is equipped with a shooting device 11. The shooting device 11 is connected to the body of the drone 10 through a supporting member such as a gimbal 12. The interface 13 is sent to the ground control terminal 14. In this embodiment, the ground control terminal 14 is communicatively connected to the ground shooting device 15. In some embodiments, the ground shooting device 15 may be integrated in the ground control terminal 14, such as a ground shooting device 15 can be provided on the ground control terminal 14. In some other embodiments, the ground shooting device 15 may be a shooting device external to the ground control terminal 14. In this embodiment, the ground control terminal 14 can receive the image captured by the shooting device 11 of the drone 10 or the image captured by the ground shooting device 15, and the ground control terminal 14 can capture the image captured by the shooting device 11 Image processing is performed with the image captured by the ground shooting device 15, and the image processing method will be described in detail below through specific embodiments.

2 is a flowchart of an image processing method provided by an embodiment of the present invention. As shown in FIG. 2, the method in this embodiment may include:

Step S201: Acquire a first image and a second image, where the first image is one of a captured image of a drone camera and a captured image of a ground camera connected to the ground control terminal, and The second image is another image different from the first image in the captured image of the shooting device of the drone and the captured image of the ground shooting device communicatively connected to the ground control terminal.

As shown in FIG. 1, the ground control terminal 14 may receive the image captured by the shooting device 11 of the drone 10 or the image captured by the ground shooting device 15. The image captured by the shooting device 11 may specifically be a drone 10 aerial image during flight. The image captured by the ground shooting device 15 may specifically be an image around the ground control terminal 14, for example, images of characters such as pilots, narrators, and moderators near the ground control terminal 14, and/or scene images around the ground control terminal 14 . Optionally, one of the image captured by the shooting device 11 and the image captured by the ground shooting device 15 is recorded as the first image, and the other of the image captured by the shooting device 11 and the image captured by the ground shooting device 15 is the other. One image is recorded as the second image. That is to say, the first image may be an image captured by the shooting device 11, the second image may be an image captured by the ground shooting device 15, or the first image may be an image captured by the ground shooting device 15, and the second image may be The image captured by the shooting device 11.

Optionally, the ground control terminal and the ground shooting device are connected by wireless communication or wired communication. As shown in FIG. 1, the ground control terminal 14 and the ground shooting device 15 may be connected by wired communication, or may be connected by wireless communication.

When the ground control terminal 14 and the ground shooting device 15 are connected by wireless communication, the wireless communication method may specifically be a wireless fidelity (WIreless-Fidelity, WIFI) communication method.

When the ground control terminal and the ground shooting device are communicatively connected by wired communication, the connection interface between the ground control terminal and the ground shooting device includes at least one of the following: composite synchronous video broadcast signal (Composite Video Broadcast Signal) , CVBS) interface, high definition multimedia interface (High Definition Multimedia Interface, IDMI), digital component serial interface (Serial) digital interface (SDI), mobile industry processor interface (Mobile Industry Processor Interface, MIPI), universal serial bus (Universal Serial Bus, USB) interface.

In addition, when the ground control terminal 14 and the ground shooting device 15 are in communication connection through at least one interface of CVBS, IDMI, SDI, and MIPI, the image acquired by the ground control terminal 14 from the ground shooting device 15 is taken by the ground shooting device 15 The original image of, that is, the original image captured by the ground shooting device 15 can be sent to the ground control terminal 14 through at least one interface of CVBS, IDMI, SDI, and MIPI without being encoded.

In addition, when the ground control terminal 14 and the ground shooting device 15 are communicatively connected via at least one of USB and WIFI, the image acquired by the ground control terminal 14 from the ground shooting device 15 is an encoded image, that is, the ground shooting device 15 The original image captured needs to be encoded, for example, encoded by H.264 encoding to obtain an encoded image, and further, the encoded image is sent to the ground control terminal 14 via at least one of USB and WIFI.

In addition, the drone 10 can encode the image captured by the shooting device 11, for example, encoding by H.264 encoding to obtain an encoded image, and further, the encoded image can be encoded by wireless communication, such as WIFI, software-defined radio (Software At least one of Defined Radio (SDR) is sent to the ground control terminal 14.

Therefore, when the ground control terminal 14 receives the encoded image sent by the drone 10 and/or the encoded image sent by the ground shooting device 15, the ground control terminal 14 also needs to decode the encoded image to obtain the original image.

As a possible implementation manner, the acquiring the first image and the second image includes: receiving a first encoded image sent by one of the drone and the ground shooting device; and performing the first encoded image Decoding to obtain the first image; and/or receiving a second coded image sent by the other of the drone and the ground shooting device; decoding the second coded image to obtain the first image Second image.

For example, the image captured by the ground camera 15 is recorded as the first image, and the image captured by the camera 11 of the drone 10 is recorded as the second image. The ground camera 15 may send the first image to the ground control terminal 14 Or, send the encoded image of the first image to the ground control terminal 14, where the encoded image of the first image is recorded as the first encoded image. In some embodiments, the unmanned aerial vehicle 10 may send the second image to the ground control terminal 14, or send the encoded image of the second image to the ground control terminal 14, here, the encoded image of the second image Record as the second coded image.

A possible situation is that: the ground shooting device 15 sends the first encoded image to the ground control terminal 14, and the drone 10 sends the second image to the ground control terminal 14. At this time, the ground control terminal 14 needs to encode the first image The image is decoded to obtain the first image.

Another possible situation is that: the ground shooting device 15 sends a first image to the ground control terminal 14, and the drone 10 sends a second coded image to the ground control terminal 14. At this time, the ground control terminal 14 needs to encode the second image The image is decoded to obtain the second image.

Another possible situation is that: the ground shooting device 15 sends the first coded image to the ground control terminal 14, and the drone 10 sends the second coded image to the ground control terminal 14. At this time, the ground control terminal 14 needs to The encoded image is decoded to obtain the first image, and the second encoded image is decoded to obtain the second image.

In step S202, the second image is superimposed on the first image to obtain a third image.

If the first image is an image captured by the camera 11 and the second image is an image captured by the ground camera 15, the image captured by the camera 11 can be used as the main screen, and the image captured by the ground camera 15 can be used as the auxiliary screen , The image captured by the ground imaging device 15 is superimposed on the image captured by the imaging device 11 to obtain a third image.

If the first image is an image captured by the ground camera 15 and the second image is an image captured by the camera 11, the image captured by the ground camera 15 can be used as the main screen, and the image captured by the camera 11 can be used as the auxiliary screen , The image captured by the camera 11 is superimposed on the image captured by the ground camera 15 to obtain a third image.

It can be understood that the superposition may be implemented by using a pixel superposition algorithm.

In this embodiment, the first image is an image captured by the imaging device 11 and the second image is an image captured by the ground imaging device 15 as an example, and the following schematic description will be made.

Optionally, the superimposing the second image onto the first image to obtain a third image includes adjusting the original size of the first image and/or the second image so that after the adjustment The size of the first image is larger than the size of the second image; after the adjustment, the second image is superimposed on the first image to obtain the third image.

For example, the size of the first image captured by the camera 11 and the size of the second image captured by the ground camera 15 may be different or the same. Before superimposing the second image on the first image, the first image and The original size of the second image is adjusted so that after adjustment, the size of the first image is larger than the size of the second image. After the adjustment, the second image is superimposed on the first image to obtain the third image.

As shown in FIG. 3, 31 represents the first image captured by the

camera

11, and 32 represents the second image captured by the ground camera 15. The size of the second image 32 is equal to the size of the first image 31. In other embodiments, The size of the second image 32 may be larger than the size of the first image 31. In this case, the size of the first image 31 and/or the size of the second image 32 needs to be adjusted. After the adjustment, the second image is superimposed The third image is obtained from the first image.

A possible implementation manner is: performing reduction processing on the second image 32, as shown in FIG. 3, 33 represents an image after performing reduction processing on the second image 32. The superimposed image 33 on the second image 32 is superimposed on the first image 31 to obtain the third image 34.

Another possible implementation manner is: performing an enlargement process on the first image 31, as shown in FIG. 4, 41 represents an image after performing the enlargement process on the first image 31, and superimposing the second image 32 on the first image 31 The third image 42 is obtained from the image 41 subjected to the enlargement process.

Another possible implementation is to perform a reduction process on the second image 32, and at the same time, perform an enlargement process on the first image 31, as shown in FIG. 5, 33 represents the image after the second image 32 is reduced, 41 It represents an image after the first image 31 is enlarged, and the image 33 obtained by performing the reduction process on the second image 32 is superimposed on the image 41 after performing the enlargement process on the first image 31 to obtain a third image 42.

In some embodiments, the superimposing the second image on the first image to obtain the third image includes superimposing the second image on a preset position in the first image to obtain the third image Describe the third image. Optionally, the preset position is determined according to the original size of the first image or the adjusted size of the first image.

For example, after adjusting the size of the first image 31 and/or the size of the second image 32 so that the size of the first image 31 is larger than the size of the second image 32, when the second image 32 is superimposed on the first image 31 Specifically, the second image 32 is superimposed on the preset position in the first image 31 to obtain the third image. For example, the second image 32 is superimposed on the lower left corner of the first image 31. In some embodiments, the preset position is determined according to the original size of the first image 31 or the adjusted size of the first image 31.

In some other embodiments, the size of the first image 31 may be greater than the size of the second image 32, and the ratio of the size of the first image 31 to the size of the second image 32 is greater than the preset ratio, or, the size of the first image 31 If the absolute value of the difference between the size and the size of the second image 32 is greater than the preset difference, you can directly superimpose the second image 32 without adjusting the size of the first image 31 and/or the size of the second image 32 A third image is obtained from the first image 31. Specifically, the second image 32 is superimposed on a preset position in the first image 31 to obtain a third image.

In some embodiments, after the second image is superimposed on the first image to obtain a third image, the method further includes: displaying the third image on a display component.

As shown in FIG. 3, after superimposing the image 33 after the second image 32 is reduced on the first image 31 to obtain the third image 34, the third image 34 may be further displayed on the display component. The display component may specifically be a screen of the ground control terminal 14 or a display device external to the ground control terminal 14, such as a display screen. Similarly, the third image 42 shown in FIGS. 4 and 5 can also be displayed on the display component.

In this embodiment, the captured image of the drone camera and the captured image of the ground camera connected to the ground controller are acquired through the ground control terminal, and the captured image of the ground camera is superimposed on the drone camera. In the captured image, or superimpose the captured image of the drone camera to the captured image of the ground camera, so that the superimposed image includes both the captured image of the drone camera and the captured image of the ground camera, The captured images of the ground shooting device may include images of characters such as flying pilots, narrators, moderators, etc., and may also include scene images around the ground control terminal, so that users can see the shooting of the drone in one image at the same time Environment and the shooting environment of the ground shooting device, thereby improving the presence of the environment around the ground control end.

An embodiment of the present invention provides an image processing method. Based on the embodiment shown in FIG. 2, the adjusting the original size of the first image and/or the second image includes: adjusting the original size of the second image according to the original size of the first image Original size.

Specifically, as shown in FIG. 3, keeping the size of the first image 31 unchanged, the original size of the second image 32 is adjusted according to the original size of the first image 31.

Optionally, the adjusting the original size of the second image according to the original size of the first image includes the following feasible implementation manners:

A feasible implementation manner is: adjusting the original height of the second image according to the original size of the first image; according to the adjusted height of the second image and the original height of the second image and The ratio of the original width adjusts the original width of the second image.

As shown in FIG. 6, when the original height of the second image 32 is adjusted according to the original size of the first image 31, specifically the original height of the second image 32 may be adjusted according to the original height of the first image 31, for example, the first image 31 The original height of is recorded as H, and the original height of the second image 32 is adjusted according to the original height H of the first image 31, so that the height of the adjusted second image 32 is smaller than H, and 61 indicates that the height of the second image 32 is adjusted After the image, the height of the adjusted second image 32 is recorded as h, for example, h is equal to a quarter of H. Further, according to the adjusted height h of the second image 32 and the ratio of the original height of the second image 32 to the original width, the width of the image 61 is adjusted to obtain the image 33.

Another possible implementation is to adjust the original width of the second image according to the original size of the first image; according to the adjusted width of the second image and the original height of the second image Adjust the original height of the second image in proportion to the original width.

As shown in FIG. 7, when the original width of the second image 32 is adjusted according to the original size of the first image 31, specifically the original width of the second image 32 may be adjusted according to the original width of the first image 31, for example, the first image 31 The original width of W is denoted as W, and the original width of the second image 32 is adjusted according to the original width W of the first image 31, so that the width of the adjusted second image 32 is smaller than W, and 71 indicates that the width of the second image 32 is adjusted After the image is adjusted, the width of the adjusted second image 32 is recorded as w, for example, w is equal to a quarter of W. Further, according to the adjusted width w of the second image 32 and the ratio of the original height of the second image 32 to the original width, the height of the image 71 is adjusted to obtain the image 33.

After the second image is superimposed on the first image to obtain the third image, the method further includes: adjusting the size of the third image according to at least one resolution supported by the display component. As shown in FIG. 6 or FIG. 7, the image 33 after the second image 32 is subjected to the reduction process is superimposed on the first image 31 to obtain a third image 34 whose size is the same as the original size of the first image 31 , When the third image 34 is displayed on the display component, the resolution of the display component and the resolution of the main screen, that is, the first image 31 may not match. In this case, the third image 34 needs to be adjusted according to the resolution of the display component In particular, the display component can support at least one resolution, and the size of the third image 34 can be adjusted according to the at least one resolution supported by the display component.

The adjusting the size of the third image according to at least one resolution supported by the display component includes: determining, from the at least one resolution supported by the display component, the resolution closest to the resolution of the first image Target resolution; adjust the size of the third image according to the target resolution.

For example, a target resolution closest to the resolution of the first image 31 is determined from at least one resolution supported by the display component, and the size of the third image 34 is adjusted according to the target resolution. For example, the resolutions supported by the display component include: 320*240 and 640*480, and the resolution of the first image 31 is 640*640, compared to the resolution supported by the display component 320*240, The resolution 640*480 supported by the display component is closer to the resolution of the first image 31, and the size of the third image 34 is adjusted according to the resolution 640*480 supported by the display component to reduce the The third image 34 is scaled so that the third image 34 is less distorted. At the same time, the current resolution of the display component is adjusted to 640*480.

In this embodiment, the original size of the first image is adjusted to adjust the original size of the second image, and the adjusted second image is superimposed on the first image to obtain the third image, and further distinguished according to at least one resolution supported by the display component Adjust the size of the third image to adapt the size of the third image to the resolution supported by the display component, and in addition, determine the resolution from the first image from at least one resolution supported by the display component The target resolution with the closest rate; adjusting the size of the third image according to the target resolution can reduce the zoom ratio for scaling the third image, so that the third image is less distorted.

An embodiment of the present invention provides an image processing method. Based on the embodiment shown in FIG. 2, the adjusting the original size of the first image and/or the second image includes: adjusting the original size of the first image; according to the adjusted For the size of an image, adjust the original size of the second image.

As shown in FIG. 5, the first image 31 is enlarged to obtain the image 41, and further, the original size of the second image 32 is adjusted according to the size of the image 41 to obtain the image 33, and the image 33 is superimposed on the image 41 to obtain the first Three images 42.

Optionally, the adjusting the original size of the first image includes: adjusting the original size of the first image according to at least one resolution supported by the display component.

For example, the first image 31 is the main screen of the display component, and the resolution of the first image 31 and the resolution of the display component may not match. In this case, the original size of the first image 31 needs to be adjusted according to the resolution of the display component Specifically, the display component may support at least one resolution, and the original size of the first image 31 is adjusted according to the at least one resolution supported by the display component.

Optionally, the adjusting the original size of the first image according to at least one resolution supported by the display component includes: determining the resolution of the first image from the at least one resolution supported by the display component The target resolution with the closest resolution; adjusting the original size of the first image according to the target resolution.

For example, the target resolution closest to the resolution of the first image 31 is determined from at least one resolution supported by the display component, and the original size of the first image 31 is adjusted according to the target resolution, for example , The resolution supported by the display component includes: 320*240 and 640*480, the resolution of the first image 31 is 640*640, compared to the resolution supported by the display component 320*240, the The resolution 640*480 supported by the display component is closer to the resolution of the first image 31, and the size of the first image 31 is adjusted according to the resolution 640*480 supported by the display component to reduce The image 31 is scaled so that the first image 31 is less distorted. At the same time, the current resolution of the display component is adjusted to 640*480.

Optionally, the adjusting the original size of the second image according to the adjusted size of the first image includes the following feasible implementation methods:

A feasible implementation manner is: adjusting the original height of the second image according to the adjusted size of the first image; according to the adjusted height of the second image and the original of the second image The ratio of the height to the original width adjusts the original width of the second image.

As shown in FIG. 8, the original size of the first image 31 is adjusted according to the resolution of the display component to obtain the image 41, and when the original height of the second image 32 is adjusted according to the size of the image 41, the height can be adjusted specifically according to the height of the image 41 The original height of the second image 32, for example, the height of the image 41 is recorded as H, and the original height of the second image 32 is adjusted according to the height H of the image 41, so that the height of the adjusted second image 32 is smaller than H, and 61 indicates The height of the second image 32 is adjusted. The height of the adjusted second image 32 is denoted as h. For example, h is equal to a quarter of H. Further, according to the adjusted height h of the second image 32 and the ratio of the original height of the second image 32 to the original width, the width of the image 61 is adjusted to obtain the image 33.

Another feasible implementation manner is: adjusting the original width of the second image according to the adjusted size of the first image; according to the adjusted width of the second image and the size of the second image The ratio of the original height to the original width adjusts the original height of the second image.

As shown in FIG. 9, the original size of the first image 31 is adjusted according to the resolution of the display component to obtain the image 41, and when the original width of the second image 32 is adjusted according to the size of the image 41, it can be specifically adjusted according to the width of the image 41 The original width of the second image 32, for example, the width of the image 41 is denoted as W, and the original width of the second image 32 is adjusted according to the width W of the image 41, so that the width of the adjusted second image 32 is smaller than W, and 71 indicates An image after the width of the second image 32 is adjusted, and the width of the adjusted second image 32 is denoted by w, for example, w is equal to a quarter of W. Further, according to the adjusted width w of the second image 32 and the ratio of the original height of the second image 32 to the original width, the height of the image 71 is adjusted to obtain the image 33.

As shown in FIG. 8 or FIG. 9, the original size of the first image 31 is adjusted according to the resolution of the display component to obtain the image 41, that is, the size of the image 41 has been adapted to the resolution of the display component, so , When the image 33 is superimposed on the image 41 to obtain the third image 42, the size of the third image 42 is the same as the size of the image 41. Therefore, the size of the third image 42 is also adapted to the resolution of the display component. The size of the third image 42 is not adjusted again.

In this embodiment, by determining the target resolution closest to the resolution of the first image from at least one resolution supported by the display component, and adjusting the original size of the first image according to the target resolution, The zoom ratio of the first image is reduced to make the first image less distorted.

An embodiment of the present invention provides an image processing method. Based on the above embodiment, after the second image is superimposed on the first image to obtain a third image, the method further includes: running an application in the ground control terminal, and using the application to The third image performs a sharing operation.

As shown in FIG. 1, the ground control terminal 14 may be installed with an application program, and the application program may be an instant messaging application, such as WeChat, Facebook, Douyin, etc. The ground control terminal 14 may start according to the user's start operation of the application Run the application program, and perform a sharing operation on the third image through the application program.

In some embodiments, after the second image is superimposed on the first image to obtain a third image, the method further includes: sending the third image to at least one of a live broadcast device, a relay device, and a server .

For example, the ground control terminal 14 may also send the third image to at least one of a live broadcast device, a relay device, and a server, and the live broadcast device may send the third image to a television station to achieve live TV broadcast. Alternatively, the live broadcast device may perform live webcast of the third image. The rebroadcasting device may send the third image to a television station to achieve television rebroadcasting.

Optionally, the ground control terminal 14 receives the first image sent by the drone 10 in real time and the second image captured by the ground shooting device 15. The ground control terminal 14 can superimpose the second image on the first image in real time The third image, and send the third image to the server 16 in real time, or the ground control terminal 14 encodes the third image in real time to obtain an encoded image, and sends the encoded image to the server 16, the server 16 may specifically be A streaming media server. Other user terminals may download the third image from the streaming media server.

In this embodiment, the superimposed image of the ground shooting device is superimposed on the drone's shooting device, or the drone's shooting device is superimposed on the ground shooting device. The application in the control terminal performs a sharing operation on the superimposed image, or sends the superimposed image to at least one of the live broadcast device, the relay device, and the server through the ground control terminal, thereby realizing the superimposed image Live broadcast, rebroadcast or share.

An embodiment of the present invention provides a ground control terminal. 10 is a structural diagram of a ground control terminal provided by an embodiment of the present invention. As shown in FIG. 10, the ground control terminal 100 includes: a memory 101, a processor 102, and a first communication interface 103; The ground shooting device is communicatively connected; the memory 101 is used to store the program code; the processor 102 calls the program code, and when the program code is executed, it is used to perform the following operations: acquire the first image and the second image, wherein, the first One image is one of the captured image of the drone camera and the captured image of the ground camera connected to the ground control terminal, and the second image is the camera of the drone The captured image of the camera and another image of the grounded camera that is in communication with the ground control terminal are different from the first image; the second image is superimposed on the first image to obtain The third image.

Optionally, after the processor 102 superimposes the second image on the first image to obtain a third image, the processor 102 is further used to display the third image on the display component.

Optionally, when the processor 102 superimposes the second image onto the first image to obtain a third image, it is specifically used to: adjust the original size of the first image and/or the second image to After the adjustment, the size of the first image is larger than the size of the second image; after the adjustment, the second image is superimposed on the first image to obtain the third image.

Optionally, when the processor 102 adjusts the original size of the first image and/or the second image, it is specifically used to: adjust the original size of the second image according to the original size of the first image.

Optionally, when the processor 102 adjusts the original size of the second image according to the original size of the first image, it is specifically used to: adjust the original size of the second image according to the original size of the first image Height; adjust the original width of the second image according to the adjusted height of the second image and the ratio of the original height of the second image to the original width.

Optionally, when the processor 102 adjusts the original size of the second image according to the original size of the first image, it is specifically used to: adjust the original size of the second image according to the original size of the first image Width; adjust the original height of the second image according to the adjusted width of the second image and the ratio of the original height of the second image to the original width.

Optionally, after the second image is superimposed on the first image to obtain the third image, the processor 102 is further used to: adjust the third image according to at least one resolution supported by the display component the size of.

Optionally, when the processor 102 adjusts the size of the third image according to at least one resolution supported by the display component, it is specifically used to: determine the resolution of the third image from at least one resolution supported by the display component The resolution of an image is closest to the target resolution; according to the target resolution, the size of the third image is adjusted.

Optionally, when the processor 102 adjusts the original size of the first image and/or the second image, it is specifically used to: adjust the original size of the first image; according to the adjusted Size, adjust the original size of the second image.

Optionally, when the processor 102 adjusts the original size of the first image, it is specifically used to adjust the original size of the first image according to at least one resolution supported by the display component.

Optionally, when the processor 102 adjusts the original size of the first image according to at least one resolution supported by the display component, it is specifically used to: determine and determine from the at least one resolution supported by the display component The resolution of the first image is closest to the target resolution; according to the target resolution, the original size of the first image is adjusted.

Optionally, when the processor 102 adjusts the original size of the second image according to the adjusted size of the first image, it is specifically used to: adjust the first size according to the adjusted size of the first image The original height of the second image; adjusting the original width of the second image according to the adjusted height of the second image and the ratio of the original height of the second image to the original width.

Optionally, when the processor 102 adjusts the original size of the second image according to the adjusted size of the first image, it is specifically used to: adjust the first size according to the adjusted size of the first image The original width of the second image; adjusting the original height of the second image according to the adjusted width of the second image and the ratio of the original height of the second image to the original width.

Optionally, when the processor 102 superimposes the second image on the first image to obtain the third image, it is specifically used to: superimpose the second image on the preset in the first image Position to get the third image.

Optionally, the preset position is determined according to the original size of the first image or the adjusted size of the first image.

Optionally, the ground control terminal further includes: a second communication interface; when the processor 102 acquires the first image and the second image, it is specifically used to: receive the drone and the ground through the second communication interface 104 A first encoded image sent by one of the shooting devices; decode the first encoded image to obtain the first image; and/or, receive the drone and the ground shot through the second communication interface 104 A second encoded image sent by another in the device; decoding the second encoded image to obtain the second image.

Optionally, the ground control terminal and the ground shooting device are connected by wireless communication or wired communication.

Optionally, the connection interface between the ground control terminal and the ground shooting device includes at least one of the following: composite synchronous video broadcasting signal CVBS interface, high-definition multimedia interface IDMI, digital component serial interface SDI, mobile industry processor Interface MIPI, universal serial bus USB interface.

Optionally, after the processor 102 superimposes the second image on the first image to obtain a third image, it is also used to: run an application program on the ground control terminal, and use the application program to The third image performs a sharing operation.

Optionally, the ground control terminal further includes: a second communication interface; after the processor 102 superimposes the second image on the first image to obtain a third image, it is also used to: through the second communication interface 104 Sending the third image to at least one of a live broadcast device, a relay device, and a server.

Optionally, the first communication interface 103 and the second communication interface 104 may be the same interface or different interfaces.

The specific principles and implementation manners of the ground control terminal provided by the embodiments of the present invention are similar to the foregoing embodiments, and details are not described herein again.

In this embodiment, the captured image of the drone camera and the captured image of the ground camera connected to the ground controller are acquired through the ground control terminal, and the captured image of the ground camera is superimposed on the drone In the captured image, or superimpose the captured image of the drone camera to the captured image of the ground camera, so that the superimposed image includes both the captured image of the drone camera and the ground camera The captured images of the ground shooting device may include images of characters such as flying pilots, narrators, moderators, etc., and may also include scene images around the ground control terminal, so that users can see the shooting of the drone in one image Environment and the shooting environment of the ground shooting device, thereby improving the presence of the environment around the ground control end.

In addition, this embodiment also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the image processing method described in the above embodiment.

In the several embodiments provided by the present invention, it should be understood that the disclosed device and method may be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a division of logical functions. In actual implementation, there may be other divisions, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware, or in the form of hardware plus software functional units.

The above integrated unit implemented in the form of a software functional unit may be stored in a computer-readable storage medium. The above software functional units are stored in a storage medium, and include several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute the method described in each embodiment of the present invention Partial steps. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

Those skilled in the art can clearly understand that, for convenience and conciseness of description, only the above-mentioned division of each functional module is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated by different functional modules according to needs, that is, the device The internal structure of is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not described herein again.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than limiting it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced; and these modifications or replacements do not deviate from the essence of the corresponding technical solutions of the technical solutions of the embodiments of the present invention. range.

Claims

An image processing method, applied to the ground control terminal, characterized by including:

Obtaining a first image and a second image, wherein the first image is one of a captured image of a drone camera and a captured image of a ground camera connected to the ground control terminal, and the second image is Another image different from the first image in the captured image of the shooting device of the drone and the captured image of the ground shooting device communicatively connected to the ground control terminal;

The second image is superimposed on the first image to obtain a third image.
The method according to claim 1, wherein after the second image is superimposed on the first image to obtain a third image, the method further comprises:

The third image is displayed on the display component.
The method according to claim 1 or 2, wherein the superimposing the second image on the first image to obtain a third image includes:

Adjusting the original size of the first image and/or the second image so that the size of the first image is greater than the size of the second image after adjustment;

After the adjustment, the second image is superimposed on the first image to obtain the third image.
The method according to claim 3, wherein the adjusting the original size of the first image and/or the second image comprises:

Adjust the original size of the second image according to the original size of the first image.
The method according to claim 4, wherein the adjusting the original size of the second image according to the original size of the first image includes:

Adjust the original height of the second image according to the original size of the first image;

The original width of the second image is adjusted according to the adjusted height of the second image and the ratio of the original height and the original width of the second image.
The method according to claim 4, wherein the adjusting the original size of the second image according to the original size of the first image includes:

Adjust the original width of the second image according to the original size of the first image;

The original height of the second image is adjusted according to the adjusted width of the second image and the ratio of the original height of the second image to the original width.
The method according to any one of claims 4-6, wherein after the second image is superimposed on the first image to obtain the third image, the method further includes:

Adjust the size of the third image according to at least one resolution supported by the display component.
The method according to claim 7, wherein the adjusting the size of the third image according to at least one resolution supported by the display component comprises:

Determining a target resolution closest to the resolution of the first image from at least one resolution supported by the display component;

Adjust the size of the third image according to the target resolution.
The method according to claim 3, wherein the adjusting the original size of the first image and/or the second image comprises:

Adjust the original size of the first image;

Adjust the original size of the second image according to the adjusted size of the first image.
The method according to claim 9, wherein the adjusting the original size of the first image comprises:

Adjust the original size of the first image according to at least one resolution supported by the display component.
The method according to claim 10, wherein the adjusting the original size of the first image according to at least one resolution supported by the display component comprises:

Determining a target resolution closest to the resolution of the first image from at least one resolution supported by the display component;

Adjust the original size of the first image according to the target resolution.
The method according to any one of claims 9-11, wherein the adjusting the original size of the second image according to the adjusted size of the first image includes:

Adjust the original height of the second image according to the adjusted size of the first image;

The original width of the second image is adjusted according to the adjusted height of the second image and the ratio of the original height and the original width of the second image.
The method according to any one of claims 9-11, wherein the adjusting the original size of the second image according to the adjusted size of the first image includes:

Adjust the original width of the second image according to the adjusted size of the first image;

The original height of the second image is adjusted according to the adjusted width of the second image and the ratio of the original height of the second image to the original width.
The method according to claim 3, wherein the superimposing the second image on the first image to obtain the third image includes:

The third image is obtained by superimposing the second image on a preset position in the first image.
The method according to claim 14, wherein the preset position is determined according to the original size of the first image or the adjusted size of the first image.
The method according to any one of claims 1-15, wherein the acquiring the first image and the second image includes:

Receiving the first coded image sent by one of the drone and the ground shooting device;

Decoding the first encoded image to obtain the first image; and/or,

Receiving a second coded image sent by the other of the drone and the ground shooting device;

Decoding the second encoded image to obtain the second image.
The method according to any one of claims 1 to 16, wherein the ground control terminal and the ground shooting device are connected by wireless communication or wired communication.
The method according to claim 17, wherein the connection interface between the ground control terminal and the ground shooting device includes at least one of the following:

Composite synchronous video broadcast signal CVBS interface, high-definition multimedia interface IDMI, digital component serial interface SDI, mobile industry processor interface MIPI, universal serial bus USB interface.
The method according to claim 1, wherein after the second image is superimposed on the first image to obtain a third image, the method further comprises:

Run an application program in the ground control terminal, and perform a sharing operation on the third image through the application program.
The method according to claim 1, wherein after the second image is superimposed on the first image to obtain a third image, the method further comprises:

Sending the third image to at least one of a live broadcast device, a relay device, and a server.
A ground control terminal, characterized by comprising: a memory, a processor and a first communication interface;

The first communication interface is used for communication connection with the ground shooting device;

The memory is used to store program codes;

The processor calls the program code, and when the program code is executed, it is used to perform the following operations:

Acquiring a first image and a second image, wherein the first image is one of a captured image of a drone camera and a captured image of the ground camera connected to the ground control terminal, The second image is another image different from the first image in the captured image of the shooting device of the drone and the captured image of the ground shooting device communicatively connected to the ground control terminal;

The second image is superimposed on the first image to obtain a third image.
The ground control terminal according to claim 21, wherein after the processor superimposes the second image onto the first image to obtain a third image, the processor is further used to:

The third image is displayed on the display component.
The ground control terminal according to claim 21, wherein when the processor superimposes the second image on the first image to obtain a third image, it is specifically used to:

Adjusting the original size of the first image and/or the second image so that the size of the first image is greater than the size of the second image after adjustment;

After the adjustment, the second image is superimposed on the first image to obtain the third image.
The ground control terminal according to claim 23, wherein when the processor adjusts the original size of the first image and/or the second image, it is specifically used to:

Adjust the original size of the second image according to the original size of the first image.
The ground control terminal according to claim 24, wherein when the processor adjusts the original size of the second image according to the original size of the first image, it is specifically used to:

Adjust the original height of the second image according to the original size of the first image;

The original width of the second image is adjusted according to the adjusted height of the second image and the ratio of the original height and the original width of the second image.
The ground control terminal according to claim 24, wherein when the processor adjusts the original size of the second image according to the original size of the first image, it is specifically used for:

Adjust the original width of the second image according to the original size of the first image;

The original height of the second image is adjusted according to the adjusted width of the second image and the ratio of the original height of the second image to the original width.
The ground control terminal according to any one of claims 24-26, wherein after the processor superimposes the second image on the first image to obtain the third image, the processor is further used to:

Adjust the size of the third image according to at least one resolution supported by the display component.
The ground control terminal according to claim 27, wherein when the processor adjusts the size of the third image according to at least one resolution supported by the display component, it is specifically used to:

Determining a target resolution closest to the resolution of the first image from at least one resolution supported by the display component;

Adjust the size of the third image according to the target resolution.
The ground control terminal according to claim 23, wherein when the processor adjusts the original size of the first image and/or the second image, it is specifically used to:

Adjust the original size of the first image;

Adjust the original size of the second image according to the adjusted size of the first image.
The ground control terminal according to claim 29, wherein when the processor adjusts the original size of the first image, it is specifically used to:

Adjust the original size of the first image according to at least one resolution supported by the display component.
The ground control terminal according to claim 30, wherein when the processor adjusts the original size of the first image according to at least one resolution supported by the display component, it is specifically used to:

Determining a target resolution closest to the resolution of the first image from at least one resolution supported by the display component;

Adjust the original size of the first image according to the target resolution.
The ground control terminal according to any one of claims 29 to 31, wherein the processor is specifically used when adjusting the original size of the second image according to the adjusted size of the first image :

Adjust the original height of the second image according to the adjusted size of the first image;

The original width of the second image is adjusted according to the adjusted height of the second image and the ratio of the original height and the original width of the second image.
The ground control terminal according to any one of claims 29 to 31, wherein the processor is specifically used when adjusting the original size of the second image according to the adjusted size of the first image :

Adjust the original width of the second image according to the adjusted size of the first image;

The original height of the second image is adjusted according to the adjusted width of the second image and the ratio of the original height of the second image to the original width.
The ground control terminal according to claim 23, wherein when the processor superimposes the second image onto the first image to obtain the third image, it is specifically used to:

The third image is obtained by superimposing the second image on a preset position in the first image.
The ground control terminal according to claim 34, wherein the preset position is determined according to the original size of the first image or the adjusted size of the first image.
The ground control terminal according to any one of claims 21 to 35, wherein the ground control terminal further comprises: a second communication interface;

When the processor acquires the first image and the second image, it is specifically used to:

Receiving the first coded image sent by one of the drone and the ground shooting device through the second communication interface;

Decoding the first encoded image to obtain the first image; and/or,

Receiving a second coded image sent by the other of the drone and the ground shooting device through the second communication interface;

Decoding the second encoded image to obtain the second image.
The ground control terminal according to any one of claims 21 to 36, wherein the ground control terminal and the ground photographing device are connected by wireless communication or wired communication.
The ground control terminal according to claim 37, wherein the connection interface between the ground control terminal and the ground shooting device includes at least one of the following:

Composite synchronous video broadcast signal CVBS interface, high-definition multimedia interface IDMI, digital component serial interface SDI, mobile industry processor interface MIPI, universal serial bus USB interface.
The ground control terminal according to claim 21, wherein after the processor superimposes the second image onto the first image to obtain a third image, the processor is further used to:

Run an application program in the ground control terminal, and perform a sharing operation on the third image through the application program.
The ground control terminal according to claim 21, wherein the ground control terminal further comprises: a second communication interface;

After the processor superimposes the second image on the first image to obtain a third image, the processor is further used to:

The third image is sent to at least one of the live broadcast device, the relay device and the server through the second communication interface.
A computer-readable storage medium, characterized in that a computer program is stored thereon, the computer program is executed by a processor to implement the method according to any one of claims 1-20.