KR20180078931A - A drone and a drone system which has a switching transmission capability of video streaming from dual cameras - Google Patents

Abstract

The present invention relates to a drone. The drone of the present invention comprises: an EO camera for shooting a first image; an IR camera for shooting a second image; a first converter for converting the first image from the EO camera into a set standard; a second converter for converting the second image from the IR camera into a set standard; a camera module multiplexer for selectively receiving the first image and the second image from the first converter and the second converter; and a control unit for outputting a control signal for selection to the camera module multiplexer such that the first image or the second image is selectively received from the camera module multiplexer. The drone of the present invention can process two images by using one image processing board and one multiplexer. Therefore, lightening of the drone can be implemented.

Description

Description of the Related Art [0002] A drone and a drone system having a switching transmission function of video streaming from a dual camera, which has a switching transmission capability of video streaming from dual cameras,

The present invention relates to a drones and drones system incorporating an EO camera and an IR marketer. More particularly, the present invention relates to a drones and a drone system for switching transmission of video streaming selectively transmitting an image captured by an EO camera and an IR camera to the outside.

Unmanned aerial vehicles, aka drone, have been developed and commercialized. Drones, for example, are equipped with cameras and various sensors to fly unmanned. Drones are used for a wide range of applications, for example, environmental monitoring, broadcast relaying, sports relaying, pesticide application, road surveillance, security, military, leisure and sports.

Conventional drones can be equipped with EO cameras and IR cameras, which can be applied to the environment, forest fire, and disaster relief. The EO camera is an optical high performance camera, and the image of the EO camera should be transmitted to the terrestrial control terminal or the video server. An IR camera is an infrared camera that transmits nighttime or special-purpose images to a terrestrial control terminal or a video server on the ground.

In this case, when the image board supporting two cameras is mounted on the dron, the weight or resources of the dron become large. Since the dragon is a flying body flying by the battery, when the weight of the dragon becomes heavy, the capacity of the battery must be increased and the flying time is shortened.

It is an object of the present invention to provide a dron equipped with a dual camera of an EO camera and an IR camera, which selectively drives each camera, switches video streams photographed by the cameras, and transmits them to a terrestrial control terminal or a video server on the ground Thereby providing a dron and dron system capable of reducing the resources and flying for a long time.

According to an aspect of the present invention, there is provided a drones comprising: an EO camera for capturing a first image; An IR camera for capturing a second image; A first converter for converting the first image from the EO camera into a predetermined standard; A second converter for converting the second image from the IR camera to a predetermined standard; A camera module multiplexer for selectively receiving the first image and the second image from the first converter and the second converter; And a controller for outputting a selection control signal to the camera module multiplexer so as to selectively receive the first image or the second image from the camera module multiplexer. The control unit of the drone transmits a power-on command to one of the two cameras, and selectively controls the camera module multiplexer to receive an image of a camera that is powered on and captures an image. Therefore, the hardware resources can be made compact and lightweight.

The control unit may transmit a power on command to the EO camera or the IR camera, and may select the first image from the first converter when the EO camera is powered on, And outputs the selection control signal to select and receive the second image from the second converter when the IR camera is powered on.

According to another aspect of the present invention, there is provided a drone system comprising: an EO camera that captures a first image; an IR camera that captures a second image; a first converter that converts the first image into a predetermined standard; A second converter for converting the second image into a predetermined standard, a camera module multiplexer for selectively receiving the first image or the second image from the first converter and the second converter, And a dron control unit for outputting a selection control signal to the camera module's meltdyplexer to selectively receive a second image from the camera multiplexer; A ground control terminal for transmitting a command to the drones; And an RTSP server for selectively receiving and storing the EO camera or the first image or the second image of the IR camera in response to the command from the terrestrial control terminal. . The drones system consists of a drone, a terrestrial control terminal and an RTSP server, and the dron camera is selectively operated by a command from the terrestrial control terminal, and the dron camera module multiplexer is selected to receive the image of the currently operating camera.

When the drone and the terrestrial control terminal access the mobile communication network through the mobile communication base station, the drones and the terrestrial control terminal are assigned IP addresses from the unique ID of the drones communication module and the unique ID of the terrestrial control terminal communication module, And a mobile communication server connecting the drones and the terrestrial control terminals to each other. The mobile communication server connects the drones and the terrestrial control terminals to each other so that control commands of the holding period can be exchanged with each other.

And the RTSP server provides the first image or the second image to the another terminal when the terminal other than the terrestrial control terminal accesses the mobile communication base station. When a terminal such as a mobile terminal or a notebook computer other than the terrestrial control terminal is connected to the mobile network, the RTSP server transmits the camera image to another connected terminal.

In the present invention, the drones and drone systems have built-in dual cameras, and selectively receive and process camera images, thereby making hardware resources smaller and lighter.

In addition, devices such as a mobile phone, a mobile device, a notebook PC, and the like, as well as a drones and a ground control device, can be connected to a mobile communication network and receive video streaming images taken by a drones from an RTSP server.

1 is a use state diagram of a drones and drones system according to the present invention.
2 is a block diagram of a drones according to the present invention.
FIG. 3 is a conceptual diagram illustrating a connection process using a mobile communication network between a drones and a ground control terminal in a dronesystem according to the present invention, and transmitting a camera control signal to a drones from a terrestrial control terminal and receiving image streaming information.

Embodiments of the present invention will be described in detail with reference to the accompanying drawings. The features and advantages of the present invention will become more apparent from the following description of the preferred embodiments with reference to the accompanying drawings.

1 is a use state diagram of a drones and drones system according to the present invention.

The dron system 1000 according to the present invention includes a drones 100, a mobile communication base station 410, an RTSP server 200, and a terrestrial control terminal 300.

The drones 100 and the terrestrial control terminals 300 are connected to the mobile communication network through the mobile communication base station 410, respectively.

The mobile communication base station 410 corresponds to a mobile communication network infrastructure for recognizing a wireless communication terminal such as a smart phone and transmitting and receiving voice and data. As examples of mobile communication technologies, technologies such as 3G, 4G, and 5G are being commercialized.

In the case of the Republic of Korea, mobile telecommunication companies such as KT and SKT install basic infrastructure for mobile communication on the territory of the Republic of Korea to connect mobile phones and enable voice and data communication.

As in the US and China, mobile telecommunications companies also install mobile telecommunications bases in their territories and provide voice and data communications to subscribers.

The present invention is to connect the terrestrial control terminal 300 for controlling the drones 100 and the drones 100 to the mobile communication wireless infrastructure so that the drones are controlled at a remote place within the space supported by the mobile communication network connected to the drones .

The drone 100 is equipped with two cameras such as an EO camera 10 and an IR camera 20, capturing an image suitable for each application and transmitting the captured image stream to an external device. (See Fig. 2)

The terrestrial control terminal 300 transmits the dron control command to the drones 100 via the mobile communication base station 420 in a wireless manner.

The terrestrial control terminal 300 is also called a GCS terminal 300. GCS is an abbreviation of Ground Control System and means a terminal that controls the drones on the ground.

The drone control command may be, for example, an instruction for imaging an EO camera 10 or an IR camera 20 mounted on the drone 100. [ The drone control command is not limited to the image shooting command for the cameras.

The terrestrial control terminal 300 may be used to select one of the two cameras and transmit the command.

The dron system 1000 of the present invention includes an RTSP server 200. The RTSP server 200 stores unique IDs of wireless communication modules installed in all drones 100 manufactured and sold. In addition, unique IDs of wireless communication modules mounted on the terrestrial control terminal 300 corresponding to the drone 100 are stored in pairs.

The RTSP server 200 can transmit and receive data to and from the dron 100 and the terrestrial control terminal 300 through the communication module It checks the unique ID to determine whether the two devices are connected to each other, and authenticates and connects the two devices.

When the drones 100 and the terrestrial control terminal 300 are connected to the mobile communication network, the mobile communication server 400 of the mobile communication company (see FIG. 3) And gives an IP address.

The RTSP server 200 receives the IP address from the drone 100 and the terrestrial control terminal 300, and then connects the two devices to each other.

The drones 100 and the terrestrial control terminals 300 can be connected to each other at a remote site supported by the mobile communication base station without restriction on the distance.

RTSP of the RTSP server 200 is an abbreviation of Real Time Streaming Service Protocol. The RTSP server 200 also receives and stores or retransmits the video stream captured by the drones 100 in real time.

It is possible to selectively control the two cameras mounted on the drones 100 by using the terrestrial control terminal 300 and to receive the images photographed by the respective cameras via the terrestrial control terminal 300 at a remote place.

At this time, video streaming can be received through the RTSP server 200. In some cases, the terrestrial control terminal 300 can receive directly from the drones 100.

If the portable terminal 500 or the personal computer 600 is connected to the mobile communication network and the IP address of the dron 100 is received and connected to the drones 100, The EO camera 10 and the IR camera 20 of the portable terminal 100 can selectively receive the image stream.

The portable terminal 500 or the embedded computer 600 may receive the video stream directly through the RTSP server 200 or directly from the drones 100.

It is possible to control the operation of the drones 100 through a separate cordless drone operation controller not shown in FIG.

2 is a block diagram of a drones 100 according to the present invention.

The drones 100 include a drones control unit 46, an EO camera 10, an IR camera 20, an LTE modem 50 and an image processing board 40.

The drone 100 also includes a drive unit such as a propeller, a power unit such as a battery, a memory containing various sensors and a flight control program, and the like.

The EO camera 10 refers to an electro-optical camera and corresponds to a general optical camera. The EO camera 10 captures a precise optical image.

The IR camera 120 corresponds to an infrared camera and captures a special image in a special case such as night.

The images captured by the EO camera 10 and the IR camera 120 are transmitted to an external device in a streaming manner.

The LTE modem 50 is mounted on the USB port 42 of the image processing board 40 and serves as a drone communication module for connecting the drones 100 to the mobile communication base station 410. The USB port corresponds to an embodiment, and the LTE modem 50 can be mounted with or without a port other than the USB port.

The image processing board 40 receives an image from the EO camera 10 and the IR camera 20, processes the image, and transmits the processed image to the outside in a streaming manner using the mobile communication network.

At this time, the video stream is transmitted to the RTSP server 200 by using an image streamer 44 composed of hardware and / or software in the image processing board 40.

The camera module multiplexer 30 is located at the input end of the image processing board 40.

The camera module multiplexer 30 has two input sections.

The first image photographed by the EO camera 10 is converted into a standard specified by the HDMI to MIPI converter board 12, which is a video converter.

HDMI stands for High Definition Multimedia Interface, which means a high definition multimedia interface, and is one of the uncompressed digital video interface standards.

MIPI stands for Mobile Industry Processor Interface and is a standard for open membership organization for mobile ecosystem.

The normal EO camera 10 captures an image of the HDMI standard and converts it into the MIPI standard for the image processing board 40 and the camera module multiplex 30 that support the MIPI standard.

The second image photographed by the IR camera 20 is converted to a standard that has been set via the CVBS to HDMI board 22 and the HDMI to MIPI convert board 24, which are video converters.

CVBS stands for Composite Video Baseband Signal, which converts composite video signals taken by an IR camera to HDMI. And then converts the HDMI to MIPI.

The first input unit of the camera module multiplexer 30 converts the first image from the EO camera 10 into a MIPI standard.

The second input portion of the camera module multiplexer 30 converts the second image from the IR camera 20 into the MIPI standard in the same manner.

At this time, the dron control unit 46 of the image processing board 40 outputs a camera selection signal to the camera module multiplexer 30 to select either the first input unit (EO camera image) or the second input unit (IR camera image) .

Upon receiving an instruction to turn on the EO camera 10 from the terrestrial control terminal 300, the drones control unit 46 transmits a control command to turn on the EO camera 10.

The EO camera 10 is turned on by the instruction of the drones control unit 46 and shoots the first image. The photographed first image is converted into the MIPI hall via the HDMI to MIPI converter board 12 and input to the first input unit of the camera module multiplexer 30.

The drone control unit 46 recognizes that the current state of the EO camera 10 is in the operating state, and outputs a control signal to select the " EO camera "

The camera module multiplexer 30 outputs a first image of the EO camera 10 to the image processing board 40 in response to a camera selection signal from the control unit.

Similarly, when receiving a command to turn on the IR camera 20 from the terrestrial control terminal 300, the drones control unit 46 turns the IR camera 20 on.

The IR camera 20 takes a second image, converts the second image into a MIPI sight and inputs it to the second input of the camera module multiplexer 30.

Since the drone control unit 46 knows the operation state of the current IR camera 20, it outputs a control signal to select the " IR camera "

The camera module multiplexer 30 outputs the second image of the IR camera 10 to the image processing board 40 in accordance with the command of the camera selection signal.

Accordingly, the image from two cameras can be processed and transmitted in real time using one image processing board 40 and the camera module multiplexer 30.

FIG. 3 is a conceptual diagram illustrating a connection process using a mobile communication network between a drones and a ground control terminal in a dronesystem according to the present invention, and transmitting a camera control signal to a drones from a terrestrial control terminal and receiving image streaming information.

First, the connection procedure between the drones 100 and the terrestrial control terminal 300 is shown. (S100-S300)

Next, transmission of control signals and reception of video streaming information between the drones 100 and the terrestrial control terminal 300 are shown. (S320 - S390)

The drones 100 are turned on by the user. (S100)

And transfers the unique ID stored in the drone communication module installed in the drone 100 to the mobile communication server 400. (S110)

The mobile communication server 400 is a server managed by a mobile communication company and receives a unique ID of the drones 100 detected by the mobile communication base station 410.

The mobile communication server 400 assigns a dynamic IP address to the drone 100 and transmits the assigned dynamic IP address to the drone. (S120)

A dynamic IP address refers to a dynamic Internet Protocol (IP) address. Since a drone is not always powered on, it only assigns an IP address when the drone is powered on. Therefore, even when the same drones are powered on, different IP addresses can be assigned.

The drone 100 transmits its unique ID and the dynamic IP address allocated from the mobile communication server 400 to the RTSP server 200. [ (S130)

Meanwhile, the terrestrial control terminal (GCS terminal 300) is also turned on by the user. (S200)

The terrestrial control terminal 300 transmits the unique ID stored in the built-in terminal communication module to the mobile communication server 400 via the mobile communication base station 410. (S210)

The mobile communication server 400 allocates a dynamic IP address to the terrestrial control terminal 300 and transmits the assigned dynamic IP address to the terrestrial control terminal 300. [ (S220)

The terrestrial control terminal 300 transmits its own unique ID and the dynamic IP address allocated from the mobile communication server 400 to the RTSP server 200. (S230)

The RTSP server 200 is a server managed by a drones manufacturer or service company. The drone IP management unit of the RTSP server 200 stores in advance the unique IDs of all the drone and ground control terminals produced and sold.

Therefore, it is determined from the unique IDs and IP addresses received from the drones 100 and the terrestrial control terminal 300 whether the corresponding drones 100 and terrestrial control terminals 300 are legitimate devices capable of connecting with each other. Authenticates the legitimate drones 100 and the terrestrial control terminal 300 and connects the authenticated drones 100 and terrestrial control terminals 300 to each other using respective IP addresses. (S300)

When the drones 100 and the terrestrial control terminal 300 are connected to each other, a control command sent from the terrestrial control terminal 300 is transmitted to the dron 100, and the dron 100 is operated according to the control command. (S320, S330)

At this time, the control command of the terrestrial control terminal 300 may be an instruction to turn on the EO camera. The control command may be transmitted to the drone 100 via the drone management server 200 or directly to the drone 100 without going through the drone management server 200. [

The drone 100 operates the internal EO camera 10 in response to the EO camera on command from the terrestrial control terminal 300.

Thereafter, the first image from the EO camera 10 is converted into a predetermined standard, the camera module multiplexer 30 is selectively operated to process the image of the EO camera 10, and the image of the EO camera 10 And transmits the streaming to the RTSP server. (S340)

The RTSP server 200 stores the image received from the drone 100 or transmits it to the terrestrial control terminal 100, the portable terminal 500, and the personal computer 600. (S350)

The terrestrial control terminal 300 transmits a control command to turn on the IR camera. (S360)

The control command may be transmitted to the drone 100 via the drone management server 200 or directly to the drone 100 without going through the drone management server 200. [ (S370)

The drones 100 operate the internal IR camera 20 in response to the IR camera on command from the terrestrial control terminal 300.

Thereafter, the second image from the IR camera 20 is converted into a predetermined standard, the camera module multiplexer 30 is selectively operated to process the image of the IR camera 20, and the streaming of the IR camera 20 And transmits the image to the RTSP server 200. (S380)

The RTSP server 200 stores the second image of the IR camera 20 received from the drones 100 or transmits the second image to the terrestrial control terminal 100, the portable terminal 500, and the personal computer 600. (S390)

The present invention is characterized in that a drones 100 equipped with an EO camera 10 and an IR camera 20 use one image processing board 40 and one camera module multiplexer 30 for processing respective images, Is small and lightweight. Therefore, the drones are stable and can fly for a long time.

The present invention is not limited to the above-described embodiments, and it is possible to control the image of the drone at a remote place or real-time reception of the video stream using the concept of the present invention.

1000: Dron system
100: Drones
10: EO camera
12: HDMI to MIPI Converted Board
20: IR camera
22: CVBS to HDMI Conversion Board
24: HDMI to MIPI Converted Board
30: Camera module multiplexer
40: Image processing board
42: USB port
44: Video Streamer
46: Drone control
50: LTE modem
200: RTSP server
300: Ground control terminal (GCS terminal)
400: mobile communication server
410: mobile communication base station
500: mobile phone terminal
600: personal computer

Claims (5)

In the drones,
An EO camera for capturing a first image;
An IR camera for capturing a second image;
A first converter for converting the first image from the EO camera into a predetermined standard;
A second converter for converting the second image from the IR camera to a predetermined standard;
A camera module multiplexer for selectively receiving the first image and the second image from the first converter and the second converter; And
And a controller for outputting a selection control signal to the camera module multiplexer to selectively receive the first image or the second image from the camera module multiplexer.
The method according to claim 1,
The control unit may transmit a power on command to the EO camera or the IR camera, and may select the first image from the first converter when the EO camera is powered on, And outputs the selection control signal to select and receive the second image from the second converter when the IR camera is powered on.
In the drones system,
An EO camera for capturing a first image, an IR camera for capturing a second image, a first converter for converting the first image into a predetermined standard, a second converter for converting the second image to a predetermined standard, And a camera module multiplexer for selectively receiving the first image or the second image from the second converter and a camera module multiplexer for selectively receiving the first image or the second image from the camera multiplexer A dron including a dron controller for outputting a selection control signal;
A ground control terminal for transmitting a command to the drones; And
And an RTSP server for selectively receiving and storing the EO camera or the first image or the second image of the IR camera in response to the command from the terrestrial control terminal.
The method of claim 3,
When the drone and the terrestrial control terminal access the mobile communication network through the mobile communication base station, the drones and the terrestrial control terminal are assigned IP addresses from the unique ID of the drones communication module and the unique ID of the terrestrial control terminal communication module, And a mobile communication server connecting the drones and the terrestrial control terminals to each other.
5. The method of claim 4,
Wherein the RTSP server provides the first image or the second image to the another terminal when the terminal other than the terrestrial control terminal accesses the mobile communication base station.

Images