CN113507638B

CN113507638B - Display equipment and screen projection method

Info

Publication number: CN113507638B
Application number: CN202110766329.9A
Authority: CN
Inventors: 徐建锋
Original assignee: Vidaa Netherlands International Holdings BV
Current assignee: Vidaa Netherlands International Holdings BV
Priority date: 2021-07-07
Filing date: 2021-07-07
Publication date: 2023-05-05
Anticipated expiration: 2041-07-07
Also published as: CN113507638A

Abstract

The application discloses a display device and a screen projection method, comprising the following steps: a display; a controller in communication with the display, the controller configured to: responding to the received screen throwing signal, and starting an RTSP interaction thread; receiving screen throwing data sent by screen throwing equipment, wherein the screen throwing data comprises a plurality of video frames, and the video frames comprise a plurality of RTP data packets; detecting an RTP data packet, and sending an RTSP message to the screen throwing equipment when the current RTP data packet is discontinuous with the last received data packet so as to enable the screen throwing equipment to reduce the resolution; and when the current RTP data packet is continuous with the last received RTP data packet, controlling the display to play the screen throwing data. According to the method and the device, the display device informs the screen throwing device to reduce the resolution of the screen throwing data when the playing is not smooth, so that the data volume is reduced, the display device achieves the effect of smooth playing after receiving the low-resolution screen throwing data, a user can smoothly watch the screen throwing data, and the user experience is improved.

Description

Display equipment and screen projection method

Technical Field

The application relates to the technical field of display, in particular to display equipment and a screen projection method.

Background

The display device is a television product which can realize the bidirectional man-machine interaction function and integrates multiple functions of video, entertainment, data and the like. In order to meet the diversified demands of users, the display device is provided with a plurality of applications such as screen projection, video and entertainment, and the like, and interacts and exchanges information with the users through a user interface.

For the screen-casting application, miracast screen-casting is an important resource sharing mode between display devices, and media sharing is achieved through RTSP (Real TimeStreaming Protoco, real-time streaming protocol). For example, when Miracast is used for screen projection, the mobile phone needs to acquire video frames of media to be shared, compress and encode the video frames, and transmit the video frames to the smart television. The intelligent television decompresses the received media data and displays the decompressed media data. Because some mobile phones still adopt old version compression algorithm when compressing for the data volume that sends to intelligent TV is too big, and intelligent TV comes not as far as to handle, thereby leads to the media to play delay or intelligent TV flower screen problem, makes user experience effect not good.

Disclosure of Invention

The application provides display equipment and a screen projection method, which are used for solving the technical problem that in the prior art, due to the fact that the data size of media assets is large after mobile phone compression, intelligent television processing is not timely carried out, so that the experience effect of a user is poor.

In order to solve the technical problems, the embodiment of the application discloses the following technical scheme:

in a first aspect, an embodiment of the present application discloses a display device, including:

a display;

a controller in communication with the display, the controller configured to:

in response to receiving the screen throwing signal, starting an RTSP interaction thread to enable the display equipment to conduct data interaction with the screen throwing equipment;

receiving screen projection data sent by the screen projection equipment, wherein the screen projection data comprises a plurality of video frames, and the video frames comprise a plurality of RTP data packets;

the RTP data packets are detected and,

when the current RTP data packet is discontinuous with the last received RTP data packet, sending an RTSP message to the screen throwing device so that the screen throwing device reduces the resolution according to the RTSP message; and when the current RTP data packet is continuous with the last received RTP data packet, controlling the display to play the screen throwing data.

In some embodiments, the RTP packet is marked with a packet sequence number, and in the step of sending an RTSP message to the screen-casting device when the current RTP packet is discontinuous with the last received RTP packet, the controller is further configured to:

detecting whether the packet sequence number of the current RTP data packet is continuous with the packet sequence number of the last received RTP data packet; and when the packet sequence numbers are discontinuous, sending RTSP information to the screen throwing equipment.

In some embodiments, prior to said sending an RTSP message to the screen casting device, the controller is further configured to:

when the packet sequence numbers are discontinuous, calculating the number of lost RTP data packets; and when the number of the lost RTP data packets is larger than a preset value, sending an RTSP message to the screen throwing equipment.

In some embodiments, in the step of counting the number of lost RTP packets when the packet sequence number is discontinuous, the controller is further configured to:

and the packaging sequence number of the current RTP data packet is differed from the packaging sequence number of the last received RTP data packet, so that the lost number of the RTP data packets is obtained.

In some embodiments, after sending an RTSP message to the screen casting device, the controller is further configured to:

recording the current playing progress of the screen throwing data; receiving a feedback message of successful resolution modification sent by the screen-throwing equipment, and receiving screen-throwing data after resolution reduction sent by the screen-throwing equipment; and based on the playing progress, controlling the display to continuously play the screen throwing data according to the reduced resolution.

a feedback message of successful resolution modification sent by the screen throwing equipment is not received; and stopping the RTSP interaction thread, and controlling the display to exit the playing process of the screen throwing data.

In a second aspect, an embodiment of the present application discloses a screen projection method, where the screen projection method includes:

the RTP data packets are detected and,

when the current RTP data packet is discontinuous with the last received RTP data packet, sending an RTSP message to the screen throwing device so that the screen throwing device reduces the resolution according to the RTSP message; and when the current RTP data packet is continuous with the last received RTP data packet, controlling a display to play the screen throwing data.

Compared with the prior art, the beneficial effects of this application are:

the application provides display equipment and a screen projection method, when a user sends a screen projection signal to the display equipment through the screen projection equipment, the display equipment responds to the screen projection signal, and the screen projection equipment and the display equipment start RTSP interaction threads. The screen throwing device compresses and packages the screen throwing data into a plurality of RTP data packets, and sends the RTP data packets to the display device. The display device receives and detects the RTP data packet sent by the screen throwing device, and judges whether the current RTP data packet is continuous with the last received RTP data packet. If the display device is continuous, the display device displays the screen throwing data on the display, and if the display device is discontinuous, the display device sends RTSP information to the screen throwing device to request the screen throwing device to reduce the resolution ratio. In the application, the display device reduces the resolution of the screen data playing by sending the RTSP message for indicating the screen throwing device to reduce the resolution, so that the data size is reduced, the display device achieves the effect of smooth playing after receiving the screen throwing data after the resolution reduction sent by the screen throwing device, so that a user can smoothly watch the screen throwing data, and the user experience is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

In order to more clearly illustrate the technical solutions of the present application, the drawings that are needed in the embodiments will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.

A schematic diagram of an operational scenario between a display device and a control apparatus according to some embodiments is schematically shown in fig. 1;

a hardware configuration block diagram of the control apparatus 100 according to some embodiments is exemplarily shown in fig. 2;

a hardware configuration block diagram of a display device 200 according to some embodiments is exemplarily shown in fig. 3;

a schematic diagram of the software configuration in a display device 200 according to some embodiments is exemplarily shown in fig. 4;

a flow diagram of a screen projection method according to some embodiments is schematically shown in fig. 5;

another flow diagram of a screen casting method according to some embodiments is schematically shown in fig. 6;

a timing diagram of a screen casting method according to some embodiments is illustrated in fig. 7.

Detailed Description

For purposes of clarity and implementation of the present application, the following description will make clear and complete descriptions of exemplary implementations of the present application with reference to the accompanying drawings in which exemplary implementations of the present application are illustrated, it being apparent that the exemplary implementations described are only some, but not all, of the examples of the present application.

It should be noted that the brief description of the terms in the present application is only for convenience in understanding the embodiments described below, and is not intended to limit the embodiments of the present application. Unless otherwise indicated, these terms should be construed in their ordinary and customary meaning.

The terms "first," second, "" third and the like in the description and in the claims and in the above-described figures are used for distinguishing between similar or similar objects or entities and not necessarily for limiting a particular order or sequence, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements explicitly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code that is capable of performing the function associated with that element.

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in fig. 1, a user may operate the display device 200 through the smart device 300 or the control apparatus 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or bluetooth protocol communication, and other short-range communication modes, and the display device 200 is controlled by a wireless or wired mode. The user may control the display device 200 by inputting user instructions through keys on a remote control, voice input, control panel input, etc.

In some embodiments, a smart device 300 (e.g., mobile terminal, tablet, computer, notebook, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on a smart device.

In some embodiments, the display device 200 may also perform control in a manner other than the control apparatus 100 and the smart device 300, for example, the voice command control of the user may be directly received through a module configured inside the display device 200 device for acquiring voice commands, or the voice command control of the user may be received through a voice control device configured outside the display device 200 device.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be permitted to make communication connections via a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200. The server 400 may be a cluster, or may be multiple clusters, and may include one or more types of servers.

Fig. 2 exemplarily shows a block diagram of a configuration of the control apparatus 100 in accordance with an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction of a user and convert the operation instruction into an instruction recognizable and responsive to the display device 200, and function as an interaction between the user and the display device 200.

Fig. 3 shows a hardware configuration block diagram of the display device 200 in accordance with an exemplary embodiment.

In some embodiments, display apparatus 200 includes at least one of a modem 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, memory, a power supply, a user interface.

In some embodiments the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, a first interface for input/output to an nth interface.

In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, for receiving image signals from the controller output, for displaying video content, image content, and a menu manipulation interface, and for manipulating a UI interface by a user.

In some embodiments, the display 260 may be a liquid crystal display, an OLED display, a projection device, and a projection screen.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, or other network communication protocol chip or a near field communication protocol chip, and an infrared receiver. The display device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

In some embodiments, the user interface may be configured to receive control signals from the control device 100 (e.g., an infrared remote control, etc.).

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for capturing the intensity of ambient light; alternatively, the detector 230 includes an image collector such as a camera, which may be used to collect external environmental scenes, user attributes, or user interaction gestures, or alternatively, the detector 230 includes a sound collector such as a microphone, or the like, which is used to receive external sounds.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, or the like. The input/output interface may be a composite input/output interface formed by a plurality of interfaces.

In some embodiments, the modem 210 receives broadcast television signals via wired or wireless reception and demodulates audio-video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, the controller 250 and the modem 210 may be located in separate devices, i.e., the modem 210 may also be located in an external device to the main device in which the controller 250 is located, such as an external set-top box or the like.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored on the memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command to select a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other operable control. The operations related to the selected object are: displaying an operation of connecting to a hyperlink page, a document, an image, or the like, or executing an operation of a program corresponding to the icon.

In some embodiments the controller includes at least one of a central processing unit (Central Processing Unit, CPU), video processor, audio processor, graphics processor (Graphics Processing Unit, GPU), RAM Random Access Memory, RAM), ROM (Read-Only Memory, ROM), first to nth interfaces for input/output, a communication Bus (Bus), and the like.

And the CPU processor is used for executing the operating system and application program instructions stored in the memory and executing various application programs, data and contents according to various interaction instructions received from the outside so as to finally display and play various audio and video contents. The CPU processor may include a plurality of processors. Such as one main processor and one or more sub-processors.

In some embodiments, a graphics processor is used to generate various graphical objects, such as: icons, operation menus, user input instruction display graphics, and the like. The graphic processor comprises an arithmetic unit, which is used for receiving various interactive instructions input by a user to operate and displaying various objects according to display attributes; the device also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, the video processor is configured to receive an external video signal, perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image composition, etc., according to a standard codec protocol of an input signal, and may obtain a signal that is displayed or played on the directly displayable device 200.

In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like. The demultiplexing module is used for demultiplexing the input audio and video data stream. And the video decoding module is used for processing the demultiplexed video signal, including decoding, scaling and the like. And an image synthesis module, such as an image synthesizer, for performing superposition mixing processing on the graphic generator and the video image after the scaling processing according to the GUI signal input by the user or generated by the graphic generator, so as to generate an image signal for display. And the frame rate conversion module is used for converting the frame rate of the input video. And the display formatting module is used for converting the received frame rate into a video output signal and changing the video output signal to be in accordance with a display format, such as outputting RGB data signals.

In some embodiments, the audio processor is configured to receive an external audio signal, decompress and decode the audio signal according to a standard codec protocol of an input signal, and perform noise reduction, digital-to-analog conversion, and amplification processing to obtain a sound signal that can be played in a speaker.

In some embodiments, a user may input a user command through a Graphical User Interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.

In some embodiments, a "user interface" is a media interface for interaction and exchange of information between an application or operating system and a user that enables conversion between an internal form of information and a form acceptable to the user. A commonly used presentation form of the user interface is a graphical user interface (Graphic User Interface, GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in a display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

In some embodiments, a system of display devices may include a Kernel (Kernel), a command parser (shell), a file system, and an application program. The kernel, shell, and file system together form the basic operating system architecture that allows users to manage files, run programs, and use the system. After power-up, the kernel is started, the kernel space is activated, hardware is abstracted, hardware parameters are initialized, virtual memory, a scheduler, signal and inter-process communication (IPC) are operated and maintained. After the kernel is started, shell and user application programs are loaded again. The application program is compiled into machine code after being started to form a process.

Referring to FIG. 4, in some embodiments, the system is divided into four layers, from top to bottom, an application layer (referred to as an "application layer"), an application framework layer (Application Framework layer) (referred to as a "framework layer"), a An Zhuoyun row (Android run) and a system library layer (referred to as a "system runtime layer"), and a kernel layer, respectively.

In some embodiments, at least one application program is running in the application program layer, and these application programs may be a Window (Window) program of an operating system, a system setting program, a clock program, or the like; or may be an application developed by a third party developer. In particular implementations, the application packages in the application layer are not limited to the above examples.

The framework layer provides an application programming interface (application programming interface, API) and programming framework for the application. The application framework layer includes a number of predefined functions. The application framework layer corresponds to a processing center that decides to let the applications in the application layer act. Through the API interface, the application program can access the resources in the system and acquire the services of the system in the execution.

As shown in fig. 4, the application framework layer in the embodiment of the present application includes a manager (manager), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an Activity Manager (Activity Manager) is used to interact with all activities that are running in the system; a Location Manager (Location Manager) is used to provide system services or applications with access to system Location services; a Package Manager (Package Manager) for retrieving various information about an application Package currently installed on the device; a notification manager (Notification Manager) for controlling the display and clearing of notification messages; a Window Manager (Window Manager) is used to manage bracketing icons, windows, toolbars, wallpaper, and desktop components on the user interface.

In some embodiments, the activity manager is used to manage the lifecycle of the individual applications as well as the usual navigation rollback functions, such as controlling the exit, opening, fallback, etc. of the applications. The window manager is used for managing all window programs, such as obtaining the size of the display screen, judging whether a status bar exists or not, locking the screen, intercepting the screen, controlling the change of the display window (for example, reducing the display window to display, dithering display, distorting display, etc.), etc.

In some embodiments, the system runtime layer provides support for the upper layer, the framework layer, and when the framework layer is in use, the android operating system runs the C/C++ libraries contained in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 4, the kernel layer contains at least one of the following drivers: audio drive, display drive, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (e.g., fingerprint sensor, temperature sensor, pressure sensor, etc.), and power supply drive, etc.

The hardware or software architecture in some embodiments may be based on the description in the foregoing embodiments, and in some embodiments may be based on other similar hardware or software architectures, which may implement the technical solutions of the present application.

Based on the above display device 200, a user may download a screen-throwing application in an application center of the display device 200, where the screen-throwing application may be an application program based on a Miracast (wireless display) function, and the Miracast uses a WIFI Point-to-Point (Point 2 Point) network transmission function to transmit screen-throwing data of the screen-throwing device to the display device 200 for playing, so as to achieve the purpose of resource sharing, where the screen-throwing data may be audio and video data, and the screen-throwing device may be a terminal device such as a smart phone, a tablet computer, a computer, and the like.

When Miracast realizes the screen-throwing function, a WFD (Wi-Fi Display) interaction protocol is required to be utilized to realize media resource sharing. Whereas the WFD interaction protocol utilizes the RTSP (Real Time Streaming Protocol) protocol to enable capability negotiation between the screen-casting device and the display device. According to RTSP, the screen throwing device, the display device and the stages M3 and M4, some capability parameters which need to be negotiated at two ends of the device are specified, one of the capability parameters is video capability negotiation, and the two ends negotiate the resolution, refresh rate, compression algorithm and the like of the screen throwing data to be played. The display device 200 generally expects the projection device to employ a higher compression algorithm to reduce the amount of data as much as possible by compression, but still use older versions of compression algorithms, such as h.264level 3.2 or earlier, due to the older or lower end of the partial projection device. This makes the amount of screen-throwing data sent from the screen throwing device very large, resulting in the display device 200 not being able to deal with, and thus presenting a delay in media resources or screen-throwing problems due to packet loss. In order to solve the above problems, in some embodiments, the present application provides a display device and a screen projection method. It should be noted that, the display device in the application not only refers to an intelligent television, but also refers to a computer, a tablet computer, and the like.

The screen projection process provided by the embodiment of the application is described below with reference to the accompanying drawings.

A flow diagram of a screen projection method according to some embodiments is schematically shown in fig. 5. The present application provides in some embodiments a display device 200 comprising a display 260 and a controller 250, the controller 250 being communicatively coupled to the display 260, the controller being configured to perform the screen casting process of fig. 5. With reference to fig. 5, the screen projection process is as follows:

s501: and in response to receiving the screen throwing signal, starting an RTSP interaction thread so as to enable the display device to conduct data interaction with the screen throwing device.

In some embodiments, a user may input a screen-cast signal at a screen-cast device, wherein the screen-cast device is provided with a screen-cast application that is based on a Miracast screen-cast function. For example, a screen throwing control is arranged on the screen throwing device, and a user starts a Miracast screen throwing function by clicking or selecting the screen throwing control through touch. After the Miracast screen-throwing function is started, the screen-throwing device sends a screen-throwing signal to the display device 200, and the display device 200 responds to the screen-throwing signal to start the Miracast screen-throwing function of the display device, so that the display device and the screen-throwing device perform data interaction based on the Miracast.

In some embodiments, the display device 200 starts its own P2P network when it starts the Miracast screen-casting function. Of course, when the Miracast screen-throwing function is started, the screen-throwing equipment also starts the P2P network of the screen-throwing equipment. Based on the P2P network, the display device 200 establishes a connection with the screen-throwing device, and the two ends perform interaction of the network parts to obtain IP addresses of each other.

In some embodiments, after the P2P network connection is established, the display device 200 and the screen-throwing device each start an RTSP interaction thread to perform a handshake interaction phase, so as to implement data interaction between the display device 200 and the screen-throwing device.

S502: and receiving the screen projection data sent by the screen projection equipment, wherein the screen projection data comprises a plurality of video frames, and the video frames comprise a plurality of RTP data packets.

In some embodiments, in the RTSP interactive threads of the display device 200 and the screen device, the screen device 200 parses the screen data into a plurality of video frames, each video frame is composed of a plurality of RTP data packets, and each RTP data packet is marked with a sequace number (packet sequence number), where the packet sequence number indicates the sequential continuity of the plurality of RTP data packets. The screen throwing device sends the RTP data packets to the display device 200, and the display device 200 receives the RTP data packets.

S503: and detecting the RTP data packet.

In some embodiments, the display device 200 may turn on the gatekeeper package thread after receiving the screen cast data. RTP data packets are detected by a gatekeeper-frame. Here, the guard line Cheng Zhu is to detect continuity of RTP packets.

S504: and when the current RTP data packet is continuous with the last received RTP data packet, controlling the display to play the screen throwing data.

In some embodiments, if the display device 200 can process the received screen projection data in time, that is, decode the screen projection data in time, the packet-guarding thread can detect that the RTP packet is always in a continuous state, the display device 200 removes the header of the RTP packet, sends the RTP packet to the player for playing and displays the RTP packet on the display.

S505: and when the current RTP data packet is discontinuous with the last received RTP data packet, sending an RTSP message to the screen throwing device so that the screen throwing device reduces the resolution ratio according to the RTSP message.

In some embodiments, when the gatekeeper thread detects the current RTP packet and finds that the current RTP packet is discontinuous with the last received RTP packet, that is, the display device 200 detects that the RTP packet is lost, sends an RTSP message to the screen throwing device. After receiving the RTSP message, the screen-throwing device reduces the resolution of the screen-throwing data and re-packs the screen-throwing data.

In some embodiments, in the RTSP interactive thread of the display device 200 and the screen casting device, after the display device 200 detects that the RTP packet is lost, a new RTSP message is added, which modifies the resolution by set_parameter (setting PARAMETER).

For example, in the RTSP interactive thread at the beginning, the display device 200 tells the display device 200 that the video negotiation parameters are as follows during the M3/M4 phase video stream negotiation process:

wfd_video_formats:00 01 01 02 00000080 00000000 0000000 000000 0000 00none none. According to Wi-Fi display_specification section6.1.3 (Wi-Fi standard protocol Specification section 6.1.3), 0080 in the RTSP message indicates that the message is played at 1920 x 1080p30 (resolution 1920 x 1080, refresh rate 30 fps).

After the display device 200 detects that the RTP packet is lost, an RTSP message M17 is sent to the screen-throwing device, where the content of the message is as follows:

wfd_video_formats:00 01 01 02 00000020 00000000 0000000 000000 0000 00none none. 0020 in the RTSP message indicates playback at 1280 x 720p30 (resolution 1280 x 720, refresh rate 30 fps).

In some embodiments, the display device 200 detects whether the packet sequence number of the current RTP packet is consecutive with the packet sequence number of the last received RTP packet based on the packet sequence number marked in the RTP packet. And when the packet sequence numbers are discontinuous, sending RTSP information to the screen throwing equipment.

In some embodiments, according to previous playing experience, if the number of lost RTP packets is within a certain range, the smoothness of playing the whole screen-throwing data is not basically affected. Thus, prior to the sending of RTSP messages to the screen casting device, the controller 250 is further configured to: and when the packet sequence numbers are discontinuous, calculating the number of lost RTP data packets. And when the number of the lost RTP data packets is larger than a preset value, sending an RTSP message to the screen throwing equipment. Here, the controller 250 may set a preset value to 5, and a specific value of the preset value may be set by a worker according to past experience.

In some embodiments, when detecting that the packet sequence number of the current RTP packet is discontinuous with the packet sequence number of the last received RTP packet, the packet-guarding thread in the display device 200 may further calculate the number of lost RTP packets, and make the difference between the packet sequence number of the current RTP packet and the packet sequence number of the last received RTP packet to obtain the number of lost RTP packets.

For example, the thread in the display device 200 detects that the packet number of the current RTP packet is 6, the packet number of the last received RTP packet is 3, the two packet numbers are discontinuous, and the number of the current lost RTP packets is 3 after 6-3. And 3 is smaller than the preset value 5, then the RTSP message may not be sent first. If the packet sequence number of the current RTP data packet is detected to be 9, the packet sequence number of the last received RTP data packet is detected to be 3, the two packet sequence numbers are discontinuous, and the two packet sequence numbers are different by 6. The number of lost RTP packets has exceeded the preset value of 5, the display device 200 sends an RTSP message to the screening device.

After the display device 200 transmits the RTSP message, the display device 200 needs to further determine whether the screen casting device has modified the resolution according to the message. Another flow diagram of a screen projection method according to some embodiments is schematically shown in fig. 6. In connection with fig. 6, the controller 250 also needs to be configured to perform the following process:

s601: and recording the current playing progress of the screen throwing data.

In some embodiments, after the display device 200 transmits the RTSP message, the display device 200 records the current play progress of the screen-cast data. For example, the display device 200 records that the current screen-casting data is played 48%, and for example, the display device 200 records that the current screen-casting data is played to 20 minutes and 35 seconds.

S602: and judging whether a feedback message of successful resolution modification is received.

In some embodiments, after the screen casting device receives the RTSP message, the playback resolution of the screen casting data is reduced based on the indication in the RTSP message. After the modification is successful, the screen projection device sends a feedback message of the successful modification to the display device 200, so that the display device 200 knows that the screen projection device has successfully modified the resolution. Of course, if the screen-throwing device is not successfully modified, no feedback message of successful resolution modification is sent.

S603: and receiving the screen projection data after resolution reduction sent by the screen projection equipment.

In some embodiments, after the resolution of the screen-throwing data is reduced, the screen-throwing device packages the screen-throwing data from new to new, packages the screen-throwing data after the resolution is reduced into several RTP data packets, and sends the screen-throwing data after the re-package to the display device 200. The display device 200 receives the low resolution screen projection data transmitted by the screen projection device.

S604: and based on the playing progress, controlling the display to continuously play the screen throwing data according to the reduced resolution.

In some embodiments, the display device 200, after receiving the screen-cast data after the resolution reduction, looks up the play progress of the screen-cast data recorded before, and plays the screen-cast data continuously based on the recorded play progress.

S605: and stopping the RTSP interaction thread, and controlling the display to exit the playing process of the screen throwing data.

In some embodiments, the display device 200 does not receive the feedback message that the resolution modification sent by the screen-throwing device is successful, that is, the screen-throwing device does not perform the operation of reducing the resolution, and the display device 200 may stop the RTSP interaction thread and exit the Miracast screen-throwing function, that is, exit the playing process of the screen-throwing data, because the user has problems such as delay or screen-throwing when watching the screen-throwing data at this time.

The process of dropping the screen will be further described with reference to the accompanying drawings.

A timing diagram of a screen casting method according to some embodiments is illustrated in fig. 7. As shown in fig. 7, a user triggers a screen-throwing function through a screen-throwing application program installed on the display device 200, after the display device 200 starts the Miracast screen-throwing function, a screen-throwing signal is sent to the screen-throwing device, the display device 200 and the screen-throwing device both start respective P2P networks, and a P2P network connection is established between the two. Then, both the display device 200 and the screen casting device start respective RTSP interactive threads. The screen throwing device 200 sends the screen throwing data group packet as an RTP data packet to the display device 200, and the display device 200 detects the received RTP data packet. When the number of lost RTP packets is greater than a preset value, the display apparatus 200 transmits an RTSP message to the screen casting apparatus. The screen throwing device receives the RTSP message and reduces the resolution of the screen throwing data according to the RTSP message. The screen throwing device feeds back the message of successful resolution modification to the display device 200 and feeds back the screen throwing data after resolution reduction to the display device 200. The display device 200 continuously broadcasts the screen throwing data after resolution reduction based on the previous playing progress so as to realize smooth playing of the screen throwing data.

In the application, the display device reduces the resolution of the screen data playing by sending the RTSP message for indicating the screen throwing device to reduce the resolution, so that the data size is reduced, the display device achieves the effect of smooth playing after receiving the screen throwing data after the resolution reduction sent by the screen throwing device, so that a user can smoothly watch the screen throwing data, and the user experience is improved.

Based on the same inventive concept as the display device, the embodiment of the application also provides a screen projection method, which comprises the following steps: in response to receiving the screen-cast signal, the display device 200 initiates an RTSP interaction thread to cause the display device 200 to interact data with the screen-cast device. The display device 200 receives the screen projection data sent by the screen projection device, wherein the screen projection data comprises a plurality of video frames, and the video frames comprise a plurality of RTP data packets. After receiving the screen-throwing data, the display device 200 needs to detect the RTP data packet, and when the current RTP data packet is discontinuous with the last received RTP data packet, an RTSP message is sent to the screen-throwing device, so that the screen-throwing device reduces the resolution according to the RTSP message; and when the current RTP data packet is continuous with the last received RTP data packet, controlling a display to play the screen throwing data.

In some embodiments, the RTP packet is marked with a packet sequence number, and the display device 200 detects whether the packet sequence number of the current RTP packet is consecutive to the packet sequence number of the last received RTP packet. When the packet sequence number is discontinuous, the display device 200 transmits an RTSP message to the screen casting device.

In some embodiments, the display device 200 further counts the number of lost RTP packets when the packet sequence number is discontinuous. When the number of the lost RTP packets reaches a preset value, the display device 200 sends an RTSP message to the screen projection device.

Since the foregoing embodiments are all described in other modes by reference to the above, the same parts are provided between different embodiments, and the same and similar parts are provided between the embodiments in the present specification. And will not be described in detail herein.

It should be noted that in this specification, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a circuit structure, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such circuit structure, article, or apparatus. Without further limitation, the statement "comprises" or "comprising" a … … "does not exclude the presence of other identical elements in a circuit structure, article or apparatus that comprises the element.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure of the invention herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

The above embodiments of the present application are not intended to limit the scope of the present application.

Claims

1. A display device, the display device comprising:

a display;

a controller in communication with the display, the controller configured to:

the RTP data packets are detected and,

2. The display device of claim 1, wherein the RTP packets are marked with a packet sequence number, and wherein, in the step of sending an RTSP message to the screen casting device when the current RTP packet is not consecutive to a last received RTP packet, the controller is further configured to:

detecting whether the packet sequence number of the current RTP data packet is continuous with the packet sequence number of the last received RTP data packet;

and when the packet sequence numbers are discontinuous, sending RTSP information to the screen throwing equipment.

3. The display device of claim 2, wherein prior to the sending of the RTSP message to the screen casting device, the controller is further configured to:

when the packet sequence numbers are discontinuous, calculating the number of lost RTP data packets;

and when the number of the lost RTP data packets is larger than a preset value, sending an RTSP message to the screen throwing equipment.

4. A display device according to claim 3, wherein in the step of counting the number of lost RTP packets when the packet sequence numbers are discontinuous, the controller is further configured to:

5. The display device of claim 1, wherein after sending an RTSP message to the screen casting device, the controller is further configured to:

recording the current playing progress of the screen throwing data;

receiving a feedback message of successful resolution modification sent by the screen-throwing equipment, and receiving screen-throwing data after resolution reduction sent by the screen-throwing equipment;

and based on the playing progress, controlling the display to continuously play the screen throwing data according to the reduced resolution.

6. The display device of claim 1, wherein after sending an RTSP message to the screen casting device, the controller is further configured to:

a feedback message of successful resolution modification sent by the screen throwing equipment is not received;

and stopping the RTSP interaction thread, and controlling the display to exit the playing process of the screen throwing data.

7. The display device of claim 3, wherein the controller is further configured to:

setting the preset value to 5.

8. The screen projection method is characterized by comprising the following steps of:

the RTP data packets are detected and,

9. The method for screen projection according to claim 8, wherein the RTP packet is marked with a packet sequence number, and wherein the sending an RTSP message to the screen projection device when the current RTP packet is discontinuous from the last received RTP packet comprises:

10. The method for projecting a screen according to claim 9, wherein when the packet sequence number is discontinuous, sending an RTSP message to the screen projecting device comprises:

and when the number of the lost RTP data packets reaches a preset value, sending an RTSP message to the screen throwing equipment.