WO2018040943A1

WO2018040943A1 - Display data control method, device, and system

Info

Publication number: WO2018040943A1
Application number: PCT/CN2017/097952
Authority: WO
Inventors: 王任
Original assignee: 阿里巴巴集团控股有限公司
Priority date: 2016-08-31
Filing date: 2017-08-18
Publication date: 2018-03-08
Also published as: CN107783702A; TW201812567A

Abstract

Disclosed in the present invention are a display data control method, device, and system. The method comprises: a control apparatus acquiring, from a server, display configuration information of a display apparatus, wherein the display configuration information comprises desktop information of the display apparatus and object information of display objects displayed on all display screens included in the display apparatus; the control apparatus drawing, according to the display configuration information, a corresponding operation interface, wherein the operation interface comprises a simulation desktop corresponding to the desktop information and simulation objects corresponding to the display objects; the operation interface of the control apparatus detecting an operating action, and generating a corresponding operation event, wherein the operation event is used to control an operation the same as the operating action to be executed on the display apparatus. The present invention solves the technical problem of the prior art in which flexibility of an interactive process with display information is poor when data content is displayed on a large display screen.

Description

Display data control method, device and system

The present application claims the priority of the Chinese Patent Application Serial No. PCT Application No.

Technical field

The present invention relates to the field of data processing, and in particular to a method, device and system for controlling display data.

Background technique

The prior art provides a large display screen which is obtained by splicing a plurality of small display screens. The large display screen is used as a display device, and each small display screen included needs to be connected to a computer PC. Show the desktop content of each computer PC.

Based on the large-screen display device shown in Figure 1, the traditional large-screen display project is based on the architecture shown in Figure 2. As shown in Figure 2, the control signals of each computer PC are spliced and assembled to obtain the entire large display. The final display content, for example, using 18 PC computers, outputs a total of 18 signals, each PC runs an H5 page (ie, page.html in Figure 1), and projects a small display corresponding to a large screen display. In the screen.

The layout of the control signals received by each small display is realized by the control software provided by the large-screen manufacturer. The control software needs to be handed over to the professional IT staff for processing. Specifically, the R&D personnel execute the video controller. Signal control and layout, that is, the staff can control the video controller to remotely control the switching and layout of the signal through the control device. The In interface represents the multi-channel HD signal of the video controller, and the Out interface represents the multi-channel signal output of the video controller. .

The control business process is as follows: First, the UED interacts with the h5 page to the service party, and the service party deploys to the respective server. Then, each PC browser accesses the server of the service party, and projects the desktop to the screen group through the video controller, and A dedicated staff controls the switching and layout of the signals through the control device.

The analysis shows that the solution provided by the above prior art mainly includes the following two disadvantages: each time the signal content on the large display screen is to be modified, it is necessary to remotely log in to the computer PC playing the signal to modify the content accessed by the chrome browser; If you modify the signal layout of the large display, such as modifying the position of the signal, scaling the signal, etc., professional IT personnel are required to control the manufacturer's control software, which is not very flexible.

Therefore, in a scenario where a large display screen is used to display business content, if the speaker performs a speech based on the displayed business content, each time the business content is explained, an extra IT is required in addition to the presenter. The staff cooperates with the switching signal to reduce the entire interpretation experience. And the control software has no public standards, each manufacturer has its own implementation, not universal and not open source, bringing some trouble to maintenance.

In response to the above problems, no effective solution has been proposed yet.

Summary of the invention

The embodiment of the invention provides a method, a device and a system for controlling display data, so as to at least solve the technical problem that the interaction process of displaying information is inflexible when displaying data content by using a large display screen in the prior art.

According to an aspect of an embodiment of the present invention, a control system for displaying data is provided, the system comprising: a display device including a plurality of display screens, each display screen for displaying page content; a server connected to the display device, And displaying display configuration information, where the display configuration information includes: desktop information of the display device and object information of a display object displayed on each display screen included in the display device; and the control device is connected to the server, and configured to: The corresponding operation interface is drawn according to the obtained display configuration information, and the corresponding operation event is generated, wherein the operation interface includes: a simulated desktop corresponding to the desktop information and a simulated object corresponding to the display object; wherein The server is also configured to receive an operation event and perform the same operation on the display device as the operation behavior according to the operation event control

According to an aspect of the embodiments of the present invention, a method for controlling display data is provided. The method includes: the control device acquires display configuration information of the display device from the server, where the display configuration information includes: desktop information of the display device and the display device The object information of the display object displayed by each display screen is included; the control device draws a corresponding operation interface according to the display configuration information, wherein the operation interface comprises: an analog desktop corresponding to the desktop information and a simulated object corresponding to the display object; The operation interface of the device detects the operation behavior and generates a corresponding operation event, wherein the operation event is used to control the same operation on the display device as the operation behavior.

According to an aspect of the embodiments of the present invention, a method for controlling display data is provided. The method includes: the server acquiring display configuration information of the display device, where the displaying the configuration information includes: desktop information of the display device and the display device includes Each of the display screens displays object information of the display object; the server generates a corresponding operation event according to the operation behavior detected by the control device on the operation interface, wherein the operation interface is drawn by the control device according to the display configuration information, and the operation interface includes: The simulated desktop corresponding to the information and the simulated object corresponding to the display object; the server performs the same operation on the display device as the operation behavior according to the operation event control

According to another aspect of the present invention, a control device for displaying data is provided, the device comprising: an obtaining module, configured to acquire display configuration information of a display device from a server, where displaying the configuration information comprises: displaying the device Desktop information and object information of a display object displayed on each display screen included in the display device; The module is configured to draw a corresponding operation interface according to the display configuration information, where the operation interface includes: an analog desktop corresponding to the desktop information and a simulation object corresponding to the display object; and a generation module, configured to detect an operation behavior if the operation interface is A corresponding operational event is generated, wherein the operational event is used to control the same operation as the operational behavior on the display device.

In the embodiment of the present invention, the display configuration information of the display device is obtained from the server by using the control device, where the display configuration information includes: desktop information of the display device and object information of the display object displayed on each display screen included in the display device; The control device draws a corresponding operation interface according to the display configuration information, where the operation interface includes: an analog desktop corresponding to the desktop information and a simulation object corresponding to the display object; the operation interface of the control device detects the operation behavior, and generates a corresponding operation event, The operation event is used to control the manner of performing the same operation as the operation behavior on the display device, and the display device obtains the display configuration information of the display device from the server, and draws a corresponding operation interface according to the display configuration information, and the user uses the operation. When the interface performs the corresponding operation, the control device can detect the above operation behavior, thereby generating a corresponding operation event, and the server responds to the operation event, and performs the same operation on the display device as the operation behavior based on the operation event. The control device can be implemented by the server, and the local operation event of the control device is synchronized to the corresponding display device, thereby realizing that the user can control the operation process of the large-screen display device flexibly by the control device, thereby reducing the large-screen display device. The interactive flexibility of the display information further solves the technical problem that the interactive process of displaying information is inflexible when displaying data content on a large display screen in the prior art.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

1 is a schematic view showing the display of a large display screen according to the prior art;

2 is a schematic diagram of a system control structure of a large display screen according to the prior art;

3 is a schematic structural diagram of a video data control system according to an embodiment of the present invention;

4 is a schematic diagram showing display of event distribution for controlling a Windows desktop in a display device according to an embodiment of the present invention;

FIG. 5 is a block diagram showing a hardware configuration of a computer terminal for implementing a control method for displaying data;

6 is a flowchart of a method of controlling display data according to a first embodiment of the present invention;

7 is a schematic diagram of displaying object information of respective display objects in a control device according to an embodiment of the present invention;

8 is a schematic diagram of a control for controlling an abstract display object in a device according to an embodiment of the present invention;

Figure 9a is a business flow diagram of an operational event in accordance with an embodiment of the present invention;

Figure 9b is another service flow diagram of an operational event in accordance with an embodiment of the present invention;

Figure 9c is a further operational flow diagram of an operational event in accordance with an embodiment of the present invention;

Figure 9d is a further operational flow diagram of an operational event in accordance with an embodiment of the present invention;

FIG. 10 is a diagram of a service interaction of a control device according to an embodiment of the present invention; FIG.

11 is a flow chart showing a control device interacting with a server according to an embodiment of the present invention;

FIG. 12 is a flowchart of a method of controlling display data according to Embodiment 1 of the present invention; FIG.

FIG. 13 is a schematic diagram of a control device for displaying data according to an embodiment of the present invention;

FIG. 14 is a structural block diagram of a control device according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is an embodiment of the invention, but not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of the present invention.

It is to be understood that the terms "first", "second" and the like in the specification and claims of the present invention are used to distinguish similar objects, and are not necessarily used to describe a particular order or order. It is to be understood that the data so used may be interchanged where appropriate, so that the embodiments of the invention described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "comprises" and "comprises" and "the" and "the" are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to Those steps or units may include other steps or units not explicitly listed or inherent to such processes, methods, products or devices.

First, some of the nouns or terms that appear in the process of describing the embodiments of the present application are applicable to the following explanations:

Signal source: refers to the video signal source, such as the DVI signal output of the graphics card, PC desktop signal.

Information piece app: An object abstracted at the software level, logically used to represent block data information, such as a web page, and an implementation of an iframe is a basic concept in this technical solution.

Example 1

FIG. 3 is a schematic structural diagram of a video data control system according to an embodiment of the present invention. As shown in FIG. 3, the control system may include: a display device 10, a server 30, and a control device 50.

The display device 10 includes a plurality of display screens 0, 1, . . . , n, each of which is used to display page content. The server 30 is connected to the display device 10 and is configured to save display configuration information associated with the page content. The display configuration information includes: desktop information of the display device and object information of the display object displayed by each display screen included in the display device; the control device 50 is connected to the server 30, and is configured to draw according to the acquired display configuration information. Corresponding operation interface, and detecting an operation behavior, generating a corresponding operation event, wherein the operation interface includes: an analog desktop corresponding to the desktop information and a simulation object corresponding to the display object; wherein the server is further configured to receive an operation event, And the same operation as the operation behavior is performed on the display device according to the operation event control.

In the foregoing embodiment provided by the present application, the control device acquires display configuration information of the display device from the server, and draws a corresponding operation interface according to the display configuration information. When the user performs the corresponding operation by using the operation interface, the control device may detect the above. The action is actuated to generate a corresponding operational event, at which point the server responds to the operational event and controls the same operation on the display device as the operational behavior based on the operational event. It can be seen that one end of the control device can display the information of the corresponding display object app in the display device of the large screen to the user, and receive the interactive instruction and control instruction input by the user, and convert the instruction into an operation event recognizable by the server system. Sending to the server server, the server server can perform corresponding logic processing according to different instructions. Since the above control device can be implemented by the server, the local operation event of the control device is synchronized to the corresponding display device, so that the user can flex through the control device. The operation of the display device for the large screen reduces the interactive flexibility of the display information of the large screen display device.

The above solution further defines a set of operation event distribution mechanism, so that various control devices can access the display device system to complete flexible control of the displayed content.

It should be noted that the desktop information may include at least: a desktop size of the display device; and the object information of the display object includes at least: a size of the display object, location information in the display device, display content, and display depth. The above operational events may include at least one of the following: a click event, a zoom event, a move event, and a scroll event.

Specifically, as shown in FIG. 3, the display device 10 is a full-screen large-screen display device, for example, an oversized rectangular desktop, on which a plurality of object information having different display contents are placed (ie, may be information pieces). App1, app2, app3, and app4), each of these apps can be placed anywhere on the large desktop, and the size of the app can be extended to the extent allowed by the desktop.

For example, in order to obtain the location information of the display object in the display device, it is necessary to first know the position of each display screen in the display device, firstly, the large screen can be abstracted into a desktop of the [M x N] small screen matrix. System, each point in the matrix represents a physical display screen screen, all screen resolutions are the same, such as 1400x1050. Each screen-specific attribute is its position in the matrix (x, y), such as the display device shown in Figure 3. In the display0 display block screen position is (0,0), and the display6 screen position is (1,2). Each display screen runs a full-screen chrome browser. The address accessed is the display.html provided by the server. Id=(x,y). The id parameter tells the server which screen it represents.

The server 30 can implement the resource file for managing the display object, and provides the resource service to the external device as the web server webserver, and can create and delete the corresponding app instance for each object information, and the location information of all the apps in the abstract desktop. It also saves the connection status of each display display screen in all large display screens, and can broadcast the data of key information (such as app and screen) to the corresponding control device in real time.

The above control device 50 abstractly displays the display device 10 of the large screen and the information of the app in the server to the user, accepts the user's interaction and control instructions, and converts the instructions into system identifiable and sends them to the server server, and the server server responds according to different instructions. Logical processing.

In an optional embodiment, the control device may send information to the server transmitter for obtaining display configuration information of the display system desktop of the display device, where the display configuration information may include desktop information configured by the desktop (for example, total length and width of the display device). And object information splicing the display object of each display screen of the display device.

It can be seen from the above that the present invention implements an example as shown in FIG. 4, and the control device designs a reasonable event distribution system, for example, simulates the event distribution mechanism of the Windows desktop in FIG. 4, and the distribution system of the operation event passes through the server. Synchronize control of the display objects on the display screen. Thereby achieving flexible interactive control.

Therefore, it can be seen that at least the following two problems are encountered in the conventional solution by the design provided by the present invention:

Instead of remotely logging in to the PC, the web page that you want to render is opened directly on the console.

No need for professional IT intervention, away from the limitations of large-screen hardware manufacturers, directly on the iPad to place the position and size of the web page on the big screen.

Example 2

In accordance with an embodiment of the present invention, there is also provided an embodiment of a method of controlling the display of data, it being noted that the steps illustrated in the flowchart of the figures may be performed in a computer system such as a set of computer executable instructions, Also, although logical sequences are shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than the ones described herein.

The method embodiment provided in Embodiment 1 of the present application can be executed in a mobile terminal, a computer terminal or the like. FIG. 5 shows a hardware block diagram of a computer terminal (or mobile device) for implementing a control method of displaying data. As shown in FIG. 5, computer terminal 10 (or mobile device 10) may include one or more (figure The processor 102 is illustrated by 102a, 102b, ..., 102n (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, And a transmission module for communication functions. In addition, it can also include: display, input/output interface (I/O interface), universal serial bus (USB) port (which can be included as one of the ports of the I/O interface), network interface, power supply And / or camera. It will be understood by those skilled in the art that the structure shown in FIG. 1 is merely illustrative and does not limit the structure of the above electronic device. For example, computer terminal 10 may also include more or fewer components than shown in FIG. 5, or have a different configuration than that shown in FIG.

It should be noted that one or more of the above-described processors 102 and/or other data processing circuits may be referred to herein generally as "data processing circuits." The data processing circuit may be embodied in whole or in part as software, hardware, firmware or any other combination. Moreover, the data processing circuitry can be a single, separate processing module, or incorporated in whole or in part into any of the other components in computer terminal 10 (or mobile device). As referred to in the embodiments of the present application, the data processing circuit is controlled as a processor (e.g., selection of a variable resistance terminal path connected to the interface).

The memory 104 can be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the control method for displaying data in the embodiment of the present invention, and the processor 102 runs the software programs and modules stored in the memory 104. Thereby performing various functional applications and data processing, that is, implementing the vulnerability detection method of the above application. Memory 104 may include high speed random access memory, and may also include non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, memory 104 may further include memory remotely located relative to processor 102, which may be coupled to computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device is for receiving or transmitting data via a network. The network specific examples described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device can include a Network Interface Controller (NIC) that can be connected to other network devices through the base station to communicate with the Internet. In one example, the transmission device can be a Radio Frequency (RF) module for communicating with the Internet wirelessly.

It should be noted here that in some embodiments, the computer device (or mobile device) shown in FIG. 5 described above has a touch display (also referred to as a “touch screen” or “touch display screen”), which can make The user is able to interact with the user interface of computer terminal 10 (or mobile device). In some embodiments, the computer device (or mobile device) shown in FIG. 1 described above has an image user interface (GUI) that can be touch sensitive by the user. Finger contact and/or gestures on the surface to interact with the GUI. The human-computer interaction function here optionally includes the following interactions: creating a web page, drawing, word processing, making electronic documents, games, video conferencing, instant messaging. Executable instructions for performing the above-described human-computer interaction functions, such as transmitting and receiving e-mail, a call interface, playing digital video, playing digital music, and/or web browsing, are configured/stored in one or more processor-executable computer programs Product or readable storage media.

In the above operating environment, the present application provides a method of controlling display data as shown in FIG. 6. FIG. 6 is a flowchart of a method of controlling display data according to a first embodiment of the present invention.

As shown in FIG. 6, the control method of the display data may include the following implementation steps:

In step S20, the control device acquires display configuration information of the display device from the server, where the display configuration information includes: desktop information of the display device and object information of a display object displayed on each display screen included in the display device.

In an optional embodiment, the control device may send information to the server transmitter for obtaining display configuration information of the display system desktop of the display device, where the display configuration information may include desktop information configured by the desktop (for example, total length and width of the display device). And object information splicing the display object of each display screen of the display device (for example, app information as in FIG. 6).

It can be seen that the server can implement the resource file for managing the display object, and provides the resource service to the external device as the web server webserver, and can also create and delete the corresponding app instance for each object information, and the location of all the apps in the abstract desktop. Information, and saves the connection of each display display screen in all large display screens, and can broadcast the data of key information (such as app and screen) to the corresponding control device in real time.

Preferably, the desktop information includes at least: a desktop size of the display device; and the object information of the display object includes at least: a size of the display object, location information in the display device, display content and display depth z-index, display depth z-index It is used to indicate that when a plurality of display object apps overlap at the same position, each display object is in a position in the layer, for example, it can be distinguished which display object is on the upper layer of other display objects.

For example, in order to obtain the location information of the display object in the display device, it is necessary to first know the position of each display screen in the display device, firstly, the large screen can be abstracted into a desktop of the [M x N] small screen matrix. System, each point in the matrix represents a physical display screen screen, all screen resolutions are the same, such as 1400x1050. The unique attribute of each screen is its position in the matrix (x, y). For example, in the display device shown in Figure 3, the position of display0 screen display screen is (0,0), and display6 is the screen. The location is (1, 2). Each screen runs a full-screen chrome browser. The address accessed is the display.html provided by the server. Id=(x,y). The id parameter tells the server which screen it represents.

In an optional embodiment, as shown in FIG. 7 , the object information of each display object is displayed in the control device. In the example, the desktop of the display device may be a rectangular desktop as shown in FIG. 7 , and the rectangular desktop is displayed on the rectangular desktop. Multiple object information with different display contents (ie, information pieces app1, app2, app3, and app4) can be placed. Each of the above apps can be placed anywhere on the large desktop, and the size of the app can also be within the range allowed by the desktop. Expansion.

Step S22: The control device draws a corresponding operation interface according to the display configuration information, where the operation interface includes: an analog desktop corresponding to the desktop information and a simulated object corresponding to the display object.

Specifically, the control device may draw a display diagram of the simulated desktop and the simulated object app according to the display configuration information, and the user may obtain the layout of the content of each display screen in the current large display device according to the image currently displayed in the control device. .

Step S24, the operation interface of the control device detects the operation behavior, and generates a corresponding operation event, wherein the operation event is used to control the same operation on the display device as the operation behavior.

The user interaction event can be collected, for example, in the case that the control device is a mobile terminal, the user gesture operation can be collected by using the hammer.js library. Preferably, the operation event includes at least one of the following: a click event, a zoom event, a move event, and Scroll events.

Taking the operation event as a zoom event and a move event as an example, the user can control the display object in the corresponding display screen on the display device to perform movement and zoom by moving and zooming the app on the control device. The display device is a large-screen display screen, and the desktop of the display device can be abstracted into a desktop container, and the object information displayed by the display device can be abstracted into an information piece app in the container, thereby displaying the result of the large-screen display device. Abstract is a combination of multiple display object apps.

It can be seen that, in the foregoing embodiment provided by the present application, the control device acquires display configuration information of the display device from the server, and draws a corresponding operation interface according to the display configuration information, and controls the device when the user performs corresponding operations by using the operation interface. The above operational behavior can be detected to generate a corresponding operational event, at which time the server responds to the operational event and controls the same operation on the display device as the operational behavior based on the operational event. It can be seen that one end of the control device can display the information of the corresponding display object app in the display device of the large screen to the user, and receive the interactive instruction and control instruction input by the user, and convert the instruction into an operation event recognizable by the server system. Sending to the server server, the server server can perform corresponding logic processing according to different instructions. Since the above control device can be implemented by the server, the local operation event of the control device is synchronized to the corresponding display device, so that the user can flex through the control device. The operation of the display device for the large screen reduces the interactive flexibility of the display information of the large screen display device.

The above solution further defines a distribution mechanism for operating events, so that various control devices can be connected. Enter the system of the display device to complete flexible control of the displayed content.

In an optional embodiment, after the operation interface of the control device is detected in step S24, the operation behavior is detected, and after the corresponding operation event is generated, the following implementation steps may also be performed:

In step S241, an operation event is sent to the server.

Step S243, the server controls the application logic corresponding to the display object execution operation event displayed on the display device.

It can be seen that, in the implementation of the foregoing steps S241 and S243, the control device can convert the interaction behavior of the user into an operation pointer event identifiable by the server server and send it to the server server. After the server server runs the corresponding application logic, The app information can be updated and broadcast to the control device so that one end of the control device can update the mock UI in the simulated user interface. The above application logic can be run on the server side of the server, and the trigger of the operation event (such as the mouse seems to be) is implemented by the control device. The UI obtained by the control device is shown in Figure 8.

Preferably, in order to implement flexible interactive control on one end of the control device, a reasonable event distribution system needs to be designed. For example, when the operating system of the display device is windows, the control device can implement an event distribution mechanism for simulating the windows desktop. The defined event distribution system is described as follows: the class of each display object app is corresponding to an application window of the windows system. The class of the server server is corresponding to the windows system, and the class of the control command from the operation event of one end of the control device is corresponding to the mouse operation.

Based on the above design, the following event message is abstracted:

pointerPress (point, button) simulates the mouse point (button distinguishes the left and right buttons)

pointerRelease(point,button) simulates mouse release (button distinguishes left and right buttons)

pointerMove(point) simulates mouse movement

pointerScroll(point, wheelDelta) simulates the middle mouse button scrolling to zoom in or out of the app.

Therefore, the application logic corresponding to the foregoing execution operation event may include at least the following application modes:

Method 1: In the case that the operation event is a click event, the server selects the display object with the highest display depth to execute the click event.

Specifically, as shown in FIG. 9a, after detecting the click event, the selected display object with the highest display depth is found from all display objects, and a click operation is performed on the object, thereby implementing a display screen of the slave control device. Select the object to be operated and save the object's click command or cache selectdApp.prevPt=pointer.

Method 2: In the case that the operation event is a mobile event, the server performs a movement event on the selected display object.

Specifically, as shown in FIG. 9b, after detecting the motion event, searching for whether there is a selected display object, If there is, the selected display object is moved to the target position, and the movement instruction of the object is saved, that is, the selected pointer of the display object that needs to be moved is saved selectdApp.prevPt=pointer.

Mode 3: In the case that the operation event is a mouse lift event, a zoom event, or a browse event, the server performs a zoom event on the selected display object.

Specifically, as shown in FIG. 9c, if a mouse lift event is detected, the selected display object is discarded. In addition, after detecting the zoom event or the browsing event, it is searched whether there is a selected display object, and if so, the selected display object is zoomed or browsed to display the display object in the target area, and the instruction of the object is saved.

Mode 4: In the case that the operation event is a scroll event, the server performs a scroll event on the browsed display object.

Specifically, as shown in FIG. 9d, after the scroll is detected, it is searched whether there is a selected display object, and if so, the selected display object is scrolled and the scroll instruction of the object is saved.

In an optional embodiment, after performing the step S243 to control the application logic corresponding to the display object execution operation event displayed on the display device, the following steps may be implemented:

Step S245, the server updates the object information of the corresponding display object according to the execution result of the operation event, and obtains new display configuration information.

In step S246, the server returns the new display configuration information to the control device, so that the control device displays a new operation interface.

In an optional embodiment, before the execution of the display configuration information of the display device is obtained from the server in step S20, the following steps may be implemented:

In step S10, the server receives an object open instruction sent by the control device, where the object open command includes at least: an access address of the object to be displayed;

Step S12: The server creates object information of the object to be displayed according to the object opening instruction, where the object information of the object to be displayed includes at least: a ruler length, a display depth, and position information of the object to be displayed;

In step S14, the server controls the display device to display the object to be displayed.

FIG. 10 is a diagram of a service interaction of a control device according to an embodiment of the present invention. In conjunction with the embodiment shown in FIG. 10, the process flow is as follows for how to open a new information tablet app:

Step 90: The control device receives the access address of the object to be displayed input by the user, where the access address may be a webpage url;

Step 92: The control device sends an object open command to the server, and then opens the APP on the display device. The instruction carries the access address of the object to be displayed.

Step 94: The server creates new object information according to the instruction, including the position and size of the display screen corresponding to the object information on the large-screen display device.

In step S96, the server sends the created new object information to the display device, so that the display device acquires the DOM corresponding to the object information APP, so that the object is displayed at the corresponding position of the display device.

Preferably, the step S14 described above by the server controlling the display device to display the object to be displayed may include the following optional embodiments:

Step S141, the display device downloads object information related to the object to be displayed from the server.

Step S143, the display device generates DOM data corresponding to the object to be displayed.

Step S145, the display device loads and displays the DOM data according to the screen information of any one or more display screens.

It can be seen from the above analysis that in the implementation provided in the above steps S141 to S145, the implementation of the screen information in the display screen in the display device of the large screen needs to include the following key points: First, each display screen screen will be all The running class related definition file (js file) is downloaded, and the app instance is generated in the current running environment; each app instance actually corresponds to a dom node, so the app instance is actually hanged on the corresponding dom after being generated. In order to achieve the final visual presentation. The app definition file is essentially a definition of a reactjs component, and the reactjs component can be easily mounted to a dom. Each screen has its own identifier. For example, a screen with an id of (i, j) will be inside the container. All apps do a certain offset, the implementation is to translate (offsetX, offsetY) to the container dom of the app, where offsetX=(1400*i+app.x), offsetY=(1050*j+app.y).

11 is a flow diagram of a control device interacting with a server in accordance with an embodiment of the present invention. The core scheme of the data flow of the control device end and the server server of the present invention is described in detail below based on the embodiment shown in FIG. 11 as follows:

Step S101, the control device requests the server server to know the desktop configuration of the system (the total length and width of the large screen), and the information of all the apps, including the location, length and width, and z-index of the app, and who is represented when the app positions overlap. On the upper level.

Step S103: Draw a mock desktop and an app diagram according to the above information, prompt the user for the current layout situation, and collect the user interaction event, and the mobile terminal collects the user gesture operation by using the hammer.js library.

In step S105, the user controls the system by moving and zooming the app.

In step S107, the control end converts the interaction behavior of the user into a pointer event identifiable by the server end and sends it to the server.

Step S109: After the server server runs the corresponding logic, the object app information is updated and broadcast to one end of the control device.

In step S111, the control device updates the virtual control object mock UI at one end.

At the same time, not only the above two control problems are solved, but by writing simple scripts, we can also get many functions that are not possible with traditional solutions, such as controlling the app to slide in from the right side of the big screen, draw on the left side, and slide in. The time of slipping out and the speed of change can be precisely controlled. These have been well applied in the third phase of the intelligence project.

It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence. Because certain steps may be performed in other sequences or concurrently in accordance with the present invention. In addition, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases, the former is A better implementation. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, The optical disc includes a number of instructions for causing a terminal device (which may be a cell phone, a computer, a server, or a network device, etc.) to perform the methods described in various embodiments of the present invention.

Example 3

Also based on the operating environment shown in FIG. 4, the present application provides a method of controlling display data as shown in FIG. Figure 12 is a flow chart showing a method of controlling display data according to a first embodiment of the present invention.

As shown in FIG. 12, the method for controlling display data may include the following implementation steps:

In step S121, the server acquires display configuration information of the display device, where the display configuration information includes: desktop information of the display device and object information of a display object displayed on each display screen included in the display device;

Step 123: The server generates a corresponding operation event according to the operation behavior detected by the control device on the operation interface, where the operation interface is drawn by the control device according to the display configuration information, and the operation interface includes: The simulated desktop and the simulated object corresponding to the display object;

In step 125, the server performs the same operation on the display device as the operation behavior according to the operation event.

Embodiment 3 of the present application protects the core of the present invention from the server side, and the embodiment may also provide the same or corresponding optional or preferred embodiments provided by the foregoing Embodiment 2, and the implemented examples and application scenarios are the same. However, it is not limited to the contents disclosed in the above second embodiment.

Example 4

According to an embodiment of the present invention, there is further provided a control device for displaying data for implementing the above-described control method of display data, and FIG. 13 is a schematic diagram of a control device for displaying data according to an embodiment of the present invention. As shown in FIG. 13, the apparatus may include an acquisition module 130, a rendering module 133, and a generation module 135.

The obtaining module 130 is configured to acquire display configuration information of the display device from the server, where the display configuration information includes: desktop information of the display device and object information of a display object displayed on each display screen included in the display device;

The drawing module 133 is configured to draw a corresponding operation interface according to the display configuration information, where the operation interface includes: an analog desktop corresponding to the desktop information and a simulated object corresponding to the display object;

The generating module 135 is configured to generate a corresponding operation event if the operation interface detects the operation behavior, wherein the operation event is used to control the same operation performed on the display device as the operation behavior.

It should be noted that the foregoing obtaining module 130, drawing module 133, and generating module 135 correspond to steps S20 to S24 in Embodiment 2, and the three modules are the same as the examples and application scenarios implemented by the corresponding steps. However, it is not limited to the contents disclosed in the first embodiment. It should be noted that the above module can be operated as part of the device in the computer terminal 10 provided in the second embodiment.

The embodiment 4 of the present invention can also provide an optional or preferred embodiment corresponding to the foregoing embodiment 1. The implemented example and the application scenario are the same, but are not limited to the content disclosed in the first embodiment.

Example 5

Embodiments of the present invention may provide a control device, which may be any one of a group of computer terminals. Optionally, in this embodiment, the foregoing control device may also be replaced with a terminal device such as a mobile terminal.

Optionally, in this embodiment, the foregoing control device may be located in at least one of the plurality of network devices of the computer network.

In this embodiment, the computer terminal may execute the program code of the following steps in the vulnerability detection method of the application: the control device acquires display configuration information of the display device from the server, where the display configuration information includes: desktop information of the display device and display And the control device displays the corresponding operation interface according to the display configuration information, where the operation interface includes: a simulated desktop corresponding to the desktop information and a simulated object corresponding to the display object; The operation interface of the control device detects the operation behavior and generates a corresponding operation event, wherein the operation event is used to control the same operation performed on the display device as the operation behavior.

Optionally, FIG. 14 is a structural block diagram of a control device according to an embodiment of the present invention. As shown in FIG. 14, the control device 14 may include one or more (only one shown) processor, memory, and other necessary devices.

The memory can be used to store software programs and modules, such as the security vulnerability detection method and the program instruction/module corresponding to the device in the embodiment of the present invention. The processor executes various functions by running a software program and a module stored in the memory. Application and data processing, that is, the detection method for implementing the above system vulnerability attack. The memory may include a high speed random access memory, and may also include non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, the memory can further include memory remotely located relative to the processor, which can be connected to the terminal 14 over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The processor may call the information and the application stored in the memory by the transmission device to perform the following steps: the control device acquires display configuration information of the display device from the server, where the display configuration information includes: desktop information of the display device and the display device includes Each of the display screens displays object information of the display object; the control device draws a corresponding operation interface according to the display configuration information, wherein the operation interface includes: a simulation table corresponding to the desktop information And an analog object corresponding to the display object; the operation interface of the control device detects the operation behavior, and generates a corresponding operation event, wherein the operation event is used to control the same operation performed on the display device as the operation behavior.

Optionally, the foregoing processor may further execute the following program code: send an operation event to the server; the server controls application logic corresponding to the display object displayed on the display device, where the operation event is a click event. In case, the server selects the display object with the highest depth to execute the click event; in the case that the operation event is the mobile event, the server performs a movement event on the selected display object; in the case that the operation event is a scaling event, the server displays the selected The object performs a zoom event; in the case where the action event is a scroll event, the server performs a scroll event on the browsed display object.

Optionally, the processor may further execute the following program code: the server updates the object information of the corresponding display object according to the execution result of the operation event, to obtain new display configuration information; and the server returns the new display configuration information to the control device. So that the control device displays a new operating interface.

Optionally, the processor may further execute the following steps: the server receives an object open instruction sent by the control device, where the object open instruction includes at least: an access address of the object to be displayed; and the server creates a to-be-displayed according to the object open instruction. Object information of the object, wherein the object information of the object to be displayed includes at least: a ruler length, a display depth, and position information of the object to be displayed; and the server controls the display device to display the object to be displayed.

Optionally, the processor may further execute the following program code: the display device downloads object information related to the object to be displayed from the server; the display device generates DOM data corresponding to the object to be displayed; and the display device follows any one or more The screen information of the display screen loads and displays the DOM data.

With the embodiment of the present invention, a control scheme for displaying data is provided. The control device obtains the display configuration information of the display device from the server, and draws a corresponding operation interface according to the display configuration information. When the user performs the corresponding operation by using the operation interface, the control device may detect the operation behavior, thereby generating a corresponding operation. The event, at this time, the server responds to the operation event, and performs the same operation on the display device as the operation behavior based on the operation event, thereby achieving the operation process of the display device of the large screen that the user can flexibly control by the device, and the operation is achieved. The purpose of reducing the interactive flexibility of the display information of the large-screen display device is to solve the technical problem that the interactive process of displaying information is inflexible when displaying the data content on the large display screen in the prior art.

A person skilled in the art can understand that the structure shown in FIG. 14 is only a schematic, and the computer terminal can also be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, an applause computer, and a mobile Internet device (Mobile Internet Devices, MID). ), PAD and other terminal devices. FIG. 10 does not limit the structure of the above electronic device. For example, computer terminal 10 may also include more or fewer components (such as a network interface, display device, etc.) than shown in FIG. 10, or have a different configuration than that shown in FIG.

A person of ordinary skill in the art may understand that all or part of the steps of the foregoing embodiments may be completed by a program to instruct terminal device related hardware, and the program may be stored in a computer readable storage medium, and the storage medium may be Including: flash disk, read-only memory (ROM), random access memory (RAM), disk or optical disk.

Example 5

Embodiments of the present invention also provide a storage medium. Optionally, in the embodiment, the foregoing storage medium may be used to save the program code executed by the control method of the display data provided in the first embodiment.

Optionally, in this embodiment, the foregoing storage medium may be located in any one of the computer terminal groups in the computer network, or in any one of the mobile terminal groups.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: the control device acquires display configuration information of the display device from the server, where displaying the configuration information comprises: displaying desktop information of the device and Displaying object information of the display object displayed on each display screen included in the display device; the control device draws a corresponding operation interface according to the display configuration information, wherein the operation interface includes: an analog desktop corresponding to the desktop information and a simulated object corresponding to the display object The operation interface of the control device detects the operation behavior and generates a corresponding operation event, wherein the operation event is used to control the same operation on the display device as the operation behavior.

Optionally, in the present embodiment, the storage medium is arranged to store program code for performing the following steps:

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: the server updates the object information of the corresponding display object according to the execution result of the operation event to obtain new display configuration information; the server will The new display configuration information is returned to the control device, causing the control device to display a new operational interface.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: the server receives an object open instruction sent by the control device, where the object open instruction includes at least: an access address of the object to be displayed The server creates object information of the object to be displayed according to the object opening instruction, wherein the object information of the object to be displayed includes at least: a ruler length, a display depth, and position information of the object to be displayed; and the server controls the display device to display the object to be displayed.

Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: the display device downloads object information related to the object to be displayed from the server; and the display device generates DOM data corresponding to the object to be displayed The display device loads and displays the DOM data according to the screen information of any one or more of the display screens.

The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

In the above-mentioned embodiments of the present invention, the descriptions of the various embodiments are different, and the parts that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments.

In the several embodiments provided by the present application, it should be understood that the disclosed technical contents may be implemented in other manners. The device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, unit or module, and may be electrical or otherwise.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and the like. .

The above description is only a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It should be considered as the scope of protection of the present invention.

Claims

A control system for displaying data, comprising:

a display device comprising a plurality of display screens, each display screen for displaying page content;

a server, connected to the display device, for saving display configuration information associated with the page content, wherein the display configuration information includes: desktop information of the display device and each display included in the display device The object information of the display object displayed on the screen;

The control device is connected to the server, and is configured to draw a corresponding operation interface according to the obtained display configuration information, and generate a corresponding operation event when the operation behavior is detected, where the operation interface includes: a simulated desktop corresponding to the desktop information and a simulated object corresponding to the display object;

The server is further configured to receive the operation event, and perform the same operation on the display device as the operation behavior according to the operation event.
The control system according to claim 1, wherein the desktop information comprises at least: a desktop size of the display device; and the object information of the display object comprises at least: a size of the display object, located in the display Location information, display content, and display depth in the device.
The control system of claim 2 wherein said operational event comprises at least one of: a click event, a zoom event, a move event, and a scroll event.
A method for controlling display data, comprising:

The control device acquires display configuration information of the display device from the server, where the display configuration information includes: desktop information of the display device and object information of a display object displayed on each display screen included in the display device;

The control device draws a corresponding operation interface according to the display configuration information, where the operation interface includes: a simulated desktop corresponding to the desktop information and a simulated object corresponding to the display object;

The operation interface of the control device detects an operation behavior and generates a corresponding operation event, wherein the operation event is used to control performing the same operation on the display device as the operation behavior.
The method according to claim 4, wherein the desktop information comprises at least: a desktop size of the display device; and the object information of the display object includes at least: a size of the display object, located in the display device Location information, display content, and display depth.
The method of claim 5 wherein the operational event comprises at least one of: a click event, a zoom event, a move event, and a scroll event.
The method according to claim 6, wherein the operation interface of the control device detects After the operation behavior is generated, the method further includes:

Transmitting the operational event to the server;

The server controls application logic corresponding to the display object displayed on the display device to perform the operation event, where

In a case that the operation event is the click event, the server selects a display object with the highest display depth to execute the click event;

In the case that the operation event is the movement event, the server performs the movement event on the selected display object;

In the case that the operation event is the scaling event, the server performs the scaling event on the selected display object;

In the case that the operation event is the scroll event, the server performs the scroll event on the browsed display object.
The method according to claim 7, wherein after the server controls the display object displayed on the display device to execute the application logic corresponding to the operation event, the method further includes:

Updating, by the server, the object information of the corresponding display object according to the execution result of the operation event, to obtain new display configuration information;

The server returns the new display configuration information to the control device such that the control device displays a new operational interface.
The method of claim 4, wherein before the controlling device obtains the display configuration information of the display device from the server, the method further includes:

Receiving, by the server, an object open instruction sent by the control device, where the object open instruction includes at least: an access address of an object to be displayed;

The server creates object information of the object to be displayed according to the object opening instruction, where the object information of the object to be displayed includes at least: a ruler length, a display depth, and position information of the object to be displayed;

The server controls the display device to display the object to be displayed.
The method according to claim 9, wherein the controlling the display device to display the object to be displayed comprises:

The display device downloads object information related to the object to be displayed from the server;

The display device generates DOM data corresponding to the object to be displayed;

The display device loads and displays the number of DOMs according to screen information of any one or more display screens according to.
A method for controlling display data, comprising:

The server acquires display configuration information of the display device, where the display configuration information includes: desktop information of the display device and object information of a display object displayed on each display screen included in the display device;

The server generates a corresponding operation event according to the operation behavior detected by the control device on the operation interface, where the operation interface is drawn by the control device according to the display configuration information, where the operation interface includes: An analog desktop corresponding to the desktop information and a simulated object corresponding to the display object;

The server controls to perform the same operation on the display device as the operation behavior according to the operation event.
A control device for displaying data, comprising:

An obtaining module, configured to acquire display configuration information of the display device from the server, where the display configuration information includes: desktop information of the display device and object information of a display object displayed by each display screen included in the display device ;

a drawing module, configured to draw a corresponding operation interface according to the display configuration information, where the operation interface includes: a simulated desktop corresponding to the desktop information and a simulated object corresponding to the display object;

And a generating module, configured to generate a corresponding operation event if the operation interface detects the operation behavior, wherein the operation event is used to control performing the same operation on the display device as the operation behavior.