[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2018184443A1 - 终端屏幕显示处理方法、装置及存储介质 - Google Patents

终端屏幕显示处理方法、装置及存储介质 Download PDF

Info

Publication number
WO2018184443A1
WO2018184443A1 PCT/CN2018/078425 CN2018078425W WO2018184443A1 WO 2018184443 A1 WO2018184443 A1 WO 2018184443A1 CN 2018078425 W CN2018078425 W CN 2018078425W WO 2018184443 A1 WO2018184443 A1 WO 2018184443A1
Authority
WO
WIPO (PCT)
Prior art keywords
terminal
picture
screen
terminals
interface
Prior art date
Application number
PCT/CN2018/078425
Other languages
English (en)
French (fr)
Inventor
罗刚
谢守军
Original Assignee
博彦科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 博彦科技股份有限公司 filed Critical 博彦科技股份有限公司
Publication of WO2018184443A1 publication Critical patent/WO2018184443A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1407—General aspects irrespective of display type, e.g. determination of decimal point position, display with fixed or driving decimal point, suppression of non-significant zeros
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units

Definitions

  • the present invention relates to the field of network technologies, and in particular, to a terminal screen display processing method, apparatus, and storage medium.
  • the functions and interfaces of terminal devices have been continuously optimized, and corresponding, user-friendly interfaces and intuitive operating experiences have become important issues in the field of terminal devices.
  • users have also pursued new conveniences in controlling terminal devices.
  • the display of the screen of the controlled terminal is real-time, and the screen information of the terminal to be controlled is transmitted to the master device.
  • the bandwidth of the connection established between the master device and the controlled terminal is always occupied, which seriously causes waste of bandwidth, and further causes a jam when transmitting other data, so that The user experience is also poor.
  • the embodiment of the invention provides a method, a device and a storage medium for processing a terminal screen display, so as to at least solve the technical problem of wasting bandwidth when the related art displays the screen of the terminal.
  • a method for processing a terminal screen display includes: capturing a picture of each of the plurality of terminals, wherein the picture of each terminal is used to display the terminal The content displayed on the screen, the picture of each terminal includes a plurality of pictures, the multiple pictures are pictures when the terminal screen changes; the pictures are sent; and the plurality of pictures are used to display the plurality of pictures in one interface The content displayed on each terminal screen in the terminal.
  • the capturing the picture of each terminal includes: acquiring the number of terminals displayed in the one interface; setting a resolution according to the number of the terminals; and capturing each terminal according to the resolution
  • the picture, or, after the picture of each terminal is captured, is converted according to the resolution.
  • the capturing the picture of each terminal includes: acquiring a picture of each terminal according to a predetermined time interval; determining whether a change of the acquired two adjacent pictures exceeds a threshold, and exceeding a threshold In the case, a picture captured later in the two pictures is used as a picture displayed in the interface.
  • determining whether the acquired change of the two pictures exceeds the threshold includes: acquiring pixel points of each of the two pictures; comparing whether each pixel in the two pictures is A change occurs, the pixel point at which the change occurs is recorded; and it is determined whether the number of changed pixel points exceeds the threshold.
  • sending the picture includes: acquiring between the device that grabs the picture and the device that uses the picture for display a network condition; processing the captured picture according to the network condition; and transmitting the processed picture.
  • the network status includes at least one of the following: a packet loss rate, a bandwidth.
  • a storage medium including a stored program, wherein, when the program is running, controlling a device in which the storage medium is located to execute the terminal described above
  • the screen displays the processing method.
  • a processor is provided, the processor being configured to execute a program, wherein the program is executed to execute the terminal screen display processing method described above.
  • a terminal screen display processing apparatus including: a capture module, configured to capture a picture of each of the plurality of terminals, wherein each of the terminals The picture is used to display the content displayed on the screen of the terminal, the picture of each terminal includes a plurality of pictures, the multiple pictures are pictures when the terminal screen changes, and the sending module is configured to send the picture;
  • the first display module is configured to display content displayed by each of the plurality of terminals in an interface using the picture.
  • the capturing module includes: a first acquiring unit configured to acquire a quantity of terminals displayed in the one interface; a setting unit configured to set a resolution according to the number of the terminals; a grabbing unit, And being configured to capture the picture of each terminal according to the resolution, or to perform the conversion according to the resolution after the picture of each terminal is captured.
  • a picture of each terminal of the multiple terminals is captured, where the picture of each terminal is set to display the content displayed on the screen of the terminal, and the picture of each terminal includes multiple pictures and multiple pictures.
  • the picture of each terminal screen changes; send a picture; use the picture to display the content displayed on each of the plurality of terminals in one interface.
  • FIG. 1 is a flowchart of a terminal screen display processing method according to an embodiment of the present invention.
  • FIG. 2 is a flowchart of a terminal control method according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a preferred acquisition of a first terminal according to an embodiment of the present invention.
  • FIG. 4 is a schematic illustration of a preferred pre-set area in accordance with an embodiment of the present invention.
  • Figure 5 is a schematic illustration of a preferred selection control in accordance with an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of a terminal screen display processing apparatus according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of a terminal control apparatus according to an embodiment of the present invention.
  • FIG. 8 is a block diagram showing the hardware structure of a computer terminal according to an embodiment of the present invention.
  • FIG. 9 is a second block diagram of a hardware structure of a computer terminal according to an embodiment of the invention.
  • a method embodiment of a terminal screen display process wherein the steps shown in the flowchart of the drawing may be performed in a computer system such as a set of computer executable instructions. Also, although logical sequences are shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than the ones described herein.
  • the operating systems used on mobile terminals are Microsoft's WP operating system, Android system, and Apple's iOS system. In addition to these operating systems, there may be other types of operating systems.
  • a variety of operating systems can also be used for computing (eg, a personal computer PC), such as a Windows operating system, an Apple operating system, a Linux operating system, or a Unix operating system.
  • control modes for the terminals. These control methods have different processing methods for the controlled terminals. For example, in a control mode, when the screen of the controlled terminal is displayed on the main control device, it is necessary to The data for displaying the screen of the controlled terminal is transmitted to the master device in real time. For this control method, the bandwidth for controlling the connection of the data transmitted by both parties is kept in an occupied state, which not only wastes bandwidth but also affects other data. Transmission.
  • the present embodiment describes a method for controlling the terminal from the perspective of the display of the terminal screen.
  • the technical solution involved in this embodiment can be made into an application in the application market, and the user downloads the application and installs it when using it. Or, if you want to promote an operating system, you can also build this function into the operating system. As a selling point of the operating system, increase the installed capacity of the operating system.
  • FIG. 1 is a flowchart of a terminal screen display processing method according to an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps:
  • Step S102 Capture a picture of each terminal of the multiple terminals, where the picture of each terminal is used to display the content displayed on the screen of the terminal, and the picture of each terminal includes multiple pictures, and multiple pictures are the terminal screen. The picture when the change occurred;
  • Step S104 sending a picture
  • Step S106 using the picture to display the content displayed on each of the plurality of terminals in one interface.
  • the user may first select a plurality of terminals that need to be controlled. Generally, the operation of the current terminal is accessible to the network. In the above steps, the controlled terminal can be connected through the network.
  • the above steps describe the operation of displaying the screen of the plurality of terminals by the terminal screen display processing method, and the control application performs operations such as capturing and displaying the screen display contents of the plurality of terminals.
  • a server can be used to connect terminals of different operating systems.
  • an Android phone and an iOS phone can be connected through a Mac Mini.
  • the processing of displaying content on multiple terminal screens is supported in general operating systems, for example, even as the most restrictive iOS system is supported, for other relatively open systems, such as Android or Linux systems. It is also supported by the operating system. Therefore, the technical solutions involved in the above steps can support a wider range of operating systems.
  • the multiple terminals may be smart terminal devices such as a computer, a notebook, a tablet computer, a mobile phone, a smart wearable device, or a smart home device such as an air conditioner, a refrigerator, a washing machine, or a television.
  • the smart device such as the smart terminal device and the smart home device, refers to a device that has a computing processing capability and can perform connection control.
  • the foregoing smart device is only an example of the embodiment, which is not limited in this embodiment.
  • multiple pictures from the plurality of terminals for displaying the screen display content of the terminal are detected and captured in real time, and the picture is sent and used.
  • the picture displays the content displayed on each of the terminal screens in an interface, so that the content displayed on the screen of the terminal is captured when the screen of the terminal changes, and the captured picture is displayed.
  • the operations of capturing a picture and displaying a picture may occur on the same device or on different devices.
  • the captured picture may be, but is not limited to, a key frame of terminal screen data, and a screen of multiple terminals is displayed on one interface, and the screens of the displayed terminals are key frames instead of Streaming media, the key frames of multiple terminals are captured as pictures, and the screen of the terminal is displayed through the transmission of pictures.
  • the picture when the screen of each of the plurality of terminals changes is captured, and the content displayed on each of the plurality of terminals is displayed and displayed in an interface. Therefore, when the screen of the terminal is displayed, the picture for indicating the change of the screen is transmitted only when the screen of the terminal being displayed changes, so that the connection for controlling the plurality of terminals does not need to transmit the screen data in real time, thereby reducing the screen display process.
  • the content displayed on multiple terminal screens is displayed on one interface, and the purpose of the picture when the terminal screen changes is captured, thereby realizing the technology for saving the bandwidth of the display terminal screen. effect. Therefore, the above embodiment of the present application solves the technical problem of wasting bandwidth when displaying the screen of the terminal in the related art.
  • the image of each terminal can be captured according to the number of terminals displayed on the interface.
  • an optional implementation manner is provided, and the optional implementation manner may include The following steps:
  • Step 202 Acquire a number of terminals displayed in one interface
  • Step 204 setting a resolution according to the number of terminals
  • Step 206 Grab the picture of each terminal according to the resolution, or capture the picture of each terminal and convert according to the resolution.
  • the resolution may be set for the captured picture according to the number of terminals displayed on the interface, and may be in one of two ways. Obtain the picture of the resolution: mode one, capture the picture of the terminal according to the above resolution; mode two, grab the picture of each terminal, and then convert the picture according to the above resolution.
  • the resolution of the displayed picture may be set according to the bandwidth status of the control side.
  • the lower resolution may be set to display the picture.
  • the resolution when the image is captured on the terminal may be set according to the resolution of the terminal itself.
  • the control side may display with a lower resolution to adapt to the condition of the network. This makes the terminal screen display more efficient.
  • the image after the image is captured, the image may be processed into a corresponding resolution according to requirements.
  • the size of the image after processing may be determined according to the number of terminals displayed on one page; The size to determine the size of the image after processing.
  • the resolution of the terminal For example, if you want to display 9 terminals on an interface with a resolution of 1920 ⁇ 1080 pixels, you can set the resolution of the terminal to 640 ⁇ 360 pixels, and then grab the picture of each terminal with a resolution of 640 ⁇ 360 pixels. Or, after capturing the picture of each terminal, the resolution of the captured picture is converted to 640 ⁇ 360 pixels.
  • the image resolution when the image is captured may be, but is not limited to, a fixed value.
  • the resolution of the captured image may be fixed (for example, 1920 ⁇ 1080), and when the captured image is displayed, the resolution of the displayed image may be based on the current network condition.
  • Make settings for example, the resolution of the displayed picture can be set according to the current data transmission rate. For example: obtaining the data transmission rate of the current network; setting the resolution for the displayed picture according to the data transmission rate; displaying the picture according to the set resolution.
  • the image displayed on the interface may be determined according to the change of the two images captured before and after.
  • the method may further include the following steps:
  • Step 302 Obtain a picture of each terminal according to a predetermined time interval.
  • Step 304 Determine whether the acquired change of two adjacent pictures exceeds a threshold, and if the threshold value is exceeded, use one picture captured in the two pictures as the picture displayed in the interface.
  • the picture of each terminal is obtained every 2 seconds, and it is determined whether the change of the two pictures acquired before and after each terminal exceeds a preset threshold. For the terminal whose change exceeds the threshold, the picture that is captured later is taken as The image displayed in the interface can keep the image displayed on the current interface unchanged for images that do not exceed the threshold.
  • the terminal screen display processing method when the terminal screen display processing method is applied to the test scenario of the application, comparing the adjacent images captured before and after is helpful to discover problems in the running of the application, and may be based on the application.
  • the stage of operation on the device: installation, startup, verification, operation, uninstallation, etc. intercept the screen image to test the various stages of the application.
  • the picture of each terminal is captured according to a predetermined time interval, and the picture displayed on the interface is determined according to the change of the two pictures captured before and after, thereby saving the bandwidth occupied by the screen display of the terminal.
  • determining whether the acquired changes of the two pictures exceed a threshold may include the following steps:
  • Step 402 Obtain pixel points of each picture in the two pictures
  • Step 404 comparing whether each pixel in the two pictures changes, and recording the changed pixel point
  • Step 406 Determine whether the number of pixel points that have changed exceeds a threshold.
  • the change of the picture may be determined according to the change of the pixel points in the two pictures, thereby determining the picture displayed on the interface. For example, obtaining the pixels of each picture in the two pictures, and comparing whether each pixel in the two pictures changes, recording the changed pixel points, and obtaining the number of changed pixels is 15000, and The preset threshold is 10000 pixels, and the number of currently changed pixels has exceeded the threshold, so that the change of the obtained two pictures has exceeded the threshold, and the two pictures can be grasped later. An image taken is used as the image displayed in the interface.
  • the image may also be captured by acquiring a picture of each terminal according to a preset time, and using the currently captured picture as the picture displayed in the interface.
  • the preset time may be, but is not limited to, selected by a user or set by a system.
  • the time interval for capturing the picture and the number of the captured pictures may be preset, and when the picture is captured, the preset capture is performed.
  • the time interval of the image is captured, and the image is stopped when the number of captured images reaches the preset number of captured images.
  • the time interval for capturing the image is set to 5s and the number of pre-set images is 20, and when the image is captured, one image is captured every 5 seconds. Stop crawling when 20 images are taken.
  • the picture before the picture is captured, it may be determined whether the number of pictures that have been captured has reached a preset number. If the preset number is not reached, the picture of each terminal is obtained. If the preset number has been reached, stop capturing the image.
  • the preset number may be, but is not limited to, selected by a user or set by a system.
  • the network condition between the device that captures the picture and the device that uses the picture is displayed, and according to The network condition processes the captured image and then sends the processed image.
  • the foregoing network status may include, but is not limited to, at least one of the following: a packet loss rate and a bandwidth.
  • an embodiment of the present invention further provides a storage medium, the storage medium including a stored program, wherein, when the program is running, controlling a device where the storage medium is executed The terminal screen display processing method described above.
  • an embodiment of the present invention further provides a processor, where the processor is configured to run a program, where the program is executed to execute the terminal screen display processing method described above. .
  • the operating systems used on mobile terminals are Microsoft's WP operating system, Android system, and Apple's iOS system. In addition to these operating systems, there may be other types of operating systems.
  • a variety of operating systems can also be used for computing (eg, a personal computer PC), such as a Windows operating system, an Apple operating system, a Linux operating system, or a Unix operating system.
  • control modes for the terminals. These control methods are different for the controlled terminal. For example, one control mode is to display the interface of a controlled terminal on the control interface. Since the controlled terminal may have many, for This control method can only control a single controlled terminal.
  • the operation efficiency of this method is rather low for the case where there are a large number of controlled terminals.
  • the manner of interface display may not be the same, or even the gap is large.
  • the user When the user opens the interface of the controlled terminal, the user will find the The interface is full of the screen of the main control terminal, which may cause an uncomfortable feeling under certain circumstances, which may affect the user's experience.
  • the present embodiment describes a method of controlling the terminal from the perspective of the controller in the terminal control.
  • the technical solution involved in this embodiment can be made into an application in the application market, and the user downloads the application and installs it when using it. Or, if you want to promote an operating system, you can also build this function into the operating system. As a selling point of the operating system, increase the installed capacity of the operating system.
  • the technical solution involved in this embodiment may also be configured as a website, and the user may access the website when using a browser or using other applications, thereby implementing control of the terminal on the webpage.
  • FIG. 2 is a flowchart of a method for controlling a terminal according to an embodiment of the present invention. As shown in FIG. 2, the method includes the following steps:
  • Step S202 acquiring multimedia data from each of the plurality of terminals, wherein the multimedia data of each terminal is used to display the content displayed on the screen of the terminal;
  • Step S204 using multimedia data to display content displayed on each terminal screen of the plurality of terminals in one interface
  • Step S206 receiving an operation signal input by the user through the interface
  • Step S208 acquiring a first terminal corresponding to the operation signal
  • Step S210 Convert the operation signal into a corresponding instruction and send it to the first terminal.
  • the terminal that needs to be controlled may be selected by the user first.
  • the operation of the current terminal is accessible to the network.
  • the controlled terminal can be connected through the network.
  • the above steps describe the operation of the control application to acquire a connection of a plurality of terminals, and the control application performs operations such as acquisition and connection of a plurality of terminals. If the content in the above steps is built into the operating system as a function, it is the operation of acquiring and connecting a plurality of terminals by the function.
  • a server can be used to connect mobile phones of different operating systems.
  • an Android phone and an iOS phone can be connected through a Mac Mini.
  • the terminal can be directly connected to the hub through the USB cable, and the hub is connected to the server, thereby realizing the connection between the terminal and the server.
  • authorization may be set for the activation of the control application or the use of the function of the operating system.
  • the use of the control application may require the user to input a password or use biometric information to identify the user, if the user password The input is correct or it is recognized that the authorized user is allowed to use the function. Identification of biological information is currently widely used for fingerprint recognition. Of course, other biometric technologies can also be used, for example, iris recognition, facial recognition, voiceprint recognition, and the like.
  • the user can obtain the terminal control function by purchasing, and the price of the purchase can be determined according to the number of terminals that can be controlled. The more the number of terminals that can be controlled, the higher the price, and the user can provide the user after purchasing. A license is used as a proof of purchase.
  • the multiple terminals may be smart terminal devices such as a computer, a notebook, a tablet computer, a mobile phone, a smart wearable device, or a smart home device such as an air conditioner, a refrigerator, a washing machine, or a television.
  • the smart device such as the smart terminal device and the smart home device, refers to a device that has a computing processing capability and can perform connection control.
  • the foregoing smart device is only an example of the embodiment, which is not limited in this embodiment.
  • the multimedia data from each of the plurality of terminals is detected and acquired in real time, and the content displayed on each of the plurality of terminals is displayed in an interface using the multimedia data.
  • the multimedia data of each terminal is used to display the content displayed on the screen of the corresponding terminal.
  • the acquired multimedia data may be, but is not limited to, displayed on the screen by the plurality of terminals.
  • the multimedia data that is converted and transmitted by the content, or obtained may be, but is not limited to, obtained by converting the data sent by the plurality of terminals for displaying the content displayed on the screen.
  • the operation signal may be, but is not limited to, a signal generated by a user's touch on the screen, and may be, but is not limited to, a signal input by a user through some input devices.
  • the above input device may include, but is not limited to, a mouse, a keyboard, a touchpad, and the like. For example, taking the user's touch on the screen as an example, the signal generated by the user's touch interface can be received, and the signal is converted into an operation signal.
  • the multimedia data for displaying the content displayed by the screen from each of the plurality of terminals is acquired, and the screens of the plurality of terminals are displayed in an interface using the acquired multimedia data. Displaying content, thereby realizing display of a plurality of terminal screens on one interface, receiving an operation signal input by the user through the interface, acquiring a first terminal corresponding to the operation signal, and converting the operation signal into a corresponding instruction, the instruction Send to the first terminal to implement control of the terminal displayed on the interface.
  • the above embodiments of the present application solve the technical problem of low control efficiency of the related art terminal control.
  • the first terminal may be acquired according to the location of the operation signal on the interface and the location of each terminal displayed on the interface.
  • an optional implementation manner is provided. An optional implementation may include the following steps:
  • Step 502 Acquire first location information input by the operation signal in the interface.
  • Step 504 Acquire second location information displayed by each terminal in the interface, where the second location information is a range occupied by the terminal in the interface.
  • Step 506 Determine, according to the first location information and the second location information, the first terminal corresponding to the operation signal.
  • the first location information and the second location information may be, but are not limited to, represented by coordinate values, or may be, but not limited to, represented by location information of pixel points, for example, may be used in
  • the center of the area where the operation signal is generated on the interface to represent the operation signal obtain the coordinate value of the center in the coordinate system of the interface as the first position information
  • the second position information may be the area displayed by the terminal on the interface.
  • the coordinate interval corresponding to the boundary determines the first terminal corresponding to the operation signal by determining a coordinate interval corresponding to the second position information in which the coordinate value corresponding to the first position information falls.
  • the terminal corresponding to the range in which the first location information falls within the range occupied by each terminal in the interface is used as the first terminal, where the first terminal includes one or more terminals.
  • the interface may be pre-divided into a plurality of areas, and each area displays a terminal, and the first terminal is obtained by determining an area in which the operation signal falls.
  • FIG. 3 is a schematic diagram of a preferred acquisition of a first terminal according to an embodiment of the present invention. As shown in FIG. 3, the interface is divided into nine areas, and the screens of the terminals 1 to 9 are respectively displayed, and an operation signal is obtained. The first location information input in the interface falls into the area where the terminal 4 is located, and then the terminal 4 can be determined to be the first terminal.
  • the purpose of acquiring the first terminal to be controlled by the operation signal and the position information of the terminal is achieved.
  • the function of multi-screen synchronization of the operation signals input by the user can also be implemented. That is to say, when the number of the first terminals is plural, the operation signal input by the user is converted into a corresponding instruction and transmitted to the plurality of terminals, thereby controlling the plurality of terminals to perform the same operation.
  • the user can simultaneously control multiple test phones to test the application. After the user selects the multi-screen control function, the selected test phone will remain synchronized in any operation behavior initiated by one of the test phones.
  • the iOS and Android test mobile phones can simultaneously perform multi-screen synchronization operations.
  • Android test phone multi-screen operation synchronization response time can be controlled within 0.1 seconds.
  • the iOS test phone multi-screen operation synchronization response time can be controlled within 0.5 seconds.
  • test mobile phone screen in which any synchronous operation is performed may be hidden or closed.
  • a multi-screen synchronous click, slide, text input, and the like test operation of the tested application may be performed. It is also possible to perform multi-screen synchronized screenshots, horizontal screen, vertical screen, return to the previous step, and return to the main screen for the tested application.
  • the screen of the first terminal to be controlled may be enlarged to be a full screen display.
  • the user can control multiple terminals.
  • the user can select the test terminal and Test the terminal screen for full-screen zoom operation.
  • the full screen magnification of the single device screen of the mobile phone can be tested for iOS and Android.
  • the control device when performing full-screen enlargement of the terminal screen, may perform operations such as clicking, sliding, etc. to enlarge the screen of the controlled terminal.
  • a full-screen zoom operation can be identified by double-clicking, two-finger dragging, or other gestures.
  • a test operation such as clicking, sliding, text input, etc. of the tested application is supported.
  • the tested application when performing full-screen enlargement of the terminal screen, the tested application may perform a function operation such as taking a screenshot, a horizontal screen, a vertical screen, returning to the previous step, and returning to the home screen.
  • a function operation such as taking a screenshot, a horizontal screen, a vertical screen, returning to the previous step, and returning to the home screen.
  • the keyboard ESC key when performing full-screen enlargement of the terminal screen, may be clicked to quickly exit the full-screen zoom mode. You can also manually turn off full screen zoom mode.
  • the operation signal may be converted into its corresponding instruction according to the type of the operating system of the first terminal or the type of the application on the first terminal.
  • the foregoing method may also Including the following steps:
  • Step 602 Convert the operation signal into an instruction corresponding to the type of the operating system or the type of the application according to the type of the operating system of the first terminal acquired in advance or the type of the application on the first terminal;
  • Step 604 sending an instruction to the first terminal.
  • the type of the operating system of the first terminal acquired in advance is an Android system
  • the operation signal is converted into an instruction corresponding to the Android system, and the instruction is sent to the first terminal.
  • the type of the application on the first terminal acquired in advance is an instant messaging application
  • the operation signal is converted into an instruction corresponding to the instant messaging application, and the instruction is sent to the first terminal.
  • the operation signal is converted into an instruction corresponding to the type of the operating system, if the operation signal is according to the type of the application on the first terminal
  • the operation signal is converted into an instruction corresponding to the type of the application, so that the conversion of the operation signal can be applied to various operating systems or applications.
  • the method may further include the following steps in order to implement the function of converting and transmitting the command:
  • Step 702 Convert an operation signal into an operation signal to the first terminal.
  • Step 704 Send an operation signal to the first terminal to the first terminal, where the operation signal to the first terminal is used as an input to the first terminal.
  • the operation signal is to input the content "auto show” to be searched in the search field on the browser application opened by the first terminal, and perform a search. Then, the operation signal is converted into an operation signal to the first terminal, and the operation signal to the first terminal may be inputting "auto show” in the search bar of the browser, and clicking the search button. And transmitting the operation signal of the first terminal to the first terminal, thereby controlling the first terminal to perform the foregoing operation.
  • the operation signal is converted into an operation signal to the first terminal as an input to the first terminal, so that the first terminal can be input on the interface, thereby making the control of the terminal more convenient and improving the user's Experience.
  • an operation signal input by the user may be received by, but not limited to, an area on the interface that does not overlap with the screen content of the displayed terminal.
  • the operation signal input by the user through the interface may be received by, but not limited to, receiving an operation signal input by the user in a preset area, wherein the area does not overlap with an area displaying the screen content of each terminal.
  • FIG. 4 is a schematic diagram of a preferred pre-set area according to an embodiment of the present invention.
  • the interface may be divided into two areas, and an area above the interface is used.
  • the screen of the terminal is displayed, and the area below the interface can be used as the above-mentioned preset area for receiving an operation signal.
  • the selection control for selecting the first terminal may be preset on the interface, and the terminal selected by the user through the selection control is received, wherein the selected terminal is used as the first terminal. Acquiring the first terminal and transmitting an operation signal to the first terminal.
  • the selection control may be, but is not limited to, an option that provides one of the following: an option for a single terminal, an option for a group of terminals, and an option for all terminals. That is to say, the above multiple terminals can be controlled individually or in batch by selecting a control.
  • FIG. 5 is a schematic diagram of a preferred selection control according to an embodiment of the present invention. As shown in FIG. 5, a selection control may be disposed in the preset area, and the selection control is used. In selecting the first terminal.
  • an embodiment of the present invention further provides a storage medium, the storage medium including a stored program, wherein the terminal control method described above is executed when the program is running .
  • an embodiment of the present invention further provides a processor configured to execute a program, wherein the program is executed to execute the terminal control method described above.
  • the application restriction method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases The former is a better implementation.
  • the technical solution of the present invention in essence or the contribution to the related art can be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, CD-ROM).
  • the instructions include a number of instructions for causing a terminal device (which may be a cell phone, computer, server, or network device, etc.) to perform the methods described in various embodiments of the present invention.
  • FIG. 6 is a schematic diagram of a terminal screen display processing apparatus according to an embodiment of the present invention.
  • the device includes a grab module 61, a transmitting module 63, and a first display module 65.
  • the capture module 61 is configured to capture a picture of each terminal of the multiple terminals, where the picture of each terminal is used to display the content displayed on the screen of the terminal, and the picture of each terminal includes multiple pictures, and more The picture is a picture when the terminal screen changes; the sending module 63 is configured to send a picture; the first display module 65 is configured to display the content displayed by each of the plurality of terminals in one interface using the picture.
  • the above-mentioned capture module 61, the sending module 63, and the first display module 65 may correspond to the steps S102 to S106 in the first embodiment, and the examples and application scenarios implemented by the three modules and corresponding steps. The same, but not limited to, the contents disclosed in the above embodiment 1.
  • the picture when the screen of each of the plurality of terminals changes is captured, and the content displayed on each of the plurality of terminals is displayed and displayed in an interface. Therefore, when the screen of the terminal is displayed, the picture for indicating the change of the screen is transmitted only when the screen of the terminal being displayed changes, so that the connection for controlling the plurality of terminals does not need to transmit the screen data in real time, thereby reducing the screen display process.
  • the content displayed on multiple terminal screens is displayed on one interface, and the purpose of the picture when the terminal screen changes is captured, thereby realizing the technology for saving the bandwidth of the display terminal screen. effect. Therefore, the above embodiment of the present application solves the technical problem of wasting bandwidth when displaying the screen of the terminal in the related art.
  • the capturing module includes: a first acquiring unit configured to acquire the number of terminals displayed in one interface; a setting unit configured to set a resolution according to the number of terminals; the capturing unit , set to capture the picture of each terminal according to the resolution, or, after grabbing the picture of each terminal, convert according to the resolution.
  • the first acquiring unit, the setting unit, and the capturing unit may correspond to steps 202 to 206 in Embodiment 1, and the three modules are the same as the examples and application scenarios implemented by the corresponding steps, but It is not limited to the contents disclosed in the above embodiment 1.
  • the capturing unit includes: a second acquiring unit, configured to acquire a picture of each terminal according to a predetermined time interval; and the processing unit is configured to determine the acquired two adjacent pictures. Whether the change exceeds the threshold, and if the threshold is exceeded, a post-fetched picture in the two pictures is used as the picture displayed in the interface.
  • the foregoing second obtaining unit and the processing unit may correspond to step 302 to step 304 in Embodiment 1, and the two modules are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the above.
  • the processing unit is configured to: acquire pixel points of each picture in the two pictures; compare whether each pixel point in the two pictures changes, and record the changed pixel point; Determine if the number of pixels that have changed exceeds the threshold.
  • processing unit may correspond to step 402 to step 406 in Embodiment 1, which is the same as the example and application scenario implemented by the corresponding step, but is not limited to the content disclosed in Embodiment 1 above. .
  • the sending module is configured to: acquire a network condition between the device that captures the picture and the device that uses the picture for display. ; Process the captured image according to the network status; send the processed image.
  • the foregoing network condition includes at least one of the following: a packet loss rate, a bandwidth.
  • FIG. 7 is a schematic diagram of a terminal control apparatus according to an embodiment of the present invention. As shown in FIG. 7, the apparatus includes a first The acquisition module 71, the second presentation module 73, the receiving module 75, the second acquisition module 77, and the processing module 79.
  • the first obtaining module 71 is configured to acquire multimedia data from each of the plurality of terminals, wherein the multimedia data of each terminal is used to display the content displayed on the screen of the terminal; and the second display module 73 is configured to The multimedia data is used to display the content displayed on each of the plurality of terminals in an interface; the receiving module 75 is configured to receive an operation signal input by the user through the interface; and the second obtaining module 77 is configured to acquire the first corresponding to the operation signal.
  • a terminal; the processing module 79 is configured to convert the operation signal into a corresponding instruction and send the signal to the first terminal.
  • first acquiring module 71 the second displaying module 73, the receiving module 75, the second obtaining module 77, and the processing module 79 may correspond to steps S202 to S210 in the first embodiment, and three modules.
  • the examples and application scenarios implemented by the corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment 1.
  • the multimedia data for displaying the content displayed by the screen from each of the plurality of terminals is acquired, and the screens of the plurality of terminals are displayed in an interface using the acquired multimedia data. Displaying content, thereby realizing display of a plurality of terminal screens on one interface, receiving an operation signal input by the user through the interface, acquiring a first terminal corresponding to the operation signal, and converting the operation signal into a corresponding instruction, the instruction Send to the first terminal to implement control of the terminal displayed on the interface.
  • the above embodiments of the present application solve the technical problem of low control efficiency of the related art terminal control.
  • the second obtaining module 77 includes: a first acquiring unit, configured to acquire first position information input by the operation signal in the interface; and a second acquiring unit configured to acquire Second location information displayed by the terminal in the interface, wherein the second location information is a range occupied by the terminal in the interface; and the determining unit is configured to be based on the first location information and the location The second location information determines the first terminal corresponding to the operation signal.
  • the foregoing first obtaining unit, the second obtaining unit, and the determining unit may correspond to steps 502 to 506 in Embodiment 1, and the three modules are the same as the examples and application scenarios implemented by the corresponding steps. However, it is not limited to the contents disclosed in the above embodiment 1.
  • the determining unit may use, as the first terminal, a terminal corresponding to a range in which the first location information falls within a range occupied by each terminal in the interface, where the first terminal includes one or more Terminals.
  • the processing module 79 is configured to: convert the operation signal into the type or application of the operating system according to the type of the operating system of the first terminal acquired in advance or the type of the application on the first terminal.
  • the instruction corresponding to the type; the instruction is sent to the first terminal.
  • processing module may also correspond to step 602 to step 604 in Embodiment 1, which is the same as the example and application scenario implemented by the corresponding step, but is not limited to the one disclosed in Embodiment 1 above. content.
  • the processing module is further configured to: convert the operation signal into an operation signal to the first terminal; and send an operation signal to the first terminal to the first terminal, where the first terminal is The operation signal is used as an input to the first terminal.
  • processing module may also correspond to step 702 to step 704 in Embodiment 1, which is the same as the example and application scenario implemented by the corresponding step, but is not limited to the one disclosed in Embodiment 1 above. content.
  • the receiving module is configured to: receive an operation signal input by the user in a preset area, where the area does not overlap with an area displaying the screen content of each terminal; the second acquiring module The method is configured to: receive a terminal selected by the user by using a selection control, where the selected terminal is used as the first terminal; and send an operation signal to the first terminal.
  • the selection control provides one of the following options: an option for a single terminal, an option for a group of terminals, an option for all terminals.
  • Embodiments of the present invention may provide a computer terminal, which may be any one of computer terminal groups.
  • the foregoing computer terminal may also be replaced with a terminal device such as a mobile terminal.
  • the computer terminal may be located in at least one network device of the plurality of network devices of the computer network.
  • the computer terminal may execute the program code of the following steps in the application restriction method of the application: fetching a picture of each of the plurality of terminals, wherein the picture of each terminal is used to display the screen of the terminal
  • the displayed content, the picture of each terminal includes a plurality of pictures, the multiple pictures are pictures when the screen of the terminal changes; the pictures are sent; the pictures are displayed in an interface to display the contents displayed on each of the plurality of terminals.
  • the computer terminal may execute the program code of the following steps in the data transmission processing method of the application program: acquiring the number of terminals displayed in one interface; setting the resolution according to the number of terminals; and crawling each according to the resolution A picture of the terminal, or, after capturing the picture of each terminal, converting according to the resolution.
  • FIG. 8 shows a hardware structure block diagram of an optional computer terminal.
  • the computer terminal 8 may include one or more (81a, 81b, ..., 81n are used in the figure).
  • Processor 81 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 83 configured to store data, and a transmission device 85 configured as a communication function.
  • a processing device such as a microprocessor MCU or a programmable logic device FPGA
  • memory 83 configured to store data
  • a transmission device 85 configured as a communication function.
  • it can also include: display, input/output interface (I/O interface), universal serial bus (USB) port (which can be included as one of the ports of the I/O interface), network interface, power supply And / or camera.
  • USB universal serial bus
  • FIG. 8 is merely illustrative and does not limit the structure of the above electronic device.
  • computer terminal 8 may also include more or fewer components than shown in FIG. 8, or have a different configuration than that shown in FIG.
  • processors 81 and/or other data processing circuits may be referred to herein generally as "data processing circuits.”
  • the data processing circuit may be embodied in whole or in part as software, hardware, firmware or any other combination.
  • the data processing circuitry can be a single, separate processing module, or incorporated in whole or in part into any of the other components in computer terminal 8.
  • the data processing circuit is controlled as a processor (e.g., selection of a variable resistance terminal path connected to the interface).
  • the memory 83 can be configured as a software program and a module for storing application software, such as a program instruction/data storage device corresponding to the data transmission processing method in the embodiment of the present invention, and the processor 81 runs the software program and module stored in the memory 83. Thereby, various functional applications and data processing are performed, that is, a data transmission processing method of the above-described application is implemented.
  • Memory 83 may include high speed random access memory and may also include non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, memory 83 may further include memory remotely located relative to processor 81, which may be coupled to computer terminal 8 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • Transmission device 85 is arranged to receive or transmit data via a network.
  • the above specific network example may include a wireless network provided by a communication provider of the computer terminal 8.
  • transmission device 85 includes a Network Interface Controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet.
  • NIC Network Interface Controller
  • transmission device 85 can be a Radio Frequency (RF) module configured to communicate with the Internet wirelessly.
  • RF Radio Frequency
  • the display can be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with the user interface of computer terminal 8.
  • LCD liquid crystal display
  • the computer terminal 8 shown in FIG. 8 above may include hardware components (including circuits), software components (including computer code stored on a computer readable medium), or A combination of both hardware and software components. It should be noted that FIG. 8 is only one example of a specific specific example, and is intended to show the types of components that may be present in the above computer terminal.
  • the computer terminal 8 shown in FIG. 8 described above has a touch display (also referred to as a "touch screen” or a “touch display screen”).
  • the computer terminal 8 shown in FIG. 8 above has an image user interface (GUI), and the user can interact with the GUI by touching a finger contact and/or gesture on the touch-sensitive surface, where the person is
  • GUI image user interface
  • the optional machine interaction includes the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, emailing, calling interfaces, playing digital video, playing digital music and/or web browsing, etc.
  • the executable instructions for performing the human-machine interaction functions described above are configured/stored in one or more processor-executable computer program products or readable storage media.
  • the processor may call the information stored in the memory and the application program by the transmission device to perform the following steps: acquiring a picture of each terminal according to a predetermined time interval; determining whether the acquired change of the two adjacent pictures exceeds a threshold, and exceeding In the case of a threshold, a picture captured after the two pictures is used as a picture displayed in the interface.
  • the processor may call the memory stored information and the application by the transmission device to perform the following steps: acquiring pixels of each picture in the two pictures; comparing whether each pixel in the two pictures changes, the recording occurs. A pixel that changes; determines if the number of pixels that have changed exceeds a threshold.
  • the foregoing processor may further execute the following program code: obtain a network condition between the device that captures the image and the device that uses the image to display; and process the captured image according to the network condition; after the sending process picture of.
  • FIG. 8 is only an illustration, and the computer terminal can also be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, an applause computer, and a mobile Internet device (Mobile Internet Devices, MID). ), PAD and other terminal devices.
  • FIG. 8 does not limit the structure of the above electronic device.
  • computer terminal 8 may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in Figure 8, or have a different configuration than that shown in Figure 8.
  • Embodiments of the present invention may provide a computer terminal, which may be any one of computer terminal groups.
  • the foregoing computer terminal may also be replaced with a terminal device such as a mobile terminal.
  • the computer terminal may be located in at least one network device of the plurality of network devices of the computer network.
  • the computer terminal may execute the program code of the following steps in the application restriction method of the application: acquiring multimedia data from each of the plurality of terminals, wherein the multimedia data of each terminal is used for displaying the terminal The content displayed on the screen; displaying the content displayed on each terminal screen of the plurality of terminals in an interface; receiving an operation signal input by the user through the interface; acquiring the first terminal corresponding to the operation signal; converting the operation signal into a corresponding The instructions are sent to the first terminal.
  • the computer terminal may execute the program code of the following steps in the data transmission processing method of the application program: acquiring the first location information input by the operation signal in the interface; and acquiring the second location displayed by each terminal in the interface. Information, wherein the second location information is a range occupied by the terminal in the interface; and determining, by the first location information and the second location information, the first terminal corresponding to the operation signal.
  • FIG. 9 shows a hardware structure block diagram of an optional computer terminal.
  • the computer terminal 9 may include one or more (91a, 91b, ..., 91n are used in the figure).
  • Processor 91 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 93 configured to store data, and a transmission device 95 configured as a communication function.
  • a processing device such as a microprocessor MCU or a programmable logic device FPGA
  • memory 93 configured to store data
  • a transmission device 95 configured as a communication function.
  • it can also include: display, input/output interface (I/O interface), universal serial bus (USB) port (which can be included as one of the ports of the I/O interface), network interface, power supply And / or camera.
  • USB universal serial bus
  • computer terminal 9 may also include more or fewer components than shown in FIG. 9, or have a different configuration than that shown in FIG.
  • processors 91 and/or other data processing circuits may be referred to herein generally as "data processing circuits.”
  • the data processing circuit may be embodied in whole or in part as software, hardware, firmware or any other combination.
  • the data processing circuitry can be a single, separate processing module, or incorporated in whole or in part into any of the other components in computer terminal 9.
  • the data processing circuit is controlled as a processor (e.g., selection of a variable resistance terminal path connected to the interface).
  • the memory 93 can be configured as a software program and a module for storing application software, such as a program instruction/data storage device corresponding to the data transmission processing method in the embodiment of the present invention, and the processor 91 runs the software program and the module stored in the memory 93. Thereby, various functional applications and data processing are performed, that is, a data transmission processing method of the above-described application is implemented.
  • Memory 93 may include high speed random access memory and may also include non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory.
  • memory 93 may further include memory remotely located relative to processor 91, which may be connected to computer terminal 9 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • Transmission device 95 is arranged to receive or transmit data via a network.
  • the above specific network example may include a wireless network provided by a communication provider of the computer terminal 9.
  • transmission device 95 includes a Network Interface Controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet.
  • NIC Network Interface Controller
  • the transmission device 95 can be a Radio Frequency (RF) module for communicating with the Internet wirelessly.
  • RF Radio Frequency
  • the display can be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with the user interface of computer terminal 9.
  • LCD liquid crystal display
  • the computer terminal 9 shown in FIG. 9 above may include hardware components (including circuits), software components (including computer code stored on a computer readable medium), or A combination of both hardware and software components. It should be noted that FIG. 9 is only one example of a specific specific example, and is intended to show the types of components that may be present in the above computer terminal.
  • the computer terminal 9 shown in FIG. 9 described above has a touch display (also referred to as a "touch screen” or a “touch display screen”).
  • the computer terminal 9 shown in FIG. 9 above has an image user interface (GUI), and the user can interact with the GUI by touching a finger contact and/or gesture on the touch-sensitive surface, where the person is
  • GUI image user interface
  • the optional machine interaction includes the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, emailing, calling interfaces, playing digital video, playing digital music and/or web browsing, etc.
  • the executable instructions for performing the human-machine interaction functions described above are configured/stored in one or more processor-executable computer program products or readable storage media.
  • the processor may call the information and the application stored in the memory by the transmission device to perform the following steps: the terminal corresponding to the range in which the first location information falls within the range occupied by each terminal in the interface is used as the first terminal, where The first terminal includes one or more terminals.
  • the processor may call the information and the application stored by the memory through the transmission device to perform the following steps: converting the operation signal into an operating system according to the type of the operating system of the first terminal acquired in advance or the type of the application on the first terminal.
  • the type or the instruction corresponding to the type of the application; the instruction is sent to the first terminal.
  • the foregoing processor may further execute the following steps: converting the operation signal into an operation signal to the first terminal; and transmitting an operation signal to the first terminal to the first terminal, where The operational signal acts as an input to the first terminal.
  • the processor may further execute the following program code: receiving an operation signal input by the user in a preset area, where the area does not overlap with an area displaying the screen content of each terminal;
  • the first terminal includes: receiving the terminal selected by the user through the selection control, wherein the selected terminal is used as the first terminal; and the operation signal is sent to the first terminal.
  • the selection control provides one of the following options: options for a single terminal, options for a group of terminals, options for all terminals.
  • FIG. 9 is only an illustration, and the computer terminal can also be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, an applause computer, and a mobile Internet device (Mobile Internet Devices, MID). ), PAD and other terminal devices.
  • FIG. 9 does not limit the structure of the above electronic device.
  • computer terminal 9 may also include more or fewer components (such as a network interface, display device, etc.) than shown in FIG. 9, or have a different configuration than that shown in FIG.
  • Embodiments of the present invention also provide a storage medium.
  • the foregoing storage medium may be configured to save the program code executed by the terminal screen display processing method provided in Embodiment 1 above.
  • the foregoing storage medium may be located in any one of the computer terminal groups in the computer network, or in any one of the mobile terminal groups.
  • the storage medium is configured to store program code for performing the steps of: capturing a picture of each of the plurality of terminals, wherein the picture of each terminal is used to display the terminal
  • the content displayed on the screen, the picture of each terminal includes a plurality of pictures, the plurality of pictures are pictures when the screen of the terminal changes; the pictures are sent; the pictures are displayed in an interface to display the contents displayed on each of the plurality of terminals .
  • the storage medium is configured to store program code for performing the steps of: acquiring the number of terminals displayed in one interface; setting the resolution according to the number of terminals; crawling each according to the resolution A picture of the terminal, or, after capturing the picture of each terminal, converting according to the resolution.
  • the storage medium is configured to store program code for performing the following steps: acquiring a picture of each terminal according to a predetermined time interval; determining whether the acquired change of the two adjacent pictures exceeds Threshold, if a threshold is exceeded, a picture captured in the two pictures is used as the picture displayed in the interface.
  • the storage medium is configured to store program code for performing the following steps: acquiring pixel points of each picture in the two pictures; comparing whether each pixel point in the two pictures occurs Change, record the pixel that has changed; determine if the number of pixels that have changed exceeds the threshold.
  • the storage medium is configured to store program code for performing the following steps: obtaining a network condition between the device that captures the picture and the device that uses the picture for display; and fetching according to the network condition The image to be processed is processed; the processed image is sent.
  • the network status includes at least one of the following: a packet loss rate, and a bandwidth.
  • Embodiments of the present invention also provide a storage medium.
  • the foregoing storage medium may be used to save the program code executed by the terminal control method provided in Embodiment 1 above.
  • the foregoing storage medium may be located in any one of the computer terminal groups in the computer network, or in any one of the mobile terminal groups.
  • the storage medium is configured to store program code for performing the following steps: acquiring multimedia data from each of the plurality of terminals, wherein the multimedia data of each terminal is used for displaying The content displayed on the screen of the terminal; displaying the content displayed on each terminal screen of the plurality of terminals in one interface by using the multimedia data; receiving an operation signal input by the user through the interface; acquiring the first terminal corresponding to the operation signal; converting the operation signal into The corresponding instruction is sent to the first terminal.
  • the storage medium is configured to store program code for performing the following steps: acquiring first location information input by the operation signal in the interface; acquiring a second location displayed by each terminal in the interface Information, wherein the second location information is a range occupied by the terminal in the interface; and determining, by the first location information and the second location information, the first terminal corresponding to the operation signal.
  • the storage medium is configured to store program code for performing the following steps: the terminal corresponding to the range in which the first location information falls within the range occupied by each terminal in the interface as the first A terminal, wherein the first terminal comprises one or more terminals.
  • the storage medium is configured to store program code for performing the following steps: operating signals according to the type of the operating system of the first terminal acquired in advance or the type of the application on the first terminal The instruction corresponding to the type of the operating system or the type of the application; the instruction is sent to the first terminal.
  • the storage medium is configured to store program code for performing the following steps: converting the operation signal into an operation signal to the first terminal; transmitting an operation signal to the first terminal to the first a terminal, wherein an operation signal to the first terminal is used as an input to the first terminal.
  • the storage medium is configured to store program code for performing the following steps: receiving an operation signal input by the user in a preset area, wherein the area and the screen content of each terminal are displayed. The area does not overlap; the first terminal corresponding to the operation signal includes: receiving the terminal selected by the user through the selection control, wherein the selected terminal is used as the first terminal; and the operation signal is sent to the first terminal.
  • the selection control provides one of the following options: an option for a single terminal, an option for a group of terminals, and an option for all terminals.
  • the disclosed technical contents may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, unit or module, and may be electrical or otherwise.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the technical solution of the present invention may contribute to the related art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause a computer device (which may be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and the like. .
  • the solution provided by the embodiment of the present invention can be applied to the display process of the terminal screen.
  • the technical problem of wasting bandwidth when displaying the screen of the terminal in the related art is solved by the embodiment of the invention, and the bandwidth of the screen of the display terminal is saved.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Digital Computer Display Output (AREA)

Abstract

一种终端屏幕显示处理方法、装置及存储介质，其中，该方法包括：抓取多个终端中的每个终端的图片，其中，每个终端的图片用于展示终端的屏幕显示的内容，每个终端的图片包括多张图片，多张图片为该终端屏幕发生变化时的图片（S102）；发送图片（S104）；使用图片在一个界面中展示多个终端中的每个终端屏幕显示的内容（S106）。该方法解决了相关技术显示终端屏幕时浪费带宽的技术问题。

Description

终端屏幕显示处理方法、装置及存储介质
本申请要求于2017年04月05日提交中国专利局、申请号为201710217952.2、发明名称“终端屏幕显示处理方法、装置及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及网络技术领域，具体而言，涉及一种终端屏幕显示处理方法、装置及存储介质。
背景技术
随着电子和软件技术的发展，终端设备的功能和界面得到不断的优化，相应的，友好的用户界面和直观的操作体验便成为了终端设备领域的重要课题。随着终端设备种类和数量的增多，用户对控制终端设备时的便捷程度也有了新的追求。但是，现有的终端控制方式中对于被控制的终端的屏幕的显示是实时的将被控制的终端的屏幕信息传送给主控的设备。在这种情况下，就导致主控设备与被控终端之间建立的连接的带宽一直是被占用的状态，严重的造成了带宽的浪费，进一步也将导致传输其他数据时的卡顿，使得用户体验也较差。
针对上述相关技术显示终端屏幕时浪费带宽的问题，目前尚未提出有效的解决方案。
发明内容
本发明实施例提供了终端屏幕显示处理方法、装置及存储介质，以至少解决相关技术显示终端屏幕时浪费带宽的技术问题。
根据本发明实施例的一个方面，提供了一种终端屏幕显示处理方法，包括：抓取多个终端中的每个终端的图片，其中，所述每个终端的图片用于展示所述终端的屏幕显示的内容，所述每个终端的图片包括多张图片，所述多张图片为该终端屏幕发生变化时的图片；发送所述图片；使用所述图片在一个界面中展示所述多个终端中的每个终端屏幕显示的内容。
可选地，抓取所述每个终端的图片包括：获取在所述一个界面中显示的终端的数 量；根据所述终端的数量设置分辨率；根据所述分辨率抓取所述每个终端的图片，或者，抓取所述每个终端的图片之后根据所述分辨率进行转换。
可选地，抓取所述每个终端的图片包括：按照预定时间间隔获取所述每个终端的图片；判断获取到的相邻的两张图片的变化是否超过阈值，在超过所述阈值的情况下，将所述两张图片中的后抓取的一张图片用作在所述界面中展示的图片。
可选地，判断获取到的所述两张图片的变化是否超过所述阈值包括：获取所述两张图片中的每张图片的像素点；比较所述两张图片中的每个像素点是否发生变化，记录发生变化的像素点；判断发生变化的像素点的数量是否超过所述阈值。
可选地，在抓取所述图片和使用所述图片进行展示不在同一装置的情况下，发送所述图片包括：获取抓取所述图片的装置与使用所述图片进行展示的装置之间的网络状况；根据所述网络状况对抓取到的图片进行处理；发送处理后的所述图片。
可选地，所述网络状况包括以下至少之一：丢包率、带宽。
为了实现上述目的，根据本发明的另一方面，提供了一种存储介质，所述存储介质包括存储的程序，其中，在所述程序运行时控制所述存储介质所在设备执行上述所述的终端屏幕显示处理方法。
为了实现上述目的，根据本发明的另一方面，提供了一种处理器，所述处理器设置为运行程序，其中，所述程序运行时执行上述所述的终端屏幕显示处理方法。
根据本发明实施例的另一方面，还提供了一种终端屏幕显示处理装置，包括：抓取模块，设置为抓取多个终端中的每个终端的图片，其中，所述每个终端的图片用于展示所述终端的屏幕显示的内容，所述每个终端的图片包括多张图片，所述多张图片为该终端屏幕发生变化时的图片；发送模块，设置为发送所述图片；第一展示模块，设置为使用所述图片在一个界面中展示所述多个终端中的每个终端屏幕显示的内容。
可选地，所述抓取模块包括：第一获取单元，设置为获取在所述一个界面中显示的终端的数量；设置单元，设置为根据所述终端的数量设置分辨率；抓取单元，设置为根据所述分辨率抓取所述每个终端的图片，或者，抓取所述每个终端的图片之后根据所述分辨率进行转换。
在本发明实施例中，抓取多个终端中的每个终端的图片，其中，每个终端的图片设置为展示终端的屏幕显示的内容，每个终端的图片包括多张图片，多张图片为该终端屏幕发生变化时的图片；发送图片；使用图片在一个界面中展示多个终端中的每个终端屏幕显示的内容。通过本发明实施例解决了相关技术显示终端屏幕时浪费带宽的 技术问题，节约了显示终端屏幕时带宽。
附图说明
此处所说明的附图用来提供对本发明的进一步理解，构成本申请的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：
图1是根据本发明实施例的一种终端屏幕显示处理方法的流程图；
图2是根据本发明实施例的一种终端控制方法的流程图；
图3是根据本发明实施例的一种优选的获取第一终端的示意图；
图4是根据本发明实施例的一种优选的预先设置的区域的示意图；
图5是根据本发明实施例的一种优选的选择控件的示意图；
图6是根据本发明实施例的一种终端屏幕显示处理装置示意图；
图7是根据本发明实施例的一种终端控制装置示意图；
图8是根据本发明实施例的一种计算机终端的硬件结构框图一；以及
图9是根据本发明实施例的一种计算机终端的硬件结构框图二。
具体实施方式
为了使本技术领域的人员更好地理解本发明方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本发明保护的范围。
需要说明的是，本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本发明的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
实施例1
根据本发明实施例，还提供了一种终端屏幕显示处理的方法实施例，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
目前在移动终端(或称为移动设备)上使用的操作系统比较多的是微软的WP操作系统、安卓系统、和苹果公司的iOS系统，除了这些操作系统以外可能还有其他类型的操作系统。
在计算上(例如，个人电脑PC)上也可以使用多种操作系统，例如，Windows操作系统、苹果公司的操作系统、linux操作系统或者Unix操作系统等。
目前有多种对于终端的控制方式，这些控制方式对于被控制的终端的处理方法不同，例如，在一种控制方式中，将被控制的终端的屏幕显示在主控设备上的时候，需要将用于显示被控制的终端的屏幕的数据实时的传输给主控设备，对于这种控制方法，控制双方传输数据的连接的带宽一直保持在占用的状态，不仅会浪费带宽，还会影响其他数据的传输。
基于上述考虑，本实施例从终端屏幕显示的角度来对终端的控制方法进行说明。本实施例中涉及到的技术方案，可以做成一个应用放在应用市场中，用户在使用的时候下载该应用并进行安装。或者，如果为了推广某种操作系统，也可以将该功能内置在操作系统中，作为该操作系统的一个卖点，增加操作系统的装机量。
在上述运行环境下，本实施例提供了如图1所示的一种终端屏幕显示处理方法。图1是根据本发明实施例的一种终端屏幕显示处理方法的流程图，如图1所示，包括如下步骤：
步骤S102，抓取多个终端中的每个终端的图片，其中，每个终端的图片用于展示终端的屏幕显示的内容，每个终端的图片包括多张图片，多张图片为该终端屏幕发生变化时的图片；
步骤S104，发送图片；
步骤S106，使用图片在一个界面中展示多个终端中的每个终端屏幕显示的内容。
在上述步骤中，可以首先由用户选择需要被控制的多个终端，一般情况下现在的终端的运行是可以对网络进行访问的。在上述步骤中，可以通过网络连接被控制的终端。
上述步骤中描述了该终端屏幕显示处理方法对多个终端的屏幕进行显示的操作，该控制应用进行了多个终端的屏幕显示内容的抓取和展示等操作。
作为一种可选的实施例，可以使用一台服务器来连接不同操作系统的终端，例如，可以通过一个Mac Mini连接安卓手机和iOS手机。
对于对多个终端屏幕显示内容的处理在一般的操作系统中均是支持的，例如，即使作为限制最严格的iOS系统也是支持的，对于其他比较开放的系统，例如，安卓系统或者Linux系统等其也操作系统也都支持。因此上述步骤中涉及到的技术方案可支持的操作系统更加广泛。
作为一种可选的实施例，上述多个终端可以为电脑、笔记本、平板电脑、手机、智能穿戴设备等智能终端设备，也可以是空调、冰箱、洗衣机、电视等智能家居设备。需要说明的是，上述智能终端设备和智能家居设备等智能设备指具有计算处理能力并且可以进行连接控制的设备，上述的智能设备只是本实施例的示例，本实施例对此不做限定。
基于上述步骤S102至S106公开的方案中，实时检测并抓取终端屏幕发生变化时来自多个终端中的每个终端的用于展示该终端的屏幕显示内容的多张图片，发送该图片并使用该图片在一个界面中展示上述每个终端屏幕显示的内容，从而使得在终端屏幕发生变化时对终端屏幕上显示的内容进行抓取，并对抓取到的图片进行展示。一种可选的实施例中，执行抓取图片和展示图片的操作可以发生在同一个装置上，也可以发生在不同的装置上。
一种可选的实施例中，上述抓取的图片可以但不限于是终端屏幕数据的关键帧，在一个界面上显示了多个终端的屏幕，显示的这些终端的屏幕是关键帧，而不是流媒体，将多个终端的关键帧抓取为图片，通过图片的传输展示终端的屏幕。
由上可知，在本实施例中，抓取多个终端中的每个终端的屏幕发生变化时的图片，发送并使用该图片在一个界面中展示上述多个终端中每个终端屏幕显示的内容，从而在展示终端屏幕时只有在被展示的终端的屏幕发生变化时传输用来表示屏幕变化的图片，使得用于控制上述多个终端的连接不必实时进行屏幕数据的传输，减少了屏幕显示过程对带宽的占用。
通过本申请上述实施例公开的方案，达到了在一个界面上展示多个终端屏幕显示的内容，并抓取终端屏幕发生变化时的图片的目的，从而实现了节约了显示终端屏幕时带宽的技术效果。由此，本申请上述实施例解决了相关技术显示终端屏幕时浪费带宽的技术问题。
在本实施例中，可以但不限于根据界面上显示的终端的数量来抓取每个终端的图片，在本实施例中提供了一种可选的实施方式，该可选的实施方式可以包括如下步骤：
步骤202，获取在一个界面中显示的终端的数量；
步骤204，根据终端的数量设置分辨率；
步骤206，根据分辨率抓取每个终端的图片，或者，抓取每个终端的图片之后根据分辨率进行转换。
在上述可选的实施方式中，为了使得抓取到的图片能够适应展示图片的界面，可以根据界面上显示的终端的数量为抓取的图片设置分辨率，并可以通过以下两种方式之一获取到该分辨率的图片：方式一，按照上述分辨率抓取终端的图片；方式二，抓取每个终端的图片，再按照上述分辨率对图片进行转换。
在一个可选的实施方式中，显示图片的分辨率可以根据控制侧的带宽状况进行设置，当网络状况不好的时候，可以设置较低的分辨率对图片进行显示。在终端上抓取图片时的分辨率可以是根据终端本身的分辨率设定的，在网络状况不好的情况下，在控制侧可以使用较低的分辨率来显示，以便适应网络的状况，从而使得终端屏幕显示时的效率更高。
在一个可选的实施方式中，图片抓取之后，可以根据需要处理成相应分辨率的图片，例如，可以根据在一个页面上显示的终端的数量来确定图片处理后的大小；还可以根据带宽的大小来确定图片处理后的大小。
例如：如果要在一个分辨率为1920×1080像素的界面上显示9个终端，可以将终端的分辨率设置为640×360像素，再抓取分辨率为640×360像素的每个终端的图片，或者，抓取每个终端的图片后将抓取到的图片的分辨率转换为640×360像素。
在一个可选的实施方式中，抓取图片时的图片分辨率可以但不限于为一个固定值。
在一个可选的实施方式中，抓取的图片的分辨率可以是固定的(比如：1920x1080)，而在对抓取的图片进行显示时，显示的图片的分辨率可以是根据当前的网络状况进行设置的。比如：可以根据当前的数据传输速率对显示的图片的分辨率进行设置。例如：获取当前网络的数据传输速率；根据数据传输速率为显示的图片设置分辨率；根据设置的分辨率显示图片。
通过上述实施例，实现了通过界面显示的终端数量来抓取每个终端的图片的目的。
在抓取每个终端图片的过程中，可以根据前后抓取的两张图片的变化来确定在界 面上展示的图片，在一种可选的实施例中，上述方法还可以包括如下步骤：
步骤302，按照预定时间间隔获取每个终端的图片；
步骤304，判断获取到的相邻的两张图片的变化是否超过阈值，在超过阈值的情况下，将两张图片中的后抓取的一张图片用作在界面中展示的图片。
例如：每隔2秒获取每个终端的图片，并分别判断每个终端前后获取的两张图片的变化是否超过了预先设置的阈值，对于变化超过阈值的终端，将后抓取的图片作为在界面中展示的图片，对于变化未超过阈值的图片，可以保持当前界面上展示的图片不变。
在本实施例中，将上述终端屏幕显示处理方法应用于应用程序的测试场景中时，对前后抓取的相邻的图片进行对比有助于发现应用程序运行中的问题，可以根据应用程序在设备上运行的阶段：安装，启动，验证，运行，卸载等对屏幕图片进行截取，从而对应用程序的各阶段进行测试。
通过上述步骤，按照预定的时间间隔来抓取每个终端的图片，根据前后抓取的两张图片的变化来确定在界面上展示的图片，从而节约了终端的屏幕展示时占用的带宽。
为了实现根据图片的变化确定界面上展示的图片的功能，在一种可选的实施例中，判断获取到的两张图片的变化是否超过阈值可以包括如下步骤：
步骤402，获取两张图片中的每张图片的像素点；
步骤404，比较两张图片中的每个像素点是否发生变化，记录发生变化的像素点；
步骤406，判断发生变化的像素点的数量是否超过阈值。
在本可选实施例中，可以根据两张图片中像素点的变化来确定图片的变化，从而确定界面上展示的图片。例如：获取两张图片中的每张图片的像素点，并比较两张图片中的每个像素点是否发生变化，记录发生变化的像素点，得到发生变化的像素点的数量为15000个，而预先设置的阈值为10000个像素点，那么当前发生变化的像素点的数量已经超过了该阈值，从而判断出获取到的两张图片的变化已经超过了阈值，可以将两张图片中的后抓取的一张图片用作在界面中展示的图片。
在一个可选的实施方式中，还可以通过以下方式抓取图片：按照预设时间获取每个终端的图片，将当前抓取的一张图片用作在界面中展示的图片。
可选地，上述预设时间可以但不限于是用户选择的或者是由系统设定的。
可选地，在本实施方式中，在抓取图片之前，可以预先设定抓取图片的时间间隔 以及预先设定抓取图片的数量，在抓取图片时，则按照预先设定的抓取图片的时间间隔抓取图片，当抓取的图片数量达到预先设定的抓取图片的数量时停止抓取图片。例如：在抓取图片之前，预先设定抓取图片的时间间隔为5s以及预先设定抓取图片的数量为20张，在抓取图片时，每隔5s抓取一张图片，当抓取了20张图片时则停止抓取。
可选地，在本实施方式中，在抓取图片之前，还可以判断已抓取的图片的数量是否已达到了预设数量，如果未达到预设数量，则获取每个终端的图片。如果已经达到了预设数量，则停止抓取图片。
可选地，上述预设数量可以但不限于是用户选择的或者是由系统设定的。
在一个可选的实施方式中，在抓取所述图片和使用所述图片进行展示不在同一装置的情况下，获取抓取图片的装置与使用图片进行展示的装置之间的网络状况，并根据网络状况对抓取到的图片进行处理，再发送处理后的图片。其中，上述网络状况可以但不限于包括以下至少之一：丢包率、带宽。
为了实现上述目的，根据本发明的另一方面，本发明实施例还提供了一种存储介质，所述存储介质包括存储的程序，其中，在所述程序运行时控制所述存储介质所在设备执行上述所述的终端屏幕显示处理方法。
为了实现上述目的，根据本发明的另一方面，本发明实施例还提供了一种处理器，所述处理器用于运行程序，其中，所述程序运行时执行上述所述的终端屏幕显示处理方法。
根据本发明实施例，还提供了一种终端控制的方法实施例，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
目前在移动终端(或称为移动设备)上使用的操作系统比较多的是微软的WP操作系统、安卓系统、和苹果公司的iOS系统，除了这些操作系统以外可能还有其他类型的操作系统。
在计算上(例如，个人电脑PC)上也可以使用多种操作系统，例如，Windows操作系统、苹果公司的操作系统、linux操作系统或者Unix操作系统等。
目前有多种对于终端的控制方式，这些控制方式对于受控终端的处理方法不同，例如，一种控制方式是在控制界面上显示一个被控终端的界面，由于被控终端可能有很多，对于这种控制方法，只能对单一的被控终端进行控制。
但是，对于存在大量被控终端的情况来说，这种方法的操作效率就相当的低。另外，对于界面显示的方式也存在一些弊端，例如，用户使用的主控终端和被控终端的屏幕大小可能并不相同，甚至差距很大，当用户打开被控终端的界面时，会发现该界面充满了主控终端的屏幕，在某种情况下会产生不舒服的感觉，这样会影响用户的体验度。
基于上述考虑，本实施例从终端控制中的控制方的角度来对终端的控制方法进行说明。本实施例中涉及到的技术方案，可以做成一个应用放在应用市场中，用户在使用的时候下载该应用并进行安装。或者，如果为了推广某种操作系统，也可以将该功能内置在操作系统中，作为该操作系统的一个卖点，增加操作系统的装机量。
此外，本实施例中涉及到的技术方案，还可以做成一个网站，用户可以在使用浏览器或者使用其他应用时访问该网站，从而在网页上实现对终端的控制。
在上述运行环境下，本实施例提供了如图2所示的一种终端控制方法。图2是根据本发明实施例的一种终端控制方法的流程图，如图2所示，包括如下步骤：
步骤S202，获取来自多个终端中的每个终端的多媒体数据，其中，每个终端的多媒体数据用于展示终端的屏幕显示的内容；
步骤S204，使用多媒体数据在一个界面中展示多个终端中每个终端屏幕显示的内容；
步骤S206，接收用户通过界面输入的操作信号；
步骤S208，获取操作信号对应的第一终端；
步骤S210，将操作信号转换成对应的指令并发送给第一终端。
在上述步骤中，可以首先由用户选择需要被控制的终端，一般情况下现在的终端的运行是可以对网络进行访问的。在上述步骤中，可以通过网络连接被控制的终端。
上述步骤中描述了该控制应用对多个终端的获取连接的操作，该控制应用进行了多个终端的获取和连接等操作。如果上述步骤中的内容作为一个功能被内置到了操作系统中，则是由该功能进行多个终端的获取和连接等操作。
作为一种可选的实施例，可以使用一台服务器来连接不同操作系统的手机，例如，可以通过一个Mac Mini连接安卓手机和iOS手机。或者，终端可以通过USB线与hub直连，hub与服务器连接，从而实现终端与服务器的连接。
作为一个可选的实施方式，可以为该控制应用的启用或者操作系统该功能的使用 设置授权，例如，该控制应用启动的使用可以要求用户输入密码或者使用生物信息对用户进行识别，如果用户密码输入正确或者识别出是被授权用户才允许使用该功能。生物信息的识别目前被广泛使用的是指纹识别，当然，其他的生物识别技术也可以被使用，例如，虹膜识别、面部识别、声纹识别等等。
作为一个可选的实施方式，用户可以通过购买来获取终端控制功能，购买的价格可以根据可控制的终端的数量确定，可控制的终端的数量越多价格越高，用户购买后可以为用户提供许可证(License)来作为购买的凭证。
对于终端控制在一般的操作系统中均是支持的，例如，即使作为限制最严格的iOS系统也是支持的，对于其他比较开放的系统，例如，安卓系统或者Linux系统等其也操作系统也都支持。因此上述步骤中涉及到的技术方案可支持的操作系统更加广泛。
作为一种可选的实施例，上述多个终端可以为电脑、笔记本、平板电脑、手机、智能穿戴设备等智能终端设备，也可以是空调、冰箱、洗衣机、电视等智能家居设备。需要说明的是，上述智能终端设备和智能家居设备等智能设备指具有计算处理能力并且可以进行连接控制的设备，上述的智能设备只是本实施例的示例，本实施例对此不做限定。
基于上述步骤S202至S210公开的方案中，实时检测并获取来自多个终端中的每个终端的多媒体数据并使用该多媒体数据在一个界面中展示上述多个终端中每个终端屏幕显示的内容，其中，每个终端的多媒体数据用于展示相应终端的屏幕显示的内容，一种可选的实施例中，获取到的上述多媒体数据可以但不限于是由上述多个终端将其屏幕上显示的内容转换并发送的，或者，获取到的上述多媒体数据还可以但不限于是接收到上述多个终端发送的用于展示屏幕上显示的内容的数据后再对这些数据进行转换得到的。
一种可选的实施例中，上述操作信号可以但不限于是用户对屏幕的触摸产生的信号，还可以但不限于是用户通过一些输入设备输入的信号。上述输入设备可以但不限于包括：鼠标、键盘、触控板等等。例如：以用户对屏幕的触摸为例，可以接收到用户触摸界面产生的信号，再将该信号转换成操作信号。
由上可知，在本实施例中，获取来自多个终端中的每个终端的用于展示屏幕显示的内容的多媒体数据，并使用获取到的多媒体数据在一个界面中展示这多个终端的屏幕显示的内容，从而实现在一个界面上对多个终端屏幕的显示，接收用户通过界面输入的操作信号并获取该操作信号对应的第一终端，再将操作信号转换成对应的指令，将该指令发送给第一终端，从而实现对界面上显示的终端的控制。
通过本申请上述实施例公开的方案，达到了在一个界面上显示多个终端，并向需要控制的终端发送指令的目的，从而实现了提高了终端控制的控制效率的技术效果。由此，本申请上述实施例解决了相关技术终端控制的控制效率低的技术问题。
在本实施例中，可以但不限于根据操作信号在界面上的位置以及界面上显示的每个终端的位置来获取第一终端，在本实施例中提供了一种可选的实施方式，该可选的实施方式可以包括如下步骤：
步骤502，获取操作信号在界面中输入的第一位置信息；
步骤504，获取每个终端在界面中显示的第二位置信息，其中，第二位置信息为该终端在界面中所占的范围；
步骤506，根据第一位置信息和第二位置信息确定操作信号对应的第一终端。
在上述可选的实施方式中，上述第一位置信息和上述第二位置信息可以但不限于用坐标值来表示，或者还可以但不限于用像素点的位置信息来表示，例如，可以用在界面上产生操作信号的区域的中心来代表该操作信号，获取该中心在界面的坐标系中的坐标值作为上述第一位置信息，而第二位置信息则可以是终端在界面上显示的区域的边界对应的坐标区间，通过判断第一位置信息对应的坐标值落入的第二位置信息对应的坐标区间来确定操作信号对应的第一终端。
在一个可选的实施方式中，可以根据每个终端在界面中所占的范围将第一位置信息落入的范围对应的终端作为第一终端，其中，第一终端包括一个或多个终端。例如：可以将界面预先划分为多个区域，每一个区域显示一个终端，通过判断操作信号落入的区域来获取第一终端。图3是根据本发明实施例的一种优选的获取第一终端的示意图，如图3所示，界面被划分成9个区域，分别显示了终端1至终端9的屏幕，获取到操作信号在界面中输入的第一位置信息落入了终端4所在的区域，则可以确定终端4为第一终端。
通过上述实施例，实现了通过操作信号和终端的位置信息来获取待被控制的第一终端的目的。
在本实施例中，当第一终端的数量为多个时，还可以实现用户输入的操作信号的多屏同步的功能。也就是说，当第一终端的数量为多个时，用户输入的操作信号将被转换为对应的指令发送给这多个终端，从而控制多个终端执行相同的操作。
例如：在进行应用程序测试时，用户登录DevicePass测试平台后，可同时控制多台测试手机进行应用程序的测试。用户选择多屏控制功能后，在其中一台测试手机发 起的任何操作行为，被选中的测试手机将保持同步操作。
可选地，在本实施例中，可以对iOS和Android测试手机同时多屏同步操作。Android测试手机多屏操作同步响应时间可以控制在0.1秒内。iOS测试手机多屏操作同步响应时间可以控制在0.5秒内。
可选地，在本实施例中，在进行多屏同步控制时，可以隐藏、关闭其中任意同步操作的测试手机屏幕。
可选地，在本实施例中，可以进行被测应用程序的多屏同步点击、滑动、文本输入等测试操作。也可以对被测应用程序进行多屏同步截图、横屏、竖屏、返回上一步、返回主屏幕等功能操作。
作为一个可选的实施方式，可以将待控制的第一终端的屏幕放大为全屏显示。例如：在进行应用程序测试的过程中，用户登录DevicePass测试平台后，可以对多个终端进行控制，在用户需要对其中的一个测试终端进行控屏操作时，用户可以选择该测试终端并对该测试终端屏幕进行全屏放大操作。
可选地，在本实施例中，可以对iOS和Android测试手机单台设备屏幕全屏放大。
可选地，在本实施例中，在进行终端屏幕的全屏放大时，可以对控制设备进行点击、滑动等操作来放大被控终端的屏幕。例如：全屏放大操作可以通过双击，双手指拖动或其他手势进行识别。
可选地，在本实施例中，在进行终端屏幕的全屏放大时，支持对被测应用程序的点击、滑动、文本输入等测试操作。
可选地，在本实施例中，在进行终端屏幕的全屏放大时，可以对被测应用程序进行截图、横屏、竖屏、返回上一步、返回主屏幕等功能操作。
可选地，在本实施例中，在进行终端屏幕的全屏放大时，可以点击键盘ESC键快速退出全屏放大模式。还可以手动关闭全屏放大模式。
在指令的转换过程中，可以根据第一终端的操作系统的类型或者第一终端上的应用的类型将操作信号转换成其对应的指令，在一种可选的实施例中，上述方法还可以包括如下步骤：
步骤602，根据预先获取到的第一终端的操作系统的类型或者第一终端上的应用的类型将操作信号转换成操作系统的类型或者应用的类型对应的指令；
步骤604，将指令发送给第一终端。
例如：预先获取到的第一终端的操作系统的类型为安卓系统，将操作信号转换成安卓系统对应的指令，并将该指令发送给第一终端。或者，预先获取到的第一终端上的应用的类型为即时通讯类应用，则将操作信号转换成即时通讯类应用对应的指令，并将该指令发送给第一终端。
通过上述步骤，如果根据预先获取到的第一终端的操作系统的类型进行指令的转换，则将操作信号转换成操作系统的类型对应的指令，如果根据第一终端上的应用的类型对操作信号进行转换，则将操作信号转换成应用的类型对应的指令，从而使得操作信号的转换能够适用于各种操作系统或者应用。
为了实现指令的转换和发送的功能，在一种可选的实施例中，上述方法还可以包括如下步骤：
步骤702，将操作信号转换为对第一终端的操作信号；
步骤704，将对第一终端的操作信号发送给第一终端，其中，对第一终端的操作信号作为对第一终端的输入。
例如：操作信号为在第一终端打开的浏览器应用上的搜索栏输入要搜索的内容“车展”，并进行搜索。那么，将操作信号转换为对第一终端的操作信号，对第一终端的操作信号可以是在浏览器的搜索栏中输入“车展”，并点击搜索按钮。再将该对第一终端的操作信号发送给第一终端，从而控制第一终端执行上述操作。
通过上述步骤，将操作信号转换成作为对第一终端的输入的对第一终端的操作信号，可以实现在界面上对第一终端进行输入，从而使得对终端的控制更加的便捷，提高用户的体验。
在一个可选的实施方式中，可以但不限于在界面上预先设置与显示的终端的屏幕内容不重合的区域来接收用户输入的操作信号。例如：可以但不限于通过以下方式接收用户通过界面输入的操作信号：接收用户在预先设置的区域中输入的操作信号，其中，区域与显示每个终端的屏幕内容的区域不重叠。
在本可选的实施方式中，图4是根据本发明实施例的一种优选的预先设置的区域的示意图，如图4所示，可以将界面划分为两个区域，界面上方的区域用来显示终端的屏幕，界面下方的区域可以作为上述预先设置的区域用来接收操作信号。
在一个可选的实施方式中，可以但不限于在界面上预先设置用于选择第一终端的选择控件，通过接收用户通过选择控件选择出的终端，其中，选择出的终端作为第一终端来获取第一终端，并将操作信号发送给第一终端。
可选地，在上述实施方式中，选择控件可以但不限于提供以下之一的选项：单个终端的选项、一组终端的选项、全部终端的选项。也就是说，可以通过选择控件单个或者批量控制上述多个终端。
作为一种可选的实施方式，图5是根据本发明实施例的一种优选的选择控件的示意图，如图5所示，在上述预先设置的区域中可以设置有选择控件，该选择控件用于选择第一终端。
为了实现上述目的，根据本发明的另一方面，本发明实施例还提供了一种存储介质，所述存储介质包括存储的程序，其中，在所述程序运行时执行上述所述的终端控制方法。
为了实现上述目的，根据本发明的另一方面，本发明实施例还提供了一种处理器，所述处理器设置为运行程序，其中，所述程序运行时执行上述所述的终端控制方法。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明并不受所描述的动作顺序的限制，因为依据本发明，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本发明所必须的。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的应用限制方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本发明各个实施例所述的方法。
实施例2
根据本发明实施例，还提供了一种设置为实施上述终端屏幕显示处理方法的装置实施例，图6是根据本发明实施例的一种终端屏幕显示处理装置示意图，如图6所示，该装置包括抓取模块61、发送模块63和第一展示模块65。
其中，抓取模块61，设置为抓取多个终端中的每个终端的图片，其中，每个终端的图片用于展示终端的屏幕显示的内容，每个终端的图片包括多张图片，多张图片为该终端屏幕发生变化时的图片；发送模块63，设置为发送图片；第一展示模块65，设置为使用图片在一个界面中展示多个终端中的每个终端屏幕显示的内容。
此处需要说明的是，上述抓取模块61、发送模块63和第一展示模块65可以对应于实施例1中的步骤S102至步骤S106，三个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。
由上可知，在本实施例中，抓取多个终端中的每个终端的屏幕发生变化时的图片，发送并使用该图片在一个界面中展示上述多个终端中每个终端屏幕显示的内容，从而在展示终端屏幕时只有在被展示的终端的屏幕发生变化时传输用来表示屏幕变化的图片，使得用于控制上述多个终端的连接不必实时进行屏幕数据的传输，减少了屏幕显示过程对带宽的占用。
通过本申请上述实施例公开的方案，达到了在一个界面上展示多个终端屏幕显示的内容，并抓取终端屏幕发生变化时的图片的目的，从而实现了节约了显示终端屏幕时带宽的技术效果。由此，本申请上述实施例解决了相关技术显示终端屏幕时浪费带宽的技术问题。
在一种可选的实施例中，上述抓取模块包括：第一获取单元，设置为获取在一个界面中显示的终端的数量；设置单元，设置为根据终端的数量设置分辨率；抓取单元，设置为根据分辨率抓取每个终端的图片，或者，抓取每个终端的图片之后根据分辨率进行转换。
此处需要说明的是，上述第一获取单元、设置单元和抓取单元可以对应于实施例1中的步骤202至步骤206，三个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。
在一种可选的实施例中，上述抓取单元包括：第二获取单元，设置为按照预定时间间隔获取每个终端的图片；处理单元，设置为判断获取到的相邻的两张图片的变化是否超过阈值，在超过阈值的情况下，将两张图片中的后抓取的一张图片用作在界面中展示的图片。
此处需要说明的是，上述第二获取单元和处理单元可以对应于实施例1中的步骤302至步骤304，两个个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。
在一种可选的实施例中，上述处理单元设置为：获取两张图片中的每张图片的像素点；比较两张图片中的每个像素点是否发生变化，记录发生变化的像素点；判断发生变化的像素点的数量是否超过阈值。
此处需要说明的是，上述处理单元可以对应于实施例1中的步骤402至步骤406，该模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的 内容。
在一种可选的实施例中，在抓取图片和使用图片进行展示不在同一装置的情况下，上述发送模块设置为：获取抓取图片的装置与使用图片进行展示的装置之间的网络状况；根据网络状况对抓取到的图片进行处理；发送处理后的图片。
在一种可选的实施例中，上述网络状况包括以下至少之一：丢包率、带宽。
根据本发明实施例，还提供了一种设置为实施上述终端控制方法的装置实施例，图7是根据本发明实施例的一种终端控制装置示意图，如图7所示，该装置包括第一获取模块71、第二展示模块73、接收模块75、第二获取模块77和处理模块79。
其中，第一获取模块71，设置为获取来自多个终端中的每个终端的多媒体数据，其中，每个终端的多媒体数据用于展示终端的屏幕显示的内容；第二展示模块73，设置为使用多媒体数据在一个界面中展示多个终端中每个终端屏幕显示的内容；接收模块75，设置为接收用户通过界面输入的操作信号；第二获取模块77，设置为获取操作信号对应的第一终端；处理模块79，设置为将操作信号转换成对应的指令并发送给第一终端。
此处需要说明的是，上述第一获取模块71、第二展示模块73、接收模块75、第二获取模块77和处理模块79可以对应于实施例1中的步骤S202至步骤S210，三个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。
由上可知，在本实施例中，获取来自多个终端中的每个终端的用于展示屏幕显示的内容的多媒体数据，并使用获取到的多媒体数据在一个界面中展示这多个终端的屏幕显示的内容，从而实现在一个界面上对多个终端屏幕的显示，接收用户通过界面输入的操作信号并获取该操作信号对应的第一终端，再将操作信号转换成对应的指令，将该指令发送给第一终端，从而实现对界面上显示的终端的控制。
通过本申请上述实施例公开的方案，达到了在一个界面上显示多个终端，并向需要控制的终端发送指令的目的，从而实现了提高了终端控制的控制效率的技术效果。由此，本申请上述实施例解决了相关技术终端控制的控制效率低的技术问题。
在一种可选的实施例中，上述第二获取模块77包括：第一获取单元，设置为获取所述操作信号在所述界面中输入的第一位置信息；第二获取单元，设置为获取每个终端在所述界面中显示的第二位置信息，其中，所述第二位置信息为该终端在所述界面中所占的范围；确定单元，设置为根据所述第一位置信息和所述第二位置信息确定所述操作信号对应的所述第一终端。
此处需要说明的是，上述第一获取单元、第二获取单元和确定单元可以对应于实施例1中的步骤502至步骤506，三个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。
在一个可选的实施方式中，上述确定单元可以根据每个终端在界面中所占的范围将第一位置信息落入的范围对应的终端作为第一终端，其中，第一终端包括一个或多个终端。
在一种可选的实施例中，上述处理模块79设置为：根据预先获取到的第一终端的操作系统的类型或者第一终端上的应用的类型将操作信号转换成操作系统的类型或者应用的类型对应的指令；将指令发送给第一终端。
此处需要说明的是，上述处理模块还可以对应于实施例1中的步骤602至步骤604，该模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。
在一种可选的实施例中，上述处理模块还设置为：将操作信号转换为对第一终端的操作信号；将对第一终端的操作信号发送给第一终端，其中，对第一终端的操作信号作为对第一终端的输入。
此处需要说明的是，上述处理模块还可以对应于实施例1中的步骤702至步骤704，该模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。
在一种可选的实施例中，上述接收模块设置为：接收用户在预先设置的区域中输入的操作信号，其中，区域与显示每个终端的屏幕内容的区域不重叠；上述第二获取模块设置为：接收用户通过选择控件选择出的终端，其中，选择出的终端作为第一终端；将操作信号发送给第一终端。
在一种可选的实施例中，所述选择控件提供以下之一的选项：单个终端的选项、一组终端的选项、全部终端的选项。
实施例3
本发明的实施例可以提供一种计算机终端，该计算机终端可以是计算机终端群中的任意一个计算机终端设备。可选地，在本实施例中，上述计算机终端也可以替换为移动终端等终端设备。
可选地，在本实施例中，上述计算机终端可以位于计算机网络的多个网络设备中的至少一个网络设备。
在本实施例中，上述计算机终端可以执行应用程序的应用限制方法中以下步骤的程序代码：抓取多个终端中的每个终端的图片，其中，每个终端的图片用于展示终端的屏幕显示的内容，每个终端的图片包括多张图片，多张图片为该终端屏幕发生变化时的图片；发送图片；使用图片在一个界面中展示多个终端中的每个终端屏幕显示的内容。
在本实施例中，上述计算机终端可以执行应用程序的数据发送处理方法中以下步骤的程序代码：获取在一个界面中显示的终端的数量；根据终端的数量设置分辨率；根据分辨率抓取每个终端的图片，或者，抓取每个终端的图片之后根据分辨率进行转换。
可选地，图8示出了一种可选的计算机终端的硬件结构框图一，如图8所示，计算机终端8可以包括一个或多个(图中采用81a、81b，……，81n来示出)处理器81(处理器81可以包括但不限于微处理器MCU或可编程逻辑器件FPGA等的处理装置)、设置为存储数据的存储器83以及设置为通信功能的传输装置85。除此以外，还可以包括：显示器、输入/输出接口(I/O接口)、通用串行总线(USB)端口(可以作为I/O接口的端口中的一个端口被包括)、网络接口、电源和/或相机。本领域普通技术人员可以理解，图8所示的结构仅为示意，其并不对上述电子装置的结构造成限定。例如，计算机终端8还可包括比图8中所示更多或者更少的组件，或者具有与图8所示不同的配置。
应当注意到的是上述一个或多个处理器81和/或其他数据处理电路在本文中通常可以被称为“数据处理电路”。该数据处理电路可以全部或部分的体现为软件、硬件、固件或其他任意组合。此外，数据处理电路可为单个独立的处理模块，或全部或部分的结合到计算机终端8中的其他元件中的任意一个内。如本申请实施例中所涉及到的，该数据处理电路作为一种处理器控制(例如与接口连接的可变电阻终端路径的选择)。
存储器83可设置为存储应用软件的软件程序以及模块，如本发明实施例中的数据发送处理方法对应的程序指令/数据存储装置，处理器81通过运行存储在存储器83内的软件程序以及模块，从而执行各种功能应用以及数据处理，即实现上述的应用程序的数据发送处理方法。存储器83可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器83可进一步包括相对于处理器81远程设置的存储器，这些远程存储器可以通过网络连接至计算机终端8。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
传输装置85设置为经由一个网络接收或者发送数据。上述的网络具体实例可包括 计算机终端8的通信供应商提供的无线网络。在一个实例中，传输装置85包括一个网络适配器(Network Interface Controller，NIC)，其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中，传输装置85可以为射频(Radio Frequency，RF)模块，其设置为通过无线方式与互联网进行通讯。
显示器可以例如触摸屏式的液晶显示器(LCD)，该液晶显示器可使得用户能够与计算机终端8的用户界面进行交互。
此处需要说明的是，在一些可选实施例中，上述图8所示的计算机终端8可以包括硬件元件(包括电路)、软件元件(包括存储在计算机可读介质上的计算机代码)、或硬件元件和软件元件两者的结合。应当指出的是，图8仅为特定具体实例的一个实例，并且旨在示出可存在于上述计算机终端中的部件的类型。
此处需要说明的是，在一些实施例中，上述图8所示的计算机终端8具有触摸显示器(也被称为“触摸屏”或“触摸显示屏”)。在一些实施例中，上述图8所示的计算机终端8具有图像用户界面(GUI)，用户可以通过触摸触敏表面上的手指接触和/或手势来与GUI进行人机交互，此处的人机交互功能可选的包括如下交互：创建网页、绘图、文字处理、制作电子文档、游戏、视频会议、即时通信、收发电子邮件、通话界面、播放数字视频、播放数字音乐和/或网络浏览等、用于执行上述人机交互功能的可执行指令被配置/存储在一个或多个处理器可执行的计算机程序产品或可读存储介质中。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：按照预定时间间隔获取每个终端的图片；判断获取到的相邻的两张图片的变化是否超过阈值，在超过阈值的情况下，将两张图片中的后抓取的一张图片用作在界面中展示的图片。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：获取两张图片中的每张图片的像素点；比较两张图片中的每个像素点是否发生变化，记录发生变化的像素点；判断发生变化的像素点的数量是否超过阈值。
可选的，上述处理器还可以执行如下步骤的程序代码：获取抓取图片的装置与使用图片进行展示的装置之间的网络状况；根据网络状况对抓取到的图片进行处理；发送处理后的图片。
本领域普通技术人员可以理解，图8所示的结构仅为示意，计算机终端也可以是智能手机(如Android手机、iOS手机等)、平板电脑、掌声电脑以及移动互联网设备(Mobile Internet Devices，MID)、PAD等终端设备。图8其并不对上述电子装置的结构造成限定。例如，计算机终端8还可包括比图8中所示更多或者更少的组件(如网 络接口、显示装置等)，或者具有与图8所示不同的配置。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令终端设备相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：闪存盘、只读存储器(Read-Only Memory，ROM)、随机存取器(Random Access Memory，RAM)、磁盘或光盘等。
本发明的实施例可以提供一种计算机终端，该计算机终端可以是计算机终端群中的任意一个计算机终端设备。可选地，在本实施例中，上述计算机终端也可以替换为移动终端等终端设备。
可选地，在本实施例中，上述计算机终端可以位于计算机网络的多个网络设备中的至少一个网络设备。
在本实施例中，上述计算机终端可以执行应用程序的应用限制方法中以下步骤的程序代码：获取来自多个终端中的每个终端的多媒体数据，其中，每个终端的多媒体数据用于展示终端的屏幕显示的内容；使用多媒体数据在一个界面中展示多个终端中每个终端屏幕显示的内容；接收用户通过界面输入的操作信号；获取操作信号对应的第一终端；将操作信号转换成对应的指令并发送给第一终端。
在本实施例中，上述计算机终端可以执行应用程序的数据发送处理方法中以下步骤的程序代码：获取操作信号在界面中输入的第一位置信息；获取每个终端在界面中显示的第二位置信息，其中，第二位置信息为该终端在界面中所占的范围；根据第一位置信息和第二位置信息确定操作信号对应的第一终端。
可选地，图9示出了一种可选的计算机终端的硬件结构框图二，如图9所示，计算机终端9可以包括一个或多个(图中采用91a、91b，……，91n来示出)处理器91(处理器91可以包括但不限于微处理器MCU或可编程逻辑器件FPGA等的处理装置)、设置为存储数据的存储器93以及设置为通信功能的传输装置95。除此以外，还可以包括：显示器、输入/输出接口(I/O接口)、通用串行总线(USB)端口(可以作为I/O接口的端口中的一个端口被包括)、网络接口、电源和/或相机。本领域普通技术人员可以理解，图9所示的结构仅为示意，其并不对上述电子装置的结构造成限定。例如，计算机终端9还可包括比图9中所示更多或者更少的组件，或者具有与图9所示不同的配置。
应当注意到的是上述一个或多个处理器91和/或其他数据处理电路在本文中通常可以被称为“数据处理电路”。该数据处理电路可以全部或部分的体现为软件、硬件、固件或其他任意组合。此外，数据处理电路可为单个独立的处理模块，或全部或部分 的结合到计算机终端9中的其他元件中的任意一个内。如本申请实施例中所涉及到的，该数据处理电路作为一种处理器控制(例如与接口连接的可变电阻终端路径的选择)。
存储器93可设置为存储应用软件的软件程序以及模块，如本发明实施例中的数据发送处理方法对应的程序指令/数据存储装置，处理器91通过运行存储在存储器93内的软件程序以及模块，从而执行各种功能应用以及数据处理，即实现上述的应用程序的数据发送处理方法。存储器93可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器93可进一步包括相对于处理器91远程设置的存储器，这些远程存储器可以通过网络连接至计算机终端9。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
传输装置95设置为经由一个网络接收或者发送数据。上述的网络具体实例可包括计算机终端9的通信供应商提供的无线网络。在一个实例中，传输装置95包括一个网络适配器(Network Interface Controller，NIC)，其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中，传输装置95可以为射频(Radio Frequency，RF)模块，其用于通过无线方式与互联网进行通讯。
显示器可以例如触摸屏式的液晶显示器(LCD)，该液晶显示器可使得用户能够与计算机终端9的用户界面进行交互。
此处需要说明的是，在一些可选实施例中，上述图9所示的计算机终端9可以包括硬件元件(包括电路)、软件元件(包括存储在计算机可读介质上的计算机代码)、或硬件元件和软件元件两者的结合。应当指出的是，图9仅为特定具体实例的一个实例，并且旨在示出可存在于上述计算机终端中的部件的类型。
此处需要说明的是，在一些实施例中，上述图9所示的计算机终端9具有触摸显示器(也被称为“触摸屏”或“触摸显示屏”)。在一些实施例中，上述图9所示的计算机终端9具有图像用户界面(GUI)，用户可以通过触摸触敏表面上的手指接触和/或手势来与GUI进行人机交互，此处的人机交互功能可选的包括如下交互：创建网页、绘图、文字处理、制作电子文档、游戏、视频会议、即时通信、收发电子邮件、通话界面、播放数字视频、播放数字音乐和/或网络浏览等、用于执行上述人机交互功能的可执行指令被配置/存储在一个或多个处理器可执行的计算机程序产品或可读存储介质中。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：根据每个终端在界面中所占的范围将第一位置信息落入的范围对应的终端作为第一终端，其中，第一终端包括一个或多个终端。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：根据预先获取到的第一终端的操作系统的类型或者第一终端上的应用的类型将操作信号转换成操作系统的类型或者应用的类型对应的指令；将指令发送给第一终端。
可选的，上述处理器还可以执行如下步骤的程序代码：将操作信号转换为对第一终端的操作信号；将对第一终端的操作信号发送给第一终端，其中，对第一终端的操作信号作为对第一终端的输入。
可选的，上述处理器还可以执行如下步骤的程序代码：接收用户在预先设置的区域中输入的操作信号，其中，区域与显示每个终端的屏幕内容的区域不重叠；获取操作信号对应的第一终端包括：接收用户通过选择控件选择出的终端，其中，选择出的终端作为第一终端；将操作信号发送给第一终端。
可选的，选择控件提供以下之一的选项：单个终端的选项、一组终端的选项、全部终端的选项。
本领域普通技术人员可以理解，图9所示的结构仅为示意，计算机终端也可以是智能手机(如Android手机、iOS手机等)、平板电脑、掌声电脑以及移动互联网设备(Mobile Internet Devices，MID)、PAD等终端设备。图9其并不对上述电子装置的结构造成限定。例如，计算机终端9还可包括比图9中所示更多或者更少的组件(如网络接口、显示装置等)，或者具有与图9所示不同的配置。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令终端设备相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：闪存盘、只读存储器(Read-Only Memory，ROM)、随机存取器(Random Access Memory，RAM)、磁盘或光盘等。
实施例4
本发明的实施例还提供了一种存储介质。可选地，在本实施例中，上述存储介质可以设置为保存上述实施例1所提供的终端屏幕显示处理方法所执行的程序代码。
可选地，在本实施例中，上述存储介质可以位于计算机网络中计算机终端群中的任意一个计算机终端中，或者位于移动终端群中的任意一个移动终端中。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：抓取多个终端中的每个终端的图片，其中，每个终端的图片用于展示终端的屏幕显示的内容，每个终端的图片包括多张图片，多张图片为该终端屏幕发生变化时的图片；发送图片；使用图片在一个界面中展示多个终端中的每个终端屏幕显示的内容。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：获取在一个界面中显示的终端的数量；根据终端的数量设置分辨率；根据分辨率抓取每个终端的图片，或者，抓取每个终端的图片之后根据分辨率进行转换。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：按照预定时间间隔获取每个终端的图片；判断获取到的相邻的两张图片的变化是否超过阈值，在超过阈值的情况下，将两张图片中的后抓取的一张图片用作在界面中展示的图片。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：获取两张图片中的每张图片的像素点；比较两张图片中的每个像素点是否发生变化，记录发生变化的像素点；判断发生变化的像素点的数量是否超过阈值。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：获取抓取图片的装置与使用图片进行展示的装置之间的网络状况；根据网络状况对抓取到的图片进行处理；发送处理后的图片。
可选地，在本实施例中，网络状况包括以下至少之一：丢包率、带宽。
本发明的实施例还提供了一种存储介质。可选地，在本实施例中，上述存储介质可以用于保存上述实施例1所提供的终端控制方法所执行的程序代码。
可选地，在本实施例中，上述存储介质可以位于计算机网络中计算机终端群中的任意一个计算机终端中，或者位于移动终端群中的任意一个移动终端中。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：获取来自多个终端中的每个终端的多媒体数据，其中，每个终端的多媒体数据用于展示终端的屏幕显示的内容；使用多媒体数据在一个界面中展示多个终端中每个终端屏幕显示的内容；接收用户通过界面输入的操作信号；获取操作信号对应的第一终端；将操作信号转换成对应的指令并发送给第一终端。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：获取操作信号在界面中输入的第一位置信息；获取每个终端在界面中显示的第二位置信息，其中，第二位置信息为该终端在界面中所占的范围；根据第一位置信息和第二位置信息确定操作信号对应的第一终端。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：根据每个终端在界面中所占的范围将第一位置信息落入的范围对应的终端作为第一终端，其中，第一终端包括一个或多个终端。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：根据预先获取到的第一终端的操作系统的类型或者第一终端上的应用的类型将操作信号转换成操作系统的类型或者应用的类型对应的指令；将指令发送给第一终端。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：将操作信号转换为对第一终端的操作信号；将对第一终端的操作信号发送给第一终端，其中，对第一终端的操作信号作为对第一终端的输入。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：接收用户在预先设置的区域中输入的操作信号，其中，区域与显示每个终端的屏幕内容的区域不重叠；获取操作信号对应的第一终端包括：接收用户通过选择控件选择出的终端，其中，选择出的终端作为第一终端；将操作信号发送给第一终端。
可选地，在本实施例中，选择控件提供以下之一的选项：单个终端的选项、一组终端的选项、全部终端的选项。
上述本发明实施例序号仅仅为了描述，不代表实施例的优劣。
在本发明的上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。
在本申请所提供的几个实施例中，应该理解到，所揭露的技术内容，可通过其它的方式实现。其中，以上所描述的装置实施例仅仅是示意性的，例如所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，单元或模块的间接耦合或通信连接，可以是电性或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质 上或者说对相关技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。
工业实用性
本发明实施例提供的方案，可以应用于终端屏幕的显示过程中。通过本发明实施例解决了相关技术显示终端屏幕时浪费带宽的技术问题，节约了显示终端屏幕时带宽。

Claims (10)

  1. 一种终端屏幕显示处理方法，包括：
    抓取多个终端中的每个终端的图片，其中，所述每个终端的图片用于展示所述终端的屏幕显示的内容，所述每个终端的图片包括多张图片，所述多张图片为该终端屏幕发生变化时的图片；
    发送所述图片；
    使用所述图片在一个界面中展示所述多个终端中的每个终端屏幕显示的内容。
  2. 根据权利要求1所述的方法，其中，抓取所述每个终端的图片包括：
    获取在所述一个界面中显示的终端的数量；
    根据所述终端的数量设置分辨率；
    根据所述分辨率抓取所述每个终端的图片，或者，抓取所述每个终端的图片之后根据所述分辨率进行转换。
  3. 根据权利要求1所述的方法，其中，抓取所述每个终端的图片包括：
    按照预定时间间隔获取所述每个终端的图片；
    判断获取到的相邻的两张图片的变化是否超过阈值，在超过所述阈值的情况下，将所述两张图片中的后抓取的一张图片用作在所述界面中展示的图片。
  4. 根据权利要求3所述的方法，其中，判断获取到的所述两张图片的变化是否超过所述阈值包括：
    获取所述两张图片中的每张图片的像素点；
    比较所述两张图片中的每个像素点是否发生变化，记录发生变化的像素点；
    判断发生变化的像素点的数量是否超过所述阈值。
  5. 根据权利要求1所述的方法，其中，抓取所述每个终端的图片包括：
    按照预设时间获取所述每个终端的图片；
    将当前抓取的一张图片用作在所述界面中展示的图片。
  6. 根据权利要求1至5中任一项所述的方法，其中，在抓取所述图片和使用所述图片进行展示不在同一装置的情况下，发送所述图片包括：
    获取抓取所述图片的装置与使用所述图片进行展示的装置之间的网络状况；
    根据所述网络状况对抓取到的图片进行处理；
    发送处理后的所述图片。
  7. 根据权利要求6所述的方法，其中，所述网络状况包括以下至少之一：丢包率、带宽。
  8. 一种终端屏幕显示处理装置，包括：
    抓取模块，设置为抓取多个终端中的每个终端的图片，其中，所述每个终端的图片用于展示所述终端的屏幕显示的内容，所述每个终端的图片包括多张图片，所述多张图片为该终端屏幕发生变化时的图片；
    发送模块，设置为发送所述图片；
    第一展示模块，设置为使用所述图片在一个界面中展示所述多个终端中的每个终端屏幕显示的内容。
  9. 一种存储介质，所述存储介质包括存储的程序，其中，在所述程序运行时执行权利要求1至7中任意一项所述的终端屏幕显示处理方法。
  10. 一种处理器，所述处理器设置为运行程序，其中，所述程序运行时执行权利要求1至权利要求7中任意一项所述的终端屏幕显示处理方法。
PCT/CN2018/078425 2017-04-05 2018-03-08 终端屏幕显示处理方法、装置及存储介质 WO2018184443A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710217952.2A CN108694029A (zh) 2017-04-05 2017-04-05 终端屏幕显示处理方法、装置及存储介质
CN201710217952.2 2017-04-05

Publications (1)

Publication Number Publication Date
WO2018184443A1 true WO2018184443A1 (zh) 2018-10-11

Family

ID=63712246

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/078425 WO2018184443A1 (zh) 2017-04-05 2018-03-08 终端屏幕显示处理方法、装置及存储介质

Country Status (2)

Country Link
CN (1) CN108694029A (zh)
WO (1) WO2018184443A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109617859B (zh) * 2018-11-13 2021-12-03 视联动力信息技术股份有限公司 一种分屏模式的实现方法和装置
CN109542379B (zh) * 2018-11-22 2021-09-14 联想(北京)有限公司 内容展示方法及装置、计算机系统及计算机可读存储介质
CN110381345B (zh) * 2019-07-05 2020-12-15 华为技术有限公司 一种投屏显示方法及电子设备
CN112416278B (zh) * 2020-11-10 2021-12-03 北京五八信息技术有限公司 一种屏幕共享方法、装置、电子设备及存储介质
CN112732160A (zh) * 2021-01-11 2021-04-30 微视显示器科技（深圳）有限公司 一种显示系统触控识别方法及显示系统
CN112732159A (zh) * 2021-01-11 2021-04-30 微视显示器科技（深圳）有限公司 一种智能显示系统及控制方法
CN113645369B (zh) * 2021-08-13 2022-11-18 展讯通信（上海）有限公司 多个屏幕的显示方法及装置、计算机可读存储介质、终端

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100245388A1 (en) * 2006-02-21 2010-09-30 Thomas Bauch Computer network system and method for operating a network system screen shot and source shot control
CN102883135A (zh) * 2012-11-01 2013-01-16 成都飞视美视频技术有限公司 屏幕共享及控制方法
CN104506907A (zh) * 2014-11-25 2015-04-08 上海众应信息科技有限公司 控制端与多个远程被控终端之间的交互操作方法及系统
CN104808898A (zh) * 2015-04-13 2015-07-29 深圳市金立通信设备有限公司 一种终端
CN106375804A (zh) * 2016-05-30 2017-02-01 深圳市美贝壳科技有限公司 移动端同步模拟智能设备界面的改良方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104216630B (zh) * 2014-08-21 2019-01-11 小米科技有限责任公司 界面共享方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100245388A1 (en) * 2006-02-21 2010-09-30 Thomas Bauch Computer network system and method for operating a network system screen shot and source shot control
CN102883135A (zh) * 2012-11-01 2013-01-16 成都飞视美视频技术有限公司 屏幕共享及控制方法
CN104506907A (zh) * 2014-11-25 2015-04-08 上海众应信息科技有限公司 控制端与多个远程被控终端之间的交互操作方法及系统
CN104808898A (zh) * 2015-04-13 2015-07-29 深圳市金立通信设备有限公司 一种终端
CN106375804A (zh) * 2016-05-30 2017-02-01 深圳市美贝壳科技有限公司 移动端同步模拟智能设备界面的改良方法

Also Published As

Publication number Publication date
CN108694029A (zh) 2018-10-23

Similar Documents

Publication Publication Date Title
WO2018184443A1 (zh) 终端屏幕显示处理方法、装置及存储介质
US11128802B2 (en) Photographing method and mobile terminal
WO2018184442A1 (zh) 终端控制方法及装置
US10551987B2 (en) Multiple screen mode in mobile terminal
WO2020238744A1 (zh) 分屏显示的处理方法、装置、设备和存储介质
WO2015172507A1 (zh) 一种应用启动的方法、装置及移动终端
WO2022089330A1 (zh) 截图方法和装置、电子设备和可读存储介质
US20120289290A1 (en) Transferring objects between application windows displayed on mobile terminal
WO2015180621A1 (en) Method and apparatus for playing im message
US10228835B2 (en) Method for displaying information, and terminal equipment
CN105045485A (zh) 一种屏幕截图方法及移动终端
CN112099707A (zh) 显示方法、装置和电子设备
US20160371048A1 (en) Remote gesture control, input monitor, systems including the same, and associated methods
US20140043255A1 (en) Electronic device and image zooming method thereof
WO2019129161A1 (zh) 移动终端后台应用程序查看方法及装置
CN107102800A (zh) 消息的显示方法以及显示消息的电子设备
CN107153546B (zh) 一种视频播放方法及移动设备
WO2024045985A1 (zh) 屏幕控制方法、屏幕控制装置、电子设备、程序及介质
EP3647970A1 (en) Method and apparatus for sharing information
CN111913675A (zh) 显示控制方法和装置、电子设备及计算机可读存储介质
CN111880706B (zh) 功能切换方法、装置、电子设备和可读存储介质
CN112099714B (zh) 截图方法、装置、电子设备及可读存储介质
US9823778B2 (en) Method and apparatus for controlling touch device, and touch device
CN113835577A (zh) 显示方法、装置、电子设备及存储介质
CN112911147A (zh) 显示控制方法、显示控制装置及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18780730

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18780730

Country of ref document: EP

Kind code of ref document: A1