CN107992256B - Window control method, device and system - Google Patents
Window control method, device and system Download PDFInfo
- Publication number
- CN107992256B CN107992256B CN201711251902.2A CN201711251902A CN107992256B CN 107992256 B CN107992256 B CN 107992256B CN 201711251902 A CN201711251902 A CN 201711251902A CN 107992256 B CN107992256 B CN 107992256B
- Authority
- CN
- China
- Prior art keywords
- image
- window
- mouse
- desktop
- control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Controls And Circuits For Display Device (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention relates to a window control method, device and system, and belongs to the technical field of large screen display. The method comprises the following steps: acquiring a window image and a desktop image, separating a mouse image from the desktop image, and superposing the separated mouse image on the window image and the desktop image to obtain a display image of front-end display equipment; after receiving mouse operation information detected by desktop image equipment through a simulation window control and generating a corresponding mouse control instruction, background control equipment receives the mouse control instruction; wherein the simulated window control corresponds to the window image; and controlling the display image of the front-end display equipment according to the mouse control instruction so as to correspondingly adjust the window image. By the technical scheme, the problem that the window operation of a user is inconvenient due to the separation between the large-screen window and the background control system is solved, and the front-end display of the window operation process is realized.
Description
Technical Field
The invention relates to the technical field of large screen display, in particular to a window control method, device and system and a storage medium.
Background
The large-screen window display provides rich information for users, and the window often needs to be moved, reduced, deleted and the like in the window control process. Traditionally, window operation on a large screen is generally realized through background control equipment, and the background control equipment does not have actual image data access, but replaces a window image with a simulated image control. In the process of implementing the invention, the inventor finds that at least the following problems exist in the traditional technology: because the front-end display device is separated from the background control device, when the window is operated, a user cannot judge the relationship between the screen window and the background control component, so that the user needs to operate on the background control device firstly, check whether the image on the window is correct or not, and repeat the operation on the window on the background control device again if the image is wrong. The window control process is very troublesome, inconvenience is brought to user operation, and misoperation is easy to occur. Therefore, it is necessary to find a method for displaying the window operation process on the window.
Disclosure of Invention
Based on this, the embodiment of the invention provides a window control method, device and system, which can realize front-end display of window operation.
The content of the embodiment of the invention is as follows:
a window control method, comprising: acquiring a window image and a desktop image, wherein the desktop image comprises a mouse image; separating the mouse image from the desktop image; superposing the window image on the desktop image to generate a composite image; superposing the separated mouse image on the synthetic image to obtain a display image of the front-end display device; after receiving mouse operation information detected by desktop image equipment through a simulation window control and generating a corresponding mouse control instruction, background control equipment receives the mouse control instruction sent by the background control equipment; wherein the simulated window control corresponds to the window image; and controlling the display image of the front-end display equipment according to the mouse control instruction so as to correspondingly adjust the window image.
In one embodiment, before the superimposing the window image on the desktop image and generating the composite image, the method further includes: and adjusting the window image according to the size information of the window image and the desktop image. The embodiment can realize the adjustment of the window image before the window image is displayed.
A window control method, comprising: acquiring windowing information of a window image from background control equipment, and drawing a corresponding simulation window control according to the windowing information to obtain a desktop image; the desktop image also comprises a mouse image; sending the desktop image containing the mouse image to a hardware processor, so that the hardware processor separates the mouse image from the desktop image and superimposes the separated mouse image on a synthetic image to obtain a display image of front-end display equipment; the composite image is composed of a desktop image and a window image superposed on the desktop image; detecting local mouse operation information through the simulation window control, and sending the mouse operation information to background control equipment so that the background control equipment generates a corresponding mouse control instruction according to the mouse operation information and sends the mouse control instruction to a hardware processor; the hardware processor is used for controlling a display image of front-end display equipment so as to enable the window image to be correspondingly adjusted.
In one embodiment, the windowing information includes size and position information of the window image; the drawing of the corresponding simulation window control according to the windowing information to obtain the desktop image comprises the following steps: and drawing a simulation window control on the initial desktop image according to the size and the position information to obtain the desktop image. And drawing a simulation window control according to the size and position information of the window image, so that the operation on the window image can be acquired through the simulation window control.
In one embodiment, after the detecting local mouse operation information through the simulated window control and sending the mouse operation information to the background control device, the method further includes: and adjusting the simulation window control according to the mouse operation information. And adjusting the simulation window control according to the mouse operation information to enable the simulation window control to correspond to the window image.
A window control method, comprising: the method comprises the steps that a desktop image device obtains windowing information of a window image from a background control device, draws a corresponding simulation window control according to the windowing information, and obtains a desktop image, wherein the desktop image also comprises a mouse image; sending the desktop image containing the mouse image to a hardware processor; a hardware processor acquires a window image and the desktop image; separating the mouse image from the desktop image; superposing the window image on the desktop image to generate a composite image, superposing the separated mouse image on the composite image to obtain a display image, and sending the display image to a front-end display device for displaying; the desktop image equipment detects local mouse operation information through the simulation window control and sends the mouse operation information to the background control equipment; adjusting the simulation window control according to the mouse operation information; the background control equipment generates a mouse control instruction according to the mouse operation information and sends the mouse control instruction to the hardware processor; and the hardware processor controls the display image of the front-end display equipment according to the mouse control instruction so as to correspondingly adjust the window image.
Correspondingly, an embodiment of the present invention provides a window control apparatus, including: the mouse separation module is used for acquiring a window image and a desktop image, wherein the desktop image comprises a mouse image; separating the mouse image from the desktop image; the display image acquisition module is used for superposing the window image on the desktop image to generate a synthetic image; superposing the separated mouse image on the synthetic image to obtain a display image of the front-end display device; the control instruction receiving module is used for receiving a mouse control instruction sent by the background control equipment after the background control equipment receives mouse operation information detected by the desktop image equipment through the simulation window control and generates a corresponding mouse control instruction; wherein the simulated window control corresponds to the window image; and the control instruction execution module is used for controlling the display image of the front-end display equipment according to the mouse control instruction so as to correspondingly adjust the window image.
A window control apparatus comprising: the window control drawing module is used for acquiring windowing information of a window image from the background control equipment and drawing a corresponding simulation window control according to the windowing information to obtain a desktop image; the desktop image also comprises a mouse image; the desktop image sending module is used for sending the desktop image containing the mouse image to a hardware processor, so that the hardware processor separates the mouse image from the desktop image and superposes the separated mouse image on a synthetic image to obtain a display image of the front-end display device; the composite image is composed of a desktop image and a window image superposed on the desktop image; the display image control module is used for detecting local mouse operation information through the simulation window control and sending the mouse operation information to the background control equipment, so that the background control equipment generates a corresponding mouse control instruction according to the mouse operation information and sends the mouse control instruction to the hardware processor; the hardware processor is used for controlling a display image of front-end display equipment so as to enable the window image to be correspondingly adjusted.
An embodiment of the present invention further provides a window control system, including: the system comprises desktop image equipment, a hardware processor and background control equipment; the desktop image equipment is used for acquiring windowing information of a window image from the background control equipment, drawing a corresponding simulation window control according to the windowing information, and obtaining a desktop image, wherein the desktop image also comprises a mouse image; sending the desktop image containing the mouse image to a hardware processor; the hardware processor is used for acquiring a window image and the desktop image; separating the mouse image from the desktop image; superposing the window image on the desktop image to generate a composite image, superposing the mouse image on the composite image to obtain a display image, and sending the display image to a front-end display device for displaying; the desktop image equipment is also used for detecting local mouse operation information through the simulation window control and sending the mouse operation information to the background control equipment; adjusting the simulation window control according to the mouse operation information; the background control equipment is used for generating a mouse control instruction according to the mouse operation information and sending the mouse control instruction to the hardware processor; and the hardware processor is also used for controlling a display image of front-end display equipment according to the mouse control instruction so as to correspondingly adjust the window image.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method described above, the computer program being stored thereby.
The window control method comprises the steps of enabling a mouse image on a desktop image to be arranged on the window image and the desktop image in a propping mode, drawing a simulation window control through desktop image equipment, and obtaining mouse operation information through the simulation window control so as to control a display image of front-end display equipment and display an operation process for controlling the window image. The association between the front-end screen mouse and the control equipment is established, and the front-end display of the desktop mouse and the front-end display of the window operation process can be realized.
Drawings
FIG. 1 is a diagram illustrating an application of a window control method in an embodiment;
FIG. 2 is a schematic flowchart of a window control method in the first embodiment;
FIG. 3 is a diagram illustrating an application of a window control method in another embodiment;
FIG. 4 is a schematic flowchart of a window control method in a second embodiment;
FIG. 5 is a schematic flowchart of a window control method in a third embodiment;
FIG. 6 is a diagram illustrating an image output by a desktop image device in one embodiment;
FIG. 7 is a schematic structural diagram of a window control apparatus according to an embodiment;
fig. 8 is a schematic structural diagram of a window control device in another embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, in the current large screen control system, the background control device 150 is connected to the hardware processor 130 through the low-speed control line 131, there is no actual display image, and the user cannot visually see the display image at the front end when scheduling a signal through the background control device 150. The actual image signal flow is as the first signal line 121 and the second signal line 141 in the figure, the signals of the desktop image 111 and the window image 112 are all directly sent to the front-end display device 110 through the hardware processor 130 for displaying, and only the control signal interaction is realized between the background control device 150 and the hardware processor 130.
In order to enrich the output display effect, a desktop signal is generally added to a signal to be displayed, and the desktop signal is equivalent to the bottommost window. When no window signal exists, displaying a desktop signal; when a window signal exists, the window image is displayed on the desktop signal. As shown in fig. 1, data communication cannot be performed between the desktop image device 120 and the background control device 150, the desktop image is sent to the hardware processor 130 as a desktop image layer, and the desktop mouse is not connected to the control system of the hardware processor 130, and the operation of the mouse cannot be displayed under the control of the hardware processor 130, so that the desktop signal is only equivalent to a common window signal at the bottom layer, and the mouse does not have any other special function. After the desktop image is covered by the upper layer image, all actions of the desktop do not have any significance, and the operation of the user on the window can only be executed by controlling the PC, so that the actual large-screen windowing display effect cannot be seen.
Fig. 2 is a schematic flowchart of a window control method according to a first embodiment, which is described in this embodiment by taking a hardware processor as an example, and the window control method provided in the embodiment mainly includes steps S110 to S140, which are described in detail as follows:
s110, acquiring a window image and a desktop image, wherein the desktop image comprises a mouse image; and separating the mouse image from the desktop image.
In this step, the hardware processor 130 obtains a window image transmitted by the window image device 140 (through the fourth signal line 142) and a desktop image transmitted by the desktop image device 120 (through the third signal line 122). The hardware processor 130 performs a separation process on the desktop image including the mouse image to obtain a separated mouse image.
The window images are images, videos and the like displayed in large-screen windows such as a splicing wall and a splicing screen; the desktop image is an image which is obtained through a desktop signal and is used as a large-screen display desktop. In the embodiment of the present invention, the desktop image device 120 is configured to process a desktop image and acquire mouse operation information for operating a window image.
Alternatively, the image corresponding to the mouse image in the separated desktop image may replace pixels of the image corresponding to the mouse image with pixels of a nearby image according to surrounding pixels. If the window image is on the image corresponding to the mouse image, no processing may be performed. Other processing methods are also possible as long as the entire display of the separated desktop image is not affected.
In order to facilitate the window operation and the control of the whole large-screen window, a mouse (input device) is connected to the desktop image device 120, and the desktop image device 120 obtains the operation information of the window image and controls the window image and the mouse image. After the desktop image device 120 is connected to the mouse, the desktop image has the image of the mouse, and the step separates the image of the mouse from the desktop image, so that the mouse image is prevented from being covered by the window image on the desktop image.
S120, superimposing the window image on the desktop image to generate a synthetic image; the separated mouse image is superimposed on the composite image to obtain a display image of the front-end display device 110.
The image superposition process is realized by superposing a window image to be displayed on a separated desktop image to obtain a composite image, superposing a mouse image obtained by separation on the composite image to obtain an image to be displayed, sending the image to be displayed to the front-end display device 110 to realize the image display, wherein the mouse image 113 is always positioned at the top layer of the whole screen, and the desktop image 111 is positioned at the bottom layer of the window.
S130, after receiving mouse operation information detected by the desktop image device 120 through the simulated window control and generating a corresponding mouse control instruction at the background control device 150, receiving the mouse control instruction sent by the background control device 150; wherein the simulated window control corresponds to the window image.
In this step, the hardware processor 130 receives a mouse control instruction, and specifically, the generation process of the mouse control instruction is as follows: the desktop image device 120 detects mouse operation information through the simulated window control, and sends the mouse operation information to the background control device 150 through the control signal line 123 (as shown in fig. 3, fig. 3 is an application schematic diagram of a window control method in an embodiment, a control signal line 123 exists between the desktop image device 120 and the background control device 150 to implement data synchronization of the desktop image device and the background control device 150), and the background control device 150 generates a corresponding mouse control instruction according to the mouse operation information and sends the mouse control instruction to the hardware processor 130.
The mouse operation information is window control information which is input through a mouse connected to desktop image equipment when a user needs to control a window image, and the mouse operation information comprises information for controlling the window image and display of the mouse image.
The simulation window control is used for acquiring operation information of a window, and the simulation window control corresponds to the size and the position of the window image, namely the window image is overlapped with the simulation window control. If the simulation window control is displayed on a large-screen window, the window image can completely cover the simulation window control, so that the simulation window control can completely acquire information for operating the window image. Of course, the analog window control may not be displayed on the large-screen window, as long as the analog window control corresponds to the window image and can acquire the mouse operation information for controlling the window image and the mouse image. The simulation window control can be in various shapes such as a square shape or a circular shape.
And S140, controlling the display image of the front-end display device 110 according to the mouse control instruction so as to correspondingly adjust the window image.
In this step, after receiving the mouse control instruction sent by the background control device 150, the hardware processor 130 executes control on the window image 112 in the display image according to the mouse control instruction. Optionally, this step may also include control of the mouse image 113.
In the above embodiment, the mouse image on the desktop image is placed on top of the window image and the desktop image, the desktop image device 120 draws the simulation window control on the desktop image, and the mouse operation information is acquired through the simulation window control, so as to control the display image of the front-end display device 110 and display the operation process for controlling the real image. The association between the front-end screen mouse and the control equipment is established, and the front-end display of the desktop mouse and the front-end display of the window operation process can be realized.
In one embodiment, step S110 includes: and separating continuous pixel points with the same preset pixel values as those of the mouse image from the desktop image to obtain a separated mouse image.
Optionally, the desktop image and the mouse image may be separated by other manners, for example, an image with similarity to a preset shape of the mouse image higher than a certain threshold is separated from the desktop image, so as to obtain a separated mouse image.
The embodiment realizes the separation of the mouse image and the desktop image according to the difference between the desktop image and the mouse image.
In one embodiment, step S140 includes: when the mouse operation information detected by the desktop image device 120 through the analog window control is the adjustment operation information of the analog window control, the mouse control instruction is a control instruction of a window image in the display image of the front-end display device 110, and the size or the position of the window image in the display image of the front-end display device 110 is adjusted according to the mouse control instruction; when the mouse operation information detected by the desktop image device 120 through the analog window control is deletion operation information of the analog window control, the mouse control instruction is a control instruction of a window image in a display image of the front-end display device 110, and the window image in the display image of the front-end display device 110 is hidden according to the mouse control instruction.
Optionally, step S140 may further include control of the mouse image, namely: when the mouse operation information detected by the desktop image device 120 through the simulation window control is mouse movement operation information, the mouse control instruction is a movement control instruction for a mouse image in the display image of the front-end display device 110, and the position of the mouse image in the display image of the front-end display device 110 is moved according to the mouse control instruction. In this embodiment, the background control device 150 generates corresponding mouse control instructions for different mouse operation information to control the window image and the mouse image, and the process of controlling the window image and the mouse image is displayed through the front-end display device 110. Front-end display of window operations is achieved.
Optionally, the window image may also be a video image, and the implementation process of step S140 may also be: and when the mouse operation information is pause operation information of the analog window control, continuously displaying the current window image or not updating the window image according to the mouse control instruction. The implementation process can also be other implementation processes for controlling window images and videos.
In one embodiment, the window control method further includes the steps of: and adjusting the window image according to the size information of the window image and the desktop image. Optionally, the adjusting of the window image may be moving, zooming, and the like of the window image and an overlapping process of overlapping the window image on the desktop image; or may be a related process to the desktop image. After the desktop image and the window image complete the relevant operation, the separated mouse image is superposed to the processed image again and output. The window image is adjusted before the window image is displayed, the window image display effect is improved, and the problems of window image inclination, improper size and the like can be prevented.
Fig. 4 is a schematic flowchart of a window control method according to a second embodiment, which is described in this embodiment by taking a desktop image device as an example, and the window control method provided in the embodiment mainly includes steps S210 to S230, which are described in detail as follows:
s210, acquiring windowing information of a window image from the background control equipment 150, and drawing a corresponding simulation window control according to the windowing information to obtain a desktop image; the desktop image also comprises a mouse image.
In this step, after the background control device 150 sends the windowing information of the window image to the desktop image device 120, the desktop image device 120 draws a simulated window control corresponding to the window image according to the windowing information, so as to obtain the desktop image.
Alternatively, the process of drawing the corresponding simulated window control may be before the hardware processor 130 sends the display image to the front-end display device 110, or may be after the hardware processor 130 sends the display image to the front-end display device 110. The simulated window control may be displayed through the front-end display device 110, or may not be displayed, and if the simulated window control is displayed, the display of the simulated window control on the front-end display device is completely covered by the window image corresponding to the simulated window control, and therefore is invisible.
S220, sending the desktop image including the mouse image to the hardware processor 130, so that the hardware processor 130 separates the mouse image from the desktop image and superimposes the separated mouse image on the synthesized image to obtain a display image of the front-end display device 110; the composite image is composed of a desktop image and a window image superimposed thereon.
In this step, the desktop image device 120 sends the desktop image including the mouse image to the hardware processor 130, so that the hardware processor 130 realizes the processes of mouse separation and image superposition to obtain the composite image to be displayed (i.e., the aforementioned display image).
S230, detecting local mouse operation information through the simulation window control, and sending the mouse operation information to the background control device 150, so that the background control device 150 generates a corresponding mouse control instruction according to the mouse operation information and sends the mouse control instruction to the hardware processor 130; the hardware processor 130 is configured to control a display image of the front-end display device 110, so that the window image is adjusted accordingly.
The step is realized by the process of acquiring the mouse operation information through the simulation window control. The desktop image device 120 is connected to a mouse, and when the user operates the window image through the mouse, the desktop image device 120 obtains local mouse operation information through the simulated window control and sends the mouse operation information to the hardware processor 130 to implement front-end display of the window operation process.
The embodiment establishes the association between the front-end screen mouse and the control equipment, can realize the front-end display of the desktop mouse, and enables a user to intuitively sense the operation of the relevant window through the large-screen desktop mouse. Taking the spliced wall as an example, an operation interface is provided for a user on the spliced wall through a mouse image on a desktop image, so that the user can realize various window operations directly on the spliced wall without going to the background control equipment 150 for operation, then looking at whether the image on the spliced wall is correct or not, and then determining whether the step of adjusting the image needs to be repeated or not.
In an embodiment, the windowing information includes size and position information of the window image. Further, step S210 includes: and drawing a simulation window control on the desktop image according to the size and the position information. According to the embodiment, the simulation window control is drawn according to the size and the position information of the window image, so that the operation on the window image can be acquired through the simulation window control.
Optionally, the simulated window control may also be drawn in a desktop window instead of only on a desktop image, and the simulated window control is used to obtain operation information about the window image and the mouse image, and the operation information may be obtained by the simulated window control.
In one embodiment, the window control method further includes the steps of: judging whether pixel points with the same value as the preset pixel values of the mouse image exist in the pixel points of the desktop image; and if so, replacing the pixel value of the pixel point with other pixel values.
The embodiment realizes the process of desktop image preprocessing. Alternatively, the mouse image is generally a white background black frame, and therefore, the black color of the mouse may be set to (r, g, b) ═ 0,0,0, and the white color may be set to (r, g, b) ═ 255, 255. When there are full black and full white pixel values on the desktop image, i.e., other pixel values in the desktop image are the same pixel values as the mouse image, then the full black and full white pixel values are replaced with other pixel values (e.g., neighboring values): (0,0,0) → (1,1,1), (255,255,255) → (254,254,254), and after completing the above processing, the desktop image device 120 transmits the desktop image including the mouse graphics to the hardware processor 130. Alternatively, the preset pixel value of the mouse image may be other pixel values, for example, the mouse image may be a color image or a gray image. The desktop image preprocessing process can also be realized in other ways, for example, by using a pattern recognition technology, an image with similarity higher than a certain threshold value with a preset shape of the mouse image is marked, and the mouse image and the desktop image are distinguished. The present embodiment distinguishes the mouse image from the desktop image, which facilitates the hardware processor 130 to separate the mouse image from the desktop image.
In one embodiment, the window control method further includes the steps of: and adjusting the simulation window control according to the mouse operation information. In this embodiment, the simulated window control is adjusted according to the mouse operation information, so that the simulated window control corresponds to the window image.
Fig. 5 is a schematic flowchart of a window control method according to a third embodiment, where the window control method provided in the embodiment mainly includes steps S310 to S350, and the following is described in detail:
s310, the desktop image equipment acquires windowing information of a window image from the background control equipment, draws a corresponding simulation window control according to the windowing information, and obtains a desktop image which also comprises a mouse image; and sending the desktop image containing the mouse image to a hardware processor.
S320, acquiring a window image and the desktop image by a hardware processor; separating the mouse image from the desktop image; and superposing the window image on the desktop image to generate a synthetic image, superposing the separated mouse image on the synthetic image to obtain a display image, and sending the display image to a front-end display device for displaying.
S330, the desktop image equipment detects local mouse operation information through the simulation window control and sends the mouse operation information to background control equipment; and adjusting the simulation window control according to the mouse operation information.
And S340, the background control equipment generates a mouse control instruction according to the mouse operation information and sends the mouse control instruction to the hardware processor.
And S350, controlling the display image of the front-end display equipment by the hardware processor according to the mouse control instruction so as to correspondingly adjust the window image.
According to the embodiment, the association between the front-end screen mouse and the control equipment is established, and the front-end display of the desktop mouse and the front-end display of the window operation process can be realized.
In order to better understand the above method, an application example of the window control method of the present invention is described in detail below, and the implementation process is specifically as follows:
1) preprocessing a desktop image, wherein a mouse is a white background black frame, and the pixel value of a preset mouse image is as follows: black (r, g, b) — (0,0,0), and white (r, g, b) — (255,255,255). If a pixel point with the same color value as the mouse exists on the desktop image, the pixel value of the pixel point is replaced by an adjacent value, for example, (0,0,0) is replaced by (1,1,1), and (255 ) is replaced by (254,254,254). After the above processing, the desktop image device 120 sends the desktop image containing the mouse graphics to the hardware processor 130.
2) And (3) separating the mouse, detecting the pixel value of the desktop image by the hardware processor 130, and when the same continuous pixel points with the preset pixel values (r, g and b) (0,0,0) or (r, g and b) (255,255 and 255) of the mouse image exist, extracting and identifying the continuous pixel points, completing the separation of the mouse, and obtaining the separated mouse image.
3) And processing the desktop image and the window image, wherein the hardware processor 130 performs moving and zooming processing on the window image according to the size position relation between the window image and the desktop image, performs superposition processing, and superimposes the window image on the desktop image to obtain a composite image.
4) And (3) synthesizing the mouse images, and superposing the mouse image 113 separated in the step 2) on the synthesized image processed in the step 3) by the hardware processor 130 to obtain a display image.
5) The desktop image device data is synchronized, and the desktop image device 120 obtains the windowing information of the window image 112 from the background control device 150 (for example, the control signal line 123 between the desktop image device and the background control device 150 in fig. 3 realizes data synchronization between the desktop image device and the background control device 150). A simulated window control is drawn on a desktop window, the simulated window control displayed on a desktop image device is shown in fig. 6, fig. 6 is a schematic diagram of an image output by a mouse desktop processing device in an embodiment, and the simulated window control is located above the desktop image and below the mouse image. When the simulated window control is displayed and images (desktop image, window image and mouse image) are pushed to the front-end display device through the hardware processor 150, the simulated window control in the desktop image is covered by the window image of the top layer and becomes invisible because the actual window image is located at the top layer of the same position. Also, as shown in FIG. 3, the mouse image is placed on top of the image.
6) Mouse processing, when the user operates the foreground window through the mouse connected to the desktop image device 120, it is the operation of the analog window control for the desktop image device 120, and the desktop image device 120 sends the information of the operation of the foreground window to the background control device 150 (sends data via the control signal line 123).
7) The background control device 150 sends a mouse operation control instruction to the hardware processor 130 to complete the corresponding window control and the display control of the window operation process.
It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present invention is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present invention.
Based on the same idea as the window control method in the above embodiment, the present invention also provides a window control apparatus that can be used to execute the above window control method. For convenience of illustration, only the parts related to the embodiments of the present invention are shown in the schematic structural diagram of the embodiments of the window control device, and those skilled in the art will understand that the illustrated structure does not constitute a limitation of the device, and may include more or less components than those illustrated, or combine some components, or arrange different components.
Fig. 7 is a schematic structural diagram of a window control device according to an embodiment, where the window control device includes a mouse separation module 310, a display image obtaining module 320, a control instruction receiving module 330, and a control instruction executing module 340, and the detailed description is as follows:
a mouse separation module 310, configured to obtain a window image and a desktop image, where the desktop image includes a mouse image; and separating the mouse image from the desktop image.
A display image obtaining module 320, configured to superimpose the window image on the desktop image to generate a composite image; and superposing the separated mouse image on the synthetic image to obtain a display image of the front-end display equipment.
The control instruction receiving module 330 is configured to receive, by a background control device, a mouse control instruction sent by the background control device after receiving mouse operation information detected by the desktop image device through the simulation window control and generating a corresponding mouse control instruction; wherein the simulated window control corresponds to the window image.
And a control instruction execution module 340, configured to control a display image of a front-end display device according to the mouse control instruction, so that the window image is correspondingly adjusted.
The window control device of the embodiment establishes the association between the front-end screen mouse and the control equipment, can realize the front-end display of the desktop mouse, and enables a user to intuitively sense the relevant window operation through the large-screen desktop mouse.
In an embodiment, the mouse separating module 310 includes a mouse separating sub-module, configured to separate consecutive pixel points having the same preset pixel value as the mouse image from the desktop image to obtain the mouse image.
In one embodiment, the control instruction execution module 340 includes: a size adjustment sub-module, configured to, when the mouse operation information detected by the desktop image device 120 through the analog window control is adjustment operation information for the analog window control, adjust a size or a position of a window image in a display image of the front-end display device 110 according to the mouse control instruction, where the mouse control instruction is a control instruction for a window image in a display image of the front-end display device 110; a deletion processing sub-module, configured to, when the mouse operation information detected by the desktop image device 120 through the analog window control is deletion operation information for the analog window control, cause the mouse control instruction to be a control instruction for a window image in a display image of the front-end display device 110, and hide the window image in the display image of the front-end display device 110 according to the mouse control instruction; and a movement processing sub-module, configured to, when the mouse operation information detected by the desktop image device 120 through the analog window control is mouse movement operation information, move the position of the mouse image in the display image of the front-end display device 110 according to the mouse control instruction, where the mouse control instruction is a movement control instruction for the mouse image in the display image of the front-end display device 110.
In one embodiment, the window control apparatus further includes: and the image processing module is used for adjusting the window image according to the size information of the window image and the desktop image.
Fig. 8 is a schematic structural diagram of a window control apparatus according to another embodiment, where the window control apparatus includes a desktop image sending module 410, a window control drawing module 420, and a display image control module 430, and the detailed description is as follows:
the window control drawing module 410 is configured to obtain windowing information of a window image from the background control device 150, and draw a corresponding simulated window control according to the windowing information to obtain a desktop image; the desktop image also comprises a mouse image
A desktop image sending module 420, configured to send the desktop image including the mouse image to the hardware processor 130, so that the hardware processor 130 separates the mouse image from the desktop image and superimposes the separated mouse image on the synthesized image to obtain a display image of the front-end display device 110; the composite image is composed of a desktop image and a window image superimposed thereon.
The display image control module 430 is configured to detect local mouse operation information through the simulation window control, and send the mouse operation information to the background control device 150, so that the background control device 150 generates a corresponding mouse control instruction according to the mouse operation information and sends the mouse control instruction to the hardware processor 130; the hardware processor 130 is configured to control a display image of the front-end display device 110, so that the window image is adjusted accordingly.
In one embodiment, the windowing information includes size and position information of the window image; the window control drawing module 410 further includes a control drawing submodule, configured to draw a simulated window control on the initial desktop image according to the size and the position information, so as to obtain a desktop image.
In one embodiment, the window control apparatus further includes: the pixel point judging module is used for judging whether pixel points with the same value as the preset pixel values of the mouse image exist in the pixel points of the desktop image; and the pixel value replacing module is used for replacing the pixel value of the pixel point with other pixel values if the pixel value is the same as the pixel value of the pixel point.
In an embodiment, the window control apparatus further includes a simulation window adjusting module, configured to adjust the simulation window control according to the mouse operation information.
An embodiment of the present invention further provides a window control system, where the window control system includes: desktop image device 120, hardware processor 130, and background control device 150; the desktop image device 120 is configured to obtain windowing information of a window image from the background control device 150, and draw a corresponding simulated window control according to the windowing information to obtain a desktop image, where the desktop image further includes a mouse image; sending the desktop image containing the mouse image to the hardware processor 130; the hardware processor 130 is configured to obtain a window image and the desktop image; separating the mouse image from the desktop image; superimposing the window image on the desktop image to generate a composite image, superimposing the mouse image on the composite image to obtain a display image, and sending the display image to the front-end display device 110 for displaying; the desktop image device 120 is further configured to detect local mouse operation information through the simulated window control by the desktop image device 120, and send the mouse operation information to the background control device 150; adjusting the simulation window control according to the mouse operation information; the background control device 150 is configured to generate a mouse control instruction according to the mouse operation information and send the mouse control instruction to the hardware processor 130; the hardware processor 130 is further configured to control the display image of the front-end display device 110 according to the mouse control instruction, so that the window image is correspondingly adjusted.
Optionally, the window control system further includes a window image device 140, where the window image device 140 is configured to send a window image and windowing information of the window image to the hardware processor 130, the hardware processor 130 sends the windowing information of the window image to the background control device 150, and the desktop image device 120 obtains the windowing information from the background control device 150 and draws a corresponding simulated window control according to the windowing information.
The window control system comprises desktop image equipment 120, a hardware processor 130, window image equipment 140 and background control equipment 150, wherein a mouse is connected to the desktop image equipment 120, a desktop image containing a mouse image is obtained by drawing a simulation window control on an original desktop image, the mouse image is separated, front-end display of the mouse image is realized, association between a front-end screen mouse and the control equipment is established, front-end display of window operation can be realized, and a user can intuitively sense related window operation through a large-screen desktop mouse.
It will be understood by those skilled in the art that all or part of the processes of the methods of the above embodiments may be implemented by a computer program, which is stored in a computer readable storage medium and sold or used as a stand-alone product. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It should be noted that the terms "first \ second \ third" related to the embodiments of the present invention are merely used for distinguishing similar objects, and do not represent a specific ordering for the objects, and it should be understood that "first \ second \ third" may exchange a specific order or sequence order if allowed. It should be understood that the terms first, second, and third, as used herein, are interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in other sequences than those illustrated or otherwise described herein.
The terms "comprises" and "comprising," and any variations thereof, of embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or (module) elements is not limited to only those steps or elements but may alternatively include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-described examples merely represent several embodiments of the present invention and should not be construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A window control method, comprising:
acquiring a window image and a desktop image, wherein the desktop image comprises a mouse image; separating the mouse image from the desktop image; the desktop image is obtained by the desktop image equipment after drawing the simulation window control according to the windowing information sent by the background control equipment;
superposing the window image on the desktop image to generate a composite image; superposing the separated mouse image on the synthetic image to obtain a display image of the front-end display device;
after receiving mouse operation information detected by desktop image equipment through a simulation window control and generating a corresponding mouse control instruction, background control equipment receives the mouse control instruction sent by the background control equipment; wherein the simulated window control corresponds to the window image;
controlling a display image of front-end display equipment according to the mouse control instruction so as to correspondingly adjust the window image;
before the superimposing the window image on the desktop image and generating the composite image, the method further includes:
and adjusting the window image according to the size information of the window image and the desktop image.
2. The window control method according to claim 1,
the window image is adjusted to be moved, zoomed and the like, and the window image is overlapped on the desktop image.
3. A window control method, comprising:
acquiring windowing information of a window image from background control equipment, and drawing a corresponding simulation window control according to the windowing information to obtain a desktop image; the desktop image also comprises a mouse image;
sending the desktop image containing the mouse image to a hardware processor, so that the hardware processor separates the mouse image from the desktop image and superimposes the separated mouse image on a synthetic image to obtain a display image of front-end display equipment; the composite image is composed of a desktop image and a window image superposed on the desktop image;
detecting local mouse operation information through the simulation window control, and sending the mouse operation information to background control equipment so that the background control equipment generates a corresponding mouse control instruction according to the mouse operation information and sends the mouse control instruction to a hardware processor; the hardware processor is used for controlling a display image of front-end display equipment so as to correspondingly adjust the window image;
wherein, the hardware processor superimposes the window image on the desktop image, and before generating the composite image, the method further comprises:
and adjusting the window image according to the size information of the window image and the desktop image.
4. The window control method according to claim 3, wherein the windowing information includes information on the size and position of the window image;
the drawing of the corresponding simulation window control according to the windowing information to obtain the desktop image comprises the following steps:
and drawing a simulation window control on the initial desktop image according to the size and the position information to obtain the desktop image.
5. The window control method according to claim 3, wherein after detecting local mouse operation information through the simulated window control and sending the mouse operation information to a background control device, the method further comprises:
and adjusting the simulation window control according to the mouse operation information.
6. A window control method, comprising:
the method comprises the steps that a desktop image device obtains windowing information of a window image from a background control device, draws a corresponding simulation window control according to the windowing information, and obtains a desktop image, wherein the desktop image also comprises a mouse image; sending the desktop image containing the mouse image to a hardware processor;
a hardware processor acquires a window image and the desktop image; separating the mouse image from the desktop image; superposing the window image on the desktop image to generate a composite image, superposing the separated mouse image on the composite image to obtain a display image, and sending the display image to a front-end display device for displaying;
the desktop image equipment detects local mouse operation information through the simulation window control and sends the mouse operation information to the background control equipment; adjusting the simulation window control according to the mouse operation information;
the background control equipment generates a mouse control instruction according to the mouse operation information and sends the mouse control instruction to the hardware processor;
the hardware processor controls the display image of the front-end display equipment according to the mouse control instruction so as to correspondingly adjust the window image;
wherein, the hardware processor superimposes the window image on the desktop image, and before generating the composite image, the method further comprises:
and adjusting the window image according to the size information of the window image and the desktop image.
7. A window control apparatus, comprising:
the mouse separation module is used for acquiring a window image and a desktop image, wherein the desktop image comprises a mouse image; separating the mouse image from the desktop image; the desktop image is obtained by the desktop image equipment after drawing the simulation window control according to the windowing information sent by the background control equipment;
the display image acquisition module is used for superposing the window image on the desktop image to generate a composite image, and superposing the mouse image on the composite image to obtain a display image of front-end display equipment;
the control instruction receiving module is used for receiving a mouse control instruction sent by the background control equipment after the background control equipment receives mouse operation information detected by the desktop image equipment through the simulation window control and generates a corresponding mouse control instruction; wherein the simulated window control corresponds to the window image;
the control instruction execution module is used for controlling the display image of the front-end display equipment according to the mouse control instruction so as to correspondingly adjust the window image;
the image display module is further configured to adjust the window image according to the size information of the window image and the desktop image.
8. A window control apparatus, comprising:
the window control drawing module is used for acquiring windowing information of a window image from the background control equipment and drawing a corresponding simulation window control according to the windowing information to obtain a desktop image; the desktop image also comprises a mouse image;
the desktop image sending module is used for sending the desktop image containing the mouse image to a hardware processor, so that the hardware processor separates the mouse image from the desktop image and superposes the separated mouse image on a synthetic image to obtain a display image of the front-end display device; the composite image is composed of a desktop image and a window image superposed on the desktop image;
the display image control module is used for detecting local mouse operation information through the simulation window control and sending the mouse operation information to the background control equipment, so that the background control equipment generates a corresponding mouse control instruction according to the mouse operation information and sends the mouse control instruction to the hardware processor; the hardware processor is used for controlling a display image of front-end display equipment so as to correspondingly adjust the window image;
wherein, the hardware processor superimposes the window image on the desktop image, and before generating the composite image, the method further comprises:
and adjusting the window image according to the size information of the window image and the desktop image.
9. A window control system, comprising: the system comprises desktop image equipment, a hardware processor and background control equipment;
the desktop image equipment is used for acquiring windowing information of a window image from the background control equipment, drawing a corresponding simulation window control according to the windowing information, and obtaining a desktop image, wherein the desktop image also comprises a mouse image; sending the desktop image containing the mouse image to a hardware processor;
the hardware processor is used for acquiring a window image and the desktop image; separating the mouse image from the desktop image; superposing the window image on the desktop image to generate a composite image, superposing the mouse image on the composite image to obtain a display image, and sending the display image to a front-end display device for displaying;
the desktop image equipment is also used for detecting local mouse operation information through the simulation window control and sending the mouse operation information to the background control equipment; adjusting the simulation window control according to the mouse operation information;
the background control equipment is used for generating a mouse control instruction according to the mouse operation information and sending the mouse control instruction to the hardware processor;
the hardware processor is also used for controlling a display image of front-end display equipment according to the mouse control instruction so as to correspondingly adjust the window image;
wherein, the hardware processor superimposes the window image on the desktop image, and before generating the composite image, the method further comprises:
and adjusting the window image according to the size information of the window image and the desktop image.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the window control method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711251902.2A CN107992256B (en) | 2017-12-01 | 2017-12-01 | Window control method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711251902.2A CN107992256B (en) | 2017-12-01 | 2017-12-01 | Window control method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107992256A CN107992256A (en) | 2018-05-04 |
| CN107992256B true CN107992256B (en) | 2020-05-12 |
Family
ID=62035231
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711251902.2A Active CN107992256B (en) | 2017-12-01 | 2017-12-01 | Window control method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107992256B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112764853A (en) * | 2021-01-19 | 2021-05-07 | 深圳乐播科技有限公司 | Screen projection method, equipment and system |
| CN114296586B (en) * | 2021-12-28 | 2023-10-10 | 威创集团股份有限公司 | Content pushing method of seat system, storage medium and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101093653A (en) * | 2007-06-26 | 2007-12-26 | 广东威创日新电子有限公司 | Device and method for multiuser interactive controlled jogged wall, and for retrieving information displayed on the jogged wall |
| CN102411487A (en) * | 2011-10-28 | 2012-04-11 | 广东威创视讯科技股份有限公司 | Desktop control method and device |
| CN103049191A (en) * | 2012-12-14 | 2013-04-17 | 广东威创视讯科技股份有限公司 | Method and device for realizing mouse interactive operation on splicing wall display system |
| CN105426146A (en) * | 2015-11-06 | 2016-03-23 | 北京淳中科技股份有限公司 | Splicing processor and processing method thereof |
| CN106708455A (en) * | 2016-12-15 | 2017-05-24 | 广东威创视讯科技股份有限公司 | Virtual window-based splicing wall control method and apparatus |
-
2017
- 2017-12-01 CN CN201711251902.2A patent/CN107992256B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101093653A (en) * | 2007-06-26 | 2007-12-26 | 广东威创日新电子有限公司 | Device and method for multiuser interactive controlled jogged wall, and for retrieving information displayed on the jogged wall |
| CN102411487A (en) * | 2011-10-28 | 2012-04-11 | 广东威创视讯科技股份有限公司 | Desktop control method and device |
| CN103049191A (en) * | 2012-12-14 | 2013-04-17 | 广东威创视讯科技股份有限公司 | Method and device for realizing mouse interactive operation on splicing wall display system |
| CN105426146A (en) * | 2015-11-06 | 2016-03-23 | 北京淳中科技股份有限公司 | Splicing processor and processing method thereof |
| CN106708455A (en) * | 2016-12-15 | 2017-05-24 | 广东威创视讯科技股份有限公司 | Virtual window-based splicing wall control method and apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107992256A (en) | 2018-05-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106254933B (en) | Subtitle extraction method and device | |
| CN106664465B (en) | System for creating and reproducing augmented reality content and method using the same | |
| KR101733246B1 (en) | Apparatus and method for composition of picture for video call using face pose | |
| KR101463038B1 (en) | Display device and driving method of the same | |
| JP5495930B2 (en) | Image processing apparatus, method, and program | |
| CN101263723A (en) | Methods and apparatus for detecting and adjusting over-scanned images | |
| CN113645494B (en) | Screen fusion method, display device, terminal device and server | |
| CN107209922A (en) | Image processing equipment, image processing system and image processing method | |
| KR20130136774A (en) | Method of selective removal of text in video and apparatus for performing the same | |
| JP5509173B2 (en) | Display device and display method | |
| CN103000054B (en) | Intelligent teaching machine for kitchen cooking and control method thereof | |
| CN107992256B (en) | Window control method, device and system | |
| CN111787240B (en) | Video generation method, apparatus and computer readable storage medium | |
| CN114299100A (en) | Screen positioning method, terminal device and display device | |
| CN103543916A (en) | Information processing method and electronic equipment | |
| US11659134B2 (en) | Image processing apparatus and image processing method | |
| KR20150078229A (en) | Display apparatus and channel map manage method thereof | |
| JPH08123658A (en) | Method and device for registering icon pattern or icon, and method and device for displaying icon | |
| CN112567735A (en) | Multi-video signal pre-monitoring method and multi-video signal pre-monitoring system | |
| EP3641290B1 (en) | Electronic apparatus and control method thereof | |
| US11494934B2 (en) | Image processing device, image processing method, and monitoring system | |
| KR20120035360A (en) | Apparatus for recognizing character and method thereof | |
| JP2011239172A (en) | Video processing apparatus | |
| KR20120115898A (en) | Display apparatus having camera, remote controller for controlling the display apparatus, and display control method thereof | |
| CN112394815A (en) | Augmented reality-based drawing assistance method, augmented reality device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |