[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN112015508B - Screen-throwing interaction control method, equipment and computer-readable storage medium - Google Patents

Screen-throwing interaction control method, equipment and computer-readable storage medium Download PDF

Info

Publication number
CN112015508B
CN112015508B CN202010890879.7A CN202010890879A CN112015508B CN 112015508 B CN112015508 B CN 112015508B CN 202010890879 A CN202010890879 A CN 202010890879A CN 112015508 B CN112015508 B CN 112015508B
Authority
CN
China
Prior art keywords
screen
coverage
throwing
sensing data
distance sensing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010890879.7A
Other languages
Chinese (zh)
Other versions
CN112015508A (en
Inventor
吴承伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nubia Technology Co Ltd
Original Assignee
Nubia Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nubia Technology Co Ltd filed Critical Nubia Technology Co Ltd
Priority to CN202010890879.7A priority Critical patent/CN112015508B/en
Publication of CN112015508A publication Critical patent/CN112015508A/en
Application granted granted Critical
Publication of CN112015508B publication Critical patent/CN112015508B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/451Execution arrangements for user interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Position Input By Displaying (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

The invention discloses a screen-throwing interaction control method, equipment and a computer-readable storage medium, wherein the method comprises the following steps: judging that when the equipment is detected to be in a screen-throwing display state, acquiring distance sensing data of at least one surface of the equipment; then, analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface; then, generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark; and finally, regulating and controlling the operation items of the control object according to the dynamic information. The humanized screen-throwing interaction control scheme is realized, so that the user can more easily perform visual control on the display content of the screen-throwing end in the screen-throwing process, and the screen-throwing control experience of the user is improved.

Description

Screen-throwing interaction control method, equipment and computer-readable storage medium
Technical Field
The present invention relates to the field of mobile communications, and in particular, to a method and apparatus for controlling screen interaction, and a computer readable storage medium.
Background
In the prior art, with rapid development of intelligent terminal devices, the display requirement of users on the terminal devices is higher and higher, for example, in the process of playing games, enjoying video and other multimedia activities, in order to pursue better display effects or feeling of reality, the screen display content of the mobile terminal is usually put on a larger screen, for example, the game interface or video playing interface in progress in the mobile terminal is projected to a projector or a television, at this time, in order to enable the mobile terminal to perform corresponding control on the game interface or video playing interface in real time, the original game interface or video playing interface is generally reserved, or the state of displaying the projected content is activated from the state of screen extinction projection, at this time, the content displayed on the terminal device and the projection device by the game interface or video playing interface is completely the same. It is conceivable that only the main manipulation options need to be focused on the terminal device, since the user only needs to focus on the main display content on the projection device. Therefore, in the prior art, the screen-throwing control scheme has the advantages that the display scheme is single, the operation mode on the terminal equipment is not adaptively configured according to the specific practical requirement of screen throwing, the operation efficiency is reduced to a certain extent, and the user experience is poor.
Disclosure of Invention
In order to solve the technical defects in the prior art, the invention provides a screen-throwing interaction control method, which comprises the following steps:
when detecting that the equipment is in a screen-throwing display state, acquiring distance sensing data of at least one surface of the equipment;
analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface;
generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark;
regulating and controlling operation items of the control object according to the dynamic information.
Optionally, when the device is detected to be in the screen projection display state, acquiring distance sensing data of at least one surface of the device includes:
when the equipment is detected to be in a screen projection display state, acquiring first distance sensing data of a dot matrix type through one or more of a screen surface, a side surface and a back surface of the equipment, and/or detecting a holding state of the equipment;
and continuously acquiring the distance sensing data if the first distance sensing data meets a preset starting condition or the holding state is a preset state.
Optionally, the analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface includes:
Acquiring and analyzing the lattice type distance sensing data in real time;
and taking the point with the distance value within the preset distance value as the coverage point.
Optionally, the analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface further includes:
determining one or more blocky coverage surfaces formed by the coverage points in real time;
and determining the morphological state and the displacement state of each coverage surface in real time, wherein the morphological state comprises a coverage area variable of the coverage surface, and the displacement state comprises a coverage degree variable of the coverage surface.
Optionally, the generating the indication identifier of the screen throwing end according to the coverage information, and determining the control object corresponding to the indication identifier at the same time includes:
determining a covering position corresponding to the projection to the screen projection end according to the covering surface;
and selecting at least one control object at the covering position, and generating the indication mark in the area where the control object is located.
The invention also provides a screen-throwing interaction control device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the computer program is realized when being executed by the processor:
When detecting that the equipment is in a screen-throwing display state, acquiring distance sensing data of at least one surface of the equipment;
analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface;
generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark;
regulating and controlling operation items of the control object according to the dynamic information.
Optionally, the computer program is implemented when executed by the processor:
when the equipment is detected to be in a screen projection display state, acquiring first distance sensing data of a dot matrix type through one or more of a screen surface, a side surface and a back surface of the equipment, and/or detecting a holding state of the equipment;
and continuously acquiring the distance sensing data if the first distance sensing data meets a preset starting condition or the holding state is a preset state.
Optionally, the computer program is implemented when executed by the processor:
acquiring and analyzing the lattice type distance sensing data in real time;
and taking the point with the distance value within the preset distance value as the coverage point.
Optionally, the computer program is implemented when executed by the processor:
Determining one or more blocky coverage surfaces formed by the coverage points in real time;
determining the morphological state and the displacement state of each coverage surface in real time, wherein the morphological state comprises a coverage area variable of the coverage surface, and the displacement state comprises a coverage degree variable of the coverage surface;
determining a covering position corresponding to the projection to the screen projection end according to the covering surface;
and selecting at least one control object at the covering position, and generating the indication mark in the area where the control object is located.
The invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a screen-throwing interaction control program, and the screen-throwing interaction control program realizes the steps of the screen-throwing interaction control method according to any one of the above steps when being executed by a processor.
According to the screen-throwing interaction control method, the screen-throwing interaction control equipment and the computer readable storage medium, when the screen-throwing display state of the equipment is detected, distance sensing data of at least one surface of the equipment are obtained; then, analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface; then, generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark; and finally, regulating and controlling the operation items of the control object according to the dynamic information. The humanized screen-throwing interaction control scheme is realized, so that the user can more easily perform visual control on the display content of the screen-throwing end in the screen-throwing process, and the screen-throwing control experience of the user is improved.
Drawings
The invention will be further described with reference to the accompanying drawings and examples, in which:
fig. 1 is a schematic diagram of a hardware structure of a mobile terminal according to the present invention;
fig. 2 is a schematic diagram of a communication network system according to an embodiment of the present invention;
FIG. 3 is a flowchart of a first embodiment of the screen-shot interactive control method of the present invention;
FIG. 4 is a flow chart of a second embodiment of the screen-shot interactive control method of the present invention;
FIG. 5 is a flow chart of a third embodiment of a screen shot interaction control method of the present invention;
FIG. 6 is a flow chart of a fourth embodiment of the screen-shot interactive control method of the present invention;
fig. 7 is a flowchart of a fifth embodiment of the screen-shot interactive control method of the present invention.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In the following description, suffixes such as "module", "component", or "unit" for representing elements are used only for facilitating the description of the present invention, and have no specific meaning per se. Thus, "module," "component," or "unit" may be used in combination.
The terminal may be implemented in various forms. For example, the terminals described in the present invention may include mobile terminals such as cell phones, tablet computers, notebook computers, palm computers, personal digital assistants (Personal Digital Assistant, PDA), portable media players (Portable Media Player, PMP), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs, desktop computers, and the like.
The following description will be given taking a mobile terminal as an example, and those skilled in the art will understand that the configuration according to the embodiment of the present invention can be applied to a fixed type terminal in addition to elements particularly used for a moving purpose.
Referring to fig. 1, which is a schematic diagram of a hardware structure of a mobile terminal implementing various embodiments of the present invention, the mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an a/V (audio/video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111. Those skilled in the art will appreciate that the mobile terminal structure shown in fig. 1 is not limiting of the mobile terminal and that the mobile terminal may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
The following describes the components of the mobile terminal in detail with reference to fig. 1:
the radio frequency unit 101 may be used for receiving and transmitting signals during the information receiving or communication process, specifically, after receiving downlink information of the base station, processing the downlink information by the processor 110; and, the uplink data is transmitted to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 101 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication, global System for Mobile communications), GPRS (General Packet Radio Service ), CDMA2000 (Code Division Multiple Access, CDMA 2000), WCDMA (Wideband Code Division Multiple Access ), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access, time Division synchronous code Division multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution, frequency Division Duplex Long term evolution), and TDD-LTE (Time Division Duplexing-Long Term Evolution, time Division Duplex Long term evolution), etc.
WiFi belongs to a short-distance wireless transmission technology, and a mobile terminal can help a user to send and receive e-mails, browse web pages, access streaming media and the like through the WiFi module 102, so that wireless broadband Internet access is provided for the user. Although fig. 1 shows a WiFi module 102, it is understood that it does not belong to the necessary constitution of a mobile terminal, and can be omitted entirely as required within a range that does not change the essence of the invention.
The audio output unit 103 may convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output unit 103 may also provide audio output (e.g., a call signal reception sound, a message reception sound, etc.) related to a specific function performed by the mobile terminal 100. The audio output unit 103 may include a speaker, a buzzer, and the like.
The a/V input unit 104 is used to receive an audio or video signal. The a/V input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, the graphics processor 1041 processing image data of still pictures or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the graphics processor 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a phone call mode, a recording mode, a voice recognition mode, and the like, and can process such sound into audio data. The processed audio (voice) data may be converted into a format output that can be transmitted to the mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting the audio signal.
The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of ambient light, and the proximity sensor can turn off the display panel 1061 and/or the backlight when the mobile terminal 100 moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and direction when stationary, and can be used for applications of recognizing the gesture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; as for other sensors such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that may also be configured in the mobile phone, the detailed description thereof will be omitted.
The display unit 106 is used to display information input by a user or information provided to the user. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an Organic Light-Emitting Diode (OLED), or the like.
The user input unit 107 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the mobile terminal. In particular, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on the touch panel 1071 or thereabout by using any suitable object or accessory such as a finger, a stylus, etc.) and drive the corresponding connection device according to a predetermined program. The touch panel 1071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends the touch point coordinates to the processor 110, and can receive and execute commands sent from the processor 110. Further, the touch panel 1071 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. The user input unit 107 may include other input devices 1072 in addition to the touch panel 1071. In particular, other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, mouse, joystick, etc., as specifically not limited herein.
Further, the touch panel 1071 may overlay the display panel 1061, and when the touch panel 1071 detects a touch operation thereon or thereabout, the touch panel 1071 is transferred to the processor 110 to determine the type of touch event, and then the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of touch event. Although in fig. 1, the touch panel 1071 and the display panel 1061 are two independent components for implementing the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 may be integrated with the display panel 1061 to implement the input and output functions of the mobile terminal, which is not limited herein.
The interface unit 108 serves as an interface through which at least one external device can be connected with the mobile terminal 100. For example, the external devices may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.
Memory 109 may be used to store software programs as well as various data. The memory 109 may mainly include a storage program area that may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and a storage data area; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
The processor 110 is a control center of the mobile terminal, connects various parts of the entire mobile terminal using various interfaces and lines, and performs various functions of the mobile terminal and processes data by running or executing software programs and/or modules stored in the memory 109 and calling data stored in the memory 109, thereby performing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 110.
The mobile terminal 100 may further include a power source 111 (e.g., a battery) for supplying power to the respective components, and preferably, the power source 111 may be logically connected to the processor 110 through a power management system, so as to perform functions of managing charging, discharging, and power consumption management through the power management system.
Although not shown in fig. 1, the mobile terminal 100 may further include a bluetooth module or the like, which is not described herein.
In order to facilitate understanding of the embodiments of the present invention, a communication network system on which the mobile terminal of the present invention is based will be described below.
Referring to fig. 2, fig. 2 is a schematic diagram of a communication network system according to an embodiment of the present invention, where the communication network system is an LTE system of a general mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an e-UTRAN (Evolved UMTS Terrestrial Radio Access Network ) 202, an epc (Evolved Packet Core, evolved packet core) 203, and an IP service 204 of an operator that are sequentially connected in communication.
Specifically, the UE201 may be the terminal 100 described above, and will not be described herein.
The E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. The eNodeB2021 may be connected with other eNodeB2022 by a backhaul (e.g., an X2 interface), the eNodeB2021 is connected to the EPC203, and the eNodeB2021 may provide access from the UE201 to the EPC 203.
EPC203 may include MME (Mobility Management Entity ) 2031, hss (Home Subscriber Server, home subscriber server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, pgw (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function, policy and tariff function entity) 2036, and so on. The MME2031 is a control node that handles signaling between the UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as home location registers (not shown) and to hold user specific information about service characteristics, data rates, etc. All user data may be sent through SGW2034 and PGW2035 may provide IP address allocation and other functions for UE201, PCRF2036 is a policy and charging control policy decision point for traffic data flows and IP bearer resources, which selects and provides available policy and charging control decisions for a policy and charging enforcement function (not shown).
IP services 204 may include the internet, intranets, IMS (IP Multimedia Subsystem ), or other IP services, etc.
Although the LTE system is described above as an example, it should be understood by those skilled in the art that the present invention is not limited to LTE systems, but may be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, and future new network systems.
Based on the above mobile terminal hardware structure and the communication network system, various embodiments of the method of the present invention are provided.
Example 1
Fig. 3 is a flowchart of a first embodiment of the screen-shot interactive control method of the present invention. A screen-throwing interaction control method comprises the following steps:
s1, when detecting that equipment is in a screen-throwing display state, acquiring distance sensing data of at least one surface of the equipment;
s2, analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface;
s3, generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark;
s4, regulating and controlling operation items of the control object according to the dynamic information.
In this embodiment, first, when detecting that a device is in a screen-projection display state, distance sensing data of at least one surface of the device is acquired; then, analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface; then, generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark; and finally, regulating and controlling the operation items of the control object according to the dynamic information.
Specifically, for example, firstly, an area above a screen is detected through an ultrasonic or infrared sensing device, and when a user's hand or finger stays above the screen or strokes on the screen, the sensing device captures the area where the hand or finger is located or the area where the finger presses and maps the area to a mobile phone end; then, the identified area dynamically covered by the hand or the finger is projected to the television end through a screen projection technology, a user can intuitively see the area where the finger is or the position where the key point is located when the user performs touch typing on the television, and the user can clearly see the general azimuth and accurate click position of the finger operated by the user on the screen at every moment in the whole operation process, so that the screen projection blind operation experience is improved; it can be seen that, in this embodiment, for the general situation, when the method is mainly used for a horizontal screen game, the distance sensing device is mainly concentrated on the left and right half screens of the screen, the middle dotted line is the dividing line, the sensing device is placed in the area which can be covered approximately when the mobile phone is held, the larger the laying range and density are, and the higher the sensitivity and accuracy can be sensed.
The method has the advantages that distance sensing data of at least one face of the equipment are obtained through judging when the equipment is detected to be in a screen-throwing display state; then, analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface; then, generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark; and finally, regulating and controlling the operation items of the control object according to the dynamic information. The humanized screen-throwing interaction control scheme is realized, so that the user can more easily perform visual control on the display content of the screen-throwing end in the screen-throwing process, and the screen-throwing control experience of the user is improved.
Example two
Fig. 4 is a flowchart of a second embodiment of the screen-projection interaction control method according to the present invention, based on the above embodiment, optionally, when detecting that the device is in a screen-projection display state, acquiring distance sensing data of at least one surface of the device includes:
s11, when the equipment is detected to be in a screen projection display state, acquiring first distance sensing data of a dot matrix type through one or more of a screen surface, a side surface and a back surface of the equipment, and or detecting a holding state of the equipment;
and S12, continuously acquiring the distance sensing data if the first distance sensing data meets a preset starting condition and/or the holding state is a preset state.
In this embodiment, first, when it is detected that a device is in a screen-projection display state, first distance sensing data of a dot matrix is acquired through one or more of a screen surface, a side surface, and a back surface of the device, and or a holding state of the device is detected; and then, continuously acquiring the distance sensing data if the first distance sensing data meets a preset starting condition and/or the holding state is a preset state.
Optionally, when the device is detected to be in a screen-throwing display state, acquiring first distance sensing data of a dot matrix type through one or more of a screen surface, a side surface and a back surface of the device, wherein the first distance sensing data is used for determining whether to start visual control of the screen-throwing side in the screen-throwing state;
Optionally, when the device is detected to be in a screen-throwing display state, acquiring first distance sensing data of a dot matrix through one or more of a screen surface, a side surface and a back surface of the device, and/or detecting a holding state of the device, that is, when a user adopts a certain holding mode, the embodiment may judge whether to start visual control of a screen throwing side in the screen throwing state through the first distance sensing data.
The method has the advantages that when the equipment is detected to be in a screen projection display state, first distance sensing data of a dot matrix type are obtained through one or more of a screen surface, a side surface and a back surface of the equipment, and or the holding state of the equipment is detected; and then, continuously acquiring the distance sensing data if the first distance sensing data meets a preset starting condition and/or the holding state is a preset state. The screen-throwing interaction control scheme is more humanized, so that the user can more easily control the display content of the screen-throwing end in a visual manner in the screen-throwing process, and the screen-throwing control experience of the user is improved.
Example III
Fig. 5 is a flowchart of a third embodiment of the screen-projection interaction control method according to the present invention, based on the above embodiment, the analyzing the distance sensing data to obtain coverage information and dynamic information of at least one of the surfaces includes:
S21, acquiring and analyzing the lattice type distance sensing data in real time;
s22, taking the point with the distance value within the preset distance value as the coverage point.
In this embodiment, first, the lattice type distance sensing data is acquired and analyzed in real time; then, a point whose distance value is within a preset distance value is taken as a coverage point.
Alternatively, a point whose distance value is within a preset distance value is taken as a coverage point, for example, a point within 2cm from the screen is taken as a coverage point;
optionally, a point whose distance value is within a preset distance range value is taken as a coverage point, for example, a point between 1-2cm distance from the screen is taken as a coverage point;
optionally, when the distance value is not within the preset distance range value, reminding the user to readjust the position of the screen-throwing control finger through vibration.
The method has the beneficial effects that the lattice type distance sensing data is obtained and analyzed in real time; then, a point whose distance value is within a preset distance value is taken as a coverage point. The screen-throwing interaction control scheme is more humanized, so that the user can more easily control the display content of the screen-throwing end in a visual manner in the screen-throwing process, and the screen-throwing control experience of the user is improved.
Example IV
Fig. 6 is a flowchart of a fourth embodiment of the screen-projection interaction control method according to the present invention, based on the above embodiment, the analyzing the distance sensing data to obtain coverage information and dynamic information of at least one of the surfaces, and further includes:
s23, determining one or more blocky coverage surfaces formed by the coverage points in real time;
s24, determining the morphological state and the displacement state of each coverage surface in real time, wherein the morphological state comprises coverage area variables of the coverage surfaces, and the displacement state comprises coverage degree variables of the coverage surfaces.
In this embodiment, first, one or more block-shaped coverage surfaces composed of the coverage points are determined in real time; and then determining the morphological state and the displacement state of each coverage surface in real time, wherein the morphological state comprises a coverage area variable of the coverage surface, and the displacement state comprises a coverage degree variable of the coverage surface.
Optionally, determining one or more block-shaped coverage surfaces formed by the coverage points in real time, wherein it is understood that the coverage surfaces can be coverage surfaces generated by different surfaces of the terminal equipment shell;
optionally, determining, in real time, a morphological state of each coverage surface for coverage surfaces generated by different surfaces, wherein a part of the surfaces comprises coverage area variables of the coverage surface; and determining the displacement state of each covering surface on the other part of the surfaces, wherein the displacement state comprises the covering degree variable of the covering surface, so that different control recognition bases and subsequent control modes of different surfaces of the shell are realized.
The embodiment has the beneficial effects that the coverage of one or more blocks formed by the coverage points is determined in real time; and then determining the morphological state and the displacement state of each coverage surface in real time, wherein the morphological state comprises a coverage area variable of the coverage surface, and the displacement state comprises a coverage degree variable of the coverage surface. The screen-throwing interaction control scheme is more humanized, so that the user can more easily control the display content of the screen-throwing end in a visual manner in the screen-throwing process, and the screen-throwing control experience of the user is improved.
Example five
Fig. 7 is a flowchart of a fifth embodiment of a screen-throwing interaction control method according to the present invention, based on the foregoing embodiment, the generating, according to the coverage information, an indication identifier of a screen-throwing end, and determining a control object corresponding to the indication identifier, where the determining includes:
s31, determining a covering position corresponding to the projection to the screen projection end according to the covering surface;
s32, selecting at least one control object at the covering position, and generating the indication mark in the area where the control object is located.
In this embodiment, first, determining, according to the coverage surface, a coverage position corresponding to the projection to the screen projection end; then, at least one control object is selected at the covering position, and the indication mark is generated in the area where the control object is located.
Optionally, determining a covering position corresponding to the projection to the screen projection end according to the generated covering surfaces corresponding to different surfaces of the shell;
optionally, selecting at least one control object at a covering position generated on the first surface of the shell, and generating a first indication mark in a region where the control object is located, wherein the first indication mark is used for selecting a display target on the projection side;
optionally, at least one control object is selected at the covering position generated on the second surface of the shell, and a second indication mark is generated in the area where the control object is located, wherein the second indication mark is used for moving (or dragging) the display target on the projection side;
optionally, at least one control object is selected at the coverage position generated on the third surface of the shell, and a third indication mark is generated in the area where the control object is located, wherein the third indication mark is used for deleting the display target on the projection side.
The method has the beneficial effects that the coverage position corresponding to the projection to the screen projection end is determined through the coverage surface; then, at least one control object is selected at the covering position, and the indication mark is generated in the area where the control object is located. The screen-throwing interaction control scheme is more humanized, so that the user can more easily control the display content of the screen-throwing end in a visual manner in the screen-throwing process, and the screen-throwing control experience of the user is improved.
Example six
Based on the above embodiment, the present invention further provides a screen-throwing interaction control device, where the device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the computer program is implemented when executed by the processor:
when detecting that the equipment is in a screen-throwing display state, acquiring distance sensing data of at least one surface of the equipment;
analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface;
generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark;
regulating and controlling operation items of the control object according to the dynamic information.
In this embodiment, first, when detecting that a device is in a screen-projection display state, distance sensing data of at least one surface of the device is acquired; then, analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface; then, generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark; and finally, regulating and controlling the operation items of the control object according to the dynamic information.
Specifically, for example, firstly, an area above a screen is detected through an ultrasonic or infrared sensing device, and when a user's hand or finger stays above the screen or strokes on the screen, the sensing device captures the area where the hand or finger is located or the area where the finger presses and maps the area to a mobile phone end; then, the identified area dynamically covered by the hand or the finger is projected to the television end through a screen projection technology, a user can intuitively see the area where the finger is or the position where the key point is located when the user performs touch typing on the television, and the user can clearly see the general azimuth and accurate click position of the finger operated by the user on the screen at every moment in the whole operation process, so that the screen projection blind operation experience is improved; it can be seen that, in this embodiment, for the general situation, when the method is mainly used for a horizontal screen game, the distance sensing device is mainly concentrated on the left and right half screens of the screen, the middle dotted line is the dividing line, the sensing device is placed in the area which can be covered approximately when the mobile phone is held, the larger the laying range and density are, and the higher the sensitivity and accuracy can be sensed.
The method has the advantages that distance sensing data of at least one face of the equipment are obtained through judging when the equipment is detected to be in a screen-throwing display state; then, analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface; then, generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark; and finally, regulating and controlling the operation items of the control object according to the dynamic information. The humanized screen-throwing interaction control scheme is realized, so that the user can more easily perform visual control on the display content of the screen-throwing end in the screen-throwing process, and the screen-throwing control experience of the user is improved.
Example seven
Based on the above embodiments, the computer program when executed by the processor implements:
when the equipment is detected to be in a screen projection display state, acquiring first distance sensing data of a dot matrix type through one or more of a screen surface, a side surface and a back surface of the equipment, and/or detecting a holding state of the equipment;
and continuously acquiring the distance sensing data if the first distance sensing data meets a preset starting condition or the holding state is a preset state.
In this embodiment, first, when it is detected that a device is in a screen-projection display state, first distance sensing data of a dot matrix is acquired through one or more of a screen surface, a side surface, and a back surface of the device, and or a holding state of the device is detected; and then, continuously acquiring the distance sensing data if the first distance sensing data meets a preset starting condition and/or the holding state is a preset state.
Optionally, when the device is detected to be in a screen-throwing display state, acquiring first distance sensing data of a dot matrix type through one or more of a screen surface, a side surface and a back surface of the device, wherein the first distance sensing data is used for determining whether to start visual control of the screen-throwing side in the screen-throwing state;
Optionally, when the device is detected to be in a screen-throwing display state, acquiring first distance sensing data of a dot matrix through one or more of a screen surface, a side surface and a back surface of the device, and/or detecting a holding state of the device, that is, when a user adopts a certain holding mode, the embodiment may judge whether to start visual control of a screen throwing side in the screen throwing state through the first distance sensing data.
The method has the advantages that when the equipment is detected to be in a screen projection display state, first distance sensing data of a dot matrix type are obtained through one or more of a screen surface, a side surface and a back surface of the equipment, and or the holding state of the equipment is detected; and then, continuously acquiring the distance sensing data if the first distance sensing data meets a preset starting condition and/or the holding state is a preset state. The screen-throwing interaction control scheme is more humanized, so that the user can more easily control the display content of the screen-throwing end in a visual manner in the screen-throwing process, and the screen-throwing control experience of the user is improved.
Example eight
Based on the above embodiments, the computer program when executed by the processor implements:
Acquiring and analyzing the lattice type distance sensing data in real time;
and taking the point with the distance value within the preset distance value as the coverage point.
In this embodiment, first, the lattice type distance sensing data is acquired and analyzed in real time; then, a point whose distance value is within a preset distance value is taken as a coverage point.
Alternatively, a point whose distance value is within a preset distance value is taken as a coverage point, for example, a point within 2cm from the screen is taken as a coverage point;
optionally, a point whose distance value is within a preset distance range value is taken as a coverage point, for example, a point between 1-2cm distance from the screen is taken as a coverage point;
optionally, when the distance value is not within the preset distance range value, reminding the user to readjust the position of the screen-throwing control finger through vibration.
The method has the beneficial effects that the lattice type distance sensing data is obtained and analyzed in real time; then, a point whose distance value is within a preset distance value is taken as a coverage point. The screen-throwing interaction control scheme is more humanized, so that the user can more easily control the display content of the screen-throwing end in a visual manner in the screen-throwing process, and the screen-throwing control experience of the user is improved.
Example nine
Based on the above embodiments, the computer program when executed by the processor implements:
determining one or more blocky coverage surfaces formed by the coverage points in real time;
determining the morphological state and the displacement state of each coverage surface in real time, wherein the morphological state comprises a coverage area variable of the coverage surface, and the displacement state comprises a coverage degree variable of the coverage surface;
determining a covering position corresponding to the projection to the screen projection end according to the covering surface;
and selecting at least one control object at the covering position, and generating the indication mark in the area where the control object is located.
In this embodiment, first, one or more block-shaped coverage surfaces composed of the coverage points are determined in real time; and then determining the morphological state and the displacement state of each coverage surface in real time, wherein the morphological state comprises a coverage area variable of the coverage surface, and the displacement state comprises a coverage degree variable of the coverage surface.
Optionally, determining one or more block-shaped coverage surfaces formed by the coverage points in real time, wherein it is understood that the coverage surfaces can be coverage surfaces generated by different surfaces of the terminal equipment shell;
Optionally, determining, in real time, a morphological state of each coverage surface for coverage surfaces generated by different surfaces, wherein a part of the surfaces comprises coverage area variables of the coverage surface; and determining the displacement state of each covering surface on the other part of the surfaces, wherein the displacement state comprises the covering degree variable of the covering surface, so that different control recognition bases and subsequent control modes of different surfaces of the shell are realized.
In another embodiment, first, determining a coverage position corresponding to a projection to the projection end according to the coverage surface; then, at least one control object is selected at the covering position, and the indication mark is generated in the area where the control object is located.
Optionally, determining a covering position corresponding to the projection to the screen projection end according to the generated covering surfaces corresponding to different surfaces of the shell;
optionally, selecting at least one control object at a covering position generated on the first surface of the shell, and generating a first indication mark in a region where the control object is located, wherein the first indication mark is used for selecting a display target on the projection side;
optionally, at least one control object is selected at the covering position generated on the second surface of the shell, and a second indication mark is generated in the area where the control object is located, wherein the second indication mark is used for moving (or dragging) the display target on the projection side;
Optionally, at least one control object is selected at the coverage position generated on the third surface of the shell, and a third indication mark is generated in the area where the control object is located, wherein the third indication mark is used for deleting the display target on the projection side.
The method has the beneficial effects that the coverage position corresponding to the projection to the screen projection end is determined through the coverage surface; then, at least one control object is selected at the covering position, and the indication mark is generated in the area where the control object is located. The screen-throwing interaction control scheme is more humanized, so that the user can more easily control the display content of the screen-throwing end in a visual manner in the screen-throwing process, and the screen-throwing control experience of the user is improved.
Examples ten
Based on the above embodiment, the present invention further provides a computer readable storage medium, where a screen-throwing interaction control program is stored, where the screen-throwing interaction control program when executed by a processor implements the steps of the screen-throwing interaction control method according to any one of the above embodiments.
According to the screen-throwing interaction control method, the screen-throwing interaction control equipment and the computer readable storage medium, when the screen-throwing display state of the equipment is detected, distance sensing data of at least one surface of the equipment are obtained; then, analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface; then, generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark; and finally, regulating and controlling the operation items of the control object according to the dynamic information. The humanized screen-throwing interaction control scheme is realized, so that the user can more easily perform visual control on the display content of the screen-throwing end in the screen-throwing process, and the screen-throwing control experience of the user is improved.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.

Claims (7)

1. The screen-throwing interaction control method is characterized by comprising the following steps of:
when detecting that the equipment is in a screen-throwing display state, acquiring distance sensing data of at least one surface of the equipment;
analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface;
generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark;
regulating and controlling operation items of the control object according to the dynamic information;
the analyzing the distance sensing data to obtain at least one of coverage information and dynamic information of the surface comprises the following steps:
acquiring and analyzing first distance sensing data of a matrix through one or more of a screen surface, a side surface and a back surface of the equipment in real time;
Taking a point with the distance value within a preset distance value as a coverage point;
determining one or more blocky coverage surfaces formed by the coverage points in real time;
and determining the morphological state and the displacement state of each coverage surface in real time, wherein the morphological state comprises a coverage area variable of the coverage surface, and the displacement state comprises a coverage degree variable of the coverage surface.
2. The method for controlling screen-projection interaction according to claim 1, wherein when the device is detected to be in a screen-projection display state, acquiring distance sensing data of at least one face of the device comprises:
when the equipment is detected to be in a screen-throwing display state, acquiring the first distance sensing data of the dot matrix and/or detecting the holding state of the equipment;
and continuously acquiring the distance sensing data if the first distance sensing data meets a preset starting condition and/or the holding state is a preset state.
3. The screen projection interaction control method according to claim 2, wherein the generating the indication identifier of the screen projection end according to the coverage information, and determining the control object corresponding to the indication identifier at the same time, includes:
Determining a covering position corresponding to the projection to the screen projection end according to the covering surface;
and selecting at least one control object at the covering position, and generating the indication mark in the area where the control object is located.
4. A screen shot interactive control device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being implemented when executed by the processor:
when detecting that the equipment is in a screen-throwing display state, acquiring distance sensing data of at least one surface of the equipment;
analyzing the distance sensing data to obtain coverage information and dynamic information of at least one surface;
generating an indication mark of a screen throwing end according to the coverage information, and determining a control object corresponding to the indication mark;
regulating and controlling operation items of the control object according to the dynamic information;
the analyzing the distance sensing data to obtain at least one of coverage information and dynamic information of the surface comprises the following steps:
acquiring and analyzing first distance sensing data of a matrix through one or more of a screen surface, a side surface and a back surface of the equipment in real time;
Taking a point with the distance value within a preset distance value as a coverage point;
determining one or more blocky coverage surfaces formed by the coverage points in real time;
and determining the morphological state and the displacement state of each coverage surface in real time, wherein the morphological state comprises a coverage area variable of the coverage surface, and the displacement state comprises a coverage degree variable of the coverage surface.
5. The screen shot interactive control device of claim 4, wherein the computer program when executed by the processor implements:
when the equipment is detected to be in a screen-throwing display state, acquiring the first distance sensing data of the dot matrix and/or detecting the holding state of the equipment;
and continuously acquiring the distance sensing data if the first distance sensing data meets a preset starting condition and/or the holding state is a preset state.
6. The screen shot interactive control device of claim 5, wherein the computer program when executed by the processor implements:
determining a covering position corresponding to the projection to the screen projection end according to the covering surface;
and selecting at least one control object at the covering position, and generating the indication mark in the area where the control object is located.
7. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a screen-shot interaction control program, which when executed by a processor, implements the steps of the screen-shot interaction control method according to any one of claims 1 to 3.
CN202010890879.7A 2020-08-29 2020-08-29 Screen-throwing interaction control method, equipment and computer-readable storage medium Active CN112015508B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010890879.7A CN112015508B (en) 2020-08-29 2020-08-29 Screen-throwing interaction control method, equipment and computer-readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010890879.7A CN112015508B (en) 2020-08-29 2020-08-29 Screen-throwing interaction control method, equipment and computer-readable storage medium

Publications (2)

Publication Number Publication Date
CN112015508A CN112015508A (en) 2020-12-01
CN112015508B true CN112015508B (en) 2024-01-09

Family

ID=73503241

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010890879.7A Active CN112015508B (en) 2020-08-29 2020-08-29 Screen-throwing interaction control method, equipment and computer-readable storage medium

Country Status (1)

Country Link
CN (1) CN112015508B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112905136A (en) * 2021-03-11 2021-06-04 北京小米移动软件有限公司 Screen projection control method and device and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109177899A (en) * 2018-08-30 2019-01-11 上海天马微电子有限公司 Interaction method of vehicle-mounted display device and vehicle-mounted display device
CN110362231A (en) * 2019-07-12 2019-10-22 腾讯科技(深圳)有限公司 The method and device that new line touch control device, image are shown
CN111061445A (en) * 2019-04-26 2020-04-24 华为技术有限公司 Screen projection method and computing equipment

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010030668A1 (en) * 2000-01-10 2001-10-18 Gamze Erten Method and system for interacting with a display

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109177899A (en) * 2018-08-30 2019-01-11 上海天马微电子有限公司 Interaction method of vehicle-mounted display device and vehicle-mounted display device
CN111061445A (en) * 2019-04-26 2020-04-24 华为技术有限公司 Screen projection method and computing equipment
CN110362231A (en) * 2019-07-12 2019-10-22 腾讯科技(深圳)有限公司 The method and device that new line touch control device, image are shown

Also Published As

Publication number Publication date
CN112015508A (en) 2020-12-01

Similar Documents

Publication Publication Date Title
CN108566479B (en) Screen state control method, mobile terminal and computer readable storage medium
CN109195213B (en) Mobile terminal screen control method, mobile terminal and computer readable storage medium
CN109471579B (en) Terminal screen information layout adjustment method and device, mobile terminal and storage medium
CN108449513B (en) Interactive regulation and control method, equipment and computer readable storage medium
CN112102780A (en) Display frame rate regulation and control method, device and computer readable storage medium
CN109542325B (en) Double-sided screen touch method, double-sided screen terminal and readable storage medium
CN110052024B (en) Game vibration adjusting method, adjusting device, mobile terminal and storage medium
CN110096213B (en) Terminal operation method based on gestures, mobile terminal and readable storage medium
CN109389394B (en) Multi-screen payment control method, equipment and computer readable storage medium
CN109683778B (en) Flexible screen control method and device and computer readable storage medium
CN109491577B (en) Holding interaction method and device and computer readable storage medium
CN109445945B (en) Memory allocation method of application program, mobile terminal, server and storage medium
CN108984063B (en) Desktop interaction regulation and control method and device and computer readable storage medium
CN112437472B (en) Network switching method, equipment and computer readable storage medium
CN111443818B (en) Screen brightness regulation and control method, equipment and computer readable storage medium
CN112015508B (en) Screen-throwing interaction control method, equipment and computer-readable storage medium
CN109710149B (en) Interactive display method, equipment and computer readable storage medium
CN109462829B (en) Call transfer method, device and computer readable storage medium
CN109710168B (en) Screen touch method and device and computer readable storage medium
CN109561221B (en) Call control method, device and computer readable storage medium
CN109711850B (en) Secure payment method, device and computer readable storage medium
CN109669616B (en) Side screen interaction control method and device and computer readable storage medium
CN109684020B (en) Theme switching method, device and computer readable storage medium
CN107256108B (en) Screen splitting method, device and computer readable storage medium
CN112947831B (en) Screen-throwing reverse control method, mobile terminal and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant