WO2011079640A1

WO2011079640A1 - Method for determining whether target point belongs to flat plane or not, mouse and touch screen

Info

Publication number: WO2011079640A1
Application number: PCT/CN2010/078055
Authority: WO
Inventors: 胡世曦
Original assignee: Hu Shixi
Priority date: 2009-12-29
Filing date: 2010-10-25
Publication date: 2011-07-07
Also published as: CN101882014A; CN101963848B; CN101963848A; CN101776953A

Abstract

A method for determining whether a target point belongs to a flat plane or not relates to a display screen control technology, the method is that two or more camera heads shoot the target points on the flat plane and both sides of the flat plane, and an image identification unit identifies the image of the target point from the images shot by the camera heads, wherein the target point belongs to the flat plane if the positions for the same image points of the target point corresponding to different camera heads are the same, otherwise the target point locates on both sides of the flat plane. A new mouse which is combined with a keyboard and controls movement of a cursor on the display screen through movement of fingers directly is provided to avoid deficiency of prior mouse and make control of a computer more conveniently so as to be more practical and make low cost, accurate positioning and no effect on screen brightness for the touch screen.

Description

Method for judging whether target point belongs to plane and mouse and touch screen

Technical field:

The present invention relates to a method of determining whether a target point belongs to a plane method, and thereby positioning a target point, and a finger mouse and a touch screen fused with a keyboard to which the method is applied.

Background technique:

Since the operating system entered the graphical user interface, the mouse has become one of the most important peripheral input devices in addition to the keyboard due to the need to control the cursor displayed on the display.

Optical mouse and computer are often connected by COM, PS/2 or USB interface, and the cursor displayed on the display moves with the movement of the optical mouse.

The structure of the existing optical mouse comprises a housing, a light emitting diode, an optical lens, a light receiving processing device, a button and a printed circuit board. The light emitting diode is disposed in the housing and emits light toward an opening at the bottom of the housing, so that the photoelectric When the mouse is placed on a desktop or mouse pad, light is reflected when the light hits the table or mouse pad. The light lens is used to collect the reflected light emitted by the self-illuminating diode and reflected by the desktop or mouse pad. The light receiving processing device receives the reflected light collected by the optical lens, and the light receiving processing device has a light sensor unit and an image processing unit; the light sensor unit is constituted by a charge coupled device (CCD), and thus can be acquired after being collected via the optical lens The image, the typical light processing device chip is the optical light receiving processing device chip of HDNS-2000, HDNS-2051 and other optical mouse of American AGILENT Company. The light receiving processing device chip such as HDNS-2000 type chip can be taken from the photosensitive window every second. Pick up 1500 frames of images. Next, by the image processing unit for the light sensor The images extracted at different times are compared to the difference, thereby determining the displacement, speed, etc. of the mouse movement, and converting it into an electrical signal corresponding to the computer's corresponding coordinates (ΔΧ, ΔΥ), and then sending it to the computer to control the cursor. The movement.

When the button of the mouse is pressed by the user, the button will interlock the micro switch to control the cursor to perform a click action. Since the optical mouse uses optical technology to determine the moving position, it is not easy to contaminate the dust and distinguish The advantages of high speed, smooth movement, etc., have gradually replaced the traditional wheel mouse.

However, users of optical mice need a certain amount of space to hold the mouse along a plane to effectively control the movement of the cursor, so it must take up a lot of space, which is not very convenient to use, especially for laptops that are often carried around. It is as light and compact as possible, and the mouse takes up a certain amount of space or weight, which is not convenient enough to carry and use.

If both typing and mouse use are required, the user's hand has to move back and forth between the keyboard and the mouse, which reduces user productivity.

Especially important, long-term operation of the mobile mouse is not only easy to make the muscles of the wrist and the joints and joints are sore, it is more likely to cause sports injuries due to long-term accumulation.

As the latest computer input device, the touch screen is the simplest, most convenient and natural way of human-computer interaction. The application range of touch screen is very broad, mainly the inquiry of public information; business inquiry of departments such as telecommunication bureau, tax bureau, bank, electric power, etc.; information inquiry of city street; also applied to leading office, industrial control, military command, video game , ordering songs, multimedia teaching, real estate pre-sales, etc.

From the technical principle to distinguish the touch screen, it can be divided into five basic categories: vector pressure sensing technology touch screen, resistance technology touch screen, capacitive technology touch screen, infrared technology touch screen, surface acoustic wave technology Touch screen. Among them, the vector pressure sensing technology touch screen has withdrawn from the historical stage; the infrared technology touch screen is inexpensive, but its frame is fragile, easy to produce light interference, and distortion under the surface condition; the capacitive technology touch screen design is reasonable, but its image distortion problem is difficult to obtain. The fundamental solution is that the position of the resistive touch screen is accurate, but its price is quite high, and it is afraid of scratching. The surface acoustic wave touch screen solves various defects of the touch screen in the past, and is not easy to be damaged. It is suitable for various occasions. The disadvantage is the screen surface. If there are water droplets and dust, the touch screen will become dull or even not working.

The main part of the resistive touch screen is a resistive film screen that is very compatible with the surface of the display. It is a multi-layer composite film with a glass or hard plastic plate as the base layer and a transparent oxide metal on the surface (transparent conductive). Resistor) The conductive layer is covered with a layer of externally hardened, smooth, anti-scratch plastic layer, a coating on its inner surface, and a number of tiny (less than 1/1000 inch) transparent between them. The isolation point separates the two conductive layers from each other. When the finger touches the screen, the two conductive layers have contact at the touch point position, the resistance changes, and signals are generated in both X and Y directions, and then sent to the touch screen controller, and the controller detects the contact and calculates The position of (X, Y) is output, and then it works according to the way of simulating the mouse.

The capacitive touch screen is a four-layer composite glass screen. The inner surface and the interlayer of the glass screen are coated with a layer of IT0. The outermost layer is a thin layer of bauxite glass protective layer. The interlayer IOT coating is used as the working surface and is taken out at the four corners. Four electrodes, the inner layer ιτο is a shielding layer to ensure a good working environment. When the finger touches the metal layer, the user and the surface of the touch screen form a coupling capacitor due to the electric field of the human body. For the high-frequency current, the capacitor is a direct conductor, and the finger sucks a small current from the contact point. This current is discharged from the electrodes on the four corners of the touch screen, and the current flowing through the four electrodes is proportional to the distance from the finger to the four corners. The controller calculates the position of the touch point by accurately calculating the ratio of the four currents. .

The capacitive screen is heavily reflective, and the four-layer composite touch screen of capacitive technology transmits light to each wavelength. The rate is uneven, there is a problem of color distortion, and the reflection of light between layers also causes blurring of image characters. In principle, the capacitive screen uses the human body as an electrode of a capacitor element. When a conductor is close to a capacitor with a sufficient capacitance between the working surface of the interlayer, the current flowing away is enough to cause the capacitive screen to malfunction. . When a large area of the palm or hand-held conductor is close to the capacitive screen instead of touching, it can cause the capacitive screen to malfunction. In humid weather, this situation is particularly serious. Hold the monitor, the palm is close to the display within 7 cm or If the body is within 15 cm of the display, it can cause malfunction of the capacitive screen. Another disadvantage of capacitive screens is that they do not respond when touched with a gloved hand or a non-conductive object, because of the addition of a more insulating medium. The main disadvantage of the capacitive screen is drift: When the ambient temperature and humidity change, the environmental field changes, causing the drift of the capacitive screen, resulting in inaccuracy. For example: When the monitor is turned on, the temperature rise will cause drift: when the user touches the screen, the other hand or the side of the body will drift near the display; the larger object near the capacitive touch screen will move back and drift, if you touch it, if someone is watching it, It will cause drift; the drift of the capacitive screen is a technical innate deficiency. The environmental potential surface (including the user's body) is far away from the capacitive touch screen, but much larger than the finger area. They directly affect the measurement of the touch position. .

The infrared touch screen uses an infrared matrix densely packed in the X and Y directions to detect and locate the user's touch. The infrared touch screen is provided with a circuit board outer frame on the front side of the display. The circuit board arranges the infrared transmitting tube and the infrared receiving tube on the four sides of the screen, and one-to-one correspondingly forms an infrared matrix which is horizontally and vertically crossed. When the user touches the screen, the finger blocks the two infrared rays passing through the position, so that the position of the touch point on the screen can be judged. Any touch object can change the infrared light on the contact to achieve touch screen operation.

The three corners of the surface acoustic wave touch screen are respectively affixed with transducers for transmitting and receiving acoustic waves in the X and Y directions (transducers: made of special ceramic materials, divided into transmitting transducers and receiving transducers. The electrical signal sent by the controller through the touch screen cable is converted into acoustic energy and the surface acoustic wave condensed by the reflective fringes becomes an electrical signal. ), the four sides are engraved with reflective stripes that reflect the surface ultrasonic waves. When a finger or a soft object touches the screen, part of the sound wave energy is absorbed, and then the received signal is changed, and the X, Y coordinates of the touch are obtained by the processing of the controller.

Optical touch screen: The CCD camera installed in the upper left corner of the top of the screen emits light through the LED light. It is reflected by the surrounding reflection bar and enters the CCD camera in the upper right corner. Similarly, the light emitted by the CCD camera in the upper right corner is transmitted to the left side. In the CCD camera, the dense light forms a light net in the touch area, and the space between the multiple reflected light is within 1 inch. When touching a little, the emitted light and the received light of the point form an angle, and the CCD camera at both ends and the two rays and the line formed between the two cameras form two angles, so that the point is accurate. The coordinates are entered by the controller to implement a touch response.

Summary of the invention:

The object of the present invention is to disclose a method for judging whether a point belongs to a plane, and a mouse device and a touch screen device using the method; to avoid the inadequacy of the existing mouse, and to provide a finger-integrated movement with the keyboard The new mouse device that directly controls the cursor movement on the display screen makes the computer control more convenient and thus more practical; the touch screen has low cost and accurate positioning, and has no influence on the screen brightness.

The object of the present invention can be achieved by a method for determining whether a target point belongs to a plane, two or more cameras photographing target points on both sides of the plane and the plane, and the image recognition unit is from the above camera. The image of the target point is recognized in the captured image. If the position of the same image point corresponding to the different cameras is the same, the target point belongs to the plane. Otherwise, the target point is located on both sides of the plane.

The method for determining whether a target point belongs to a plane, and determining that the target point corresponds to different There are two methods for correcting the position of the same image point of the camera: a. Superimposing the images taken by multiple cameras. If the image positions of the target points coincide, the position is the same, otherwise, the position is different; b according to the imaging point The correspondence between the position of the position and the actual position on the plane, from the position of the image of the target point in the image, the actual position of the same image point on the plane is calculated, and the same image point calculated from the images taken by the plurality of cameras is compared. s position.

A mouse device includes a left button and a right button, and is characterized by:

Two or more cameras: photographing moving objects above the work plane and the work plane; image recognition unit: for recognizing an image of the moving object from the work plane background image;

Judging unit: for judging whether the moving object is in contact with the working plane;

Calculation unit: if the moving object is in contact with the working plane, the speed of the motion of the image of the contact portion or the speed at which the contact portion moves along the working plane is calculated;

Control communication unit: Transfer the speed of the contact part along the working plane or the speed of the movement of the contact part image and the left button and right click key information to the host system to control the movement of the cursor on the screen or perform related operations.

In the mouse device, the moving object is a user's finger.

In the mouse device, the working plane is a keyboard surface.

In the mouse device, the keyboard surface is smooth.

In the mouse device, a finger rest for holding a finger is installed between two adjacent rows of buttons on the keyboard.

In the mouse device, the left button and the right button are located on the finger rest, or are located near the space bar on the standard keyboard.

A touch screen includes:

Two or more cameras: for shooting moving objects on the screen surface and on one side of the screen; An image recognition unit: an image for identifying a moving object from a screen background image; a determining unit: configured to determine whether the object is in contact with the screen surface;

Calculation unit: If the object is in contact with the surface of the screen, calculate the position of the contact portion relative to the reference object and the coordinates on the screen, or the speed of the movement of the contact portion of the object;

Control unit: Perform the corresponding operation according to the position of the contact area of the object or the speed of the movement. The touch screen is based on the edge of the screen.

Compared with the prior art, the invention avoids the deficiencies of the existing mouse, and provides a novel mouse device which is integrated with the keyboard and directly controls the movement of the cursor on the screen of the display by the movement of the finger, so that the computer control is more convenient. Therefore, it is more practical; the touch screen is low in cost and accurate in positioning, and has no effect on screen brightness.

DRAWINGS

1 is a schematic diagram of a method for determining whether a point belongs to a plane according to the present invention;

2 is a schematic diagram of an image taken by the camera 2 of the present invention;

3 is a schematic diagram of an image taken by the camera 1 of the present invention;

4 is a schematic diagram of a finger mouse fused with a keyboard according to the present invention;

Figure 5 is a schematic diagram of a touch screen of the present invention;

FIG. 6 is a schematic diagram of a system with a mouse and a touch screen according to the present invention.

The symbols in the figure are: 1 is the camera, 2 is the camera, 3 is the point, 4 is the same image point of the camera 2, 5 is the same image point of the camera 1, 6 is the reference object, a, b, c are the reference objects Three reference points, 6a is the screen reference, 6b is the keyboard reference, 7 is the finger, 8 is the left button, and 9 is the right button.

A physical point in space, the imaged point on the photoreceptor of the camera is called the image point, a spatial point and a point on the plane. If their imaging points are in the same position, the points on the plane The same image point called the space point. The positions of the points belonging to the plane and their corresponding pixels are actually coincident, and the points on both sides of the plane and the actual positions of their same pixels do not coincide. When the spatial point is located on both sides of the plane, the position of the same image point on the plane is related to the position of the camera. Two cameras with different positions correspond to different positions of the same image on the plane of the two cameras. of. The position of the same image point is also related to the position of the plane. At the same point, the positions of the same image points in different planes corresponding to the same camera are also different.

The position of the point on the plane refers to its relative position to the reference object. The position of the image point is the relative position of the image point relative to the image of the reference object. The reference object is located on a plane and must include at least 3 points that are not on a straight line. These three points determine the position of the plane. The reference object can be a tangible sign, such as a flat texture or a pattern on a plane; it can also be a virtual logo. When a tangible mark is used, the imaging position of the reference can be obtained by image recognition. When using a virtual reference object, the actual position of the virtual reference object should be known, and the position of the image of the virtual reference object can be theoretically calculated, if the position of the virtual reference object is not known, or the imaging position cannot be calculated. , can not be used as a reference. There are various types of virtual markers, such as: Draw a fixed working area on the plane, calculate the imaging position of the edge of the area and the actual position of the edge of the area as a reference.

As shown in FIG. 1 , one point 3 is photographed by two cameras 1 and 2, and a certain mark on the plane is taken as a reference object 6, and the mark includes at least 3 points a, b, C o 4 which are not on the same straight line. 5 is the same image point of the camera 2 and 1 respectively. As can be seen from Fig. 2 and Fig. 3, the positions of points 4 and 5 are different with respect to the reference object 6.

Each point on the plane corresponds to an imaging point. The position of the plane point and the position of the imaging point are in one-to-one correspondence. When the position of the camera is fixed, the position correspondence is also fixed, which can be calculated or measured in advance. When the position of the camera is changed, the position correspondence is also changed. At this time, to calculate the correspondence, the reference object needs to be taken on the plane, and the reference object includes at least three points that are not on a straight line, and the position and size thereof are known and unchanged. First place the camera in the standard position, shoot the reference object, record the standard image position of the reference object, and then, when the position of the camera changes, take a picture of the reference object and record the position of the image of the reference object, and the standard reference The object image positions are compared to calculate the positional correspondence between the image point and the actual point. According to the position correspondence, the position of the same image point can be calculated from the position of the image point.

The image processing unit recognizes the image of the target point from the image captured by the camera, and calculates the position of the same image point on the plane according to the positional correspondence of the target point, if the same image is calculated from the image captured by the camera The positions of the image points are the same, that is: the position of the same image point of the target point corresponding to different cameras is the same on the plane, then the point belongs to this plane; otherwise, the point is located on both sides of the plane.

Alternatively, after the image recognition unit recognizes the image of the target point, the images of the plurality of cameras are superimposed to compare whether the image positions of the target points coincide, and if they are coincident, it indicates that: the same image point corresponding to the different cameras of the target point is in the plane. The upper position is the same, then the target point is on the plane, otherwise, the point is located on both sides of the plane.

A mouse device, including a left button and a right button; further comprising the following components:

Two or more cameras: Shooting moving objects above the work plane and the work plane; using a marker on the work plane as a reference, the reference contains at least three points that are not in a straight line;

The image recognition unit: identifying an image of the moving object from the working plane background image; determining unit: determining whether the moving object is in contact with the working plane by comparing whether the same image point position of the moving object in the plurality of cameras is the same; Calculation unit: If the moving object is in contact with the work plane, the speed of movement of the image of the contact portion or the actual speed of movement of the contact portion is calculated.

Calculating the speed of the image is relatively simple, and you can simply measure and calculate the image directly. Calculating the actual moving speed of the object is complicated: According to the correspondence between the position of the contact part image and the actual position of the contact part, the actual position of the contact part is calculated, and the displacement of the contact part in a time interval is further calculated, thereby calculating the contact part. The speed of motion along the working plane; the position of the image is in one-to-one correspondence with the actual position, and the correspondence is calculated in two ways:

If the parameters of the camera and the relative position and relative height with the working plane are fixed, the correspondence between the position of the image and the actual position of the contact portion is a constant, and the correspondence can be calculated or measured at one time, and can be used as a constant.

If the parameters of the camera and the relative position and relative height with the working plane are not fixed, the correspondence between the position of the image and the actual position is changed. In this case, the size and position of the reference must be a known constant. The value of the size and position of the reference object is measured, and the image size and position value of the reference object are obtained by the image recognition method, and the proportional relationship between the actual position and size of the reference object and the position and size of the image is obtained. Next, image recognition is used to determine the size and position of the image of the contact portion relative to the reference object. Based on the proportional relationship obtained in the previous step, the actual position of the contact portion on the work plane is calculated.

Control communication unit: transmit the motion information of the contact part along the working plane and the left and right keystroke key information to the host system, control the movement of the cursor on the screen or perform related operations. If there are multiple moving objects, the multiple moving objects are different. The motion state represents a different operation.

A touch screen that includes the following components:

Two or more cameras that shoot moving objects on the screen surface and on one side of the screen; Use a mark on the screen as a reference, such as the edge of the screen;

Image recognition unit: identifying an image of a moving object from a background image of the screen;

Judging unit: judging whether the object is in contact with the surface of the screen;

Calculating unit: If the moving object bit is in contact with the working plane, the actual position of the contact portion is calculated according to the correspondence between the position of the contact portion image and the actual position of the contact portion, and the displacement of the contact portion in a time interval is further calculated, thereby Calculate the speed at which the contact portion moves along the surface of the screen; the position of the image is in one-to-one correspondence with the actual position. There are two ways to calculate the actual position based on the position of the image:

If the parameters of the camera and the relative position and relative height with the working plane are fixed, the correspondence between the position of the image and the actual position is a constant, and the correspondence is calculated or measured at one time, and can be used multiple times.

If the parameters of the camera and the relative position and relative height with the working plane are not fixed, the correspondence between the position of the image and the actual position is changed. In this case, the size and position of the reference must be a known constant. The value of the size and position of the reference object is measured, and the image size and position value of the reference object are obtained by the image recognition method, and the proportional relationship between the actual position and size of the reference object and the position and size of the image is obtained. Next, the size and position of the image of the contact portion relative to the reference object are obtained by image recognition, and the actual position of the moving object on the screen surface is calculated based on the proportional relationship obtained in the previous step.

Control unit: Perform the corresponding operation according to the position of the object or the speed of movement.

detailed description

Embodiment 1 A finger mouse combined with a keyboard, as shown in FIG. 4:

Includes the following units: Two cameras, camera 1, camera 2: Shoot the movement of the keyboard surface and the finger above the keyboard, the camera is slightly higher than the keyboard surface, the position is above the standard keyboard function keys, above the ' and ' keys, the camera is aligned with the keyboard Central, the field of view covers the central area of the keyboard. The two cameras work synchronously, and their parameters are the same. When working, the parameters of the camera are fixed, such as the direction of the optical axis, focal length, position, height, etc.

Image recognition unit: The image of the finger and the image of the reference object are recognized from the background image of the keyboard. The parameters of the camera are fixed. Therefore, the background image of the keyboard is a constant and does not change. At the same time, the shape of the user's finger is also approximate. The same, so it is very easy to recognize the image of the finger from the background image of the keyboard. Usually, the part where the finger touches the surface of the keyboard is a fingertip, and the fingertip is generally located closest to the camera or closest to the keyboard, and the fingertip has obvious features such as nails, contour edges, etc., according to these features, the image recognition unit The image of the fingertip is further recognized from the finger image, and the corresponding point or the corresponding part is found. The corresponding point refers to an imaging point in which the physical point is respectively in the two cameras, and the corresponding points constitute the corresponding part. The corresponding points on the fingertip are processed by subsequent units. Image recognition technology is currently quite mature and will not be described here.

Judgment unit: Determine if the finger is in contact with the surface of the keyboard. With the numeric keys '6 'Ί' and 'Ψ above the standard keyboard as the reference object, ie the keyboard reference object 6b, the position and parameters of the camera are fixed, so the position and actual position of the imaging point are also fixed, which can be regarded as a constant. .

The movement of the finger on the keyboard is divided into three categories: 1. Keystroke action: The finger moves from top to bottom, and the direction is substantially perpendicular to the surface of the keyboard. At the same time, there is generally an electrical signal that the button is pressed; 2. Effective action: Fingers are attached The surface of the keyboard moves, and a finger moves against the surface of the keyboard to move the cursor. At this time, the movement of the finger controls the movement of the cursor; when multiple fingers are attached to the surface of the keyboard, different gestures represent different operations, meaning of the gesture. , customized by the user. If two fingers move down at the same time, it means the scroll bar moves downwards; four fingers move down, which means the window minimizes the operation; two fingers Opposite motion, means to close a program; two fingers move backwards to indicate that a program is open. 3, other actions: In addition to the vertical movement and the movement of the surface of the keyboard, such actions are often the user's habits or unintentional movements.

When the fingertip of the finger is located on the surface of the keyboard, the position of the fingertip (or the corresponding point of the fingertip) corresponding to the two cameras corresponds to the position of the same image point (relative to the reference object); if it is higher than the surface of the keyboard The fingertip corresponds to the position of the same image point of the two cameras. According to whether the position of the same image point of the different camera corresponding to the fingertip image (or the corresponding point of the fingertip) is the same, the judgment unit can determine whether the fingertip is posted. The keyboard surface. If the fingertip is not on the surface of the keyboard, it means that the action of the finger is 1 keystroke or 3 other actions. At this time, the mouse system does not do any processing. If the fingertip is attached to the surface of the keyboard and there is no electrical signal that the button is pressed, it indicates that the action of the finger belongs to 2 effective action, and the computing unit of the mouse starts to work.

Calculation unit: Calculates the speed at which the fingertip moves along the surface of the keyboard, including the rate and direction. In the image taken by the same camera at two times, the distance between the image of the fingertip (or the corresponding point of the fingertip) is divided by the time interval between the two moments, and the moving speed of the image of the fingertip of the finger is calculated, thereby calculating the fingertip of the finger. The speed of movement.

Control communication unit: Send the movement speed information of the fingertip, the gesture information and the keystroke information of the left and right keys to the host system to perform the corresponding operations.

There is a difference between the keys on the ordinary keyboard and the keys on the ordinary keyboard. In the present invention, the surface of the keyboard is smooth to facilitate the movement of the fingers. At the same time, a fixing bracket for holding a finger is arranged between the upper and lower rows of buttons in the center of the keyboard, which is called a finger rest. When the user's finger moves on the keyboard surface, other fingers can be placed on the finger rest, which can Avoiding other fingers erroneously pressing the button also provides a support point and a force point for the movement of the finger. The left button 8 and the right button 9 can be mounted on the finger rest, operated with the index finger, the middle finger or the ring finger, or installed near the space bar of the standard keyboard, and operated by the thumb. When using the keyboard input, the user presses the key in the normal way, the same way as the ordinary keyboard; when using the mouse, the hand does not need to leave the keyboard, the other fingers are placed on the finger rest to remain still, a finger is attached When the surface of the keyboard moves, the cursor moves with the finger. The operation of the left and right keys is the same as that of the normal mouse. When the gesture operation is required, multiple fingers are attached to the keyboard and moved at the same time, and the corresponding operation can be performed.

At present, the keyboard and mouse of the desktop are separated. When operating, the user's hand needs to move back and forth between the keyboard and the mouse, which is inconvenient. The mouse device used in the notebook computer is generally a touch bar or a touch pad. The touch bar occupies a small volume and is convenient to carry, but the difference between the use mode and the conventional mouse is too large, and the use is not accustomed. The touchpad takes up a lot of space and is actually separate from the keyboard. The mouse and the keyboard disclosed by the invention are perfectly integrated, do not occupy the keyboard volume, and are very convenient to carry. The hand does not need to leave the keyboard when using, and at the same time, the movement of the cursor is directly controlled by the finger, which is more convenient than the use of the conventional mouse.

Embodiment 2 Touch screen, as shown in Figure 5:

Including two cameras, the camera is positioned at two corners above the screen, the height of the camera is higher than the surface of the screen, and its field of view covers the entire screen. The parameters of the camera are fixed, such as focal length, orientation of the optical axis, height, position, etc. With at least two intersecting edges of the screen as the screen reference 6a, the imaging position and the actual position of the reference object are fixed and can be regarded as constant. The camera captures the image on the screen surface and on the side of the screen, and the two cameras work in sync.

The image recognition unit recognizes an image of an object (touch body) from the background image of the screen, and determines corresponding points in the images of the two cameras. The background image of the screen is the image normally displayed on the screen, which is generally changed and located on the surface of the screen. The object image is located on the side of the screen except the contact surface. According to this feature, the background image and the object image can be distinguished. .

Judging unit: judging whether the object is in contact with the screen, if the object is not in contact with the screen, The relative positions of the objects corresponding to the two cameras (relative to the reference object) are different; if the object is in contact with the screen, the objects in the corresponding two cameras are the same as the image point.

Calculation unit: Calculates the speed at which a moving object moves along the surface of the screen, including the velocity and direction. When multiple objects are in contact, the position and velocity are calculated separately.

Control communication unit: Send the moving speed information of the moving object, multi-touch information to the host system, and perform corresponding operations.

Embodiment 3 A notebook computer that simultaneously integrates a finger mouse and a touch screen with a keyboard, as shown in FIG. 6:

The mouse and the touch screen are common input devices on the computer. In the above two embodiments, the principle used by the mouse and the touch screen is the same, and the main components are the same, so they can be integrated into one system, that is, A system with both touch screen and mouse functions.

On the two corners above the laptop screen, two cameras are mounted. The height of the camera is higher than the surface of the screen. The field of view covers the surface of the screen and the keyboard. Each point on the screen and keyboard ensures that it is captured by two cameras. The two cameras capture the surface of the screen and the side of the screen, as well as the surface of the keyboard and objects above the keyboard, and the two cameras shoot simultaneously. In the figure, 1 and 2 are screen cameras, and 6a is a screen reference.

The surface of the keyboard is smooth to facilitate the movement of the fingers. The finger pad is mounted on the gap between the upper and lower rows of buttons. The finger rest is a fixed bracket for lifting the finger. When the user's finger moves on the surface of the keyboard, other fingers can be placed. On the finger rest, this can prevent other fingers from erroneously pressing the button, and also provides support points and stress points for the movement of the finger. The left and right buttons can be mounted on the finger rest, operated with the index finger, middle finger or ring finger. They can also be installed near the space bar of the standard keyboard and operated by the thumb.

The image recognition unit recognizes the image of the finger from the screen and the keyboard background, different user's fingers The image does not have much difference, so the success rate of recognition is high.

Using the edge of the screen as the screen reference 6a, the power light on the surface of the keyboard, the 3' and 6' keys are used as the keyboard reference 6b. The actual position of the two reference objects and the imaging position are calculated differently: Since the position and height of the camera relative to the screen are fixed, the imaging position and size of the screen reference object on the photosensitive material of the camera are fixed, once determined It can be used as a constant directly. Its actual position and size are also fixed and can be used as a constant. Since the screen of the notebook computer can be opened and closed, the position of the camera is changed with respect to the keyboard, and the imaging position and size of the keyboard reference object on the photosensitive material of the camera are also changed, and the specific imaging position and size are recognized by the image recognition unit in real time. And calculated. The actual position and size of the keyboard reference is constant and can be measured during the production commissioning phase.

The judging unit judges the type of operation of the finger. After recognizing the finger image, first calculate the relative position of the finger image (usually the fingertip) relative to the same image point of the keyboard. If the positions of the same image points of the two cameras relative to the keyboard plane are the same, the finger is indicated. On the keyboard surface, if there is no electrical signal that the keyboard button is pressed at this time, it indicates that the finger is operating the mouse; if the positions of the same image points of the two cameras relative to the keyboard plane are different, continue to compare with the screen plane, if The relative position of the finger (usually the fingertip) corresponding to the same image point of the two cameras with respect to the plane of the screen is the same, indicating that the finger is operating the touch screen; otherwise, the finger is neither operating the keyboard nor operating the mouse.

The calculation unit, if the finger is operating the mouse, calculates the speed of movement of the finger (usually the fingertip) against the surface of the keyboard; if the finger is operating the touch screen, calculates the position of the finger (usually the fingertip) on the surface of the screen and the finger The speed of movement (usually the fingertip).

The communication and control unit sends the movement information of the finger, the gesture information and the click information of the left and right keys to the host system, and if the finger is operating the mouse, the cursor moves with the finger, or according to the custom The meaning of the meaning of the gesture, the corresponding gesture operation is performed; if the finger is operating the touch screen, the program function at the position of the touch point of the screen is executed, or the corresponding gesture operation is performed according to the meaning of the customized gesture.

A system that implements two functions, both a touch screen and a mouse, is highly cost-effective. The keyboard and mouse are combined to make the user's use and operation more convenient.

Claims

Claim

A method for determining whether a target point belongs to a plane, wherein: two or more cameras capture a target point on a plane and on both sides of the plane, and the image recognition unit captures from the camera The image of the target point is recognized in the image. If the position of the same image point corresponding to the different cameras is the same, the target point belongs to the plane. Otherwise, the target point is located on both sides of the plane.

2. The determining method according to claim 1, wherein: the method for determining whether the target point corresponds to the same image point of different cameras is the same: a comparing the images captured by the plurality of cameras, if If the image positions of the target points coincide, the positions are the same, otherwise, the positions are different. b According to the correspondence between the position of the imaged point and the actual position on the plane, the image point is calculated from the position of the image of the target point in the image. The actual position on the plane compares the position of the same image point calculated from the images taken by multiple cameras.

3. A mouse device, including a left button and a right button, the features of which are:

4. The mouse device according to claim 3, wherein: the moving object is a user's finger.

5. The mouse device according to claim 3, wherein: the work plane is a keyboard surface.

6. The mouse device according to claim 5, wherein: the surface of the keyboard is smooth.

7. The mouse device according to claim 5, wherein: the finger holders for holding the fingers are mounted between the adjacent two rows of buttons on the keyboard.

8. The mouse device according to claim 3, wherein: the left button and the right button are located on the finger rest, or are located near the space bar on the standard keyboard.

9. A touch screen, characterized by:

Two or more cameras: for capturing moving objects on the screen surface and on one side of the screen; image recognition unit: for recognizing an image of a moving object from the background image of the screen;

Judging unit: used to determine whether an object is in contact with the surface of the screen;

Control unit: Perform the corresponding operation according to the position of the contact area of the object or the speed of the movement.

10. The touch screen of claim 9, wherein: the edge of the screen is used as a reference.