JP2014235478A - Touch input device, input detection method for touch input device, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a coordinate and pressing force at each of a plurality of touch input points when the plurality of touch inputs are performed on a panel with force sensors.SOLUTION: A panel part 100 is arranged with a plurality of force sensors around the panel. Each of the force sensors outputs signals in response to a touch to the panel. A computing part 200 executes predetermined computing processing on the basis of signals from the plurality of force sensors, and thereby computing a coordinate and pressing force at each touch input point of the panel. The computing part 200 determines whether the computed coordinate and pressing force of the touch input satisfy a predetermined pattern determination condition that is previously set, and, if the predetermined pattern determination condition is determined to be satisfied, executes predetermined computing processing, thereby also computing a coordinate and pressing force at each of a plurality of touch input points of the panel.

Description

  The present invention relates to a touch input device, a touch input device input detection method, and a computer program.

  There is known a touch input device such as a touch panel that receives an input by touching a display surface. As a method for detecting the XY coordinates of the touched position in the touch input device, a resistance film method, a capacitance method, a force sensor method, and the like are known. The force sensor type touch input device can detect a pressing pressure in addition to the XY coordinates.

  In Patent Document 1, since a touch input device is realized using a glass plate that is an operation panel and a force sensor, a film having optical loss like a resistive film type or a capacitance type is not required. It is described that the light transmittance increases.

  Patent Document 2 describes a tablet terminal capable of detecting so-called multi-touch.

JP 2005-78194 A JP 2012-173890 A

  In a force sensor type touch input device, a pressure applied to a panel is measured by a plurality of force sensors around the panel, and a touched location is calculated from the measured values of each force sensor. For this reason, in a force sensor type touch input device, when a plurality of locations are touched almost simultaneously, it is difficult to specify those locations. This is because the combined value of the pressure applied to a plurality of locations is distributed to each force sensor and measured.

  The present invention has been made in view of the above problems, and an object thereof is to provide a technique capable of detecting a plurality of touch inputs in a touch input device using a force sensor.

  A touch input device according to one aspect of the present invention includes a panel, a plurality of force sensors that are provided around the panel and output signals according to force, and a predetermined calculation based on signals from the plurality of force sensors. And a calculation unit that calculates coordinates and pressing force at each point of a plurality of touch inputs to the panel by performing the processing.

  The calculation unit calculates coordinates and pressing force input by touching the panel based on signals from a plurality of force sensors, and sets predetermined pattern determination conditions in which the calculated coordinates and pressing force of the touch input are set in advance. If it is determined whether or not a predetermined pattern determination condition is satisfied, a predetermined calculation process is performed to calculate the coordinates and pressing force at each point of a plurality of touch inputs to the panel.

  The predetermined pattern determination condition includes a first pattern determination condition for determining a first pattern having a predetermined time difference between a plurality of touch inputs to the panel. When satisfying the above, by executing the predetermined first calculation process included in the predetermined calculation process, the coordinates and the pressing force at each point of the plurality of touch inputs having a predetermined time difference can be calculated.

  The predetermined pattern determination condition includes a second pattern determination condition for determining a second pattern in which a plurality of touch inputs to the panel are performed within a predetermined time. When the condition is satisfied, by executing the predetermined second calculation process included in the predetermined calculation process, the coordinates and the pressing force at each point of the plurality of touch inputs performed within the predetermined time can be calculated. it can.

1 is a block diagram illustrating a configuration of a touch input device according to Embodiment 1. FIG. FIG. 3A is a perspective view illustrating a configuration of the panel unit 100, and FIG. The top view which shows the coordinate system of the panel part 100 surface. 3 is a flowchart illustrating input detection processing according to the first embodiment. Explanatory drawing which shows the detection method of time difference multi-touch. The flowchart which shows a time difference multi-touch detection process. Explanatory drawing which shows the mode of the gravity center at the time of simultaneous multi-touch. Explanatory drawing which shows the detection method of simultaneous multi-touch. The flowchart which shows the detection process of simultaneous multi-touch. FIG. 6 is a block diagram illustrating a configuration of a touch input device according to a second embodiment. The flowchart which shows the process which detects multi-touch with a touch input device provided with the several panel from which a detection system differs. Explanatory drawing which shows the detection method when three points | pieces are touched simultaneously. Explanatory drawing which shows the detection method when three points on a straight line are touched simultaneously.

  In this embodiment, as will be described in detail below, when a multi-touch operation is performed on the panel based on a signal from a force sensor arranged around the panel, the coordinates and pressures of the plurality of pressing points are performed. Is detected.

  Multi-touch as an example of “multiple touch inputs” means that the user simultaneously touches a plurality of positions on the panel with a fingertip or the like. However, since it is a human operation, even if the operation can be said to be “simultaneous” from the user's side, multiple touches have the same meaning in a relatively strict sense when viewed from the touch input device side where the operation is input. Little is done at the time. Therefore, in this embodiment, multi-touch in which a plurality of distant places on the panel are touched at the same time is dealt with by dividing into a plurality of patterns according to the time difference.

  In the first pattern, there is a predetermined time difference between one touch and the other touch. The predetermined time difference is set relatively long within the range of the multi-touch recognition time in which the touch input device should be recognized that the panel operation is multi-touch. This is because if the predetermined time difference is set to be longer than the multi-touch recognition time, the touch input device cannot be recognized as multi-touch.

  In the second pattern, one touch and the other touch are performed within a predetermined time. The predetermined time is set relatively short.

  It can be understood that both the first pattern and the second pattern are cases where “a plurality of touch inputs to the panel are simultaneously performed”. Considering the difference between the two patterns, the first pattern can be defined as a case where a plurality of touch inputs to the panel are performed simultaneously within a range that allows a predetermined time difference. Alternatively, the first pattern can be defined as a case where a plurality of touch inputs to the panel are performed almost simultaneously.

  The touch input device of this embodiment is a portable information terminal (including a mobile phone, a smartphone, a camera, a personal computer, a game machine, etc.), a vehicle (including a car navigation device), a kiosk terminal, an ATM (Automated Teller Machine). The present invention may be applied to a system that accepts an operation by pressing a touch surface, such as a vending machine (including an automatic ticket vending machine). The touch input device according to the present embodiment may be applied to an HMI (Human Machine Interface) device that is connected to a computer and performs input / output. Furthermore, the touch input device of the present embodiment can also be applied to a touch pad used as a pointing device.

  Embodiment 1 will be described with reference to FIGS. The touch input device of this embodiment calculates and outputs coordinates based on the force (pressing force) detected by the force sensor.

  FIG. 1 is a block diagram illustrating the configuration of the touch input device according to the first embodiment. The touch input device according to the present embodiment includes a panel unit 100, a signal conversion unit 500, a calculation unit 200, an application execution unit 300, and a display control unit 400.

  The panel unit 100 outputs four output signals respectively indicating the forces applied to the four locations of the panel unit 100 to the signal conversion unit 500. The signal conversion unit 500 includes four A / D conversion units 510 corresponding to the four output signals output from the panel unit 100, respectively. The A / D converter 510 converts an analog output signal into a digital output value at a predetermined sample period by A / D conversion.

  The calculation unit 200 detects an input to the panel unit 100 based on an output value from each A / D conversion unit 510 and outputs the detected input to the application execution unit 300.

  The application execution unit 300 executes the application based on the detected input, generates screen information based on the execution result, and outputs the screen information to the display control unit 400. The display control unit 400 further generates a display signal for controlling the panel unit 100 based on the screen information, and outputs the display signal to the panel unit 100. Panel unit 100 displays a screen based on the display signal.

  The calculation unit 200 includes, for example, an input calculation unit 210, a storage unit 220, and a multi-touch detection unit 230. The input calculation unit 210 acquires an output value from the signal conversion unit 500 at a predetermined measurement cycle, and calculates a measurement value that indicates a variation in the output value. The input calculation unit 210 calculates a pressing force indicating a pressing force based on the measurement value. Further, the input calculation unit 210 calculates the pressed coordinates that are the coordinates of the pressed point based on the measured value and the pressing force. The storage unit 220 is a buffer for storing the pressing force and the pressing coordinates calculated by the input calculation unit 210.

  The multi-touch detection unit 230 detects the pressing force and the pressing coordinate at each multi-touched point (each pressing point) based on a plurality of sets of pressing force and pressing coordinate data stored in the storage unit 220. To do.

  The multi-touch detection unit 230 includes a time difference multi-touch detection unit 231 and a simultaneous multi-touch detection unit 232. The time difference multi-touch detection unit 231 is a function for “calculating coordinates and pressing force at each point of a plurality of touch inputs having a predetermined time difference”. The simultaneous multi-touch detection unit 232 is a function for “calculating coordinates and pressing force at each point of a plurality of touch inputs performed within a predetermined time”. Details of the detection units 231 and 232 will be described later.

  The application execution unit 300 executes an application program that uses touch input to the panel unit 100. Examples of the application program include document creation software, mail management software, web browser, game software, image management software, music management software, electronic book browsing software, and map browsing software.

  The display control unit 400 generates a display signal for controlling the display of the panel unit 100 based on the screen information from the application execution unit 300 and outputs the display signal to the panel unit 100.

  The calculation unit 200, the application execution unit 300, and the display control unit 400 are realized by a computer, for example. This computer includes a memory for storing programs and data, and a microprocessor such as a CPU (Central Processing Unit) that executes processing of the arithmetic unit 200, the application execution unit 300, and the display control unit 400 according to the program. This program may be stored in a computer-readable medium and read from the medium to the computer.

  The calculation unit 200, the application execution unit 300, and the display control unit 400 may be realized by different microprocessors. The application execution unit 300 and the display control unit 400 may be realized by a single microprocessor. Programs corresponding to the calculation unit 200, the application execution unit 300, and the display control unit 400 may be prepared. Any of the arithmetic unit 200, the application execution unit 300, and the display control unit 400 may be realized by a dedicated electronic circuit. Either the application execution unit 300 or the display control unit 400 may be provided outside the touch input device. In this case, the arithmetic unit 200 has a communication interface and performs communication with the outside of the touch input device. The calculation unit 200 may include a signal conversion unit 500.

  The structure of the panel part 100 is demonstrated using FIG. 2A shows a perspective view of the panel unit 100, and FIG. 2B shows an exploded perspective view of the panel unit 100. FIG.

  The panel unit 100 includes, for example, a display panel 110, a cover panel 120, four force sensors 130, and a base unit 150. A display panel 110 is disposed on the base unit 150. The display panel 110 includes, for example, an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display, or the like. In this embodiment, the case where the display area of the display panel 110 is rectangular will be described. However, the present invention is not limited to this, and may be a square, a triangle, a pentagon or more polygon, a circle, an ellipse, or the like.

  A plurality of force sensors 130 are arranged on the upper surface side of the base portion 150 so as to be located around the display panel 110. For example, four force sensors 130 are arranged corresponding to the four vertices of the display panel 110, respectively.

  Examples of the force sensor 130 include a piezoelectric element such as a piezoelectric element, a capacitor, and a strain gauge. In this embodiment, a piezoelectric element is used as an example of the force sensor 130. The number of the force sensors 130 is not limited to four. For example, five or more force sensors 130 may be provided, or a configuration using three or less force sensors may be used.

  A cover panel 120 is provided to face the surface of the display panel 110. The touch surface is the surface of the cover panel 120. The surface shape of the cover panel 120 is formed in a rectangular shape, for example, similarly to the surface shape of the display panel 110.

  The four force sensors 130 respectively support the vicinity of the four vertices on the back surface of the cover panel 120. Thereby, there is a predetermined interval between the back surface of the cover panel 120 and the front surface of the display panel 110. The cover panel 120 transmits the display on the display panel 110. The cover panel 120 is made of a material that is transparent to visible light, such as glass or plastic.

  A signal conversion unit 500 is further provided on the upper surface side of the base unit 150. The signal conversion unit 500 is connected to the four force sensors 130 via signal lines. The signal conversion unit 500 only needs to be connected to the force sensor 130 and does not necessarily have to be disposed on the base unit 150. On the base unit 150, a calculation unit 200, an application execution unit 300, and the like may be further provided. The four force sensors 130 may support the vicinity of the four vertices on the back surface of the display panel 110, and the front surface of the display panel 110 may be in contact with the back surface of the cover panel 120.

  In order to remove an unstable operation region due to a minute pressure, it is preferable that a slight pressure is applied to the cover panel 120 in the direction of the base portion 150 as an initial load. In order to apply an initial load, an elastic body such as a spring may be provided between the cover panel 120 and the base portion 150.

  The cover panel 120 covers the surface of the display panel 110 and protects the surface of the display panel 110. Furthermore, the surface of the cover panel 120 receives a press (touch) by the user. The four force sensors 130 output an output signal corresponding to the force received from the cover panel 120 to the signal conversion unit 500 as an electrical signal. Each force sensor 130 continuously measures the force by the user's touch.

  The arithmetic unit 200 performs an input detection process for detecting an input based on an output from the force sensor 130 at every predetermined input detection cycle.

  FIG. 3 shows a coordinate system on the surface of the panel unit 100. In this embodiment, the center of the display area of the display panel 110 is the origin, and the surface of the display panel 110 is the XY plane. Of the four force sensors 130, the force sensor 130 arranged in the first quadrant of the XY plane is set as S1, the force sensor 130 arranged in the second quadrant is set as S2, and the force sensor 130 arranged in the third quadrant. Is S3, and the force sensor 130 arranged in the fourth quadrant is S4. Here, the measured values obtained from the force sensors S1, S2, S3, and S4 are F1, F2, F3, and F4, respectively.

  Here, the distance on the surface of the panel unit 100 is defined in units of pixels in the display panel 110. The size of the display area of the display panel 110 in the X direction is Wd, and the size of the display area in the Y direction is Hd.

  Further, the distance between the two force sensors 130 adjacent in the X direction (the distance between S1 and S2, the distance between S3 and S4) is Ws, and the distance between the two force sensors 130 adjacent in the Y direction. Let Hs be the distance (distance between S1 and S4, distance between S2 and S3). For example, the inter-sensor pixel number Ws in the X direction is 422 and Hs is 266. Furthermore, if the distance in the X direction from the origin to the force sensor 130 is Xs, Xs is Ws / 2, and if the distance in the Y direction from the origin to the force sensor 130 is Ys, Ys is Hs / 2.

  In the calculation method of the pressing coordinates, the input calculation unit 210 is based on the coordinates of the force sensors S1, S2, S3, S4 represented by Xs and Ys, the measured values F1, F2, F3, F4, and the pressing force Fs. The press coordinates PX and PY are calculated using the following formula.

  FIG. 4 is a flowchart showing the input detection process.

  The input calculation unit 210 stores in advance initial loads Fw1, Fw2, Fw3, and Fw4 corresponding to the four force sensors 130, respectively. When the input detection process is started, the input calculation unit 210 acquires output values G1, G2, G3, and G4 respectively corresponding to the four force sensors 130 from the signal conversion unit 500 (S10). The input calculation unit 210 calculates the measured values F1, F2, F3, and F4 by subtracting the initial loads Fw1, Fw2, Fw3, and Fw4 from the output values G1, G2, G3, and G4, respectively (S11). Thereby, the influence of the initial load can be removed from the output value.

  The input calculation unit 210 calculates a pressing force Fs that is the sum of the measured values F1, F2, F3, and F4 (S12). The pressing force Fs indicates a force that presses the surface of the cover panel 120. The input calculation unit 210 determines whether the pressing force is equal to or greater than a predetermined pressing force threshold Fth (S13). If the input calculation unit 210 determines that the pressing force Fs is smaller than the pressing force threshold Fth (S13: NO), the process returns to step S10. Thereby, since a noisy force is not detected as a pressing (touch) operation, the detection accuracy of touch input can be increased.

  When the input calculation unit 210 determines that the pressing force Fs is equal to or greater than the pressing force threshold Fth (S13: YES), the pressing coordinates PX and PY, which are the coordinates of the touched point (pressing point), are expressed by Equation 1. (S14). The input calculation unit 210 stores the calculated pressing coordinates and pressing force in the storage unit 220 and outputs them to the multi-touch detection unit 230 (S15).

  FIG. 5 is an explanatory diagram illustrating a method of detecting time difference multi-touch. The time difference multi-touch means a multi-touch operation with a short predetermined time difference between the first touch and the second touch. When the first touch and the second touch are operated with a predetermined time difference or more, it is detected that the touch is performed at two places.

  The time difference multi-touch has the following characteristics. The first feature is that the detected coordinates move so as to fly from the first point to the second point. The second feature is that the pressing force Fs when the second point is touched is larger than the pressing force when the first point is touched. The third feature is that the center of gravity exists on the line connecting the first point and the second point.

  FIG. 5A shows a moment when the first point is touched. The coordinates of the first touch P1 are detected as (P1X, P1Y), and the pressing force is detected as Fs1. In the figure, the pressed points detected by the calculation unit 200 are hatched, and the pressed points that are not detected are outlined. Hereinafter, the first touch P1 may be abbreviated as the first point P1. Similarly, the second touch P2 may be abbreviated as the second P2.

  FIG. 5B shows a case where the second point is touched after the first point P1. When the two points P1 and P2 are touched almost simultaneously on the panel unit 100, the pressed coordinates move from the first point P1 toward the second point P2. Due to the relationship between the detection cycle of the force sensor 130 and the calculation cycle of the calculation unit 200, the pressed coordinates are calculated at a predetermined cycle. The pressed coordinates stop near the center of gravity G between the first point P1 and the second point P2.

  The coordinates of the center of gravity G are indicated as (PgX, PgY), and the pressing force is indicated as Fsg. The coordinates of the second point P2 that has not been detected yet are indicated as (P2X, P2Y), and the pressing force is indicated as Fs2. In the following description, the center of gravity G may be abbreviated as the center of gravity G.

  As shown in FIG. 5C, the pressing coordinates when the two places P <b> 1 and P <b> 2 of the panel unit 100 are touched are the center of gravity G.

  As shown in FIG. 5D, by defining two right triangles, the coordinates of the second point P2 and the pressing force are geometrically calculated. The first triangle ΔGP1W1 has three points, the center of gravity G, the first point P1, the perpendicular line descending from the center of gravity G, and the point W1 at which the line parallel to the X axis passing through the first point P1 intersects. Is a triangle. The other triangle ΔP2GW2 is a triangle having apexes at three points: the second point P2, the center of gravity G, a perpendicular line drawn from the center of gravity G, and a point W2 where a line parallel to the X axis passing through the center of gravity G intersects. It is. The length from the first point P1 to the center of gravity G is R1, and the length from the center of gravity G to the second point P2 is R2.

  The angle θ that the line segment R1 from the first point P1 to the center of gravity G forms with the X axis can be obtained by the following formula 2.

  θ = atan (PgY-P1Y) / (PgX-P1X) * (180 / π) (Equation 2)

  The length of the line segment R1 can be obtained by the following formula 3 or formula 4.

  R1 = (PgY-P1Y) / sinθ (Formula 3)

  R1 = (Pgx-P1X) / cosθ (Formula 4)

  Here, when (PgY−P1Y) is DiffY1 and (Pgx−P1X) is DiffX1, Equation 2 becomes Equation 5, Equation 3 becomes Equation 6, and Equation 4 becomes Equation 7.

θ = atan (DiffY1) / (DiffX1) * (180 / π) (Formula 5)

  R1 = DiffY1 / sinθ (Formula 6)

  R1 = DiffX1 / cosθ (Formula 7)

  Here, when the balance law is applied to FIG. The line segment R2 is a line connecting the center of gravity G and the second point P2. When Formula 8 is solved for R2, Formula 9 is obtained. By using Equation 9, the length of the line segment R2 can be obtained.

  Since the pressing force Fsg at the center of gravity G is obtained as the sum of the pressing force Fs1 at the first point P1 and the pressing force Fs2 at the second point P2, the pressing force Fs2 at the second point P2 is (Fsg−Fs1). ). In Expressions 8 and 9, it is a precondition that the pressing force Fsg of the center of gravity G is larger than the pressing force Fs1 of the first point P1.

  Fs1 * R1 = (Fsg-Fs1) * R2 (Formula 8)

  R2 = (Fs1 * R1) / (Fsg-Fs1) (Formula 9)

  Here, since the center of gravity G is located somewhere on the line connecting the first point P1 and the second point P2, the two right triangles ΔGP1W1 and ΔP2GW2 have a common angle θ. The two right triangles ΔGP1W1 and ΔP2GW2 are similar because the two angles perpendicular to the angle θ are equal. Therefore, if the base length DiffX1 and the height DiffY1 of the first right triangle ΔGP1W1 are known, the base length DiffX2 and the height DiffY2 of the second right triangle ΔP2GW2 are obtained from Equations 10 and 11. Recognize.

  DiffX2 = (DiffX1 * R2) / R1 (Equation 10)

  DiffY2 = (DiffY1 * R2) / R1 (Formula 11)

  If DiffX2 obtained by Equation 10 is added to the X-axis coordinate PgX of the center of gravity G, the X-axis coordinate P2X of the second point P2 is obtained as shown in Equation 12. If DiffY2 obtained by Equation 11 is added to the Y-axis coordinate PgY of the center of gravity G, the Y-axis coordinate P2Y of the second point P2 is obtained as shown in Equation 13.

  P2X = PgX + DiffX2 (Formula 12)

  P2Y = PgY + DiffY2 (Formula 13)

  As described above, if it is detected that the operation is a “time difference multi-touch” operation based on the above three features and the coordinates of the first point P1, the pressing force, the coordinates of the center of gravity G, and the pressing force are obtained, the second point P2 is obtained. Coordinates and pressing force can also be calculated. If the coordinates and pressing force of the first point P1 and the second point P2 are known, the coordinates (PmX, PmY) of the center point M and the pressing force Fsm can also be obtained.

  FIG. 6 is a flowchart illustrating the time difference multi-touch detection process executed by the time difference multi-touch detection unit 231. Hereinafter, the detection unit 231 may be abbreviated.

  The input calculation unit 210 performs calculation at a predetermined cycle, and this processing is executed every time a new calculation result is obtained. The time difference multi-touch detection process and the simultaneous multi-touch detection process described later are repeatedly executed every time new data is detected. The time difference multi-touch process and the simultaneous multi-touch process are executed in parallel. By exclusive control, the value of the earlier output of the time difference multi-touch detection process and the simultaneous multi-touch detection process is adopted.

  The detecting unit 231 determines whether the new data (pressed coordinates and pressing force) calculated by the input calculating unit 210 is stored in the storage unit 220 (S20). As described in FIG. 4, when a pressing force exceeding a predetermined threshold is detected, new data calculated by the input calculation unit 210 is stored in the storage unit 220. When the detection unit 231 determines that new data is stored in the storage unit 220 (S20: YES), the detection unit 231 determines whether the multi-touch flag is on (S21). The multi-touch flag is determination information indicating that it has been determined that multi-touch has been performed, and is set in step S25 described later. If the multi-touch flag is on (S21: YES), the detection unit 231 skips steps S22 to S25 and moves to step S26 because it has already been determined that multi-touch has been performed. On the other hand, when the multi-touch flag is off (S21: NO), the detection unit 231 analyzes the relationship between the latest data detected in step S20 and the past data (S22). Here, since it is the first execution, NO is determined in step S21 and the process proceeds to step S22.

  The detection unit 231 determines whether or not the currently calculated pressing force has increased by a predetermined value or more than the previous pressing force (S23). That is, the detection unit 231 determines whether or not the above-described second feature has been found in the data analysis result.

  Furthermore, the detection unit 231 determines whether the coordinates calculated this time have moved in a certain direction from the previous coordinates (S24). That is, the detection unit 231 determines whether the first feature described above has been found in the data analysis result. The order of step S23 and step S24 is not particularly limited.

  When the detection unit 231 satisfies both the determinations of step S23 and step S24 (S23: YES AND S24: YES), the detection unit 231 sets the multi-touch flag to on (S25). And the detection part 231 calculates the coordinate and pressing force of several touches with a time difference by calculating using a predetermined | prescribed arithmetic expression, respectively (S26). Since the predetermined arithmetic expression has been described with reference to FIG.

  The detection unit 231 stores the calculation result of step S26 in the storage unit 220 (S27), and further outputs it to the application execution unit 300 (S28). The application execution unit 300 generates screen information based on the coordinates of each point of multi-touch, or based on the coordinates and the pressing force, and outputs the screen information to the display control unit 400. If the detection unit 231 determines that the new data is not stored in the storage unit 220 (S20: NO), the detection unit 231 turns off the multi-touch flag (S29). The second execution of this process will be briefly described. When new data is continuously calculated after the first execution (S20: YES), since the multi-touch flag is set to ON (S21: YES), the detection unit 231 skips steps S22 to S25. Then, the process proceeds to S26 to execute a predetermined calculation. The calculation result is stored in the storage unit 220 (S27) and output to the application execution unit 300 (S28). Then, when new data is no longer detected (S20: NO), for example, when the user removes his / her finger from the panel, the multi-touch flag is reset and turned off (S29).

  FIG. 7 is an explanatory diagram showing changes in the center of gravity G during simultaneous multi-touch. Simultaneous multi-touch is a plurality of touches performed within a predetermined short time, and is that a plurality of locations (here, two locations) are touched almost simultaneously. In simultaneous multi-touch, since two places are touched almost simultaneously, there is no time ahead, but for convenience of explanation, the first point touch P1 (or first point P1), the second point touch P2 (or two points) This will be described as eye P2).

  As shown in FIG. 7A, when the user touches the first point P1 and the second point P2 almost simultaneously with the fingertip or the like, the center of gravity G appears on the line connecting the first point P1 and the second point P2. Appear.

  As shown in FIG. 7B, the center of gravity G slightly moves on a line connecting the touched points P1 and P2. It is difficult for the user to press the points P1 and P2 evenly and the pressing force at the points P1 and P2 varies in a short time, so the coordinates of the center of gravity G and the pressing force also change.

  FIG. 7C shows an enlarged center of gravity G that slightly fluctuates on the same line. Here, for convenience, numerals 1 to 5 are attached to the center of gravity G in the order from the first point P1 to the second point P2. However, it does not move in the order of G1, G2, G3, G4, and G5. The slight fluctuation of the center of gravity G depends on the change in force at the points P1 and P2, etc. For example, G3 to G1, G1 to G4, G4 to G2, G2 to G5, and so on. Sometimes.

  FIG. 8 is an explanatory diagram showing a method for detecting simultaneous multi-touch. FIG. 8A shows the relationship between the actually touched points P1 and P2 and the estimated points P1a and P2a detected by the simultaneous multi-touch detection process. FIG. 8B shows a state where the estimated points P1a and P2a are detected based on one center of gravity G3.

  As shown in FIG. 8A, the center of gravity G moves on a line connecting the first point P1 and the second point P2. That is, the center of gravity G moves at the same angle θ with respect to the X axis or the Y axis. In other words, the displacement of the center of gravity G can be expressed by a linear function. Therefore, when the center of gravity G that slightly displaces on the same line is found, it can be estimated that simultaneous multi-touch is performed. Therefore, a centroid serving as a reference for determining whether or not they are moving on the same line is determined, and an angle θ between the centroid and the centroid is calculated to check whether they match. Note that the angles θ do not have to be exactly the same, and need only match within a predetermined margin. Furthermore, simultaneous multi-touch detection accuracy can be improved by selecting a plurality of reference centroids and calculating and confirming angles for each.

  Even if it is known that the center of gravity G slightly fluctuates on the same line at the time of simultaneous multi-touch, the coordinates of the first point P1 and the second point P2 are obtained as shown by white stars in FIG. It is not possible. This is because information necessary for calculating the coordinates of the points P1 and P2 cannot be obtained.

  Therefore, in the present embodiment, as shown in FIG. 8B, the center of gravity G is considered to coincide with the center point M between the first point P1 and the second point P2. Then, from the pressing force at the center of gravity G, the coordinates and pressing force of the first point P1a and the second point P2a are estimated. Since P1a and P2a, which are estimates of the points P1 and P2, are obtained, the detection accuracy is lower than the time difference multi-touch detection, but it is possible to detect that the multi-touch operation has been performed at a certain accuracy position. Applications can be moved using information.

  A specific estimation method will be described. As shown in Equations 14 and 15, a distance R1 from the estimated first point P1a to the center of gravity G and a distance R2 from the center of gravity G to the second point P2 are calculated.

  In the following formulas 14 and 15, QT is a conversion coefficient for changing the pressing force into a length, and is set to a value such as 0.8 based on the size of the panel unit 100 or the like. The value of the conversion coefficient QT is set, for example, at the time of manufacturing the touch input device so that the distance calculated from the pressing force Fsg of the center of gravity G falls within the size of the display panel 110. For example, a configuration in which the value of the conversion coefficient QT is automatically changed by a learning algorithm or the like may be used.

  R1 = (Fsg * QT) / 2 (Formula 14)

  R2 = (Fsg * QT) / 2 (Formula 15)

  Next, the difference TempX in the X-axis direction and the difference TempY in the Y-axis direction between the center of gravity G and the estimated pressed points P1a and P2a are calculated according to Equations 16 and 17. Note that R = R1 = R2.

  TempX = R * cosθ (Formula 16)

  TempY = R * sinθ (Expression 17)

  From the difference TempX in the X-axis direction and the difference TempY in the Y-axis direction obtained by Expressions 16 and 17, the first estimated coordinates P1a (P1aX, P1aY) and the second estimated coordinates P2a (P2aX, P2aY) are obtained. Estimation is performed according to Equation 18, Equation 19, Equation 20, and Equation 21.

  P1aX = PgX-TempX (Formula 18)

  P1aY = PgY-TempY (Equation 19)

  P2aX = PgX + TempX (Equation 20)

  P2aY = PgY + TempY (Formula 21)

  In this manner, even when two places on the panel unit 100 are touched almost simultaneously, it is possible to detect that multi-touch has been performed and an approximate region where the multi-touch has been performed.

  FIG. 9 is a flowchart showing the simultaneous multi-touch detection process executed by the simultaneous multi-touch detection unit 232. The simultaneous multi-touch process is repeatedly executed every time new data is calculated, similar to the time difference multi-touch process described above. Hereinafter, the detection unit 232 may be abbreviated.

  The input calculation unit 210 performs calculation at a predetermined cycle, and this processing is executed every time a new calculation result is obtained.

  The detecting unit 232 determines whether the new data (pressed coordinates and pressing force) calculated by the input calculating unit 210 is stored in the storage unit 220 (S30). If it is determined that new data is stored in the storage unit 220 (S30: YES), the detection unit 232 analyzes the relationship between the latest data and past data (S31).

  The detection unit 232 determines whether the currently calculated center of gravity position (the coordinates of the center of gravity point G) is on the same line as the previous center of gravity position within a predetermined margin (S32). Further, the detection unit 232 changes the reference center of gravity point to determine whether the currently calculated center of gravity position and the previous center of gravity position are on the same line within a predetermined margin (S33).

  If the detection unit 232 determines that the current center-of-gravity position is on a predetermined line even when viewed from a plurality of reference center-of-gravity points (S32: YES and S33: YES), the calculation is performed based on the above-described Expressions 14 to 21. Then, the coordinates and pressing force of the two touch points P1a and P2a that have been multi-touched are estimated (S34). The detection unit 232 stores the calculation result of step S34 in the storage unit 220 (S35), and further outputs it to the application execution unit 300 (S36).

  According to this embodiment configured as described above, even in the force sensor type touch input device, a so-called multi-touch operation of touching a plurality of locations can be detected at a relatively low cost. Therefore, the operability of the force sensor touch input device can be improved, and the possibilities, application range, and application fields of the force sensor touch input device can be expanded.

  A second embodiment will be described with reference to FIGS. In the second embodiment, a so-called hybrid touch input device including a panel unit 100 for force detection and a panel unit 101 for position detection will be described as an example. In this hybrid touch input device, the coordinates of the XY plane are directly detected by the position detection panel unit 101, and further, the force (pressing force) in the Z-axis direction is detected by the force sensor type panel unit 100. That is, in the second embodiment, a touch input device that can measure a user's touch operation in three dimensions will be described.

  FIG. 10 is a block diagram illustrating the configuration of the touch input device according to the second embodiment. The hybrid panel unit 10 includes the force sensor panel unit 100 described in the first embodiment and the position detection panel unit 101.

  The position detection panel unit 101 is a panel unit that can directly measure the position touched by the user, and can be configured as, for example, a capacitive touch input device or a multilayer resistive touch input device. . The position detection panel unit 101 has a built-in sensor function for detecting the position. A signal from the position detection panel unit 101 is input to the calculation unit 200 via the signal processing unit 501. Note that the signal processing unit 501 may be provided in the position detection panel unit 101.

  The calculation unit 200 according to the second embodiment includes a coordinate detection unit 240 in addition to the input calculation unit 210, the storage unit 220, and the multi-touch detection unit 230a. The coordinate detection unit 240 detects the coordinates of the pressed point based on the signal from the position detection panel unit 101, stores the detected coordinates in the storage unit 220, and sends them to the multi-touch detection unit 230a.

  The multi-touch detection unit 230a according to the second embodiment does not need to calculate the coordinates of the two pressing points, and calculates only the pressing force at each pressing point.

  FIG. 11 is a flowchart illustrating a process of detecting multi-touch with the touch input device according to the second embodiment.

  The coordinate detection unit 240 of the calculation unit 200 detects the coordinates of each multi-touched point from the signal of the position detection panel unit 101, and outputs it to the multi-touch detection unit 230a (S40). The input calculation unit 210 of the calculation unit 200 detects the pressing force at the time of multi-touch based on the signal from the force sensor 130 (S41).

  The multi-touch detection unit 230a performs a predetermined calculation from the coordinates of each point of the multi-touch (each pressing point) detected in step S40 and the pressing force at the time of multi-touch detected in step S41. The pressing force of each multitouched point is calculated (S42). The multi-touch detection unit 230a outputs the coordinates and pressing force of each multi-touched point to the application execution unit 300 (S43).

  In this process, the coordinates (P1X, P1Y) of the first point P1 and the coordinates (P2X, P2Y) of the second point P2 can be detected by signals from the position detection panel unit 101 (S40), and the points P1, P2 are detected. The pressing force Fsg at the center of gravity G and the coordinate conversion position (PgX, PgY) of the center of gravity G can be detected by a signal from the force sensor 130 (S41).

  Accordingly, as in the case described with reference to FIG. 5D, the angle θ formed between the line segment R1 from the first point P1 to the center of gravity G and the X axis can be obtained by Equation 2. The length of the line segment R1 can be obtained by Equation 3 or Equation 4.

  Since the pressing force Fsg at the center of gravity G is obtained as the sum of the pressing force Fs1 at the first point P1 and the pressing force Fs2 at the second point P2, the pressing force Fs2 at the second point P2 is (Fsg−Fs1). ). Therefore, it is possible to detect the pressing forces Fs1 and Fs2 at the points P1 and P2 from Expression 8.

  FIG. 12 shows a method of detecting coordinates and pressing force when the three points P1, P2, and P3 are simultaneously touched with the hybrid touch input device.

  The coordinates of the first pressing point P1 are (P1X, P1Y), the coordinates of the second pressing point P2 are (P2X, P2Y), and the coordinates of the third pressing point P3 are (P3X, P3Y). The coordinates of these points are directly measured by signals from the position detection panel unit 101.

  The total load Fsg applied to the panel unit 100 (the sum of the pressing forces at the points P1, P2, and P3) and the coordinate conversion position at which the total load appears, that is, the coordinates of the barycentric point G (PgX, PgY) are a force sensor. It can be detected by analyzing 130 signals.

  By performing a predetermined calculation based on these known information (P1X, P1Y), (P2X, P2Y), (P3X, P3Y), Fsg, (PgX, PgY), three pressing points P1, P2, P3 The pressing force at can be calculated.

  As shown in FIG. 12A, an intersection point CP1 with a line perpendicular to the line connecting the point P1 and the point P3 from the point P2 is calculated. Specifically, a linear function is obtained from the points P1 and P3, a linear function is obtained from the point P2 and the center of gravity G, and an intersection point CP1 is calculated from these linear functions.

  The length R1 of the line connecting the point P1 and the intersection CP1, the length R2 of the line connecting the intersection CP1 and the point P2, the length R3 of the line connecting the point P2 and the center of gravity G, and the length of the line connecting the center of gravity G and the intersection CP1 R4 is calculated respectively. Since the coordinates of each point P1, P2, P3, G, CP1 are known, the line lengths R1, R2, R3, R4 can be determined.

  Next, Equation 22, Equation 23, and Equation 24 are solved as simultaneous equations.

  Fsg = Fs1 + Fs2 + Fs3 (Formula 22)

  R1: R2 = Fs3: Fs1 (Formula 23)

  R3: R4 = (Fs1 + Fs3): Fs2 (Equation 24)

  By solving the simultaneous equations, it is possible to calculate the pressing forces at the three pressing points P1, P2, and P3 that are not aligned on a straight line.

  As shown in FIG. 12B, even when calculation is performed using other sides, the pressing force at each of the points P1, P2, and P3 can be obtained in the same manner as described in FIG. That is, the calculation may be performed using the intersection point CP2 where the perpendicular line drawn from the point P3 intersects the line connecting the points P1 and P2, or the line connecting the point P2 and the point P3 is dropped from the point P1. The calculation may be performed by using the intersection point CP3 where the vertical line has a tolerance.

  FIG. 13 shows a case where the hybrid touch input device is touched simultaneously at three points P1, P2, and P3 arranged in a straight line. Similar to FIG. 12, (P1X, P1Y), (P2X, P2Y), (P3X, P3Y), Fsg, (PgX, PgY) are known.

  First, the intermediate point P3 is temporarily ignored, and the pressing force Fs1 at each point P1 and the pressing force Fs2 at the point P2 are calculated based on the outermost points P1 and P2 and the center of gravity G. This calculation method has been described with reference to FIG.

  Next, paying attention to the points P1 and P3 and the center of gravity G, the previously calculated pressing force Fs1 at the point P1 is divided into a pressing force only at the point P1 and a pressing force at the point P3. Thereby, it is possible to calculate the pressing force at the three points P1, P2, and P3 arranged in a straight line.

  Configuring this embodiment like this also achieves the same effects as the first embodiment. Furthermore, since the hybrid type panel unit 10 of the present embodiment includes the position detection panel unit 101 and the force sensor type panel unit 100, it is possible to more accurately detect the coordinates of each multi-touched point. Pressure can also be detected. Thereby, the range of input operation can be expanded and the application range and application field of a touch input device can be expanded.

The present invention is not limited to the above-described embodiment. A person skilled in the art can make various additions and changes within the scope of the present invention.
“Expression 1.
A first panel unit that detects and outputs a pressing force at a touched position with a plurality of force sensors;
A second panel unit that is arranged over the first panel unit and detects and outputs coordinates of a touched position;
A calculation unit that calculates coordinates and pressing force at each point of a plurality of touch inputs by performing predetermined calculation processing based on the signal from the first panel unit and the signal from the second panel unit;
A touch input device comprising:
Expression 2.
The first panel unit includes a panel and a plurality of force sensors arranged around the panel and outputting a signal corresponding to the force,
The calculation unit detects the coordinates of each point of the plurality of touch inputs with a signal from the second panel unit, and based on the detected coordinates of the point and signals from the plurality of force sensors, By performing a predetermined calculation process, the pressing force at each point of the plurality of touch inputs is calculated.
The touch input device according to expression 1.
Expression 3.
Obtain the pressing force at the center of gravity by multiple touch inputs as a signal from the force sensor,
Obtaining the coordinates of each point in the plurality of touch inputs as a signal from a position detection sensor;
By performing predetermined calculation processing based on the coordinates of each point and the pressing force at the position of the center of gravity, the pressing force at each point is calculated, and the coordinates of each point and the pressing force are output in association with each other.
An input detection method for a touch input device. "

100: Panel unit, 101: Panel unit (for position detection) 110: Display panel, 120: Cover panel, 130: Force sensor, 150: Base unit, 200, 200b: Calculation unit, 210: Input calculation unit, 220: Storage Unit, 230, 230a: multi-touch detection unit, 240: coordinate detection unit, 300: application execution unit, 400: display control unit

Claims (11)

A panel,
A plurality of force sensors provided around the panel and outputting a signal corresponding to the force;
A calculation unit that calculates coordinates and pressing force at each point of a plurality of touch inputs to the panel by performing predetermined calculation processing based on signals from the plurality of force sensors;
A touch input device comprising: The computing unit is
Based on signals from the plurality of force sensors, the coordinates and pressing force input to the panel are calculated,
Determining whether the calculated coordinates and pressure of the touch input satisfy a predetermined pattern discrimination condition set in advance;
When it is determined that the predetermined pattern determination condition is satisfied, by performing a predetermined calculation process, the coordinates and pressing force at each point of the plurality of touch inputs to the panel are calculated.
The touch input device according to claim 1. The predetermined pattern determination condition includes a first pattern determination condition for determining a first pattern having a predetermined time difference between a plurality of touch inputs to the panel,
When the first pattern determination condition is satisfied, the calculation unit performs a predetermined first calculation process included in the predetermined calculation process, so that each calculation input point has a predetermined time difference. Calculate the coordinates and pressing force of
The touch input device according to claim 2. The predetermined pattern determination condition includes a second pattern determination condition for determining a second pattern in which a plurality of touch inputs to the panel are performed within a predetermined time.
When the second pattern determination condition is satisfied, the calculation unit performs each of the plurality of touch inputs performed within the predetermined time by executing a predetermined second calculation process included in the predetermined calculation process. Calculate the coordinates and pressing force at the point,
The touch input device according to claim 2. The first pattern determination condition is that the pressing force of the touch input detected this time is larger than the pressing force of the touch input detected last time, and the coordinates of the touch input detected this time are those of the touch input detected so far. It is defined to determine that it is the first pattern when moving along a predetermined direction obtained from a change in coordinates.
The touch input device according to claim 3. The second pattern determination condition is determined to be the second pattern when the coordinates of the touch input detected this time are on a predetermined line obtained from the change in the coordinates of the touch input detected up to the previous time. Stipulated,
The touch input device according to claim 4. In the second pattern discrimination condition, a plurality of different coordinates are selected as reference coordinates from the coordinates of the touch input detected so far, and the predetermined lines are set based on the plurality of reference coordinates, respectively, and detected this time. When the coordinates of the touch input are present on any of the plurality of predetermined lines, the second pattern is defined to be determined.
The touch input device according to claim 6. It further includes a position detection panel unit that directly detects the touch-input coordinates,
The calculation unit calculates the pressing force at each point of the plurality of touch inputs to the panel using the coordinates of the touch input detected by the position detection panel unit.
The touch input device according to claim 1. A method of detecting an input using a touch input device including a panel, a plurality of force sensors provided around the panel, and a calculation unit that receives and calculates signals from the force sensors,
The computing unit is
Obtaining signals from the plurality of force sensors;
By performing predetermined calculation processing based on signals acquired from the plurality of force sensors, the coordinates and pressing force at each point of the plurality of touch inputs to the panel are calculated.
An input detection method for a touch input device. The computing unit is
Obtaining signals from the plurality of force sensors;
Based on the signals acquired from the plurality of force sensors, the coordinates and pressing force input by touching the panel are calculated,
Determining whether the calculated coordinates and pressure of the touch input satisfy a predetermined pattern discrimination condition set in advance;
When it is determined that the predetermined pattern determination condition is satisfied, by performing a predetermined calculation process, the coordinates and pressing force at each point of the plurality of touch inputs to the panel are calculated.
The input detection method of the touch input device according to claim 9. A computer program for detecting multi-touch to a touch input device using a computer that receives signals from a plurality of force sensors provided on a panel,
Obtaining signals from the plurality of force sensors;
By performing predetermined calculation processing based on signals acquired from the plurality of force sensors, the coordinates and pressing force at each point of the plurality of touch inputs to the panel are calculated.
A computer program for causing the computer to execute the above.

Images