WO2011145330A1 - 座標決定装置、座標決定方法及び座標決定プログラム - Google Patents
座標決定装置、座標決定方法及び座標決定プログラム Download PDFInfo
- Publication number
- WO2011145330A1 WO2011145330A1 PCT/JP2011/002729 JP2011002729W WO2011145330A1 WO 2011145330 A1 WO2011145330 A1 WO 2011145330A1 JP 2011002729 W JP2011002729 W JP 2011002729W WO 2011145330 A1 WO2011145330 A1 WO 2011145330A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- coordinate
- input
- unit
- information
- time
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0416—Control or interface arrangements specially adapted for digitisers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0489—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using dedicated keyboard keys or combinations thereof
- G06F3/04892—Arrangements for controlling cursor position based on codes indicative of cursor displacements from one discrete location to another, e.g. using cursor control keys associated to different directions or using the tab key
Definitions
- the present invention relates to a coordinate determination device, a coordinate determination method, and a coordinate determination program, and more particularly to a coordinate determination device that determines a coordinate position of a pointer on a display unit included in a display device based on input information acquired from the coordinate input device. .
- TVs can be used not only for viewing broadcast programs and movies, but also for various information browsing via the Internet connection, linkage with home devices connected to the network, New functions such as various applications such as photo browsing and games are being realized and used.
- the pointing device is an input device that can control a pointer or cursor displayed on the screen and can specify an arbitrary position and direction, such as a touch panel, a trackball, a joystick, and a mouse.
- a touch panel such as a touch panel, a trackball, a joystick, and a mouse.
- a pointer position presentation method and operation method on a display device based on such a conventional coordinate input device includes a classification axis “whether the coordinate input device and the display device are integrated or separated”, and “user Can be roughly divided into four categories according to the classification axis “whether the coordinate system on the coordinate input device side that is input by and the coordinate system on the display device side displayed by the system corresponds to one body 1”.
- the integrated type refers to “a shape that overlaps like a touch panel as one device”
- the separation type refers to a “separated shape such as a personal computer display and a mouse”.
- the coordinate system input by the user and the coordinate system displayed by the system corresponding to one body 1 will be referred to as an absolute coordinate mode, and the other will be referred to as a relative coordinate mode for convenience. Further, switching between the two coordinate modes is referred to as coordinate mode switching for convenience.
- the display unit of the display device and the input detection unit of the coordinate input device do not physically overlap, and the coordinate input is indirect to the display device.
- Coordinates on the input detection unit such as a touch pad or tablet are associated with the coordinates on the display unit on a one-to-one basis.
- Touchpad for handwritten character input devices external input tablet for drawing tools, etc.
- Wacom's pen input tablet and touch tablet (not integrated with liquid crystal). Since the coordinate point touched (started to input) on the input detection unit is directly converted into the corresponding coordinate position on the display unit, it is used for input of handwritten Chinese characters, illustration drawing, and the like.
- Integrated and in relative coordinate mode In some game applications using a touch panel, in order to operate the main character in the game, a part of the display unit of the display device and the input detection unit of the coordinate input device is used. There is an example to use.
- the control device / method corresponding to this case is not common compared to the above (1) to (3).
- FIGS. 16 and 17 show pointing examples using “separate type absolute coordinate mode” and “separate type relative coordinate mode”.
- the input detection unit 210 is, for example, a touch pad.
- the display unit 230 is an LCD (Liquid Crystal Display) or the like, for example.
- the coordinate mode it is assumed that the trace of the pointer is displayed on the display unit 230 when the input detection unit 210 is traced with a finger.
- FIG. 16 is a diagram showing a correspondence between a finger trajectory in the input detection unit 210 and a pointer trajectory in the display unit 230 in the absolute coordinate mode.
- a locus display A620 is displayed on the display unit 230.
- the contact coordinates on the input detection unit 210 correspond one-to-one with the coordinates on the display unit 230.
- FIG. 17 is a diagram showing the correspondence between the finger trajectory in the input detection unit 210 and the pointer trajectory in the display unit 230 in the relative coordinate mode.
- the display unit 230 displays a continuous trajectory display B640, trajectory display C642, and trajectory display D644.
- the trajectory display B640, the trajectory display C642, and the trajectory display D644 respectively correspond to the input of the trajectory B630, the trajectory C632, and the trajectory D634.
- JP 2001-117713 A JP 10-340153 A Japanese Patent Laid-Open No. 11-095912 JP 2006-201916 A JP 2001-228964 A
- the conventional coordinate input device has a problem that the coordinate mode cannot be automatically switched in a natural operation / operation system for the user.
- relative operation is automatically performed without operation of a physical switch or the like for switching the coordinate mode. It is necessary to switch between coordinate mode and absolute coordinate mode.
- the conventional technique cannot be realized by a coordinate input device that automatically switches the coordinate mode.
- the present invention solves such conventional problems, and provides a coordinate determination device that automatically recognizes a coordinate mode that is easier for the user to use while performing natural operations and switches the coordinate mode. With the goal.
- a coordinate determination apparatus is a coordinate determination apparatus that determines output coordinates according to input coordinates, and acquires coordinate information indicating the input coordinates and input time sequentially and stores the information in a recording medium.
- a storage unit a time correlation specifying unit for specifying a correlation between input times indicated by the first and second coordinate information among the plurality of coordinate information stored in the recording medium as a time correlation value;
- a coordinate correlation specifying unit that specifies a correlation between input coordinates indicated by the second and third coordinate information of the coordinate information as a coordinate correlation value, and a first coordinate based on the coordinate correlation value and the time correlation value
- a coordinate mode selection unit that selects one of the mode and the second coordinate mode as a coordinate mode, and an input coordinate indicated by the second coordinate information by the coordinate mode selection unit.
- a coordinate conversion unit that determines the coordinates as the output coordinates by converting the coordinates according to the selected coordinate mode, and the coordinate conversion unit, when the first coordinate mode is selected, When the input coordinate indicated by the second coordinate information is converted into a coordinate predetermined and associated with the input coordinate and the second coordinate mode is selected, the input coordinate indicated by the second coordinate information is displayed. Is converted into coordinates corresponding to the output coordinates already determined for other input coordinates.
- the feature amount corresponding to the time of the touch operation and the coordinate position is calculated without any special operation such as a physical switch by the user, and these are integrated to determine the user's mental model.
- the coordinate mode that is easier for the user to use can be automatically recognized and the coordinate mode can be switched while the user performs a natural motion.
- the coordinate correlation specifying unit specifies a distance between the input coordinates as the coordinate correlation value, and determines whether or not the specified coordinate correlation value is less than a predetermined threshold value.
- the time correlation specifying unit specifies the difference between the input times as the time correlation value, determines whether the specified time correlation value is less than a predetermined time, and the coordinate mode
- the selection unit is The coordinate mode may be selected as the coordinate mode.
- the coordinate mode according to the mental model of the user who operates the coordinate input device is automatically recognized using the operation time at the time of the touch operation and the distance between the touched coordinates as the feature amount. Can do.
- the acquisition storage unit acquires coordinate information from a coordinate input device having an input detection unit
- the first coordinate information is a coordinate corresponding to a time when a user lifts a finger from the input detection unit.
- the second coordinate information may be coordinate information corresponding to a time when the user again touches the input detection unit with the finger after the first coordinate information is detected.
- the third coordinate information is coordinate information corresponding to a time when a user touches the input detection unit with a finger, and the first coordinate information is detected from the third coordinate information. Is coordinate information corresponding to a time when the user lifts his / her finger from the input detection unit, and the second coordinate information is detected by the user after the first coordinate information is detected.
- the coordinate information may correspond to the time when the finger is brought into contact again.
- the acquisition storage unit further acquires selection completion information including a selection completion time that is a completion time of a selection operation by a user and stores the selection completion information in the recording medium, and the coordinate mode selection unit selects the selection from the recording medium.
- selection completion information including a selection completion time that is a completion time of a selection operation by a user and stores the selection completion information in the recording medium
- the coordinate mode selection unit selects the selection from the recording medium.
- This process prevents the user from unintentionally switching the coordinate mode with a secondary release accompanying a click.
- the coordinate determination device further includes the coordinate input device.
- the coordinate input device accepts a coordinate input operation and a selection operation by a user, and acquires the coordinate information and the selection completion information from the acquisition storage unit.
- the input detection unit is a clickable touch pad capable of detecting a mechanical push-down operation, and the return operation of the clickable touch pad caused by a user releasing a finger that pushed down the clickable touch pad. May be specified as the selection completion time.
- the user in a coordinate input device equipped with a clickable touchpad, the user automatically recognizes a coordinate mode that is easier to use for the user while performing natural operations without any special operation such as a physical switch by the user.
- the coordinate mode can be switched.
- the coordinate mode selection unit has an operation degree indicating a user operation characteristic based on at least one or more of the plurality of coordinate information and selection information acquired by the acquisition storage unit before the second coordinate information. It may be calculated and at least one of the specific time, the specific threshold, and the specific grace time may be changed based on the calculated degree of operation.
- a coordinate mode that is easier to use for the user can be automatically recognized by each user while performing a more natural operation, and the coordinate mode can be switched.
- the coordinate mode selection unit calculates a difference between an input time indicated by the coordinate information and an input time indicated by other coordinate information.
- the reciprocal of the calculated average value of the plurality of differences may be calculated as the operation degree.
- the user's familiarity with the coordinate input system can be used as the degree of operation.
- the coordinate mode selection unit may change the specific time, the specific threshold value, and the specific grace time so as to decrease as the calculated operation degree increases.
- the user can quickly recognize the natural motion.
- the position indicated by the third coordinate information may coincide with the position indicated by the first coordinate information.
- the first coordinate information is included in a first coordinate series that is a series of coordinate information that is input from when the user once touches the input detection unit until the user releases the finger
- the second coordinate information is included.
- Is included in the second coordinate series which is a series of coordinate information input from when the user touches the input detection unit again after the input of the first coordinate series until the user releases the finger.
- the coordinate information 3 may be included in the first coordinate series.
- the coordinate mode selection unit has not passed a predetermined time from the input time indicated by the first coordinate information, and the acquisition storage unit stores the second coordinate information. If it is not acquired, a specific image may be displayed on the display unit.
- This display allows the user to predict the coordinate mode that the system automatically recognizes.
- the coordinate determination apparatus further includes an operation recognition unit that determines whether or not the first coordinate series is a gesture for instructing a display device to start a specific computer process.
- the coordinate mode selection unit may not select the coordinate mode.
- a coordinate mode that is easy for the user to use can be automatically recognized and the coordinate mode can be switched while the user performs natural operations.
- the acquisition storage unit further acquires grip information indicating which position of the coordinate input device the user is gripping and stores the grip information in the recording medium
- the coordinate determination device further includes the recording A grip estimation unit configured to estimate a grip state of the coordinate input device by a user using the grip information acquired from a medium, and the coordinate mode selection unit, according to the grip state estimated by the grip estimation unit, You may change at least 1 value among a specific threshold value, the said specific time, and the said specific grace time.
- the coordinate input device includes at least two or more input detection units, and the acquisition storage unit acquires coordinate information acquired from a first input detection unit among the plurality of input detection units and a second input detection unit.
- the coordinate information acquired from the recording medium may be stored in the recording medium.
- a coordinate determination device that determines output coordinates according to input coordinates
- an acquisition storage unit that sequentially acquires coordinate information indicating the input coordinates from the coordinate input device and stores them in a recording medium, and is displayed on a display device.
- a coordinate on the display device corresponding to the center of the icon and an output coordinate obtained by converting the input coordinate indicated by the second coordinate information among the plurality of coordinate information stored in the recording medium according to the absolute coordinate mode.
- a coordinate correlation specifying unit that specifies a distance, a coordinate mode selection unit that selects one of an absolute coordinate mode and a relative coordinate mode as a coordinate mode based on the distance, and the second coordinate information.
- the coordinates are determined as the output coordinates.
- a coordinate conversion unit and when the absolute coordinate mode is selected, the coordinate conversion unit converts the input coordinate indicated by the second coordinate information into a coordinate that is predetermined and associated with the input coordinate.
- the coordinate determination device may convert the input coordinates indicated by the second coordinate information into coordinates corresponding to output coordinates already determined with respect to other input coordinates. .
- the coordinate determination device can automatically select the absolute coordinate mode, and the coordinate mode according to the user's intention can be selected. You can choose.
- a coordinate determination device that automatically recognizes a coordinate mode that is easier to use for the user while the user performs a natural motion and switches the coordinate mode.
- FIG. 1 is an external view of a coordinate input system that realizes a coordinate determination device according to the present embodiment.
- FIG. 2 is a functional block diagram showing an example of a coordinate determination device in the present embodiment.
- FIG. 3 is a schematic diagram illustrating an example of a user operation in the input detection unit according to the present embodiment.
- FIG. 4 is a schematic diagram showing an example of contact coordinates used for determination of the coordinate mode in the present embodiment.
- FIG. 5 is a state transition diagram showing a coordinate mode determination process for input information obtained after the end of pointer display in the present embodiment.
- FIG. 6 is a state transition diagram showing a coordinate mode determination process for input information acquired before the end of pointer display in the present embodiment.
- FIG. 7 is a flowchart showing the processing of the coordinate determination apparatus in the present embodiment.
- FIG. 8 is a flowchart showing details of event processing other than the release event in the present embodiment.
- FIG. 9 is a flowchart showing processing for changing a threshold value or the like according to the degree of operation in the first modification example of the present embodiment.
- FIG. 10 is a schematic diagram illustrating an example of a distance used for determination of the coordinate mode in the second modification example of the present embodiment.
- FIG. 11 is a flowchart showing the processing of the coordinate determination device in the third modification example of the present embodiment.
- FIG. 12 is an external view showing an example of a computer system that realizes the coordinate determination device according to the embodiment of the present invention.
- FIG. 13 is a block diagram showing a hardware configuration of a computer system that realizes the coordinate determination apparatus according to the embodiment of the present invention.
- FIG. 14A is a flowchart of the coordinate switching means disclosed in the conventional example (Patent Document 1).
- FIG. 14B is a flowchart of the coordinate switching means disclosed in the conventional example (Patent Document 1).
- FIG. 15a is a flowchart of the coordinate switching means disclosed in the conventional example (Patent Document 5).
- FIG. 15b is a flowchart of the coordinate switching means disclosed in the conventional example (Patent Document 5).
- FIG. 16 is a diagram showing the correspondence between the locus on the input detection unit and the locus on the display device in the absolute coordinate mode.
- FIG. 17 is a diagram illustrating the correspondence between the locus on the input detection unit and the locus on the display device in the relative coordinate mode.
- FIG. 14a and 14b are flowcharts showing a conventional coordinate mode switching process described in Patent Document 1.
- FIG. 14a and 14b are flowcharts showing a conventional coordinate mode switching process described in Patent Document 1.
- step S2 the coordinate input system sets the coordinate mode to the absolute coordinate mode.
- the coordinate input system corrects the coordinates input to the touchpad by the user in step S4 within the size determined in step S5 (in the absolute coordinate mode).
- step S8 the coordinate input system continues the above processing.
- step S8 If the coordinate input system does not detect the input of the ESC key from the user in step S8, the coordinate input system switches to the relative coordinate mode in step S9.
- step S14 the coordinate input system determines whether or not the mouse cursor (pointer) is within a predetermined area.
- the coordinate input system switches the coordinate mode from the relative coordinate mode to the absolute coordinate mode in step S15. Thereafter, the coordinate input system sets the coordinate mode to the absolute coordinate mode until the coordinate input system detects the input of the ESC key in step S19.
- the coordinate mode switching in Patent Document 1 is performed when the coordinate input system detects the input of the ESC key by the user.
- Patent Document 5 it is proposed to switch the “input mode” using a time interval of input to the touch pad.
- steps S44 to S49 of FIG. 15a an average value of n coordinate inputs on the touch pad is calculated, and this is used as an input coordinate value, and the coordinate mode is determined in step S50.
- the determination of the coordinate mode is performed based on “whether the input interval on the touch pad is within a predetermined time” as shown in step S34 of FIG. 15b. That is, if it is outside the predetermined time, the mode is switched to the input mode called “first mode” in step S35, while if it is within the predetermined time, the mode is switched to “second mode or absolute value mode” in step S36.
- the difference between the first mode and the second mode disclosed in Patent Document 5 is only the presence / absence of the determination part “presence of previous coordinate value?” In step S55. In actual coordinate processing, the first mode and the second mode are different. In both the second mode, the “relative value of the current input coordinate value with respect to the previous input coordinate value” in steps S57 to S59 is output.
- both the first mode and the second mode are so-called relative coordinate modes that output “relative values between input coordinate values”.
- Patent Document 5 does not disclose a criterion for the coordinate input system to determine whether the second mode or the absolute value mode should be used. That is, it is only described that whether or not the mode is the absolute value mode is determined only by setting with a physical switch (described as “switch 5” in Patent Document 5) (In Patent Document 5, the absolute value mode is described). "[0032] The switch 5 switches between the absolute value mode and the relative value mode of the coordinate input panel 10 according to the power supply voltage 29. By switching the switch 5, the CPU 3 switches to the absolute value mode. And the relative value mode is switched to detect the coordinate value.
- the user's input intention (that is, whether the user wants to input as absolute coordinates or relative coordinates) is estimated from the user's input operation regardless of the application.
- switching the coordinate mode used by the coordinate input device is not an object of the invention.
- the touch pad will be described as an example.
- the prior art has the following problems.
- Patent Documents 1 and 2 and Patent Documents 4 to 5 Each time the user desires to switch the coordinate mode, it is necessary to perform a coordinate mode switching operation using a specific button, switch, input detection unit, or the like.
- Patent Documents 1 and 3 The user is forced to move the pointer to a specific position in order to switch the coordinate mode, and if the pointer is moved to that position, the switch is forcibly generated, so it is impossible to switch the operation system freely.
- Non-Patent Document 1 The user is forced to learn a new (unusual) operation such as operating the pointer while controlling the contact area of the finger in order to switch the coordinate mode, and the operation is also difficult on the operation surface. Forced (Non-Patent Document 1).
- the present invention solves the above conventional problems by providing a coordinate determination device that can appropriately select a coordinate mode according to a user's mental model at the time of the input operation based on the input operation of the user. can do.
- FIG. 1 is an external view of a coordinate input system 90 realized by using a coordinate determination device 200 according to the present invention in the present embodiment.
- the coordinate input system 90 includes a display device 100 and a coordinate input device 201.
- the display device 100 is a device that receives a user input from the coordinate input device 201 and displays a processing result corresponding to the input on the display unit 230.
- a television or a video recording / playback device can be considered, but the present invention is not limited thereto.
- the coordinate input device 201 is an input device that inputs user operation information to the display device 100 connected by wire or wirelessly.
- a remote controller such as a television or a video recording / playback device is conceivable, but is not limited thereto.
- the display device 100 includes a coordinate determination device 200 and a display unit 230.
- the coordinate determination device 200 is a processing device that acquires user input information input to the coordinate input device 201 and displays it on the display unit 230.
- the coordinate determination device 200 automatically determines a coordinate mode that is easier for the user to use, and after converting the input coordinates acquired from the coordinate input device 201 into the pointer coordinate position according to the determined coordinate mode. To the display unit 230.
- the pointer coordinate position is an output coordinate of the coordinate determination device 200.
- the display unit 230 is a visual display device included in the display device 100.
- LCD and CRT Cathode Ray Tube
- the display unit 230 displays the pointer 110 at the pointer coordinate position output by the coordinate determination device 200.
- the pointer 110 is a small graphic that is displayed to indicate the object of investigation in a GUI (Graphical User Interface) environment.
- GUI Graphic User Interface
- the pointer 110 displayed on the display unit 230 moves.
- the pointer 110 is a figure or the like shaped like an arrow or a finger.
- the coordinate input apparatus 201 includes an input detection unit 210, a grip detection sensor 212, and a transmission unit 220.
- the input detection unit 210 is a pointing device that acquires input information.
- a device used for the input detection unit 210 for example, a touch pad, a tablet, a trackball, or the like can be considered.
- the input information acquired by the input detection unit 210 is mainly coordinate information and selection information.
- the coordinate information is, for example, input coordinates and the input time that the user inputs by touching the input detection unit 210 with a finger. That is, the coordinate determination device 200 determines the pointer coordinate position that is the output coordinate from the input coordinates included in the coordinate information.
- the input coordinates included in the coordinate information correspond to the position where the user touches the finger on the input detection unit 210.
- the input coordinates are values representing points on a two-dimensional or three-dimensional space, for example.
- the user can move the pointer 110, for example, by inputting coordinate information using the input detection unit 210.
- the selection information is input information indicating either the start or the end of the selection operation that the user inputs by tapping or clicking the input detection unit 210, for example.
- the selection information is one of selection start time information indicating the selection start time and selection end time information indicating the selection end.
- the input detection unit 210 when the input detection unit 210 is a clickable touch pad, the input detection unit 210 outputs the time when the user has pushed the input detection unit 210 as selection start time information.
- the time when the input detection unit 210 returns to the original position is selected by the spring mechanism of the clickable touchpad.
- the input detection unit 210 outputs the time information.
- the user can select the icon displayed on the display unit 230 and execute processing associated with the icon by inputting selection information using the input detection unit 210, for example.
- a drag operation can be performed by combining the selection information and the coordinate information.
- the grip detection sensor 212 is a sensor that detects which position of the casing of the coordinate input device 201 the user is gripping and outputs as grip information.
- the coordinate determination device 200 can hold the coordinate input device 201 with the left hand or the right hand, for example. Get information to determine whether or not.
- the detection method of the grip detection sensor 212 may be a resistance film type, an infrared type, a SAW type, an electrostatic type, or the like, but is not limited thereto.
- the transmission unit 220 is a communication interface.
- the transmission unit 220 transmits the input information detected by the input detection unit 210 to the coordinate determination device 200.
- the input information transmitted by the transmission unit 220 includes a plurality of coordinate information, selection information, grip information, and the like.
- the communication method used by the transmission unit 220 includes, but is not limited to, wireless LAN, infrared communication, short-range wireless communication, and the like.
- the coordinate input device 201 and the coordinate determination device 200 include an arithmetic device such as a CPU (Central Processing Unit), a storage such as a ROM (Read Only Memory) and a RAM (Random Access Memory).
- arithmetic device such as a CPU (Central Processing Unit)
- a storage such as a ROM (Read Only Memory) and a RAM (Random Access Memory).
- An apparatus may be provided.
- the user can move the pointer 110 displayed on the display unit 230 included in the display device 100 to a desired position by performing an operation on the input detection unit 210 included in the coordinate input device 201. it can.
- a desired icon can be selected.
- the input detection unit 210 in FIG. 1 is a personal computer mouse (in addition to a general scroll mouse, pointing in the air such as Logitech Air Mouse (registered trademark) MX) Air) It is assumed that the device is generally available, such as an arbitrary mouse, such as a mouse that can be used), an aerial pointing device (such as a Wii (registered trademark) remote control using a camera), and a tablet-type input device.
- a personal computer mouse in addition to a general scroll mouse, pointing in the air such as Logitech Air Mouse (registered trademark) MX) Air
- the device is generally available, such as an arbitrary mouse, such as a mouse that can be used), an aerial pointing device (such as a Wii (registered trademark) remote control using a camera), and a tablet-type input device.
- the coordinate input device 201 and the coordinate determination device 200 are separated, but the coordinate input device 201 and the coordinate determination device 200 may be integrated.
- FIG. 2 is a block diagram showing functional configurations of the coordinate input device 201 and the display device 100 in the present embodiment.
- the same elements as those in FIG. 1 are denoted by the same reference numerals and description thereof is omitted.
- the coordinate determination device 200 includes an acquisition storage unit 204, a detection result series storage unit 314, a grip estimation unit 316, an operation recognition unit 318, a time correlation specification unit 320, a coordinate correlation specification unit 324, and a coordinate mode selection unit. 326 and a coordinate conversion unit 328.
- the acquisition storage unit 204 sequentially acquires input information (coordinate information, selection information, grip information, etc.) transmitted from the coordinate input device 201 by wireless or wired, and stores it in the detection result sequence storage unit 314.
- the detection result series storage unit 314 is a recording medium and can be realized by, for example, a RAM.
- the grip estimation unit 316 acquires the grip information output from the grip detection sensor 212 from the detection result series storage unit 314. Thereafter, the grip estimation unit 316 estimates the gripping state of the user from the grip information and outputs it to the coordinate mode selection unit 326.
- the operation recognizing unit 318 acquires input information (coordinate information, selection information, etc.) from the detection result series storage unit 314, and whether the input information matches any of a plurality of predetermined “events”. Recognize whether or not. Thereafter, the operation recognition unit 318 notifies the coordinate mode selection unit 326 of the recognition result.
- the types of events recognized by the operation recognition unit 318 include, for example, “(pointer) movement”, “release”, “gesture (slide, flick, etc.)” and “selection”.
- the selection event and the release event are events that can be recognized from one selection information.
- a movement event and a gesture event can be distinguished only after acquiring a series of coordinate information.
- the operation recognition unit 318 may recognize a series of coordinate information once recognized as a movement event, for example, as a gesture for drawing a circle and then separately recognize it as a gesture event.
- the time correlation specifying unit 320 includes an input time indicated by the first coordinate information and an input time indicated by the second coordinate information among the plurality of coordinate information acquired from the detection result series storage unit 314.
- the correlation between the two is specified as a time correlation value, and is output to the coordinate mode selection unit 326.
- “specify the correlation as a time correlation value” means, for example, that after calculating the difference between the input time when the first coordinate information is input and the input time when the second coordinate information is input, It is to compare a specific time set in advance.
- the time correlation identification unit 320 outputs the magnitude relationship between the difference between the two input times and the specific time to the coordinate mode selection unit 326 as a time correlation value.
- the coordinate correlation specifying unit 324 includes an input coordinate indicated by the third coordinate information and an input coordinate indicated by the second coordinate information among the plurality of coordinate information acquired from the detection result series storage unit 314.
- the correlation between the two is specified as a coordinate correlation value, and is output to the coordinate mode selection unit 326.
- “identify correlation as a coordinate correlation value” is determined in advance after calculating the distance between the input coordinates of the third coordinate information and the input coordinates of the second coordinate information, for example. Compare specific thresholds.
- the coordinate correlation specifying unit 324 outputs the magnitude relationship between the distance between two input coordinates and a specific threshold value to the coordinate mode selection unit 326 as a coordinate correlation value.
- the coordinate mode selection unit 326 selects the absolute coordinate mode (that is, the first coordinate mode) and the relative coordinate mode (that is, the second coordinate mode) based on the identification results acquired from the coordinate correlation identification unit 324 and the time correlation identification unit 320.
- the coordinate mode is determined to be one of the following.
- the coordinate mode selection unit 326 may determine the coordinate mode to be the relative coordinate mode. A more detailed determination method will be described later.
- the coordinate conversion unit 328 converts the input coordinates indicated by the coordinate information into a pointer coordinate position on the display unit 230 according to the coordinate mode determined by the coordinate mode selection unit 326, and the converted pointer coordinate position is displayed on the display unit 230. Output to.
- the coordinate mode selection unit 326 determines the coordinate mode as the relative coordinate mode.
- the coordinate conversion unit 328 converts the input coordinate corresponding to the start end of the trajectory C632 into a pointer coordinate position corresponding to the end of the trajectory display B640 and outputs the converted coordinate to the display unit 230.
- the coordinate conversion unit 328 converts the input coordinate corresponding to the start end of the trajectory C632 into a coordinate corresponding to the end of the trajectory display B640, which is an output coordinate already determined with respect to other input coordinates.
- the coordinate conversion unit 328 converts the input coordinates corresponding to the starting end of the trajectory C632 into coordinates that are predetermined and associated with the input coordinates.
- the coordinate conversion unit 328 performs this association according to a static rule (that is, determined independently of coordinate information other than the coordinate information corresponding to the starting end of the trajectory C632).
- the coordinate determination apparatus 200 includes only the acquisition storage unit 204, the time correlation specification unit 320, the coordinate correlation specification unit 324, the coordinate mode selection unit 326, and the coordinate conversion unit 328. Can also achieve the object of the invention.
- the detection result sequence storage unit 314 is not included as a component of the coordinate determination device 200, for example, the coordinate input device 201 or the display device 100 other than the coordinate determination device 200 includes the detection result sequence storage unit 314.
- the invention can be implemented.
- the present invention is implemented even if the coordinate input device 201 estimates the grip state and the coordinate determination device 200 acquires the estimation result, for example. be able to. Further, as will be described later, the coordinate mode selection unit 326 only uses the gripping state as additional information for selecting the coordinate mode, so that the present invention can be used even if the coordinate mode selection unit 326 does not use the gripping state. Can be implemented.
- the present invention is implemented. be able to. Further, when the types of events that occur are limited, for example, in a system that aims only at the movement of the pointer 110, it is possible to know the events that have occurred without recognizing the user's operation. Therefore, the present invention can be implemented without the operation recognition unit 318.
- FIG. 3 is a schematic diagram showing an example of a user operation in the input detection unit related to the determination of the coordinate mode in the present embodiment.
- the user operates the input detection unit 210 in the order of the first input state 810 to the fourth input state 816.
- the current finger position 800 is at the contact start point 904.
- the input coordinates and input time of the contact start point 904 can be used.
- the user is about to release the finger from the movement end point 906 on the input detection unit 210.
- the first coordinate information in the present invention it is conceivable to use the input coordinates and input time of the movement end point 906 immediately before the finger is released from the input detection unit 210.
- the user In the third input state 814, the user once lifts the finger from the movement end point 906 and puts the finger again at the contact start point 910.
- the input coordinates and input time of the contact start point 910 can be used.
- the user In the fourth input state 816, the user is moving his / her finger on the input detection unit 210, and the current finger position 800 is at the moving point 912.
- a series of coordinate information input from when the user once touches the contact start point 904 on the input detection unit 210 until the user releases the finger at the movement end point 906 is defined as a first coordinate series.
- FIG. 4 is a diagram showing G_it (x, y) 510, G_n (x, y) 512, and G_cur (x, y) 520, which are coordinates on the input detection unit 210.
- G_it (x, y) 510 is the beginning of the first coordinate series. Therefore, the contact start point 904 in FIG. 3 is G_it (x, y) 510.
- G_n (x, y) 512 is the start of the second coordinate series. Therefore, the contact start point 910 in FIG. 3 is G_n (x, y) 512.
- G_cur (x, y) 520 is a coordinate currently in contact with the input detection unit 210. Therefore, the movement point 912 in FIG. 3 is G_cur (x, y) 520.
- the coordinate mode selection unit 326 selects a coordinate mode using the input coordinates, input time, grip information, etc., such as G_it (x, y) 510, G_n (x, y) 512, and G_cur (x, y) 520. To do.
- FIG. 5 is a state transition diagram illustrating an example of a coordinate mode determination process performed by the coordinate mode selection unit 326 in the present embodiment.
- the coordinate mode selection unit 326 uses the absolute coordinate mode 130 and the relative coordinates as coordinate modes for displaying G_n (x, y) 512, which is the start of the second coordinate series acquired from the input detection unit 210, on the display unit 230. One of the modes 132 is selected. That is, the coordinate mode selection unit 326 selects a coordinate mode for outputting the input coordinates indicated by the second coordinate information as the pointer coordinate position.
- the current coordinate mode is the absolute coordinate mode 130.
- Wr be the time from when the user who inputs coordinate information releases his / her finger from the input detection unit 210 until he / she touches the input detection unit 210 again.
- the time difference from when the user removes the finger from the input detection unit 210 at the movement end point 906 to when the user places the finger again at the contact start point 910 on the input detection unit 210 is Wr. More specifically, let Wr be the difference between the input time indicated by the first coordinate information and the input time indicated by the second coordinate information.
- the time correlation identification unit 320 compares Wr with the specific time Wd, and notifies the coordinate mode selection unit 326 of a time correlation value indicating whether or not Wr is equal to or greater than the specific time Wd.
- the coordinate mode selection unit 326 acquires the notification, and if Wr is equal to or greater than the specific time Wd, the coordinate mode selection unit 326 selects the absolute coordinate mode 130 as the coordinate mode (S800).
- the coordinate mode selection unit 326 obtains the notification, and if Wr is less than the specific time Wd, the coordinate mode selection unit 326 selects the relative coordinate mode 132 as the coordinate mode (S806).
- the current coordinate mode is the relative coordinate mode 132.
- the time correlation identification unit 320 compares Wr with the specific time Wd and notifies the coordinate mode selection unit 326 of a time correlation value indicating whether or not Wr is equal to or greater than the specific time Wd.
- the coordinate mode selection unit 326 acquires the notification, and if Wr is equal to or greater than the specific time Wd, the coordinate mode selection unit 326 selects the absolute coordinate mode 130 as the coordinate mode (S802).
- the coordinate mode selection unit 326 obtains the notification, and if Wr is less than the specific time Wd, the coordinate mode selection unit 326 selects the relative coordinate mode 132 as the coordinate mode (S804).
- FIG. 6 is a state transition diagram showing another example of the coordinate mode selection process performed by the coordinate mode selection unit 326 in the present embodiment.
- the coordinate mode selection unit 326 displays the coordinate mode for displaying G_n (x, y) 512, which is the start of the second coordinate series acquired from the input detection unit 210, on the display unit 230 as an absolute coordinate.
- G_n (x, y) 512 which is the start of the second coordinate series acquired from the input detection unit 210, on the display unit 230 as an absolute coordinate.
- One of the mode 130 and the relative coordinate mode 132 is selected.
- the current coordinate mode is the absolute coordinate mode 130.
- Rm be the distance between two points of the initial contact position G_it (x, y) 510 and the initial position G_n (x, y) 512.
- the distance between the contact start point 904 and the contact start point 910 is Rm. More specifically, the distance between the input coordinates indicated by the third coordinate information and the input coordinates indicated by the second coordinate information is Rm.
- the coordinate correlation specifying unit 324 compares Rm with a specific threshold value Rr, and notifies the coordinate mode selection unit 326 as a coordinate correlation value whether or not Rm is larger than the specific threshold value Rr.
- the coordinate mode selection unit 326 acquires the notification, and if Rm is larger than the specific threshold value Rr, the coordinate mode selection unit 326 selects the absolute coordinate mode 130 as the coordinate mode (S820).
- the coordinate mode selection unit 326 acquires the notification, and if the comparison result by the coordinate correlation specification unit 324 indicates that Rm is equal to or less than the specific threshold value Rr, the coordinate mode selection unit 326 selects the relative coordinate mode 132 as the coordinate mode. (S836).
- the current coordinate mode is the relative coordinate mode 132.
- the coordinate correlation specifying unit 324 compares Rm with a specific threshold value Rb, and notifies the coordinate mode selection unit 326 as a coordinate correlation value whether Rm is larger than the specific threshold value Rb.
- the coordinate mode selection unit 326 acquires the notification, and if Rm is larger than the specific threshold value Rb, the coordinate mode selection unit 326 selects the absolute coordinate mode 130 as the coordinate mode (S822).
- the coordinate mode selection unit 326 obtains the notification, and if Rm is equal to or less than the specific threshold value Rb, the coordinate mode selection unit 326 selects the relative coordinate mode 132 as the coordinate mode (S824).
- FIG. 7 is a flowchart showing a specific process of the coordinate determination apparatus 200 in the present embodiment.
- the initial state is the absolute coordinate mode.
- the transmission unit 220 transmits signals detected by the input detection unit 210 and the grip detection sensor 212 to the acquisition storage unit 204 as input information.
- the acquisition storage unit 204 stores the acquired input information in the detection result series storage unit 314.
- the coordinate mode selection unit 326 acquires coordinate information from the detection result series storage unit 314, and stores the input coordinates included in the coordinate information as G_it (x, y) 510 (S214).
- G_it (x, y) 510 and G_cur (x, y) 520 match.
- the coordinate conversion unit 328 converts the acquired input coordinate G_cur (x, y) 520 into a pointer coordinate position corresponding to the absolute coordinate mode 130 that is the current coordinate mode, and outputs the pointer coordinate position to the display unit 230.
- the display unit 230 updates the pointer coordinate position held for displaying the pointer 110 to the pointer coordinate position corresponding to G_cur (x, y) 520 (S216).
- the pointer 110 is displayed at the pointer coordinate position on the display unit 230 corresponding to the input coordinates on the input detection unit 210 where the user touches the finger.
- the operation recognition unit 318 attempts to acquire new input information that has not been acquired from the detection result series storage unit 314 (S218).
- the operation recognition unit 318 tries to acquire again (No in S218).
- the operation recognition unit 318 determines which event the input information can be recognized (Yes in S218, S220, S231, S232).
- the events recognized by the operation recognition unit 318 include a release event, a movement event, a selection event, a gesture event, and the like.
- the release event is an event indicating that the finger once touched by the input detection unit 210 is released from the input detection unit 210.
- the operation recognizing unit 318 can recognize the occurrence of a release event by determining whether the difference between adjacent input times is greater than a specific value or whether the distance between adjacent input coordinates is greater than a specific value.
- the movement event is an event indicating that the finger once touched on the input detection unit 210 is moved without being separated from the input detection unit 210.
- the operation recognition unit 318 obtains coordinate information including an input time within a certain time from the input time included in the immediately preceding coordinate information and including an input coordinate within a certain distance from the input coordinates included in the immediately preceding coordinate information. Thus, the occurrence of the movement event can be recognized.
- the selection event is an event that means the completion of the selection operation by the user.
- the operation recognition unit 318 recognizes the occurrence of a selection event when the selection information acquired from the detection result series storage unit 314 is selection completion information.
- the selection completion information is one of selection start time information and selection end time information. Which of the selection start time information and the selection end time information is used as the selection completion information is determined in advance depending on the coordinate determination device 200.
- the gesture event is performed by causing the user to repel the input detection unit 210 or by performing a specific input such as writing a circle or a square, thereby causing the display device 100 to perform a specific computer process associated with the specific input.
- the operation recognizing unit 318 can recognize the occurrence of a movement event, for example, when input coordinates are greatly different at adjacent times. However, the operation recognizing unit 318 can recognize the occurrence of the gesture event only after acquiring continuous input information for a certain period.
- the operation recognition unit 318 determines whether or not the input information acquired from the detection result series storage unit 314 can be recognized as a release event (S220).
- the display device 100 performs event processing according to the event type (S219). Details thereof will be described later.
- the coordinate mode selection unit 326 inputs the input time (included in the coordinate information that is the input information recognized as the release event) ( The time when the user lifts the finger from the coordinate input device 201 is recorded. Hereinafter, this time is assumed to be Tr.
- the operation recognition unit 318 determines whether or not the input information recognized as the release event can be recognized as a part of the gesture event (S231). That is, the operation recognition unit 318 determines whether or not a series of coordinate information acquired from the acquisition storage unit 204 by the operation recognition unit 318 can be recognized as a gesture event.
- the operation recognition unit 318 performs a series of inputs from the contact start point 904 (input coordinates indicated by the third coordinate information) to the movement end point 906 (input coordinates indicated by the first coordinate information). It is determined whether or not the first coordinate series as coordinates can be recognized as a gesture event (S231).
- the operation recognizing unit 318 needs to determine whether or not a series of coordinate information input by the user is a gesture event when the user releases the finger from the input detecting unit 210 (S231).
- the operation recognition unit 318 determines that the coordinate information recognized as the release event is a part of the gesture event (that is, the end of a series of coordinate information recognized as the gesture event) (Yes in S231). ), The display device 100 performs processing corresponding to the gesture event (S230).
- the display device 100 changes or controls the screen display according to the operation direction of the flick gesture (flicked in the horizontal direction or flicked in the vertical direction, etc.).
- the coordinate mode selecting unit 326 uses the time correlation specifying unit 320 to generate the release event occurrence time. It is determined whether or not the selection event has occurred within a specific grace period (hereinafter referred to as Wt) from Tr (S232).
- the time correlation identification unit 320 stores the selection event occurrence time (selection operation completion time) Ts stored in the coordinate mode selection unit 326 (FIG. 8, S412 described later), and the release event occurrence time (finger detection of input).
- the difference in Tr) is calculated, and the difference is compared with a specific delay time Wt, and the comparison result is output to the coordinate mode selection unit 326 as a time correlation value.
- the coordinate mode selection unit 326 indicates that “This release event is secondary to the occurrence of the selection event. It is determined that the event has occurred (Yes in S232), and the process proceeds to processing according to the selected event (S234).
- the coordinate mode selection unit 326 does not select the coordinate mode or change the coordinate mode.
- the display device 100 displays a display process indicating that the button has been pressed on the screen.
- substantial processing expected by the user is performed (S234).
- the coordinate mode selection unit 326 starts the coordinate mode selection process.
- Wr be the elapsed time from the occurrence of the release event (the elapsed time from the input time included in the coordinate information recognized by the operation recognition unit 318 as being a release event).
- the time correlation specifying unit 320 determines whether or not a specific time Wp has elapsed since the release event occurred, that is, whether or not Wr ⁇ Wp is satisfied, and uses the result as a time correlation value in the coordinate mode.
- the data is output to the selection unit 326 (S240).
- the coordinate mode selection unit 326 determines whether new coordinate information can be acquired from the detection result series storage unit 314 (S242).
- the coordinate mode selection unit 326 When the coordinate mode selection unit 326 cannot acquire new coordinate information (No in S242), the coordinate mode selection unit 326 displays an image such as the pointer 110 indicating that Wp has not elapsed on the display unit 230 ( S244). Thereafter, when Wr ⁇ Wp is satisfied, the coordinate mode selection unit 326 returns to the new coordinate information acquisition determination process (S242) (loop from S246 to S240).
- the process of the coordinate mode selection unit 326 is in a polling state (or a callback state), and until the coordinate information is acquired or a specific time Wp elapses after the release event occurs.
- the image display such as the pointer 110 for 100 is continued (S244).
- the coordinate mode selection unit 326 exits the loop processing from S240 to S246, and then performs the pointer 110 for the display unit 230. Is terminated (S250).
- the coordinate mode selection unit 326 determines whether or not new coordinate information can be acquired from the detection result series storage unit 314 for a certain time (S251).
- the coordinate mode selection unit 326 acquires new coordinate information from the detection result series storage unit 314 (Yes in S251), the coordinate mode selection unit 326 performs a coordinate mode selection process (S252).
- the coordinate mode selection unit 326 selects the absolute coordinate mode 130 as the coordinate mode (“absolute” in S252), the absolute coordinate mode 130 is selected as the coordinate mode (S254).
- the coordinate mode selection unit 326 determines the relative coordinate mode 132 (“relative” in S252)
- the relative coordinate mode 132 is selected as the coordinate mode (S256).
- the coordinate determining apparatus 200 restarts the process from the beginning of the loop 1 (S214).
- a pointer 110 for example, an image imitating the shape of a finger
- the coordinate determination apparatus 200 processes the input as an input in relative coordinates from the position of the pointer 110.
- an input to the input detection unit 210 after the display of the pointer 110 disappears is processed as an input in absolute coordinates.
- the display contents of the input detection unit 210 and the display unit 230 such that the input start point input after the display of the pointer 110 disappears is treated as input in absolute coordinates.
- the technology for changing the correspondence relationship with the user's operation time and the premise problem thereof are not disclosed in the prior art.
- the coordinate mode selection unit 326 sets the default value of the coordinate determination apparatus 200 as the coordinate mode. select. For example, if the relative coordinate mode 132 is a default value, the relative coordinate mode 132 is selected as the coordinate mode (S256). Thereafter, the coordinate determining apparatus 200 restarts the process from the beginning of the loop 1 (S214).
- the coordinate mode selection unit 326 selects a coordinate mode using the acquired coordinate information (S280).
- operation mode selection (S280) performed by the coordinate mode selection unit 326 may be performed as shown in FIG. 6, for example.
- the coordinate mode selection unit 326 selects the absolute coordinate mode 130 (“Absolute” in S280), the absolute coordinate mode 130 is selected as the coordinate mode (S282). If the coordinate mode selection unit 326 determines the relative coordinate mode 132 (“relative” in S280), the relative coordinate mode 132 is selected as the coordinate mode (S284).
- the coordinate determination apparatus 200 automatically recognizes a coordinate mode that is easier to use for the user while the user performs a natural operation, which cannot be realized by the following processing, and automatically switches the coordinate mode. Processing is realized.
- the coordinate determination apparatus 200 calculates, as the time correlation value, Wr, which is the difference between the occurrence time of the release event and the current time, by the time correlation identification unit 320 (S240), and the time correlation value is calculated as the coordinate mode selection unit 326. Is output.
- the coordinate correlation identification unit 324 calculates the distance Rm between G_it (x, y) 510 and G_n (x, y) 512. Then, the comparison result (magnitude relationship) between the distance and one of Rr and Rb, which are specific threshold values, is output to the coordinate mode selection unit 326 as a coordinate correlation value.
- the coordinate mode selection unit 326 automatically selects the coordinate mode as the absolute coordinate mode 130 if Rm is larger than a specific threshold value, and as the relative coordinate mode 132 if smaller than Rm (S280).
- the display device 100 restarts the process from updating the display coordinates according to the coordinate mode (S216).
- the coordinate mode selection unit 326 selects the absolute coordinate mode 130 as the coordinate mode (S282), the coordinate mode selection unit 326 sets G_cur (x, y) 520 as a new G_it (x, y) 510. Store (S286).
- FIG. 8 is a flowchart showing details of event processing other than release events in the present embodiment.
- the display device 100 performs processing according to the movement event (S404). Specifically, the display device 100 updates the value of G_cur (x, y) 520 stored in the coordinate mode selection unit 326.
- the coordinate conversion unit 328 converts G_cur (x, y) 520 to the pointer coordinate position on the corresponding display unit 230. Further, the display unit 230 draws the pointer 110 (S216).
- the operation recognition unit 318 determines that the input information is a gesture event (No in S402 and Yes in S406), the display device 100 performs processing according to the gesture event (S408).
- the gesture may be any gesture.
- the operation recognition unit 318 recognizes an operation similar to “rotation” on the input detection unit 210, the event is recognized as a “rotation gesture”. Is done. Thereafter, the display device 100 performs a process corresponding to the “rotation gesture”. Specifically, for example, scrolling the screen display can be considered.
- the coordinate mode selection unit 326 selects the selection completion time (selection start) indicated by the selection completion information. Any one of the time and the selection end time), the coordinates of G_cur (x, y) 520 at the selection completion time, the type of icon selected as necessary, and the like are recorded (S412).
- the display device 100 performs processing associated with the selection completion information (S416). Specifically, when the input detection unit 210 that is a clickable touch pad is pressed on a specific icon, the display device 100 draws a state in which the button at the position of the pointer 110 is recessed on the display unit 230. Etc.
- the coordinate input device can dynamically select an operation mode according to the user's operation, and does not require an explicit operation switching by the user, and each user can be seamlessly operated. It is possible to provide an easy-to-use operation mode for every situation.
- the coordinate mode selection unit 326 waits for input (S251) and shifts to the operation mode determination. (S252).
- the coordinate determination apparatus 200 may return the process to the beginning of the loop 1. That is, step S251, step S252, step S254, and step S256 may be omitted from the processing performed by the coordinate determination apparatus 200. In that case, the coordinate determination apparatus 200 executes step S214 after step S250.
- the coordinate mode selection unit 326 switches the content of the selection process (S252) according to the time from the occurrence of the release event (S220) to the next contact (input) (S242, S251). , S280).
- the coordinate mode selection unit 326 may perform the selection process without switching the operation mode according to time (S252). Specifically, step S240, step S242, step S244, step S246, step S280, step S282, step S284, and step S286 may be omitted from the processing performed by the coordinate determination apparatus 200. In this case, if the predetermined condition is not satisfied in step S232 (No in S2332), the coordinate determining apparatus 200 performs the process of S250 (the loop process from S240 to S246 is not necessary).
- step S240 when it is desired to switch the coordinate mode selection processing performed by the coordinate mode selection unit 326 according to the time until the next contact (input) when an arbitrary event occurs, the processing before step S240 is omitted, and step S240 and subsequent steps are omitted. It can be applied to any input system.
- the coordinate mode selection process (S252 and S280) by the coordinate mode selection unit 326 is not limited to the above.
- the coordinate mode selection unit 326 may select the coordinate mode according to arbitrary data that can be known by the coordinate determination apparatus 200 such as a device or a device.
- the coordinate determination device 200 can arbitrarily determine the specified value of the coordinate mode. For example, even if the coordinate determination device 200 dynamically determines the coordinate mode based on an artificial or past use history or history, or the coordinate determination device 200 acquires the coordinate mode from a ROM or the like as an initial setting value. Good.
- “move” is taken up as a pointing function
- “gesture” is taken as a general motion recognition function
- selection is taken up as a function to convey an intention such as a decision.
- the input information to be processed by the coordinate determination device 200 is not limited to these three events, but “information that can be acquired from the generally available input detection unit 210 and the recognition that is generally available.
- the event may include “recognition result information obtained by combining technologies”.
- the movement event process (S404) performed by the display device 100 includes not only the update of the internal pointer coordinates but also an arbitrary process generally performed when the pointer is moved in a personal computer or an embedded device.
- the gesture event processing (S408) performed by the display device 100 includes not only rotation but also arbitrary processing based on a general gesture recognition technique. For example, there are screen scroll processing by a slide operation and screen enlargement processing by a pinch-out operation, but the present invention is not limited to this.
- the selection event processing (S412) performed by the display device 100 may be triggered by an arbitrary selection event generated by a selection operation method by the general input detection unit 210.
- the input detection unit 210 is a touch pad
- an input switch mechanism is provided below the touch pad so that the entire touch pad functions as a clickable touch pad, and this touch pad is used as a physical switch (button). It is good also as a trigger of selection operation.
- the selection operation is not limited to these.
- gesture event process (S408) performed by the display device 100 is not limited to the flick operation described above, and may be a gesture operation with any release.
- the operation according to the selection event (S234) performed by the display device 100 is not limited to the button display example, and is an arbitrary process or display during a general selection event in a personal computer or a general device. Also good.
- loop 2 (S240, S242, S244, S246) waiting for input by the coordinate mode selection unit 326 is not necessarily loop control, and is equivalent to any programming means or system configuration means such as system interrupt processing. The function may be realized. The same applies to loop 1 and input waiting (S218, S251).
- the pointer display end processing (S250) after loop control waiting for input performed by the coordinate mode selection unit 326 not only stops (erase) the immediate display after the end of the loop from S246 to S246, but also gradually reduces the display. Or blink, or display animation effects, tooltips, comments, etc. to indicate that it will soon expire.
- the coordinate mode selection unit 326 may display a comment or the like on the display unit 230. Furthermore, the user's attention and understanding may be promoted by changing these expressions over time.
- the selection (S232) regarding whether or not the selection event has occurred within the specific grace time Wt performed by the coordinate mode selection unit 326 is not limited to the selection event, and may be determined for any of the above-described events. .
- the determination criterion “within a specific grace period Wt” is set based on the occurrence of the corresponding past event. Timing. Specifically, when the coordinate mode selection unit 326 obtains the nth event occurrence, the coordinate mode selection unit 326 determines the elapsed time from the occurrence of the n ⁇ 1th same type of event and the specific grace time Wt. Are compared (S232).
- the coordinate mode selection unit 326 includes a comparison result by the coordinate correlation specification unit 324, a comparison result by the time correlation specification unit 320, a grip state estimation result by the grip estimation unit 316, and a user operation recognition result by the operation recognition unit 318. , At least one result may be obtained, and the coordinate mode may be determined based on the result.
- Wt, Wp, Wd, Rr, and Rb are set to predetermined values.
- Wt, Wp, Wd, Rr, and Rb are moved according to the “operation degree” of the user. Change.
- degree of operation examples include the number of operations per unit time, the operation period of the coordinate determination device 200, the usage history for each coordinate mode, and the like, but are not limited thereto.
- FIG. 9 is a flowchart illustrating a process for changing a threshold value or the like depending on the degree of operation in the first modification example of the present embodiment.
- the coordinate mode selection unit 326 acquires the use history of each coordinate mode determined in the past by the coordinate mode selection unit 326 (S289). That is, the usage history of the absolute coordinate mode 130 and the relative coordinate mode 132 is acquired.
- a past history mode notified by the coordinate mode selection unit 326 to the coordinate conversion unit 328 is recorded in a RAM, a hard disk, a flash memory, or the like, so that a history can be acquired later.
- the coordinate mode selection unit 326 also acquires a history of event processing data (S289). For example, the type, occurrence time, end time, and the like of each of the generated movement event, gesture event, and selection event can be considered.
- the coordinate mode selection unit 326 calculates the degree of operation based on the acquired coordinate mode and event processing data (S290).
- the operation recognition unit 318 selects the number of times per unit time recognized as the movement event, the gesture event, and the selection event, the average value of the input time for each movement event, the gesture event, and the selection event, etc.
- the unit 326 calculates.
- the coordinate mode selection unit 326 may calculate a plurality of differences from the user selection completion time to the release event occurrence time, and calculate the reciprocal of the average value of the calculated differences as the operation degree. .
- the degree of operation is not limited to these, and may be another feature amount that characterizes the input operation to the input detection unit 210 by the user.
- the coordinate mode selection unit 326 corrects the value of the specific grace period Wt based on the calculated degree of operation (S292).
- Wt is made smaller.
- the coordinate mode selection unit 326 determines that the user is used to the current coordinate mode when the degree of operation, which is the operation period of the coordinate determination device 200, is large, and sets a specific grace period Wt short. Good (S232).
- the coordinate mode selection unit 326 corrects the values of specific times Wp and Wd based on the calculated operation degree (S294 and S297).
- the coordinate mode selection unit 326 determines that the operation degree as the operation frequency is larger than before because the number of movement events generated per unit time is large, the values of Wp and Wd are made smaller. Thereby, even if it is a case where it is desired for the user to recognize the absolute coordinate mode 130, it is possible to improve the tendency when the relative coordinate mode 132 is recognized.
- the coordinate mode selection unit 326 corrects the specific threshold values Rr and Rb based on the calculated degree of operation (S299).
- the coordinate mode selection unit 326 determines that the input degree corresponding to the start end of the movement event is always within a certain range, and the degree of operation that is the reciprocal of the variation in the contact start point is greater than before, Rr And the value of Rb is made smaller. Therefore, even if it is a case where it is desired for the user to recognize the absolute coordinate mode 130, it is possible to improve the tendency when the relative coordinate mode 132 is recognized.
- the method of correcting parameters such as Wt, Wp, Wd, Rr, and Rb based on the degree of operation may be changed according to the purpose of the coordinate input system 90.
- the coordinate mode selection unit 326 may determine the degree of operation based on the history of the coordinate mode, the duration of the current determination mode, and the length of the usage time, and may change parameters such as Wd, Rr, and Rb. Specifically, the coordinate mode selection unit 326 sets parameters such as Wd, Rr, and Rb so that the change of the coordinate mode is less likely to occur as the current coordinate mode continues for a long time. It is possible to correct it.
- the coordinate mode selection unit 326 increases the parameter as the degree of operation is higher, that is, as the user is used to input using the coordinate determination apparatus 200.
- the second or third photo can be selected by moving the finger touching the input detection unit 210.
- the coordinate determining apparatus 200 provides an application-specific operation method.
- the user who is used to the coordinate determination apparatus 200 determines that he / she wants to perform a specific operation, and by setting Wt longer, the user can easily select many photographs at the same time. Leads to.
- the coordinate mode selection unit 326 may calculate a plurality of operation degrees.
- the operation degree information may be held as a time series, or the operation degree may be calculated for each event.
- the coordinate mode selection unit 326 is individually or combined with respect to a plurality of components / information that the system can know, such as by user, by time zone used, by day of the week, by application, etc.
- the degree of operation may be set to.
- “composite” means a combination of the above-described constituent elements such as “operation degree in application A on Tuesday” and “frequency of occurrence of selection event in application B of user A”.
- a specific time, a specific threshold value, or the like may be corrected depending on the coordinate mode regardless of the degree of operation. Specifically, when operating in the relative coordinate mode 132, a specific time, a specific threshold, or the like is set long (or short), and in the absolute coordinate mode 130, a specific time or a specific threshold is set. You may set a threshold value etc. short (or long).
- Wt, Wp, Wd, and the like which are predetermined times, may be set according to the generation time distribution of events such as selection, the average generation time, the longest time, and the shortest time under each coordinate mode.
- the coordinate mode selection unit 326 may correct the value of each parameter based on an operation degree and a correction method that are different for each parameter such as Wt, Wp, Wd, Rr, and Rb.
- the operation degree calculated by the coordinate mode selection unit 326 is not only “the number of operations within a unit time”, but also “the number and frequency of occurrence of each event”, “the coordinate mode when the event occurs, and the pointer mode. It may be determined according to “position, position distribution”, and the like.
- the coordinate mode selection unit 326 calculates the degree of operation based on, for example, “an arbitrary combination of internal data that the system can know and information that the user can input in advance or dynamically”. May be.
- the detection result series storage unit 314 may hold the generation time, the coordinate mode selection result by the coordinate mode selection unit 326 and the time, the ID for identifying the operating user, the operation date and time, and the like.
- the coordinate mode selection unit 326 may change the erased representation of the pointer.
- the coordinate mode selection unit 326 may display the increase / decrease state of Wp itself with an animation effect, a tool tip, a comment, or the like.
- the coordinate mode selection unit 326 may correct parameters such as Wt, Wp, Wd, Rr, and Rb according to the operation proficiency level in each coordinate mode.
- the operation proficiency level is a kind of operation level, and is determined according to the frequency of event occurrence (number of selected events, etc.), the number of errors, usage time, etc. in each coordinate mode.
- the coordinate mode may be selected according to the level of operation proficiency.
- the “operation degree” corresponding to the familiarity, habit, usage, etc. of the operation is calculated for each user, and using this, the time used by the coordinate mode selection unit 326 and A distance threshold or the like (Wt, Wp, Wd, Rr, and Rb) can be changed.
- the coordinate determination apparatus 200 can more precisely cope with a natural input operation for each user, and can automatically recognize and switch to a coordinate mode that is easier for the user to use.
- the display time of the pointer 110 (Wp described above) is shortened and the relative coordinate mode is entered. It is also possible to perform processing such as making it difficult to determine or determining the absolute coordinate mode even if the finger is left. Further, as in the case of typing in general keyboard input, for example, even if the distance between each input point is shorter than a threshold value, the absolute coordinate mode may be determined. In addition, such a process and the subject used as the premise are not shown by the prior art.
- G_it (x, y) 510 that is the input coordinate indicated by the third coordinate information as the distance used by the coordinate mode selection unit 326 to select the coordinate mode and the input indicated by the second coordinate information.
- the coordinate mode selection unit 326 uses a different distance in this modification.
- FIG. 10 is a schematic diagram illustrating an example of a distance used by the coordinate mode selection unit 326 to determine the coordinate mode in the second modification example of the present embodiment.
- G_it (x, y) 510 corresponds to the contact start point 904.
- G_n (x, y) 512 corresponds to the contact start point 910.
- a distance between the contact start point 904 and the contact start point 910 is a distance Dgg740.
- the display unit 230 displays the corresponding pointer 110 from the display contact start point 944 to the display movement end point 946.
- a display position on the display unit 230 when the contact start point 910 is interpreted as the absolute coordinate mode 130 is set as a contact start assumed point 950.
- the display position on the display unit 230 when the movement point 912 is interpreted as the absolute coordinate mode 130 is set as the assumed pointer position 952.
- an icon 924 as an icon being displayed on the display unit 230, and the center coordinate of the icon 924 is an icon center 928.
- the distance between the display movement end point 946 and the assumed contact start point 950 is defined as a distance Dgp720.
- the distance between the icon center 928 and the assumed contact start point 950 is a distance Dgd710.
- the distance between the assumed pointer position 952 and the icon center 928 is a distance Dpd730.
- the coordinate mode selection unit 326 can select a coordinate mode using a conditional expression that arbitrarily combines these distances.
- the coordinate mode selection unit 326 may select the absolute coordinate mode 130 as the coordinate mode. Conceivable.
- the coordinate mode selection unit 326 It is conceivable to select the relative coordinate mode 132 as the coordinate mode.
- the screen display part with which the coordinate mode selection unit 326 compares the distance with the pointer 110 is not limited to the closest screen display part.
- the coordinate mode selection unit 326 can use the distance between the pointer 110 and an arbitrary part and the multiple distances between the pointer 110 and a plurality of parts in combination.
- the coordinate mode selection unit 326 may select the coordinate mode according to information that the pointer 110 is on the screen display part (does not exist).
- the assumed pointer position 952 is defined as the display position of the pointer 110 on the display unit 230 in the absolute coordinate mode 130, but the actual display position of the pointer 110 in the absolute coordinate mode 130 or the relative coordinate mode 132. You may think.
- the display device 100 may display an image having an arbitrary shape such as a finger shape in addition to a general cursor shape as the shape of the pointer 110 displayed on the display unit 230.
- the distance Dgd 710 is defined as the distance between the icon center 928 and the assumed contact start point 950.
- the distance information to any place of the screen display part is not limited to the icon center 928. You may obtain
- the number of screen display parts for calculating the distance information used by the coordinate mode selection unit 326 is not limited to one.
- the coordinate mode selection unit 326 may calculate and use a distance from a plurality of screen display parts, and may further use an average value thereof.
- the method used in the coordinate mode determination by the coordinate mode selection unit 326 before the standby pointer display ends is partially used. Or may be used in combination or in part.
- the coordinate mode selection unit 326 may select the coordinate mode according to the distance between G_n (x, y) 512 and the screen display (GUI).
- the coordinate mode selection unit 326 may select the coordinate mode according to the position (or history) of the current (or past) pointer on the screen.
- the coordinate mode selection unit 326 selects a coordinate mode according to the mutual distance between the three points G_n (x, y) 512, the position of the screen display (GUI), and G_cur (x, y) 520. Also good.
- G_it (x, y) 510 may be added to the previous three, and the coordinate mode selection unit 326 may select a coordinate mode according to the mutual distance of the four points.
- the coordinate mode selection unit 326 is not limited to G_it (x, y) 510, but arbitrary coordinate information included in the first coordinate series (for example, at the end of the first coordinate series or in the middle of the start and end points).
- the coordinate mode may be selected using coordinate information corresponding to a certain point).
- the coordinate mode selection unit 326 can display the pointer on the GUI part by selecting the absolute coordinate mode 130 as the coordinate mode (S280).
- the coordinate mode selection unit 326 may select the absolute coordinate mode 130 as the coordinate mode in preference to other conditions. Good.
- G_it (x, y) 510, G_n (x, y) 512, the screen display (GUI) position, and the current position of the pointer 110 on the display unit 230 depend on the mutual distance.
- the coordinate mode selection unit 326 may select the coordinate mode.
- the coordinate mode selection unit 326 may select the coordinate mode by combining a plurality of mutual distances or weighting.
- the coordinate input device 201 may have a plurality of input detection units 210 (for example, touch pads) like the remote-control type coordinate input device 201 with a touch pad shown in FIG.
- the acquisition storage unit 204 stores all inputs from the plurality of input detection units 210 in the detection result series storage unit 314.
- the coordinate mode selection unit 326 may return the coordinate mode to the default value when the input detection unit 210 (touch pad) used for input is changed.
- the coordinate mode selection unit 326 may select the coordinate mode according to the estimation result of the grip estimation unit 316 (for example, how to hold the user such as the right hand, the left hand, and both hands).
- the coordinate mode selection unit 326 determines at least one of a specific grace time Wt, specific times Wp and Wd, and specific thresholds Rr and Rb according to the gripping state estimated by the grip estimation unit 316. It may be increased or decreased.
- the left side and the right side of the input detection unit 210 are easier operations for the user as the positions where the thumb is brought into contact. This is because touching the right side of the input detection unit 210 with the thumb while gripping with the right hand becomes unstable as the gripping state.
- the coordinate mode selection unit 326 may return the coordinate mode to a specified value (either one of the absolute coordinate mode 130 and the relative coordinate mode 132) when the user's way of holding changes. Also, the coordinate mode selection unit 326 may return the coordinate mode to the specified value once the user's hand has left the coordinate input device 201.
- the coordinate mode selection unit 326 may change the specified value of the coordinate mode when the user holds the coordinate input device 201 with both hands and when the user holds it with one hand.
- the coordinate mode selection unit 326 may change the specified value of the coordinate mode depending on whether the user holds the coordinate input device 201 with the right hand or the left hand.
- the coordinate mode selection unit 326 may change the specified value of the coordinate mode according to the user's dominant hand.
- the coordinate mode selection unit 326 may select the coordinate mode by other methods without being limited to these.
- the specified value of the coordinate mode could not be changed.
- the specified value of the coordinate mode can be changed according to, for example, the user's preference and purpose of use.
- FIG. 11 is a flowchart showing the processing of the coordinate determining apparatus 200 in the third modification example of the present embodiment.
- the coordinate determining apparatus 200 first sets the default value of the coordinate mode to one of the absolute coordinate mode 130 and the relative coordinate mode 132 (S310).
- the coordinate mode selection unit 326 displays a GUI screen on the display unit 230 and acquires and sets the result of selection by the user using the coordinate input device 201, or the coordinate mode selection unit A method in which 326 determines from the past operation history is conceivable.
- the coordinate mode selection process (S350) performed by the coordinate mode selection unit 326 is not limited to the above.
- the input time of the contact start point 910 is within a certain time from the input time of the movement end point 906, and the distance between the input coordinates of the contact start point 910 and the input coordinates of the contact start point 904 is within a certain distance.
- the coordinate mode selection unit 326 may select the relative coordinate mode 132 as the coordinate mode, and may select the absolute coordinate mode 130 in other cases.
- the coordinate mode selection unit 326 may select the coordinate mode using the determination process (S232, S252, S280) used in the other modified example described above.
- coordinate determination device 200 described in the present embodiment can be realized by a computer.
- coordinate determination apparatus 200 includes a computer 34, a keyboard 36 and mouse 38 for giving instructions to computer 34, a display 32 for presenting information such as calculation results of computer 34, a computer 34 includes a CD-ROM (Compact Disc-Read Only Memory) device 40 for reading the program executed in 34 and a communication modem (not shown).
- CD-ROM Compact Disc-Read Only Memory
- a program for realizing each process performed by the coordinate determination device 200 is stored in a CD-ROM 42 which is a computer-readable medium and is read by the CD-ROM device 40. Alternatively, it is read by a communication modem through the computer network 26.
- FIG. 13 is a block diagram showing a hardware configuration of a computer system that implements the coordinate determination apparatus 200.
- the computer 34 includes a CPU 44, a ROM 46, a RAM 48, a hard disk 50, a communication modem 52, and a bus 54.
- the CPU 44 executes the program read via the CD-ROM device 40 or the communication modem 52.
- the ROM 46 stores programs and data necessary for the operation of the computer 34.
- the RAM 48 stores data such as parameters at the time of program execution.
- the hard disk 50 stores programs and data.
- the communication modem 52 communicates with other computers via the computer network 26.
- the bus 54 connects the CPU 44, the ROM 46, the RAM 48, the hard disk 50, the communication modem 52, the display 32, the keyboard 36, the mouse 38, and the CD-ROM device 40 to each other.
- the system LSI is a super multifunctional LSI manufactured by integrating a plurality of components on one chip, and specifically, a computer system including a microprocessor, a ROM, a RAM, and the like. .
- a computer program is stored in the RAM.
- the system LSI achieves its functions by the microprocessor operating according to the computer program.
- a part or all of the constituent elements constituting each of the above devices may be constituted by an IC card or a single module that can be attached to and detached from each device.
- the IC card or module is a computer system that includes a microprocessor, ROM, RAM, and the like.
- the IC card or the module may include the super multifunctional LSI described above.
- the IC card or the module achieves its function by the microprocessor operating according to the computer program. This IC card or this module may have tamper resistance.
- the present invention may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the computer program.
- the present invention relates to a computer-readable recording medium such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray Disc). (Registered trademark)), a memory card such as a USB memory or an SD card, or a semiconductor memory. Further, the digital signal may be recorded on these recording media.
- a computer-readable recording medium such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray Disc). (Registered trademark)
- a memory card such as a USB memory or an SD card
- the digital signal may be recorded on these recording media.
- the computer program or the digital signal may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.
- the present invention may also be a computer system including a microprocessor and a memory, in which the memory stores the computer program, and the microprocessor operates according to the computer program.
- program or the digital signal is recorded on the recording medium and transferred, or the program or the digital signal is transferred via the network or the like and executed by another independent computer system. You may do that.
- the present invention can be applied to a coordinate determination device or the like, and in particular to a coordinate determination device or the like that determines the coordinate position of a pointer on a display unit of a display device based on input information acquired from the coordinate input device.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Position Input By Displaying (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
表示装置の表示部(ディスプレイ)と座標入力装置の入力検知部(タッチパネル)が重畳した構成で、直接、表示部上の操作対象物(アイコン等)を触ることでポインティングを行うもの。任天堂社のNintendoDS(登録商標)や、アップル社のiPhone(登録商標)など。
表示装置の表示部と座標入力装置の入力検知部が物理的に重なり合っておらず、座標入力が表示装置に対して間接的になるもの。デスクトップパソコンのマウス、ラップトップパソコンのタッチパッドなど。ラップトップパソコン全体を1つの機器とみなした場合、タッチパッドと表示装置は一体型になっているとも言えるが、本願では、表示部と入力検知部が前例(1)のように物理的に重なり合っていないことから、分離型として扱う。
タッチパッドやタブレットなどの入力検知部上の座標が、表示部上の座標に1対1で対応付けられているもの。手書き文字入力機器用のタッチパッド、描画ツール用の外付け入力タブレットなど。ワコム社(Wacom社)のペン入力型タブレットやタッチタブレット(液晶との一体型ではないもの)など。入力検知部に触れた(入力し始めた)座標点がそのまま表示部上の対応する座標位置に変換されるため、手書き漢字の入力やイラスト描画等に用いられる。
タッチパネルを用いた一部のゲームアプリケーションで、ゲーム中の主人公を操作するために、表示装置の表示部兼座標入力装置の入力検知部の一部を用いる例などがある。このケースに相当するコントロールデバイス・方法は、上記(1)~(3)に比べ一般的ではない。
・ユーザにESCキーを押下させるなど、アプリケーションに依存させて絶対座標モードと相対座標モードを切り替えるもの(例えば、特許文献1参照)、
・入力検知部210上に備えた物理的な切換えボタン(スイッチ)をユーザに操作させるもの(例えば、特許文献2参照)、
・表示部230上でのポインタ位置によって切り替えるもの(例えば、特許文献1及び特許文献3参照)、
・絶対座標モードの入力と相対座標モードの入力にそれぞれ別々の入力検知部210を仮定しているもの(例えば、特許文献4参照)、
・タッチパッド上での指の接触面積の大きさによって座標モード切替を行うもの(例えば、非特許文献1参照)、
・タッチパッドへ入力する時間間隔によって座標モード切替を行うもの(例えば、特許文献5参照)などが提案されている。
以下、本実施の形態による変形例について説明する。
以下、本実施の形態による第2の変形例について説明する。
34 コンピュータ
36 キーボード
38 マウス
40 CD-ROM装置
44 CPU
46 ROM
48 RAM
50 ハードディスク
52 通信モデム
54 バス
90 座標入力システム
100 表示装置
110 ポインタ
130 絶対座標モード
132 相対座標モード
200 座標決定装置
201 座標入力装置
204 取得格納部
210 入力検知部
212 把持検出センサ
220 送信部
230 表示部
314 検知結果系列記憶部
316 把持推定部
318 操作認識部
320 時間相関特定部
324 座標相関特定部
326 座標モード選択部
328 座標変換部
510 G_it(x,y)
512 G_n(x,y)
520 G_cur(x,y)
610 軌跡A
620 軌跡表示A
630 軌跡B
632 軌跡C
634 軌跡D
640 軌跡表示B
642 軌跡表示C
644 軌跡表示D
710 距離Dgd
720 距離Dgp
730 距離Dpd
740 距離Dgg
800 現在の指の位置
810 第1入力状態
812 第2入力状態
814 第3入力状態
816 第4入力状態
904、910 接触開始点
906 移動終了点
912 移動点
924 アイコン
928 アイコン中心
944 表示接触開始点
946 表示移動終了点
950 接触開始想定点
952 想定ポインタ位置
Claims (19)
- 入力座標に応じた出力座標を決定する座標決定装置であって、
前記入力座標及び入力時刻を示す座標情報を順次取得して記録媒体に格納する取得格納部と、
前記記録媒体に格納されている複数の座標情報のうち第1及び第2の座標情報によって示される入力時刻間の相関を時刻相関値として特定する時間相関特定部と、
前記複数の座標情報のうち第2及び第3の座標情報によって示される入力座標間の相関を座標相関値として特定する座標相関特定部と、
前記座標相関値及び前記時刻相関値に基づいて、第一の座標モード及び第二の座標モードのうちの何れか一方を座標モードとして選択する座標モード選択部と、
前記第2の座標情報によって示される入力座標を、前記座標モード選択部によって選択された座標モードにしたがった座標に変換することによって、当該座標を前記出力座標として決定する座標変換部とを備え、
前記座標変換部は、
前記第一の座標モードが選択されたときには、前記第2の座標情報によって示される入力座標を、当該入力座標に予め定め対応付けられた座標に変換し、
前記第二の座標モードが選択されたときには、前記第2の座標情報によって示される入力座標を、他の入力座標に対して既に決定された出力座標に応じた座標に変換する
座標決定装置。 - 前記座標相関特定部は前記入力座標間の距離を前記座標相関値として特定し、特定した前記座標相関値が事前に定められた特定の閾値未満であるか否かを判定し、
前記時間相関特定部は前記入力時刻間の差を前記時刻相関値として特定し、特定した前記時刻相関値が事前に定められた特定の時間未満であるか否かを判定し、
前記座標モード選択部は前記座標相関特定部及び前記時間相関特定部によって、前記座標相関値が特定の閾値未満であり、かつ前記時刻相関値が特定の時間未満であると判断された場合、前記第二の座標モードを座標モードとして選択する
請求項1に記載の座標決定装置。 - 前記取得格納部は、入力検知部を有する座標入力装置から座標情報を取得しており、
前記第1の座標情報は、ユーザが前記入力検知部から指を離した時刻に対応する座標情報であり、
前記第2の座標情報は、前記第1の座標情報が検知された後、ユーザが前記入力検知部に再度指を接触させた時刻に対応する座標情報である
請求項2に記載の座標決定装置。 - 前記第3の座標情報は、ユーザが前記入力検知部に指を接触させた時刻に対応する座標情報であり、
前記第1の座標情報は、前記第3の座標情報が検知された後、ユーザが前記入力検知部から指を離した時刻に対応する座標情報であり、
前記第2の座標情報は、前記第1の座標情報が検知された後、ユーザが前記入力検知部に再度指を接触させた時刻に対応する座標情報である
請求項2に記載の座標決定装置。 - 前記取得格納部は、さらに、ユーザによる選択操作の完了時刻である選択完了時刻を含む選択完了情報を取得して前記記録媒体に格納し、
前記座標モード選択部は前記記録媒体から前記選択完了情報を取得し、前記第1の座標情報によって示される入力時刻が、前記選択完了時刻から事前に定められた特定の猶予時間内であれば、座標モードの選択を行わない若しくは座標モードの変更を行わない、
請求項4に記載の座標決定装置。 - 前記座標決定装置は、さらに、前記座標入力装置を備えており、
前記座標入力装置は、ユーザによる座標入力操作及び選択操作を受け付けて、前記座標情報及び前記選択完了情報を前記取得格納部に送信し、
前記入力検知部は、機械的な押し下げ動作を検知可能なクリッカブルタッチパッドであり、
ユーザが前記クリッカブルタッチパッドを押し下げた指を離したことに起因して前記クリッカブルタッチパッドの戻し動作が完了する時刻を前記選択完了時刻として特定する
請求項5に記載の座標決定装置。 - 前記座標モード選択部は、
前記第2の座標情報より前に前記取得格納部が取得した複数の座標情報及び選択情報のうち少なくとも1以上に基づいて、ユーザの操作特性を示す操作度合いを算出し、前記算出された操作度合いに基づいて、前記特定の時間、前記特定の閾値及び前記特定の猶予時間のうち少なくとも1つを変更する
請求項5に記載の座標決定装置。 - 前記座標モード選択部は、前記複数の座標情報に含まれる一の座標情報ごとに、当該座標情報によって示される入力時刻と、他の座標情報によって示される入力時刻との差を算出し、算出された前記複数の差の平均値の逆数を前記操作度合いとして算出する
請求項7に記載の座標決定装置。 - 前記座標モード選択部は、前記算出された操作度合いが大きいほど、前記特定の時間、前記特定の閾値及び前記特定の猶予時間のうち少なくとも1つを小さくなるように変更する
請求項7又は8に記載の座標決定装置。 - 前記第3の座標情報で示される位置が、前記第1の座標情報で示される位置と一致する
請求項1又は2に記載の座標決定装置。 - 前記第1の座標情報は、ユーザが前記入力検知部に一旦指を触れてから離すまでに入力された一連の座標情報である第1の座標系列に含まれ、
前記第2の座標情報は、前記第1の座標系列の入力後、ユーザが再度前記入力検知部に指を触れてから指を離すまでに入力された一連の座標情報である第2の座標系列に含まれ、
前記第3の座標情報は、前記第1の座標系列に含まれる
請求項1又は2に記載の座標決定装置。 - 前記座標モード選択部は、前記第1の座標情報によって示される入力時刻から事前に定められた特定の時間が経過しておらず、かつ、前記取得格納部が前記第2の座標情報を取得していない場合には、表示部へ特定の画像を表示させる
請求項2に記載の座標決定装置。 - 前記座標決定装置は、さらに、
前記第1の座標系列が、表示装置に対して特定の計算機処理の開始を指令するジェスチャであるか否かを判定する操作認識部を有し、
前記操作認識部が前記第1の座標系列をジェスチャと判定した場合、前記座標モード選択部は、前記座標モードの選択を行わない
請求項11に記載の座標決定装置。 - 前記取得格納部は、さらに、ユーザが前記座標入力装置のどの位置を把持しているのかを示す把持情報を取得して前記記録媒体に格納し、
前記座標決定装置は、さらに、前記記録媒体から取得した前記把持情報を用いてユーザによる前記座標入力装置の把持状況を推定する把持推定部を有し、
前記座標モード選択部は、前記把持推定部が推定した把持状況に応じて、前記特定の閾値、前記特定の時間及び前記特定の猶予時間のうち少なくとも1つの値を変更する
請求項3に記載の座標決定装置。 - 前記座標入力装置は、前記入力検知部を少なくとも2以上備え、
前記取得格納部は、前記複数の入力検知部のうち第1の入力検知部から取得する座標情報及び第2の入力検知部から取得する座標情報を前記記録媒体に格納する
請求項1又は2に記載の座標決定装置。 - 入力座標に応じた出力座標を決定する座標決定方法であって、
前記入力座標及び入力時刻を示す座標情報を順次取得して記録媒体に格納する取得格納ステップと、
前記記録媒体に格納されている複数の座標情報のうち第1及び第2の座標情報によって示される入力時刻間の相関を時刻相関値として特定する時刻相関特定ステップと、
前記複数の座標情報のうち第2及び第3の座標情報によって示される入力座標間の相関を座標相関値として特定する座標相関特定ステップと、
前記座標相関値及び前記時刻相関値に基づいて、第一の座標モード及び第二の座標モードのうちの何れか一方を座標モードとして選択する座標モード選択ステップと、
前記第2の座標情報によって示される入力座標を、前記座標モード選択部によって選択された座標モードにしたがった座標に変換することによって、当該座標を前記出力座標として決定する座標変換ステップとを含み、
前記座標変換ステップは、
前記第一の座標モードが選択されたときには、前記第2の座標情報によって示される入力座標を、当該入力座標に予め定め対応付けられた座標に変換し、
前記第二の座標モードが選択されたときには、前記第2の座標情報によって示される入力座標を、他の入力座標に対して既に決定された出力座標に応じた座標に変換する
座標決定方法。 - 請求項16に記載の座標決定方法をコンピュータで実行させる
プログラム。 - 請求項17に記載のプログラムを記録した
コンピュータ読み取り可能な記録媒体。 - 入力座標に応じた出力座標を決定する集積回路であって、
前記入力座標及び入力時刻を示す座標情報を順次取得して記録媒体に格納する取得格納部と、
前記記録媒体に格納されている複数の座標情報のうち第1及び第2の座標情報によって示される入力時刻間の相関を時刻相関値として特定する時間相関特定部と、
前記複数の座標情報のうち第2及び第3の座標情報によって示される入力座標間の相関を座標相関値として特定する座標相関特定部と、
前記座標相関値及び前記時刻相関値に基づいて、第一の座標モード及び第二の座標モードのうちの何れか一方を座標モードとして選択する座標モード選択部と、
前記第2の座標情報によって示される入力座標を、前記座標モード選択部によって選択された座標モードにしたがった座標に変換することによって、当該座標を前記出力座標として決定する座標変換部とを備え、
前記座標変換部は、
前記第一の座標モードが選択されたときには、前記第2の座標情報によって示される入力座標を、当該入力座標に予め定め対応付けられた座標に変換し、
前記第二の座標モードが選択されたときには、前記第2の座標情報によって示される入力座標を、他の入力座標に対して既に決定された出力座標に応じた座標に変換する
集積回路。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/383,966 US9041649B2 (en) | 2010-05-18 | 2011-05-17 | Coordinate determination apparatus, coordinate determination method, and coordinate determination program |
CN201180002866.0A CN102473060B (zh) | 2010-05-18 | 2011-05-17 | 坐标决定装置、坐标决定方法 |
JP2011536220A JP5762967B2 (ja) | 2010-05-18 | 2011-05-17 | 座標決定装置、座標決定方法及び座標決定プログラム |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2010114752 | 2010-05-18 | ||
JP2010-114752 | 2010-05-18 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011145330A1 true WO2011145330A1 (ja) | 2011-11-24 |
Family
ID=44991448
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2011/002729 WO2011145330A1 (ja) | 2010-05-18 | 2011-05-17 | 座標決定装置、座標決定方法及び座標決定プログラム |
Country Status (4)
Country | Link |
---|---|
US (1) | US9041649B2 (ja) |
JP (1) | JP5762967B2 (ja) |
CN (1) | CN102473060B (ja) |
WO (1) | WO2011145330A1 (ja) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2013088559A1 (ja) * | 2011-12-15 | 2013-06-20 | トヨタ自動車株式会社 | 操作装置 |
JP2015041189A (ja) * | 2013-08-21 | 2015-03-02 | ソニー株式会社 | 表示制御装置、表示制御方法およびプログラム |
JP2015115038A (ja) * | 2013-12-16 | 2015-06-22 | セイコーエプソン株式会社 | 情報処理装置および情報処理装置の制御方法 |
JP5739015B2 (ja) * | 2013-03-28 | 2015-06-24 | 株式会社東芝 | 通信装置、通信方法及びプログラム |
CN103376926B (zh) * | 2012-04-16 | 2016-07-06 | 汉王科技股份有限公司 | 可切换坐标的绘画板及其坐标切换方法 |
JP2016130975A (ja) * | 2015-01-14 | 2016-07-21 | シャープ株式会社 | 情報処理装置およびプログラム |
US10031610B2 (en) | 2013-02-19 | 2018-07-24 | Toyota Jidosha Kabushiki Kaisha | Operation device for vehicle |
JPWO2020217336A1 (ja) * | 2019-04-24 | 2020-10-29 | ||
CN115185411A (zh) * | 2022-07-08 | 2022-10-14 | 北京字跳网络技术有限公司 | 光标移动方法、装置和电子设备 |
Families Citing this family (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9746995B2 (en) * | 2011-07-14 | 2017-08-29 | Microsoft Technology Licensing, Llc | Launcher for context based menus |
US9491146B2 (en) | 2011-09-07 | 2016-11-08 | Elwha Llc | Computational systems and methods for encrypting data for anonymous storage |
JP6091829B2 (ja) * | 2011-09-28 | 2017-03-08 | 京セラ株式会社 | 装置、方法、及びプログラム |
US9924907B2 (en) * | 2011-09-30 | 2018-03-27 | Google Technology Holdings LLC | Method and system for identifying location of a touched body part |
US20130127738A1 (en) * | 2011-11-23 | 2013-05-23 | Microsoft Corporation | Dynamic scaling of touch sensor |
US20130176218A1 (en) * | 2012-01-10 | 2013-07-11 | Cywee Group Limited | Pointing Device, Operating Method Thereof and Relative Multimedia Interactive System |
KR20140010780A (ko) * | 2012-07-17 | 2014-01-27 | 최경순 | 터치 유저 인터페이스, 영상 장치, 시스템 및 원격 터치 입력 장치 |
US9348512B2 (en) * | 2012-08-08 | 2016-05-24 | Nuance Communications, Inc. | Methods for facilitating text entry |
CN103019375B (zh) * | 2012-11-28 | 2016-02-10 | 广东远峰电子科技有限公司 | 一种基于图像识别的光标控制方法及其系统 |
KR20140089660A (ko) * | 2013-01-04 | 2014-07-16 | 삼성전자주식회사 | 터치 패드, 디스플레이 장치 및 터치 패드의 제어 방법 |
US10175874B2 (en) * | 2013-01-04 | 2019-01-08 | Samsung Electronics Co., Ltd. | Display system with concurrent multi-mode control mechanism and method of operation thereof |
US9584849B2 (en) | 2013-07-17 | 2017-02-28 | Kyung Soon CHOI | Touch user interface method and imaging apparatus |
EP2853992A1 (en) * | 2013-09-27 | 2015-04-01 | Movea | Air pointer with improved user experience |
KR20150046852A (ko) * | 2013-10-23 | 2015-05-04 | 삼성전자주식회사 | 전자 장치 및 전자 장치의 암호를 이용한 인증 방법 |
CN103645847A (zh) * | 2013-12-02 | 2014-03-19 | 乐视致新电子科技(天津)有限公司 | 通过移动终端模拟鼠标控制智能电视的方法和系统 |
JP6034281B2 (ja) * | 2013-12-18 | 2016-11-30 | インターナショナル・ビジネス・マシーンズ・コーポレーションInternational Business Machines Corporation | オブジェクト選択方法、装置及びコンピュータ・プログラム |
CN103727899B (zh) * | 2013-12-31 | 2015-07-01 | 京东方科技集团股份有限公司 | 一种电视系统中检测遥控器旋转角度的方法及电视系统 |
JP2015128918A (ja) * | 2014-01-06 | 2015-07-16 | 株式会社東海理化電機製作所 | 操作装置 |
CN103699244B (zh) * | 2014-01-08 | 2016-08-17 | 福建利利普光电科技有限公司 | 测量仪器中轨迹球的识别方法 |
US10019155B2 (en) * | 2014-06-30 | 2018-07-10 | Honda Motor Co., Ltd. | Touch control panel for vehicle control system |
US9727236B2 (en) * | 2014-08-28 | 2017-08-08 | Peigen Jiang | Computer input device |
CN104866136B (zh) * | 2015-05-11 | 2019-02-15 | 努比亚技术有限公司 | 一种确定终端操作模式的方法及装置 |
US11578470B2 (en) * | 2017-03-30 | 2023-02-14 | Komatsu Ltd. | Control system for work vehicle, method for setting trajectory of work implement, and work vehicle |
CN107145259A (zh) * | 2017-05-09 | 2017-09-08 | 深圳Tcl新技术有限公司 | 触摸图标校对方法及装置 |
US11222295B2 (en) * | 2017-08-24 | 2022-01-11 | Mitsubishi Electric Corporation | Activity recording device, activity recording program, and activity recording method |
CN107728837A (zh) * | 2017-09-27 | 2018-02-23 | 惠州Tcl移动通信有限公司 | 一种触摸屏定位输出的方法、存储介质及智能终端 |
KR102492560B1 (ko) * | 2017-12-12 | 2023-01-27 | 삼성전자주식회사 | 전자 장치 및 그의 입력 제어 방법 |
CN110069147B (zh) * | 2018-01-23 | 2023-02-03 | 可赛尔内存股份有限公司 | 操控装置及其控制方法 |
JP7308188B2 (ja) * | 2018-05-14 | 2023-07-13 | 株式会社ワコム | 学習支援システム |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2009276819A (ja) * | 2008-05-12 | 2009-11-26 | Fujitsu Ltd | ポインティング装置の制御方法およびポインティング装置、並びにコンピュータプログラム |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4049877B2 (ja) * | 1997-03-31 | 2008-02-20 | 株式会社ワコム | 電子ペンを用いたコンピュータシステム |
JPH10340153A (ja) | 1997-06-06 | 1998-12-22 | Smk Corp | 座標入力装置 |
JPH1195912A (ja) | 1997-09-22 | 1999-04-09 | Sanyo Electric Co Ltd | 座標入力装置、座標入力方法及び座標入力プログラムを記録したコンピュータ読み取り可能な記録媒体 |
JP2001117713A (ja) | 1999-10-19 | 2001-04-27 | Casio Comput Co Ltd | データ処理装置、及び記憶媒体 |
JP4878667B2 (ja) | 2000-02-17 | 2012-02-15 | 富士通コンポーネント株式会社 | 座標検出装置及び座標検出方法 |
JP2003167669A (ja) * | 2001-11-22 | 2003-06-13 | Internatl Business Mach Corp <Ibm> | 情報処理装置、プログラム及び座標入力方法 |
JP3952896B2 (ja) | 2002-07-30 | 2007-08-01 | キヤノン株式会社 | 座標入力装置及びその制御方法、プログラム |
JP4776237B2 (ja) | 2005-01-19 | 2011-09-21 | 富士通コンポーネント株式会社 | 座標入力制御システムおよび座標入力制御処理をコンピュータに実行させるためのプログラム |
JP2007164470A (ja) | 2005-12-14 | 2007-06-28 | Alps Electric Co Ltd | 入力装置およびこの入力装置を使用した電子機器 |
KR20080021906A (ko) * | 2006-09-05 | 2008-03-10 | 삼성전자주식회사 | 휴대용 단말기에서 아날로그 조작을 하기 위한 장치 및방법 |
JP4966636B2 (ja) * | 2006-12-13 | 2012-07-04 | 株式会社ワコム | 座標入力装置 |
-
2011
- 2011-05-17 WO PCT/JP2011/002729 patent/WO2011145330A1/ja active Application Filing
- 2011-05-17 CN CN201180002866.0A patent/CN102473060B/zh not_active Expired - Fee Related
- 2011-05-17 US US13/383,966 patent/US9041649B2/en not_active Expired - Fee Related
- 2011-05-17 JP JP2011536220A patent/JP5762967B2/ja not_active Expired - Fee Related
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2009276819A (ja) * | 2008-05-12 | 2009-11-26 | Fujitsu Ltd | ポインティング装置の制御方法およびポインティング装置、並びにコンピュータプログラム |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9176634B2 (en) | 2011-12-15 | 2015-11-03 | Toyota Jidosha Kabushiki Kaisha | Operation device |
WO2013088559A1 (ja) * | 2011-12-15 | 2013-06-20 | トヨタ自動車株式会社 | 操作装置 |
CN103376926B (zh) * | 2012-04-16 | 2016-07-06 | 汉王科技股份有限公司 | 可切换坐标的绘画板及其坐标切换方法 |
US10031610B2 (en) | 2013-02-19 | 2018-07-24 | Toyota Jidosha Kabushiki Kaisha | Operation device for vehicle |
JP5739015B2 (ja) * | 2013-03-28 | 2015-06-24 | 株式会社東芝 | 通信装置、通信方法及びプログラム |
US9736513B2 (en) | 2013-03-28 | 2017-08-15 | Kabushiki Kaisha Toshiba | Communication device and communication method |
JP2015041189A (ja) * | 2013-08-21 | 2015-03-02 | ソニー株式会社 | 表示制御装置、表示制御方法およびプログラム |
JP2015115038A (ja) * | 2013-12-16 | 2015-06-22 | セイコーエプソン株式会社 | 情報処理装置および情報処理装置の制御方法 |
JP2016130975A (ja) * | 2015-01-14 | 2016-07-21 | シャープ株式会社 | 情報処理装置およびプログラム |
JPWO2020217336A1 (ja) * | 2019-04-24 | 2020-10-29 | ||
WO2020217336A1 (ja) * | 2019-04-24 | 2020-10-29 | 株式会社ワコム | タッチパッド用のシステム |
JP7342111B2 (ja) | 2019-04-24 | 2023-09-11 | 株式会社ワコム | タッチパッド用のシステム |
US12019825B2 (en) | 2019-04-24 | 2024-06-25 | Wacom Co., Ltd. | System for touch pad |
JP7514989B2 (ja) | 2019-04-24 | 2024-07-11 | 株式会社ワコム | デバイスドライバ |
CN115185411A (zh) * | 2022-07-08 | 2022-10-14 | 北京字跳网络技术有限公司 | 光标移动方法、装置和电子设备 |
CN115185411B (zh) * | 2022-07-08 | 2024-03-15 | 北京字跳网络技术有限公司 | 光标移动方法、装置和电子设备 |
Also Published As
Publication number | Publication date |
---|---|
CN102473060A (zh) | 2012-05-23 |
US9041649B2 (en) | 2015-05-26 |
US20120113001A1 (en) | 2012-05-10 |
JP5762967B2 (ja) | 2015-08-12 |
JPWO2011145330A1 (ja) | 2013-07-22 |
CN102473060B (zh) | 2016-03-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5762967B2 (ja) | 座標決定装置、座標決定方法及び座標決定プログラム | |
US12093525B2 (en) | Character recognition on a computing device | |
US11520467B2 (en) | Input device and user interface interactions | |
US8217905B2 (en) | Method and apparatus for touchscreen based user interface interaction | |
US8407623B2 (en) | Playback control using a touch interface | |
US8610678B2 (en) | Information processing apparatus and method for moving a displayed object between multiple displays | |
US20110060986A1 (en) | Method for Controlling the Display of a Touch Screen, User Interface of the Touch Screen, and an Electronic Device using The Same | |
US20180088896A1 (en) | Devices, Methods, and Graphical User Interfaces for Media Playback Control Using Intensity-Based User Inputs | |
US20160299659A1 (en) | Remote multi-touch control | |
JP6141301B2 (ja) | 間接的対話装置の対話モデル | |
US9691270B1 (en) | Automatically configuring a remote control for a device | |
US11107345B1 (en) | Automatically configuring a remote control for a device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201180002866.0 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2011536220 Country of ref document: JP |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11783267 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13383966 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 11783267 Country of ref document: EP Kind code of ref document: A1 |