JP2015141686A - Pointing device, information processing device, information processing system, and method for controlling pointing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the operation performance of a pointing device.SOLUTION: A pointing device 10 includes: main operation means (an optical sensor unit 30, displacement detection unit 31, operation button 40, and operation detection unit 41) that performs at least the operation of specifying a coordinate position; and sub-operation means (a touch panel 20 and touch operation detection unit 21) that performs a predetermined operation on the basis of the coordinate position specified by the main operation means. The main operation means and sub-operation means can be switched.

Description

  The present invention relates to a pointing device, an information processing apparatus, an information processing system, and a pointing device control method.

  A mouse is widely used as a pointing device that is an input device of an information processing apparatus such as a personal computer. Conventional ball-type mice have been easily degraded in accuracy due to dirt on the mounting surface, but recently, optical (optical) mice that are less dirty and easy to clean have become mainstream. Such an optical mouse detects displacement between the mouse and the mounting surface of the mouse by observing reflected waves such as infrared rays.

  In addition, as a pointing device, a pen-shaped light receiving device is brought into contact with the display device, the timing when the position shines is picked up, and a light pen that recognizes the position on the screen or a flat sensor is traced with a finger. Touchpads that operate pointers have become popular.

  Here, when a predetermined range is selected using a pointing device and a button associated with the pointing device is operated with respect to the selected range to execute a predetermined process, the desired selection range cannot be selected well. , It may shift unintentionally.

  For example, consider an operation of selecting a plurality of icons within a predetermined range from a plurality of objects (icons) displayed on a display using a mouse and deleting the selected plurality of icons. In general, selection of a predetermined range using a mouse is performed by a mouse drag-and-drop operation. The selection start position is the position at the start of dragging (when the movement starts while pressing the button), and the selection end position is the position at the time of dropping (the time when the movement is completed and the button is released).

  However, when a click operation is performed at the start of dragging, there is a case where the button is deviated from a desired start position due to the reaction of pressing the button. If such a shift in the designated position occurs, it is not possible to select some of the plurality of objects to be selected, and it is necessary to designate coordinates by drag and drop operation again. Become.

  Further, when the click releasing operation at the time of the drop operation is performed, the cursor (pointer) is at the end point (end position) of the selection range. For this reason, for example, if the next operation is to be executed by right-clicking or the like, the operation is performed outside the selection range when the pointing position of the mouse is slightly shifted, and the selection range is released. This shift in the designated position of the mouse is likely to occur frequently at the time of dropping because it occurs when the timing for stopping the mouse is later than the timing for releasing the click. In this case as well, it is necessary to designate coordinates again by a drag-and-drop operation.

  Similarly, a pointing device that acquires absolute coordinates specified when an operation is instructed, such as a light pen or a touchpad, similarly causes an unintended coordinate shift. In particular, when a button attached to a light pen or a stylus of a tablet device is pressed, a force point that is a button to be pressed exists between a fulcrum holding the pointing device and an action point (tip) indicating the position. Therefore, deviation is likely to occur.

  For example, Patent Literature 1 discloses a touch screen display, a motor that vibrates the touch screen display, a software touch pad display unit that displays a software touch pad on the touch screen display (LCD), and a touch position on the touch screen display. And a vibration control unit that controls the motor to vibrate while the touch position is outside the software touch pad when the determined touch position moves from the software touch pad to the outside of the software touch pad. Are disclosed.

  In Patent Document 1, the operability of a specific area set on the touch screen is improved, but the touch screen display included in the information processing device is configured to vibrate, and the above problem is caused by the configuration of the pointing device. Cannot be solved.

  Therefore, the present invention provides a pointing device capable of improving the operation performance by sharing the operation between the main pointing device and the sub pointing device in accordance with the purpose of the operation content in one pointing device. Objective.

  In order to achieve such an object, a pointing device according to the present invention executes at least a main operation unit that performs an operation of designating a coordinate position and a predetermined operation based on the coordinate position designated by the main operation unit. Sub-operating means, and the main operating means and the sub-operating means can be switched.

  According to the present invention, the operating performance of the pointing device can be improved.

1 is an external perspective view of a pointing device according to a first embodiment. 1 is a block diagram of an information processing system including a pointing device and an information processing apparatus (first embodiment). 6 is a flowchart illustrating an example of an operation of a pointing device. It is explanatory drawing explaining a mode that three objects are displayed on the display part of information processing apparatus, and selection operation of an object (1st Embodiment). It is explanatory drawing of spread operation. It is explanatory drawing explaining a mode that three objects are displayed on the display part of information processing apparatus, and selection operation of an object (2nd Embodiment). It is an external appearance perspective view of the pointing device which concerns on 3rd Embodiment. It is a block diagram of the information processing system which consists of a pointing device and information processing apparatus (3rd Embodiment). It is explanatory drawing explaining a mode that three objects are displayed on the display part of information processing apparatus, and selection operation of an object (3rd Embodiment). It is an external view of the pointing device which concerns on 4th Embodiment.

  Hereinafter, a configuration according to the present invention will be described in detail based on the embodiment shown in FIGS.

[First Embodiment]
(Pointing device configuration)
FIG. 1 shows an external view of a pointing device according to the present embodiment. A pointing device 10 shown in FIG. 1 is an optical mouse provided with a touch panel 20. The pointing device 10 is provided with the touch panel 20 on the upper surface side and the operation buttons 40 on the left side surface. An optical sensor unit 30 that includes a light emitter and a light receiver and optically reads the amount, direction, and speed of movement is provided on the bottom surface (not shown).

  For example, when the user performs an operation with the right hand, the pointing device 10 illustrated in FIG. 1 can operate the operation button 40 with a thumb and the touch panel 20 with an index finger, a middle finger, or the like.

  In addition, the arrangement position of the operation button 40 and the touch panel 20 is an example, and is not limited thereto. FIG. 1 shows an example in which the operation button 40 includes two buttons, a so-called left click L button 40a and a right click R button 40b. However, the number of the operation buttons 40 is not limited. There may be one or three or more. Further, a wheel or the like that enables a scroll operation may be provided.

  FIG. 2 is a block diagram of an information processing system including the pointing device 10 and the information processing apparatus 70 shown in FIG. As shown in FIG. 2, the pointing device 10 includes a touch panel 20, an optical sensor unit 30, an operation button 40, a controller 50 that controls the entire pointing device, and a signal output unit that outputs a signal to the information processing apparatus 70. 60.

  The touch operation detection unit 21, the displacement detection unit 31, and the operation detection unit 41 are provided. The touch operation detection unit 21 detects the operation content of the user on the touch panel 20 with a finger, a sli- tus pen, and the like. Is sent to the controller 50. Further, the displacement detection unit 31 detects the displacement amount and the displacement direction in the optical sensor unit 30 and sends them to the controller 50. Further, the operation detection unit 41 detects the presence / absence of a click on the operation button 40 and sends it to the controller 50. Further, the controller 50 generates a signal based on the operation content, the amount of displacement, the displacement direction, the presence / absence of a click, and the like, and causes the signal output unit 60 to output the signal to the information processing device 70.

  Further, the controller 50 can discriminate between a state in which the pointing device 10 functions as a main pointing device and a state in which the pointing device 10 functions as a sub pointing device in a signal output from the signal output unit 60 to the information processing apparatus 70. Output a signal. The control unit of the information processing device 70 determines in which state the pointing device 10 is based on the signal, and realizes the operation content corresponding to the state.

  Note that the communication between the pointing device 10 and the information processing apparatus 70 may be wired or wireless. In the case of wireless communication, the signal output unit 60 uses infrared rays or electromagnetic waves (Bluetooth (registered trademark), ZigBee, NFC (Near Field Communication), etc.) to operate the information processing device 70, operation amount, displacement amount, displacement direction. A wireless signal including the presence / absence of a click is transmitted. In the case of performing wireless communication, the pointing device 10 includes a rechargeable or battery-type power source.

(Control of pointing device)
The pointing device according to the present embodiment includes at least a main operation unit that executes an operation that specifies a coordinate position, and a sub operation unit that executes a predetermined operation based on the coordinate position specified by the main operation unit, The main operation means and the sub operation means can be switched.

  Here, in the present embodiment, the main pointing device (main operation means) means detection of the relative movement distance, movement direction, and speed of the pointing device 10 by the optical sensor unit 30 and the displacement detection unit 31, and the operation buttons 40 and This is detection of a button operation by the operation detection unit 41. Further, the sub pointing device (sub operation means) is detection of a touch operation by the touch panel 20 and the touch operation detection unit 21. In the present embodiment, an example in which the touch panel 20 is a touch panel capable of detecting coordinates of at least two points (also referred to as a multi-point detection touch panel) will be described.

  FIG. 3 is a flowchart showing the operation of the pointing device 10. FIG. 4 is a schematic diagram showing a state in which three objects 81, 82, 83 are displayed on a display unit 71 such as a monitor of the information processing apparatus 70. Hereinafter, an operation (copy operation, cut operation, etc.) for selecting the three objects 81, 82, 83 at the same time and performing predetermined processing on the three objects 81, 82, 83 will be described.

  First, as shown in FIG. 4A, the pointer 80 is moved to an operation target object (here, object 81) by the main pointing device (S101). The displacement detector 31 detects the displacement amount and the displacement direction of the pointing device 10, and the pointer 80 moves corresponding to the displacement amount and the displacement direction. That is, it is a normal operation of moving the pointer position using an optical mouse.

  Next, the selection operation by the sub pointing device is started (S102). In the present embodiment, as shown in FIG. 5, an operation (spread operation) is performed on the touch panel 20 to move two fingers away from one place (points in the figure). Corresponding to this spread operation, the size of the selection frame 84 indicated by the dotted line shown in FIG. 4B can be operated. Note that the operation method of the selection frame 84 on the touch panel 20 is not limited to the spread operation. In the present embodiment, the center position of the touch panel 20 at the start of the operation by the sub pointing device is the coordinate position designated by the main pointing device. As shown in FIG. 4B, a cross pointer 85 is displayed at the operation start position when the operation of the auxiliary pointing device is started.

  At the start of the operation of S102, the pointing device 10 is switched from the operation by the main pointing device to the operation by the sub pointing operation. That is, when the user's finger or slate pen is touching the touch panel 20, the operation start trigger by the sub pointing device is used.

  When the selection frame 84 indicated by the dotted line has a desired size including all objects to be operated, the selection operation by the sub pointing device is ended (S103). When using the multi-point detection touch panel, a rectangle circumscribing each detected point at the end of the operation of the sub pointing device becomes the selection range (selection frame 84).

  At the end of the operation of S103, the pointing device 10 returns from the operation by the sub pointing device to the operation by the main pointing operation. Note that the main pointing device is locked during the sub-pointing operation (S102 to S103), and the pointer 80 is not moved even if the pointing device is displaced.

  In the present embodiment, the time when the user's finger or the sli- tus pen is released from the touch panel 20 (when the contact state is released) is set as a trigger for ending the operation by the sub pointing device.

  At the end of the operation of S103, as shown in FIG. 4C, the display of the cross pointer 85 and the selection frame 84 is canceled, and the objects 81, 82, and 83 within the rectangular range selected by the selection frame 84 are selected. It becomes a state. At this time, the main pointing device is operated from the position of the pointer 80. The operation to be performed next can instruct a desired operation targeted by the selected object (S104). For example, the operation buttons 40 can be used to instruct operations such as cutting, copying, and deleting objects.

  According to the pointing device according to the present embodiment, in one pointing device, the operation performance can be improved by sharing the operation between the main pointing device and the sub pointing device in accordance with the purpose of the operation content. In other words, by separating the main pointing device for specifying the position (coordinates) of the object to be operated and the sub pointing device for performing the accompanying operation, the operability is improved by eliminating the coordinate movement accompanying the operation content. In addition, it is possible to easily hold the designated position even if the pointing device in the pointing device is an unstable pointing device.

  Therefore, when designating the coordinates to be operated, an unintended coordinate shift due to an operation error is unlikely to occur, and a coordinate re-designation operation when a shift occurs can be eliminated.

  In the first embodiment, an example in which the optical mouse unit and the operation button are used as the main pointing device and the touch panel is used as the sub pointing device has been described. However, the optical mouse unit and the operation button are used as the sub pointing device with the touch panel as the main pointing device. You may make it function as a device. The same applies to the following embodiments.

[Second Embodiment]
Hereinafter, other embodiments of the pointing device according to the present invention will be described. In addition, description about the same point as the said embodiment is abbreviate | omitted suitably.

  In the first embodiment, the example including the multi-point detection touch panel has been described. However, in the second embodiment, the touch panel has a configuration capable of detecting only a single coordinate (also referred to as a single point detection touch panel). explain. When a single point detection touch panel is used, an operation of switching between selection of the start point and end point of the selection range is required.

  FIG. 6 is a schematic diagram illustrating a state in which three objects 81, 82, and 83 are displayed on the display unit 71 such as a monitor of the information processing apparatus 70. Hereinafter, an operation of simultaneously selecting the three objects 81, 82, and 83 and performing predetermined processing on the three objects 81, 82, and 83 will be described.

  First, as shown in FIG. 6A, the pointer 80 is moved to the operation target object (object 81) by the main pointing device (S101). The displacement detector 31 detects the displacement amount and the displacement direction of the pointing device 10, and the pointer 80 moves corresponding to the displacement amount and the displacement direction.

  Next, the selection operation by the sub pointing device is started (S102). That is, when the touch panel 20 is touched, the main pointing device is locked, and a touch operation can be performed from the coordinate position designated by the main pointing device. A crosshair pointer 85 is displayed at the position of the pointer 80.

  Then, as shown in FIG. 6B, the start point (selection start position) of the selection frame 84 is determined by a touch operation. Since the touch panel is a single point detection touch panel, the user moves the cross pointer 85 by the slide operation of the touch panel 20 to determine the start point.

  Then, when a predetermined switching button is pressed, switching to designation of the end point (selection end position) of the selection frame 84 by the touch operation is performed. As a switching button at this time, a dedicated operation button may be provided in the pointing device 10, but an existing button (L button 40a) or the like can be switched. That is, for example, when the L button 40a is pressed during an operation with the sub-pointing device, control is performed to switch from the position designation of the selection start position to the position designation of the selection end position. In addition, when the button is pressed again, control is performed to return from the designation start / end position designation to the selection start position designation.

  When the switch button is pressed, as shown in FIG. 6C, the user moves the cross pointer 85 by the slide operation of the touch panel 20 to determine the end point. In addition, when using a single point detection touch panel, the rectangle formed including the start point and the end point becomes the selection range (selection frame 84).

  When the selection frame 84 indicated by the dotted line has a desired size including all objects to be operated, the touch on the touch panel 20 is released, and the selection operation by the sub pointing device is ended (S103).

  At the end of the operation in S103, as shown in FIG. 6D, the display of the cross pointer 85 and the selection frame 84 is canceled, and the objects 81, 82, and 83 within the rectangular range selected by the selection frame 84 are selected. It becomes a state. At this time, the main pointing device is operated from the position of the pointer 80. The operation to be performed next can instruct a desired operation targeted by the selected object (S104).

  In the second embodiment, the position designation of the selection start position and the position designation of the selection end position are switched by pressing the switching button, but this may be determined by a predetermined touch operation on the touch panel 20. good. For example, this switching may be performed by an operation (tap operation) in which contact with the touch panel 20 is stopped and contact is immediately made again. In this case, the determination of the contact release in S103 may be made when the contact is released for a predetermined time or more in order to determine the tap operation.

[Third Embodiment]
(Pointing device configuration)
In the first and second embodiments, the pointing device 10 including the touch panel 20 has been described. However, the functions of the main pointing device and the sub pointing device may be switched by a switching button. That is, as with the main pointing device, the secondary pointing device is the relative movement distance detection of the pointing device by the optical sensor unit 30 and the displacement detection unit 31, and the button operation by the operation button 40 and the operation detection unit 41.

  FIG. 7 shows an external view of the pointing device according to the present embodiment. The pointing device 10 shown in FIG. 7 is an optical mouse provided with a main / sub switching button 22 as switching means. The pointing device 10 is provided with operation buttons 40 (L button 40a and R button 40b) on the upper surface side, and a main / sub switch button 22 is provided on the left side surface. An optical sensor unit 30 is also provided. If there are three or more operation buttons 40, one of them may be the main / sub switch button 22.

  The pointing device 10 shown in FIG. 7 can be operated, for example, when the user operates with the right hand, the main / sub switching button 22 with the thumb and the operation button 40 with the index finger, the middle finger, or the like.

  FIG. 8 is a block diagram of the pointing device shown in FIG. As shown in FIG. 8, the pointing device 10 includes a main / sub switching button 22, an optical sensor unit 30, an operation button 40, a controller 50, a signal output unit 60 that outputs a signal to the information processing device 70, It has.

  The switching operation detection unit 23, the displacement detection unit 31, and the operation detection unit 41 are provided. The switching operation detection unit 23 detects that the main / sub switching button 22 has been clicked, and this is detected by the controller 50. Send to.

(Control of pointing device)
In the present embodiment, when the pointing device 10 is used as a main pointing device, the L button 40a is used for a left click operation, the R button 40b is used for a right click operation, and the main / sub switching button 22 is used for switching to a sub pointing device. To do. When the pointing device 10 is used as a sub-pointing device, the L button 40a is used to select a selection start position, the R button 40b is used to select a selection end position, and the main / sub switch button 22 is used to switch to the main pointing device. To do.

  FIG. 9 is a schematic diagram showing a state where three objects 81, 82, 83 are displayed on a display unit 71 such as a monitor of the information processing apparatus 70. Hereinafter, an operation of simultaneously selecting the three objects 81, 82, and 83 and performing predetermined processing on the three objects 81, 82, and 83 will be described.

  First, as shown in FIG. 9A, the pointer 80 is moved to the operation target object (object 81) by the main pointing device (S101). The displacement detector 31 detects the displacement amount and the displacement direction of the pointing device 10, and the pointer 80 moves corresponding to the displacement amount and the displacement direction.

  Next, the selection operation by the sub pointing device is started (S102). That is, when the main / sub switching button 22 is pressed, the operation is switched from the main pointing device to the operation with the sub pointing device.

  When the main / sub switching button 22 is pressed, a cross pointer 85 is displayed at the position of the pointer 80, and the selection start position of the selection frame 84 becomes possible as shown in FIG. 9B. The selection start position is specified by pressing the L button 40a. Next, as shown in FIG. 9C, the selection end position is designated. The selection end position is specified by pressing the R button 40b.

  When the selection frame 84 indicated by the dotted line has a desired size including all objects to be operated, the selection operation by the sub pointing device is ended (S103).

  Next, when the main / sub switching button 22 is pressed, the operation returns to the operation with the main pointing device from the sub pointing device. Then, as shown in FIG. 9D, the display of the cross pointer 85 and the selection frame 84 is canceled, and the objects 81, 82, and 83 in the rectangular range selected by the selection frame 84 are selected. At this time, the main pointing device is operated from the position of the pointer 80. The operation to be performed next can instruct a desired operation targeted by the selected object (S104).

  According to the pointing device according to the third embodiment, a single pointing device can be operated with the main pointing device and the sub pointing device according to the purpose of the operation content with a simple configuration without providing a touch panel. The operating performance can be improved by sharing.

  As described above, in the first to third embodiments, the optical mouse is taken as an example of the pointing device 10, but a mechanical mouse may be used. The pointing device 10 is not limited to a mouse, and may be applied to a light pen, a tablet, a touch pad, a touch panel, and the like. Further, the pointing device 10 may be provided in the information processing apparatus 70 main body.

[Fourth Embodiment]
Further, the present invention can be applied to a glasses-type display device. FIG. 10 is an enlarged view of a main part of a glasses-type display device which is an embodiment of the pointing device according to the present invention. The pointing device (glasses-type display device 90) is a head-mounted display type, and includes an imaging device (imaging device 91) that captures an operator's eyeball, and a control unit that tracks the operator's line of sight based on the captured image. And a display device 93 for displaying the operator's line-of-sight position and a touch panel 94. The line-of-sight display on the display device by the imaging device 91 and the control unit functions as a main pointing device, and the touch panel 94 functions as a sub-pointing device. Is.

  The arrows shown in FIG. 10 indicate the direction (front) of the wearer's line of sight, and in FIG. 10, the line-of-sight side portion (front portion) of the glasses-type display device 90 and the right part of the temple 92 are illustrated.

  As shown in FIG. 10, an imaging device (camera) 91 that captures the eye movement of the wearer is provided at the front portion of the glasses-type display device 90. Further, the frame 95 has a built-in control unit that tracks the operator's line of sight based on the photographed image and displays it on the display device 93. Further, a touch panel 94 is provided on the right temple 92.

  Here, the imaging device 91 tracks which position on the display device 93 the wearer is gazing at, that is, the wearer's line of sight, and based on the tracking result, determines a predetermined position on the display device 93. Acts as the main pointing device for selection. Then, for example, by starting an operation of the touch panel 94 at a desired selection position, the touch panel 94 is caused to function as a sub pointing device.

  The pointing device according to the present embodiment described above can be used in an information processing apparatus that requires a graphic or text display device and a pointing device for designating arbitrary coordinates in the display device. Further, the present invention can be applied to a mouse, a light pen, a tablet, a touch pad, a touch panel, and the like, and can be used for an information processing apparatus such as a personal computer (PC), a workstation, a portable terminal, a smartphone, and a game machine. Further, it can be used for an information processing apparatus that uses motion capture and line-of-sight tracking by a camera as a pointing device.

  The above-described embodiment is a preferred embodiment of the present invention, but is not limited thereto, and various modifications can be made without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 10 Pointing device 20 Touch panel 21 Touch operation detection part 22 Main / sub switching button 23 Switching operation detection part 30 Optical sensor part 31 Displacement detection part 40 Operation button 40a L button 40b R button 41 Operation detection part 50 Controller 60 Signal output part 70 Information Processing device 71 Display unit 80 Pointer 81, 82, 83 Object 84 Selection frame 85 Cross pointer 90 Glasses-type display device 91 Imaging device 92 Temple 93 Display device 94 Touch panel 95 Frame

JP 2011-181004 A

Claims (9)

A main operation means for performing at least an operation of designating a coordinate position;
Sub-operation means for executing a predetermined operation based on the coordinate position designated by the main operation means,
A pointing device characterized in that the main operating means and the sub operating means can be switched. The main operation means performs an object selection operation,
The sub-operation means executes an operation for designating a predetermined range including the object selected by the main operation means,
The pointing device according to claim 1, wherein an operation for executing a predetermined process is executed by the main operation unit with respect to the predetermined range selected by the sub-operation unit. The main operation means includes a displacement detection means for detecting a displacement direction and a displacement amount of the pointing device, and an operation button for performing a selection operation.
The pointing device according to claim 1, further comprising a touch panel as the sub operation unit.   The operation by the main operation unit is stopped when contact with the touch panel is detected, and the operation by the main operation unit is resumed when contact release to the touch panel is detected. 3. The pointing device according to 3. A switching means for switching between the operation by the main operation means and the operation by the sub operation means,
The pointing device according to claim 1 or 2, wherein when a predetermined operation is performed on the switching means, the main operating means and the sub operating means are switched. It is a head mounted display type,
An imaging device that captures the operator's eye and tracks the operator's line of sight;
A display device for displaying the line-of-sight position of the operator imaged by the imaging device;
The pointing device according to claim 1, further comprising: a touch panel, wherein the imaging apparatus functions as the main operation unit, and the touch panel functions as the sub operation unit. A pointing device according to any one of claims 1 to 6;
When a signal is input from the pointing device, the case where the pointing device functions as the main operation unit and the case where the pointing device functions as the sub operation unit are determined, and processing according to the determination result is executed. And an information processing means. A pointing device according to any one of claims 1 to 6;
When a signal is input from the pointing device, the case where the pointing device functions as the main operation unit and the case where the pointing device functions as the sub operation unit are determined, and processing according to the determination result is executed. And an information processing apparatus. A main operation means for performing at least an operation of designating a coordinate position;
A pointing device comprising: sub-operating means for executing a predetermined operation based on the coordinate position designated by the main operating means;
The main operation means performs an object selection operation,
Causing the sub-operation means to execute an operation for designating a predetermined range including the object selected by the main operation means;
A pointing device control method, comprising: causing the main operation means to execute an operation for executing a predetermined process for the predetermined range selected by the sub operation means.