JP2017143530A - Mobile terminal device, method of controlling terminal, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make a mobile terminal device convenient for use when it is used as an operation device relative to a display device as an external apparatus.SOLUTION: In a mouse style comprising a touch display unit 108 provided on a surface side of a case, an extraction unit 113 housed in the case in an extractable manner, and a mouse sensor 109 mounted on the extraction unit 113 with the surface side of the case directed to a plane, the extraction unit 113 is mounted on the case so that a space is formed by a step (gp) between a top end face of the extraction unit 113 and a display surface of the touch display unit 108, from which the extraction unit 113 is extracted. By this, the whole case obliquely rises and comes to have a shape of protruding upward. When the case is mounted on the plane and held so that a back face side of the case is covered with a palm, the shape thereof produces a form easy to be held as a mouse.SELECTED DRAWING: Figure 8

Description

  The present invention relates to a terminal device and a program.

Conventionally, as a mobile terminal device that can be used as an operation device for a display device as an external device, there is a technology that enables a mobile phone to be used as a pointing device (mouse) for the external display device (Patent Documents 1 to 3). 4). Further, in the techniques of Patent Document 1 and Patent Document 4 described above, on the front side of the casing constituting the mobile terminal, that is, on the back side opposite to the front side where the key operation unit and the display unit are disposed. A mouse detection unit (light emitting unit or light receiving unit for detecting movement of the casing) is provided, and the movement signal is sent in response to a moving operation that moves the casing on the plane with the mouse detection unit facing the plane such as a desk surface. However, if you operate while holding the back side of the housing with your palm, your hands will touch the key operation unit and display unit on the front side, and your hands may be greasy. There was a possibility of incorrect key operation.
Therefore, in the techniques of Patent Document 2 and Patent Document 3 described above, the casing is folded to prevent the hand from touching the key operation unit and the display unit when used as a mouse. I have to.

JP 2007-123962 A JP 2008-263454 A JP 2007-150695 A JP 2010-165289 A

  However, in the technique of the above-mentioned patent documents 2 and patent documents 3, since it is on the condition that the case is made into a folding shape, the recent mainstream smartphone type, that is, the entire surface side of the case The technology cannot be adopted in a style in which a touch display unit with a touch panel is provided and no major device is provided on the back side. Further, each of the above-described prior arts is limited to the case where the mobile phone is used as a pointing device (mouse) for an external display device, and thus there remains a problem in terms of versatility.

  An object of the present invention is to make it usable with ease when used as an operation device for an external device.

The invention according to claim 1 is a terminal device that communicates with a neighboring external device, and when a specific instruction operation instructing communication with the external device is detected, the terminal device displays display information. It is determined whether or not the specific instruction operation is invalidated based on whether or not the display instruction is displayed, and the terminal device is shifted to the device mode for pairing with the external device on the condition that the display information is displayed. The display information displayed on the terminal device is transmitted to the external device, and the movement operation of the terminal device in the plane direction on the predetermined plane is moved and displayed on the cursor displayed on the external device. First control means for controlling the cursor movement signal to be transmitted to the external device, and when the specific instruction operation is detected after the transition to the device mode, And determining whether or not to invalidate the specific instruction operation based on whether or not the display position of the cursor is the display position of the display object. On the other hand, there is provided second control means for controlling to newly transmit the display contents executed by the terminal device to the external device.
The invention according to claim 3 is a terminal device that performs communication with a neighboring external device, and when a specific instruction operation that instructs communication with the external device is detected, the terminal device is connected to the external device. Switch to the pairing device mode, and when switching to the device mode, operate the terminal device as the mouse device mode for the external device or the touch input device mode depending on the orientation of the terminal device body. A determination means for determining whether or not the mouse device mode is determined by the determination means; and a cursor displayed on the external device for a movement operation in a plane direction on the predetermined plane of the terminal device. A first control means for controlling to send a cursor movement signal as an operation for movement to the external device; A second control unit configured to control the display content processed based on detection of various touch operations on the terminal device to be transmitted to the external device when the touch input device mode is determined; It is characterized by that.

  An object of the present invention is to make it usable with ease when used as an operation device for an external device.

The figure which showed the various display apparatuses 200 as an external apparatus which can perform near field communication between the mobile telephones 100. FIG. The figure which showed the mode at the time of using the mobile telephone 100 as a pointing device (mouse) with respect to the display apparatus 200. FIG. The figure which showed the mode at the time of using the mobile telephone 100 as a touch input device with respect to the display apparatus 200. FIG. FIG. 3 is a block diagram showing basic components of the mobile phone 100. The figure for demonstrating the display position management table DT. The block diagram which showed the basic component of the display apparatus 200 (TV receiver 200A). FIGS. 2A and 2B are schematic external views of the mobile phone 100, in which FIG. 1A is a view of the mobile phone 100 viewed from the direction of the surface, and FIG. (1) and (2) are schematic external views as viewed from one side of the mobile phone 100, and (2) further functions as the operation device (mouse) for the display device 200. The figure including the partially expanded schematic sectional view in a state. The flowchart which showed operation | movement of the mobile telephone 100 started execution by monitor link button operation (pairing operation). The flowchart which showed the operation | movement following FIG. The flowchart which showed the operation | movement following FIG. 7 is a flowchart showing an interrupt process that is started when a monitor search signal is received from the mobile phone 100 as an operation of the display device 200. The figure which showed the example of a display in the state where the screen by the side of the mobile telephone 100 and the screen by the side of the display apparatus 200 are linked when using the mobile telephone 100 as a pointing device (mouse) of the display apparatus. The figure which showed the other display example in the state where the screen of the mobile telephone 100 side and the screen of the display apparatus 200 are linked, when using the mobile telephone 100 as a touch input device of the display apparatus 200. FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS.
The present embodiment is applied to a mobile phone as a mobile terminal device. FIG. 1 is a diagram showing various display devices 200 as external devices capable of short-range communication with the mobile phone 1.
The mobile phone 100 includes a voice call function, an e-mail function, an Internet connection function (Web access function), a short-range communication function, a touch display function, and the like, and a display device 200 as an external device (electronic device including a display unit). Data communication is possible. The display device 200 is a variety of electronic devices installed in the home, such as a television receiver 200A, a projector 200B, a digital photo frame 200C, a personal computer (PC) 200D, an in-vehicle terminal 200E, and the like. The display device 200 is directly connected for communication via short-range communication, for example, Bluetooth (registered trademark).

  Here, the mobile phone 100 and the display device 200 are connected by short-range communication when the device itself functions as an operation device for the display device 200, and the device itself functions as a pointing device (mouse) for the display device 200. Or the device itself functions as a touch input device for the display device 200. 2 shows a case where the mobile phone 100 is used as a pointing device (mouse) for the display device 200 (the television receiver 200A in the illustrated example), and FIG. 3 shows that the mobile phone 100 is used as a touch input device for the display device 200. In the figure showing the state in use, how to place and hold a finger, how to operate, etc. is not limited to the illustrated example, but is arbitrary.

  When the mobile phone 100 performs an access operation on an object to be processed, that is, a monitor link button (to be described later) is operated to link the mobile phone 100 and the display device 200 via short-range communication. When there is a display device 200 in the vicinity of the mobile phone 100 (for example, within a radius of 5 m) is searched for whether there is a display device 200 capable of communication connection (short-distance communication), the display device 200 exists in the vicinity. If so, the display device 200 is specified as the output monitor of the mobile phone 100.

  When the display device 200 is used as an output monitor for the mobile phone 100 and the mobile phone 100 is used as an operation device for the output monitor and it is desired to link the mobile phone 100 and the display device 200, the monitor described above is used. The link button is operated. Here, the object to be processed is an icon, file, image data, text data, etc. held on the mobile phone 100 side. When the access operation is performed on the object to be processed (when the monitor link button is operated), the mobile phone 100 identifies the object or the processing result as the output monitor 200. Control is performed to display on the screen.

  That is, the object to be processed as described above, that is, an object accessible on the mobile phone 100 side (an object held on the mobile phone 100 side) is referred to as a terminal object, or the object to be displayed on the display device 200 specified as the output monitor or the object When the processing result is referred to as a monitor link object, the mobile phone 100 performs control for displaying the monitor link object on the display device 200 in accordance with an access operation to the terminal object. For example, if the terminal object is an icon, in order to display the terminal object (icon) on the display device 200 as a monitor link object, direct communication transmitted to the display device 200 via short-range communication is performed. ing.

  In this way, the mobile phone 100 performs control to display the monitor link object on the display device 200 specified as the output monitor, and then causes the mobile phone 100 to function as an operation device for the display device 200. The current style of the telephone 100 is determined, the type of device is selected according to the style, and the type of operation signal according to the selected type of device is transmitted to the display device 200 identified as the output monitor via short-range communication. To send.

  That is, when a pointing device (mouse) is selected as the device type, the cursor on the display device 200 is moved and displayed according to the moving operation of the mobile phone 100, or the display is displayed according to the pressing operation of the monitor link button. Processing related to the monitor link object on the apparatus 200, that is, processing (described later) for the terminal object corresponding to the monitor link object is executed. When a touch input device is selected as the device type, a key operation on a soft key (touch key) is detected by a touch display function, or a touch action operation is detected. Here, the touch action operation includes a tap operation, a double tap operation, a hold (hold) operation, a pitch-in / pitch-out (enlargement / reduction) operation, a flick (pay = scroll) operation, a drag (move) operation, and the like. The object on the display screen is controlled according to the touch operation signal, and the monitor link object on the display device 200 is controlled.

FIG. 4 is a block diagram showing basic components of the mobile phone 100.
The mobile phone 100 includes a voice call function, an e-mail function, an Internet connection function (Web access function), a near field communication function, and the like. The CPU 101 serves as a core, and includes a power supply unit 102, a storage unit 103, a memory 104, and a wireless communication unit 105. , An audio signal processing unit 106, an operation unit 107, a touch display unit 108, a mouse sensor 109, a short-range communication unit 110, a housing orientation detection unit 111, and a drawer detection unit 112.

  The CPU 101 is a central processing unit that operates by supplying power from a power supply unit 102 having a secondary battery (not shown) and controls the overall operation of the mobile phone 100 according to various programs in the storage unit 103. is there. The storage unit 103 stores a program and various applications for realizing the present embodiment in accordance with the operation procedure shown in FIGS. 9 to 11, and stores information necessary for the program. Yes. The storage unit 103 may include, for example, a removable portable memory (recording medium) such as an SD card, or may include a storage area on a predetermined external server (not shown). . The memory 104 is a work area for temporarily storing various information necessary for the operation of the mobile phone 100 such as flag information, and is provided with a display position management table DT described later.

  The radio communication unit 105 demodulates the received signal into a received baseband signal during the operation of the call function, and then outputs the audio from the call speaker SP via the audio signal processing unit 106, and the input audio from the call microphone MC Data is taken from the audio signal processing unit 106, encoded into a transmission baseband signal, and then transmitted from an antenna (not shown). The operation unit 107 includes a push button type power key, a side key SK such as a menu key, and the above-described monitor link button MB. The CPU 101 determines a predetermined value according to an input operation signal from the operation unit 107. Execute the process. The monitor link button MB is used when the mobile phone 100 is caused to function as an operation device (mouse) for the display device 200, or is used when a monitor link object displayed on the display device 200 is operated. Key.

  The touch display unit 108 is disposed on the surface side of the casing constituting the mobile phone 100. For example, the touch display unit 108 displays character data, a standby image, various objects on a high-definition liquid crystal display or an organic EL display, A touch key group such as a soft keyboard is displayed. It has a screen with different aspect ratios, and a transparent touch sensor (not shown) is stacked over the entire area of the screen to allow touch operation with a finger. And a touch screen for inputting data corresponding to the touch operation. Note that the touch sensor employs a capacitive method that is excellent in weight reduction, light transmission, durability, etc., and detects a key operation on a soft key (touch key) or a touch operation action (for example, scrolling, Bitch-in, Bitch-out, etc.).

  The mouse sensor 109 constitutes a light emitting unit and a light receiving unit of an optical mouse using infrared rays or laser light, and optically detects the moving direction and moving amount of the mobile phone 100 in the mouse mode. The short-range communication unit 110 performs Bluetooth communication, and its communicable area has, for example, a substantially radius of 5 m, and performs serial / parallel conversion or the like on data to be communicated with Bluetooth. The housing orientation detection unit 111 is an acceleration sensor that detects the orientation (posture) of the housing. The drawer detection unit 112 detects whether the mouse sensor 109 housed in the mobile phone 100 so as to be pulled out is in a state of being pulled out by any one of magnetic, mechanical, and electrical. The detection method is arbitrary.

FIG. 5 is a diagram for explaining the display position management table DT.
The display position management table DT is a table that manages the display position of the object displayed on the mobile phone 100 side in association with the display position of the object displayed on the display device 200 side. “Display position of object” and “Display position of monitor link object” are included. That is, the display position management table DT corresponds to “object” “a”, “b”,..., “Monitor link object display position” on the display device 200, and on the mobile phone 100. It is a table that stores and manages “terminal object display position” in association with each other.

FIG. 6 is a block diagram showing basic components of a television receiver 200A that is the display device 200. As shown in FIG.
The television receiver 200 </ b> A has a CPU 201 as a core, in addition to basic components such as a power supply unit 202, a storage unit 203, a television broadcast receiving unit 204, an operation unit 205, a display unit 206, and a speaker 207, in this embodiment. The infrared communication unit 208 and the short-range communication unit 209 are included. The storage unit 203 stores a program and various applications for realizing the present embodiment in accordance with the operation procedure illustrated in FIG. 12, and stores information necessary for the program. The TV receiver 200A can perform infrared communication with a TV remote controller (not shown) through the infrared communication unit 208, and can perform short-range communication with the mobile phone 100 through the short-range communication unit 208. (Bluetooth communication) can be performed.

  The mobile phone 100 and the television receiver 200A establish a communication link through Bluetooth communication (communication connection is set), whereby the short-range communication unit 110 on the mobile phone 100 side and the short-range communication unit on the TV receiver 200A side Via 208, the cellular phone 100 and the television receiver 200A can perform data communication with each other. In this case, when a monitor search operation (link operation: pairing operation) for specifying a connection partner is performed from the mobile phone 100, the mobile phone 100 has a display device 200 capable of communication connection nearby. After transmitting a monitor search signal for searching whether or not, a response signal is received from the television receiver 200A to establish a communication link between the mobile phone 100 and the television receiver 200A.

7 and 8 are schematic external views of the mobile phone 100, and FIG. 7 (1) is a view of the mobile phone 100 as viewed from the front (front) direction. FIG. 7B is a view as seen from the back (back) direction. FIGS. 8A and 8B are views seen from the direction of one side surface of the mobile phone 100.
The mobile phone 100 has an overall rectangular parallelepiped shape, and the back surface of the case has a curved shape that is slightly raised in an arc as shown in FIG. On the surface of the cellular phone 100, as shown in FIG. 5 (1), touch display units 108 having different aspect ratios are arranged. FIG. 7A is a front view of the mobile phone 100 showing a state in which the casing is oriented vertically so that the touch display unit 108 becomes a vertical screen. One end side in the longitudinal direction of the vertical screen (upper side in the drawing) ) Is provided with a call speaker SP, and a call microphone MC is provided on the other longitudinal end of the vertical screen (lower side in the figure). Furthermore, the drawer part 113 is accommodated in the longitudinal direction other end side (lower side in a figure) of a vertical screen so that extraction | drawer is possible. In addition, the above-described push button type monitor link button MB is disposed at a substantially central portion of the back surface of the housing.

  The drawer portion 113 is housed in a housing so that it can be pulled out (slidable). When the mobile phone 100 is used as a pointing device (mouse), as shown in FIG. The drawer 113 is pulled out from the one end of the casing. FIG. 8A shows a state in which the drawer portion 113 is housed in the housing, and FIG. 8B shows a state in which the drawer portion 113 is pulled out from the housing. 8 (1) and 8 (2) show a state where the housing is placed on a flat surface (for example, a desk surface) with the front surface side of the housing facing down in the drawing. A mouse sensor 109 is housed in the drawer 113, and its light emitting surface and light receiving surface are exposed from the inner surface opening of the drawer 113.

  In this way, in the mouse style in which the front side of the housing is on a flat surface, when the drawer portion 113 is pulled out from the inside of the housing, a space gp is formed between the front end surface of the drawer portion 113 and the display surface of the touch display portion 108. The drawer part 113 is attached to the housing so that is formed. In this case, the drawing portion 113 is drawn in an oblique direction (in the illustrated example, in an oblique direction of approximately 60 °) from the vicinity of one end portion on the surface side of the housing. As a result, the leading end portion of the lead-out portion 113 and the other end portion on the surface side of the housing are placed on the flat surface in a state of contact (two-point contact state) on the flat surface (for example, the desk surface).

  The drawer portion 113 is pulled in an oblique direction from the vicinity of one end of the surface of the housing, so that one end of the housing is raised, that is, the entire housing is raised obliquely and protrudes upward in a curved line. Form (mouse style) that is easy to hold as a mouse when the case is placed on a flat surface such as a desk surface and the case is sandwiched so that the back side of the case is wrapped with a palm. It becomes. As described above, the back surface of the housing has a curved shape that is slightly raised in an arc shape. However, the entire drawing portion 113 is also a slightly arc-shaped streamline type, so that when the drawing portion 113 is pulled out, the drawing portion 113 is pulled out. The outer surface of the part 113 and the outer surface of the end portion of the housing are curvedly continuous.

  Although not shown in the drawing, the drawing portion 113 is attached with a locking mechanism and a stopper mechanism. This locking mechanism is configured so that the drawer portion 113 is elastically locked in a state in which the drawer portion 113 is completely pulled out from the housing and in a state in which the drawer portion 113 is completely stored in the housing, thereby maintaining the state. Therefore, when the drawer portion 113 is pulled out against the elastic force, the locking is released. In addition, the stopper mechanism is configured to have a stopper that restricts the drawer portion 113 from being pulled out at the longest position in order to suppress the drawer from being drawn out beyond the longest drawer amount shown in FIG. In addition, about the latching mechanism and the stopper mechanism, since the mechanism used generally is used, the illustration and detailed description shall be abbreviate | omitted.

  Further, as shown in the partial enlarged view shown in FIG. 8B, a mouse sensor 109 is mounted in the drawer portion 113 as a movement sensor for detecting the movement of the mobile phone 100. This mouse sensor (movement sensor) 109 is an optical sensor using infrared rays or laser light, has a light emitting part and a light receiving part, and irradiates the desk surface on which the housing is placed and receives the reflected light. I have to. It should be noted that when the above-described drawer detection unit 112 detects whether or not the drawer unit 113 (mouse sensor 109) has been pulled out, it is determined whether or not the drawer unit 113 has been pulled out to its longest drawer position. I try to detect it. Note that a power supply battery (for example, a button battery) 102a of the power supply unit 102 is stored in the drawer unit 113 so as to be freely inserted and removed (replaceable).

Next, the operation concept of the mobile phone 100 and the display device 200 (television receiver 200A) in the present embodiment will be described with reference to the flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. Further, it is possible to sequentially execute the operation according to the above-described program code transmitted via a transmission medium such as a network. In other words, in addition to the recording medium, an operation unique to the present embodiment can be executed using a program / data supplied externally via a transmission medium.
9 to 11 are flowcharts showing an outline of the operation of the characteristic part of the present embodiment in the entire operation of the mobile phone 100. When the flow of FIG. 9 to FIG. Return to the main flow of operation (not shown).

  9 to 11 are flowcharts showing operations of the mobile phone 100 that are started to be executed by a monitor link button operation (pairing operation). FIG. 12 is a flowchart showing the operation on the display device 200 (television receiver 200A) side. FIG. 13 shows a display example in a state where the screen on the mobile phone 100 side and the screen on the display device 200 side are linked when the mobile phone 100 is used as a pointing device (mouse) of the display device 200. 14 is a diagram illustrating another display example in which the screen on the mobile phone 100 side and the screen on the display device 200 side are linked when the mobile phone 100 is used as a touch input device of the display device 200. is there. Hereinafter, the flow of FIGS. 9 to 12 will be specifically described with reference to FIGS. 13 and 14.

  First, in the cellular phone 100, the CPU 101 checks whether the terminal object to be processed is displayed on the touch display unit 108 (step A1 in FIG. 9). If the terminal object is not currently displayed (NO in step A1), the operation of the current monitor link button is invalidated, and the flow of FIGS. 9 to 11 is exited. Is displayed (YES in step A1), a monitor search signal is transmitted from the short-range communication unit 110 to search for the display device 200 existing in the vicinity (step A2), It is checked whether the response is received from the display device 200 by short-range communication (step A3).

  If there is no response from any of the display devices 200 (NO in step A3), the mobile phone 100 exits the flow of FIGS. 9 to 11, but receives the response from the surrounding display devices 200 (step The display device 200 is specified as an output monitor, and the ID (monitor ID) received from the display device 200 at the time of the response is temporarily stored in the memory 14 (step A4). The display device 200 specified as the output monitor is simply referred to as an output monitor. Then, the terminal object displayed in the terminal screen is transmitted as a monitor link object to the output monitor (step A5). Thus, when the display position of the monitor link object and the display position of the cursor transmitted from the output monitor are received (step A6), the display position of the terminal object corresponding to this monitor link object is detected, and the display of this terminal object is displayed. The position and the display position of the monitor link object are associated with each other and stored in the display position management table DT (step A7).

FIG. 12 is a flowchart that is started when a monitor search signal is received from the mobile phone 100 as an operation of the display device 200 (television receiver 200A).
First, when a monitor search signal is received from the cellular phone 100, the preset identification information (monitor ID) is read, and this monitor ID is transmitted to the cellular phone 100 to respond to the monitor search signal (step). B1). Then, it is checked whether the monitor link object has been received from the mobile phone 100 by short-range communication (step B2). If the monitor link object is not received (NO in step B2), a predetermined time (for example, It is checked whether a time-out has occurred after 3 seconds) (step B3). Further, when the time-out occurs (YES in step B3), the process exits from the interrupt processing flow of FIG.

  When the monitor link object is received (YES in step B2), the monitor link object is displayed on the monitor screen and the cursor is temporarily displayed in the vicinity thereof (step B4). Note that “temporary display of the cursor” means that after the cursor is displayed together with the monitor link object, if the operation is not performed on the mobile phone 100, the cursor display is erased after the cursor display position is saved, and thereafter This means that the cursor is displayed at the display position saved at the timing of starting the operation (hereinafter the same). After the monitor link object and the cursor display position are detected and transmitted to the mobile phone 100 (step 5), it is checked whether a cursor movement signal is received from the mobile phone 100 (step 6), or a new It is checked whether a monitor link object has been received (step B8), or whether a link end instruction has been received from the mobile phone 1 (step B9).

  On the other hand, the mobile phone 100 stores the display position of the monitor link object in the display position management table DT in response to the reception of the display position of the monitor link object as described above (step A6 in FIG. 9). (Step A7) acquires the detection result of the casing direction detection unit 111 (Step A8), acquires the detection result of the drawer detection unit 112 (Step A9), and detects the casing direction detection unit 111 and the drawer detection unit. On the basis of the detection result 112, it is a mouse style, that is, as shown in FIG. 8 (2), the drawer portion 113 is pulled out of the casing and the rear side of the casing is raised (mouse (Style A) is checked (step A10).

  The case orientation detection unit 111 detects that the case is sandwiched so that the back side of the case is wrapped with a palm, for example, the posture of the case (touch display unit 108 or mouse sensor 109) is downward. If the drawing detection unit 112 detects that the drawing unit 113 is pulled out from the housing (mouse style) (YES in step A10), the mode shifts to the pointing device (mouse) mode. Then (step A11), the process proceeds to the flow of FIG. 10, and processing related to the monitor link object (processing for the terminal object corresponding to the monitor link object) is executed. If it is not the mouse style (NO in step A10), the mode is shifted to the touch input device mode (step A12), and the process proceeds to the flow of FIG. 11 to process the monitor link object (terminal object corresponding to the monitor link object). Process).

  When the mobile device 100 shifts to the pointing device (mouse) mode as described above, the mobile phone 100 performs the following operation. First, when a moving operation for moving the casing in the direction desired by the user is performed (YES in step A13 in FIG. 10), the cursor movement signal indicates the moving direction and moving speed of the casing detected by the mouse sensor 109. As described above, after transmitting to the output monitor (step A14) and receiving and acquiring the new cursor display position updated in accordance with the cursor movement process on the output monitor side (step A15), Return to Step A13.

  When the monitor link button MB is operated (YES in step A16), the display position management table DT is referred to and it is checked whether the current cursor display position matches the display position of the monitor link object (step S16). A17). As a result, if the two do not match (NO in step A17), the mouse style has been released, that is, the drawer 113 is pulled out of the housing as shown in FIG. (Step A18). Here, when the mouse style is released (NO in step A18), a link end signal is transmitted to the output monitor to release the pointing device mode (mouse mode) (step A19). ~ Exit from the flow of Fig. 11.

  If the cursor position coincides with the monitor link object position when the monitor link button MB is operated (YES in step A17), that is, if the monitor link object is selected by moving the cursor, the monitor Processing related to the link object (processing for the terminal object corresponding to the monitor link object) is performed. First, referring to the display position management table DT, a terminal object corresponding to the monitor link object is selected (step A20), and processing (for example, processing according to an icon) for the terminal object is executed (step A21). ). The processing result is acquired as a new monitor link object (step A22) and transmitted to the output monitor (step A23).

  Then, after receiving the display position of the monitor link object and the cursor on the output monitor from the output monitor (step A24), the display position of the terminal object and the display position of the monitor link object are displayed in association with each other. Store in the position management table DT (step A25). Then, the process proceeds to step A18 described above to check whether the mouse style is released. On the other hand, when the mouse style is released (YES in step A18), a link end signal is transmitted to the output monitor (step A19), and the flow of FIGS. 9 to 11 is exited.

  On the other hand, when the mode is changed to the touch input device mode, the mobile phone 100 performs the following operation. First, the presence / absence of a touch operation on the touch display unit 108 is checked (step A26 in FIG. 11), or whether a link end instruction is received (step A37). When an operation for instructing link release is performed in the touch input device mode (YES in step A37), a link end signal is transmitted to the output monitor (step A38), and then the flow of FIGS. 9 to 11 is exited. Now, when a touch action operation is any of tap operation, double tap operation, hold operation, pitch-in / pitch-out operation, flick operation, drag operation, etc. (YES in step A26), the touch position and action content Is detected (step A27), and it is checked whether the touch position is the display position of the terminal object (step A28).

  If the touch position is the display position of the terminal object (YES in step A28), a command corresponding to the action content is generated (step A29). For example, if the action content is one of tap, double tap, hold, pitch-in / pitch-out, flick, and drag, a command corresponding to that is generated. Then, the terminal object is processed according to the above-described generation command (step A30). For example, an object selection process, an object movement process, an object enlargement / reduction process, and the like are executed. After the processing result is transmitted to the output monitor as a new terminal object (step A31), when the display position of the new terminal object is received from the output monitor (step A32), the display of the monitor link object is displayed. The position and the display position of the terminal object are associated with each other and stored in the display position management table DT (step A33). Thereafter, the process returns to step A26 described above.

  In FIG. 13, a plurality of (three) icons are displayed as terminal objects on the terminal screen, and the plurality of icons are displayed as monitor link objects on the output monitor (in the illustrated example, the contents of the terminal screen are fully displayed). It is the figure which showed the example of a display when the icon position desired as a touch action on a terminal screen is touched in the state which is carrying out. Here, in the illustrated example, when the position of the icon “a” is touched, the mobile phone 100 executes icon processing corresponding to the action content (tap) and displays the processing result on the terminal screen. Since a new monitor link object is transmitted to the output monitor, a new monitor link object is displayed on the monitor screen.

  On the other hand, if the touch position does not match the display position of the terminal object (NO in step A28), a command corresponding to the action content is generated (step A34), and then the entire terminal screen is displayed according to the generated command. As the control process, for example, a scroll process, an enlargement / reduction process, and the like are executed (step A35). Then, the contents of the terminal screen updated according to the processing result of the screen control are transmitted to the output monitor as a new monitor link object (step A36). Then, after receiving and acquiring the display position of the new monitor link object from the output monitor (step A32), the display position management table DT associates the display position of the monitor link object with the display position of the terminal object. (Step A33). Thereafter, the process returns to step A26 described above.

  FIG. 14 shows a pitch for instructing image enlargement as a touch action on the terminal screen in a state where an image is displayed as a terminal object on the terminal screen and this image is also displayed as a monitor link object on the output monitor. FIG. 10 is a diagram illustrating a display example when an out touch operation is performed. Here, in the illustrated example, when a touch-out operation for pitch-out is performed, the mobile phone 1 executes an enlargement process corresponding to the action content (pitch-out), displays the processing result on the terminal screen, and outputs it. Since a new monitor link object is transmitted to the monitor, a new monitor link object is displayed on the monitor screen.

  As described above, in the present embodiment, the mobile phone 100 includes the touch display unit 108 with a touch panel provided on the surface side of the housing, the drawer portion 113 that is housed in the housing so as to be able to be pulled out, and the drawer portion 113. The mouse is equipped with a mouse sensor 109, and the drawer 113 is pulled out of the housing so that the back side of the housing is sandwiched between palms so that the front side of the housing is oriented on a plane. In this case, since the drawer portion 113 is attached to the housing so that a space is formed by the step between the leading end surface of the drawer portion 113 and the display surface of the touch display portion 108, the cellular phone is used as an operation device for the display device 200. When 100 is used, it can be used with ease of use and has both operability and convenience.

  That is, in the mouse style that is sandwiched so that the back side of the chassis is wrapped with a palm with the front side of the chassis facing the plane, the touch display unit 108 is hidden on the plane side. The finger can be operated without touching the touch display surface, and the one end side of the casing is raised by the pulling out of the drawer 113, that is, the entire casing rises diagonally. It becomes a shape like a state where the back side is raised, and it becomes a form that is easy to hold (use) as a mouse.

  In the mouse style with the front side of the housing facing the plane, the drawer 113 is pulled obliquely from the vicinity of one end on the front side of the housing, so that the leading end of the drawer 113 and the surface side of the housing are separated. Since the end portion is in contact with a flat surface, contact with a flat surface such as a desk is reduced, and a smooth movement operation can be performed.

  Based on the detection result of the drawer detection unit 112, it is determined whether the mouse style is selected. Therefore, the drawer operation of the drawer unit 113 can be performed without a separate operation button for instructing switching to the mouse mode. It can replace the operation to switch to mouse mode.

  Based on the detection results of the casing orientation detection unit 111 and the drawer detection unit 112, it is determined whether the mouse style is selected. Therefore, it is possible to more accurately determine the mouse style.

  In response to the monitor link button operation, the operation of determining whether the mouse style is selected is started, so that the user's intention can be reflected.

  If there is a display device 200 capable of communication connection in the vicinity of the mobile phone 100, the display device 200 is specified as an output monitor. Therefore, it is assumed that the user who owns the mobile phone 100 is located. However, if there is a display device 200 capable of communication connection in the vicinity, there is no need to be aware of communication connection with the display device 200 or switching to the operation device mode.

  The mobile phone 100 specifies the display device 200 that can be connected to the neighborhood as an output monitor, selects the type of operation device according to the current style, and displays the display content of the display device 200 according to the selected device type. This allows you to select the type of operation device without performing any special operations, and it can function as a device that matches your style, improving usability and improving operability and convenience. It becomes possible to combine.

  Since the mobile phone 100 selects a device type that functions as a pointing device (mouse) for the display device 200 or selects a device type that functions as a touch input device that uses the touch display unit 108, this Various kinds of operations can be realized by two kinds of device functions.

  When a device type that functions as a pointing device is selected, the cursor displayed on the display device 200 can be moved only by moving the housing.

  When the device type that functions as the touch input device is selected, the display content of the display device 200 can be changed according to the touch action on the touch display unit 108.

  Since the casing style is determined in accordance with the positional relationship of the sensor unit (mouse sensor 109) for the operation device provided in the casing, accurate determination is possible.

  In the above-described embodiment, in the touch input device mode, a processing result obtained by executing, for example, an object selection process, an object movement process, an object enlargement / reduction process, etc., is output as a new terminal object in response to a touch action operation. Although it is transmitted to the monitor, the touch action content may be transmitted to the output monitor together with the touch position.

  In the above-described embodiment, Bluetooth communication is used as short-range communication between the mobile phone 100 and the display device 200, but infrared communication or the like may be used. Further, although the information distribution apparatus 5 is connected via the Internet 4, it may be connected via a local communication network.

  In the embodiment described above, when the mobile phone 100 is used as a pointing device (mouse), as shown in FIG. 8 (2), the surface side of the housing faces downward, that is, the touch display unit 108 is flat (for example, Although the case where it mounts toward a (desk surface) was shown, the plane is not limited to a horizontal plane and may be a vertical plane (for example, a wall surface). In the embodiment described above, the mobile phone 100 is used as a mouse function, but may be used as a motion sensor function (space movement operation device).

  Moreover, although the case where it applied to a mobile telephone was illustrated in embodiment mentioned above, not only this but PDA, a digital camera, a music player, etc. may be sufficient.

  Further, the “machine” and “unit” shown in the above-described embodiment may be separated into a plurality of cases according to function, and are not limited to a single case. In addition, each step described in the above-described flowchart is not limited to time-series processing, and a plurality of steps may be processed in parallel or separately.

The embodiment of the present invention has been described above. However, the present invention is not limited to this, and includes the invention described in the claims and the equivalent scope thereof.
Hereinafter, the invention described in the claims of the present application will be appended.
(Appendix)
(Claim 1)
The invention described in claim 1
A mobile terminal device that functions as a pointing device for the display device via communication means that performs data communication directly or indirectly with a display device as an external device,
A touch display unit with a touch panel provided on the surface side of the casing constituting the terminal device;
A drawer portion housed in the housing so as to be capable of being pulled out;
A movement sensor mounted on the drawer and detecting movement of the terminal device;
With
A style that functions as a pointing device for the display device is a style in which the drawer portion is pulled out from the housing so as to wrap the back side of the housing with a palm, and the style of the pointing device is the surface side of the housing. Is attached to the housing so that a space is formed by a step between the leading end surface of the drawer portion and the display surface of the touch display portion.
This is a portable terminal device that is characterized as described above.
(Claim 2)
The invention according to claim 2 is the portable terminal device according to claim 1,
When the surface side of the housing is oriented on a plane in the style of the pointing device, the leading portion of the leading portion is pulled out in an oblique direction from the vicinity of one end on the surface side of the housing. And the other end on the surface side of the housing are in contact with the plane,
This is a portable terminal device that is characterized as described above.
(Claim 3)
The invention according to claim 3 is the portable terminal device according to claim 1 or 2,
A drawer detecting means for detecting whether the drawer portion housed in the housing is pulled out;
Style discrimination means for discriminating whether or not the terminal device is a style that functions as a pointing device for the display device according to a detection result of the drawer detection means;
Function control means for controlling the terminal device to function as a pointing device for the display device when it is determined by the style determination means that the style functions as a pointing device;
In addition,
This is a portable terminal device that is characterized as described above.
(Claim 4)
According to a fourth aspect of the present invention, in the mobile terminal device according to the third aspect,
It further comprises posture detecting means for detecting the posture of the terminal device,
The style determination means determines whether the terminal device is a style that functions as a pointing device for the display device according to detection results of the drawer detection means and posture detection means.
This is a portable terminal device that is characterized as described above.
(Claim 5)
The invention according to claim 5 is the portable terminal device according to claim 3 or 4,
The style determination means starts a determination operation as to whether or not the style functions as a pointing device for the display device according to a predetermined user operation.
A portable terminal device characterized by that.
It is.
(Claim 6)
The invention according to claim 6 is the portable terminal device according to any one of claims 1 to 5,
Connection determining means for determining whether or not the display device capable of communication connection exists in the vicinity of the terminal device;
A specifying means for specifying the display device as an output monitor when the determining device determines that the display device exists;
Display control means for controlling the display content of the display device specified as the output monitor by the specifying means;
Further comprising
This is a portable terminal device that is characterized as described above.
(Claim 7)
The invention described in claim 7
A mobile terminal device that functions as an operation device for the display device via communication means for performing data communication directly or indirectly with a display device as an external device,
Connection determining means for determining whether or not the display device capable of communication connection exists in the vicinity of the terminal device in accordance with a predetermined user operation;
When it is determined by the connection determining means that a display device exists in the vicinity, a specifying means for specifying the display device as an output monitor of the terminal device;
Style discriminating means for discriminating the style of the terminal device;
Device type selection means for selecting the type of operation device according to the style determined by the style determination means;
Display control means for controlling the display content of the display device according to the device type selected by the device type selection means;
It is a portable terminal device provided with.
(Claim 8)
The invention according to claim 8 is the portable terminal device according to claim 7,
A touch display unit with a touch panel;
The device type selection means selects a device type that functions as a pointing device for the display device or a device type that functions as a touch input device that uses the touch display unit.
This is a portable terminal device that is characterized as described above.
(Claim 9)
The invention according to claim 9 is the portable terminal device according to claim 7 or claim 8,
The display control means includes a movement operation signal detected by the movement sensor or the operation signal when the terminal apparatus is moved in a state where the device type that functions as a pointing device is selected by the device type selection means. A processing result corresponding to the control unit is transmitted to the display unit via the communication unit, thereby controlling the movement of the cursor displayed on the display device.
This is a portable terminal device that is characterized as described above.
(Claim 10)
The invention according to claim 10 is the portable terminal device according to claim 8,
The display control means, when a device type that functions as a touch input device is selected by the device type selection means, when a touch action is performed on the touch display unit, the touch operation signal or the operation signal Control to change the display content of the display device to the content corresponding to the touch action by transmitting the execution result of the processing according to the control device to the display device specified as the output monitor via the communication means. Do,
The mobile terminal device according to claim 8, wherein the mobile terminal device is configured as described above.
(Claim 11)
The invention according to claim 11 is the portable terminal device according to any one of claims 7 to 10,
The style determining means determines the style of the casing according to the positional relationship of the sensor unit for the operation device provided in the casing.
This is a portable terminal device that is characterized as described above.

DESCRIPTION OF SYMBOLS 100 Mobile phone 200 Display apparatus 200A Television receiver 101 CPU
DESCRIPTION OF SYMBOLS 103 Memory | storage part 107 Operation part 108 Touch display part 109 Mouse sensor 111 Case direction detection part 112 Draw-out detection part DT Display position management table

The present invention relates to a terminal device , a terminal control method, and a program.

Claim 1, upon detection of a specific operation for instructing the short-range wireless communication with the external device, and determines whether the display information to be transmitted to the terminal device is present, if there the Rutotomoni transitions the terminal device to the external device and Pearingusu Ru mode, a first control means for controlling so that sends the display information to the external device, the specific operation or, after transition to the front Symbol mode A second control means for executing a predetermined process based on the content of the display information and controlling to transmit new display information updated with the processing result to the external device . It is characterized by that.
Claim 5 is a terminal apparatus that performs short-range wireless communication with the neighboring external device, and pairing control means for paired an equivalent terminal device as a device for the external device, there at the time of the pairing a determination unit that the state of the terminal device determines whether the mouse style Te, when it is determined that the mouse style said determination means, said movement signal based on a moving operation of the terminal device main body external device When the determination unit determines that the mouse style is not the first control unit that controls to transmit to the terminal device, the control unit controls to transmit contents based on detection of various operations on the terminal device to the external device . And a second control means.

Claims (7)

A terminal device that communicates with a nearby external device,
When a specific instruction operation for instructing communication with an external device is detected, it is determined whether or not the specific instruction operation is invalidated based on whether or not the terminal device displays display information, and the display The terminal device is shifted to the device mode for pairing with the external device on condition that the information is displayed, and the display information displayed on the terminal device is transmitted to the external device, and the terminal device First control means for controlling to transmit a cursor movement signal to the external device, the moving operation in a plane direction on a predetermined plane being an operation for moving and displaying a cursor displayed on the external device;
When the specific instruction operation is detected after the transition to the device mode, whether or not the specific instruction operation is invalidated based on whether or not the display position of the cursor at that time is the display position of the display object Second control means for controlling the display device to newly transmit to the external device the display content executed by the terminal device for the display object on the condition that the display object is at the display position. When,
A terminal device comprising: Release means for releasing the pairing with the external device in the device mode when the terminal device main body is not in a predetermined orientation after the transition to the device mode;
The terminal device according to claim 1, further comprising: A terminal device that communicates with a nearby external device,
When a specific instruction operation for instructing communication with an external device is detected, the terminal device is shifted to a device mode for pairing with the external device, and when the device mode is shifted to, the orientation of the main body of the terminal device is set. In response, a determination means for determining whether to operate the terminal device as a mouse device mode for the external device or a touch input device mode;
If the mouse device mode is determined by the determining means, a cursor movement signal that uses a movement operation in a plane direction on the predetermined plane of the terminal device as an operation for moving a cursor displayed on the external device. First control means for controlling to transmit to the external device;
When the touch input device mode is discriminated by the discriminating means, second control means for controlling to transmit display contents processed based on detection of various touch operations on the terminal device to the external device. When,
A terminal device comprising: Second discrimination means for discriminating whether or not the movement of the terminal device is a mouse style that can be detected by the sensor by holding the housing of the terminal device with the sensor provided in the terminal device facing downward When,
Further comprising
The first control means performs control as the mouse device mode when the second discrimination means discriminates the mouse style,
The second control unit performs control as the touch input device mode when the second determination unit determines that the mouse style is not used.
The terminal device according to claim 3. A second determination means for determining whether or not the mouse sensor provided in the terminal device is exposed, and the exposed mouse sensor can detect an arbitrary movement operation of the terminal device;
Further comprising
The first control means performs control as the mouse device mode when the second discrimination means discriminates the mouse style,
The second control unit performs control as the touch input device mode when the second determination unit determines that the mouse style is not used.
The terminal device according to claim 3. A program for controlling a computer of a terminal device that communicates with a nearby external device,
The computer,
When a specific instruction operation for instructing communication with an external device is detected, it is determined whether or not the specific instruction operation is invalidated based on whether or not the terminal device displays display information, and the display The terminal device is shifted to the device mode for pairing with the external device on condition that the information is displayed, and the display information displayed on the terminal device is transmitted to the external device, and the terminal device First control means for controlling a movement operation in a plane direction on a predetermined plane to transmit a cursor movement signal to the external device as an operation for moving and displaying a cursor displayed on the external device;
When the specific instruction operation is detected after the transition to the device mode, whether or not the specific instruction operation is invalidated based on whether or not the display position of the cursor at that time is the display position of the display object Second control means for controlling the display device to newly transmit to the external device the display content executed by the terminal device for the display object on the condition that the display object is at the display position. ,
A program designed to function as A program for controlling a computer of a terminal device that communicates with a nearby external device,
The computer,
When a specific instruction operation for instructing communication with an external device is detected, the terminal device is shifted to a device mode for pairing with the external device, and when the device mode is shifted to, the orientation of the main body of the terminal device is set. In response, a determination means for determining whether to operate the terminal device as a mouse device mode for the external device or a touch input device mode;
If the mouse device mode is determined by the determining means, a cursor movement signal that uses a movement operation in a plane direction on the predetermined plane of the terminal device as an operation for moving a cursor displayed on the external device. First control means for controlling to transmit to the external device,
When the touch input device mode is discriminated by the discriminating means, second control means for controlling to transmit display contents processed based on detection of various touch operations on the terminal device to the external device. ,
A program designed to function as

Images