JP2010108447A - Processing control unit, processing execution unit, information processor, control method, control program, and computer-readable recording medium with the control program recorded thereon - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processor having a main device and a sub-device, in which processing to be executed by the main device is controlled from a sub-device according to the execution state of processing in the main device. <P>SOLUTION: A sub-display device 230 with a sensor which can be operated independently of a main display device 210 is provided with: a POST code inspection part 24 which acquires a POST code generated by the main display device 210 while executing a main side BIOS program P11, and determines whether or not the acquired POST code is a predetermined value; and a main side start OS specifying part 25 which, when it is determined that the POST code is the predetermined value, specifies any one of the plurality of main side OS programs P21 to P23 which can be executed by the main display device 210 to be executed. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a process control unit, a process execution unit, an information processing apparatus, a control method, a control program, and a computer-readable recording medium that records the control program.

  In recent years, various portable information processing apparatuses such as notebook PCs (Personal Computers) have been developed and are widely used. Such an information processing apparatus is provided with various functions in order to improve its operability. As an example, a sub display device including a sub display is provided separately from the main display device including the main display, and the sub display device has a function that enables mouse pointing operation, numeric keypad input operation, menu screen display, and the like. .

  Conventional techniques related to a notebook PC equipped with such a sub display device are disclosed in Patent Documents 1 to 4.

  In Patent Document 1, when a password is set in BIOS (Basic Input Output System), the BIOS first displays the sub display device and its controller in a displayable state, and displays a password input request on the sub display device. A technique for performing control is disclosed.

  Patent Document 2 discloses a technique for securing a display area near a mouse pointer displayed on a main display device and displaying a switching menu screen handled by a sub display device in the display area.

  Patent Document 3 discloses a technique for displaying a screen image of the entire desktop including the main operation screen displayed on the main display device on the sub display device as an operation screen for instructing the position of the mouse pointer by a touch operation. .

In Patent Document 4, when a function button included in a notebook PC is operated, a menu for selecting a function is displayed on the sub display device, and when the user operates the sub display device and selects a numeric keypad from the menu, A technique for displaying an operable numeric keypad on a sub display device is disclosed.
JP 2004-5105 A (publication date: January 8, 2004) JP 2004-5212 A (publication date: January 8, 2004) JP 2004-234504 A (publication date: August 19, 2004) JP 2000-339097 A (publication date: December 8, 2000)

  However, in the conventional notebook PC, a control unit (specifically, a CPU (central processing unit)) is provided in the main display device, and the sub display device is controlled by the control unit.

  Therefore, when a problem occurs during the processing of the main display device and not only the control of the main display device but also the display control of the sub display device is impossible, the status (status) of the failure may be presented to the user. There is a problem that you can not.

  In addition, there is a problem that the sub display device cannot execute processing according to the operation status of the main display device. To give a specific example, if there is a problem in the startup process of the main display device, an OS (Operating System) for starting the main display device if the sub display device displays that fact or there is no fault in the startup process. Cannot be performed from the sub display device.

  In order to solve the above problems, each of the main display device and the sub display device includes a control unit, and the operation status of the main display device is referred to from the sub display device, or according to the operation status of the main display device. It is conceivable to control the main display device from the sub display device. Such a notebook PC is not disclosed in Patent Documents 1 to 4 described above.

  The present invention has been made in view of the above-described problems, and an object thereof is to be executed by the main apparatus in an information processing apparatus having a main apparatus and a sub apparatus according to the execution status of processing in the main apparatus. An object of the present invention is to provide a process control unit, a process execution unit, an information processing apparatus, a control method, a control program, and a computer-readable recording medium on which the control program is recorded.

  In order to solve the above-mentioned problem, the process control unit according to the present invention generates execution status information indicating the execution status while executing the first processing, and performs the second processing after the first processing. A process control unit that is communicable with a process execution unit to be executed and is operable independently of the process execution unit, and acquires the generated execution status information from the process execution unit and the acquisition When the execution status determining means for determining whether or not the executed status information is a predetermined value, and when the execution status determining means determines that the acquired execution status information is the predetermined value, the processing execution unit And an execution designating unit that designates the process execution unit to execute any one of a plurality of executable candidate processes as the second process. It is.

  Further, the control method of the processing control unit according to the present invention is a process for generating execution status information indicating the execution status while executing the first processing, and executing the second processing after the first processing. A method for controlling a processing control unit that is communicable with an execution unit and operable independently of the process execution unit, wherein the generated execution status information is acquired from the process execution unit and the acquisition is performed. The execution state determining step for determining whether or not the executed execution state information is a predetermined value; and when the acquired execution state information is determined to be the predetermined value in the execution state determination step, the processing execution unit An execution designation step that designates the process execution unit to execute any one of a plurality of candidate processes that can be executed in the process as the second process. It is characterized in that.

  According to the above configuration, it is possible to acquire the execution status information from the process execution unit and determine whether or not the acquired execution status information is a predetermined value. Further, when it is determined that the acquired execution status information is the predetermined value, any one of the plurality of candidate processes is designated as a second process executed after the first process in the process execution unit. be able to.

  Therefore, when the execution status of the first process being executed by the process execution unit is a predetermined situation, the process control unit determines one of a plurality of candidate processes that can be executed by the process execution unit as the second process. Can be specified as

  Therefore, the process control unit can control the subsequent process (second process) executed by the process execution unit according to the execution status of the preceding process (first process) in the process execution unit.

  Since the process control unit can operate independently of the process execution unit, the execution status can be changed from the process control unit without depending on the execution result of the preceding process (first process) in the process execution unit. Can be confirmed. Therefore, even if some trouble occurs during the preceding process (first process) in the process execution unit, it is possible to confirm the execution status of the process in the process execution unit from the process control unit.

  Furthermore, the process control unit according to the present invention is the above-described configuration, wherein the execution designation unit further acquires candidate process information indicating each of the plurality of candidate processes from the process execution unit. The execution specifying means may be configured to transmit information specifying any one of the acquired candidate process information to the process execution unit.

  According to the above configuration, candidate process information indicating each of the plurality of candidate processes can be acquired from the process execution unit. Then, any one of the candidate processes indicated by the acquired candidate process information can be transmitted to the process execution unit.

  Therefore, the process control unit can designate the second process to be executed by the process execution unit by transmitting any one of the candidate process information acquired from the process execution unit to the process execution unit.

  Therefore, the process control unit has an effect that the second process can be designated without knowing in advance the candidate process that can be designated as the second process.

  Furthermore, the processing control unit according to the present invention is configured so that, in the above configuration, the execution specifying unit further presents the candidate processing information acquired by the candidate processing information acquiring unit to the user, It is good also as a structure provided with the candidate process selection means which makes a user select any one, and the said execution designation | designated means transmit the information which designates the said selected candidate process information to the said process execution unit.

  According to the above configuration, the candidate process information can be presented to the user, and the user can select any one of the presented candidate process information. Then, the selected candidate process information can be transmitted to the process execution unit.

  Thus, the user can cause the process execution unit to execute the candidate process selected by the user.

  Therefore, it is possible to cause the process execution unit to execute the process selected by the user among the plurality of candidate processes.

  Furthermore, the process control unit according to the present invention is configured so that, in the above configuration, the execution designation unit is configured to execute the predetermined candidate process when the candidate process information presented by the candidate process selection unit is not selected within a predetermined time. Information specifying the information may be transmitted to the processing execution unit.

  According to the above configuration, when the candidate process information is not selected by the user within a predetermined time, the predetermined candidate process information can be transmitted to the process execution unit.

  Therefore, even if the candidate process information is not selected by the user, the process execution unit can execute the second process.

  Therefore, there is an effect that it is possible to prevent the process execution unit from waiting for the start of execution of the second process indefinitely.

Furthermore, the process control unit according to the present invention has the above-described configuration, wherein the process execution unit executes any one of a plurality of operating systems executable by the own unit after executing the BIOS as the first process. Which is executed as the second process,
The execution designation unit may be configured to designate any one of the operating systems as the second process.

  According to the above configuration, the process execution unit further executes the operating system as the second process after executing the BIOS as the first process, and the process control unit performs the operation system as the second process. Can be specified.

  Therefore, when the execution status information generated by the BIOS in the process execution unit is a predetermined value, the operating system to be started up in the process execution unit can be specified from the process control unit.

  In this case, the execution status information is specifically a POST (Power On Self Test) code output by executing the BIOS, and its value indicates the normal end of the POST process. In this case, the process control unit designates an operating system to be activated by the process execution unit.

  In order to solve the above-mentioned problem, the process execution unit according to the present invention generates execution status information indicating the execution status while executing the first process, and after the first process, A process execution unit that communicates with the process control unit and executes any one of a plurality of executable candidate processes as the second process, and the second process from the process control unit that can read the execution status information And an execution control means for executing the designated second process after the first process.

  The method for controlling a process execution unit according to the present invention generates execution status information indicating the execution status while executing the first process, and can be executed by the own unit after the first process. A method for controlling a process execution unit capable of communicating with a process control unit, executing any one of a plurality of candidate processes as a second process, wherein the second process is performed from a process control unit capable of reading the execution status information. And an execution control step for executing the designated second process after the first process.

  According to the above configuration, after the first process is executed and the predetermined execution status information is generated, the second process specified by the process control unit can be executed.

  Therefore, the process execution unit can execute the second process in response to an instruction from the process control unit when the execution status of the first process reaches a predetermined status.

  Therefore, the process execution unit can perform the subsequent process (second process) designated by the process control unit in accordance with the execution status of the previous process (first process).

  Since the process control unit can operate independently of the process execution unit, the execution status can be changed from the process control unit without depending on the execution result of the preceding process (first process) in the process execution unit. Can be confirmed. Therefore, even if some trouble occurs during the preceding process (first process) in the process execution unit, it is possible to confirm the execution status of the process in the process execution unit from the process control unit.

  Furthermore, in the above configuration, the process execution unit according to the present invention is configured so that, when the process control unit does not designate the second process within a predetermined time after the execution of the first process, the execution control means It is good also as a structure which performs the said 2nd process.

  According to the above configuration, when the process control unit does not designate the second process within a predetermined time, the predetermined second process can be executed.

  Therefore, the process execution unit can execute the second process even if the second process is not designated after the execution of the first process.

  Therefore, there is an effect that it is possible to prevent the process execution unit from waiting for the start of execution of the second process indefinitely.

  The information processing apparatus according to the present invention is characterized in that the process control unit and the process execution unit are connected by wire.

  The information processing apparatus has the same effects as the process control unit and the process execution unit.

  The process control unit and the process execution unit may be realized by a computer. In this case, the process control unit and the process execution unit are realized by a computer by operating the computer as each of the means. The control program for the display device and the computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention.

  As described above, the processing control unit according to the present invention generates the execution status information indicating the execution status while executing the first processing, and executes the second processing after the first processing. A process control unit that is communicable with the execution unit and operable independently of the process execution unit, and obtains the generated execution status information from the process execution unit, and the acquired execution status Execution status determining means for determining whether or not the information is a predetermined value, and when the execution status determining means determines that the acquired execution status information is the predetermined value, the process execution unit can execute the information. Execution designation means for designating the process execution unit to execute any one of a plurality of candidate processes as the second process.

  Further, the control method of the processing control unit according to the present invention is a process for generating execution status information indicating the execution status while executing the first processing, and executing the second processing after the first processing. A method for controlling a processing control unit that is communicable with an execution unit and operable independently of the process execution unit, wherein the generated execution status information is acquired from the process execution unit and the acquisition is performed. The execution state determining step for determining whether or not the executed execution state information is a predetermined value; and when the acquired execution state information is determined to be the predetermined value in the execution state determination step, the processing execution unit An execution designation step that designates the process execution unit to execute any one of a plurality of candidate processes that can be executed in the process as the second process. They are out.

  Therefore, the process control unit can designate the second process to be executed by the process execution unit when the execution status of the first process being executed by the process execution unit is a predetermined situation.

  Therefore, the process control unit can control the subsequent process (second process) executed by the process execution unit according to the execution status of the preceding process (first process) in the process execution unit.

  Further, the process execution unit according to the present invention generates execution status information indicating the execution status while executing the first process, and a plurality of candidates that can be executed by the own unit after the first process. A process execution unit that executes any one of the processes as a second process and is communicable with the process control unit, and receives an instruction to execute the second process from the process control unit that can read the execution status information. And an execution control means for executing the designated second process after the first process.

  The method for controlling a process execution unit according to the present invention generates execution status information indicating the execution status while executing the first process, and can be executed by the own unit after the first process. A method for controlling a process execution unit capable of communicating with a process control unit, executing any one of a plurality of candidate processes as a second process, wherein the second process is performed from a process control unit capable of reading the execution status information. And an execution control step for executing the designated second process after the first process.

  Therefore, the process execution unit can execute the second process in response to an instruction from the process control unit when the execution status of the first process reaches a predetermined status.

  Therefore, the process execution unit can perform the subsequent process (second process) designated by the process control unit in accordance with the execution status of the previous process (first process).

  An embodiment of the present invention will be described below with reference to FIGS.

(Summary of Invention)
The outline of the present invention is that when the execution state of the first process executed by the main display device (process execution unit) 210 is a predetermined state, the second process that can be executed by the main display device 210 is displayed as a main display. It is specified from the sub display device with a sensor (processing control unit) 230 that can operate independently of the device. In the present embodiment, an OS (Operating System) that can be executed by the main display device 210 is executed particularly when the execution status of the basic input output system (BIOS) immediately after the main display device 210 is turned on is a predetermined status. A case where the activation process is designated from the sub display device with sensor 230 will be described as an example. Note that the main display device 210 is installed with a plurality of OSs that can be executed by the device itself, and any one of them is activated (in a multi-boot state).

  First, an outline of the two-screen display device (information processing device) 200 in which the sensor-equipped sub-display device 230 is mounted on the main display device 210 will be described, and then the main display device 210 and the sensor-equipped sub-display device 230 will be described. .

(Outline of the two-screen display device)
The two-screen display device 200 includes a main display device 210 including a liquid crystal panel 311 capable of displaying data, and a sensor built-in liquid crystal panel 301 (described later) capable of detecting an image of a nearby object in addition to displaying data. This is a device having a sub-display device 230 with a sensor.

  The main display device 210 and the sensor-equipped sub display device 230 are detachable. When the sensor-equipped sub-display device 230 is mounted on the main display device 210, it becomes possible to perform wired communication with each other, and by transmitting an “inter-device command” to the partner device, the partner device Can be controlled. Details of the inter-device command will be described later.

  Each of the main display device 210 and the sensor-equipped sub display device 230 includes a control unit that controls the device itself. Therefore, in a state where the main display device 210 and the sensor-equipped sub display device 230 are separated from each other, each can operate independently.

  A specific example of the two-screen display device 200 will be described with reference to FIG. FIG. 2A is a schematic diagram illustrating a specific example of the two-screen display device 200. As shown in the figure, in the present embodiment, a notebook PC (Personal Computer) including a sensor-equipped sub display device 230 instead of a conventional touch pad is assumed as the two-screen display device 200. In this case, a portion other than the sensor-equipped sub display device 230 (a portion including the display unit 310 and the keyboard 920) corresponds to the main display device 210. The state shown in FIG. 2A is a state where the sub display device with sensor 230 is mounted inside the main display device 210. The portion corresponding to the touch pad of the main display device 210 is transparent, and the user can operate the sensor built-in liquid crystal panel 301 of the sub display device with sensor 230 mounted inside.

  FIG. 2B is a schematic diagram showing a state where the sensor-equipped sub display device 230 is taken out from the main display device 210 and separated in the notebook PC shown in FIG. As described above, the main display device 210 and the sensor-equipped sub-display device 230 can each operate alone. In this state, the main display device 210 can operate as a notebook PC having no touch pad function. It is. The sensor-equipped sub-display device 230 can operate as, for example, a remote controller including a sensor-equipped liquid crystal panel 301 or a PDA (Personal Digital Assistant).

(Outline of LCD panel with built-in sensor)
As described above, the sensor built-in liquid crystal panel 301 included in the sensor-equipped sub-display device 230 is a liquid crystal panel capable of detecting an image of a nearby object in addition to displaying data. Here, the image detection of the object is, for example, detection of a position pointed (touched) by the user with a finger or a pen, or reading (scanning) of an image of a printed material. The device used for display is not limited to a liquid crystal panel, and may be an organic EL (Electro Luminescence) panel or the like.

  The structure of the sensor built-in liquid crystal panel 301 will be described with reference to FIG. FIG. 3 is a diagram schematically showing a cross section of the sensor built-in liquid crystal panel 301. The sensor built-in liquid crystal panel 301 described here is an example, and any structure can be used as long as the display surface and the reading surface are shared.

  As shown in the figure, the sensor built-in liquid crystal panel 301 includes an active matrix substrate 51A disposed on the back surface side and a counter substrate 51B disposed on the front surface side, and has a structure in which a liquid crystal layer 52 is sandwiched between these substrates. is doing. The active matrix substrate 51A is provided with a pixel electrode 56, a data signal line 57, an optical sensor circuit 32 (not shown), an alignment film 58, a polarizing plate 59, and the like. The counter substrate 51B is provided with color filters 53r (red), 53g (green), 53b (blue), a light shielding film 54, a counter electrode 55, an alignment film 58, a polarizing plate 59, and the like. In addition, a backlight 307 is provided on the back surface of the sensor built-in liquid crystal panel 301.

  The photodiode 6 included in the photosensor circuit 32 is provided in the vicinity of the pixel electrode 56 provided with the blue color filter 53b, but is not limited to this configuration. It may be provided in the vicinity of the pixel electrode 56 provided with the red color filter 53r, or may be provided in the vicinity of the pixel electrode 56 provided with the green color filter 53g.

  Next, referring to FIGS. 4A and 4B, a case where the user touches the sensor built-in liquid crystal panel 301 with a finger or a pen is taken as an example, and the touched position is detected. The types of methods will be described.

  FIG. 4A is a schematic diagram showing how the position touched by the user is detected by detecting the reflected image. When the light 63 is emitted from the backlight 307, the optical sensor circuit 32 including the photodiode 6 detects the light 63 reflected by the object 64 such as a finger. Thereby, the reflected image of the target object 64 can be detected. Thus, the sensor built-in liquid crystal panel 301 can detect the touched position by detecting the reflected image.

  FIG. 4B is a schematic diagram showing a state in which a position touched by the user is detected by detecting a shadow image. As shown in FIG. 4B, the optical sensor circuit 32 including the photodiode 6 detects the external light 61 transmitted through the counter substrate 51B and the like. However, when there is an object 62 such as a pen, the incident of the external light 61 is hindered, so that the amount of light detected by the optical sensor circuit 32 is reduced. Thereby, a shadow image of the object 62 can be detected. Thus, the sensor built-in liquid crystal panel 301 can also detect a touched position by detecting a shadow image.

  As described above, the photodiode 6 may detect reflected light (shadow image) of the light emitted from the backlight 307 or may detect a shadow image caused by external light. Further, the two types of detection methods may be used in combination to detect both a shadow image and a reflected image at the same time.

(Main components of main display device and sub display device with sensor)
Next, the main configuration of the main display device 210 and the sensor-equipped sub display device 230 will be described with reference to FIG. FIG. 5 is a block diagram illustrating a main configuration of the main display device 210 and the sub display device with sensor 230.

(Main components of the sub display device with sensor)
First, the configuration of the main part of the sensor-equipped sub display device 230 will be described. As illustrated, the sensor-equipped sub-display device 230 includes a display / light sensor unit 330, a display / light sensor circuit control unit 630, a data processing unit 730, a sub-side main control unit 830, a storage unit 931, a primary storage unit 932, A communication control unit 933, a wired communication unit 934, a short-range wireless communication unit 935, an electric field strength detection unit 936, a battery 937, and a power supply switching unit 938 are provided.

  The display / light sensor unit 330 is a so-called liquid crystal display device with a built-in light sensor. The display / light sensor unit 330 includes a sensor built-in liquid crystal panel 301, a backlight 307, and a peripheral circuit 309 for driving them. A detailed configuration of the display / light sensor unit 330 will be described later.

  Next, the display / light sensor circuit control unit 630 has a function as a device driver that controls the peripheral circuit 309 of the display / light sensor unit 330. A detailed configuration of the display / light sensor circuit control unit 630 will be described later.

  Next, the data processing unit 730 has a function as middleware that gives an instruction to the display / light sensor circuit control unit 630 based on the “in-sub display device command” received from the sub-side main control unit 830. . A detailed configuration of the data processing unit 730 will be described later. Details of the sub display device commands will also be described later.

  Next, the sub-side main control unit 830 controls the operation of each unit included in the sensor-equipped sub display device 230. The sub-side main control unit 830 reads various programs stored in the storage unit 931, controls each unit of the sensor-equipped sub display device 230, and implements various functions included in the sensor-equipped sub display device 230.

  The sub-side main control unit 830 causes the data processing unit 730 to display the display data on the sensor built-in liquid crystal panel 301 and to scan the object in the sensor built-in liquid crystal panel 301. Send commands and display data. When “data type” is designated for the command within the sub display device, at least one of the whole image data, the partial image data, and the coordinate data is sent as a response to the command within the sub display device. Receive from 730.

  In addition, the sub-side main control unit 830 receives control from the main display device 210. Specifically, the inter-device command transmitted from the main-side main control unit 810 of the main display device 210 is received, and one of the following (1) to (3) is selected according to the type of the received inter-device command. The process shown is executed.

  First, (1) if the received inter-device command is a “scan command” (details will be described later) that designates scanning of an object located in the vicinity of the display / light sensor unit 330, the sub-side The main control unit 830 converts the received scan command into a sub display device command, and transmits the converted sub display device command to the data processing unit 730. The above conversion is performed by removing a “0th” field (described later) from the scan command.

  The sub-side main control unit 830 receives at least one of the whole image data, the partial image data, and the coordinate data from the data processing unit 730 as a response to the command within the sub-display device after the conversion. The received data is transmitted to the main main control unit 810.

  Next, (2) when the received inter-device command is a “display command” (details will be described later) for designating display / light sensor unit 330 to display image data such as a photograph or an icon. The sub-side main control unit 830 generates display data based on the information specified in the display command, and transmits the generated display data to the data processing unit 730.

  Next, (3) when the received inter-device command is a “processing command” (details will be described later) that designates execution of a predetermined process, the sub-side main control unit 830 is designated as the processing command. The predetermined processing is executed.

  The predetermined processing executed by the sub-side main control unit 830 includes, for example, character recognition processing and image processing for image data obtained from the data processing unit 730, and communication using the communication control unit 933 of the sensor-equipped sub display device 230. This is an execution process of various applications, middleware, and the like included in the sensor-equipped sub display device 230 such as a connection process and a file transfer process.

  Further, the sub-side main control unit 830 instructs the main display device 210 to execute a predetermined process. Specifically, an inter-device command as a processing command is transmitted to the main side main control unit 810 of the main display device 210. The predetermined processing executed by the main main control unit 810 includes, for example, OS (Operating System) startup processing of the main display device 210, communication connection processing using the communication control unit 913 of the main display device 210, and file transfer processing. And execution processing of various applications, middleware, and the like that the main display device 210 has.

  The display / light sensor circuit control unit 630, the data processing unit 730, and the sub-side main control unit 830 can be configured by a CPU (Central Processing Unit), a memory, and the like, respectively. The data processing unit 730 may be configured by a circuit such as an ASIC (application specific integrate circuit).

  Next, the storage unit 931 reads (1) the control program for each unit, (2) the OS program, (3) the application program, and (4) the program executed by the sub-side main control unit 830. Records various data. The storage unit 931 is configured by a nonvolatile storage device such as a flash memory.

  The primary storage unit 932 is configured by a volatile storage device such as a RAM (Random Access Memory), and temporarily holds data in the process in which the sub-side main control unit 830 executes the various programs described above. Used as a work area.

  Next, the communication control unit 933 allows the sensor-equipped sub display device 230 to communicate with either the wired communication unit 934 for communicating with the main display device 210, and the main display device 210 or an external device (not shown). The short-range wireless communication unit 935 is controlled.

  That is, the communication control unit 933 transmits an inter-device command and various data transmitted from the main display device 210 via either the wired communication unit 934 or the short-range wireless communication unit 935 to the sub-side main control unit 830. . Also, the communication control unit 933 sends inter-device commands and various data transmitted from the sub-side main control unit 830 in response to an instruction from the sub-side main control unit 830, and the wired communication unit 934 and the short-range wireless communication unit 935. Is transmitted to the main display device 210.

  Further, the communication control unit 933 exchanges various data with an external device via the short-range wireless communication unit 935.

  The wired communication unit 934 can be connected to the wired communication unit 914 of the main display device 210. In the present embodiment, it is assumed that the wired communication unit 934 and the wired communication unit 914 of the main display device 210 are connected by USB (universal serial bus).

  At the time of connection, the wired communication unit 934 can transmit and receive signals to and from the wired communication unit 914 of the main display device 210 using the communication line, and the wired communication of the main display device 210 using the power line. Power can be supplied from the unit 914. When the power supply is received from the wired communication unit 914 of the main display device 210, the wired communication unit 934 supplies power to the power supply switching unit 938.

  The short-range wireless communication unit 935 can communicate with each other using a wireless communication path formed with the short-range wireless communication unit 915 of the main display device 210 and with an external device. Yes. In the present embodiment, it is assumed that Bluetooth (registered trademark) communication (hereinafter simply referred to as Bluetooth) communication is applied to the wireless communication path. That is, the short-range wireless communication unit 935 is a communication unit that can transmit and receive signals by Bluetooth communication. Note that the wireless communication path is not particularly limited as long as it can transmit and receive signals, and IEEE802.11 wireless, ZigBee (registered trademark), or the like may be applied.

  Next, the electric field strength detection unit 936 detects the strength of the signal received by the short-range wireless communication unit 935 (RSSI (Received Signal Strength Indication)) in response to an instruction from the sub-side main control unit 830. To do. Then, the detection result is transmitted to the sub-side main control unit 830.

  Next, when the wired communication unit 934 is supplied with power from the wired communication unit 914 of the main display device 210, the power supply switching unit 938 supplies the power to the respective units included in the sensor-equipped sub display device 230 from the main display device 210. Supply the power that is. On the other hand, when the wired communication unit 934 does not receive power from the wired communication unit 914 of the main display device 210, the power supply switching unit 938 supplies the power of the battery 937 to each unit included in the sensor-equipped sub display device 230. .

(Main display unit configuration)
Next, a configuration of main parts of the main display device 210 will be described. As illustrated, the main display device 210 includes a display unit 310, a display circuit control unit 610, a display data processing unit 710, a main side main control unit 810, a storage unit 911, a primary storage unit 912, a communication control unit 913, and wired communication. A unit 914, a short-range wireless communication unit 915, a wired / wireless network communication unit 916, a power source 917, an operation unit 919, an audio output unit 924, and an audio input unit 925.

  The display unit 310 is for displaying data, and includes a liquid crystal panel 311, a backlight 317, and a peripheral circuit 319 for driving them. The display circuit control unit 610 has a function as a device driver that controls the peripheral circuit 319. The display data processing unit 710 has a function as middleware that gives an instruction to the display circuit control unit 610 based on the display data received from the main main control unit 810. The display unit 310, the display circuit control unit 610, and the display data processing unit 710 can be configured using a generally known technique.

  Next, the main main control unit 810 controls the operation of each unit included in the main display device 210. The main main control unit 810 reads various programs stored in the storage unit 911, controls each unit of the main display device 210, and realizes various functions included in the main display device 210.

  In addition, the main-side main control unit 810 transmits an inter-device command to the sub-side main control unit 830 of the sensor-equipped sub display device 230, and causes the sensor-equipped sub display device 230 to execute predetermined processing. When “data type” (described later) is specified for a scan command that is one of the inter-device commands, at least one of whole image data, partial image data, and coordinate data is returned as a response to the command. Is received from the sub-side main control unit 830.

  The main main control unit 810 receives a processing command that is one of the inter-device commands from the sub-side main control unit 830 of the sensor-equipped sub display device 230, and performs a predetermined process specified in the received inter-device command. (OS startup processing, execution processing of various applications, middleware, etc.) is performed.

  The display circuit control unit 610, the display data processing unit 710, and the main side main control unit 810 can each be configured with a CPU, a memory, and the like.

  Next, the storage unit 911 reads (1) the control program for each unit, (2) the OS program, (3) the application program, and (4) the program executed by the main-side main control unit 810. Records various data. The storage unit 911 is configured by a nonvolatile storage device such as a flash memory.

  The primary storage unit 912 is configured by a volatile storage device such as a RAM, and serves as a work area for temporarily storing data in the course of executing the above-described various programs by the main main control unit 810. used.

  The communication control unit 913 is a short-range wireless communication for the main display device 210 to communicate with the wired communication unit 914 for communicating with the sensor-equipped sub-display device 230, the sensor-equipped sub-display device 230, and an external device (not shown). Control unit 915 and wired / wireless network communication unit 916 for communicating with an external device (not shown).

  That is, the communication control unit 913 sends inter-device commands and various data transmitted from the sensor-equipped sub display device 230 via either the wired communication unit 914 or the short-range wireless communication unit 915 to the main main control unit 810. Send. In addition, the communication control unit 913 receives inter-device commands and various data transmitted from the main main control unit 810 in response to an instruction from the main main control unit 810, the wired communication unit 914, and the short-range wireless communication unit 915. Is transmitted to the sensor-equipped sub-display device 230.

  In addition, the communication control unit 933 exchanges various data with an external device via either the short-range wireless communication unit 935 or the wired / wireless network communication unit 916.

  As described above, the wired communication unit 914 can be connected to the wired communication unit 934 of the sensor-equipped sub display device 230. The wired communication unit 914 and the wired communication unit 934 of the sensor-equipped sub display device 230 Assumes a USB connection.

  The short-range wireless communication unit 915 is configured to be able to communicate with each other using a wireless communication path formed between the short-range wireless communication unit 935 of the sensor-equipped sub display device 230 and an external device. It is assumed that Bluetooth communication is applied to the wireless communication path. In other words, the short-range wireless communication unit 915 is a communication unit that can transmit and receive signals by Bluetooth communication.

  The wired / wireless network communication unit 916 is configured to be communicably connected to an external communication network, and performs communication with an external device. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available.

  The power source 917 supplies power to each unit included in the main display device 210. When the wired communication unit 914 is connected to the wired communication unit 934 of the sensor-equipped sub display device 230, power is also supplied to the sensor-equipped sub display device 230 via the wired communication unit 914. The power can be turned on / off by the user operating the power switch 922.

  Next, the operation unit 919 receives an input operation of the user of the main display device 210. The operation unit 919 includes, for example, an input device such as a keyboard 920, a switch, a mouse, and a remote controller. Then, the operation unit 919 generates a control signal corresponding to the input operation of the user of the main display device 210, and transmits the generated control signal to the main side main control unit 810.

  As an example of the switch, a hinge switch 921 for detecting the open / closed state of the casing, a power switch 922 for switching on / off of the power, A hardware switch such as a user switch 923 to which a function is assigned is assumed.

  In addition, the main display device 210 includes an audio output unit 924 such as a speaker for outputting audio and an audio input unit 925 such as a microphone for inputting an audio signal.

(Detailed configuration of sensor-equipped sub-display device)
Next, detailed configurations of the display / light sensor unit 330, the display / light sensor circuit control unit 630, and the data processing unit 730 of the sub display device with sensor 230 will be described with reference to FIG. FIG. 6 is a block diagram showing more detailed configurations of the display / light sensor unit 330, the display / light sensor circuit control unit 630, and the data processing unit 730.

  First, a more detailed configuration of the display / light sensor unit 330 will be described. As shown in the figure, the display / light sensor unit 330 includes a sensor built-in liquid crystal panel 301, a backlight 307, and a peripheral circuit 309 for driving them.

  The sensor built-in liquid crystal panel 301 includes a plurality of pixel circuits 31 and photosensor circuits 32 arranged in a matrix. The detailed configuration of the sensor built-in liquid crystal panel 301 will be described later.

  The peripheral circuit 309 includes a liquid crystal panel drive circuit 304, an optical sensor drive circuit 305, a signal conversion circuit 306, and a backlight drive circuit 308.

  The liquid crystal panel drive circuit 304 outputs a control signal (G) and a data signal (S) in accordance with the timing control signal (TC1) and the data signal (D) from the liquid crystal panel control unit 601 of the display / light sensor circuit control unit 630. In this case, the pixel circuit 31 is driven. Details of the driving method of the pixel circuit 31 will be described later.

  The optical sensor driving circuit 305 is a circuit that drives the optical sensor circuit 32 by applying a voltage to the signal line (R) in accordance with the timing control signal (TC2) from the sensor control unit 602 of the display / optical sensor circuit control unit 630. is there. Details of the driving method of the optical sensor circuit 32 will be described later.

  The signal conversion circuit 306 is a circuit that converts the sensor output signal (SS) output from the optical sensor circuit 32 into a digital signal (DS) and transmits the converted signal to the sensor control unit 602.

  The backlight 307 includes a plurality of white LEDs (Light Emitting Diodes) and is disposed on the back surface of the sensor built-in liquid crystal panel 301. When a power supply voltage is applied from the backlight drive circuit 308, the backlight 307 is turned on and irradiates the sensor built-in liquid crystal panel 301 with light. Note that the backlight 307 is not limited to white LEDs, and may include LEDs of other colors. The backlight 307 may include, for example, a cold cathode fluorescent lamp (CCFL) instead of the LED.

  The backlight drive circuit 308 applies a power supply voltage to the backlight 307 when the control signal (BK) from the backlight control unit 603 of the display / light sensor circuit control unit 630 is at a high level. When the control signal from the light control unit 603 is at a low level, no power supply voltage is applied to the backlight 307.

  Next, the display / light sensor circuit control unit 630 will be described. The display / light sensor circuit control unit 630 includes a liquid crystal panel control unit 601, a sensor control unit 602, a backlight control unit 603, and a display data storage unit 604.

  The liquid crystal panel control unit 601 receives display data from the display data processing unit 701 of the data processing unit 730 and, in accordance with an instruction from the display data processing unit 701, sends a timing to the liquid crystal panel drive circuit 304 of the display / light sensor unit 330. A control signal (TC1) and a data signal (D) are transmitted, and the received display data is displayed on the sensor built-in liquid crystal panel 301.

  Note that the liquid crystal panel control unit 601 primarily stores the display data received from the display data processing unit 701 in the display data storage unit 604. Then, a data signal (D) is generated based on the primary stored display data. The display data storage unit 604 is, for example, a video random access memory (VRAM).

  The sensor control unit 602 transmits a timing control signal (TC2) to the optical sensor driving circuit 305 of the display / optical sensor unit 330 in accordance with an instruction from the sensor data processing unit 703 of the data processing unit 730, and the sensor built-in liquid crystal panel 301. Run the scan with.

  In addition, the sensor control unit 602 receives a digital signal (DS) from the signal conversion circuit 306. Then, image data is generated based on the digital signal (DS) corresponding to the sensor output signal (SS) output from all the optical sensor circuits 32 included in the sensor built-in liquid crystal panel 301. That is, the image data read in the entire reading area of the sensor built-in liquid crystal panel 301 is generated. Then, the generated image data is transmitted to the sensor data processing unit 703.

  The backlight control unit 603 transmits a control signal (BK) to the backlight drive circuit 308 of the display / light sensor unit 330 in accordance with instructions from the display data processing unit 701 and the sensor data processing unit 703 to drive the backlight 307. Let

  Next, the data processing unit 730 will be described. The data processing unit 730 includes a display data processing unit 701 and a sensor data processing unit 703.

  The display data processing unit 701 receives display data from the sub-side main control unit 830 and gives instructions to the liquid crystal panel control unit 601 and the backlight control unit 603, and displays the received display data on the sensor built-in liquid crystal panel 301. Let

  The sensor data processing unit 703 gives instructions to the sensor control unit 602 and the backlight control unit 603 in accordance with the sub-display device command received by the data processing unit 730.

  The sensor data processing unit 703 receives image data from the sensor control unit 602 and stores the image data in the image data buffer 704 as it is. The sensor data processing unit 703 then selects “whole image data”, “partial image data (partial) based on the image data stored in the image data buffer 704 in accordance with the sub-display device command received by the data processing unit 730. At least one of “including coordinate data of image)” and “coordinate data” is transmitted to the sub-side main control unit 830. The whole image data, partial image data, and coordinate data will be described later. The operation of the sensor data processing unit 703 according to the sub display device command will be described later.

(Detailed configuration of main display device)
As described above, the display unit 310 and the display circuit control unit 610 of the main display device 210 can be configured using a generally known technique, but these configuration examples are described with reference to FIG. A brief explanation will be given. FIG. 7 is a block diagram illustrating a configuration example of the display unit 310 and the display circuit control unit 610.

  The display unit 310 includes a liquid crystal panel 311 including a plurality of pixel circuits 38 arranged in a matrix, a plurality of white LEDs, a backlight 317 disposed on the back surface of the liquid crystal panel 311, and drives them. And a peripheral circuit 319. The backlight 317 is not limited to the white LED, and may include another color LED, or may include a cold cathode tube instead of the LED.

  The peripheral circuit 319 includes a liquid crystal panel drive circuit 314 that drives the pixel circuit 38 and a backlight drive circuit 318 that applies a power supply voltage to the backlight 317. Note that when a power supply voltage is applied from the backlight driving circuit 318, the backlight 317 is turned on and irradiates the liquid crystal panel 311 with light.

  Next, the display circuit control unit 610 includes a liquid crystal panel control unit 611, a backlight control unit 613, and a display data storage unit 614.

  The liquid crystal panel control unit 611 receives display data from the display data processing unit 710 and transmits a control signal and a data signal to the liquid crystal panel drive circuit 314 in accordance with an instruction from the display data processing unit 710. The received display data Is displayed on the liquid crystal panel 311. The liquid crystal panel control unit 611 temporarily stores the display data received from the display data processing unit 710 in the display data storage unit 614 (VRAM or the like), and generates a data signal based on the display data stored in the primary storage. To do.

  The backlight control unit 613 transmits a control signal to the backlight driving circuit 318 according to an instruction from the display data processing unit 710 to drive the backlight 317.

(Sub display device command)
Next, with reference to FIGS. 8 and 9, details of the sub display device commands transmitted from the sub-side main control unit 830 to the data processing unit 730 will be described. FIG. 8 is a diagram schematically showing an example of the frame structure of the sub-display device command. FIG. 9 is a diagram for explaining an example of values that can be specified in each field included in the sub-display device command and an outline thereof.

  As shown in FIG. 8, the sub-display device commands include “header”, “data acquisition timing”, “data type”, “scan method”, “scanned image gradation”, “scan resolution”, and “reserved”. Each field. In each field, for example, values shown in FIG. 9 can be designated.

  The “header” field is a field indicating the start of a frame of the sub display device command. As long as it is possible to identify the start of the frame, the value of the “header” field may be any value.

  Next, the “data acquisition timing” field is a field for designating timing for acquiring data from the sensor data processing unit 703. In the “data acquisition timing” field, for example, values “00” (sense), “01” (event), and “10” (all) can be specified.

  Here, “sense” designates that the latest data is immediately transmitted to the sensor data processing unit 703. Therefore, when the sensor data processing unit 703 receives a command in the sub display device whose value in the “data acquisition timing” field is “sense”, the latest data specified in the “data type” field is immediately updated. Transmit to the sub-side main control unit 830.

  “Event” designates that the sensor data processing unit 703 transmits the image data received from the sensor control unit 602 at a timing when the change occurs. Therefore, when the sensor data processing unit 703 receives a command in the sub display device whose value of the “data acquisition timing” field is “event”, the sensor control unit 602 converts the data specified in the “data type” field. Is transmitted to the sub-side main control unit 830 at a timing when a change larger than a predetermined threshold occurs in the image data received from.

  Further, “all” designates that the sensor data processing unit 703 transmits data at a predetermined cycle. Therefore, when the sensor data processing unit 703 receives a command in the sub display device having a value of “all” in the “data acquisition timing” field, the data designated in the “data type” field is Transmit to the sub-side main control unit 830. The predetermined period coincides with the period in which the optical sensor circuit 32 performs scanning.

  Next, the “data type” field is a field for designating the type of data acquired from the sensor data processing unit 703. In the “data type” field, for example, values of “001” (coordinates), “010” (partial image), and “100” (entire image) can be specified. Furthermore, by adding these values, “coordinates” and “partial image” / “whole image” can be specified simultaneously. For example, when “coordinate” and “partial image” are specified at the same time, “011” can be specified.

  When the sensor data processing unit 703 receives a command in the sub display device whose value of the “data type” field is “whole image”, the image data itself stored in the image data buffer 704 is sent to the sub-side main control unit 830. Send. The image data itself stored in the image data buffer 704 is referred to as “whole image data”.

  In addition, when the sensor data processing unit 703 receives a command in the sub display device whose value of the “data type” field is “partial image”, the image data received from the sensor control unit 602 changes more than a predetermined threshold. An area including the generated portion is extracted, and image data of the extracted area is transmitted to the sub-side main control unit 830. Here, the image data is referred to as “partial image data”.

  When a plurality of partial image data is extracted from the image data received from the sensor control unit 602, the sensor data processing unit 703 sends each of the extracted plurality of partial image data to the sub-side main control unit 830. Send. Therefore, for example, when the user touches a plurality of locations on the sensor built-in liquid crystal panel 301 with a plurality of fingers or pens at the same time, the sensor data processing unit 703 extracts each of the touched regions as partial image data, Transmit to the sub-side main control unit 830.

  Further, the sensor data processing unit 703 detects a representative point in the partial image and receives the position of the representative point in the partial image when a command in the sub display device whose value of the “data type” field is “partial image” is received. Is sent to the sub-side main control unit 830. The representative point includes, for example, the center of the partial image data, the center of gravity of the partial image data, and the like.

  When a plurality of partial image data is extracted from the image data received from the sensor control unit 602, the sensor data processing unit 703 detects each representative point of the extracted plurality of partial image data, and The coordinate data of the representative coordinates indicating the position of each detected representative point in the partial image is transmitted to the sub-side main control unit 830. Therefore, for example, when the user touches a plurality of locations on the sensor built-in liquid crystal panel 301 with a plurality of fingers or pens at the same time, the sensor data processing unit 703 displays the position of the representative point of each touched region in each region. Is transmitted to the sub-side main control unit 830.

  Next, when the sensor data processing unit 703 receives the in-sub-display device command whose value of the “data type” field is “coordinate”, the coordinate data of the representative coordinate indicating the position of the representative point in the entire image data is obtained. Transmit to the sub-side main control unit 830.

  When a plurality of partial image data is extracted from the image data received from the sensor control unit 602, the sensor data processing unit 703 displays the position of each representative point of the extracted plurality of partial images in the entire image. Is sent to the sub-side main control unit 830. Therefore, for example, when the user simultaneously touches a plurality of locations on the sensor built-in liquid crystal panel 301 with a plurality of fingers, pens, etc., the sensor data processing unit 703 displays the positions of the touched representative points in the entire image. Is transmitted to the sub-side main control unit 830.

  That is, the sensor-equipped sub-display device 230 can detect the representative point of the touched region regardless of the number of touches of the sensor-equipped liquid crystal panel 301 by the user with a finger or a pen (multi-point detection). Then, the coordinate data of the representative coordinates indicating the position of the detected representative point in the partial image and the coordinate data of the representative coordinates indicating the position in the entire image can be used as the sub-side main control unit 830.

  Specific examples of the whole image data, the partial image data, and the coordinate data will be described later with reference to schematic diagrams.

  Next, the “scan method” field is a field for designating whether or not the backlight 307 is turned on at the time of executing the scan. In the “scan method” field, for example, values of “00” (reflection), “01” (transmission), and “10” (reflection / transmission) can be designated.

  “Reflection” designates that scanning is performed with the backlight 307 turned on. Therefore, when the sensor data processing unit 703 receives a command in the sub-display device in which the value of the “scan method” field is “reflection”, the optical sensor driving circuit 305 and the backlight driving circuit 308 operate in synchronization. In addition, an instruction is given to the sensor control unit 602 and the backlight control unit 603.

  “Transmission” specifies that scanning is performed with the backlight 307 turned off. Accordingly, when the sensor data processing unit 703 receives the sub-display device command whose “scan method” field value is “transparent”, the sensor data processing unit 703 operates the optical sensor driving circuit 305 and does not operate the backlight driving circuit 308. An instruction is given to the sensor control unit 602 and the backlight control unit 603. Note that “reflection / transmission” specifies that scanning is performed using both “reflection” and “transmission”.

  Next, the “scanned image gradation” field is a field for designating gradations of the partial image data and the entire image data. In the “scanned image gradation” field, for example, values of “00” (binary) and “01” (multivalue) can be designated.

  Here, when the sensor data processing unit 703 receives a command in the sub display device having a “binary” value in the “scanned image gradation” field, the partial image data and the whole image data are converted into monochrome data, and the main data on the sub side. Transmit to the control unit 830.

  In addition, when the sensor data processing unit 703 receives a command in the sub-display device having a “scanned image gradation” field value of “multi-value”, the partial image data and the entire image data are converted into multi-gradation data, Transmit to the main control unit 830.

  Next, the “scan resolution” field is a field for designating the resolution of the partial image data and the entire image data. In the “scan resolution” field, for example, values of “0” (high) and “1” (low) can be specified.

  Here, “high” designates a high resolution. Therefore, when the sensor data processing unit 703 receives a command in the sub display device having a value of “high” in the “scan resolution” field, it transmits the partial image data and the entire image data to the sub-side main control unit 830 at a high resolution. To do. For example, it is desirable to designate “high” for image data (image data such as a fingerprint) to be subjected to image processing such as image recognition.

  “Low” designates a low resolution. Therefore, when the sensor data processing unit 703 receives the sub-display device internal command whose “scan resolution” field value is “low”, it transmits the partial image data and the entire image data to the sub-side main control unit 830 at a low resolution. To do. For example, it is desirable to designate “low” for image data (such as touched finger or hand image data) that only needs to be recognized.

  Next, the “reserved” field is a field that is appropriately specified when it is necessary to further specify information other than information that can be specified in the above-described fields.

  Note that it is not necessary to use all of the above-described fields when transmitting a sub-display device command, and invalid values (such as a NULL value) may be set for fields that are not used.

  When the user wants to acquire coordinate data of a position touched with a finger or a pen, a command in the sub display device in which “coordinate” is specified in the “data type” field is transmitted to the data processing unit 730. When it is desired to detect the movement of a finger or a pen, it is desirable to further specify “all” in the “data acquisition timing” field of the sub display device command to acquire coordinate data. In this case, since it is only necessary to obtain coordinate data of the touched position, the scanning accuracy may not be high. Therefore, it is only necessary to designate “low” as the value of the “scan resolution” field of the sub display device command.

  In addition, when “coordinate” is specified in the “data type” field of the sub display device command, for example, when the user touches the sensor built-in liquid crystal panel 301 with a plurality of fingers or pens at the same time, the touch is performed. Each coordinate data of the position can be acquired (multi-point detection).

  Further, when acquiring image data of an object such as a document, a command in the sub display device in which “whole image” is specified in the “data type” field is transmitted to the data processing unit 730. Since it is common to scan an object in a stationary state, it is not necessary to periodically scan the object. Therefore, in this case, it is desirable to designate “sense” or “event” in the “data acquisition timing” field. When scanning an object such as a document, it is desirable that the scanning accuracy is high so that the user can easily read the characters. Therefore, it is desirable to designate “high” in the “scan resolution” field.

(Inter-device command)
Next, the frame structure of the inter-device command will be described with reference to FIG. 10 and FIG. First, FIG. 10 is a schematic diagram illustrating an example of a frame structure of an inter-device command. As illustrated, the inter-device command includes a “header” field, “0th” to “fifth” fields, and a “reserved” field in order from the top.

  The “header” field is a field indicating the start of the frame of the inter-device command. As long as it is possible to identify the start of the frame, the header field value may be any value.

  Next, the “0th” field is a field for designating a “command identifier” for identifying the type of command between devices.

  The inter-device command whose command identifier value is “000” is referred to herein as a “scan command”. The scan command is sent from the main main controller 810 to the sub display main unit 810 so that the main display device 210 causes the sub display device with sensor 230 to scan an object located in the vicinity of the sensor built-in liquid crystal panel 301. It is transmitted to the control unit 830. The sub-side main control unit 830 that has received the scan command converts the scan command into a sub-display device command by removing the 0th field from the scan command, and converts the converted sub-display device command into data. The data is transmitted to the processing unit 730.

  Further, an inter-device command whose command identifier value is “001” is referred to herein as a “display command”. The display command is sent from the main main control unit 810 to the sub display device with sensor 230 so that the display / light sensor unit 330 displays image data such as photographs and icons. It is transmitted to the side main control unit 830.

  An inter-device command whose command identifier value is “010” is referred to herein as a “processing command”. The processing command is transmitted / received between the main side main control unit 810 and the sub side main control unit 830 so that either the sub display device with sensor 230 or the main display device 210 causes the other party to execute a predetermined process. The

  Next, information that can be specified in each field from “first” to “fifth” is different for each of the scan command, the display command, and the processing command. Information that can be specified in the first to fifth fields will be described later.

  Next, the “reserved” field is a field that is appropriately specified when it is necessary to further specify information other than the information that can be specified in each field described above.

  Note that it is not necessary to use all the fields described above, and invalid values (such as NULL values) may be set for fields that are not used.

  Next, information that can be specified in each field for each of the scan command, the display command, and the processing command will be described with reference to FIG. FIG. 11 is an explanatory diagram showing information that can be specified in each field for each of a scan command, a display command, and a processing command.

  As shown in the figure, in the scan command, “data acquisition timing”, “data type”, “scan method”, “scan image gradation”, and “scan resolution” can be specified in the first to fifth fields. it can. Note that the information that can be specified in the first to fifth fields of the scan command is the same as the information that can be specified in the fields of the sub display device command.

  The information that can be specified in the first to fifth fields of the scan command is the same as the information that can be specified in each field of the command in the sub display device, so that the sub-side main control unit that has received the scan command. 830 removes the 0th field of the scan command received from the main main control unit 810 so that the received scan command can be converted into a sub-display device command.

  In the display command, “display number” and “display data / display position” can be designated in the first and second fields. Note that the third to fifth fields are not used (a NULL value).

  In the processing command, “execution processing type”, “argument number”, and “argument information” can be specified in the first to third fields. Note that the fourth and fifth fields are not used (is a NULL value).

  Next, an example of values that can be specified in each field and an outline thereof will be described for each of the scan command, the display command, and the processing command with reference to FIGS.

  FIG. 12 is an explanatory diagram of an example of values that can be specified in the first to fifth fields of the scan command and an outline thereof. As shown in the figure, the first to fifth fields of the scan command are the same as the fields of the sub-display device command, and thus description thereof is omitted here.

  Next, the display command will be described. FIG. 13 is an explanatory diagram of an example of values that can be specified in the first and second fields of the display command and an outline thereof.

  The first field (display number) of the display command is a field for designating the number of data to be displayed on the display / light sensor unit 330 of the sub display device with sensor 230. That is, it indicates the number of sets designated in the second field (display data / display position).

  The second field (display data / display position) of the display command is a field indicating a set of image data itself to be displayed on the display / light sensor unit 330 of the sub-display device with sensor 230 and the display position (coordinates) of the data. It is. When a plurality of image data are displayed, a plurality of sets are designated in the second field.

  Next, the processing command will be described. FIG. 14 is an explanatory diagram of an example of values that can be specified in the first to third fields of the processing command and an outline thereof.

  The first field (execution processing type) of the processing command is a field for specifying identification information of processing (various applications, middleware, OS, etc.) to be executed by the main control unit that is the processing command transmission destination. The identification information may be any value as long as the process can be identified.

  The second field (argument number) of the processing command is a field for designating the number of various information (arguments) used during or after the processing indicated by the identification information specified in the first field. That is, it indicates the number of data specified in the third field (argument information).

  The third field (argument information) of the processing command is a field for designating various information (arguments) used during or after the processing indicated by the identification information specified in the first field.

  The information used at the time of starting includes, for example, an operation mode of a predetermined process specified in the first field, a parameter specifying a display area, and the like.

  Further, the information used after activation includes, for example, image data to be recognized by the character recognition application when a character recognition application is specified in the first field. Further, when a file transfer application is specified in the first field, a file to be transferred by the file transfer application can be cited.

(Whole image data / Partial image data / Coordinate data)
Next, the whole image data, the partial image data, and the coordinate data will be described by way of example with reference to FIG. The image data shown in FIG. 15A is image data obtained as a result of scanning the entire sensor-equipped liquid crystal panel 301 when the object is not placed on the sensor-equipped liquid crystal panel 301. 15B is image data obtained as a result of scanning the entire sensor-equipped liquid crystal panel 301 when the user is touching the sensor-equipped liquid crystal panel 301 with a finger.

  When the user touches the sensor built-in liquid crystal panel 301 with a finger, the amount of light received by the photosensor circuit 32 in the vicinity of the touch changes, so that the voltage output from the photosensor circuit 32 changes, and as a result, In the image data generated by the sensor control unit 602, the brightness of the pixel value of the portion touched by the user changes.

  In the image data shown in FIG. 15B, the brightness of the pixel value of the portion corresponding to the user's finger is higher than that in the image data shown in FIG. In the image data shown in FIG. 15B, the smallest rectangular area (area PP) that includes all pixel values whose lightness changes more than a predetermined threshold is “partial image data”.

  The image data indicated by the area AP is “whole image data”.

  Also, the coordinate data in the whole image data (area AP) of the representative coordinates Z of the partial image data (area PP) is (Xa, Ya), and the coordinate data in the partial image data (area PP) is (Xp, Yp). It is.

(Configuration of sensor built-in liquid crystal panel)
Next, the configuration of the sensor built-in liquid crystal panel 301 included in the sensor-equipped sub display device 230 and the configuration of the peripheral circuit 309 of the sensor built-in liquid crystal panel 301 will be described with reference to FIG. FIG. 16 is a block diagram showing the main part of the display / light sensor unit 330, particularly the configuration of the sensor built-in liquid crystal panel 301 and the configuration of the peripheral circuit 309.

  The sensor built-in liquid crystal panel 301 includes a pixel circuit 31 for setting light transmittance (brightness) and an optical sensor circuit 32 that outputs a voltage corresponding to the intensity of light received by the sensor. The pixel circuit 31 is used as a general term for the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b corresponding to the red, green, and blue color filters, respectively.

  The pixel circuits 31 are arranged on the sensor built-in liquid crystal panel 301 in the column direction (vertical direction) and 3n in the row direction (horizontal direction). A set of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b is continuously arranged in the row direction (lateral direction). This set forms one pixel.

  In order to set the light transmittance of the pixel circuit 31, first, the high level voltage (TFT 33 is turned on) to the scanning signal line Gi connected to the gate terminal of the TFT (Thin Film Transistor) 33 included in the pixel circuit 31. Voltage). Thereafter, a predetermined voltage is applied to the data signal line SRj connected to the source terminal of the TFT 33 of the R pixel circuit 31r. Similarly, the light transmittance is also set for the G pixel circuit 31g and the B pixel circuit 31b. Then, by setting these light transmittances, an image is displayed on the sensor built-in liquid crystal panel 301.

  Next, the photosensor circuit 32 is arranged for each pixel. One pixel may be arranged in the vicinity of each of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b.

  In order for the optical sensor circuit 32 to output a voltage according to the light intensity, first, the sensor readout line RWi connected to one electrode of the capacitor 35 and the anode terminal of the photodiode 6 are connected. A predetermined voltage is applied to the sensor reset line RSi. In this state, when light is incident on the photodiode 6, a current corresponding to the amount of incident light flows through the photodiode 6. And according to the said electric current, the voltage of the connection point (henceforth connection node V) of the other electrode of the capacitor | condenser 35 and the cathode terminal of the photodiode 6 falls. When the power supply voltage VDD is applied to the voltage application line SDj connected to the drain terminal of the sensor preamplifier 37, the voltage at the connection node V is amplified and output from the source terminal of the sensor preamplifier 37 to the sensing data output line SPj. Based on the output voltage, the amount of light received by the optical sensor circuit 32 can be calculated.

  Next, the liquid crystal panel drive circuit 304, the optical sensor drive circuit 305, and the sensor output amplifier 44, which are peripheral circuits of the sensor built-in liquid crystal panel 301, will be described.

  The liquid crystal panel drive circuit 304 is a circuit for driving the pixel circuit 31, and includes a scanning signal line drive circuit 3041 and a data signal line drive circuit 3042.

  The scanning signal line driving circuit 3041 sequentially selects one scanning signal line from the scanning signal lines G1 to Gm for each line time based on the timing control signal TC1 received from the liquid crystal panel control unit 601. A high level voltage is applied to the selected scanning signal line, and a low level voltage is applied to the other scanning signal lines.

  Based on the display data D (DR, DG, and DB) received from the liquid crystal panel controller 601, the data signal line driver circuit 3042 applies a predetermined voltage corresponding to the display data for one row for each line time. Are applied to the data signal lines SR1 to SRn, SG1 to SGn, and SB1 to SBn (line sequential method). Note that the data signal line driver circuit 3042 may be driven by a dot sequential method.

  The optical sensor driving circuit 305 is a circuit for driving the optical sensor circuit 32. Based on the timing control signal TC2 received from the sensor control unit 602, the optical sensor driving circuit 305 selects a predetermined sensor readout signal line from the sensor readout signal lines RW1 to RWm for each line time. A read voltage is applied, and a voltage other than a predetermined read voltage is applied to the other sensor read signal lines. Similarly, based on the timing control signal TC2, a predetermined reset voltage is applied to the sensor reset signal line selected from the sensor reset signal lines RS1 to RSm for each line time, and the others. A voltage other than a predetermined reset voltage is applied to the sensor reset signal line.

  The sensing data output signal lines SP1 to SPn are grouped into p groups (p is an integer of 1 to n), and the sensing data output signal lines belonging to each group are connected via a switch 47 that is sequentially turned on in time division. And connected to the sensor output amplifier 44. The sensor output amplifier 44 amplifies the voltage from the group of sensing data output signal lines connected by the switch 47 and outputs the amplified voltage to the signal conversion circuit 306 as sensor output signals SS (SS1 to SSp).

(Detailed configuration of main display device and sub display device with sensor)
Next, a more detailed configuration of the main display device 210 and the sensor-equipped sub display device 230 will be described with reference to FIG. FIG. 1 is a block diagram showing a more detailed configuration of the main display device 210 and the sensor-equipped sub display device 230.

(Detailed configuration of main display device)
First, the storage unit 911 of the main display device 210 will be described. As illustrated, the storage unit 911 includes a main-side BIOS program (first process) P11, a main-side boot loader B11, and a set of a plurality of main-side secondary boot loaders and main-side OS programs (second process, Candidate processing) is stored. As a set of the main side secondary boot loader and the main side OS program, in FIG. 1, the main side secondary boot loader B21 and the main side OS program P21, the main side secondary boot loader B22 and the main side OS program P22, main Three sets of the secondary boot loader B23 and the main OS program P23 are stored. Note that the main-side secondary boot loader and the main-side OS program are not necessarily separate programs, and the main-side secondary boot loader and the main-side OS program may be combined into one program.

  The main-side BIOS program P11 is stored in, for example, the flash memory 16, and the set of the main-side boot loader B11 and a plurality of main-side secondary boot loaders and main-side OS programs is stored in the hard disk 17, for example. It is assumed that Each set of the main side secondary boot loader and the main side OS program is assumed to be divided and stored in predetermined partitions (areas) PA1 to PA3 as shown in the figure. Normally, at least one of the partitions is set to active (a start flag is set).

  The main BIOS program P11 is a BIOS (Basic Input Output System) program of the main display device 210. Hereinafter, the BIOS of the main display device 210 is referred to as a main BIOS. The main-side BIOS program P11 is assumed to be a generally known BIOS program, and is executed by the main-side main control unit 810 immediately after power is supplied to the main display device 210. Since the process performed by the main-side BIOS program P11 is the same as the process performed by a generally known BIOS program, detailed description thereof is omitted here.

  The main-side BIOS program P11 performs initialization processing (POST: Power On Self Test) of various devices connected to the main display device 210. During the POST process, “POST code” (execution status information) that is information indicating the progress (execution status) of the process is generated and sequentially output to the POST code holding unit 18. Therefore, the execution status of the POST process can be known by referring to the POST code output to the POST code holding unit 18. Since the POST code is generally known, detailed description thereof is omitted here.

  When the POST process is normally completed to the end, a POST code indicating that is output to the POST code holding unit 18. Further, if any problem occurs during the POST process (for example, initialization of a certain device has failed), the POST process is interrupted. In the case of the interruption, the POST code holding unit 18 outputs a POST code indicating the processing progress at the time of the interruption.

  It is assumed that the POST code holding unit 18 is a circuit or the like that can hold the value of the POST code and can hold the held value from the outside. For example, it is assumed that the POST code holding unit 18 is provided on a mother board or the like.

  Next, the main boot loader B11 is a program called by the main BIOS program P11. The main side boot loader B11 has a function provided by a generally known boot loader, and is stored in the first sector (boot sector) of a disk or the like in which the main side OS programs P21 to P23 are stored. ing. In the case of a hard disk drive, the sector is generally a sector called a master boot record (MBR).

  The main-side boot loader B11 is read and executed by the main-side main control unit 810. The main side boot loader B11 determines a partition to be referred to, and executes the main side secondary boot loader stored in the head sector of the determined partition. Normally, a partition that is set to be active is a partition to be referred to. The first sector of each partition is generally a sector called a volume boot record (VBR) or a partition boot record (PBR).

  Next, the main secondary boot loaders B21, B22, and B23 are read and executed by the main main controller 810 as a result of the execution of the main boot loader B11. It is a program for starting. The main-side secondary boot loaders B21, B22, and B23 are assumed to be generally known secondary boot loaders, and are stored in the first sector of each partition such as a disk in which the main-side OS is stored. ing. Specific examples of the main secondary boot loaders B21, B22, and B23 include LILO (LInux LOader), GRUB (GRand Unified Bootloader), Windows (registered trademark) Boot Manager, and NTLDR (NT Loader).

  When the main secondary boot loader is executed, the main OS program stored in the partition where the main secondary boot loader is stored is read into the main main control unit 810, It is activated.

  Next, the main-side OS programs P21, P22, and P23 are programs for the OS of the main display device 210 (hereinafter, main-side OS). In the main display device 210, the main side OS boot program is started by the main side secondary boot loader. The type of the main OS is not limited. For example, the main-side OS programs P21, P22, and P23 are a Windows (registered trademark) XP program, a Windows (registered trademark) Vista program, a recovery program for restoring the OS, and the like.

  It is assumed that identification information of the corresponding main OS is added to the main OS programs P21, P22, and P23 so that they can be referred to from the outside.

  Next, the main side main control unit 810 will be described. As shown in the figure, the main side main control unit 810 includes a main side boot loader execution unit 13 (execution control means), a main side secondary boot loader execution unit 14, and a main side OS activation unit 15. When the main side secondary boot loader and the main side OS program are combined into one program, the main side secondary boot loader execution unit 14 and the main side OS startup unit 15 are merged into one. Also good.

  The main-side boot loader execution unit 13 operates as a result of the main-side boot loader B11 being read by the main-side main control unit 810. The main-side boot loader execution unit 13 includes at least a timeout detection unit 131 and a secondary boot loader activation unit 132. Further, when a list of identification information (candidate processing information) of the main OS that can be booted by the main display device 210 is requested from the sub display device with sensor 230, the main boot loader execution unit 13 further It is configured to include a side OS list transmission unit 133 (specifically, the main side boot loader execution unit 13 is described in the main side boot loader B11 so as to include the main side OS list transmission unit 133). In the following, a list of identification information of main OSs that can be activated by the main display device 210 is referred to as a “main OS list”.

  The time-out detection unit 131 detects that a predetermined time has elapsed since the main-side boot loader B11 was read into the main-side main control unit 810.

  The secondary boot loader activation unit 132 normally operates as a trigger when the timeout detection unit 131 detects that the predetermined time has elapsed, and determines the partition set to be active as the partition to be referred to. To do. Then, the main secondary boot loader (any of B11 to B13) stored in the determined partition is read into the main main control unit 810.

  However, before the predetermined time elapses, the main-side boot loader execution unit 13 is activated from the main-side activation OS designation unit 25 (described later) of the sensor-equipped sub-display device 230 via the communication control unit 913. When the processing command in which the identification information of the main OS to be specified is received, the secondary boot loader activation unit 132 operates in response to the reception and stores the main OS corresponding to the specified identification information. The partition that is being used is determined as the partition to be referenced.

  Next, when the main-side boot loader execution unit 13 receives a processing command in which a request for the main-side OS list is specified via the communication control unit 913, the main-side OS list transmission unit 133 responds to the reception. Works. Then, as a response to the request, the main-side OS identification information acquired from the storage unit 911 is transmitted to the processing command request source as a main-side OS list.

  At this time, the timeout detection unit 131 further detects that a predetermined time has elapsed since the main OS list transmission unit 133 transmitted the main OS list to the request source of the received processing command. Then, when the predetermined time has passed without the main OS to be started being instructed, the secondary boot loader starting unit 132 stores the main secondary boot loader (B11 to B11) stored in the partition set to be active. (B13) is read into the main main control unit 810.

  Next, the main-side secondary boot loader execution unit 14 is executed as a result of one of the main-side secondary boot loaders B11 to B13 being read into the main-side main control unit 810 and executed by the secondary boot loader activation unit 132. It works. Then, the main OS is started by reading the main OS program stored in the same partition as the main secondary boot loader read into the main main controller 810 into the main main controller 810.

  Next, the main-side OS activation unit 15 operates as a result of the main-side secondary boot loader execution unit 14 reading and executing any of the main-side OS programs P21, P22, and P23 into the main-side main control unit 810. The main OS is started.

(Detailed configuration of the sub display device with sensor)
Next, the memory | storage part 931 of the sub display apparatus 230 with a sensor is demonstrated. As illustrated, the storage unit 931 stores a sub-side BIOS program P31, a sub-side boot loader B31, and a set of the sub-side secondary boot loader B41 and the sub-side OS program P41. The sub-side secondary boot loader B41 and the sub-side OS program P41 are not necessarily separate programs, and the sub-side secondary boot loader B41 and the sub-side OS program P41 may be combined into one program. .

  The sub-side BIOS program P31 is stored in, for example, the flash memory 26, and the set of the sub-side boot loader B31 and the sub-side secondary boot loader B41 and the sub-side OS program P41 is stored in, for example, the hard disk 27. Assuming that Further, it is assumed that a set of the sub-side secondary boot loader B41 and the sub-side OS program P41 is stored in a predetermined partition PA4.

  The sub-side BIOS program P31 is a program for the BIOS of the sub-display device with sensor 230 (hereinafter referred to as sub-side BIOS). The sub-side BIOS program P31 is assumed to be a generally known BIOS program, and is executed by the sub-side main control unit 830 immediately after power is supplied to the sensor-equipped sub display device 230. Since the processing performed by the sub-side BIOS program P31 is the same as the processing performed by the generally known BIOS program, detailed description thereof is omitted here.

  Next, the sub-side boot loader B31 is a program called by the sub-side BIOS program P31. The sub-side boot loader B31 has a function of a generally known boot loader, and is stored in the first sector (boot sector) such as a disk in which the sub-side OS program P41 is stored. .

  The sub-side boot loader B31 is read and executed by the sub-side main control unit 830. Then, the sub-side boot loader B31 executes the sub-side secondary boot loader B41 stored in the first sector of the partition.

  Next, the sub-side secondary boot loader B41 is read and executed by the sub-side main control unit 830 as a result of the execution of the sub-side boot loader B31, and starts the sub-side OS program. It is a program. The sub-side secondary boot loader B41 is assumed to be a generally known secondary boot loader, and is stored in the top sector of each partition such as a disk in which the sub-side OS is stored. Specifically, the sub-side secondary boot loader B41 is LILO, GRUB, Windows (registered trademark) Boot Manager, NTLDR, or the like.

  When the sub-side secondary boot loader is executed, the sub-side OS program P41 stored in the partition in which the sub-side secondary boot loader is stored is read into the sub-side main control unit 830. It is activated.

  Next, the sub-side OS program P41 is a program of the OS of the sensor-equipped sub-display device 230 (hereinafter referred to as sub-side OS).

  Even when the main display device 210 and the sensor-equipped sub-display device 230 are turned on simultaneously, it is possible to specify the OS activation of the main display device 210 from the sensor-equipped sub-display device 230. Therefore, the time required for starting the sub-side OS in the sub display device with sensor 230 is shorter than the time required for starting the main side OS in the main display device 210 as much as possible. Is preferable. Therefore, an OS that requires a relatively short time for the startup process, such as Linux or Tron, is suitable for the sub-side OS.

  When the sub display device with sensor 230 is attached to the main display device 210 (that is, when the wired communication unit 914 of the main display device 210 and the wired communication unit 934 of the sub display device with sensor 230 are connected). As described above, by turning on the power supply 917 of the main display device 210, power is supplied to the sub display device with sensor 230, and as a result, power is supplied to each part of the sub display device with sensor 230. The

  Next, the sub-side main control unit 830 will be described. As illustrated, the sub-side main control unit 830 includes a sub-side boot loader execution unit 21, a sub-side secondary boot loader execution unit 22, a sub-side OS activation unit 23, a POST code inspection unit (execution status determination unit) 24, And a main-side boot OS designation unit (execution designation means) 25. The POST code checking unit 24 and the main-side startup OS designation unit 25 operate after the sub-side OS is started up. When the sub-side secondary boot loader B41 and the sub-side OS program P41 are combined into one program, the sub-side secondary boot loader execution unit 22 and the sub-side OS startup unit 23 are merged into one. May be.

  The sub-side boot loader execution unit 21 operates as a result of the sub-side boot loader B31 being read by the sub-side main control unit 830. The sub-side boot loader execution unit 21 reads the sub-side secondary boot loader B41 stored in the partition into the sub-side main control unit 830.

  Next, the sub-side secondary boot loader execution unit 22 operates as a result of the sub-side secondary boot loader B 41 being read by the sub-side main load control unit 830 and executed by the sub-side boot loader execution unit 21. . Then, it starts by reading the sub-side OS program P41 into the sub-side main control unit 830.

  Next, the sub-side OS startup unit 23 operates as a result of the sub-side OS program P41 being read by the sub-side secondary boot loader execution unit 22 and executed by the sub-side main control unit 830. Start the OS.

  Next, the POST code checking unit 24 checks the POST code output from the main BIOS program P11 of the main display device 210 immediately after the sub OS is started. For this purpose, the POST code inspection unit 24 refers to the POST code output to the POST code holding unit 18 of the main display device 210 via the communication control unit 933. Based on the POST code referred to above, the progress status of the POST process performed by the main-side BIOS program P11 of the main display device 210 is checked, and (A) whether the POST process has been normally completed to the end, and (B ) It is determined whether the POST process is interrupted.

  Specifically, when the referred POST code is a value indicating that the POST process has been normally completed to the end, the POST code checking unit 24 determines that the POST process has been normally completed to the end. Then, the main side startup OS designation unit 25 is notified to that effect.

  On the other hand, when the referred POST code is other than a value indicating that the POST process has been completed normally, the POST code checking unit 24 refers to the POST code output to the POST code holding unit 18. repeat. Note that the number of repeated references and the reference interval may be determined in advance or may be determined by a predetermined calculation.

  If the POST code is always the same value as a result of the repeated reference, no new POST code is output to the POST code holding unit 18, and the POST code checking unit 24 interrupts the POST process. Judge that Then, the POST code inspection unit 24 displays a predetermined message corresponding to the last referenced POST code on the display / light sensor unit 330 to notify the user that the POST process has been interrupted. The POST code inspection unit 24 generates display data including the predetermined message, and transmits the generated display data to the data processing unit 730, thereby causing the display / light sensor unit 330 to display the predetermined message. .

  On the other hand, as a result of the repeated reference, if the value of the POST code changes, a new POST code is output to the POST code holding unit 18, and therefore the POST process is considered to be in progress. Therefore, in this case, the process of referring to the POST code output to the POST code holding unit 18 is repeated until one of the above (A) and (B) can be determined.

  Next, when receiving the notification from the POST code checking unit 24, the main-side startup OS designating unit 25 instructs the main display device 210 to start the main-side OS to be started, and the main-side startup OS designation unit 25 starts up the main display device 210. The OS identification information is transmitted to the main display device 210 via the communication control unit 933.

  More specifically, the main boot OS designation unit 25 designates information for identifying the secondary boot loader activation unit 132 in the first field, designates “1” in the second field, and designates in the third field. Then, the processing command specifying the identification information of the main-side OS to be activated on the main display device 210 is transmitted to the main display device 210.

  Note that the identification information of the main OS may be stored in the storage unit 931 in advance, or the main display device 210 may be inquired. When making an inquiry to the main display device 210, the main-side startup OS designation unit 25 is further configured to include a main-side OS list request unit (candidate processing information acquisition unit) 251. The main OS list request unit 251 requests the main OS list from the main OS list transmission unit 133 of the main display device 210 via the communication control unit 933. Then, the main OS list transmitted from the main OS list transmitter 133 is received as a response to the request.

  When requesting the main-side OS list, the main-side OS list request unit 251 designates information for identifying the main-side OS list transmission unit 133 in the first field, and designates “1” in the second field. Then, a processing command specifying the identification information of the main-side OS list request unit 251 as a return source of the main-side OS list in the third field is transmitted to the main display device 210.

  In addition, the main-side OS that the main-side startup OS designation unit 25 causes the main display device 210 to start may be calculated and specified based on a predetermined rule, or may be specified by being selected by the user. When making a user select, the main side starting OS designation | designated part 25 is further comprised so that the user presentation process part (candidate process selection means) 252 may be included. When the user presentation processing unit 252 receives the main OS list stored in the storage unit 931 in advance or the main OS list request unit 251 receives the main OS list, the user presentation processing unit 252 The sensor / built-in liquid crystal panel 301 of the display / light sensor unit 330 is displayed in a selectable manner by the user, and the main OS selected by the user is specified.

  At this time, the user presentation processing unit 252 generates display data including the main-side OS list, and transmits the generated display data to the data processing unit 730, whereby the display / light sensor unit 330 receives the main data. The side OS list is displayed.

  In addition, the user presentation processing unit 252 specifies “event” or “all” in the “data acquisition timing” field and acquires “coordinate data” indicating the position touched by the user, and the “data type” field. The sub display device command designating “coordinates” is transmitted to the data processing unit 730. If both coordinate data and partial image data (including coordinate data in a partial image) are desired, both “coordinate” and “partial image” are specified in the “data type” field.

  When the user touches the sensor built-in liquid crystal panel 301, the optical sensor circuit 32 of the sensor built-in liquid crystal panel 301 detects at least one of a shadow image and a reflected image of a finger or a pen touched by the user. The image data received by the data processing unit 730 from the sensor control unit 602 changes.

  As a result, the data processing unit 730 transmits the coordinate data to the user presentation processing unit 252 as a response to the command within the sub display device, so that the user presentation processing unit 252 indicates the coordinate data received from the data processing unit 730. The main OS selected by the user can be specified based on the location where the identification information of the main OS is displayed.

  When the main side OS list is displayed on the sensor built-in liquid crystal panel 301 of the display / light sensor unit 330 and a predetermined time elapses without selection from the user, the main side startup OS specification unit 25 displays the main display. By transmitting identification information of a predetermined main OS to the secondary boot loader activation unit 132 of the apparatus 210, the main display OS 210 is instructed to be activated. The predetermined main OS may be determined in advance or may be determined by a predetermined calculation.

(Processing flow until the OS starts on the main display device)
Next, with reference to FIG. 17, an example of a processing flow until the OS is started in the main display device in the two-screen display device 200 according to the present embodiment will be described. FIG. 17 is a flowchart illustrating an example of a processing flow until the OS is started on the main display device.

  Here, it is assumed that the sub display device with sensor 230 inquires of the main display device 210 about the identification information of the main side OS and causes the user to select the main side OS at the sub display device with sensor 230. .

  First, when the power of main display device 210 is turned on (YES in S11), power is supplied from wired communication unit 914 of main display device 210 to wired communication unit 934 of sub-display device with sensor 230 (S12). . Then, the main BIOS program P11 is executed (S13). As a result, the POST code is output to the POST code holding unit 18.

  Further, after the main-side BIOS program P11 is executed, the main-side boot loader B11 is read into the main-side main control unit 810 and executed (S14).

  On the other hand, as a result of power being supplied to the wired communication unit 934 of the sensor-equipped sub display device 230 in step S12, power is supplied to each unit of the sensor-equipped sub display device 230 (YES in S15). Thereby, the sub BIOS program P31 is executed (S16). Thereafter, the sub-side boot loader B31 is read into the sub-side main control unit 830 and executed (S17). Thereafter, the sub-side secondary boot loader B41 is read into the sub-side main control unit 830, and the sub-side OS program P41 is executed (S18). As a result, the sub-side OS is activated.

  Subsequently, the POST code checking unit 24 refers to the POST code output to the POST code holding unit 18 of the main display device 210, (A) whether the POST process has been normally completed, and (B) POST. It is determined whether the process is interrupted (S19). When it is determined that the POST process has not been completed normally until the end (POST process is interrupted) (NO in S20), POST code checking unit 24 displays / displays a predetermined message corresponding to the referred POST code. It is displayed on the optical sensor unit 330 (S21).

  On the other hand, when it is determined that the POST process has been normally completed to the end (YES in S20), main side OS list request unit 251 requests main side OS list from main display device 210 (S22).

  When main bootloader execution unit 13 receives the request (YES in S23), main OS list transmission unit 133 transmits the main OS list to main OS list request unit 251 (S24). On the other hand, the main-side boot loader execution unit 13 does not receive the above request (NO in S23), and a time-out is detected that a predetermined time has elapsed since the main-side boot loader B11 was read into the main-side main control unit 810. When the unit 131 detects (YES in S25), the secondary boot loader activation unit 132 selects the main side secondary boot loader (any one of B11 to B13) stored in the active partition as the main side. The main-side secondary boot loader reads the main-side OS program stored in the same partition and loads the main-side OS program into the main-side main control unit 810 to start the main-side OS (S26). ).

  On the other hand, when the main-side OS list request unit 251 receives the main-side OS list transmitted in step S <b> 24, the user presentation processing unit 252 displays the received main-side OS list and the sensor built-in liquid crystal of the optical sensor unit 330. It is displayed on the panel 301 (S27).

  When the user selects one of the main-side OSs displayed on the sensor built-in liquid crystal panel 301 by touching with a finger, a pen, or the like (YES in S28), the main-side startup OS specifying unit 25 performs the above operation. In order to instruct activation of the selected main-side OS, the identification information of the selected main-side OS is transmitted to the main display device 210 (S29).

  When the predetermined time has passed without selection from the user (YES in S30), the main-side startup OS designation unit 25 identifies the predetermined main-side OS in order to instruct the startup of the predetermined main-side OS. Information is transmitted to the main display device 210 (S31).

  When the main-side boot OS execution unit 13 instructs the main-side boot OS execution unit 13 to specify the main-side OS to be booted (that is, when the main-side OS identification information is received) (YES in S32). The secondary boot loader activation unit 132 operates in response to the reception, and determines a partition in which the main-side OS corresponding to the received identification information is stored as a partition to be referred to. Then, the main side secondary boot loader (any of B11 to B13) stored in the determined partition is read into the main side main control unit 810 to execute the main side secondary boot loader, and the main side OS Is activated (S33).

  The main-side boot loader execution unit 13 does not receive the identification information of the main-side OS to be booted from the main-side boot OS designation unit 25 (NO in S32), and the timeout detection unit 131 displays the main-side OS list. When the transmission unit 133 detects that a predetermined time has elapsed since the transmission of the main OS list to the main OS list request unit 251 (YES in S34), the secondary boot loader activation unit 132 is set to active. The main secondary boot loader (any of B11 to B13) stored in the partition being read is read into the main main control unit 810, and the read main secondary boot loader is stored in the same partition. The main-side OS is started by reading the main-side OS program into the main-side main control unit 810 ( 35).

[Modification]
In the above-described embodiment, the execution status of the BIOS of the main display device 210 is inspected by the sub display device with sensor 230, and the OS to be started up by the main display device 210 is designated from the sub display device with sensor 230. Although described by way of example, the present invention is applicable to other cases.

  For example, each of a plurality of users owns a sensor-equipped sub-display device 230 stored in a state where the environment settings of the main-side OS are uniquely customized, and each user owns a sensor-equipped sub-display device 230 A case where the main display device 210 is attached to the main display device 210 is used. In this case, when the OS is activated on the main display device 210, the environment setting may be designated for the OS to be activated from the sub display device with sensor 230.

  Similarly, for example, each of the plurality of users owns the sensor-equipped sub display device 230 in which the login password of the main-side OS is set, and each user owns the sensor-equipped sub display device 230. A case where the display device 210 is used while being mounted is conceivable. In this case, when the OS is started on the main display device 210, the login password may be specified from the sensor-equipped sub display device 230.

[Additional Notes]
In the above-described embodiment, the configuration in which the sub display device with sensor 230 is removable from the main display device 210 has been described. However, the sub display device with sensor 230 may be configured integrally with the main display device 210. .

  In the above-described embodiment, the configuration including the data processing unit 730 and the display / light sensor circuit control unit 630 has been described as the configuration of the sub-display device with sensor 230. However, the configuration is not limited to this configuration. For example, the sub-side main control unit 830 may directly control the display / light sensor circuit control unit 630, and the sub-side main control unit 830 may generate whole image data / partial image data / coordinate data. . The sub-side main control unit 830 may directly control the display / light sensor unit 330.

  In the above-described embodiment, the BIOS program is described as an example of the first process, but the present invention is not limited to this, and may be, for example, an environment acquisition program that acquires a temperature or the like.

  Finally, each block of the main display device 210 and the sensor-equipped sub display device 230, in particular, the main side main control unit 810, the sub side main control unit 830, the data processing unit 730, and the display / light sensor circuit control unit 630, It may be configured by wear logic, or may be realized by software using a CPU as follows.

  That is, the main display device 210 and the sensor-equipped sub display device 230 develop a CPU (central processing unit) that executes instructions of a control program for realizing each function, a ROM (read only memory) that stores the program, and the program. A random access memory (RAM), and a storage device (recording medium) such as a memory for storing the program and various data. The object of the present invention is to read the program code (execution format program, intermediate code program, source program) of the control program for the main display device 210 and the sensor-equipped sub display device 230, which is software for realizing the above-described functions, by a computer It is also achieved by supplying the recording medium recorded as possible to the main display device 210 and the sensor-equipped sub display device 230, and the computer (or CPU or MPU) reads out and executes the program code recorded on the recording medium. Is possible.

  Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the main display device 210 and the sub display device with sensor 230 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), IEEE802.11 radio, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  The present invention can be applied to an information processing apparatus including a main device (main display device) and a sub device (sub display device with a sensor). In particular, the present invention can be suitably applied to a notebook computer equipped with a main display and a sub display.

1, showing an embodiment of the present invention, is a block diagram illustrating a detailed configuration of a main display device and a sensor-equipped sub display device. FIG. FIG. 4A is a schematic diagram showing a specific example of a two-screen display device, and FIG. 4B shows a main display device and a sensor in the specific example of the two-screen display device shown in FIG. It is a schematic diagram which shows the state which isolate | separated the attached sub display apparatus. It is a figure which shows typically the cross section of the liquid crystal panel with a sensor with which the said sub display apparatus with a sensor is provided. FIG. 4A is a schematic diagram showing how the position touched by the user is detected by detecting the reflected image, and FIG. 4B is the position touched by the user by detecting the shadow image. It is a schematic diagram which shows a mode that it detects. It is a block diagram which shows the principal part structure of the said main display apparatus and a sub display apparatus with a sensor. It is a block diagram which shows the more detailed structure of the display / light sensor part with which the said sub display apparatus with a sensor is equipped, a display / light sensor circuit control part, and a data processing part. It is a block diagram which shows the structural example of the display part with which the said sub display apparatus with a sensor is equipped, and a display circuit control part. It is a figure which shows typically an example of the frame structure of the command in a sub display apparatus. It is a figure explaining an example of the value which can be specified to each field contained in the command in a sub display apparatus, and the outline | summary. It is a schematic diagram which shows an example of the frame structure of the command between apparatuses. It is explanatory drawing which shows the information which can be designated to each field about each of a scan command, a display command, and a process command. It is explanatory drawing of an example of the value which can be designated to the 1st to 5th field of a scan command, and its outline. It is explanatory drawing of an example of the value which can be designated to the 1st and 2nd field of a display command, and its outline | summary. It is explanatory drawing of an example of the value which can be designated to the 1st-3rd field of a process command, and its outline | summary. FIG. 6A is a diagram showing an example of image data obtained as a result of scanning the entire sensor-equipped liquid crystal panel when the object is not placed on the sensor-equipped liquid crystal panel. FIG. 4B is a diagram illustrating an example of image data obtained as a result of scanning the entire sensor-equipped liquid crystal panel when the user touches the sensor-equipped liquid crystal panel with a finger. It is a block diagram which shows the principal part of the said display / light sensor part, especially the structure of a liquid crystal panel with a built-in sensor, and the structure of a peripheral circuit. It is a flowchart which shows an example of the flow of a process until OS starts in the said main display apparatus.

Explanation of symbols

13 Main side boot loader execution part (execution control means)
14 Main-side secondary boot loader execution unit 15 Main-side OS startup unit 18 POST code holding unit 21 Sub-side boot loader execution unit 22 Sub-side secondary boot loader execution unit 23 Sub-side OS startup unit 24 POST code checking unit (execution status) Judgment means)
25 Main side boot OS designation part (execution designation means)
133 Main-side OS list transmission unit 200 Two-screen display device (information processing device)
210 Main display device (processing execution unit)
230 Sub-display with sensor (processing control unit)
251 Main-side OS list request section (candidate processing information acquisition means)
252 User presentation processing unit (candidate process selection means)
P11 Main BIOS program (first process)
B11 to B13 Secondary boot loader on the main side (second process, candidate process)
P21 to P23 Main OS program (second process, candidate process)

Claims (14)

While executing the first process, it generates execution status information indicating the execution status, and can communicate with the process execution unit that executes the second process after the first process. A processing control unit operable independently,
Execution status determination means for acquiring the generated execution status information from the process execution unit and determining whether the acquired execution status information is a predetermined value;
When the execution status determination unit determines that the acquired execution status information is the predetermined value, one of a plurality of candidate processes that can be executed in the process execution unit is executed as the second process. The process control unit further comprises execution designation means for designating the process execution unit. The execution designation means further includes candidate process information acquisition means for acquiring candidate process information indicating each of the plurality of candidate processes from the process execution unit,
The process control unit according to claim 1, wherein the execution designation unit transmits information designating any one of the acquired candidate process information to the process execution unit. The execution specifying means further includes candidate process selection means for presenting the candidate process information acquired by the candidate process information acquisition means to the user and allowing the user to select any one of the presented candidate process information,
The process control unit according to claim 2, wherein the execution designating unit transmits information designating the selected candidate process information to the process execution unit.   The execution specifying means transmits information specifying the predetermined candidate process information to the process execution unit when the candidate process information presented by the candidate process selection means is not selected within a predetermined time. 4. The process control unit according to claim 3, wherein The process execution unit executes a basic input output system (BIOS) as the first process, and then executes any one of a plurality of operating systems executable by the unit as the second process. And
5. The process control unit according to claim 1, wherein the execution designation unit designates any one of the operating systems as the second process. 6. While executing the first process, it generates execution status information indicating the execution status, and after the first process, any one of a plurality of candidate processes that can be executed by the own unit is set as the second process. A process execution unit that can communicate with the process control unit,
Receiving execution designation of the second process from the process control unit capable of reading the execution status information, and including execution control means for executing the designated second process after the first process. Feature processing execution unit.   The execution control means executes the predetermined second process when the designation of the second process is not received from the process control unit within a predetermined time after the execution of the first process. 6. The process execution unit according to 6.   An information processing apparatus in which the process control unit according to claim 1 and the process execution unit according to claim 6 are connected by wire. While executing the first process, it generates execution status information indicating the execution status, and can communicate with the process execution unit that executes the second process after the first process. A control method for a processing control unit that can operate independently,
An execution status determination step of acquiring the generated execution status information from the process execution unit and determining whether the acquired execution status information is a predetermined value;
If it is determined in the execution status determination step that the acquired execution status information is the predetermined value, any one of a plurality of candidate processes that can be executed in the process execution unit is executed as the second process. As described above, a method for controlling a process control unit, comprising: an execution designation step that designates the process execution unit. While executing the first process, the execution status information indicating the execution status is generated, and after the first process, any one of a plurality of candidate processes that can be executed by the own unit is set as the second process. A method for controlling a process execution unit to be executed and capable of communicating with the process control unit,
Receiving an instruction to execute the second process from a process control unit capable of reading the execution status information, and including an execution control step for executing the specified second process after the first process. A method for controlling a processing execution unit.   A control program for causing a computer to function as the processing control unit according to any one of claims 1 to 5, wherein the control program causes the computer to function as each of the above means.   A control program for causing a computer to function as the processing execution unit according to claim 6 or 7, wherein the control program causes the computer to function as each of the above means.   The computer-readable recording medium which recorded the control program of Claim 11.   A computer-readable recording medium on which the control program according to claim 12 is recorded.

Images