JP2006185235A - Information processing system and information processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processing system that can execute information processing without equipment constraints while maintaining the portability of a portable information terminal. <P>SOLUTION: The information processing system comprises a first portable information terminal 2 and an information processing apparatus 1. The first portable information terminal 2 is carried by a first user and connected to a first display device 91 on the way. The information processing apparatus 1 is connected to the first portable information terminal 2 via a network 5. The first portable information terminal 2 acquires first display device information about the first display device 91 from the first display device 91, and sends first terminal information about the first portable information terminal 2 and the first display device information to the information processing apparatus 1. The information processing apparatus 1 sends first information capable of being output by the first display device 91 and second information capable of being output by the first portable information terminal 2 to the first portable information terminal 2. The first portable information terminal 2 outputs the second information, and sends the first information to the first display device 91. The first display device 91 outputs the first information. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an information processing system and an information processing method, and more particularly to an information processing system and an information processing method using a portable information terminal.

  Mobile information terminals such as mobile phones, PDAs (Personal Digital Assistants), and mini-notebook computers are small and excellent in portability. However, there are restrictions on the size of a display device such as a display, the size of an input device such as a keyboard, and the capacity of a storage device such as a memory. Therefore, the function may be limited due to these restrictions. There is a demand for a technique that can maintain the portability of a portable information terminal and that is not subject to device restrictions.

  As a related technique, Japanese Patent Application Laid-Open No. 2003-319097 discloses a two-screen linked video distribution system and apparatus. The information processing apparatus includes means for receiving information including first and second types of data and synchronization information of the data, a storage unit for the data, a clock for counting the current time, a display unit, It has a processing part. The processing unit converts the first information based on the means for separating the two types of data, the means for decoding the first information displayed on the display unit, and the current time and the synchronization information. Controlling means for displaying on the display unit and means for outputting the second information to the external processing device is provided.

  As a related technique, for example, a large display device that can be used by the public (Japanese Patent Laid-Open No. 2004-166219) or a large display device connected to a network (Japanese Patent Laid-Open No. 2002-108752) is used. There is technology.

  Japanese Unexamined Patent Application Publication No. 2004-166219 discloses a public display device, an image providing system using the public display device, and an information processing system for the public display device. The public display device includes image display means, near field communication control means, first output means, and second output means. The image display means is installed in a publicly available place and has a display screen larger than the display screen of the small portable terminal. The first output means reproduces the first image data received via the near field communication control means and outputs it to the image display means. The second output means reproduces the second image data received via the reception control means or stored in the storage means and outputs the second image data to the image display means. Then, a communication connection is established with an unspecified small portable terminal via the short-range communication control means. The first image data is received from the small portable terminal via the short-range communication control means, and the received first image data is reproduced and output to the image display means. At the same time, the reproduction of the second image data output in advance based on the communication connection with the small portable terminal, the reception of the first image data, or the reproduction of the first image data is stopped.

  Japanese Patent Application Laid-Open No. 2002-108752 discloses a network system, a method for using other functions in the system, and a storage medium. In this network system, a plurality of service providing apparatuses that process predetermined data and perform predetermined service functions, a server, a first terminal apparatus, and a second terminal apparatus are connected via a network. A plurality of connections can be used simultaneously from the server, the first terminal device, and the second terminal device. Service providing apparatus specifying means, connection setting means, and data transmitting means are provided. The service providing device specifying unit specifies a service providing device from among the plurality of service providing devices in response to a request from at least one of the first terminal device and the second terminal device. The connection setting means sets a new connection between the service providing apparatus specified by the service providing apparatus specifying means and the first terminal apparatus. The data transmitting unit transmits the predetermined data from the first terminal device to the service providing device through the new connection set by the connection setting unit.

  In addition, the following screens of in-vehicle devices (JP 2003-244343), notebook PC screens (JP 2000-31144), dedicated independent screens (JP 11-288253, JP 2003). -2739982) is available. Regarding the keyboard, there is a technique using a projected keyboard (Japanese Patent Laid-Open No. 2000-89899) as follows.

  Japanese Patent Laid-Open No. 2003-244343 discloses a display device, a portable terminal, and an information display system. The display device includes an interface unit, a control unit, and a display unit. The interface means is connected to the mobile terminal and inputs information to be displayed on the screen of the mobile terminal. The control means converts the information into display data. The display means is larger in size than the screen of the portable terminal that displays the display data.

  Japanese Unexamined Patent Application Publication No. 2000-31144 discloses a portable information terminal device. The portable information terminal device includes a display device, an input device, a communication device, and a control board. The display device can display and browse at least information. The input device can input characters and commands. The communication device can transmit and receive information to and from a mobile telephone terminal for performing communication using a public line. The control board controls the entire apparatus. The portable information terminal device has a structure connectable to the mobile phone terminal. In the portable information terminal device, the connection part can be exchanged.

  Japanese Patent Application Laid-Open No. 11-288253 discloses a liquid crystal display device and a method of using the same. The liquid crystal display device includes at least a liquid crystal screen for displaying documents, moving images, and still images, a driver circuit for the liquid crystal screen, a first image memory, and a controller circuit. The liquid crystal display device can be connected to an external device capable of generating digital image data via a digital signal cable capable of transmitting moving images. The controller circuit includes first means, second means, and third means. The first means reads the image data from the first image memory and displays it on the liquid crystal screen. The second means displays the digital image data sent from the external device to the liquid crystal display device via the digital signal cable on the liquid crystal screen. The third means writes the digital image data sent from the external device to the liquid crystal display device via the digital signal cable directly into the first image memory.

  Japanese Unexamined Patent Application Publication No. 2003-273882 discloses a mobile phone terminal and an external display device. This external display device for a mobile phone terminal is fixed to at least both side surfaces of a panel portion having a display screen larger than the display portion of the mobile phone terminal on one side and the casing of the mobile phone terminal provided on the other side of the panel portion. And a coupling member protruding from the other surface of the panel portion. The panel portion can be attached so as to cover at least a part of the display portion of the mobile phone terminal.

  Japanese Unexamined Patent Publication No. 2000-89899 discloses a virtual display / keyboard and a method for generating the same. This method includes the following steps (a) to (c) and generates a virtual display and a keyboard for a computer. (A) projecting a virtual display of data representing a computer display on a substantially flat and light colored surface; (b) projecting a virtual keyboard pattern of keys on said substantially flat and light colored surface; (C) A remote detection means for determining which virtual key in the lower-layer keyboard is selected is provided.

JP 2003-319097 A JP 2004-166219 A JP 2002-108752 JP 2003-244343 A JP 2000-311144 A JP-A-11-288253 JP 2003-273882 A JP 2000-89899 A

  Accordingly, an object of the present invention is to provide an information processing system and an information processing method capable of performing information processing without being restricted by the device while maintaining the portability of the portable information terminal.

  Another object of the present invention is to provide an information processing system and information processing method capable of exchanging information with the same convenience as a larger display device while maintaining the portability of a portable information terminal. It is to provide.

  Still another object of the present invention is to provide an information processing apparatus and an information processing method that are capable of exchanging information with a portable information terminal having functions equivalent to those of a larger input device.

  Hereinafter, means for solving the problem will be described using the numbers and symbols used in the best mode for carrying out the invention. These numbers and symbols are added in parentheses in order to clarify the correspondence between the description of the claims and the best mode for carrying out the invention. However, these numbers and symbols should not be used for interpreting the technical scope of the invention described in the claims.

Therefore, in order to solve the above problems, the information processing system of the present invention includes the first portable information terminal (2-1) and the information processing apparatus (1). The first portable information terminal (2-1) is carried by the first user and can be connected to the first display device (91) at the destination of the first user. The information processing device (1) is communicably connected to the first portable information terminal (2-1) via the network (5). The first portable information terminal (2-1) obtains the first display device information related to the first display device (91) from the first display device (91), and the first portable information terminal (2-1) relates to the first portable information terminal (2-1). The terminal information and the first display device information are transmitted to the information processing device (1). The information processing device (1) displays the first information on the first display device (91) for the stored first information and second information based on the first display device information and the first terminal information. The second information is converted so that it can be output by the first portable information terminal (2-1), and transmitted to the first portable information terminal (2-1). The first portable information terminal (2-1) outputs the second information and transmits the first information to the first display device (91) so that the first display device (91) displays the first information. . However, the first display device (91) is larger than the display device built in the portable information terminal (2-1).
In the present invention, since the large first display device (91) at the destination of the first user is used, the same convenience as in the case of using the large display device can be obtained without carrying around the large screen display device. . That is, it is possible to perform information processing without being restricted by devices while maintaining the portability of the portable information terminal. Such a large-screen display device does not need to be a public device, and may be a PC in a destination office of the first user or a friend's PC. Further, the display device itself does not need to be connected to the network.

  In the information processing system, the first information is stored in advance in the information processing apparatus (1). The first portable information terminal (2-1) transmits a first request signal including a request for the first information to the information processing device (1). Based on the first request signal, the information processing device (1) transmits the first information converted to be displayable by the first display device (91) to the first portable information terminal (2-1).

In the information processing system, the first portable information terminal (2-1) includes a storage unit (72). The storage unit (72) includes a first storage area (72d) preset for input / output of first information and a second storage area (72a / 72b / 72c preset for input / output of second information). ).
In the present invention, by dividing the memory area in advance, it is possible to accurately and efficiently process different types of data (example: voice, image, key operation, external display device of portable information terminal). it can.

In the above information processing system, it is portable to the second user, can be connected to the second display device (91) at the destination of the second user, and can communicate with the information processing device (1) via the network (5). And a second portable information terminal (2-2) connected to the mobile phone. The second portable information terminal (2-2) acquires the second display device information relating to the second display device (91) from the second display device (91), and the second relating to the second portable information terminal (2-2). The terminal information and the display device information are transmitted to the information processing device (1). The information processing device (1) displays the third information on the second display device (91) for the stored third information and fourth information based on the second terminal information and the second display device information. The fourth information is converted to be outputable by the second portable information terminal (2-2) and transmitted to the second portable information terminal (2-2). The second portable information terminal (2-2) outputs the fourth information and transmits the third information to the second display device (91) so that the second display device (91) displays the third information. .
In the present invention, not only the first portable information terminal (2-1) but also a plurality of second portable information terminals (2-2) use a large display device without carrying a large-screen display device. Equivalent convenience can be obtained.

In the information processing system, the third information is stored in advance in the information processing apparatus (1). The first portable information terminal (2-1) or the second portable information terminal (2-2) receives a second request signal including a request to output the third information to the second portable information terminal (2-1), It transmits to a processing apparatus (1). Based on the second request signal, the information processing device (1) transmits the third information converted to be displayable by the second display device (91) to the second portable information terminal (2-2).
In the present invention, between the first portable information terminal (2-1) and the second portable information terminal (2-2) using the large second display device (91) at the destination of the second user, Information can be exchanged via the information processing device (1). Thereby, various usage forms such as a conference between a plurality of portable information terminals can be implemented.

In the information processing system, the second information is received by the first user's video taken by the camera (70) of the first portable information terminal (2-1) or the first portable information terminal (2-1). The voice of the first user. The fourth information is an image of the second user taken by the camera (70) of the second portable information terminal (2-2) or the voice of the second user received by the second portable information terminal (2-2). is there.
According to the present invention, the information processing apparatus (1) is interposed between the first portable information terminal (2-1) and the second portable information terminal (2-2) while recognizing both users by video and voice. Various information can be exchanged.

In the information processing system, the information processing apparatus (1) associates the terminal information including the first terminal information with the handicap information indicating an unfavorable condition when receiving the information processing service regarding the user including the first user. And a database (14) for storing. When converting the first information, the information processing apparatus (1) refers to the database (14) and converts the first information so as to be adapted to the handicap information.
According to the present invention, each user can receive a service from the information processing apparatus (1) regardless of the disadvantageous conditions (eg, foreigner, amblyopia, blind) of each user.

In the information processing system, the information processing apparatus (1) belongs to the first user.
According to the present invention, the environment of a PC that a user normally uses can be reproduced on the portable information terminal (1).

  In order to solve the above problems, the information processing apparatus of the present invention includes a transmission / reception unit (85), a first data processing unit (12-5), and a second data processing unit (12-3). The transmission / reception unit (85) is transmitted via the network from the first portable information terminal (2-1) that is carried by the first user and connected to the first display device (91) at the destination of the first user. The first display device information relating to the first display device (91) and the first terminal information relating to the first portable information terminal (2-1) are received. Based on the first display device information, the first data processing unit (12-5) converts the stored first information into first conversion information that can be displayed on the first display device (91). Based on the first terminal information, the second data processing unit (12-3) converts the stored second information into second conversion information that can be output by the first portable information terminal (2-1). The transmission / reception unit (85) transmits the first conversion information and the second conversion information to the first portable information terminal (2-1) via the network.

  In the above information processing apparatus, a first storage area (82d) set in advance for input / output of first information and a second storage area (82b, 82c) set in advance for input / output of second information are provided. A storage unit (82) is further included. The first data processing unit (12-5) converts the first information into the first conversion information based on the first request signal including the request for the first information transmitted from the first portable information terminal (2-1). The converted data is stored in the first storage area (82d). The transmission / reception unit (85) acquires the first conversion information from the first storage area (82d) and transmits the first conversion information to the first portable information terminal (2-1).

  In the information processing apparatus, the transmission / reception unit (85) is transmitted from the second portable information terminal (2-2) that is carried by the second user and connected to the second display device (91) at the destination of the second user. The second display device information related to the second display device (91) and the second terminal information related to the second portable information terminal (2-2) transmitted via the network are received. The first data processing unit (12-5) converts the stored third information into third conversion information that can be displayed on the second display device (91) based on the second display device information. Based on the second terminal information, the second data processing unit (12-3) converts the stored fourth information into fourth conversion information that can be output by the second portable information terminal (2-2). The transmission / reception unit (85) transmits the third conversion information and the fourth conversion information to the second portable information terminal (2-2) via the network.

  In the information processing apparatus, the third information is stored in the first storage area (82d), and the fourth information is stored in the second storage area (82b, 82c). The first data processing unit (12-5) is third to the second portable information terminal (2-2) transmitted from the first portable information terminal (2-1) or the second portable information terminal (2-2). Based on the second request signal including a request to output information, the third information is converted into third conversion information and stored in the first storage area (82d). The transmission / reception unit (85) acquires the third conversion information from the first storage area (82d) and transmits it to the second portable information terminal (2-2).

  In the information processing apparatus, the second information is the first user's video captured by the camera of the first portable information terminal (2-1) or the first information received by the first portable information terminal (2-1). It is the voice of the user. The fourth information is a video of the second user taken by the camera of the second portable information terminal (2-2) or the voice of the second user received by the second portable information terminal (2-2).

In the information processing apparatus, a database (14) for storing terminal information including first terminal information and handicap information indicating disadvantageous conditions when receiving an information processing service related to a user including the first user in association with each other; Is provided. When converting the first information into the first conversion information, the first data processing unit (12-5) refers to the database (14) and converts the first information so as to be adapted to the handicap information.
Information processing device.

  In the information processing apparatus, when the second data processing unit (12-3) converts the second information into the second conversion information, the second data processing unit (12-3) refers to the database (14) to adapt to the handicap information. 2 Convert information.

  In order to solve the above problems, an information processing method according to the present invention includes: (a) a first display device in which a first portable information terminal (2-1) carried by a first user is at a destination of the first user; A step of connecting to (91); and (b) a step of the first portable information terminal (2-1) acquiring first display device information relating to the first display device (91) from the first display device (91). (C) The first portable information terminal (2-1) transmits the first terminal information and the first display device information regarding the first portable information terminal (2-1) via the network (5) to the information processing device. A step of transmitting to (1), and (d) a first display format of the first display device (91) by using the first information stored in the information processing device (1) based on the first display device information. To the first portable information terminal (2-1) via the network (5) A step of signal, and a step of displaying the first information received in (e) a first display device (91) comprises a first portable information terminals (2-1).

  In the information processing method, (d) step includes (d1) menu information indicating a menu of a service provided by the information processing device (1) based on the first terminal information. A step of converting the portable information terminal (2-1) to be outputable and transmitting it to the first portable information terminal (2-1) via the network (5); and (d2) a first portable information terminal ( 2-1) outputting menu information; (d3) transmitting selected menu information selected from the menu information to the information processing device (1) via the network (5); Based on the one display device information, the first information related to the selection menu information is converted to correspond to the first display format of the first display device (91), and the first portable information is transmitted via the network (5). Terminal (2-1) And transmitting.

  In the information processing method, (c) step includes (c1) the first portable information terminal (2-1) issuing a connection request indicating a connection request in addition to the first terminal information and the first display device information. After the step of transmitting to the information processing device (1) and (c2) connection with the information processing device (1) is established, the first portable information terminal (2-1) sends a request for the first information to the information processing device. Transmitting to (1).

  In the above information processing method, (f) the second portable information terminal (2-2) carried by the second user connects to the second display device (91) at the destination of the second user; (G) the second portable information terminal (2-2) acquiring second display device information relating to the second display device (91) from the second display device (91); and (h) the second portable information terminal. (2-2) transmitting the second terminal information and the second display device information regarding the second portable information terminal (2-2) to the information processing device (1) via the network (5); (I) Based on the second display device information, the information processing device (1) converts the stored second information so as to correspond to the second display format of the second display device (91), Transmitting to the second portable information terminal (2-2) via the network (5); ) The second display device (91) further comprises a step of displaying the second information received in the second portable information terminal (2-2).

  In the above information processing method, (k) the first portable information terminal (2-1) or the second portable information terminal (2-2) sends the third information to the information processing device (1) via the network (5). And (l) the information processing device (1) sends the third information to the second portable information terminal (2-2) or the first portable information terminal (2-1) via the network (5). And the step of transmitting to (1).

  In the information processing method, (l) step includes: (l1) the information processing device (1) converts the third information so as to correspond to the second display format or the first display format; ) The second display device (91) displays the third information received by the second portable information terminal (2-2), or the first display device (91) receives the first portable information terminal (2-1). Displaying the third information.

  In the above information processing method, the third information includes the first user's video captured by the first portable information terminal (2-1), the first user's voice received by the first portable information terminal (2-1), The image of the second user taken by the second portable information terminal (2-2) or the voice of the second user received by the second portable information terminal (2-2) is included.

  In the information processing method, (d) step includes (d1) disadvantageous conditions when receiving the information processing service related to the terminal information including the first terminal information and the user including the first user when converting the first information. The database includes a step of converting the first information so as to be adapted to the handicap information with reference to a database (14) that stores the handicap information in association with the handicap information.

  In order to solve the above-mentioned problem, a program of the present invention includes: (m) a first portable information terminal (m) carried by a first user and connected to a first display device (91) at a location where the first user is moving. (2-1) receiving the first display device information relating to the first display device (91) and the first terminal information relating to the first portable information terminal (2-1) transmitted via the network; n) based on the first display device information, converting the stored first information into first conversion information that can be displayed on the first display device (91); and (o) based on the first terminal information. Converting the stored second information into second conversion information that can be output by the first portable information terminal (2-1); and (p) converting the first conversion information and the second conversion information to the first portable information terminal. Transmitting to the information terminal (2-1) via the network. Executing the method of Bei the computer.

  In the above program, the (m) step converts (m1) menu information indicating a menu of the service to be provided based on the first terminal information so that the first portable information terminal (2-1) can output it, Transmitting to the first portable information terminal (2-1) via the network (5); and (m2) selecting menu information selected from the menu information from the first portable information terminal (2-1) to the network ( 5), and (m3) obtaining first information related to the selected menu information.

  In the above program, the (n) step performs the first conversion on the first information based on the first request signal including the request for the first information transmitted from the (n1) first portable information terminal (2-1). A step of converting into information and storing the information in a first storage area (82d) set in advance in the storage unit (82). Here, the first storage area (82d) is different from the second areas (82b, 82c) set in advance for storing the second information. The step (p) includes the step (p1) of acquiring the first conversion information from the first storage area (82d) and transmitting it to the first portable information terminal (2-1).

  In the above program, (q) transmitted from the second portable information terminal (2-2) connected to the second display device (91) carried by the second user and connected to the second user via the network. Receiving the second display device information related to the second display device (91) and the second terminal information related to the second portable information terminal (2-2); and (r) based on the second display device information, A step of converting the stored third information into third conversion information that can be displayed on the second display device (91); and (s) second stored fourth information based on the second terminal information. A step of converting into fourth conversion information that can be output by the portable information terminal (2-2); and (t) the third conversion information and the fourth conversion information to the second portable information terminal (2-2) via the network. And transmitting.

  In the above program, the (r) step is (r1) the second portable information terminal (2-2) transmitted from the first portable information terminal (2-1) or the second portable information terminal (2-2). The step of converting the third information into the third conversion information based on the second request signal including the request for outputting the third information and storing it in the preset third storage area (82d) in the storage unit (82). Is provided. Here, the third storage area (82d) is different from the fourth areas (82b, 82c) set in advance for storing the fourth information. The step (t) includes the step (t1) of acquiring the third conversion information from the third storage area and transmitting it to the second portable information terminal (2-2).

  In the above program, the second information is the image of the first user taken by the camera of the first portable information terminal (2-1) or the first user received by the first portable information terminal (2-1). It is voice. The fourth information is a video of the second user taken by the camera of the second portable information terminal (2-2) or the voice of the second user received by the second portable information terminal (2-2).

  In the above program, the (n) step is (n2) when receiving the information processing service regarding the terminal information including the first terminal information and the user including the first user when converting the first information into the first conversion information. A step of converting the first information so as to adapt to the handicap information is provided with reference to a database (14) that associates and stores the handicap information indicating an unfavorable condition.

  In the above program, the step (o) includes the step (o1) of converting the second information so as to adapt to the handicap information with reference to the database when the second information is converted into the second conversion information. .

  In order to solve the above-mentioned problem, the program of the present invention detects (u) the connection with the first display device (91), and sets the first display device information about the first display device (91) as the first. A step of acquiring from the display device (91), (v) a step of transmitting the first display device information to the information processing device (1) via the network (5), and (w) a first display device (91). Receiving the first information converted to correspond to the first display format from the information processing apparatus (1) via the network (5), and (x) receiving the received first information as the first information. And causing the computer to execute a method comprising the step of transmitting to the display device (91).

  In the program described above, the step (v) includes (v1) a step of transmitting the first display device information to the information processing device (1), and (v2) after establishing a connection with the information processing device (1). And a step of transmitting a request for the first information to the information processing apparatus (1).

  In the above program, the (w) step includes (w1) storing the first information in the preset first storage area (72d) of the storage unit (72) and storing the second information in the first storage area (72d). And storing in a preset second storage area (72b, 72c) different from the above.

  Note that the exchange of information in the information processing method and the program described above is merely an example. If the mobile terminal 2 and the display device 91 operate simultaneously in association with the service of the information processing apparatus 1, other services ( It is also possible to apply to transmission / reception of other data.

  According to the present invention, it is possible to perform information processing without being restricted by the device while maintaining the portability of the portable information terminal.

  First, the outline of the information processing system of the present invention will be described. 1A and 1B are schematic views showing an information processing system of the present invention. In FIG. 1A, the mobile terminal 2 connected to the display device 91 is connected to the information processing device 1 via the network 5 including the base station 5a. In FIG. 1B, the mobile terminal 2 connected to the display device 91 attached to the PC is connected to the information processing device 1 via the network 5 including the base station 5a. In this case, the mobile terminal 2 may be connected to the display device 91 via the PC main body (displayed by a solid line) or directly connected to the display device 91 (displayed by a broken line).

  The user accesses the information processing apparatus 1 through the network 5 including the base station 5a after connecting the portable terminal 2 to an appropriate display device 91 at the user's destination. The information processing apparatus 1 displays a menu screen on the mobile terminal 2 and proposes various services to the user. The user selects a desired service from them. The information processing apparatus 1 transmits information related to the selected service to the mobile terminal 2 so that the display unit, the speaker, the microphone, and the camera of the mobile terminal 2 and the display apparatus 91 simultaneously operate together. Thereby, the information processing system of the present invention can provide more diverse and advanced services from the information processing apparatus 1 to the user (mobile terminal 2) via the network 5. If the user carries the mobile terminal 2, the user can receive the service by using a nearby display device without carrying the display device 91. At that time, the portable terminal 2 does not need to have a driver for the display device 91. This is because the information processing apparatus 1 functions as a driver. Accordingly, the type of the display device 91 to be used is not limited.

(First embodiment)
Hereinafter, a first embodiment of an information processing system and an information processing method of the present invention will be described with reference to the accompanying drawings. First, the configuration of the embodiment of the information processing system of the present invention will be described.

  FIG. 2 is a block diagram showing the configuration of the first embodiment of the information processing system of the present invention. The information processing system includes an information processing apparatus 1 and a mobile terminal 2-i (i = 1 to n: n is a natural number). The information processing apparatus 1, the mobile terminal 2, and the mobile terminal 3 are connected to each other via a network 5 so that bidirectional communication is possible.

  The information processing apparatus 1 is an information processing apparatus exemplified by a workstation or a server. An information processing function using the mobile terminal 2 is provided via the network 5. The information processing apparatus 1 includes a connection management unit 11, a data control unit 12, a material database 13, a participant database 14, a connected device database 15, a menu database 18, and a recording database 19 as programs for performing information processing.

  The material database 13 stores the identification data of the target person who receives the information processing service of the information processing apparatus 1 and the material data to be referenced (utilized) in association with each other. When the information processing service is a TV conference using a mobile terminal, the material data is exemplified by a material file (hereinafter also referred to as “material data file”) used for the conference. The document data file is transmitted to the information processing apparatus 1 in advance by the user of the portable terminal. The identification data is exemplified by the mobile terminal ID of the TV conference participant.

  The participant database 14 stores identification data of a subject who receives the information processing service of the information processing apparatus 1 and information related to the subject in association with each other. In the case of a TV conference, the data is exemplified by the name of a participant who attends the conference and its portable terminal ID. The user of the mobile terminal transmits the name and the mobile terminal ID to the information processing apparatus 1 in advance. The mobile terminal ID and information related to the mobile terminal (example: owner's name as a participant) are stored in association with each other.

  The connected device database 15 stores identification data and display characteristics of a connected device (display device) that can be connected to the mobile terminal in association with each other. Here, the display characteristics are exemplified by the number of displayable pixels (resolution), the number of displayable colors, and the scanning method. By storing the data of connected devices, it is possible to support not only existing display devices but also devices with other characteristics in the future and operation methods in new applications. .

  The menu database 18 stores the types of services provided by the information processing apparatus 1 and the contents of information processing performed by the information processing apparatus 1 in association with each other. Here, the types of services are exemplified by TV conferences, restaurant menus, and sports broadcasts. The contents of information processing are exemplified by speech translation, character conversion, and drawing extraction.

The recording database 19 stores an execution record of services provided by the information processing apparatus 1. The execution record includes the type of data (example: image data, character data, document data, voice data) and the data output source (example: database via the information processing device 1, mobile terminal 2-i (i = 1 to n). )) Etc. and stored.
The data control unit 12 stores the operation of each unit and data to be transmitted and received in the recording database 19 at appropriate time intervals. Thereby, progress of operation | movement of the data control part 12 can be grasped | ascertained later.

  When the information processing apparatus 1 performs information processing, the connection management unit 11 establishes a connection with the related mobile terminal 2-i (i = 1 to n). When the information processing is finished, the connection is finished.

  When the information processing apparatus 1 performs information processing, the data control unit 12 controls data transmission / reception and data processing between the portable terminal 2-i (i = 1 to n) and the information processing apparatus 1. The data control unit 12 includes a data recognition unit 12-1, a voice data processing unit 12-2, a built-in display data processing unit 12-3, an operation data processing unit 12-4, and a display data processing unit 12 as programs for performing data processing. -5 and a search unit 12-6. Each program is executed, for example, in a time-sharing manner.

  The data recognition unit 12-1 recognizes the type of data transmitted from each database or mobile terminal (2-i: transmission source). Based on the type of data, the data is stored in an appropriate area (example: 82a to 82e) in the memory 82 (described later).

  The voice data processing unit 12-2 performs data processing as necessary on data related to voice transmitted from each database or portable terminal (2-i: transmission source). The data is transmitted to the portable terminal (2-j (j = 1 to n, j ≠ i): transmission destination). Data processing is exemplified by processing for converting data so that it can be output by a portable terminal at the transmission destination, speech translation processing, and processing for converting character data to speech data. Data related to sound is input / output via the area 82 b of the memory 82.

  The built-in display data processing unit 12-3 performs data processing on data displayed on the display unit of the mobile terminal transmitted from each database or mobile terminal (2-i: transmission source) as necessary. The data is transmitted to the portable terminal (2-j (j = 1 to n, j ≠ i): transmission destination). Data processing includes processing to convert data so that it can be output on the mobile terminal of the transmission destination, processing to extract character data in the data, processing to easily convert character data, processing to convert character data into another language Is exemplified. Data to be displayed on the display unit is input / output via the area 82 c of the memory 82.

  The operation data processing unit 12-4 performs data processing on the data related to the operation transmitted from each database or mobile terminal (2-i: transmission source) as necessary. The data is transmitted to the portable terminal (2-j (j = 1 to n, j ≠ i): transmission destination). Data related to the operation is exemplified by data obtained by operating an input device such as a keyboard, a numeric keypad, and a touch panel. Data processing is exemplified by processing for converting data so that it can be output by a portable terminal as a transmission destination, and processing for selecting processing to be executed by the information processing apparatus 1 (data control unit 12). Data relating to the operation is input / output via the area 82 a of the memory 82.

  The display data processing unit 12-5 performs data processing on data displayed on an external display device transmitted from each database or portable terminal (2-i: transmission source) as necessary. The data is transmitted to the portable terminal (2-j (j = 1 to n, j ≠ i): transmission destination). Data processing is exemplified by processing for converting data so that it can be output by an external display device at the transmission destination, processing for extracting image data in the data, and processing for easily converting image data. Data to be displayed on the display device is input / output via the area 82d of the memory 82.

  The search unit 12-6 searches for data based on the search-related data transmitted from the mobile terminal (2-i: transmission source). As a search destination, not only the database in the information processing apparatus 1 but also an external database may be accessed and searched. Then, the retrieved data is transmitted to the portable terminal (2-j (j = 1 to n, j ≠ i): transmission destination). Data related to search is exemplified by search keywords and data file names. Data relating to the search is input / output via the area 82e of the memory 82.

  However, the transmission / reception itself may be executed by another transmission / reception function (transmission / reception unit: not shown).

  The portable terminal 2-i (i = 1 to n: n is a natural number) is a portable information processing device such as a cellular phone, a PDA, or a mini-notebook PC that is carried by the user. Communication via the network 5 is possible, and an external display device can be connected and used. When the portable terminal 2 is used as the information processing system, an external display device 91 is connected. The portable terminal 2 includes an accessory device management unit 21, a connection management unit 22, and a data control unit 23 as programs for performing information processing.

  When a device (eg, display device 91) is attached to an external device connection interface (described later), the accessory device management unit 21 inquires of the device and acquires the function, performance, and the like of the device.

  The connection management unit 22 establishes a connection with the information processing apparatus 1 when receiving the information processing service from the information processing apparatus 1. When the information processing service 1 is terminated from the information processing service, the connection is terminated.

  When receiving provision of information processing service from the information processing apparatus 1, the data control unit 23 controls data transmission / reception and data processing with the information processing apparatus 1. The data control unit 23 includes a data recognition unit 23-1, a voice data processing unit 23-2, a built-in display data processing unit 23-3, an operation data processing unit 23-4, and a display data processing unit 23 as programs for performing data processing. -5 and a search unit 23-6. Each program is executed, for example, in a time-sharing manner.

  The data recognition unit 23-1 recognizes the type of data transmitted from the information processing apparatus 1 (transmission source). Based on the type of data, the data is stored in an appropriate area (example: 72a to 72e) in the memory 72 (described later).

  The audio data processing unit 23-2 generates data related to audio transmitted from the information processing apparatus 1 from a speaker 76 (described later). The voice input from the microphone 75 (described later) by the user is transmitted to the information processing apparatus 1 as voice-related data. Data related to sound is input / output via the area 72 b of the memory 72.

  The built-in display data processing unit 23-3 causes the built-in display device 73 (described later) to display data to be displayed on the display unit of the mobile terminal transmitted from the information processing device 1. An image captured by a camera 70 (described later) of the mobile terminal is transmitted to the information processing apparatus 1 as data to be displayed on the display unit of the mobile terminal. Data to be displayed on the display unit is input / output via the area 72 c of the memory 72.

  The operation data processing unit 23-4 performs data processing as necessary on the data obtained by operating input devices such as the numeric keypad and function keys of the portable terminal 2-i, the attached keyboard and touch panel, and sends the data to the information processing apparatus 1. Send. Data processing is exemplified by processing for converting an input into a service request signal. Data relating to the operation is input / output via the area 72 a of the memory 72.

  The display data processing unit 23-5 outputs data to be displayed on the external display device 91 (described later) transmitted from the information processing device 1 to the display device 91. Data to be displayed on the display device is input / output via the area 72 d of the memory 72.

  The search unit 23-6 transmits data related to the search to the information processing apparatus 1 in response to execution of the search command in the mobile terminal 2-i. Data relating to the search (example: search keyword) is input / output via the area 72 e of the memory 72.

  However, the transmission / reception itself may be executed by another transmission / reception function (transmission / reception unit: not shown).

  The participant database 14 will be further described. FIG. 3 is a table showing an example of the participant database 14. The participant database 14 stores identification data of a subject who receives the information processing service of the information processing apparatus 1 and information related to the subject in association with each other. The identification data is exemplified by the ID 14-1 of the other party mobile terminal of the subject and the TEL 14-2 of the telephone number of the other party mobile terminal. Information about the subject is exemplified by the name 14-3 of the subject, the attribute 14-4 such as the company name and department name to which the subject belongs, and the handicap information 14-5 concerning the ability of the subject. The handicap information 14-5 is, for example, the nationality of a foreigner who does not understand Japanese, the degree of inconvenience in the case of a blind person, or the inconvenience of a person with a hearing impairment. It is illustrated to some extent. Nationality can be applied to speech translation and text translation. The degree of blindness can be applied to enlargement of characters, conversion to color graphic patterns, and reading of characters. The degree of deafness can be applied to the conversion of speech.

  The connected device database 15 will be further described. FIG. 4 is a table showing an example of the connected device database 15. The connected device database 15 stores identification data of connection devices (display devices) that can be connected to the mobile terminal and display characteristic data in association with each other. The identification data is exemplified by the connected device name 15-1. The display characteristic data is exemplified by the number of displayable pixels (resolution) 15-2, the number of displayable colors 15-3, the scanning method 15-4, and the color correction value 15-5.

The menu database 18 will be further described. FIG. 5 is a table showing an example of the menu database 18. The menu database 18 stores the type of information processing service provided by the information processing apparatus 1 and the content (function) of information processing to be performed in association with each other.
The type of information processing service is exemplified in each item of the menu 18-1 (example: TV conference, restaurant menu, sports broadcast, game, bid, concert and movie relay, navigation).
The contents (functions) of the information processing are: display device 18-2, built-in display device 18-3, audio data processing unit 18-4, built-in display data processing unit 18-5, operation data processing unit 18-6, display data processing. Part 18-7 is exemplified. The column of the display device 18-2 shows data displayed on the display device 91. The column of the built-in display device 18-3 shows data displayed by the built-in display device 73 of the portable terminal 2. The column of the voice data processing unit 18-4 indicates an additional function (eg, voice translation, reading) executed by the voice data processing unit 12-2. The column of the built-in display data processing unit 18-5 shows additional functions (example: character extraction, character conversion, (character) translation) executed by the built-in display data processing unit 12-3. The operation data processing unit 18-6 indicates an additional function (example: function change) executed by the operation data processing unit 12-4. The display data processing unit 18-7 indicates an additional function (example: drawing extraction, color change) executed by the display data processing unit 12-5.

Here, the content (function) of information processing will be described.
In the speech translation, for example, Japanese speech data is translated into English. In reading, character data is converted into voice data. Character extraction extracts character data from data in which characters are combined with graphics and images. Drawing extraction extracts figure and image data from data in which characters, figures and images are combined. Character conversion increases the character size of character data or makes it easier to read. Color conversion changes the color of graphics and image data, or changes the color to a texture or pattern. For (character) translation, for example, Japanese character data is translated into English. In the function change, setting screen display and setting change are performed in order to change the above-described functions.

For example, if “handicap” is described in the handicap information 14-5, the speech translation (Japanese-English, English-Japanese) of the speech data processing unit 18-4, the (character) of the built-in display data processing unit 18-5 Start translation (Japanese-English, English-Japanese). Thereby, it is possible to receive the service comfortably without being aware of the difference in language.
If the handicap information 14-5 describes “low vision”, the character extraction and character conversion of the built-in display data processing unit 18-5, the drawing extraction of the display data processing unit 18-7, and the voice as necessary. Starts reading out of the data processing unit 18-4. Thereby, it becomes easy to see a character and a drawing and can receive a service comfortably.
If “handicapped” is described in the handicap information 14-5, the voice data processing unit 18-4 reads aloud, and the built-in display data processing 18-5 performs character conversion (braille conversion separately connected to the portable terminal 2). Use the device). This makes it possible to receive a service comfortably even when characters are not visible.
If “color blindness” is described in the handicap information 14-5, drawing extraction and color conversion of the display data processing unit 18-7 are activated. Thereby, the color recognition can be changed to the pattern recognition, and the service can be received comfortably.

  In addition, by activating the function change of the operation data processing unit 18-6 at the time of service execution, a desired function can be selected or deleted by the operation of the mobile terminal 2 regardless of the handicap information 14-5. it can.

Here, in the case of the TV conference in the item of the menu 18-1, the display device 91 displays data (example: document data) transmitted from the information processing device 1 as described in the display device 18-2. . As described in the internal display device 18-3, the internal display device 73 displays input information (example: partner's camera image) from another connection destination (another mobile terminal, information processing device).
In the case of the restaurant menu, the display device 91 displays data (eg, type of dish) transmitted from the information processing device 1 as described in the display device 18-2. The built-in display device 73 displays a command list or the like (eg, a command for selecting the number or type of dishes to be ordered) that allows the display contents to be manipulated as described in the built-in display device 18-3. . In sports relay, the display device 91 displays data (eg, sports relay) transmitted from the information processing device 1 as described in the display device 18-2. The built-in display device 73 is a command list or the like that makes it possible to operate data and display contents transmitted from the information processing device 1 as described in the built-in display device 18-3 (ex. A list of commands that can request data display, such as the tone of the data, and the result of the requested data).

  Since the processing procedure is stored in the storage device 86 of the information processing apparatus 1 and executed by the data control unit 12, by increasing the contents (functions) of services and information processing stored on the information processing apparatus 1 side, Even if the functions are not installed on the mobile terminals 2-1 and 2-2, various new applications can be provided.

  FIG. 6 is a block diagram showing the configuration of the first embodiment of the portable terminal of the information processing system of the present invention. The portable terminal 2 (-1 to n) includes a camera 70, a CPU 71, a memory 72, a built-in display device 73, a key input unit 74, a microphone 75, a speaker 76, a communication unit 77, a storage device 78, an external device connection interface 79, A display device 91 is provided.

  The program shown in FIG. 2 is stored in the storage device 78. During operation, it is expanded in the memory 72 and executed by the CPU 71. In addition to the normal area, the memory 72 includes an area 72a, an area 72b, an area 72c, an area 72d, and an area 72e. The area 72a is controlled by the operation data processing unit 23-4 and is used for input / output of data related to data by operation. The area 72b is controlled by the audio data processing unit 23-2 and used for input / output of data related to audio. The area 72c is controlled by the built-in display data processing unit 23-3 and used for input / output of data to be displayed on the display unit of the mobile terminal. The area 72d is controlled by the display data processing unit 23-5 and used for data input / output for the display device 91. The area 72e is controlled by the search unit 23-6 and used for input / output of data related to the search.

  The display device 91 includes a control unit 91-1 and a display unit 91-2. The control unit 91-1 outputs information related to the display device in response to the inquiry from the accessory device management unit 21. In addition, data for the display device 91 is displayed on the display unit 91-2. If the display unit 91-2 is a touch panel type, the touched data is recognized and output to the mobile terminal. The display device 91 is larger than the built-in display device 73 of the mobile terminal 2. For example, a flat panel display is exemplified. An input-capable display such as a touch panel may be used.

  The display device 91 is, for example, a display at a destination of the user of the mobile terminal 2 (example: work place, business trip destination, restaurant, entertainment facility, etc.). In the information processing system of the present invention, even if the built-in display device 73 of the portable terminal 2 is small, data can be displayed by borrowing a display at a destination that is not carried. Thereby, two different pieces of information can be obtained simultaneously from each display device. In particular, when the display is larger than the built-in display device 73 of the portable terminal 2, it is preferable that information can be displayed in more detail.

  FIG. 7 is a block diagram showing the configuration of the embodiment of the information processing apparatus of the information processing system of the present invention. The information processing apparatus 1 includes a CPU 81, a memory 82, an external device connection interface 83, an external device connection interface 84, a communication unit 85, a storage device 86, a keyboard 96, and a display device 97.

  The program shown in FIG. 2 is stored in the storage device 86. During operation, it is expanded in the memory 82 and executed by the CPU 81. In addition to the normal area, the memory 82 includes an area 82a, an area 82b, an area 82c, an area 82d, and an area 82e. The area 82a is controlled by the operation data processing unit 12-4 and is used for data input / output by operation. The area 82b is controlled by the audio data processing unit 12-2 and used for input / output of data related to audio. The area 82c is controlled by the built-in display data processing unit 12-3 and is used for input / output of data to be displayed on the display unit of the portable terminal. The area 82d is controlled by the display data processing unit 12-5 and used for data input / output for the display device 91. The area 82e is controlled by the search unit 12-6 and used for input / output of data related to the search. The keyboard 96 and the display device 97 may be omitted.

  As shown in FIGS. 6 and 7, data processing is facilitated by using different areas in the memory depending on the type of data.

Next, the operation (information processing method) of the first embodiment of the information processing system of the present invention will be described.
8 and 9 are flowcharts showing the operation of the first embodiment of the information processing method of the present invention. However, FIG. 8 shows an operation until the mobile terminal 2 selects one of a plurality of functions (services) provided by the information processing apparatus 1. FIG. 9 shows, as an example, a case in which a conference between the portable terminal 2-1 (user) and the portable terminal 2-2 (user) is selected as a function.

  In the communication between the portable terminals 2-1 and 2-2 and the information processing apparatus 1, the ID of the transmission source and the transmission destination is always included in the header of the information to be communicated. When information to be communicated is stored (stored), the information is stored (stored) in association with the ID or the like. Their description is omitted (the same applies to other embodiments hereinafter). The ID or the like is an identification number (ID) of the mobile terminal or the information processing apparatus, and is exemplified by a URL, a mail address, an IP address, and the like.

First, with reference to FIG. 8, the operation | movement which the mobile terminal 2 selects one from the some function which the information processing apparatus 1 provides is demonstrated.
(1) Step S01
The user connects the display device 91 to the external device connection interface 79 of the portable terminal 2-1. The external device connection interface 79 has an attachment / detachment detection function for attached devices. The accessory device management unit 21 of the mobile terminal 2 detects the connection based on the detection signal from the external device connection interface 79. The display device 91 is located at the user's destination and is not particularly limited as long as it can be connected to the mobile terminal 2. For example, another person's display device in the destination office may be borrowed. Thus, in the present invention, it is not necessary to carry around the display device 91, it is not particularly necessary to be a public display device, and the display device 91 does not need to be connected to a network. Note that the driver of the display device 91 is stored in the storage device 78 in advance by the portable terminal 2-1. You may download from the information processing apparatus 1.
(2) Step S02
The accessory device management unit 21 outputs a device information request to the connected display device 91. The device information request is a request for display device information as information regarding the display device 91. The display device information is information indicating the characteristics of the connected device exemplified by the number of pixels (resolution) and the scanning method. The connected device name may be used.
(3) Step S03
In response to the device information request, the control unit 91-1 of the display device 91 outputs display device information stored in a memory (not shown). The accessory device management unit 21 stores the display device information in the memory 72.
(4) Step S04
The user inputs a connection request. The connection request is a request for connecting the portable terminal 2-1 to the information processing apparatus 1 and using a predetermined information processing function. It may include a user password that has been negotiated in advance.
(5) Step S05
The connection management unit 22 of the portable terminal 2-1 receives the connection request input and transmits the connection request and device information to the information processing apparatus 1 via the network 5. However, the device information includes terminal information and display device information as information regarding the portable terminal 2-1. The terminal information is exemplified by, for example, a communication method, the number of pixels of the built-in display device 73, a scanning method, and an audio conversion method. The connection management unit 11 of the information processing apparatus 1 stores these pieces of information in the memory 82.
(6) Step S06
In response to the connection request, the connection management unit 11 converts (corresponds) the menu information so that it can be displayed and operated on the mobile terminal 2 based on the device information (example: number of pixels and scanning method). Is generated. The menu information is information indicating the types of information processing services that the user can receive. The service indicated by the menu information is the menu 18-1 of the menu database 18.
However, when the display device information in the device information is “connected device name”, the display characteristics of the connected device are grasped from the connected device database 15 based on the name 15-1 corresponding to the connected device name. Display data (menu information) for changing the size of the display screen and unifying the operation interface is generated according to the device characteristics grasped here. In this case, the amount of data transmitted from the portable terminal 2-1 can be reduced.
Note that existing PC displays comply with the DDC (Display Data Channel) standard, and by supporting this function, information such as supported frequencies on the display side can be automatically acquired by communicating with the display. It has become.
If there is a password, it is confirmed with a correspondence table (not shown) between the password of the storage device 86 and the portable terminal ID.
(7) Step S07
The connection management unit 11 transmits menu information to the mobile terminal 2 via the network 5.
(8) Step S08
The connection management unit 22 displays the menu information on the built-in display device 73.
(9) Step S09
The user selects a desired service from the displayed menu information. Here, it is assumed that a conference (TV conference) on the mobile terminal is selected.
The type of display content (display data) for each screen of the built-in display device 73 of the portable terminal 2-1 and the display device 91 connected to the portable terminal is uniquely determined depending on what service is selected at this time. It is done. Data indicating the relationship between the service and the display content is stored in the menu database 18 as described above.
(10) Step S10
The connection management unit 22 transmits selection information indicating the selected service (TV conference) to the information processing apparatus 1 via the network 5.
(11) Step S11
The connection management unit 11 confirms selection information from the mobile terminal 2-1. Based on this, the connection management unit 11 generates start information indicating the start of the selected service (TV conference). At the same time, based on the selected service, the contents (functions) of information processing of each unit in the data control unit 12 are set to be usable with reference to the menu database 18.
(12) Step S12
The connection management unit 11 transmits start information to the mobile terminal 2-1 via the network 5.
(13) Step S13
The connection management unit 22 of the portable terminal 2-1 displays the start (start information) of the selected service (TV conference) on the built-in display device 73.

  In this way, the mobile terminal 2 selects one of a plurality of services provided by the information processing apparatus 1.

  Next, with reference to FIG. 9, a case where a conference between the portable terminal 2-1 (user) and the portable terminal 2-2 (user) is selected as a service is shown. However, the user of the portable terminal 2-1 is a meeting organizer (chairman, representative), and the user of the portable terminal 2-2 is a participant of the meeting. There may be a plurality of users of the portable terminal 2-2. At least the user of the portable terminal 2-1 has previously applied or contracted using the information processing function.

(1) Step S21
After the start information is output, the data control unit 12 of the information processing apparatus 1 generates conference input information converted (corresponding) so that the portable terminal 2-1 can perform display and operation based on the device information. To do. The conference input information indicates an input screen for information necessary for executing a conference function.
(2) Step S22
The data control unit 12 transmits the conference input information to the portable terminal 2-1 via the network 5.
(3) Step S23
The data control unit 23 of the portable terminal 2-1 displays the conference input information on the built-in display device 73.
(4) Step S24
The user inputs necessary information regarding the conference to the displayed conference input information. Here, the information necessary for the conference is exemplified by information indicating a conference partner and a material data file name. However, the information indicating the meeting partner is exemplified by the partner mobile terminal ID, mail address, URL, IP address, name, telephone number, and the like. Here, the other party mobile terminal ID is used. The partner mobile terminal ID is an identification number (mobile terminal ID) of the mobile terminal 2-2 which is a mobile terminal of another user who attends the conference. The material data file name is the name of the material data file that is the material for the meeting. The material data file is read from the material database 13 storing the material data file and the portable terminal ID in association with each other. The user of the portable terminal 2-1 stores it in the material database 13 in advance. There may be a plurality of counterpart portable terminal IDs and material data file names.
(5) Step S25
The data control unit 23 transmits information (partner mobile terminal ID) indicating the meeting partner and the material data file name to the information processing apparatus 1 via the network 5.
(6) Step S26
The data control unit 12 of the information processing device 1 refers to the participant database 14 based on the counterpart portable terminal ID and identifies the portable terminal 2-2 to be connected. At this time, referring to the handicap information 14-5 of the user of the portable terminal 2-2, it is necessary among the contents (functions) of information processing that the connection management unit 11 refers to the menu database 18 and can use. To activate a function. For example, if the user's handicap information 14-5 is a US register, the necessary function activates a translation function that translates character data into English. If the handicap information 14-5 is weakly sighted, a character conversion function for making character data large characters is activated.
(7) Step S27
The data control unit 12 generates connection request confirmation information that is converted (corresponding) so that it can be displayed and operated by the portable terminal 2-2 indicated by the partner portable terminal ID. The connection request confirmation information is information for confirming the connection. The connection management unit 11 transmits connection request confirmation information to the portable terminal 2-2 via the network 5. When there are a plurality of portable terminals 2-2, it is performed for each of them.
(8) Step S28
The connection management unit 22 of the portable terminal 2-2 displays the connection request confirmation information on the built-in display device 73.
(9) Step S29
The user of the portable terminal 2-2 confirms the displayed connection request confirmation information and inputs confirmation.
(10) Step S30
In response to the input of the connection request confirmation information, the connection management unit 22 of the portable terminal 2-2 transmits the connection request and the device information to the information processing apparatus 1 via the network 5. The device information relates to the mobile terminal 2-2.
(11) Step S31
The connection management unit 11 confirms a response (connection request, device information) from the mobile terminal 2-2. Based on this, the data control unit 12 of the information processing apparatus 1 generates connection completion information indicating that the connection has been converted (corresponding) so that it can be displayed on each of the mobile terminals 2-1 and 2-2.
(12) Step S32
The connection management unit 11 transmits connection completion information to the mobile terminal 2-1 via the network 5. The connection management unit 22 of the portable terminal 2-1 displays the connection completion information on the built-in display device 73.
(13) Step S33
Similarly, the connection management unit 11 transmits connection completion information to the portable terminal 2-2 via the network 5. The connection management unit 22 of the portable terminal 2-2 displays the connection completion information on the built-in display device 73.
(14) Step S34
The data control unit 12 of the information processing apparatus 1 executes a service selected by the mobile terminal 2-1 (here, a TV conference by the mobile terminal). In a TV conference by a portable terminal, the information processing apparatus 1, the portable terminal 2-1, and the portable terminal 2-2 perform transmission / reception of document data, display processing thereof, transmission / reception and display of camera images by the portable terminal, transmission / reception and generation of sound. Do. Details will be described later.
(15) Step S35
The user inputs an end request for ending the selected service (TV conference).
(16) Step S36
The data control unit 23 of the portable terminal 2-1 receives the termination request and transmits the termination request and the portable terminal ID to the information processing apparatus 1 via the network 5.
(17) Step S37
In response to the termination request input, the data control unit 12 stops the function related to the portable terminal ID.
(18) Step S38
The connection management unit 11 generates end information indicating the end of the selected service (TV conference) so as to be displayed on the built-in display device 73 of the mobile terminal 2-1 and transmits the information to the mobile terminal 2-1 via the network 5. To do. The connection management unit 22 of the portable terminal 2-1 disconnects the information processing apparatus 1. The data control unit 23 ends the process.
(19) Step S39
Similarly, the connection management unit 11 generates end information indicating the end of the selected service so that it can be displayed on the built-in display device 73 of the mobile terminal 3, and transmits it to the mobile terminal 2-2 via the network 5. The connection management unit 22 of the portable terminal 2-2 disconnects from the information processing apparatus 1. The data control unit 23 ends the process.

  By doing in this way, a meeting can be held between the portable terminal 2-1 (the user) and the portable terminal 2-2 (the user, a plurality of users) as a representative. At that time, since a display device larger than the portable terminal 2 can be used, the content of the conference becomes dark and efficient, and the progress can be made smooth.

  In the present invention, the display of the built-in display device 73 of the portable terminals 2-1 and 2-2 and the display of the connected display device 91 can be controlled independently. That is, since the materials and the attendees are displayed independently, the contents of the meeting and the contents of the meeting can be grasped more accurately, and the progress can be made smooth.

Next, the operation in step S34 in the flow of FIG. 9 will be described.
FIG. 10 is a flowchart showing an operation of displaying data (eg, document data) on the display device in step S34.

(1) Step S41
The user of the portable terminal 2-1 designates desired material data from the material data file used for the conference by the key input unit 74. For example, the first page is designated.
(2) Step S42
The operation data processing unit 23-4 of the data control unit 23 of the portable terminal 2-1 transmits material data designation information indicating the designation of material data to the information processing apparatus 1 via the network 5 based on designation by the user. . The material data designation information has a header indicating that it is data from the operation data processing unit (hereinafter also referred to as “operation data”).
(3) Step S43
The data recognition unit 12-1 of the data control unit 12 of the information processing device 1 recognizes that the material data designation information is operation data from the header of the material data designation information. Then, the material data designation information is stored in the area 82 a in the memory 82. The area 82a stores operation data.
The operation data processing unit 12-4 reads the material data designation information from the area 82 a and acquires the corresponding material data from the material database 13. Then, the document data is converted so that it can be displayed on the display device 91 of the portable terminal 2. The converted document data (hereinafter referred to as converted document data A) has a header indicating that it is data to be displayed on the display device 91 of the portable terminal 2-1. Similarly, it converts so that it can display with the display apparatus 91 of the portable terminal 2-2. The converted material data (hereinafter referred to as converted material data B) has a header indicating that it is data to be displayed on the display device 91 of the portable terminal 2-2.
(4) Step S44
The operation data processing unit 12-4 transmits the conversion material data A to the mobile terminal 2-1 via the network 5.
(5) Step S45
Similarly, the operation data processing unit 12-4 transmits the conversion material data B to the portable terminal 2-2 via the network 5.
(6) Step S46
The data recognition unit 23-1 of the portable terminal 2-1 recognizes from the header of the conversion material data A that the conversion material data A is data for the display device 91. Then, the converted material data A is stored in the area 72 d in the memory 72. The area 72d stores data related to the display device 91.
The display data processing unit 23-5 reads the converted material data A from the area 72d and outputs it to the display device 91.
(7) Step S47
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display the conversion material data A.
(8) Step S48
The data recognition unit 23-1 of the portable terminal 2-2 recognizes from the header of the conversion material data B that the conversion material data B is data for the display device 91. Then, the converted material data B is stored in the area 72 d in the memory 72.
The display data processing unit 23-5 reads the converted material data B from the area 72d and outputs it to the display device 91.
(9) Step S49
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display the conversion material data B.

  However, the person who designates the document data may be limited to a predetermined representative (example: user of the portable terminal 2) or permitted to other participants (example: user of the portable terminal 2-2). You may do it.

  In the example of FIG. 10, the step of inputting the number of pages (page number) by specifying the material data to be displayed has been described. However, in S41, not only “specification of the number of pages of material data to be displayed” is specified. , Add new information to material data (such as adding text and figures to material, drawing underlines and markers, surrounding certain characters with circles (circles), etc.) "Edited content" can be used as an input from the mobile terminal. As a result, it is possible to edit the material data at the time of the meeting and to promptly inform the members of the meeting of the edited contents.

  The processing procedure corresponds to S41 in the processing content of FIG. As for the specific contents of S41, the contents of the menu at the time of editing are requested from the portable terminal 2-1 to the information processing apparatus 1 in the processing flow as in S04 to S08 of FIG. Possible menu contents are fed back to the portable terminal 2-1. 9, S35: the mobile terminal 2-1 receives the edited content by the user, S36: the mobile terminal 2-1 transmits the edited content, and S37: the information processing terminal. 1 changes and reflects the document data in accordance with the requested content of the edit contents, and S38, S39: The changed document data is fed back to each portable terminal 2-1, and displayed.

  By doing in this way, the content different from the display apparatus with a built-in portable terminal can be displayed on the display apparatus externally connected to the portable terminal, without greatly increasing the burden of information processing in the portable terminal. Efficient and rich conferences can be held.

Next, another operation of step S34 in the flow of FIG. 9 will be described.
FIG. 11 is a flowchart showing an operation of displaying on the mobile terminal 3 a user image captured by the mobile terminal 2 in step S34.

(1) Step S51
The built-in display data processing unit 12-3 of the information processing apparatus 1 generates a camera image request for requesting camera images of the portable terminal 2-1 and the portable terminal 2-2. The camera image request has a header indicating that it is data related to the built-in display device 73 of the mobile terminal 2.
(2) Step S52
The data recognition unit 23-1 of the data control unit 23 of the portable terminal 2-1 recognizes that the camera image request is data related to the built-in display device 73 from the header of the camera image request. Then, the camera image request is stored in the area 72 c in the memory 72. The area 72c stores data related to the built-in display device 73.
The built-in display data processing unit 23-3 reads the camera image request from the area 72c. And the camera image of the user of the portable terminal 2-1 is acquired using the camera 70. FIG. The acquired camera image is attached with the mobile terminal ID of the captured mobile terminal (user) and stored in the area 72 c of the memory 72.
(3) Step S53
Similarly, the data recognition unit 23-1 of the data control unit 23 of the portable terminal 2-2 recognizes that the camera image request is data related to the built-in display device 73 from the header of the camera image request. Then, the camera image request is stored in the area 72 c in the memory 72. The area 72c stores data related to the built-in display equipment 73.
The built-in display data processing unit 23-3 reads the camera image request from the area 72c. And the camera image of the user of the portable terminal 2-2 is acquired using the camera 70. FIG. The acquired camera image is stored in the area 72 c of the memory 72.
(4) Step S54
The built-in display data processing unit 23-3 of the portable terminal 2-1 reads the camera image from the area 72c. Then, the camera image is transmitted to the information processing apparatus 1 via the network 5. The camera image has a header indicating data regarding the built-in display equipment 73.
(5) Step S55
Similarly, the built-in display data processing unit 23-3 of the portable terminal 2-2 reads the camera image from the area 72c. Then, the camera image is transmitted to the information processing apparatus 1 via the network 5. The camera image has a header indicating data regarding the built-in display equipment 73.
(6) Step S56
The data recognition unit 12-1 of the data control unit 12 of the information processing apparatus 1 recognizes that the camera image is data related to the built-in display equipment 73 from the header of the camera image. Then, the camera image is stored in the area 82 c in the memory 82. The area 82c stores data related to the built-in display equipment 73.
The built-in display data processing unit 12-3 reads the camera image from the portable terminal 2-2 from the area 82c. Then, the camera image is converted so that it can be displayed on the built-in display device 73 of the portable terminal 2-1. The converted camera image (hereinafter referred to as “converted camera image A”) has a header indicating that it is data to be displayed on the built-in display device 73 of the portable terminal 2-1.
Similarly, the built-in display data processing unit 12-3 reads the camera image from the portable terminal 2-1 from the area 82c. Then, the camera image is converted so that it can be displayed on the built-in display device 73 of the portable terminal 2-2. The converted camera image (hereinafter, converted camera image B) has a header indicating that it is data to be displayed on the built-in display device 73 of the portable terminal 2-2.
(7) Step S57
The built-in display data processing unit 12-3 transmits the converted camera image A to the portable terminal 2-1 via the network 5.
(8) Step S58
Similarly, the built-in display data processing unit 12-3 transmits the converted camera image B to the portable terminal 2-2 via the network 5.
(9) Step S59
The data recognition unit 23-1 of the portable terminal 2-1 recognizes that the converted camera image A is data for the built-in display device 73 from the header of the converted camera image A. Then, the converted camera image A is stored in the area 72 c in the memory 72.
The built-in display data processing unit 23-3 reads the converted camera image A from the area 72c and displays it on the built-in display device 73.
(10) Step S60
Similarly, the data recognition unit 23-1 of the portable terminal 2-2 recognizes from the header of the converted camera image B that the converted camera image B is data for the built-in display device 73. Then, the converted camera image B is stored in the area 72 c in the memory 72.
The built-in display data processing unit 23-3 reads the converted camera image B from the area 72c and displays it on the built-in display device 73.
(11) Step S61
The built-in display data processing unit 12-3 waits for a predetermined time to elapse after transmitting the camera image to the portable terminal 2-1 and the portable terminal 2-2. The predetermined time is, for example, 1 second.
(12) Step S62
The built-in display data processing unit 12-3 confirms that the conference has not ended after the predetermined time has elapsed. The end of the conference is determined by the presence / absence of an end request. If not completed, the process returns to step S51, and the camera image acquisition, transmission, and display processes are executed again. If it is done, the process is finished.

  When there are a plurality of participants in the conference, the built-in display data processing unit 23-3 divides the built-in display device 73 so that a plurality of converted camera images for all the members are displayed together with the mobile terminal ID. May be. For example, when there are four participants, the camera images of the three people except the person are displayed on the built-in display device 73 together with the mobile terminal ID.

  By doing in this way, the image of all the participants of a meeting can be displayed on a portable terminal. Thereby, each thought and atmosphere can be grasped from the facial expressions of the participants, and the presence of the conference can be enhanced, and the progress of the conference can be made smooth.

Next, still another operation in step S34 in the flow of FIG. 9 will be described.
FIG.12 and FIG.13 is a flowchart which shows the operation | movement in which a user speaks in step S34.

(1) Step S71
The user of the portable terminal 2-2 generates a speech request for requesting speech in the conference by using the key input unit 74.
(In the case of a representative (mobile terminal 2-1), step S71 ': The user of the mobile terminal 2-1 generates a request for requesting a speech at the conference by the key input unit 74.)
(2) Step S72
The operation data processing unit 23-4 of the data control unit 23 of the portable terminal 2-2 transmits a speech request to the information processing apparatus 1 via the network 5. The message request has a header indicating that it is operation data.
(In the case of a representative (mobile terminal 2-1), step S72 ': the operation data processing unit 23-4 of the data control unit 23 of the mobile terminal 2-1 sends a speech request to the information processing apparatus 1 via the network 5. Send)
(3) Step S73
The data recognition unit 12-1 of the data control unit 12 of the information processing device 1 recognizes that the message request is operation data from the message request header. Then, the speech request is stored in the area 82 a in the memory 82.
The operation data processing unit 12-4 reads the speech request from the area 82a. Then, it is determined by the mobile terminal ID from which the speech request is issued whether or not the speech request is a speech of the representative (here, the user of the mobile terminal 2-1).
(4) Step S74
When the speech request is from the representative, the operation data processing unit 12-4 cancels the speech data processing unit 12-2 if the speech data processing unit 12-2 performs processing related to speech from another user. And the permission information which shows the permission signal which shows permission of a statement is produced | generated so that display is possible with the display apparatus 91 of the portable terminal 2-1. The permission information has a header indicating that it is data for the display device 91.
(5) Step S75
The operation data processing unit 12-4 transmits the permission information to the portable terminal 2-1 (representative).
(6) Step S76
The data recognition unit 23-1 of the portable terminal 2-1 recognizes that the permission information is data for the display device 91 from the header of the permission information. Then, the permission information is stored in the area 72d in the memory 72.
The display data processing unit 23-5 reads the permission information from the area 72d and outputs it to the display device 91.
(7) Step S77
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display permission information. At this time, data that has already been displayed (example: document data) is displayed so as to open another window. By maximizing, minimizing, and changing the window to a desired size, it is possible to coexist with existing data on the display device 91 (the same applies hereinafter).
(8) Step S78
If the speech request is not from the representative in step S73, the operation data processing unit 12-4 determines whether the speech data processing unit 12-2 is performing processing related to speech from another user.
(9) Step S79
The operation data processing unit 12-4 is based on the portable terminal ID from which the request for speech is issued and the participant database 14 when the voice data processing unit 12-2 performs processing related to speech from other users. Create a waiter list. The created waiter list is stored in the area 82a of the storage unit 82. If there is already a waiting list, the waiting list is updated.
(10) Step S80
The operation data processing unit 12-4 converts the waiting list so that it can be displayed on the display device 91 of the portable terminal 2-1. The converted waiting list (hereinafter referred to as conversion waiting list) has a header indicating that the data is to be displayed on the display device 91 of the portable terminal 2-1. The operation data processing unit 12-4 transmits the waiting list to the portable terminal 2-1 via the network 5.
(11) Step S81
The data recognition unit 23-1 of the portable terminal 2-1 recognizes that the conversion waiting list is data for the display device 91 from the header of the conversion waiting list. The conversion waiting list is stored in the area 72 d in the memory 72.
The display data processing unit 23-5 reads the conversion waiting list from the area 72d and outputs it to the display device 91.
(12) Step S82
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display the conversion waiting list. At this time, the data already displayed is displayed so as to open another window.
(13) Step S83
The operation data processing unit 12-4 determines whether or not there is a waiting list stored in the area 82a of the storage unit 82 when the voice data processing unit 12-2 is not performing processing related to a speech from another user. Judging.
(14) Step S84
When there is a waiting list, the operation data processing unit 12-4 creates a speaker request for inquiring which speaker is allowed to speak. The operation data processing unit 12-4 converts the speaker request so that it can be displayed on the display device 91 of the portable terminal 2-1. The converted speaker request (hereinafter referred to as “converted speaker request”) has a header indicating that it is data to be displayed on the display device 91 of the portable terminal 2-1. The operation data processing unit 12-4 transmits a speaker request to the mobile terminal 2-1 via the network 5.
(15) Step S85
The data recognition unit 23-1 of the portable terminal 2-1 recognizes that the converted speaker request is data for the display device 91 from the header of the converted speaker request. Then, the converted speaker request is stored in an area 72 d in the memory 72.
The display data processing unit 23-5 reads the converted speaker request from the area 72d and outputs it to the display device 91.
(16) Step S86
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display the conversion speaker request. At this time, the data already displayed is displayed so as to open another window.
(17) Step S87
Based on the speaker request, the user designates the speaker from the waiting list displayed on the display device 91 and inputs the speaker from the key input unit 74. The operation data processing unit 23-1 transmits the specified speaker to the information processing apparatus 1 via the network 5, with speaker information indicating the specified speaker. The speaker information has a header indicating that it is data related to operation data.
(18) Step S88
The data recognition unit 12-1 of the data control unit 12 of the information processing device 1 recognizes that the speaker information is data related to operation data from the header of the specified speaker. Then, the speech request is stored in the area 82 a in the memory 82. The operation data processing unit 12-4 reads the speaker information from the area 82a. And based on speaker information, with reference to participant database 14, permission information which shows permission is transmitted to a speaker's portable terminal (portable terminal 2-2) corresponding to speaker information.
Or operation data processing part 12-4 transmits permission information which shows permission to a speaker's portable terminal (portable terminal 2-2) which made a statement request at Step S72, when a waiting list does not exist at Step S83. .
The permission information is generated so as to be displayed on the display device 91 of the speaker's mobile terminal, and has a header indicating that the data is data to be displayed on the display device.
(19) Step S89
The data recognition unit 23-1 of the portable terminal 2-2 recognizes that the permission information is data for the display device 91 from the header of the permission information. Then, the permission information is stored in the area 72d in the memory 72.
The display data processing unit 23-5 reads the permission information from the area 72d and outputs it to the display device 91.
(20) Step S90
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display permission information. At this time, the data already displayed is displayed so as to open another window.
(21) Step S91
The user speaks via the microphone 75 based on the speech permission displayed on the display device 91. The audio data processing unit 23-2 generates the audio information. The audio information has a header indicating that it is data related to audio data.
(22) Step S92
The audio data processing unit 23-2 transmits the audio information to the information processing apparatus 1 via the network 5.
(23) Step S93
The data recognition unit 12-1 of the data control unit 12 of the information processing apparatus 1 recognizes that the voice information is data related to the voice data from the header of the voice information. Then, the audio data is stored in the area 82b in the memory 82. The area 82b stores data related to audio data. The audio data processing unit 12-2 reads the audio information from the area 82b. Then, referring to the participant database 14, the audio information is transmitted to all other portable terminals (mobile terminals 2-1).
(24) Step S94
The data recognition unit 22-1 of the data control unit 22 of the portable terminal 2-1 recognizes that the voice information is data related to the voice data from the header of the voice information. Then, the audio data is stored in the area 72b in the memory 72. The area 72b stores data related to audio data. The voice data processing unit 22-2 reads the voice information from the area 22b. And it outputs as an audio | voice from the speaker 76. FIG.
(25) Step S95
If there is a waiting list, the operation data processing unit 12-4 updates the waiting list in the area 82a of the storage unit 82 based on the speaker information in step S87.
(26) Step S96
The operation data processing unit 12-4 converts the waiting list so that it can be displayed on the display device 91 of the portable terminal 2-1. The converted waiting list (hereinafter referred to as conversion waiting list) has a header indicating that the data is to be displayed on the display device 91 of the portable terminal 2-1. The operation data processing unit 12-4 transmits the waiting list to the portable terminal 2-1 via the network 5.
(27) Step S97
The data recognition unit 23-1 of the portable terminal 2-1 recognizes that the conversion waiting list is data for the display device 91 from the header of the conversion waiting list. The conversion waiting list is stored in the area 72 d in the memory 72.
The display data processing unit 23-5 reads the conversion waiting list from the area 72d and outputs it to the display device 91.
(28) Step S98
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display the conversion waiting list. At this time, the data already displayed is displayed so as to open another window.

By organizing the comments in the conference in this way, it is possible to prevent voices from overlapping and making it impossible to hear the contents of the comments. In addition, by giving the representative authority to select a statement, the representative can control the proceedings. As a result, the conference by the mobile terminal can be more efficiently advanced.
In addition, the camera image described in FIG. 11 not only displays all participants during the meeting, but also focuses on the speaker and displays the “speaker's face” on the display unit of the listening portable terminal. The information processing apparatus 1 can perform a process of automatically switching so that “the face of all the listeners” is displayed on the display unit of the speaking mobile terminal.

Next, another operation in step S34 in the flow of FIG. 9 will be described.
14 and 15 are flowcharts showing the operation of making a majority decision in step S34.

(1) Step S101
The user of the portable terminal 2-1 requests a menu screen in order to use a new function by using the key input unit 74.
This request is given by way of example from the portable terminal 2-1, but can also be made from another connected portable terminal (mobile terminal 2-2).
(2) Step S102
The operation data processing unit 23-4 of the portable terminal 2-1 receives the menu screen request input and transmits the menu screen request to the information processing apparatus 1 via the network 5. The menu screen request has a header indicating operation data.
(3) Step S103
The data recognition unit 12-1 of the information processing apparatus 1 recognizes that the menu screen request is operation data from the header of the menu screen request. Then, the menu screen request is stored in the area 82 a in the memory 82.
The operation data processing unit 12-4 generates corresponding menu data so that it can be displayed and operated on the display device 91 of the portable terminal 2-1. Menu data is the type of information processing function that can be received by users participating in the conference. The menu data is, for example, majority decision (two choices, three choices, etc.) and opinion aggregation. The menu data has a header indicating that it can be displayed on the display device 91.
(4) Step S104
The operation data processing unit 12-4 transmits menu data to the portable terminal 2-1 via the network 5.
(5) Step S105
The data recognition unit 23-1 of the portable terminal 2-1 recognizes that the menu data is data for the display device 91 from the menu data header. Then, the menu data is stored in the area 72 d in the memory 72. The area 72d stores data related to the display device 91.
The display data processing unit 23-5 reads the menu data from the area 72d and outputs it to the display device 91.
(6) Step S106
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display menu data.
(7) Step S107
The user of the portable terminal 2-1 selects a desired function from the menu data using the key input unit 74. For example, a majority vote is selected.
(8) Step S108
The operation data processing unit 23-4 of the portable terminal 2-1 receives the menu selection input indicating the selected menu, and transmits the menu selection to the information processing apparatus 1 via the network 5. The menu selection has a header indicating operation data.
(9) Step S109
The data recognition unit 12-1 of the information processing apparatus 1 recognizes that the menu selection is operation data from the menu selection header. Then, the menu selection is stored in the area 82a in the memory 82.
The operation data processing unit 12-4 generates menu corresponding data that can be displayed and operated on the display device 91 of the portable terminal 2-1. The menu correspondence data is an input screen necessary for executing the function of the selected menu. The menu-corresponding data is, for example, a screen for entering options for majority vote. The menu corresponding data has a header indicating that the data can be displayed on the display device 91.
(10) Step S110
The operation data processing unit 12-4 transmits the menu corresponding data to the portable terminal 2-1 via the network 5.
(11) Step S111
The data recognizing unit 23-1 of the portable terminal 2-1 recognizes that the menu corresponding data is data for the display device 91 from the header of the menu corresponding data. Then, the menu corresponding data is stored in the area 72 d in the memory 72.
The display data processing unit 23-5 reads the menu corresponding data from the area 72d and outputs it to the display device 91.
(12) Step S112
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display menu corresponding data.
(13) Step S113
The user of the portable terminal 2-1 inputs desired data with the key input unit 74 while referring to the menu corresponding data. For example, a majority voting command with option 1 as YES, option 2 as NO, and option 3 as abstention.
(14) Step S114
The operation data processing unit 23-4 of the portable terminal 2-1 receives the input of the command information indicating the input command and transmits the command information to the information processing apparatus 1 via the network 5. The command information has a header indicating operation data.
(15) Step S115
The data recognition unit 12-1 of the data control unit 12 of the information processing device 1 recognizes that the command information is operation data from the header of the command information. Then, the command information is stored in the area 82 a in the memory 82. The operation data processing unit 12-4 reads the command information from the area 82a. And referring to participant database 14, command information is converted into data for display device 91 of other portable terminals (portable terminal 2-2), and command information is transmitted to other portable terminals.
(16) Step S116
The data recognition unit 23-1 of the portable terminal 2-2 recognizes that the command information is data for the display device 91 from the header of the command information. Then, the command information is stored in the area 72 d in the memory 72.
The display data processing unit 23-5 reads the command information from the area 72d and outputs it to the display device 91.
(17) Step S117
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display the command information. At this time, the data already displayed is displayed so as to open another window.
(18) Step S118
The user performs input via the key input unit 74 based on the command information displayed on the display device 91. For example, option 1: YES, option 2: NO, option 3: abstention. The operation data processing unit 23-1 generates opinion information indicating an opinion by the key input. Opinion information has a header indicating that it is data related to operation data.
(19) Step S119
The operation data processing unit 23-1 transmits opinion information to the information processing apparatus 1 via the network 5.
(20) Step S120
The data recognition unit 12-1 of the data control unit 12 of the information processing device 1 recognizes that the opinion information is data related to the operation data from the header of the opinion information. Then, the opinion information is stored in the area 82 a in the memory 82.
The operation data processing unit 12-4 collects opinion information from all the mobile terminals (the mobile terminal 2-1 and the mobile terminal 2-2) stored in the area 82 a in the memory 82. Then, the result of opinion aggregation is generated as opinion aggregation data so as to be displayed on the display device 91. For example, option 1: 3, option 2: 1, and option 3: 0. Opinion aggregation data has a header indicating that it is data related to a display device.
(21) Step S121
The operation data processing unit 12-4 converts the opinion aggregation data into data for the display device 91 of the portable terminal 2-1, and transmits the data to the portable terminal 2-1, via the network 5, to the portable terminal 2-1.
(22) Step S122
Similarly, the operation data processing unit 12-4 converts the opinion aggregation data into data for the display device 91 of the other portable terminal 2-2, and transmits the data to the other portable terminal 2-2 via the network 5. .
(23) Step S123
The data recognition unit 22-1 of the data control unit 22 of the portable terminal 2-1 recognizes that the opinion aggregated data is data related to the display device 91 from the header of the opinion aggregated data. Then, the opinion aggregation data is stored in the area 72 d in the memory 72.
The display data processing unit 23-5 reads the opinion aggregation data from the area 72d and outputs it to the display device 91.
(24) Step S124
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display opinion aggregation data. At this time, the data already displayed is displayed so as to open another window.
(25) Step S125
The data recognition unit 32-1 of the data control unit 32 of the portable terminal 2-2 recognizes that the opinion aggregated data is data related to the display device 91 from the header of the opinion aggregated data. Then, the opinion aggregation data is stored in the area 72 d in the memory 72.
The display data processing unit 23-5 reads the opinion aggregation data from the area 72d and outputs it to the display device 91.
(26) Step S126
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display opinion aggregation data. At this time, the data already displayed is displayed so as to open another window.

  By such an operation, opinions can be collected in a conference and a majority vote can be taken. As a result, it is possible not only to collect opinions but also to make decisions.

  The editing and writing of the material shown in the part after the description of FIG. 10 is also performed based on the processing procedure as shown in FIGS. Can be edited and changed. You can also share the changes. Therefore, deeper discussions can be carried out.

  As shown in FIGS. 8 to 15 above, the flow of data is organized by providing a corresponding area in the memory based on the difference between the external device and the internal device, the difference in the type of data, and the like. I can do it. More reliable information processing can be performed.

  The operation of each unit in the above operation and data to be transmitted / received are time-stamped at appropriate time intervals by the data control unit 12 and stored in the recording database 19 for each data. Thereby, those data can be used as minutes of the TV conference. Thereby, the TV conference can be reproduced at a later date. In addition, for example, data output from a specific portable terminal 2-i (example: a specific user's remarks) can be referred to.

(Second Embodiment)
Next, a second embodiment of the information processing system and information processing method of the present invention will be described with reference to the accompanying drawings. In this embodiment, a case will be described in which the information processing system of the present invention is applied to a system that accepts orders at restaurants.

  FIG. 19 is a block diagram showing the configuration of the second embodiment of the information processing system of the present invention. In this configuration, the portable terminal is the information processing apparatus 1 (other portable terminals 2-2 are not directly related), and the database of the information processing apparatus 1 uses the restaurant menu database 16. The database database 13, the participant database 14, and the menu database 18 are the same as those in the first embodiment except that the database is not used, and the description thereof is omitted.

  However, the restaurant menu database 16 stores food menu data that can be ordered at the restaurant and data for displaying it on the display device. As the display device 91 of the portable terminal 2-1, for example, a display device installed for each table in a restaurant is used.

  Here, the configuration of the second embodiment of the portable terminal of the information processing system of the present invention (FIG. 6) and the configuration of the information processing device of the second embodiment of the information processing system of the present invention (FIG. 7). Since this is the same as in the first embodiment, its description is omitted.

Next, the operation (information processing method) of the second embodiment of the information processing system of the present invention will be described.
8 and 20 are flowcharts showing the operation of the second embodiment of the information processing method of the present invention. However, FIG. 8 shows an operation until the portable terminal 2-1 selects one of a plurality of functions provided by the information processing apparatus 1. FIG. 20 shows, as an example, a case where an order for a dish in a restaurant at the portable terminal 2-1 (user) is selected as a function.

  Referring to FIG. 8, regarding the operation of selecting one of a plurality of functions provided by information processing device 1 by portable terminal 2-1, the selected function is an order for a dish at a restaurant. Since it is the same as that of the first embodiment, its description is omitted.

  Next, with reference to FIG. 20, the case where the order of the dish in the restaurant in the portable terminal 2-1 (user) is selected as a function is shown.

(1) Step S151
After the start information is output, the operation data processing unit 12-4 of the information processing apparatus 1 acquires restaurant menu information from the restaurant menu database 16. The restaurant menu information indicates a menu screen of dishes that can be ordered at the restaurant. Then, based on the device information, the restaurant menu information converted (corresponding) so as to be displayed and operated on the display device 91 connected to the portable terminal 2-1 is generated. The converted restaurant menu information has a header indicating that it is data to be displayed on the display device 91 of the portable terminal 2-1 (2) Step S152.
The operation data processing unit 12-4 transmits the restaurant menu information to the portable terminal 2-1 via the network 5.
(3) Step S153
The data recognition unit 23-1 of the portable terminal 2-1 recognizes that the restaurant menu information is data for the display device 91 from the header of the restaurant menu information. Then, the restaurant menu information is stored in the area 72 d in the memory 72. The display data processing unit 23-5 reads the restaurant menu information from the area 72d and outputs it to the display device 91.
(4) Step S154
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display restaurant menu information.
(5) Step S155
Based on the device information, the operation data processing unit 12-4 of the information processing device 1 generates order table information converted (corresponding) so that the portable terminal 2-1 can perform display and operation. The order table information indicates a screen for inputting a dish to be ordered by the user. The converted order table information has a header indicating that it is data to be displayed on the portable terminal 2-1 (6) Step S 156.
The operation data processing unit 12-4 transmits the order table information to the portable terminal 2-1 via the network 5.
(7) Step S157
The data recognition unit 23-1 of the portable terminal 2-1 recognizes that the order table information is data for the built-in display device 73 from the header of the order table information. Then, the order table information is stored in the area 72 c in the memory 72. The built-in display data processing unit 23-3 reads the order table information from the area 72c and displays it on the built-in display device 73. A screen for inputting the food to be ordered by the user is displayed.
(8) Step S158
The operation data processing unit 23-4 of the portable terminal 2-1 determines whether ordering information has been input to the portable terminal 2-1. Order information by a user operation is stored in the area 72a. The ordering information is information indicating the dish that the user orders (selects).
(9) Step S159
When there is ordering information, the operation data processing unit 23-4 causes the built-in display data processing unit 23-3 to display a screen asking whether or not the order has been completed on the built-in display device 73.
(10) Step S160
When the user inputs that the order is not completed, the operation data processing unit 23-4 transmits the order information to the information processing apparatus 1 through the network 5. The order information has a header indicating operation data.
(11) Step S161
The data recognition unit 12-1 of the connection management unit 11 recognizes that the order information is operation data, and stores it in the area 82a. The operation data processing unit 12-4 temporarily stores the portable terminal ID and the ordering information in the storage device 82 in association with each other. Then, with respect to the restaurant menu information, screen processing such as arranging the ordered dishes in the image brought to the table, displaying the number of orders, total calories, and nutritional value is performed, and the process returns to step S151.
(12) Step S162
When the user performs input indicating the end of ordering, the operation data processing unit 23-4 transmits the order end information indicating the end of ordering and the ordering information to the information processing apparatus 1 via the network 5. The order information and the order end information have a header indicating operation data.
(13) Step S163
The data recognition unit 12-1 of the connection management unit 11 recognizes that the ordering information and the ordering end information are operation data, and stores them in the area 82a. The operation data processing unit 12-4 temporarily stores the portable terminal ID and the ordering information in the storage device 82 in association with each other. Then, the operation data processing unit 12-4 transmits all the ordering information related to the portable terminal ID to the information processing apparatus (not shown) in the kitchen via the network 5.

  With the above operation, the information processing method of the present invention relating to ordering of food at a restaurant is carried out.

  According to the present invention, a menu at a restaurant can be distributed to a mobile terminal by a server. Thereby, when the menu is updated, it is only necessary to change data in the server, and it is not necessary to create a plurality of new menus. If the customer does not have a menu, not only can the labor and time required to request the menu from the store clerk be saved, but also a visual grasp of the order details and additional information such as calories and nutritional value can be known.

  In the above embodiment, a restaurant is shown as an example, but the present invention is not limited to this. For example, it can be applied to stores that sell things.

  The operation of each unit in the above operation and data to be transmitted / received are time-stamped at appropriate time intervals by the data control unit 12 and stored in the recording database 19 for each data. Thereby, these data can be used as a record of orders by customers. Thereby, information such as order tendencies and user preferences can be obtained.

(Third embodiment)
Next, a third embodiment of the information processing system and information processing method of the present invention will be described with reference to the accompanying drawings. In this embodiment, a case where the information processing system of the present invention is applied to an on-demand sports relay system will be described.

FIG. 21 is a block diagram showing the configuration of the information processing system according to the third embodiment of the present invention. In this configuration, the portable terminal is the information processing apparatus 1 (other portable terminals 2-2 are not directly related), and the database of the information processing apparatus 1 uses the sports content database 17. The database database 13, the participant database 14, and the menu database 18 are the same as those in the first embodiment except that the data database 13, the participant database 14, and the menu database 18 are not used.
However, the sports content database 17 stores sports relay data (image data), data on players and teams, and data for displaying them on a display device.

  Here, the configuration of the third embodiment of the portable terminal of the information processing system of the present invention (FIG. 6) and the configuration of the information processing apparatus of the information processing system of the present invention (FIG. 7). Since this is the same as in the first embodiment, its description is omitted.

Next, the operation (information processing method) of the third embodiment of the information processing system of the present invention will be described.
8 and 22 are flowcharts showing the operation of the information processing method according to the third embodiment of the present invention. However, FIG. 8 shows an operation until the portable terminal 2-1 selects one of a plurality of functions provided by the information processing apparatus 1. FIG. 22 shows, as an example, a case where the on-demand sports relay on the portable terminal 2-1 (user) is selected as a function.

  Referring to FIG. 8, the operation of selecting one of a plurality of functions provided by the information processing apparatus 1 by the portable terminal 2-1 is the first except that the selected function is an on-demand sports broadcast. Since this is the same as the embodiment, the description thereof is omitted.

  Next, with reference to FIG. 22, the case where the on-demand sports relay by the portable terminal 2-1 (the user) is selected as a function is shown.

(1) Step S181
After the start information is output, the operation data processing unit 12-4 of the information processing apparatus 1 acquires image information from the sports content database 17. The image information indicates an image of a sports broadcast. Based on the device information, the image information converted (corresponding) so as to be displayed and operated on the display device 91 connected to the portable terminal 2-1 is generated. The converted image information has a header indicating that it is data to be displayed on the display device 91 of the portable terminal 2-1.
(2) Step S182
The operation data processing unit 12-4 transmits image information to the portable terminal 2-1 via the network 5.
(3) Step S183
The data recognition unit 23-1 of the portable terminal 2-1 recognizes that the image information is data for the display device 91 from the header of the image information. Then, the image information is stored in the area 72 d in the memory 72. The display data processing unit 23-5 reads the image information from the area 72d and outputs it to the display device 91.
(4) Step S184
The control unit 91-1 of the display device 91 controls the display unit 91-2 to display image information.
(5) Step S185
Based on the device information, the operation data processing unit 12-4 of the information processing device 1 generates command list information converted (corresponding) so that the portable terminal 2-1 can perform display and operation. The command list information indicates a screen for inputting operations that can be executed in connection with sports broadcasting. The operations that can be executed in connection with sports relay are exemplified by operations for displaying player data, team ranking, etc., sports relay rewinding and fast-forwarding. The converted command list information has a header indicating that it is data to be displayed on the portable terminal 2-1 (6) Step S 186.
The operation data processing unit 12-4 transmits command list information to the portable terminal 2-1 via the network 5.
(7) Step S187
The data recognition unit 23-1 of the portable terminal 2-1 recognizes that the command list information is data for the built-in display device 73 from the header of the command list information. Then, the command list information is stored in the area 72 c in the memory 72. The built-in display data processing unit 23-3 reads the command list information from the area 72c and displays it on the built-in display device 73. A screen for inputting operations that can be executed in connection with the sports broadcast is displayed.
(8) Step S188
The operation data processing unit 23-4 of the mobile terminal 2-1 determines whether command information is input to the mobile terminal 2-1. Command information by the user's operation is stored in the area 72a. The command information is information indicating a command operated (selected) by the user.
(9) Step S189
When there is command information, the operation data processing unit 23-4 causes the built-in display data processing unit 23-3 to display a screen on the built-in display device 73 asking whether or not to finish watching the sports broadcast. If the user makes an input indicating that watching a sports broadcast is to end, the process ends.
(10) Step S190
When the user performs an input indicating that watching the sports broadcast is not finished, the operation data processing unit 23-4 transmits the command information to the information processing apparatus 1 via the network 5. The command information has a header indicating operation data.
(11) Step S191
The data recognition unit 12-1 of the connection management unit 11 recognizes that the command information is operation data and stores it in the area 82a. The operation data processing unit 12-4 extracts the request content indicated by the command information from the sports content database 17. Then, based on the device information, request content information obtained by converting (corresponding to) the request content information indicating the request content so as to be displayed and operated on the portable terminal 2-1 is generated.
(12) Step S192
The operation data processing unit 12-4 transmits the request content information to the portable terminal 2-1. The request content information has a header indicating that it is data to be displayed on the built-in display device 73 of the portable terminal 2-1.
(13) Step S193
The data recognition unit 23-1 of the portable terminal 2-1 recognizes that the request content information is data for the built-in display device 73 from the header of the request content information. Then, the request content information is stored in the area 72 c in the memory 72. The built-in display data processing unit 23-3 reads the requested content information from the area 72c and displays it on the built-in display device 73. The content requested by the user is displayed on the screen. The process returns to step S185.

  With the above operation, the information processing method of the present invention related to on-demand sports relay is implemented.

  According to the present invention, it is possible to watch sports broadcasts on demand with a wide screen using a display at a destination. In addition, other information such as player information can be viewed at the same time on the screen of the portable terminal, and it is possible to dynamically enjoy sports watching.

  In the above embodiment, sports relay is shown as an example, but the present invention is not limited to this. For example, the present invention can be applied to a case where a concert, a movie, or a theater is watched by distribution of the information processing apparatus 1.

  The operation of each unit in the above operation and data to be transmitted / received are time-stamped at appropriate time intervals by the data control unit 12 and stored in the recording database 19 for each data. However, the data of the sports broadcast itself is recorded as a data file name, for example. Thereby, those data can be used as a record of a sports broadcast spectator. As a result, it is possible to obtain information such as the required trend of sports broadcast and user preferences.

  In the above-described embodiment, as the information processing, the TV conference using the mobile terminal, the order of dishes at the restaurant, and the on-demand sports relay have been described. However, the present invention is not limited to this case. In other words, it can be used in situations where it is desired to send different information to the built-in display device of the portable terminal and the display device attached to the portable terminal.

For example, when a plurality of users play a game, the built-in display device is used for displaying individual user information that is not shown to other users, and the external display device is used for displaying a common game screen. Data control regarding the game is provided by the information processing apparatus 1.
When used for bidding, the built-in display device and the mobile terminal itself are used to input the bid amount of the user, and the external display device is used to display the product description. Data processing related to bidding is provided by the information processing apparatus 1.
When used in an auction, the built-in display device and the mobile terminal itself are used for the user's bit, and the external display device is used for displaying the product description and the current bit amount. Data control related to the auction is provided by the information processing apparatus 1.
When used in public gambling such as horse racing, the built-in display device and the mobile terminal itself are used for purchasing voting tickets by the user, and the external display device is used for displaying odds and race relay. Data control related to public gambling is provided by the information processing apparatus 1.
In the case of navigation, the built-in display device and the mobile terminal itself are used for inputting and displaying data requests such as enlargement of the scale of the map, and the external display device is used for displaying the entire map. Data control relating to navigation is provided by the information processing apparatus 1.
In the case of image display, the built-in display device and the mobile terminal itself are used for inputting and displaying data requests such as image enlargement, and the external display device is used for displaying the entire image. Data control relating to image display is provided by the information processing apparatus 1.

The cooperation between the two screens of the portable terminal 2-1 and the display device 91 may be performed as follows.
The contents output to the built-in display device 73 of the portable terminal 2-1 and the display unit 91-2 of the display device 91 connected to the portable terminal 2-1 can be enhanced in convenience. For example, in the editing of document data described in the part after the description of FIG. 10, a menu screen for editing is displayed on the built-in display device 73, and the editing work and the reflected editing result are displayed on the connected display device 91. Display in conjunction. Moreover, also in the other usage examples described above, the convenience that the information on the screen displayed on the other screen can be automatically changed according to the content of one screen can be explained. Speaking of sports broadcasts, for example, when a certain player is closed up in the relay content displayed on the external display device 91, the built-in display device 73 of the portable terminal 2-1 displays information about the player. In this example, the information processing apparatus 1 automatically selects and displays the information.

  Further, the application is operated on the information processing apparatus 1, and the information processing apparatus 1 visualizes the operation state of the application so that it can be displayed on the external display device. Then, streaming distribution is performed as data for the external display device to the portable terminal. The external display device displays the operation status. The mobile terminal itself is used for application input (for example, in the case of a mobile phone, movement of the console with the arrow keys and character input with the numeric keypad). The input of the portable terminal (example: position information of the console, input characters) is sequentially transmitted to the information processing apparatus 1. The information processing apparatus 1 operates an application corresponding to the input. Then, the video in which the operation state of the application can be displayed on the external display device is sequentially streamed. In this case, for example, it can be efficiently executed by dividing the memory area of the input / output data in the portable terminal itself and the data for the external display device.

  In this way, the application can be used by transmitting / receiving the operation screen of the application without downloading or deploying the application having a large capacity in the memory of the portable terminal having a limited capacity. When the information processing apparatus 1 is a user's own personal computer (PC), information processing can be performed in the usual PC environment of the user even when away from home.

  In the above embodiment, the external device connection interface is provided with a display device such as a flat panel display, but the present invention is not limited to this example. FIG. 16 is a schematic diagram illustrating another configuration of the display device of the mobile terminal. This display device includes a projection device 91 ′ and an input device 93. That is, it has a function of an input device in addition to the display device. Both are connected to an external device connection interface.

  The input device 93 has a sheet shape and includes a touch panel 94. The projection device 91 ′ combines the content of the screen displayed by the display device 91 in the above description with the keyboard image and projects the synthesized image onto the touch panel 94 of the input device 93. By setting the portable terminal 2-1 at a preset position P and fixing the position and magnification of the image projected by the projection device 91 ′, the contents of the screen and the position of each key on the keyboard are set to the same position every time. be able to. The “keyboard image” is not necessarily required depending on the program to be used, and may be a metaphor such as a button shown in the screen. The “keyboard” here is a representative example of a metaphor such as a button.

  A table indicating the relationship between the position of the contents of the screen and the position on the touch panel 94 and the position of each key of the keyboard and the position on the touch panel 94 is stored in the storage device 72 of the portable terminal 2-1. Thereby, the user's input can be grasped based on which position on the touch panel 94 is touched. The contents of the screen and the keyboard image are synthesized by the built-in display data processing unit 23-3. The touch panel input process is performed by newly providing another operation data processing unit and a dedicated area in the memory.

  FIG. 17 is a block diagram showing a configuration of the projection device 91 ′. The projection device 91 'includes a control unit 91'-1 and a display unit 91'-2. In the display unit 91 ′-2, the light from the lamp 121 passes through the LCD panel 122 on which the screen content and the keyboard are projected, and is projected onto the touch panel 94 through the optical system 123 such as a lens. The control unit 91'-1 displays data for the display device 91 on the display unit 91-2.

  FIG. 18 is a configuration diagram showing another form of the mobile terminal. In this case, in addition to the display device 91, an input device 95 such as a keyboard is connected to the portable terminal via an external device connection interface. In this case, the input process is performed by newly providing another operation data processing unit and a dedicated area in the memory.

  In the above embodiment, different information is displayed on two types of screens. As such a display method, for example, a method disclosed in Japanese Patent Application Laid-Open No. 2003-319097 can be used. As a method of displaying information with two types of screens as one screen, MaxiVista (registered trademark) manufactured by AG-TECH Corporation (/www.agtech.co.jp/) (http: // http: // www. It is also possible to use the technique of maxivista.jp/). As another method for displaying information using two types of screens as one screen, the method disclosed in JP-A-10-198505 can also be used.

  Note that the exchange of information in the information processing systems described in the first to third embodiments is an example, and can be applied to other services (transmission and reception of other data).

1A and 1B are schematic views showing an information processing system of the present invention. FIG. 2 is a block diagram showing the configuration of the first embodiment of the information processing system of the present invention. FIG. 3 is a table showing an example of the participant database 14. FIG. 4 is a table showing an example of the connected device database 15. FIG. 5 is a table showing an example of the menu database 18. FIG. 6 is a block diagram showing the configuration of the embodiment of the portable terminal of the information processing system of the present invention. FIG. 7 is a block diagram showing the configuration of the embodiment of the information processing apparatus of the information processing system of the present invention. FIG. 8 is a flowchart showing the operation of the embodiment of the information processing method of the present invention. FIG. 9 is a flowchart showing the operation of the first embodiment of the information processing method of the present invention. FIG. 10 is a flowchart showing an operation of displaying data on the display device in step S15. FIG. 11 is a flowchart showing an operation of displaying on the mobile terminal 2-2 a user image taken by the mobile terminal 2-1, in step S15. FIG. 12 is a flowchart showing an operation in which the user speaks in step S15. FIG. 13 is a flowchart showing an operation in which the user speaks in step S15. FIG. 14 is a flowchart showing an operation of making a majority decision in step S15. FIG. 15 is a flowchart showing the operation of making a majority decision in step S15. FIG. 16 is a schematic diagram illustrating another configuration of the display device of the mobile terminal. FIG. 17 is a block diagram illustrating a configuration of the projection apparatus. FIG. 18 is a configuration diagram showing another form of the mobile terminal. FIG. 19 is a block diagram showing the configuration of the second embodiment of the information processing system of the present invention. FIG. 20 is a flowchart showing the operation of the second embodiment of the information processing method of the present invention. FIG. 21 is a block diagram showing the configuration of the information processing system according to the third embodiment of the present invention. FIG. 22 is a flowchart showing the operation of the third embodiment of the information processing method of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 2, 3 Portable terminal 5 Network 11 Connection management part 12 Data control part 12-1 Data recognition part 12-2 Voice data processing part 12-3 Built-in display data processing part 12-4 Operation data processing part 12-5 Display data processing unit 12-6 Search unit 13 Document database 14 Participant database 15 Connected device database 16 Restaurant menu database 17 Sports content database 18 Menu database 19 Record database 21, 31 Accessory device management unit 22, 32 Connection management unit 23, 33 Data control unit 23-1 Data recognition unit 23-2 Audio data processing unit 23-3 Built-in display data processing unit 23-4 Operation data processing unit 70 Camera 71 CPU
72 Memory 72a, 72b, 72c, 72d Area 73 Built-in display device 74 Key input unit 75 Microphone 76 Speaker 77 Communication unit 78 Storage device 79 External device connection interface 81 CPU
82 Memory 82a, 82b, 82c Area 83, 84 External device connection interface 85 Communication unit 86 Storage device 91 Display device 91 'Projection device 91'-1 Control unit 91'-2 Display unit 93 Input device 94 Touch panel 96 Keyboard 97 Display Device 121 Lamp 122 LCD panel 123 Optical system

Claims (34)

A first portable information terminal that is carried by the first user and is connectable to a first display device at the destination of the first user;
An information processing apparatus communicably connected to the first portable information terminal via a network,
The first portable information terminal acquires first display device information related to the first display device from the first display device, and the first terminal information related to the first portable information terminal and the first display device information are Transmitted to the information processing device,
The information processing device can display the first information on the first display device for the stored first information and second information based on the first display device information and the first terminal information. The second information is converted to be outputable by the first portable information terminal, and is transmitted to the first portable information terminal.
The first portable information terminal outputs the second information and transmits the first information to the first display device so that the first display device displays the first information. The information processing system according to claim 1,
The first information is stored in advance in the information processing apparatus,
The first portable information terminal transmits a first request signal including a request for the first information to the information processing apparatus,
The information processing device transmits the first information converted to be displayable on the first display device to the first portable information terminal based on the first request signal. The information processing system according to claim 1 or 2,
The first portable information terminal includes a storage unit,
The storage unit
A first storage area preset for input / output of the first information;
And a second storage area set in advance for input / output of the second information. In the information processing system according to any one of claims 1 to 3,
A second portable information terminal that is carried by the second user and is connectable to a second display device at the destination of the second user and is communicably connected to the information processing device via the network. ,
The second portable information terminal acquires second display device information related to the second display device from the second display device, and obtains the second terminal information related to the second portable information terminal and the display device information as the information. Sent to the processing device,
The information processing device can display the third information on the second display device for the stored third information and fourth information based on the second terminal information and the second display device information. The fourth information is converted to be output by the second portable information terminal, and is transmitted to the second portable information terminal.
The second portable information terminal outputs the fourth information and transmits the third information to the second display device so that the second display device displays the third information. The information processing system according to claim 4,
The third information is stored in advance in the information processing apparatus,
The first portable information terminal or the second portable information terminal transmits a second request signal including a request to output the third information to the second portable information terminal, to the information processing apparatus,
The information processing device transmits the third information converted to be displayable on the second display device to the second portable information terminal based on the second request signal. In the information processing system according to claim 4 or 5,
The second information is an image of the first user taken by the camera of the first portable information terminal, or the voice of the first user received by the first portable information terminal,
The fourth information is an image of the second user taken by a camera of the second portable information terminal or a voice of the second user received by the second portable information terminal. The information processing system according to any one of claims 1 to 6,
The information processing apparatus includes a database that associates and stores terminal information including the first terminal information and handicap information indicating an unfavorable condition when receiving an information processing service related to a user including the first user,
The information processing apparatus refers to the database when converting the first information, and converts the first information so as to be adapted to the handicap information. The information processing system according to any one of claims 1 to 7,
The information processing apparatus is an information processing system belonging to the first user. First display device information relating to the first display device, transmitted via a network from a first portable information terminal carried by the first user and connected to the first display device at the destination of the first user; and A transmitting / receiving unit that receives first terminal information related to the first portable information terminal;
A first data processing unit that converts the stored first information into first conversion information that can be displayed on the first display device, based on the first display device information;
A second data processing unit that converts the stored second information into second conversion information that can be output by the first portable information terminal based on the first terminal information;
The transmission / reception unit transmits the first conversion information and the second conversion information to the first portable information terminal via the network. The information processing apparatus according to claim 9,
A storage unit including a preset first storage area used for input / output of the first information and a preset second storage area used for input / output of the second information;
The first data processing unit converts the first information into the first conversion information based on a first request signal including the request for the first information transmitted from the first portable information terminal. Store in one storage area,
The transmission / reception unit acquires the first conversion information from the first storage area and transmits the first conversion information to the first portable information terminal. The information processing apparatus according to claim 9 or 10,
The transmission / reception unit relates to the second display device transmitted via the network from a second portable information terminal that is carried by the second user and connected to a second display device at a destination of the second user. 2 display device information and second terminal information related to the second portable information terminal,
The first data processing unit converts the stored third information into third conversion information that can be displayed on the second display device, based on the second display device information,
The second data processing unit converts the stored fourth information into fourth conversion information that can be output by the second portable information terminal based on the second terminal information,
The transmission / reception unit transmits the third conversion information and the fourth conversion information to the second portable information terminal via the network. The information processing apparatus according to claim 11,
The third information is stored in the first storage area, and the fourth information is stored in the second storage area.
The first data processing unit, based on a second request signal including a request to output the third information to the second portable information terminal transmitted from the first portable information terminal or the second portable information terminal, Converting third information into the third conversion information and storing it in the first storage area;
The transmission / reception unit acquires the third conversion information from the first storage area and transmits the third conversion information to the second portable information terminal. The information processing apparatus according to claim 11 or 12,
The second information is an image of the first user taken by the camera of the first portable information terminal, or the voice of the first user received by the first portable information terminal,
The fourth information is an image of the second user taken by a camera of the second portable information terminal or a voice of the second user received by the second portable information terminal. The information processing apparatus according to any one of claims 9 to 13,
A database for storing terminal information including the first terminal information and handicap information indicating disadvantageous conditions when receiving an information processing service regarding a user including the first user in association with each other;
When converting the first information into the first conversion information, the first data processing unit refers to the database and converts the first information so as to be adapted to the handicap information. The information processing apparatus according to claim 14,
When converting the second information into the second conversion information, the second data processing unit refers to the database and converts the second information so as to be adapted to the handicap information. (A) the first portable information terminal carried by the first user is connected to the first display device at the destination of the first user;
(B) the first portable information terminal obtaining first display device information relating to the first display device from the first display device;
(C) the first portable information terminal transmitting the first terminal information related to the first portable information terminal and the first display device information to the information processing device via a network;
(D) The information processing device converts the stored first information so as to correspond to the first display format of the first display device based on the first display device information, and Transmitting to the first portable information terminal via,
(E) The first display device includes a step of displaying the first information received by the first portable information terminal. The information processing method according to claim 16,
The step (d) includes:
(D1) Based on the first terminal information, the information processing apparatus converts menu information indicating a menu of a service provided by the information processing apparatus to be output by the first portable information terminal, and Transmitting to the first portable information terminal via,
(D2) the first portable information terminal outputting the menu information;
(D3) transmitting the selected menu information selected from the menu information to the information processing apparatus via the network;
(D4) Based on the first display device information, the first information related to the selection menu information is converted so as to correspond to the first display format of the first display device, and the network information is transmitted via the network. Transmitting to the first portable information terminal. An information processing method. The information processing method according to claim 16 or 17,
The step (c) includes:
(C1) the first portable information terminal transmits a connection request indicating a connection request to the information processing apparatus in addition to the first terminal information and the first display device information;
(C2) After the connection with the information processing apparatus is established, the first portable information terminal includes a step of transmitting a request for the first information to the information processing apparatus. The information processing method according to any one of claims 16 to 18,
(F) a step in which a second portable information terminal carried by a second user connects to a second display device at a destination of the second user;
(G) the second portable information terminal obtaining second display device information relating to the second display device from the second display device;
(H) the second portable information terminal transmitting the second terminal information related to the second portable information terminal and the second display device information to the information processing device via the network;
(I) Based on the second display device information, the information processing device converts the stored second information so as to correspond to the second display format of the second display device, and Transmitting to the second portable information terminal via,
(J) The second display device further includes a step of displaying the second information received by the second portable information terminal. The information processing method according to claim 19,
(K) the first portable information terminal or the second portable information terminal transmitting third information to the information processing apparatus via the network;
(L) The information processing apparatus transmits the third information to the second portable information terminal or the first portable information terminal via the network;
An information processing method further comprising: The information processing method according to claim 20,
The step (l)
(L1) The information processing apparatus converts the third information so as to correspond to the second display format or the first display format;
(L2) The second display device displays the third information received by the second portable information terminal, or the first display device displays the third information received by the first portable information terminal. An information processing method comprising steps. The information processing method according to claim 20 or 21,
The third information includes the video of the first user taken by the camera of the first portable information terminal, the voice of the first user received by the first portable information terminal, and the camera of the second portable information terminal. An information processing method including the video of the second user or the voice of the second user received by the second portable information terminal. The information processing method according to claim 16 or 17,
The step (d) includes:
(D5) When converting the first information, the terminal information including the first terminal information and the handicap information indicating an unfavorable condition when receiving the information processing service related to the user including the first user are stored in association with each other. An information processing method comprising: converting the first information so as to adapt to the handicap information with reference to a database to be processed. (M) a first display device relating to the first display device, which is transmitted via a network from a first portable information terminal which is carried by the first user and connected to the first display device which is the destination of the first user. Receiving information and first terminal information about the first portable information terminal;
(N) based on the first display device information, converting the stored first information into first conversion information that can be displayed on the first display device;
(O) converting the stored second information into second conversion information that can be output by the first portable information terminal based on the first terminal information;
(P) A program for causing a computer to execute a method comprising: transmitting the first conversion information and the second conversion information to the first portable information terminal via the network. The program according to claim 24,
The step (m)
(M1) Based on the first terminal information, the menu information indicating the menu of the service to be provided is converted to be outputable by the first portable information terminal and transmitted to the first portable information terminal via the network. Steps,
(M2) receiving the selected menu information selected from the menu information from the first portable information terminal via the network;
(M3) A program comprising: acquiring the first information related to the selection menu information. The program according to claim 25,
The step (n) includes
(N1) Based on a first request signal including the request for the first information transmitted from the first portable information terminal, the first information is converted into the first conversion information, and preset in the storage unit. Storing in the first storage area, wherein the first storage area is different from a second storage area set in advance for storing the second information,
The step (p) includes:
(P1) A program comprising the step of acquiring the first conversion information from the first storage area and transmitting the first conversion information to the first portable information terminal. The program according to any one of claims 24 to 26,
(Q) the second display device transmitted via the network from a second portable information terminal that is carried by the second user and connected to the second display device at the destination location of the second user; Receiving two display device information and second terminal information about the second portable information terminal;
(R) converting the stored third information into third conversion information that can be displayed on the second display device based on the second display device information;
(S) based on the second terminal information, converting the stored fourth information into fourth conversion information that can be output by the second portable information terminal;
(T) A program further comprising: transmitting the third conversion information and the fourth conversion information to the second portable information terminal via the network. The program according to claim 27,
The step (r) includes
(R1) Based on a second request signal including a request for outputting the third information to the second portable information terminal transmitted from the first portable information terminal or the second portable information terminal, the third information is obtained. Converting to the third conversion information and storing the third conversion information in a preset third storage area in the storage unit, wherein the third storage area is preset to store the fourth information Unlike the fourth storage area
The step (t) includes:
(T1) A program comprising a step of acquiring the third conversion information from the third storage area and transmitting the third conversion information to the second portable information terminal. The program according to claim 27 or 28,
The second information is an image of the first user taken by the camera of the first portable information terminal, or the voice of the first user received by the first portable information terminal,
The fourth information is a video of the second user taken by a camera of the second portable information terminal or a voice of the second user received by the second portable information terminal. The program according to any one of claims 24 to 29,
The step (n) includes
(N2) When converting the first information into the first conversion information, a handicap indicating an unfavorable condition when receiving terminal information including the first terminal information and an information processing service related to the user including the first user A program comprising a step of converting the first information so as to be adapted to the handicap information with reference to a database storing information in association with the handicap information. The program according to claim 30, wherein
The step (o)
(O1) A program comprising a step of converting the second information so as to be adapted to the handicap information with reference to the database when the second information is converted into the second conversion information. (U) detecting a connection with the first display device, and obtaining first display device information relating to the first display device from the first display device;
(V) transmitting the first display device information to the information processing device via a network;
(W) receiving the first information converted so as to correspond to the first display format of the first display device from the information processing device via the network;
(X) transmitting the received first information to the first display device. The program according to claim 32,
The step (v)
(V1) transmitting the first display device information to the information processing device;
(V2) A program comprising: transmitting a request for the first information to the information processing device after connection with the information processing device is established. The program according to claim 32 or 33,
The step (w)
(W1) A program comprising the steps of: storing the first information in a preset first storage area of a storage unit; and storing preset second information different from the first information in a second storage area.

Images