JP2019068443A - Information processing device and program - Google Patents

Abstract

To provide an information processing device and a program capable of dealing with a failure of execution of a cooperation function.SOLUTION: In a case where a device required to execute a cooperation function is designated and the execution of the cooperation function fails during the execution of the cooperation function, the device is changed. In the case of changing a device, a device of a change destination is selected from among devices capable of executing the cooperation function, and it is reported that only a device which has caused the failure of the cooperation function is to be changed.SELECTED DRAWING: Figure 27K

Description

  The present invention relates to an information processing apparatus and program.

  A system in which the device body and the user interface are separated may be used. For example, Patent Document 1 discloses a technique for displaying an image indicating guidance of an operation on an operation panel that can be wirelessly communicated with an apparatus body.

  Patent Document 2 discloses a relay apparatus that relays data between a client apparatus and a service providing apparatus that provides a service.

  Patent Document 3 discloses a portable terminal that generates an augmented reality image in which additional information including state information of an image forming apparatus is superimposed on a captured image captured by an imaging unit.

  Patent Document 4 discloses a technology for displaying the usage status of each cloud server by a user on a display of a device operated by the user at the time of login in a communication system that communicates with a plurality of cloud servers.

Patent No. 5737906 JP, 2014-10769, A JP, 2014-48672, A JP 2014-238786 A

  By the way, regarding the use environment of apparatuses such as an image forming apparatus, generally, a use environment in which a plurality of users use one apparatus is assumed. On the other hand, a use environment in which a plurality of users use a plurality of devices may be assumed in the future. In addition, it may be assumed that a user interface such as a touch panel is removed from the device, or the device is temporarily used at the place where the user goes. In such a usage environment, there are cases where the user can not easily understand what functions can be executed depending on the device that the user is trying to use.

  An object of the present invention is to cope with the failure of the execution of the cooperation function.

  The invention according to claim 1 is the information having control means for changing the device when the device necessary for executing the cooperation function is designated and the execution of the cooperation function fails during the execution of the cooperation function. It is a processing device.

  The invention according to claim 2 is the information processing apparatus according to claim 1, wherein, when the device is changed, the device of the change destination is selected from the devices capable of executing the cooperation function. is there.

  The invention according to claim 4 is characterized in that, when changing the device, the control means further notifies to change only the device which has caused the failure of the cooperation function. Or it is an information processing device according to claim 2.

  The invention according to claim 4 is a control means for changing the device when the device necessary for executing the cooperation function is designated and the execution of the cooperation function fails while the cooperation function is being executed. It is a program to function as

  According to the invention which concerns on Claim 1-4, the failure of execution of a cooperation function can be coped with.

FIG. 1 is a block diagram showing an image forming system according to a first embodiment of the present invention. FIG. 1 is a block diagram showing an image forming apparatus according to a first embodiment. It is a block diagram showing a server concerning a 1st embodiment. It is a block diagram showing a terminal unit concerning a 1st embodiment. FIG. 1 is a schematic view showing an appearance of an image forming apparatus. It is a figure which shows the function purchasing screen displayed on a terminal device. It is a figure which shows the function display screen displayed on a terminal device. It is a figure which shows the function display screen displayed on a terminal device. It is a figure which shows the function display screen displayed on a terminal device. It is a sequence diagram showing a function purchase process. It is a flowchart which shows the display process of a function display screen. It is a flowchart which shows the display process of a function display screen. It is a flowchart which shows the display process of a function display screen. FIG. 7 is a block diagram showing an image forming system according to a second embodiment of the present invention. It is a block diagram showing the server concerning a 2nd embodiment. It is a schematic diagram which shows the apparatus of cooperation object. It is a schematic diagram which shows the apparatus of cooperation object. It is a figure which shows the screen of the display part of a terminal device. It is a figure which shows the screen of the display part of a terminal device. It is a schematic diagram which shows each apparatus arrange | positioned in a search area. It is a sequence diagram which shows the process by the image forming system which concerns on 2nd Embodiment. It is a figure which shows the transition of the screen of a terminal device. It is a figure which shows the execution priority of a cooperation function. It is a block diagram showing the server concerning a 3rd embodiment. It is a block diagram showing a server concerning a 4th embodiment. It is a figure for demonstrating the process by the image forming system which concerns on 4th Embodiment. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of a priority display. It is a figure which shows an example of a priority display. It is a figure which shows an example of a priority display. It is a figure which shows an example of a priority display.

First Embodiment
An image forming system as an information processing system according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 shows an example of the image forming system according to the first embodiment. The image forming system according to the first embodiment includes an image forming apparatus 10 as an apparatus, a server 12 and a terminal device 14 as an information processing apparatus. The image forming apparatus 10, the server 12 and the terminal device 14 are connected to one another via a communication path N such as a network. In the example shown in FIG. 1, although one image forming apparatus 10, one server 12 and one terminal device 14 are included in the image forming system, a plurality of image forming apparatuses 10, a plurality of servers 12 and a plurality of terminals Apparatus 14 may be included in the imaging system.

  The image forming apparatus 10 is an apparatus having an image forming function. Specifically, the image forming apparatus 10 is an apparatus provided with at least one of a scan function, a print function, a copy function, and a facsimile function. In addition, the image forming apparatus 10 has a function of transmitting and receiving data to and from another apparatus.

  The server 12 is a device that manages functions available to the user for each user. For example, the function purchased by the user is a function available to the user, and the server 12 manages the function purchase history for each user. Of course, since there are also functions available free of charge, functions of additional update data, functions specially managed by the administrator, etc., the server 12 is not merely managing functions according to the presence or absence of purchase. The function purchase process is performed by, for example, the server 12. Also, the server 12 is a device that executes a specific function. The specific function executed by the server 12 is, for example, a function related to image processing. The functions managed by the server 12 are, for example, a function executed using the image forming apparatus 10, a function executed by the server 12, and the like. The management of the function purchase history and the execution of the specific function may be performed by different servers 12 or may be performed by the same server 12. The server 12 also has a function of transmitting and receiving data to and from other devices.

  The terminal device 14 is a device such as a PC (personal computer), a tablet PC, a smartphone, a mobile phone, etc., and has a function of transmitting and receiving data to and from another device. The terminal device 14 functions as a user interface unit (UI unit) of the image forming apparatus 10 when the image forming apparatus 10 is used.

  In the image forming system according to the first embodiment, a function is purchased by the user by using the terminal device 14, and the history of the purchase is managed by the server 12 as a function purchase history. The function purchased by the user is executed by, for example, the image forming apparatus 10 or the server 12.

  Hereinafter, the configuration of the image forming apparatus 10 will be described in detail with reference to FIG. The configuration of the image forming apparatus 10 is shown in FIG.

  The communication unit 16 is a communication interface, and has a function of transmitting data to another device via the communication path N, and a function of receiving data from the other device. The communication unit 16 may be a communication interface having a wireless communication function, or may be a communication interface having a wired communication function.

  The image forming unit 18 performs a function related to image formation. Specifically, the image forming unit 18 executes at least one of the scan function, the print function, the copy function, and the facsimile function. By executing the scan function, the document is read and scan data (image data) is generated. By executing the print function, an image is printed on a recording medium such as paper. By executing the copy function, the original is read and printed on the recording medium. By executing the facsimile function, image data is transmitted by facsimile or received by facsimile. Also, a function combining a plurality of functions may be executed. For example, a scan transfer function combining the scan function and the transmission function (transfer function) may be performed. By executing the scan transfer function, the document is read to generate scan data (image data), and the scan data is transmitted to a transmission destination (for example, an external device such as the terminal device 14). Of course, this combination function is only an example, and another combination function may be performed.

  The storage unit 20 is a storage device such as a hard disk. The storage unit 20 includes information (for example, job information etc.) indicating an image formation command, image data to be printed, scan data generated by executing a scan function, various control data, various programs, Etc. are stored. Of course, they may be stored in separate storage devices, or may be stored in one storage device.

  The UI unit 22 is a user interface unit, and includes a display unit and an operation unit. The display unit is, for example, a display device such as a liquid crystal display. The operation unit is, for example, an input device such as a touch panel or a keyboard. The image forming apparatus 10 may not include the UI unit 22 or may include a hardware user interface unit (hardware UI unit) as hardware without including the display unit. The hardware UI unit is, for example, a hardware key specialized for numeric input (for example, a ten key), a hardware key specialized for direction indication (for example, a direction indication key), or the like.

  The control unit 24 controls the operation of each unit of the image forming apparatus 10.

  Hereinafter, the configuration of the server 12 will be described in detail with reference to FIG. The configuration of the server 12 is shown in FIG.

  The communication unit 26 is a communication interface, and has a function of transmitting data to another device via the communication path N, and a function of receiving data from the other device. The communication unit 26 may be a communication interface having a wireless communication function, or may be a communication interface having a wired communication function.

  The storage unit 28 is a storage device such as a hard disk. The storage unit 28 stores device function information 30, function purchase history information 32, a program for executing a specific function, and the like. Of course, they may be stored in separate storage devices, or may be stored in one storage device. Here, the device function information 30 and the function purchase history information 32 will be described.

  The device function information 30 is information indicating a function group of each image forming apparatus 10 included in the image forming system, and, for example, device identification information for identifying the image forming apparatus 10 for each image forming apparatus 10, This is information indicating the correspondence with the function identification information for identifying each function that the image forming apparatus 10 has. The device identification information is, for example, a device ID, a device name, a model number, position information, and the like. The function identification information is, for example, a function ID or a function name. For example, when an image forming apparatus 10 has a scan function, a print function, a copy function, and a scan transfer function, the device identification information of the image forming apparatus 10 includes function identification information indicating a scan function, and a print function. The indicated function identification information, the function identification information indicating the copy function, and the function identification information indicating the scan transfer function are associated with each other. By referring to the device function information 30, the function group that each image forming apparatus 10 has is specified.

  The function purchase history information 32 is information indicating the purchase history of functions by each user, that is, information indicating functions purchased by each user, and, for example, user identification information for identifying the user for each user, This is information indicating the correspondence with the function identification information indicating the function purchased by the user. The user identification information is, for example, user account information such as a user ID and a name. The features purchased by a user are the features available to the user. By referring to the function purchase history information 32, the functions purchased by each user, that is, the functions available to the user for each user are identified. The function purchase history information 32 is updated, for example, each time the user purchases a function.

  The function execution unit 34 executes a specific function. For example, when the user designates a specific function using the terminal device 14 and then instructs execution of the function, the function execution unit 34 executes the function designated by the user. The function execution unit 34 executes functions related to image processing such as, for example, a character recognition function, a translation function, an image processing function, and an image forming function. Of course, the function execution unit 34 may execute a function related to processing other than image processing. By executing the character recognition function, characters in the image are recognized and character data indicating the characters is generated. By executing the translation function, characters in the image are translated into characters represented in a specific language, and character data indicating the translated characters is generated. The image is processed by executing the image processing function. For example, the function execution unit 34 receives, from the image forming apparatus 10, scan data generated by executing the scan function, and an image of a character recognition function, a translation function, an image processing function, etc. for the scan data. Execute functions related to processing. The function execution unit 34 may receive image data from the terminal device 14 and execute each function on the image data. The character data and image data generated by the function execution unit 34 are transmitted from the server 12 to the terminal device 14, for example.

  The control unit 36 controls the operation of each unit of the server 12. Further, the control unit 36 includes a purchase processing unit 38, a purchase history management unit 40, and a specifying unit 42.

  The purchase processing unit 38 executes purchase processing of the function. For example, when a paid function is purchased by a user, the purchase processing unit 38 applies a charging process to the user. The functions purchased by the user become available to the user, and the functions not purchased by the user become unavailable to the user.

  The purchase history management unit 40 manages the purchase history of the function purchased by the user for each user, and generates the function purchase history information 32 indicating the purchase history. The purchase history management unit 40, for example, updates the function purchase history information 32 each time a user purchases a function. The information included in the function purchase history information 32 is displayed on the terminal device 14 as a function purchase screen, for example, when the user purchases a function or confirms the purchased function. The function purchase screen will be described in detail later with reference to FIG.

  The specifying unit 42 receives device identification information for identifying the image forming apparatus 10 to be used, and the function of each function associated with the device identification information in the device function information 30 stored in the storage unit 28. Identify identification information. Thereby, the function group of the image forming apparatus 10 to be used is specified (recognized). For example, device identification information is transmitted from the terminal device 14 to the server 12, and the identification unit 42 identifies function identification information of each function associated with the device identification information. The function identification information (for example, information indicating the name of each function) of each function is transmitted from the server 12 to the terminal device 14, for example, and displayed on the terminal device 14. Thereby, the function identification information of each function of the image forming apparatus 10 specified by the device identification information is displayed on the terminal device 14.

  In addition, the identification unit 42 receives user identification information for identifying the user, and the function identification information of each function associated with the user identification information in the function purchase history information 32 stored in the storage unit 28 Identify. Thereby, the function group purchased by the user, that is, the function group available by the user is specified (recognized). For example, user identification information is transmitted from the terminal device 14 to the server 12, and the identification unit 42 identifies function identification information of each function associated with the user identification information. The function identification information (for example, information indicating the name of each function) of each function is transmitted from the server 12 to the terminal device 14, for example, and displayed on the terminal device 14. Thereby, the function identification information of each function available by the user specified by the user identification information is displayed on the terminal device 14.

  For example, the identification unit 42 receives the device identification information and the user identification information, identifies the function identification information of each function associated with the device identification information in the device function information 30, and, in the function purchase history information 32, The function identification information of each function associated with the user identification information is specified. As a result, a function group possessed by the image forming apparatus 10 specified by the device identification information, and a function group usable by the user specified by the user identification information is specified (recognized). The function identification information of each function that each image forming apparatus 10 has and can be used by the user is, for example, transmitted from the server 12 to the terminal device 14 and displayed on the terminal device 14. As a result, function identification information of each function that each image forming apparatus 10 has and that can be used by the user is displayed on the terminal device 14.

  The function identification information of each function of the image forming apparatus 10 to be used and the function identification information of each function available to the user are displayed on the terminal device 14 as a function display screen, for example. The function display screen will be described in detail later with reference to FIG.

  In this embodiment, as an example, by applying AR (Augmented Reality) technology (Augmented Reality technology), device identification information is acquired, and the image forming apparatus 10 to be used is identified (recognized). AR technology known as AR technology is used. For example, a marker type AR technology using a marker such as a two-dimensional barcode, a markerless type AR technology using an image recognition technology, a position information AR technology using position information, or the like is used. Of course, device identification information may be acquired without applying AR technology, and the image forming apparatus 10 to be used may be identified.

  Hereinafter, the configuration of the terminal device 14 will be described in detail with reference to FIG. The configuration of the terminal device 14 is shown in FIG.

  The communication unit 44 is a communication interface, and has a function of transmitting data to another device via the communication path N, and a function of receiving data from the other device. The communication unit 44 may be a communication interface having a wireless communication function, or may be a communication interface having a wired communication function. The camera 46 as a photographing unit generates image data (for example, still image data and moving image data) by photographing an object to be photographed. The storage unit 48 is a storage device such as a hard disk or an SSD, and includes various programs, various data, address information of the server 12, address information of each device (for example, address information of each image forming apparatus 10) It stores information on devices, information on linked functions, etc. The UI unit 50 is a user interface unit, and includes a display unit and an operation unit. The display unit is, for example, a display device such as a liquid crystal display. The operation unit is, for example, an input device such as a touch panel, a keyboard, or a mouse. The control unit 52 controls the operation of each unit of the terminal device 14. The control unit 52 functions as, for example, a display control unit, and causes the display unit of the UI unit 50 to display a function purchase screen and a function display screen.

  The device function information 30 described above may be stored in the storage unit 48 of the terminal device 14. In this case, the device function information 30 may not be stored in the storage unit 28 of the server 12. Similarly, the function purchase history information 32 described above may be stored in the storage unit 48 of the terminal device 14. In this case, the function purchase history information 32 may not be stored in the storage unit 28 of the server 12. In addition, the control unit 52 of the terminal device 14 may include the above-described purchase history management unit 40 and may manage the history of purchase of the function by the user using the terminal device 14. In this case, the server 12 may not include the purchase history management unit 40. Further, the control unit 52 of the terminal device 14 includes the above-described identification unit 42, identifies the image forming apparatus 10 based on the device identification information, and identifies a function available to the user based on the user identification information. You may In this case, the server 12 may not include the specifying unit 42.

  The process for acquiring device identification information of the image forming apparatus 10 will be described in detail below with reference to FIG. The appearance of the image forming apparatus 10 is schematically shown in FIG. Here, processing for acquiring device identification information by applying the marker type AR technology will be described. The case of the image forming apparatus 10 is provided with a marker 54 such as a two-dimensional bar code. The marker 54 is information in which device identification information of the image forming apparatus 10 is coded. The user activates the camera 46 of the terminal device 14 and photographs the marker 54 provided in the image forming apparatus 10 to be used with the camera 46. Thereby, image data representing the marker 54 is generated. The image data is transmitted from the terminal device 14 to the server 12, for example. In the server 12, the control unit 36 extracts device identification information by applying a decoding process to the marker image represented in the image data. As a result, the image forming apparatus 10 to be used (the image forming apparatus 10 having the photographed marker 54) is specified (recognized). The specifying unit 42 of the server 12 specifies the function identification information of each function associated with the extracted device identification information in the device function information 30. Thereby, the function of the image forming apparatus 10 to be used is specified.

  The control unit 52 of the terminal device 14 may extract the device identification information by applying the decoding process to the image data representing the marker 54. In this case, the extracted device identification information is transmitted from the terminal device 14 to the server 12. The specifying unit 42 of the server 12 specifies the function identification information of each function associated with the device identification information transmitted from the terminal device 14 in the device function information 30. When the device function information 30 is stored in the storage unit 48 of the terminal device 14, the control unit 52 of the terminal device 14 is associated with the device identification information extracted by the control unit 52 in the device function information 30. The function identification information of each function may be identified.

  The marker 54 may include function identification information of each function of the image forming apparatus 10 in a coded form. In this case, by applying the decoding process to the image data representing the marker 54, the device identification information of the image forming apparatus 10 is extracted, and the function identification information of each function of the image forming apparatus 10 is also extracted. . Thus, the image forming apparatus 10 is specified, and each function of the image forming apparatus 10 is specified. This decoding process may be performed by the server 12 or may be performed by the terminal device 14.

  When the device identification information is acquired by applying the markerless AR technology, for example, the user captures all or part of the appearance of the image forming apparatus 10 to be used by the camera 46 of the terminal device 14. Of course, it is useful to obtain information for identifying the device, such as the name of the device to be used (for example, a product name) and a model number from the appearance. By photographing, appearance image data representing all or part of the appearance of the image forming apparatus 10 to be used is generated. The appearance image data is transmitted from, for example, the terminal device 14 to the server 12. In the server 12, the control unit 36 specifies the image forming apparatus 10 to be used based on the appearance image data. For example, the storage unit 28 of the server 12 associates, for each image forming apparatus 10, appearance image data representing all or part of the appearance of the image forming apparatus 10 with device identification information of the image forming apparatus 10 The external appearance image correspondence information which shows is stored. For example, the control unit 36 compares the appearance image data transmitted from the terminal device 14 with the appearance image data included in the appearance image correspondence information, and based on the comparison result, the control unit 36 Identify device identification information. For example, the control unit 36 extracts the feature of the appearance of the image forming apparatus 10 to be used from the appearance image data transmitted from the terminal device 14, and the appearance image data group included in the appearance image correspondence information has the appearance Appearance image data representing a feature that is the same as or similar to the feature is identified, and device identification information associated with the appearance image data is identified. Thereby, the image forming apparatus 10 to be used (the image forming apparatus 10 photographed by the camera 46) is specified (recognized). As another example, when the name (for example, a product name) or model number of the image forming apparatus 10 is captured and appearance image data representing the name or model number is generated, the name or type represented in the appearance image data The image forming apparatus 10 to be used may be specified based on the number. The specifying unit 42 of the server 12 specifies the function identification information of each function associated with the specified device identification information in the device function information 30. Thereby, the function of the image forming apparatus 10 to be used is identified (recognized).

  The control unit 52 of the terminal device 14 compares appearance image data representing all or part of the appearance of the image forming apparatus 10 to be used with each appearance image data included in the appearance image association information, and compares the comparison. The device identification information of the image forming apparatus 10 to be used may be specified based on the result. The appearance image correspondence information may be stored in the storage unit 48 of the terminal device 14. In this case, the control unit 52 of the terminal device 14 specifies the device identification information of the image forming apparatus 10 to be used by referring to the appearance image correspondence information stored in the storage unit 48 of the terminal device 14. Alternatively, the control unit 52 of the terminal device 14 acquires appearance image correspondence information from the server 12 and specifies device identification information of the image forming apparatus 10 to be used by referring to the appearance image correspondence information. It is also good.

  When the device identification information is acquired by applying the position information AR technology, for example, by using a GPS (Global Positioning System) function, the position information indicating the position where the image forming apparatus 10 is installed is acquired. For example, each image forming apparatus 10 has a GPS function, and acquires device position information indicating the position of the image forming apparatus 10 itself. The terminal device 14 outputs information indicating an acquisition request for device position information to the image forming apparatus 10 to be used, and the device position information of the image forming apparatus 10 from the image forming apparatus 10 as a response to the acquisition request. Receive The device position information is transmitted from the terminal device 14 to the server 12, for example. In the server 12, the control unit 36 specifies the image forming apparatus 10 to be used based on the device position information. For example, the storage unit 28 of the server 12 associates, for each image forming apparatus 10, correspondence between device position information indicating the position where the image forming apparatus 10 is installed and device identification information of the image forming apparatus 10. Position correspondence information indicating is stored. The control unit 36 identifies, in the position association information, device identification information associated with the device position information transmitted from the terminal device 14. Thus, the image forming apparatus 10 to be used is identified (recognized). The specifying unit 42 of the server 12 specifies the function identification information of each function associated with the specified device identification information in the device function information 30. Thereby, the function of the image forming apparatus 10 to be used is identified (recognized).

  The control unit 52 of the terminal device 14 may specify, in the position association information, the device identification information associated with the position information of the image forming apparatus 10 to be used. The position association information may be stored in the storage unit 48 of the terminal device 14. In this case, the control unit 52 of the terminal device 14 specifies the device identification information of the image forming apparatus 10 to be used by referring to the position association information stored in the storage unit 48 of the terminal device 14. Alternatively, the control unit 52 of the terminal device 14 may specify the device identification information of the image forming apparatus 10 to be used by acquiring the position association information from the server 12 and referring to the position association information. .

  Hereinafter, the screen displayed on the terminal device 14 will be described in detail. First, referring to FIG. 6, a function purchase screen displayed when purchasing a function or when confirming a purchased function will be described. An example of the function purchase screen is shown in FIG.

  For example, when a user accesses the server 12 using the terminal device 14, user identification information (user account information) of the user is transmitted from the terminal device 14 to the server 12. In the server 12, the identifying unit 42 identifies function identification information of each function associated with the user identification information in the function purchase history information 32. Thereby, the function group purchased by the user, that is, the function group available by the user is specified (recognized). For example, function purchase screen information including function identification information indicating each function to be sold and function identification information indicating each function available to the user is transmitted from the server 12 to the terminal device 14 . The control unit 52 of the terminal device 14 causes the display unit of the UI unit 50 of the terminal device 14 to display a function purchase screen based on the function purchase screen information. The control unit 52 of the terminal device 14 displays, for example, each function identification information on the display unit of the UI unit 50, and causes the display unit to display information indicating the purchase status of each function.

  In the function purchase screens 56 and 58 shown in FIGS. 6 (a) and 6 (b), a list of information indicating the functions to be sold is displayed. Purchase status information indicating purchases is displayed in association with each other. The function associated with the purchase status information indicating the purchase is a function purchased by the user, that is, a function available to the user. The functions associated with purchase status information indicating unpurchased are functions that have not been purchased by the user, that is, functions that can not be used by the user (functions whose use is prohibited).

  In the example shown in FIG. 6A, the function purchase screen 56 is a screen showing a function purchase history for the user A. The function purchase history is displayed as a list on the function purchase screen 56 as an example. The functions A and C are functions purchased by the user A and available to the user A. The functions B, D, and E are not purchased by the user A, and can not be used by the user A. The function is purchased via the function purchase screen 56. For example, when the user A designates the unpurchased function B using the terminal device 14 and gives a purchase instruction, the terminal device 14 transmits function identification information indicating the function B and information indicating the purchase instruction from the terminal device 14 to the server 12 Be done. In the server 12, the purchase processing unit 38 executes the purchase processing of the function B. When the function B is charged, the purchase processing unit 38 executes charging processing. Further, the purchase history management unit 40 updates the function purchase history information for the user A. That is, the purchase history management unit 40 associates the function identification information indicating the function B with the user identification information of the user A in the function purchase history information. This enables the user A to use the function B. Further, on the function purchase screen 56, the purchase status of the function B is changed from “not purchased” to “purchase”. In addition, you may make it display the corresponding apparatus for every function. By doing so, the device corresponding to the function that the user wants to use can be easily identified. For example, the devices A capable of executing the functions A, B, and C are associated with the functions A, B, and C, and the information indicating the device α is displayed in association with the functions A, B, and C. ing. The devices D capable of executing the functions D and E are associated with the functions D and E, and information indicating the devices β is displayed in association with the functions D and E. Note that the description of devices that can be executed may be displayed by the name of a device group, or may be a display that enumerates each single device. Further, as in the function purchase screen 58 shown in FIG. 6B, the function may be associated with the device capable of executing the function, and the executable device may be displayed in another row. For example, models of devices capable of executing the function A are models a, b, c, d, and models of devices capable of executing the function B are model Z group. The model Z group includes models a, b, e and f.

  For example, a program of a web browser is stored in the terminal device 14, and access to the server 12 from the terminal device 14 is realized by using the web browser. When the user accesses the server 12 using a web browser, a web page representing the function purchase screen 56 or the function purchase screen 58 is displayed on the display unit of the UI unit 50 of the terminal device 14 and the function is performed via the web page. Will be purchased.

  Next, the function display screen will be described in detail with reference to FIG. The function display screen is displayed on the display unit of the UI unit 50 of the terminal device 14 when using the image forming apparatus 10 or the like. An example of the function display screen is shown in FIG.

  For example, by using any of the marker type AR technology, the markerless type AR technology, or the position information AR technology described above, device identification information of the image forming apparatus 10 to be used is acquired, and the device identification information is obtained. Function identification information indicating each function associated with each other, that is, function identification information indicating each function of the image forming apparatus 10 to be used is identified (recognized). Further, function identification information indicating each function associated with the user identification information of the target user using the image forming apparatus 10 to be used, that is, function identification information indicating each function available to the target user is specified. (Recognized). Those pieces of information are displayed on the display unit of the UI unit 50 of the terminal device 14 as a function display screen. Further, since the function group possessed by the image forming apparatus 10 to be used is specified, the function group not possessed by the image forming apparatus 10 to be used is specified among the function groups to be sold. Function identification information indicating each function that the image forming apparatus 10 to be used does not have may be displayed on the function display screen.

  On the function display screen 60 shown in FIG. 7, a button image 62 representing the function A, a button image 64 representing the function B, and a button image 66 representing the function C are displayed as an example of the function identification information. ing. The function A is a function of the image forming apparatus 10 to be used, and is a function available to the target user, that is, a function purchased by the target user. The function B is a function that the image forming apparatus 10 to be used has, and is a function that can not be used by the target user, that is, a function that has not been purchased by the target user. By the target user purchasing the function B, the target user can use the function B. The function C is a function that the image forming apparatus 10 to be used does not have, that is, a function to which the image forming apparatus 10 to be used does not correspond. The control unit 52 of the terminal device 14 changes the display mode of the button image according to whether or not the function represented by the button image is the function possessed by the image forming apparatus 10 to be used. Further, the control unit 52 changes the display mode of the button image in accordance with whether or not the function represented by the button image is a function available to the target user. For example, the control unit 52 changes the color or shape of the button image. In the example illustrated in FIG. 7, the control unit 52 displays the button images 62, 64, and 66 on the display unit so as to be distinguished from each other. The control unit 52 changes the color of each of the button images 62, 64, 66, for example. As one example, a button image (for example, a button image 62 representing function A) which is a function of the image forming apparatus 10 to be used and represents a function available to the target user is displayed in blue. A button image (button image 64 representing function B) representing a function that the image forming apparatus 10 to be used has and can not be used by the target user is displayed in yellow. A button image (for example, a button image 66 representing the function C) representing a function that the image forming apparatus 10 to be used does not have is displayed in gray. As another example, the control unit 52 may change the shapes of the button images 62, 64, 66, or may change the font of the function display name. Of course, the display mode may be changed by another method. Thereby, the user can be notified of availability of each function with high visibility.

  For example, when the target user designates the button image 62 representing the function A using the terminal device 14 and gives an instruction to execute the function A, execution instruction information indicating an instruction to execute the function A is an image from the terminal device 14 It is sent to the forming device 10. The execution instruction information includes control data for executing the function A, data such as image data to which the process by the function A is applied, and the like. When receiving the execution instruction information, the image forming apparatus 10 executes the function A in accordance with the execution instruction information. For example, when the function A is a scan transfer function, the scan function is executed by the image forming unit 18 of the image forming apparatus 10, whereby scan data (image data) is generated. The scan data is transmitted from the image forming apparatus 10 to the set destination (for example, the terminal device 14). Further, when the function A is a function realized by the cooperation of the image forming apparatus 10 and the server 12, a part of the functions of the function A is executed by the image forming apparatus 10, and the other functions of the function A are It is executed by the server 12. For example, scan data is generated by execution of the scan function by the image forming unit 18 of the image forming apparatus 10, the scan data is transmitted from the image forming apparatus 10 to the server 12, and the function execution unit 34 of the server 12 By executing the recognition function, character data is extracted from the scan data. The character data is transmitted from the server 12 to the set destination (for example, the terminal device 14).

  When the target user designates the button image 64 representing the function B using the terminal device 14 and instructs the purchase of the function B, the terminal device 14 accesses the server 12. As a result, a screen (for example, a Web site) for purchasing the function B is displayed on the UI unit 50 of the terminal device 14 as information for enabling the target user to use the function B. By performing the purchase procedure on this screen, the use of the function B by the target user is permitted, and when the target user instructs the execution of the function B, the function B is executed. As another example, the UI unit 50 has a use permission request screen (for example, a Web site) for requesting the administrator or the like to use the function B as information for enabling the target user to use the function B. It may be displayed. When the administrator or the like is requested to permit the use of the function B via the use permission request screen, and the permission is obtained, the use of the function B by the target user becomes possible.

  Another display mode may be adopted as the display mode of the function display screen. For example, the housing of the image forming apparatus 10 is provided with an installation place where the terminal device 14 is installed, and the display mode of the function screen (according to the installation mode of the terminal apparatus 14 installed in the installation place Display design may be changed. For example, the housing of the image forming apparatus 10 has a shape corresponding to the shape of the terminal device 14 and is formed with a recess used as an installation place of the terminal device 14. The shape of the depression is vertical or horizontal. By installing the terminal device 14 in the vertically long recess, the terminal device 14 is disposed vertically to the casing of the image forming apparatus 10, and by installing the terminal device 14 in the horizontally long recess, the terminal device 14 is installed. Are disposed laterally to the housing of the image forming apparatus 10. The display mode of the function display screen is changed according to the arrangement state.

  FIG. 8 shows the function display screen 68 when the terminal device 14 is arranged vertically with respect to the casing of the image forming apparatus 10. In FIG. A function display screen 72 is shown when arranged sideways with respect to the housing.

  In the case of the vertical arrangement, as shown in FIG. 8, the control unit 52 of the terminal device 14 arranges the button images 62, 64 and 66 vertically and displays them on the display unit of the UI unit 50. That is, the control unit 52 arranges the button images 62, 64, 66 along the longitudinal direction of the terminal device 14 arranged vertically, and causes the display unit of the UI unit 50 to display the button images 62, 64, 66. In addition, the control unit 52 may display a band-shaped image 70 provided along the longitudinal direction of the terminal device 14 on both sides in the longitudinal direction of the function display screen 68.

  In the case of the horizontal arrangement, as shown in FIG. 9, the control unit 52 of the terminal device 14 arranges the button images 62, 64, 66 horizontally and causes the display unit of the UI unit 50 to display the button images 62, 64, 66. That is, the control unit 52 arranges the button images 62, 64, 66 along the longitudinal direction of the terminal device 14 arranged horizontally, and causes the display unit of the UI unit 50 to display the button images. In addition, the control unit 52 may display a band-shaped image 74 provided along the longitudinal direction of the terminal device 14 on both sides in the longitudinal direction of the function display screen 72. Image 74 has a different color or design than image 70.

  As described above, by changing the display mode (display design) of the function display screen according to the installation mode of the terminal device 14, the information displayed on the function display screen can be viewed more easily than when the display mode is fixed. It can be.

  Hereinafter, processing by the image forming system according to the first embodiment will be described in detail. First, function purchase processing will be described with reference to FIG. FIG. 10 is a sequence diagram showing the function purchase process.

  First, a target user who desires to purchase a function uses the terminal device 14 to instruct activation of an application (program) for function purchase processing. The control unit 52 of the terminal device 14 activates the application according to the instruction (S01). The application may be stored in advance in the storage unit 48 of the terminal device 14 or may be downloaded from the server 12 or the like.

  Next, the control unit 52 of the terminal device 14 reads user account information (user identification information) of the target user (S02). The user account information is stored in advance in, for example, the storage unit 48 of the terminal device 14. The control unit 52 of the terminal device 14 functions as an example of a user identification unit, and the user account information of the target user is stored in the storage unit 48. Read and identify the target user. When user account information of a plurality of users is stored in the storage unit 48, the target user specifies his own user account information using the terminal device 14. Thereby, the user account information of the target user is read, and the target user is identified. Alternatively, the control unit 52 may identify the target user by reading the user account information of the user logged in to the terminal device 14. As another example, when only one piece of user account information is stored in the same terminal device 14, the control unit 52 may identify the target user by reading the user account information. When the user account is not set and the user account information is not created, the initial setting is performed, whereby the user account information is created.

  Next, the terminal device 14 accesses the server 12 via the communication path N (S03). At this time, the terminal device 14 transmits the user account information (user identification information) of the target user to the server 12.

  In the server 12, the specification unit 42 reads out the function purchase history of the target user having the user account information (S04). Specifically, the identification unit 42 identifies function identification information of each function associated with the user account information (user identification information) in the function purchase history information 32 stored in the storage unit 28 of the server 12. Do. As a result, the function group purchased by the target user, that is, the function group available by the user is identified.

  Next, the server 12 includes function identification information indicating each function to be sold and function identification information indicating each function available to the target user (function identification information indicating each function purchased by the target user) , And is transmitted to the terminal device 14 via the communication path N (S05).

  In the terminal device 14, the control unit 52 causes the display unit of the UI unit 50 of the terminal device 14 to display a function purchase screen based on the function purchase screen information transmitted from the server 12 (S06). For example, the function purchase screen 56 shown in FIG. 6 (a) or the function purchase screen 58 shown in FIG. 6 (b) is displayed. The function purchase screens 56 and 58 may display information indicating the setting contents of the purchased function.

  The target user uses the terminal device 14 to select a purchase target function on the function purchase screen 56 (S07). Alternatively, the target user may change the setting contents of the purchased function on the function purchase screen 56. For example, the target user uses the terminal device 14 to select the function to be changed, and changes the setting content of the function.

  When the function to be purchased is selected by the target user, the control unit 52 of the terminal device 14 displays a confirmation screen on the display unit of the UI unit 50 (S08). When the target user gives a purchase instruction on the confirmation screen, the terminal device 14 transmits purchase instruction information indicating the purchase instruction to the server 12 via the communication path N (S09). The purchase instruction information includes function identification information indicating a function to be purchased. Note that the display of the confirmation screen may be omitted. In this case, when the purchase target function is selected in step S 07 and a purchase instruction is given, purchase instruction information is transmitted from the terminal device 14 to the server 12. When the setting content of the function is changed by the target user, the terminal device 14 transmits information indicating the setting content after the change to the server 12 via the communication path N.

  The server 12 executes a purchase process (S10). When the function to be purchased is chargeable, the purchase processing unit 38 executes an accounting process. The purchase history management unit 40 also updates the function purchase history information 32 for the target user. That is, in the function purchase history information 32, the purchase history management unit 40 associates the user identification information (user account information) of the target user with the function identification information indicating the function to be purchased. Thereby, the use of the function to be purchased is permitted. When the setting content of the function is changed by the target user, the purchase history management unit 40 changes the setting content of the function.

  When the purchase process is completed, the server 12 transmits purchase completion information indicating that the purchase process is completed to the terminal device 14 via the communication path N (S11). As a result, information indicating that the purchase procedure has been completed is displayed on the display unit of the UI unit 50 of the terminal device 14 (S12). Next, the display unit of the UI unit 50 of the terminal device 14 displays function identification information indicating a function that has become available by the current purchase (S13). Alternatively, the function purchase screen is displayed on the display unit of the UI unit 50, and the display mode of the function made available by the current purchase can be used on the function purchase screen from the display mode indicating that the function is not available. It changes to the display mode which shows what. For example, the color or shape of the button image indicating the function is changed. In addition, when the setting content of the function is changed, the server 12 transmits procedure completion information indicating that the change process is completed to the terminal device 14 via the communication path N. As a result, information indicating that the change procedure has been completed is displayed on the display unit of the UI unit 50 of the terminal device 14.

  Hereinafter, display processing of the function display screen will be described with reference to FIG. A flowchart showing the process is shown in FIG. As an example, the case where the image forming apparatus 10 is recognized using the marker type AR technology will be described.

  First, a target user who desires to display a function display screen uses the terminal device 14 to issue an instruction to start an application (program) for display processing of the function display screen. The control unit 52 of the terminal device 14 activates the application according to the instruction (S20). The application may be stored in advance in the storage unit 48 of the terminal device 14 or may be downloaded from the server 12 or the like.

  Next, the control unit 52 of the terminal device 14 reads user account information (user identification information) of the target user (S21). This reading process is the same as the process of step S02 described above.

  Next, the target user instructs activation of the camera 46 using the terminal device 14. The control unit 52 of the terminal device 14 activates the camera 46 according to the instruction (S22). The target user shoots the marker 54 provided in the image forming apparatus 10 to be used by the camera 46 (S23). Thereby, image data representing the marker 54 is generated.

  Next, the function group of the image forming apparatus 10 to be used is specified (S24). For example, image data representing the marker 54 is transmitted from the terminal device 14 to the server 12, and the server 12 applies decoding processing to the image data. Thereby, device identification information indicating the image forming apparatus 10 to be used is extracted. Note that after the device identification information is extracted by the terminal device 14, the usable function group is displayed by the UI unit 50 without additionally receiving an operation input specifying the device (image forming apparatus 10) to be used by the user. be able to. As a result, the operation steps for registering the device to be used by the user's operation input can be simplified, and the setting time can be shortened. As another example, decoding processing may be applied to the image data in the terminal device 14 to extract device identification information. In this case, the device identification information extracted by the terminal device 14 is transmitted from the terminal device 14 to the server 12. In the server 12, the identifying unit 42 identifies function identification information of each function associated with the device identification information in the device function information 30. Thereby, the function group of the image forming apparatus 10 to be used is specified (recognized).

  Also, the function group that can be used is specified by the target user (S25). For example, user account information (user identification information) of the target user is transmitted from the terminal device 14 to the server 12. In the server 12, the identification unit 42 identifies function identification information of each function associated with the user account information in the function purchase history information 32. Thereby, the function group purchased by the target user, that is, the function group usable by the target user is identified (recognized).

  The processes of step S24 and step S25 may be performed simultaneously, or the process of step S25 may be performed before step S24.

  In the server 12, the control unit 36 generates function display screen information indicating a function display screen for displaying the function group of the image forming apparatus 10 to be used and the function group usable by the target user. The function display screen information is transmitted from the server 12 to the terminal device 14. As a result, the function display screen is displayed on the display unit of the UI unit 50 of the terminal device 14 (S26). On the function display screen, function identification information of each function of the image forming apparatus 10 to be used and function identification information of each function available to the target user are displayed. In addition, function identification information indicating each function that is to be sold and that each image forming apparatus 10 to be used does not have may be displayed on the function display screen. For example, the function display screen 60 shown in FIG. 7 is displayed on the display unit of the UI unit 50.

  In the function display screen 60, when the unpurchased function is selected by the target user and a purchase instruction is given (S27, Yes), purchase processing for the selected function is executed (S28). This makes it possible to use the purchased function. If the purchase instruction is not given (S27, No), the process proceeds to step S29.

  When the target user selects a function (purchased function) that is a function of the image forming apparatus 10 to be used and is available by the target user (S29, Yes), the selected function Is executed (S30). When the selected function is executed by the image forming apparatus 10, execution instruction information indicating an execution instruction of the function is transmitted from the terminal device 14 to the image forming apparatus 10, and the function is executed by the image forming apparatus 10. . Further, when the selected function is realized by the cooperation of the image forming apparatus 10 and the server 12, a part of the selected function is executed by the image forming apparatus 10, and another of the selected functions is performed. The part is executed by the server 12. At this time, in order to execute the selected function, control data, data to be processed, etc. are transmitted and received among the image forming apparatus 10, the server 12, and the terminal device 14.

  When the execution instruction of the function is not given by the target user (S29, No), the process returns to step S27.

  Hereinafter, another display process of the function display screen will be described with reference to FIG. A flowchart showing the process is shown in FIG. As an example, the case where the image forming apparatus 10 is recognized using the markerless AR technology will be described.

  First, in the terminal device 14, an application for display processing of the function display screen is activated (S40), and user account information (user identification information) of a target user who desires display of the function display screen is read (S41). 46 is activated (S42).

  Next, the target user shoots all or part of the appearance of the image forming apparatus 10 to be used by the camera 46 (S43). Thereby, appearance image data representing all or part of the appearance of the image forming apparatus 10 to be used is generated.

  Next, the image forming apparatus 10 to be used is specified (S44). For example, the appearance image data is transmitted from the terminal device 14 to the server 12, and the appearance image data of each image forming apparatus 10 included in the appearance image correspondence information in the server 12 and the appearance image data transmitted from the terminal device 14 The device identification information of the image forming apparatus 10 to be used is identified by comparing.

  When the plurality of image forming apparatuses 10 are not identified as a result of the above comparison processing and one image forming apparatus 10 is identified (S45, No), the process proceeds to step S24 shown in FIG. .

  On the other hand, when a plurality of image forming apparatuses 10 are specified as a result of the above comparison processing (S 45, Yes), the target user selects an image forming apparatus 10 to be used from the plurality of image forming apparatuses 10. (S46). For example, the device identification information of each of the identified image forming apparatuses 10 is transmitted from the server 12 to the terminal device 14 and displayed on the UI unit 50 of the terminal device 14. The target user uses the terminal device 14 to select the device identification information of the image forming apparatus 10 to be used from among the plurality of pieces of device identification information. The device identification information selected by the target user is transmitted from the terminal device 14 to the server 12. Next, the process proceeds to step S24 shown in FIG.

  The processes after step S24 are the same as the contents described with reference to FIG.

  Hereinafter, another display process of the function display screen will be described with reference to FIG. A flowchart showing the process is shown in FIG. As an example, the case where the image forming apparatus 10 is recognized using position information AR technology will be described.

  First, in the terminal device 14, an application for display processing of a function display screen is activated (S50), and user account information (user identification information) of a target user who desires display of the function display screen is read (S51).

  Next, the terminal device 14 acquires position information of the image forming apparatus 10 to be used (S52). For example, each image forming apparatus 10 has a GPS function, and acquires position information indicating the position of the image forming apparatus 10 itself. The terminal device 14 transmits information indicating an acquisition request for position information to the image forming apparatus 10 to be used, and receives position information of the image forming apparatus 10 from the image forming apparatus 10 as a response.

  Next, the image forming apparatus 10 to be used is specified (S53). For example, the position information of the image forming apparatus 10 to be used is transmitted from the terminal device 14 to the server 12, and the server 12 transmits the position information of each image forming apparatus 10 included in the position association information and the terminal device 14 The device identification information of the image forming apparatus 10 to be used is identified by comparing the received position information.

  If the plurality of image forming apparatuses 10 are not identified as a result of the above comparison processing and one image forming apparatus 10 is identified (S54, No), the process proceeds to step S24 shown in FIG. .

  On the other hand, when a plurality of image forming apparatuses 10 are specified as a result of the above comparison processing (S54, Yes), the target user selects an image forming apparatus 10 to be used from the plurality of image forming apparatuses 10. (S55). The device identification information of the image forming apparatus 10 selected by the target user is transmitted from the terminal device 14 to the server 12. Next, the process proceeds to step S24 shown in FIG.

  The processes after step S24 are the same as the contents described with reference to FIG.

  As described above, according to the first embodiment, the image forming apparatus 10 to be used is identified by applying the AR technology, and the function identification information indicating the function group of the image forming apparatus 10 is used by the target user Function identification information indicating possible function groups is displayed on the terminal device 14. As a result, even if it is not clear from the appearance of what function the image forming apparatus 10 to be used has, the function of the image forming apparatus 10 to be used by the target user is It becomes easy to understand whether it has, and it becomes easy to understand whether the image forming apparatus 10 to be used has a function that can be used by the target user.

  Further, according to the first embodiment, for example, in an environment where a plurality of users (for example, a plurality of image forming apparatuses 10) use a plurality of devices (for example, a plurality of users) Is displayed properly. For example, even if a user interface such as a touch panel is removed from an apparatus such as the image forming apparatus 10, the terminal device 14 is used as the user interface, and information on functions corresponding to each user is the terminal of each user. It is properly displayed on the device 14. As another scene, for example, when the device is temporarily used at the place where the user goes out, a user interface suitable for the user, that is, a user interface representing a function available to the user is displayed on the terminal device 14. Is realized.

  In the example shown in FIGS. 11, 12 and 13, after the user account information is read and the user is identified, the device to be used (image forming apparatus 10) is identified, but the device to be used (image formation) After the device 10) is identified, user account information may be read to identify the user. When the marker type AR technology or the markerless AR technology is applied, the device can not be identified until the user moves to the device (image forming apparatus 10) and captures the device with the camera. In such a case, the user can be identified first, and the device to be used can be identified later, whereby the processing can be efficiently performed.

  Hereinafter, modifications of the first embodiment will be described.

  When the function to be executed is selected in advance by the target user, the control unit 52 of the terminal device 14 causes the display unit of the UI unit 50 to display the device identification information of the image forming apparatus 10 having the function to be executed. Good. For example, the control unit 52 of the terminal device 14 acquires function purchase history information 32 for the target user from the server 12 according to an instruction from the target user, and indicates function identification information indicating a function purchased by the target user, That is, function identification information indicating each function available to the target user is displayed on the display unit of the UI unit 50. For example, a button image representing each function available to the target user is displayed on the display unit as function identification information. Next, the target user selects a function to be executed from the function group available to the target user. For example, the target user selects function identification information (button image) indicating a function to be executed from the function identification information group (for example, button image group) displayed on the display unit. As a result, the function identification information selected by the target user is transmitted from the terminal device 14 to the server 12. In the server 12, the identifying unit 42 identifies, in the device function information 30, device identification information associated with the function identification information selected by the target user. Thereby, the image forming apparatus 10 having the function selected by the target user is identified. At this time, one or more image forming apparatuses 10 are identified. The device identification information specified by the specifying unit 42 is transmitted from the server 12 to the terminal device 14 and displayed on the display unit of the UI unit 50 of the terminal device 14. This makes it easy for the target user to understand which image forming apparatus 10 has the function to be executed.

  Also, position information of the image forming apparatus 10 having a function to be executed may be transmitted from the server 12 to the terminal device 14 and displayed on the display unit of the UI unit 50 of the terminal device 14. For example, the control unit 52 of the terminal device 14 causes the display unit of the UI unit 50 to display a map, and superimposes information (for example, an image such as a mark) indicating the image forming apparatus 10 having the function to be executed on the map. It may be displayed on the display unit. This makes it easy for the target user to understand where the image forming apparatus 10 having the function to be executed is installed.

  As another modification, when the function to be executed is selected in advance by the target user, when the image forming apparatus 10 to be used has the function to be executed, the control unit 52 of the terminal device 14 The image forming apparatus 10 to be used may execute the function to be executed. In this case, the control unit 52 functions as an example of the execution control unit. For example, as described in the above example, the control unit 52 of the terminal device 14 causes the display unit of the UI unit 50 to display function identification information (for example, a button image) indicating each function available to the target user. Next, the target user selects function identification information (button image) indicating a function to be executed from the function identification information group (button image group) displayed on the display unit. On the other hand, the application of the AR technology identifies the image forming apparatus 10 to be used, and function identification information indicating each function of the image forming apparatus 10 to be used is transmitted from the server 12 to the terminal apparatus 14. When the function identification information indicating the function to be executed is included in the function identification information indicating each function of the image forming apparatus 10 to be used, that is, the function to be executed of the image forming apparatus 10 to be used is The control unit 52 of the terminal device 14 transmits information indicating an execution instruction of the function to be executed to the image forming apparatus 10 to be used. At this time, control data or the like for executing the function to be executed is transmitted from the terminal device 14 to the image forming apparatus 10. The image forming apparatus 10 having received the information indicating the execution instruction executes the function to be executed. As a result, it is possible to save time and effort of selection by the target user as compared with the case of selecting the function that is the function that is available to the target user and is the execution target from the function group of the image forming apparatus 10 to be used.

  As yet another modification, the display unit of the UI unit 50 of the terminal device 14 may expand and display information related to the UI unit 22 of the image forming apparatus 10. For example, the control unit 24 of the terminal device 14 changes the information displayed on the UI unit 50 in accordance with an operation on the UI unit 22 of the image forming apparatus 10. For example, by cooperation of the hardware user interface unit (hardware UI unit) of the image forming apparatus 10 to be used and the software user interface unit (software UI unit) realized by the UI unit 50 of the terminal device 14 The user interface unit for the image forming apparatus 10 to be used is realized. As described above, the hardware UI unit of the image forming apparatus 10 is, for example, a ten key, a direction instruction key, or the like. Further, function identification information indicating each function of the image forming apparatus 10 to be used and function identification information indicating each function permitted to be used by the target user are displayed on the UI unit 50 of the terminal device 14. , The software UI unit is realized. For example, when the terminal device 14 transmits information indicating a connection request to the image forming apparatus 10, communication is established between the terminal device 14 and the image forming apparatus 10. In this state, information indicating an instruction given using the software UI unit of the terminal device 14 is transmitted from the terminal device 14 to the image forming apparatus 10 to be used, and the hardware UI of the image forming apparatus 10 to be used is The information indicating the instruction given using the copy unit is transmitted from the image forming apparatus 10 to be used to the terminal device 14. For example, when the target user operates the numeric keypad or the direction instruction key constituting the hardware UI unit, information indicating the content of the operation is transmitted from the image forming apparatus 10 to be used to the terminal device 14. The control unit 52 of the terminal device 14 realizes the operation content on the software UI unit by functioning as an example of the operation control unit. As a result, the software UI unit is operated using the hardware UI unit. For example, when the target user operates the hardware UI unit to select function identification information (for example, a button image) displayed on the software UI unit and gives an execution instruction, the information indicating the execution instruction of the function is the terminal The image is transmitted from the apparatus 14 to the image forming apparatus 10 to be used, and the function is executed. As described above, one hardware UI unit provided in the image forming apparatus 10 and a software UI unit displayed on the terminal device 14 cooperate to realize the UI unit of the image forming apparatus 10. The operability of the UI unit can be improved as compared with the case where only the user interface of the device, for example, only the user interface of the image forming apparatus 10 or the terminal device 14 is used. Also, a fax number or the like may be input using the hardware UI unit, or a preview screen of image data may be displayed on the software UI unit.

  As still another modification, even if individual setting information of each user is not stored in the image forming apparatus 10 and is stored in an external device other than the image forming apparatus 10 (for example, the terminal device 14 or the server 12). Good. The individual setting information is, for example, the user's name, address, telephone number, fax number, e-mail address, address of the terminal device 14, fax destination managed by the user, e-mail address book, etc. As an example, it is assumed that setting information is stored in the terminal device 14. When the function is executed using setting information in the image forming apparatus 10 to be used, the setting information is transmitted from the terminal device 14 that instructed the execution of the function to the image forming apparatus 10 to be used. For example, when facsimile transmission is performed in the image forming apparatus 10 to be used, the information indicating the fax number used in the facsimile transmission is transmitted from the terminal device 14 that instructed the facsimile transmission to the image forming apparatus 10 to be used Will be sent. The image forming apparatus 10 to be used facsimile-transmits using the facsimile number transmitted from the terminal device 14. As another example, when executing the scan transfer function, the terminal device 14 transmits address information indicating a transmission destination of the image data to the image forming apparatus 10 to be used. The image forming apparatus 10 generates image data by executing a scan function, and transmits the image data to the transmission destination indicated by the address information. As described above, by not storing the setting information in the image forming apparatus 10, the leakage of the setting information from the image forming apparatus 10 is prevented or suppressed. As a result, compared with the case where the setting information is stored in the image forming apparatus 10, the security of the setting information in the image forming apparatus 10 is improved. Although the setting information is stored in the terminal device 14 in the above example, the setting information may be stored in the server 12. In this case, the terminal device 14 may access the server 12 to acquire setting information, or the image forming apparatus 10 may access the server 12 to acquire setting information.

Second Embodiment
An image forming system as an information processing system according to the second embodiment of the present invention will be described below with reference to FIG. FIG. 14 shows an example of the image forming system according to the second embodiment. The image forming system according to the second embodiment includes a plurality of devices (for example, devices 76 and 78), a server 80, and a terminal device 14. The devices 76 and 78, the server 80 and the terminal device 14 are connected to one another via a communication path N such as a network. Although two devices (devices 76 and 78) are included in the image forming system in the example illustrated in FIG. 14, three or more devices may be included in the image forming system. Also, a plurality of servers 80 and a plurality of terminal devices 14 may be included in the image forming system.

  The devices 76 and 78 are devices having a specific function, and, for example, devices such as the image forming device 10 according to the first embodiment, a personal computer (PC), a display device such as a projector, a telephone, a watch, a surveillance camera, etc. It is. The devices 76 and 78 have a function of transmitting and receiving data to and from other devices.

  The server 80 is a device that manages a cooperation function executed by causing a plurality of devices to cooperate. The server 80 also has a function of transmitting and receiving data to and from other devices.

  The terminal device 14 has the same configuration as the terminal device 14 according to the first embodiment, and for example, functions as a user interface unit (UI unit) of the device when using the device.

  In the image forming system according to the second embodiment, a plurality of devices are specified as devices to be cooperated, and the cooperation function is specified by the cooperation of the plurality of devices.

  Hereinafter, the configuration of the server 80 will be described in detail with reference to FIG. The configuration of the server 80 is shown in FIG.

  The communication unit 82 is a communication interface, and has a function of transmitting data to another device via the communication path N, and a function of receiving data from the other device. The communication unit 82 may be a communication interface having a wireless communication function, or may be a communication interface having a wired communication function.

  The storage unit 84 is a storage device such as a hard disk or an SSD. The storage unit 84 stores cooperative function information 86, various data, various programs, and the like. Of course, they may be stored in separate storage devices, or may be stored in one storage device. Further, the cooperation function information 86 stored in the storage unit 84 may be provided to the terminal device 14 periodically. By doing so, the information stored in the storage unit 48 of the terminal device 14 may be updated. Here, the cooperation function information 86 will be described.

  The cooperation function information 86 is information indicating a cooperation function executed by causing a plurality of devices to cooperate, and, for example, for each cooperation function, for identifying each equipment to be cooperated to execute the cooperation function. It is information indicating correspondence between a combination of device identification information and cooperation function identification information for identifying the cooperation function. The device identification information is, for example, a device ID, a device name, information indicating the type of device, a model number, position information, and the like, similarly to the device identification information according to the first embodiment. The cooperation function identification information is, for example, a cooperation function ID or a cooperation function name. The cooperation function may be a function executed by causing a plurality of devices having different functions to cooperate with each other, or a function performed by causing a plurality of devices having the same function to cooperate with each other. . For example, the cooperation function is a function that could not be used before cooperation. The function that can not be used before the cooperation may be a function that can be used by using the same function among the functions of the devices to be cooperated, or the functions of the devices to be cooperated with each other It may be a function available by combining different functions. For example, the copy function is executed by linking an apparatus (printer) having a print function and an apparatus (scanner) having a scan function. That is, the copy function is executed by linking the print function and the scan function. In this case, the copy function is associated with the combination of the print function and the scan function. In the cooperation function information 86, cooperation function identification information for identifying a copy function as a cooperation function, device identification information for identifying an apparatus having a print function, and device identification for identifying an apparatus having a scan function A combination of information is associated. By referring to the cooperation function information 86, a plurality of devices for executing the cooperation function are identified.

  The control unit 88 controls the operation of each unit of the server 80. The control unit 88 also includes a specifying unit 90.

  The identification unit 90 receives device identification information for identifying each device to be cooperated, and cooperates with the cooperation function associated with the combination of each device identification information in the cooperation function information 86 stored in the storage unit 84. Identify feature identification information. As a result, the cooperation function to be executed by linking the respective devices to be cooperated is specified (recognized). For example, a plurality of pieces of device identification information are transmitted from the terminal device 14 to the server 80, and the identification unit 90 identifies cooperation function identification information of the cooperation function associated with the plurality of device identification information. For example, cooperation function identification information (for example, information indicating the name of the cooperation function) of the cooperation function is transmitted from the server 80 to the terminal device 14 and displayed on the terminal device 14. As a result, cooperation function identification information of the cooperation function executed by the plurality of devices specified by the plurality of device identification information is displayed on the terminal device 14.

  The above-mentioned cooperation function information 86 may be stored in the storage unit 48 of the terminal device 14. In this case, the cooperation function information 86 may not be stored in the storage unit 84 of the server 80. In addition, the control unit 52 of the terminal device 14 may include the above-described specifying unit 90, and may specify the cooperation function based on a plurality of pieces of device identification information. In this case, the server 80 may not include the specifying unit 90.

  In the second embodiment, as an example, by applying the AR technology, the device identification information of the device to be cooperated is acquired, and the device to be cooperated is specified (recognized). As the AR technology, a marker type AR technology, a markerless type AR technology, a position information AR technology, etc. are used as in the first embodiment.

  When the marker type AR technology is used, the camera 46 of the terminal device 14 captures a marker such as a two-dimensional bar code (for example, the marker 54 provided in the image forming apparatus 10) provided in the device to be cooperated. And image data representing the marker (for example, image data representing the marker 54) is generated. The image data is transmitted from the terminal device 14 to the server 80, for example. In the server 80, the control unit 88 extracts device identification information by applying decoding processing to the marker image represented in the image data. From this, the device identification information of the device to be cooperated is acquired. By capturing the marker of each device to be cooperated, the device identification information of each device is acquired, whereby the cooperation function is specified. The control unit 52 of the terminal device 14 may extract the device identification information by applying the decoding process.

  When the markerless AR technology is used, all or a part of the appearance of the device to be cooperated is photographed by the camera 46 of the terminal device 14. Of course, it is useful to capture and obtain information for identifying a device, such as the name of a device to be linked (for example, a product name) and a model number from the appearance. By photographing, appearance image data representing all or part of the appearance of the device to be cooperated is generated. The appearance image data is transmitted from the terminal device 14 to the server 80, for example. In the server 80, as in the first embodiment, the control unit 88 compares the appearance image data transmitted from the terminal device 14 with each appearance image data included in the appearance image correspondence information, and the comparison result is The device identification information of the device to be cooperated is specified based on that. Thus, the device to be cooperated is identified. As another example, if the device name (for example, a product name) or a model number is captured and appearance image data representing the name or model number is generated, the name or model number represented in the appearance image data is used. The device to be linked may be identified. By photographing the appearance of each device to be cooperated, the device identification information of each device is acquired, whereby the cooperation function is specified. The control unit 52 of the terminal device 14 may specify the device identification information of the device to be cooperated by applying the markerless AR technology.

  When the position information AR technology is used, for example, by using the GPS function, device position information indicating the position where the device to be cooperated is installed is acquired. The terminal device 14 acquires device position information of a device to be cooperated, as in the first embodiment. The device position information is transmitted from the terminal device 14 to the server 80, for example. In the server 80, the control unit 88 specifies the device identification information of the cooperation target device by referring to the position association information, as in the first embodiment. Thus, the device to be cooperated is identified. By acquiring the device position information of each device to be cooperated, the device identification information of each device is acquired, whereby the cooperation function is specified. The control unit 52 of the terminal device 14 may specify the device identification information of the device to be cooperated by applying the position information AR technology.

  Hereinafter, a method of linking a plurality of devices by applying the AR technology will be described.

  A method of linking a plurality of devices by applying marker type AR technology or markerless type AR technology will be described with reference to FIG. FIG. 16 shows an example of the device to be cooperated. As an example, the image forming apparatus 10 according to the first embodiment is used as the device 76 to be cooperated, and the PC 92 is used as the device 78 to be cooperated. For example, a marker 54 such as a two-dimensional bar code is provided on the housing of the image forming apparatus 10, and a marker 94 such as a two-dimensional bar code is provided on the housing of the PC 92. The marker 94 is information in which the device identification information of the PC 92 is encoded. When the device identification information of the image forming apparatus 10 and the PC 92 is acquired by the marker type AR technology or the markerless type AR technology, the user captures an image of the cooperation target image forming apparatus 10 and the PC 92 by the camera 46 of the terminal device 14. In the example shown in FIG. 16, the image forming apparatus 10 and the PC 92 are photographed together in a state in which both the image forming apparatus 10 and the PC 92 are included in the photographing area of the camera 46. Thus, image data representing the markers 54 and 94 is generated and transmitted from the terminal device 14 to the server 80. In the server 80, the control unit 88 applies the decoding process to the image data to extract the device identification information of the image forming apparatus 10 and the device identification information of the PC 92. Alternatively, appearance image data representing the appearance of both the image forming apparatus 10 and the PC 92 may be generated and transmitted from the terminal device 14 to the server 80. In this case, in the server 80, the control unit 88 specifies the device identification information of the image forming apparatus 10 and the device identification information of the PC 92 by referring to the appearance image correspondence information. When the device identification information is identified, the identification unit 90 identifies, in the cooperation function information 86, cooperation function identification information associated with the combination of the device identification information of the image forming apparatus 10 and the device identification information of the PC 92. Thus, the cooperation function to be executed is specified by causing the image forming apparatus 10 and the PC 92 to cooperate. Cooperation function identification information indicating the cooperation function is transmitted from the server 80 to the terminal device 14 and displayed on the UI unit 50 of the terminal device 14. When the user instructs the execution of the cooperation function using the terminal device 14, the cooperation function is executed. The terminal identification process of the device identification information and the identification process of the cooperation function may be performed by the terminal device 14.

  The device to be cooperated may be designated by the user's operation. For example, by photographing the image forming apparatus 10 and the PC 92 with the camera 46, as shown in FIG. 16, a device image 98 representing the image forming apparatus 10 and a device representing the PC 92 are displayed on the screen 96 of the display unit of the terminal device 14. An image 100 is displayed. The image data associated with the identified device displayed on the terminal device 14 when designating the device to be cooperated may be an image of the device photographed by the camera 46 (an image having a size at the time of photographing) (May be an enlarged or reduced image), or a previously prepared appearance image data (not an image obtained by photographing itself but a schematic image) tied to the identified device It may be For example, when using image data obtained by photographing the device, the appearance of the current device itself (for example, the appearance in which a scratch, a note, a sticker attached to the device, or the like is reflected) is reflected in the image As a result, it is possible for the user to visually recognize the difference from other similar devices. The user designates the device images 98 and 100 on the screen 96 to designate the image forming apparatus 10 and the PC 92 as devices to be cooperated. For example, when the user designates the device image 98, the marker type AR technology or the markerless type AR technology is applied to the device image 98, whereby the device identification information of the image forming apparatus 10 is specified. Similarly, when the user designates the device image 100, the marker type AR technology or the markerless type AR technology is applied to the device image 100, whereby the device identification information of the PC 92 is specified. As a result, the cooperation function to be executed by the image forming apparatus 10 and the PC 92 is specified, and cooperation function identification information indicating the cooperation function is displayed on the UI unit 50 of the terminal device 14.

  Also, as indicated by the arrow in FIG. 16, the user touches the device image 98 on the screen 96 and operates the section up to 100 with a finger or the like (for example, tracing with a finger), which is the tip of the operation. The device images 98 and 100 may be designated, and the image forming apparatus 10 and the PC 92 may be designated as devices to be cooperated. The order and the direction in which the device images 98 and 100 are touched may be opposite to the above example. Of course, a screen touch instruction means other than a finger such as a pen for tracing the screen 96 may be used. In addition to simply tracing, a device to be cooperated may be designated by a drawing operation that adds a circle, or the device is identified by touching the device image of the device to be cooperated within a preset time. You may do so. When canceling the cooperation, the user may designate the device to be canceled on the screen 96 or may press the cooperation release button. When a device that is not a collaboration target is included in the image, the user may exclude the device from the group of devices to be linked by designating the device on the screen 96. The operation for specifying the release device may function as a predetermined operation such as cross marking.

  For example, when the image forming apparatus 10 has a scan function, the scan transfer function is executed as a cooperation function by causing the image forming apparatus 10 and the PC 92 to cooperate. When the scan transfer function is executed, scan data (image data) is generated by the scan function of the image forming apparatus 10, and the scan data is transmitted from the image forming apparatus 10 to the PC 92. As another example, when the image forming apparatus 10 has a print function, document data to be printed is transmitted from the PC 92 to the image forming apparatus 10, and the print function of the image forming apparatus 10 places a document based on the document data on a sheet. It may be printed on

  FIG. 17 shows another example of the device to be cooperated. As an example, the printer 102 is used as the device 76 to be cooperated, and the scanner 104 is used as the device 78 to be cooperated. The printer 102 is an apparatus having only a printing function as an image forming function, and the scanner 104 is an apparatus having only a scanning function as an image forming function. For example, a marker 106 such as a two-dimensional bar code is provided on the housing of the printer 102, and a marker 108 such as a two-dimensional bar code is provided on the housing of the scanner 104. The marker 106 is information obtained by decoding the device identification information of the printer 102, and the marker 108 is information obtained by decoding the device identification information of the scanner 104. Similar to the example shown in FIG. 16, the printer 102 and the scanner 104 are photographed together in a state in which both the printer 102 and the scanner 104 are included in the photographing area of the camera 46. The device identification information of the printer 102 and the device identification information of the scanner 104 are specified by applying the marker type AR technology or the markerless type AR to the image data generated by this photographing, and the printer 102 and the scanner 104 are made to cooperate. The collaboration function to be executed is identified. The identification processing of the device identification information and the identification processing of the cooperation function may be performed by the server 80 or may be performed by the terminal device 14.

  Similar to the example shown in FIG. 16, a device image 110 representing the printer 102 and a device image 112 representing the scanner 104 are displayed on the screen 96 of the display unit of the terminal device 14. , And 112, the printer 102 and the scanner 104 may be designated as the device to be cooperated. Thereby, the linked function identification information indicating the copy function as the linked function is displayed on the UI unit 50 of the terminal device 14.

  By linking the printer 102 and the scanner 104, the copy function is executed. In this case, the document is read by the scan function of the scanner 104, and scan data (image data) representing the document is generated. The scan data is transmitted from the scanner 104 to the printer 102, and the print function of the printer 102 prints an image based on the scan data on a sheet. As described above, even when the device to be used does not have the copy function, the copy function as the cooperation function is executed by causing the printer 102 and the scanner 104 to cooperate with each other.

  Hereinafter, with reference to FIG. 18 and FIG. 19, another method of linking a plurality of devices by applying the marker type AR technology or the markerless type AR technology will be described. The screen of the display part of the terminal device 14 is shown by FIG.18 and FIG.19. As an example, the image forming apparatus 10 is used as the device 76 to be cooperated, and the PC 92 is used as the device 78 to be cooperated. In this example, the image forming apparatus 10 and the PC 92 are photographed separately. This is because the devices to be cooperated are not necessarily arranged in the vicinity. Of course, it is also possible to respond by changing the angle of the shooting area of the shooting means, or enlarging or reducing the shooting area, but if such an operation can not cope with the situation, dividing the shooting by the shooting means into multiple times It may be possible to identify the device to be cooperated as a solution. When the photographing by the photographing means is divided into a plurality of times, the identification information of the device identified in each photographing cycle is stored in the storage unit of the terminal device 14 or the server 80. For example, as shown in FIG. 18, the image forming apparatus 10 is photographed in a state in which the image forming apparatus 10 is included in the imaging area of the camera 46, and in a state in which the PC 92 is included in the imaging area of the camera 46 as illustrated in FIG. The PC 92 is photographed. Thus, image data representing the image forming apparatus 10 and image data representing the PC 92 are generated. By applying marker type AR technology or markerless type AR technology to each image data, the device identification information of the image forming apparatus 10 and the device identification information of the PC 92 are specified, and the cooperation function is specified.

  As still another method, a device to be cooperated may be set in advance as a basic cooperative device. For example, it is assumed that the image forming apparatus 10 is preset as a basic cooperation device. The device identification information indicating the basic cooperation device may be stored in advance in the storage unit 48 of the terminal device 14 or may be stored in advance in the storage unit 84 of the server 80. Alternatively, the user may designate the basic cooperation device by using the terminal device 14. When the basic cooperation device is set, the user shoots a device to be cooperated other than the basic cooperation device by the camera 46 of the terminal device 14. For example, when using the PC 92 as a collaboration target, the user shoots the PC 92 with the camera 46, as shown in FIG. Thereby, the device identification information of the PC 92 is specified, and the cooperation function to be executed by cooperation of the image forming apparatus 10 and the PC 92 is specified.

  Next, with reference to FIG. 20, a method of linking a plurality of devices by applying position information AR technology will be described. FIG. 20 shows the devices disposed in the search area. For example, the terminal device 14 has a GPS function, acquires terminal position information indicating the position of the terminal device 14 itself, and transmits the terminal position information to the server 80. The control unit 88 of the server 80 refers to position association information indicating the association between the device position information indicating the installation position of the device and the device identification information, and the range set in advance with reference to the position of the terminal device 14 Identify each device located in the inside as a collaboration candidate device. For example, as shown in FIG. 20, it is assumed that the image forming apparatus 10, the PC 92, the printer 102, and the scanner 104 are disposed within a range 114 preset based on the terminal device 14. In this case, the image forming apparatus 10, the PC 92, the printer 102, and the scanner 104 are specified as cooperation candidate devices. The device identification information of each cooperation candidate device is transmitted from the server 80 to the terminal device 14 and displayed on the UI unit 50 of the terminal device 14. As device identification information, an image of a cooperation candidate device may be displayed, or a character string such as a device ID may be displayed. The user specifies a device to be cooperated from the group of cooperation candidate devices displayed in the UI unit 50. The device identification information of the cooperation target device designated by the user is transmitted from the terminal device 14 to the server 80, and the cooperation function is specified in the server 80 based on the device identification information of the cooperation target device. In the UI unit 50 of the terminal device 14, linked function identification information indicating the linked function is displayed. The identification processing of the cooperation candidate device and the identification processing of the cooperation function may be performed by the terminal device 14.

  Hereinafter, processing by the image forming system according to the second embodiment will be described with reference to FIG. FIG. 21 is a sequence diagram showing the process.

  First, the user uses the terminal device 14 to issue an instruction to start an application (program) for executing the cooperation function. The control unit 52 of the terminal device 14 activates the application according to the instruction (S60). The application may be stored in advance in the storage unit 48 of the terminal device 14 or may be downloaded from the server 80 or the like.

  Next, the control unit 52 of the terminal device 14 reads user account information (user identification information) of the user (S61). This reading process is the same as the process of step S02 according to the first embodiment.

  The usage history of the collaboration function may be managed for each user, and information indicating the collaboration function used in the past by the user indicated by the read user account information may be displayed on the UI unit 50 of the terminal device 14. The information indicating the usage history may be stored in the storage unit 48 of the terminal device 14 or may be stored in the storage unit 84 of the server 80. In addition, information indicating a collaboration function being used may be displayed at a frequency higher than a preset usage frequency. By providing such a shortcut function, the time and effort of the user's operation regarding the cooperation function can be reduced.

  Next, by applying the marker type AR technology, the markerless type AR technology, or the position information AR technology, a device to be cooperated is specified (S62). When the marker type AR technology or the markerless type AR technology is applied, the user shoots a device to be cooperated with the camera 46 of the terminal device 14. As an example, when the devices 76 and 78 are used as linked devices, the user shoots the devices 76 and 78 with the camera 46. As a result, image data representing the devices 76 and 78 is generated, and the device type identification information of each of the devices 76 and 78 is specified by applying the marker type AR technology or the markerless type AR technology. When the position information AR technology is applied, the device position information of each of the devices 76 and 78 is acquired, and the device identification information of each of the devices 76 and 78 is specified based on the device position information.

  Next, the terminal device 14 transmits information indicating a connection request to the devices 76, 78 targeted for cooperation (S63). For example, when the address information indicating the addresses of the devices 76 and 78 is stored in the server 80, the terminal device 14 acquires the address information of the devices 76 and 78 from the server 80. When the device identification information includes the address information, the terminal device 14 may obtain the address information of each of the devices 76, 78 from the device identification information of each of the devices 76, 78. Alternatively, the terminal device 14 may store the address information of each of the devices 76 and 78. Of course, the terminal device 14 may obtain the address information of the devices 76 and 78 by another method. The terminal device 14 transmits information indicating a connection request to the devices 76 and 78 using the address information of the devices 76 and 78, respectively.

  The devices 76 and 78 permit or disallow connection with the terminal device 14 (S64). For example, if the devices 76 and 78 correspond to devices whose connection is not permitted or if the number of terminal devices requesting connection exceeds the upper limit, the connection is not permitted. If the connection from the terminal device 14 is permitted, the change operation may be prohibited so that the setting information unique to the devices 76 and 78 is not changed from the terminal device 14. For example, the change of the color parameter of the image forming apparatus or the setting time when shifting to the power saving is prohibited. This improves the security for the devices 76, 78. Alternatively, when linking the devices 76 and 78, the change of the setting information may be restricted as compared with the case where the devices 76 and 78 are used alone without being linked. For example, as compared with the case where the devices 76 and 78 are used alone, fewer setting items may be changed. Further, browsing of personal information of other users such as operation history may be prohibited. This improves the security of the user's personal information.

  Result information indicating permission or non-permission of connection is transmitted from the devices 76 and 78 to the terminal device 14 (S65). If the connection to the device 76, 78 is authorized, communication is established between the terminal device 14 and the device 76, 78.

  When the connection to the devices 76 and 78 is permitted, linked function identification information indicating a linked function to be executed by linking the devices 76 and 78 is displayed on the UI unit 50 of the terminal device 14 (S66) . As described above, the device identification information of each of the devices 76 and 78 specifies the cooperation function to be executed by causing the devices 76 and 78 to cooperate, and the cooperation function identification information indicating the cooperation function is displayed on the terminal device 14 Be done. The specific process may be performed by the server 80 or by the terminal device 14.

  Next, the user instructs the execution of the cooperation function using the terminal device 14 (S67). In response to the instruction, execution instruction information indicating an execution instruction of the cooperation function is transmitted from the terminal device 14 to the devices 76 and 78 (S68). The execution instruction information transmitted to the device 76 includes information (for example, job information) indicating a process to be executed by the device 76, and the execution instruction information transmitted to the device 78 is executed by the device 78. Information (for example, job information) indicating the process to be performed.

  The devices 76 and 78 having received the execution instruction information execute the function according to the execution instruction information (S69). For example, as in the scan transfer function of transferring scan data from the image forming apparatus 10 to the PC 92, when the cooperation function includes a process of transmitting and receiving data between the devices 76 and 78, the devices 76 and 78 Communication is established between In this case, for example, the execution instruction information transmitted to the device 76 includes the address information of the device 78, the execution instruction information transmitted to the device 76 includes the address information of the device 78, and those address information Communication is established between the devices 76, 78 using this.

  When the execution of the cooperation function ends, result information indicating the completion of the execution of the cooperation function is transmitted from the devices 76 and 78 to the terminal device 14 (S70). The display unit of the UI unit 50 of the terminal device 14 displays that the execution of the cooperation function has been completed (S71). If the indication that the execution has been completed is not displayed even when the time set in advance has passed from the time when the execution instruction is given, the control unit 52 of the terminal device 14 causes the display unit of the UI unit 50 to display information indicating an error. It may be displayed and the execution instruction information or information indicating the connection request may be transmitted to the devices 76 and 78 again.

  Next, the user confirms whether or not to cancel the cooperation state of the devices 76 and 78 (S72), and a process according to the presence or absence of the cancellation is executed (S73). When canceling the cooperation state, the user uses the terminal device 14 to give a cancellation instruction. Thus, the communication between the terminal device 14 and the devices 76 and 78 is released. Similarly, communication between the devices 76, 78 is also released. If the cooperation state is not canceled, an execution instruction may be continuously given.

  Moreover, you may increase the apparatus of cooperation object. For example, the device identification information of the third device is acquired, and the cooperation function to be executed is specified by causing the three devices including the devices 76 and 78 to cooperate. The information indicating that the devices 76 and 78 have already been identified is stored in the terminal device 14 or the server 80.

  The device identification information of each of the devices 76 and 78 to be cooperated and the cooperation function identification information indicating the executed cooperation function may be stored in the terminal device 14 or the server 80 or the like. For example, history information in which user account information (user identification information), device identification information of each device to be cooperated, and cooperation function identification information indicating the executed cooperation function are created for each user is created. And stored in the terminal device 14 or the server 80 or the like. The history information may be created by the terminal device 14 or may be created by the server 80. By referring to the history information, it is specified what kind of cooperation function has been executed using what kind of device group.

  The devices 76 and 78 may store, as history information, user account information indicating a user who has requested connection or terminal identification information indicating the terminal device 14. By referring to this history information, the user using the devices 76 and 78 is identified. For example, when identifying the user when the devices 76 and 78 are broken or when charging for consumables or the like, the user may be identified using history information. The history information may be stored in the server 80 or the terminal device 14 or may be stored in another device.

  Next, with reference to FIG. 22, the transition of the screen displayed on the UI unit 50 of the terminal device 14 from the recognition of the device to be cooperated to the execution of the cooperation function will be described.

  As an example, as shown in FIG. 16, the case where the image forming apparatus 10 and the PC 92 are used as devices to be cooperated will be described. In the example shown in FIG. 22, the image forming apparatus 10 has at least a scan function, a print function, and a copy function as an image forming function, and functions as a so-called multifunction machine.

  First, as shown in FIG. 16, the user captures an image of the cooperation target image forming apparatus 10 (multifunction peripheral) and the PC 92 by the camera 46 of the terminal device 14. As a result, as shown in FIG. 22A, a device image 98 representing the image forming apparatus 10 and a device image 100 representing the PC 92 are displayed on the screen 96 of the UI unit 50 of the terminal device 14.

  As one example, the image forming apparatus 10 and the PC 96 are recognized by applying the marker type AR technology or the markerless type AR technology, and as shown in FIG. 22B, the recognition device screen 116 is displayed on the UI unit 50. On the recognition device screen 116, device identification information of the image forming apparatus 10 and device identification information of the PC 92 are displayed. For example, on the recognition device screen 116, (1) a character string indicating a multifunction peripheral is displayed as device identification information of the image forming apparatus 10, and (2) a character string indicating a personal computer is displayed as device identification information of the PC 92. In addition, the names or product names of the image forming apparatus 10 and the PC 92 may be displayed.

  When the device identification information of the image forming apparatus 10 and the device identification information of the PC 92 are specified, the cooperation function to be executed by causing the image forming apparatus 10 and the PC 92 to cooperate is specified, as shown in FIG. The cooperation function selection screen 118 is displayed on the UI unit 50. The linked function selection screen 118 includes, for example, (1) information indicating a scan data transfer function (scan transfer function) to a personal computer, and (2) information indicating a function for printing document data stored in the personal computer. , Are displayed as linkage function information. When execution of the cooperative function of (1) is instructed, a document is read by the scan function of the image forming apparatus 10 (multifunction machine), scan data is generated, and the scan data is transferred from the image forming apparatus 10 to the PC 92 Ru. When the execution of the cooperation function (2) is instructed, the document data stored in the PC 92 is transmitted from the PC 92 to the image forming apparatus 10, and the print function of the image forming apparatus 10 places a document based on the document data on paper. It is printed. Note that the device group selected by the user on the recognition device screen 116 shown in FIG. 22B is used as a device to be cooperated, and the cooperation function is executed by linking the device groups selected by the user. The linked function information indicating “” may be displayed on the linked function selection screen 118.

  Another display form may be adopted as the display form of the cooperation function information. For example, the control unit 52 of the terminal device 14 causes the display unit of the UI unit 50 to display information (for example, a button image group) indicating a function group including a cooperation function, and a plurality of devices that execute the cooperation function in cooperation If is not identified (recognized), linked function information (for example, an image of a button) is displayed on the display unit in a state where use of the linked function is not possible. When device identification information of a plurality of devices that execute the cooperation function in cooperation is acquired and the plurality of devices are recognized, the control unit 52 displays the cooperation function information in a state where the cooperation function can be used. Display on the department. Describing with a specific example, the control unit 52 displays, for example, information (for example, an image group of buttons) indicating the print function, the scan function, the copy function, and the scan transfer function as a cooperation function on the UI unit 50. Let When a plurality of devices that execute the scan transfer function in cooperation are not recognized, the control unit 52 causes the display unit to display the cooperation function information in a state where the scan transfer function can not be used. For example, the control unit 52 does not receive an execution instruction of the scan transfer function. Thus, even when the user designates cooperation function information (for example, an image of a button) indicating a scan transfer function and gives an execution instruction, the scan transfer function is not executed. When a plurality of devices that execute the scan transfer function in cooperation are recognized, the control unit 52 causes the display unit to display cooperative function information (for example, an image of a button) in a state where the scan transfer function can be used. When an instruction to execute the scan transfer function is given by the user, the control unit 52 receives the instruction to execute the scan transfer function and transmits execution instruction information indicating an instruction to execute the scan transfer function to the device group to be cooperated.

  For example, when (1) the scan transfer function is designated by the user, a confirmation screen 120 is displayed on the UI unit 50 as shown in FIG. When the user presses the “NO” button on the confirmation screen 120, the screen returns to the previous linked function selection screen 118. When the user presses the "YES" button, the scan transfer function is executed. When the execution of the scan transfer function is completed, as shown in FIG. 22E, the UI unit 50 displays an execution completion screen 122 indicating the completion of the execution of the cooperation function. In addition, on the execution completion screen 122, information for instructing the user whether or not to release the connection of the device to be cooperated is displayed. In the execution completion screen 122, when the user instructs to release the connection of the device, the connection between the terminal device 14, the image forming apparatus 10, and the PC 92 is released. If the user does not instruct to release the connection of the device, the screen returns to the cooperation function selection screen 118.

  As described above, according to the second embodiment, by applying the AR technology, the cooperation function to be executed by causing the group of cooperation objects to cooperate is specified, and the cooperation function identification information indicating the cooperation function is a terminal. It is displayed on the device 14. As a result, even if it is not clear from the appearance of what kind of cooperation function can be executed depending on the group of devices to be cooperated, it becomes easy for the user to understand what kind of cooperation function can be executed . In addition, it is convenient because functions that can not be performed individually by individual devices can be performed by linking a plurality of devices. In addition, it is possible to use the cooperation function simply by applying AR technology and recognizing the device to be cooperated, which is simpler than when the user performs settings etc. for executing the cooperation function by manual operation. The work of the collaboration function can be used, and the time and effort of the user can be reduced.

  Further, according to the second embodiment, for example, in an environment where a plurality of devices are used by a plurality of users, information related to the cooperation function is appropriately displayed on the terminal device 14 of each user. For example, even if a user interface such as a touch panel is removed from the device, the terminal device 14 is used as the user interface, and the information on the cooperation function executed by causing the plurality of devices to cooperate is information of each user. It is properly displayed on the terminal device 14. As another scene, for example, when a plurality of devices are temporarily used at the place where the user goes, etc., a user interface suitable for the user, that is, by linking a plurality of devices specified by the user The terminal device 14 implements a user interface representing the linked function to be performed.

  Hereinafter, specific examples of the cooperation function will be described.

(Specific example 1)
The cooperation function according to the first specific example is a cooperation function that is executed by combining the image forming apparatus 10 as a multifunction machine and a display device such as a projector. The cooperation function is a function of printing the content of the screen displayed on a display device such as a projector by the multifunction peripheral (image forming apparatus 10). As an example, it is assumed that the device 76 is a multifunction peripheral and the device 78 is a display device such as a projector. In the first example, device identification information of each of the multi-functional peripheral and the display device is acquired by applying the AR technology, and cooperation is performed by linking the multi-functional peripheral and the display device based on the device identification information. The function is identified. Cooperation function identification information indicating the cooperation function is displayed on the terminal device 14. When the user uses the terminal device 14 to instruct execution of the cooperation function, the terminal device 14 transmits execution instruction information to the multifunction peripheral and the display device. In response to this, the display device transmits the information (screen information) displayed on the screen to the multifunction device, and the multifunction device prints the screen information sent from the display device on a sheet. According to the first embodiment, information is provided so that the user can understand what functions are executed by the multifunction device and the display device only by recognizing the multifunction device and the display device by the AR technology, and the information is displayed on the display device The content of the displayed screen is printed by the multifunction peripheral. Therefore, the time and effort of the user can be reduced as compared with the case where the user performs print setting and the like by manual operation.

(Specific example 2)
The cooperation function according to the second specific example is a cooperation function executed by combining the image forming apparatus 10 as a multifunction machine and a telephone. This cooperation function is a function A for printing the content (call content) in which the user is talking using the telephone by the multi-function device (image forming apparatus 10), and the electronic document data indicating the call content is mail set in advance. At least one of a function B for transmitting an electronic mail to an address and a function C for facsimile-transmitting electronic document data to a fax number associated with a telephone number of a calling party. As an example, it is assumed that the device 76 is a multifunction peripheral and the device 78 is a telephone. In the second embodiment, device identification information of each of the multifunction peripheral and the telephone is acquired by applying the AR technology, and a cooperation function group executed by causing the multifunction peripheral and the telephone to cooperate based on the device identification information ( Functions A, B, C) are identified. Cooperation function identification information indicating functions A, B, and C as cooperation functions is displayed on the terminal device 14. When the user uses the terminal device 14 to select an execution target function from among the functions A, B, and C and instructs the execution of the cooperation function, the terminal device 14 transmits execution instruction information to the multifunction peripheral and the telephone. In response to this, the telephone transmits data indicating the content of the call to the multifunction peripheral. When execution of function A is designated, the multi-functional peripheral prints a character string indicating the contents of the call on a sheet. When execution of the function B is instructed, the multi-functional peripheral transmits electronic document data indicating the contents of the call by e-mail to a preset e-mail address (for example, to the e-mail address of the other party). When execution of the function C is instructed, the multi-functional peripheral facsimile-transmits the electronic document data to the fax number associated with the telephone number of the other party. When a plurality of functions are designated by the user from among the functions A, B, and C and an execution instruction is given, the plurality of functions may be executed. According to the second embodiment, it is possible to provide information that allows the user to know what kind of function is executed by the MFP and the telephone just by recognizing the MFP and the telephone by the AR technology, and the call contents of the telephone are composited. At least one of printing, email transmission and facsimile transmission is performed by the machine. Therefore, the time and effort of the user can be reduced as compared with the case where the user performs print setting and the like by manual operation.

(Specific example 3)
The cooperation function according to the third example is a cooperation function that is executed by combining the image forming apparatus 10 as a multifunction machine and a clock. This cooperation function is a function to add a timer function to the multifunction peripheral. As an example, it is assumed that the device 76 is a multifunction device and the device 78 is a watch. In the third example, by applying the AR technology, device identification information of each of the multifunction peripheral and the clock is acquired, and based on the device identification information, the cooperation function executed by causing the multifunction peripheral and the clock to cooperate is specified. Be done. Cooperation function identification information indicating the cooperation function is displayed on the terminal device 14. When the user uses the terminal device 14 to instruct execution of the cooperation function, image formation using the timer function is performed. For example, the multifunction peripheral executes image formation such as printing at a time designated by the user. According to the third embodiment, the information is provided so that the user can understand what functions are executed by the multifunction device and the clock only by recognizing the multifunction device and the clock by the AR technology, and the multifunction device has the timer function. Granted. Therefore, even in the case of using a multifunction peripheral that does not have a timer function, it is possible to perform image formation using the timer function.

(Specific example 4)
The cooperation function according to the fourth example is a cooperation function executed by combining the image forming apparatus 10 as a multifunction machine and the monitoring camera. The cooperative function is a function of deleting specific information (for example, job information, image data, and the like) stored in the multifunction device according to the content photographed by the monitoring camera. As an example, it is assumed that the device 76 is a multifunction device and the device 78 is a surveillance camera. In the fourth embodiment, the device identification information of each of the multifunction peripheral and the monitoring camera is acquired by applying the AR technology, and the cooperation function executed by causing the multifunction peripheral and the monitoring camera to cooperate based on the device identification information Is identified. Cooperation function identification information indicating the cooperation function is displayed on the terminal device 14. When the user uses the terminal device 14 to instruct execution of the cooperation function, the terminal device 14 transmits execution instruction information to the MFP and the monitoring camera. In response to this, the monitoring camera analyzes the shooting content and, when a specific event occurs, transmits an information deletion instruction to the multi-functional peripheral. For example, when the person who is suspected to be a suspicious person is photographed by the surveillance camera outside business hours, the surveillance camera transmits an information deletion instruction to the multifunction device. The multifunction peripheral deletes job information and image data stored in the multifunction peripheral according to the information deletion instruction. This improves the security of the multifunction device. According to the fourth embodiment, it is possible to provide information that allows the user to know what functions are to be executed by the MFP and the surveillance camera only by recognizing the MFP and the surveillance camera by the AR technology, and the surveillance camera is used. Then, monitoring of the multifunction peripheral is performed. Therefore, the time and effort of the user can be reduced as compared with the case where the user manually performs the monitoring setting and the like.

  In addition to the above-described example, by linking the image forming apparatus and the translation apparatus, characters included in a document to be printed by the image forming apparatus are translated in parallel into a language translatable by the translation apparatus, and paper is used. A combination of linked functions to be output is also conceivable.

(Specific example 5)
Although the cooperation function according to the example described above is a cooperation function executed by combining a plurality of devices having different functions from each other, the cooperation function may be executed by combining a plurality of devices having the same function. . In this case, the plurality of devices execute the same function to share the processing. As an example, the cooperation function according to the fifth example is a cooperation function executed by combining a plurality of image forming apparatuses 10 as multifunction machines. The cooperation function is, for example, an image forming function such as a print function, a copy function, and a scan function. In the fifth example, the device identification information of each of the plurality of multifunction devices is acquired by applying the AR technology, and the cooperation function (for example, performed by causing the plurality of multifunction devices to cooperate based on the device identification information) The image forming function is identified. Cooperation function identification information indicating the cooperation function is displayed on the terminal device 14. When the user uses the terminal device 14 to instruct execution of the cooperation function, the terminal device 14 transmits execution instruction information to a plurality of cooperation target multifunction devices. The terminal device 14 divides the processing content (for example, job) according to the number of multifunction devices to be cooperated, assigns the processing content (job) after division to each of the multifunction devices, and executes the execution instruction indicating the processing content after division Send information to each MFP. In response to this, each multifunction machine executes a process (job) assigned to itself. For example, the terminal device 14 divides one job relating to printing according to the number of multifunction devices to be cooperated, assigns the divided print job to each multifunction device, and executes execution instruction information indicating the divided print job Send to each MFP. In response to this, each multifunction machine executes a print job assigned to itself by executing the print function. Further, the terminal device 14 may allocate the processing content after division to each device according to the performance of each device to be cooperated. For example, a multi-functional peripheral having a color printing function may be assigned a job for which color printing is set, and a multi-functional peripheral not having a color printing function may be assigned a job for which monochrome printing is set.

  Furthermore, as another specific example, the high-speed printing mode or the preliminary printing mode (a mode for creating a plurality of copies of the same content) may be executed as a cooperation function by linking a plurality of devices having the same function.

  Hereinafter, with reference to FIG. 23, a modification according to the second embodiment will be described. FIG. 23 shows the execution priority of the cooperation function. In the modification, when a plurality of terminal devices 14 simultaneously request connection to the same device, connection is permitted in accordance with a preset execution priority. As shown in FIG. 23, in the case of the connection request in the case of an emergency (hurry), the degree of influence on the priority is “oversized”. In the case of a connection request from the owner of the device, the degree of influence is "high". For organizational ranks, the impact on priority is "medium", and connection requests from higher rank users have higher priorities. As for the scheduled completion time of the job (image formation processing), the degree of influence on the priority is “small”, and the connection request with the shorter scheduled job completion time has a higher priority. For example, in the case where a plurality of terminal devices 14 simultaneously make a connection request to the same device, the terminal device 14 which has made a connection request including information indicating an emergency situation is connected with the highest priority to the device. Be done. When there is no terminal device 14 that has made a connection request for an emergency among the plurality of terminal devices 14, the terminal device 14 of the owner of the device is connected to the device with the highest priority. In the case where there is no terminal device 14 that has made a connection request for an emergency among the plurality of terminal devices 14 and there is no terminal device 14 of the owner of the device, the terminal device of the user with high organizational rank 14 is connected to the device with priority. If there is no terminal device 14 that has made a connection request for an emergency among the plurality of terminal devices 14, there is no terminal device 14 of the owner of the device, and the class of each user is also the same. The terminal device 14 that has instructed execution of a job with a short estimated time is preferentially connected to the device. In the emergency situation, the owner of the device, the organizational rank, and the scheduled completion time of the job, which item is to be emphasized may be arbitrarily set by the administrator of the device to be cooperated. . For example, the manager may arbitrarily change the degree of influence of each item, or may not use some items in determining the priority. Further, the priority of use of the device may be displayed on the UI unit 50 of the terminal device 14 according to the attribute information of each user. The attribute information is, for example, the urgency, whether or not the owner of the device, the organizational rank, the scheduled completion time of the job, and the like. As described above, by determining the execution priority of the cooperation function, when there is a connection request simultaneously to the same device, the user with higher importance is preferentially connected to the device.

  As another modification, when a plurality of terminal devices 14 are simultaneously making connection requests to the same device, the terminal devices 14 may notify each other of the interrupt. For example, each terminal device 14 may acquire the address information of the other terminal device 14 via the same device, or may use the process such as broadcast to transmit the address information of the other terminal device 14. You may get it. For example, when the user designates an interrupt request using the terminal device 14, the terminal device 14 transmits an interrupt notification to another terminal device 14 that is simultaneously making a connection request to the same device. Information indicating an interrupt notification is displayed on the UI unit 50 of the other terminal device 14. For example, when the user of the other terminal device 14 cancels the connection request to the device in response to the interrupt notification, communication is established between the terminal device 14 that made the interrupt request and the device. Alternatively, when the user of the other terminal device 14 permits the interrupt processing, the other terminal device 14 may transmit the information indicating the permission to the terminal device 14 that has made the interrupt request. In this case, the terminal device 14 that has made the interrupt request may preferentially connect to the device by transmitting information indicating the permission to the device. By performing the interrupt notification as described above, it is possible to cope with the case where you want to execute the cooperation function in a hurry.

Third Embodiment
Hereinafter, an image forming system as an information processing system according to the third embodiment of the present invention will be described. FIG. 24 shows a server 124 according to the third embodiment. The image forming system according to the third embodiment is a system combining the image forming system according to the first embodiment and the image forming system according to the second embodiment, and a server 124 is used instead of the server 80 according to the second embodiment. including. The configuration other than the server 124 is the same as the configuration of the image forming system according to the second embodiment shown in FIG.

  Similarly to the server 12 according to the first embodiment, the server 124 manages functions available to the user for each user, and makes a plurality of devices cooperate with each other as with the server 80 according to the second embodiment. Is a device that manages the cooperation function executed by Also, the server 124 is an apparatus that executes a specific function, like the server 12 according to the first embodiment. The specific function executed by the server 124 is, for example, a function related to image processing. The functions managed by the server 124 are, for example, a function executed using the devices 76 and 78, a function executed by the server 124, and the like. The management of the functions available to the user, the management of the collaboration functions, and the execution of the specific functions may be performed by separate servers or by the same server. The server 124 also has a function of transmitting and receiving data to and from other devices.

  In the image forming system according to the third embodiment, a function is purchased by the user by using the terminal device 14, and the history of the purchase is managed by the server 124 as a function purchase history. The functions purchased by the user are executed by, for example, the devices 76 and 78 or the server 124. In addition, when the cooperation function is purchased, the cooperation function is executed by causing a plurality of devices to cooperate.

  Hereinafter, the configuration of the server 124 will be described in detail.

  The communication unit 126 is a communication interface, and has a function of transmitting data to another device via the communication path N and a function of receiving data from the other device. The communication unit 126 may be a communication interface having a wireless communication function, or may be a communication interface having a wired communication function.

  The storage unit 128 is a storage device such as a hard disk. The storage unit 128 stores device function information 30, function purchase history information 32, linked function information 86, various data, various programs, and the like. Of course, they may be stored in separate storage devices, or may be stored in one storage device. The device function information 30 and the function purchase history information 32 are the same as the device function information 30 and the function purchase history information 32 according to the first embodiment, and the cooperation function information 86 and the cooperation function information 86 according to the second embodiment. It is the same.

  The function execution unit 34 of the server 124 is the same as the function execution unit 34 of the server 12 according to the first embodiment. As in the second embodiment, the server 124 may not have the function execution unit 34.

  The control unit 130 controls the operation of each unit of the server 124. Further, the control unit 130 includes a purchase processing unit 38, a purchase history management unit 40, and a specifying unit 132.

  The purchase processing unit 38 and the purchase history management unit 40 of the server 124 are the same as the purchase processing unit 38 and the purchase history management unit 40 of the server 12 according to the first embodiment.

  Similarly to the identifying unit 42 of the server 12 according to the first embodiment, the identifying unit 132 receives the device identification information for identifying the device to be used, the device function information 30 stored in the storage unit 128. Identify the function group possessed by the device to be used. Further, as in the case of the identification unit 42 according to the first embodiment, the identification unit 132 refers to the function purchase history information 32 stored in the storage unit 128 when the user identification information for identifying the target user is received. By doing this, the functional group that can be used by the target user is identified. As in the first embodiment, for example, when receiving the device identification information of the device to be used and the user identification information of the target user, the specifying unit 132 is a function of the device to be used, and is used by the target user Identify possible features.

  Further, when the specifying unit 132 receives device identification information for identifying each device to be cooperated, as in the case of the specifying unit 90 of the server 80 according to the second embodiment, the cooperation unit stored in the storage unit 128 By referring to the function information 86, a cooperation function to be executed by making the cooperation target devices cooperate is specified.

  Furthermore, in the third embodiment, the specifying unit 132 is a cooperation function that is executed by causing the device group to be cooperated to cooperate, and specifies a cooperation function that can be used by the target user. For example, the function purchase history information 32 includes, for each user, information indicating a collaboration function available to the user, that is, information indicating a collaboration function purchased by the user. The purchase processing of the cooperation function is the same as the purchase processing of the function in the first embodiment. The identifying unit 132 receives device identification information for identifying each device to be cooperated, and executes the cooperation of each device to be cooperated by referring to the cooperation function information 86 stored in the storage unit 128. Identify the collaboration function to be Further, the identification unit 132 receives user identification information for identifying a target user, and refers to the function purchase history information 32 stored in the storage unit 128 to obtain a cooperative function purchased by the target user, that is, , And identify a collaboration function available to the target user. The identifying unit 132 is a cooperation function that is executed by causing the device group to be cooperated to cooperate by the above process, and identifies a cooperation function that can be used by the target user. Cooperation function identification information indicating the cooperation function is transmitted from the server 124 to the terminal device 14 and displayed on the UI unit 50 of the terminal device 14. This makes it easy for the target user to understand what kind of cooperation function can be used. When the execution instruction of the cooperation function is given by the target user, the cooperation function is executed by the group of devices to be cooperated, as in the second embodiment.

  The control unit 52 of the terminal device 14 causes the display unit of the UI unit 50 to display cooperation function identification information indicating each cooperation function executed by causing the device group to be cooperated to cooperate, and the cooperation available to the target user The linked function identification information indicating the function may be displayed on the display unit of the UI unit 50 in distinction from the linked function identification information indicating the linked function which can not be used by the target user. As a result, in the target user, it becomes easy to understand what kind of cooperation function can be executed by the group of devices to be cooperated, and it becomes easy to understand what kind of cooperation function can be used by the target user .

  As another example, the identifying unit 132 identifies a plurality of functions available to the target user by referring to the function purchase history information 32, and identifies a collaboration function to be executed by linking the plurality of functions. You may For example, when the target user can use the scan function and the print function as an individual function, the copy function executed by linking the scan function and the print function can be used by the target user as a cooperation function. . Further, the identifying unit 132 identifies a cooperation function group to be executed by causing a plurality of devices to be cooperated to cooperate by referring to the cooperation function information 86. According to the above process, the specifying unit 132 may be a cooperation function that is executed by causing a plurality of cooperation target devices to cooperate, and may specify a cooperation function that can be used by the target user.

  Also in the third embodiment, the device identification information of the device is acquired by applying the AR technology. Of course, the device identification information of the device may be acquired without applying the AR technology. The user operation and the process when linking a plurality of devices to be linked are the same as the operation and the process in the second embodiment. Further, as in the first and second embodiments, the device function information 30, the function purchase history information 32, and the cooperation function information 86 are stored in the storage unit 48 of the terminal device 14, and the purchase history management unit 40 and the identification unit 132. May be provided in the control unit 52 of the terminal device 14 and processing using these may be executed by the terminal device 14.

  According to the third embodiment, when the user wants to know the individual functions available to the user in the individual devices, information indicating the usable functions by recognizing the devices to be used by the AR technology Is displayed on the terminal device 14. In addition, when the user wants to know the cooperation function that can be used by the user, which is a cooperation function executed by causing a plurality of cooperation target devices to cooperate, the AR technology recognizes the plurality of cooperation target devices using AR technology. By doing this, information indicating available collaboration functions is displayed on the terminal device 14. As described above, information on available functions is displayed on the terminal device 14 according to the usage of the device.

Fourth Embodiment
Hereinafter, with reference to FIG. 25, an image forming system as an information processing system according to the fourth embodiment of the present invention will be described. FIG. 25 shows a server 134 according to the fourth embodiment. The image forming system according to the fourth embodiment includes a server 134 instead of the server 80 according to the second embodiment. The configuration other than the server 134 is the same as the configuration of the image forming system according to the second embodiment shown in FIG.

  The server 134 is a device that manages devices to be connected according to the function to be used, that is, devices to be connected to execute the function to be used. The function to be used is, for example, a cooperation function executed by linking a plurality of devices (for example, the devices 76 and 78), and the server 134 is a group of devices to be cooperated which can be executed by linking the cooperation function. to manage. Of course, the function to be used may be a function that each device can execute alone. The server 134 also has a function of transmitting and receiving data to and from another device.

  In the image forming system according to the fourth embodiment, a function to be used (for example, a function that the user wants to use) is specified by using the terminal device 14 and connection is required to execute the function to be used. Information indicating the device group is displayed on the terminal device 14.

  Hereinafter, the configuration of the server 134 will be described in detail.

  The communication unit 136 is a communication interface, and has a function of transmitting data to another device via the communication path N, and a function of receiving data from the other device. The communication unit 136 may be a communication interface having a wireless communication function, or may be a communication interface having a wired communication function.

  The storage unit 138 is a storage device such as a hard disk. The storage unit 138 stores cooperative function information 86, device management information 140, various data, various programs, and the like. Of course, they may be stored in separate storage devices, or may be stored in one storage device. The cooperation function information 86 is the same as the cooperation function information 86 according to the second embodiment.

  The device management information 140 is information for managing information related to the device, and for example, for each device, device identification information of the device and at least one of the device location information, the performance information, and the usage status information, Information indicating the correspondence between The device location information is information indicating the location where the device is installed, the performance information is information indicating the performance (spec) of the device, and the usage status information indicates the current usage status (status) of the device. It is information. For example, device location information and performance information are obtained in advance and registered in the device management information 140. The device position information of each device is acquired, for example, by using a GPS device. In addition, usage status information is transmitted from each device to the server 134 and registered in the device management information 140. For example, the usage status information is transmitted from the device to the server 134 at a preset time, at preset time intervals, or whenever the usage status changes. Of course, the usage status information may be acquired at another timing and registered in the device management information 140.

  The control unit 142 controls the operation of each unit of the server 134. The control unit 142, for example, manages the use status of each device, and updates the device management information 140 each time the use status information of each device is acquired. Further, the control unit 142 includes a specifying unit 144.

  The identifying unit 144 identifies a group of devices to be connected according to the function to be used. For example, the specifying unit 144 receives cooperation function identification information indicating a cooperation function as a use target function, and a plurality of cooperation function information 86 stored in the storage unit 138 are associated with the cooperation function identification information. Identify device identification information. As a result, the device group to be connected to execute the function to be used, that is, the device group that can be executed in cooperation with the cooperation function is identified (recognized). For example, cooperation function identification information is transmitted from the terminal device 14 to the server 134, and the identification unit 144 specifies device identification information of each device associated with the cooperation function identification information. The device identification information of each device is, for example, transmitted from the server 134 to the terminal device 14 and displayed on the terminal device 14. Thus, the terminal device 14 displays the device group to be connected in order to execute the function to be used (for example, the cooperation function), that is, information indicating the device group that can be executed by linking the function to be used. .

  In addition, when the device group to be connected is specified, the specifying unit 144 determines, in the device management information 140, device location information, performance information, and usage associated with device identification information for each individual device to be connected. Identify at least one of the status information. Information such as device position information is transmitted from the server 134 to the terminal device 14, for example, and displayed on the terminal device 14.

  The function to be used may be a function that can be performed by each device alone. In this case, the specifying unit 144 specifies a single device to be connected in order to execute the function to be used, that is, a device capable of independently executing the function to be used. The information indicating the device is transmitted from the server 134 to the terminal device 14 and displayed on the terminal device 14.

  The device management information 140 may be stored in the storage unit 48 of the terminal device 14. In this case, the device management information 140 may not be stored in the storage unit 138 of the server 134. In addition, the control unit 52 of the terminal device 14 may include the specifying unit 144 and specify the device group to be connected. In this case, the server 134 may not include the identifying unit 144.

  Hereinafter, processing by the image forming system according to the fourth embodiment will be described in detail with reference to FIG.

  For example, the control unit 52 of the terminal device 14 causes the UI unit 50 to display a list of functions, and the user selects a function (use target function) that the user wants to use from the list. As an example, as indicated by reference numeral 146 in FIG. 26, it is assumed that the function of “print contents of conversation on telephone” is selected as the function to be used. This function is a cooperative function that is executed by linking a telephone and a device having a print function (for example, a printer or a multifunction machine), and as indicated by reference numerals 148 and 150, a group of devices to be connected (a connection is required) Equipment groups) are telephones and printers. Of course, instead of the printer, a multifunction machine having a printing function may be used as the device to be connected.

  Cooperation function identification information indicating the cooperation function selected by the user is transmitted from the terminal device 14 to the server 134. In the server 134, the specifying unit 144 specifies a plurality of pieces of device identification information associated with the cooperation function identification information in the cooperation function information 86 stored in the storage unit 138. As a result, the device group to be connected to execute the cooperation function, that is, the device group that can execute the cooperation function in cooperation is identified (recognized). In the example shown in FIG. 26, as indicated by reference numerals 152, 154 and 156, the telephones A, B and the printer A recognize as devices to be connected in order to execute the function of "printing the contents of conversation on the telephone". It is done. The telephones A and B and the printer A are, like the devices 76 and 78, devices included in the image forming system.

  At this stage, even if the device identification information of each of the telephones A and B and the printer A is transmitted from the server 134 to the terminal device 14 as information on a group of devices to be connected and displayed on the UI unit 50 of the terminal device 14 Good. Thereby, the user is provided with information indicating a group of devices to be connected in order to execute the function to be used.

  When the device group to be connected is identified, the identifying unit 144 may acquire information on each of the telephones A and B and the printer A by referring to the device management information 140. For example, the specifying unit 144 acquires performance information indicating the performance (specs) of each of the telephones A and B and the printer A. In the example shown in FIG. 26, the performance indicated by reference numeral 158 is the performance of the telephone set A, the performance indicated by reference numeral 160 is the performance of the telephone set B, and the performance indicated by reference numeral 162 is the performance of the printer A. As performance of the telephones A and B, a frequency band that can be supported is defined. Further, the telephone set A is a telephone set compatible with foreign countries, and the telephone set B is a telephone set supported only in Japan. A resolution is specified as the performance of the printer A. The printer A is a printer compatible with color printing. The performance information of each of the telephones A and B and the printer A is transmitted from the server 134 to the terminal device 14 as information on a group of devices to be connected, and displayed on the UI unit 50 of the terminal device 14. This provides the user with useful information in selecting a device suitable for the function to be used. For example, when the user desires color printing, by referring to the performance information displayed on the UI unit 50, it becomes easy to find a device (printer compatible with color printing) that meets the desire.

  The screen transition of the UI unit 50 of the terminal device 14 will be described with reference to FIGS. 27A to 27N as an example of a device required to execute the cooperation function and an application requesting connection. A user is identified by launching an application and logging in to an account. Of course, it is possible to omit login, but if you want security and to execute or manage special functions individually for each user, you can do so by requesting account login. FIG. 27A shows a screen that allows the user to specify the cooperation function to be executed. The user input unit shown in FIG. 27A is a place where the user performs text input or voice input, or a place where the cooperation function that the user wants to use is input from the pull-down menu. In accordance with the content of the cooperation function input here, identification processing of devices necessary for the cooperation function is performed. If there is no problem in the input cooperation function, the screen changes to the next screen by pressing the OK button. FIG. 27B shows the result of automatically identifying the necessary device according to the content of the cooperation function input by the user input unit. As an example, since the cooperation function is a function of "printing contents of conversation on a telephone", a telephone and a printer are displayed as necessary devices.

  In FIG. 27C and FIG. 27E, among the identified necessary devices, devices of the same field that can be identified and used by the user in the past, and devices that are identified and extracted from networks newly available to the user Etc. are shown. A list of telephones is displayed on the screen shown in FIG. 27C, and a list of printers is displayed on the screen shown in FIG. 27E. The user touches and specifies the device name to be used from this list.

  FIGS. 27D and 27F show devices selected by the user from among the devices required to perform the cooperation function shown in FIGS. 27C and 27E. As shown in FIG. 27D, the telephone set B is selected, and as shown in FIG. 27F, the printer B is selected. If the device has been specified by mistake, it is possible to return to the selection screen by selecting "NO" on this confirmation screen. If "YES" is selected, the screen shifts to the next device selection screen.

  FIG. 27G shows a confirmation screen displayed after all the devices necessary to execute the cooperation function have been specified. If "NO" is selected on this confirmation screen, the screen returns to the selection screen of each device. Here, when “YES” is selected, the screen changes to a screen for sending a connection request for the selected device. The screen is shown in FIG. 27H.

  As shown in FIG. 27I, when the environment in which the cooperation function can be executed is established (for example, when the network connection or the execution function previously performed by each device is completed), the user is asked whether to execute the cooperation function immediately A message is displayed. If "YES" is selected here, the cooperation function is immediately executed, and if "NO" is selected, the connection state is held for a preset time, and the user can wait for immediate execution of the cooperation function. .

  The contents displayed on the screen can be switched depending on whether the execution of the linkage function succeeds or fails. If the execution of the cooperation function is successful, the screen transitions in the order of the screens shown in FIG. 27J, FIG. 27L and FIG. 27N. On the other hand, when the execution of the cooperation function fails, the screens transition in the order of the screens shown in FIGS. 27K, 27M and 27N. In the screen shown in FIG. 27N, execution of the same cooperation function, execution of another cooperation function, or termination of the application can be instructed again. When performing the same cooperation function, the process of connection setting is omitted. However, if the failure reason of the execution of the collaboration function is a problem unique to the collaboration function and there are other devices that can be selected, if “Execute same collaboration function” is selected on the screen shown in FIG. 27N. Even in this case, only the selection of the device causing the error may be changed. When “execution of another cooperation function” is selected, the screen changes to the screen shown in FIG. 27A. This application is terminated if "Exit application" is selected.

  As described above, the user can easily perform the setting necessary to execute the cooperation function simply by installing the device necessary for executing the cooperation function and the application requesting connection in the terminal device 14. Is possible.

  Also, the performance information of each device to be connected may be displayed according to the priority condition. The priority condition is set by the user, for example. For example, when high-quality printing is designated by the user, the identifying unit 144 may set the priority of a printer capable of color printing or the priority of a printer having a resolution higher than that of other printers to that of other printers Set high. The control unit 52 of the terminal device 14 causes the UI unit 50 to display the device identification information of the color printing compatible printer or the high resolution printer with priority over the device identification information of the other printers according to the priority. As another example, when a call to overseas is designated by the user, the identifying unit 144 sets the priority of the overseas compatible telephone higher than the priority of the international compatible telephone. In accordance with the priority, the control unit 52 causes the UI unit 50 to display the device identification information of the overseas compatible telephone prior to the device identification information of the international compatible telephone. In the case where there are a plurality of printers as candidates as other devices to be connected, printers closer to the position of the user may be preferentially displayed on the UI unit 50. For example, the control unit 52 compares the device identification information of a device with high priority with the device identification information of another device, and places the device identification information in a conspicuous position such as the center or upper part of the UI unit 50. As another example, high-priority devices may be arranged and displayed in a specific area where the user places high-priority devices in advance. Further, as another example, the device identification information of the high priority device is attached with information indicating the recommendation, the information of the high priority device is enlarged, or the font, color or display format of the character is changed. It may be displayed on the UI unit 50. This makes it easier to select devices suitable for the function to be used, as compared to the case where the device identification information of the device group to be connected is displayed randomly.

  FIGS. 28 to 31 show an example of priority display of devices. For example, as shown in FIG. 28, the size, color, and font of the character string are changed according to the priority, and the character string indicating the device is displayed on the UI unit 50 of the terminal device 14. Also, a position where the character string indicating a device with high priority (for example, the handset A for overseas use) is more visible than for a device with low priority (for example the handsets B and C for domestic support) (for example, the upper left position on the screen) It is arranged and displayed. As another example, as shown in FIG. 29, according to the priority, the shape of the image or mark showing the device is changed. In the example shown in FIG. 29, an image or a mark indicating a device with high priority (for example, a printer C compatible with color printing) shows an image or a mark indicating a device (for example, a printer D compatible with monochrome printing) with low priority. It is displayed in a noticeable shape. As still another example, as shown in FIG. 30, a character string indicating a device with high priority (for example, the handset A for overseas use) is compared with a device with low priority (for example, handsets B and C for domestic use). Are arranged and displayed at the center of the UI unit 50. As another example, as shown in FIG. 31, a character string indicating a device with high priority (for example, a printer C compatible with color printing) in a specific area 170 (a region in which priority is to be arranged) in which devices with high priority are arranged. Is displayed, and in the area other than the specific area 170, a character string indicating a device with low priority (for example, the printer D compatible with monochrome printing) is displayed. The specific area 170 may be an area specified by the user or an area set in advance. By performing the priority display as described above, the character string indicating the device with high priority can be easily viewed, and the selection of the device suitable for the function to be used becomes easy.

  The identifying unit 144 may also identify the current status of each of the telephones A and B and the printer A by referring to the device management information 140. For example, the specifying unit 144 acquires device position information of each of the telephones A and B and the printer A from the device management information 140. The identifying unit 144 also acquires user position information indicating the position of the user or the terminal device 14. The identifying unit 144 compares, for each device to be connected, the position indicated by the device position information of the device with the position indicated by the user position information, and for each device, the relative positional relationship between the user and the device. Identify In the example shown in FIG. 26, as indicated by reference numeral 164, the telephone set A is installed relatively close to the user or the terminal 14, and as indicated by reference numerals 166 and 168, the telephone set B and the printer A are shown. Is installed at a position relatively far from the user or the terminal device 14. The information indicating the relative positional relationship is transmitted from the server 134 to the terminal device 14 as information on the device group to be connected, and is displayed on the UI unit 50 of the terminal device 14. Thereby, useful information is provided to the user regarding the movement distance and the like when selecting the device to be used.

  The user position information may be transmitted to the server 134 acquired by the terminal device 14 or may be acquired by another method. For example, user position information is obtained by using the GPS function and transmitted to the server 134. As another example, the user position information may be position information registered in advance in the terminal device 14 or may be device position information of the device registered in advance in the device. For example, when the user uses the image forming system at or near the position of the device, the device position information of the device may be used as position information of the user because the position of the device can be considered to be the position of the user. Good. In this case, the specifying unit 144 acquires device identification information as user identification information from the device. The device position information may also be registered in advance in the device.

  Further, the specifying unit 144 may specify the current usage status of each of the telephones A and B and the printer A by referring to the device management information 140. For example, the specifying unit 144 acquires usage status information of each of the telephones A and B and the printer A. In the example shown in FIG. 26, as indicated by reference numerals 164 and 168, the telephone set A and the printer A are immediately available, and the telephone set B is currently unavailable. For example, the device is available when it is not used by another user or when no failure occurs. On the other hand, when used by another user or when a failure occurs, the device is not available. The usage status information indicating the current usage status is transmitted from the server 134 to the terminal device 14 as information on the device group to be connected, and displayed on the UI unit 50 of the terminal device 14. As a result, useful information is provided to the user regarding the timing of use and the like when selecting a device to be used.

  A reservation process may be performed to preferentially use the device to be connected. For example, when the user designates a use target function using the terminal device 14, the control unit 52 of the terminal device 14 makes a reservation to preferentially use the device to be connected to execute the use target function. The information is sent to the server 134. In the server 134, the control unit 142 sets a reservation of a device to be reserved, that is, a device to be connected. As an example, when there are devices that can not be used because they are currently used by other users in the device group to be connected, a reservation process for the next use may be performed. For example, when the user designates the unavailable device (for example, the telephone B) using the terminal device 14 and instructs a reservation, the control unit 52 of the terminal device 14 determines the device identification information of the designated device and the like. And transmits to the server 134 reservation information indicating a reservation for using the device next time. In the server 134, the control unit 142 sets a reservation of the device to be reserved (for example, the telephone set B). As a result, it is possible to use a device to be reserved next to the use of another user. For example, the control unit 142 issues a reservation number or the like for using the device to be reserved when it becomes available, and sets the reservation number to the device identification information of the device to be reserved in the device management information 140. Match. In the reservation state, the use of the device is permitted by using the reservation number, and the use of the device is not permitted when the reservation number is not used. Further, the information indicating the reservation number is transmitted from the server 134 to the terminal device 14 and displayed on the UI unit 50 of the terminal device 14. If it becomes possible to use the reservation target device, the user uses the reservation target device using the reservation number. For example, use of the reservation target is permitted by inputting the reservation number into the reservation target device or transmitting it to the server 134 using the terminal device 14. When a preset time elapses from the reservation start time, the reservation state is canceled, and even a user who has not made a reservation may be permitted to use. Further, when it is desired to interrupt and use a device that has already been reserved, processing of interrupt notification may be executed as in the modification according to the second embodiment.

  Further, when a plurality of users request to use the same device, as in the modification according to the second embodiment, the connection may be permitted in accordance with the execution priority, and the priority is the terminal It may be displayed on the UI unit 50 of the device 14.

  When using a device, for example, as described with reference to FIG. 21, the terminal device 14 transmits information indicating a connection request to the device to be used, and communication is established between the terminal device 14 and the device. Be done. For example, when the telephone set A and the printer A are used as devices to be cooperated, information indicating a connection request is transmitted from the terminal device 14 to the telephone set A and the printer A, whereby the terminal device 14, the telephone set A and the printer Communication is established between A and the printer A prints information indicating the contents of the conversation on the telephone A.

  As described above, according to the fourth embodiment, the terminal device 14 displays information indicating a group of devices to be connected according to the function to be used. As a result, information is provided to the user so that the group of devices capable of executing the function to be used can be identified. In addition, since the function to be used differs depending on the available function among the available apparatuses and the functions installed in the apparatus for each user, the search for the cooperation function displayed on the terminal device 14 for each user is restricted. You may limit the cooperation functions that can be executed. By doing so, for example, when there is an electronic document or the like that can not be decrypted and decrypted without executing a specific cooperation function (cooperation function using a specific function of a specific device), security is improved. It is possible to improve.

  The control unit 52 of the terminal device 14 may cause the UI unit 50 to display information on the device to be newly connected by the terminal device 14 except for the information on the device to which the terminal device 14 is already connected. . For example, in the case where the telephone set A and the printer A are used as devices to be cooperated, communication has already been established between the terminal device 14 and the telephone set A, and communication is made between the terminal device 14 and the printer A. Is not established, the control unit 52 causes the UI unit 50 to display the device identification information of the printer A without displaying the device identification information and the device management information of the telephone set A on the UI unit 50. In addition, the control unit 52 may cause the UI unit 50 to display device management information regarding the printer A. Information about the devices that have been connected and do not need to be connected is not displayed, and information about devices that have not been connected and that require connection are displayed. As compared with the case where information is also displayed, it is easier to determine whether each device to be used is a device requiring connection work.

  In addition, the control unit 52 of the terminal device 14 may cause the UI unit 50 to display information indicating connection means corresponding to the device to be connected. The connection means are the marker type AR technology, the markerless type AR technology, the position information AR technology, the network connection, etc. described above. For example, in the device management information 140, device identification information and connection means information indicating connection means suitable for the device are associated with each individual device. A device provided with a mark such as a two-dimensional barcode in which device identification information is encoded is a device suitable for a marker type AR technology, and the device identification information of the device includes marker type as connection means information. Information indicating AR technology is associated. Further, when the appearance image data of the device is generated and included in the above-mentioned appearance image correspondence information, the device is a device suitable for the markerless AR technology, and the device identification information of the device is connected to the device identification information Information indicating markerless AR technology is associated as information. Further, when the position information of the device is acquired and included in the above-described position association information, the device is a device suitable for the position information AR technology, and the device identification information of the device is the connection means information Information indicating position information AR technology is associated with each other. When the device group to be connected is identified, the identifying unit 144 of the server 134 refers to the device management information 140 to identify the connection means for each device to be connected. The information indicating the connection means is transmitted from the server 134 to the terminal device 14 and displayed on the UI unit 50 of the terminal device 14. For example, information indicating connection means is displayed for each device to be connected. A specific example will be described. When the telephone set A as a device to be connected is a device suitable for the marker type AR technology, the marker unit AR technology is used as the connection means of the telephone set A in the UI unit 50 of the terminal device 14 Information is displayed. It should be noted that the device may not be displayed if it is determined in advance that connection is not permitted to the user who requested the connection, regardless of which connection means is used. This is convenient because it is possible to know how to connect to the device to be connected.

  The first embodiment and the fourth embodiment may be combined. For example, when the UI unit 50 of the terminal device 14 displays the function group purchased by the user, that is, the function group available by the user, and a specific function is selected by the user from the function group, Information indicating a device or device group to be connected to execute the function is displayed on the UI unit 50. When a cooperation function is selected, information indicating a group of devices to be cooperated that can be executed by linking the cooperation function is displayed, and when a function that can be executed by a single device is selected, a device capable of executing that function Information is displayed.

  Each of the image forming apparatus 10, the servers 12, 80, 124, 134, the terminal device 14 and the devices 76, 78 described above is realized by cooperation of hardware resources and software as an example. Specifically, each of the image forming apparatus 10, the servers 12, 80, 124, 134, the terminal device 14 and the devices 76, 78 includes one or more processors such as a CPU (not shown). The one or a plurality of processors read and execute a program stored in a storage device (not shown) to execute each part of the image forming apparatus 10, the servers 12, 80, 124 and 134, the terminal device 14 and the devices 76 and 78 Function is realized. The program is stored in the storage device via a recording medium such as a CD or a DVD or via a communication path such as a network. Alternatively, respective units of the image forming apparatus 10, the servers 12, 80, 124, 134, the terminal device 14 and the devices 76, 78 may be realized by hardware resources such as a processor or an electronic circuit. A device such as a memory may be used in the implementation. As another example, each unit of the image forming apparatus 10, the servers 12, 80, 124, 134, the terminal device 14 and the devices 76, 78 is realized by a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array) or the like. It may be done.

  DESCRIPTION OF SYMBOLS 10 Image forming apparatus, 12, 80, 124, 134 server, 14 terminal device, 16, 26, 44, 82, 126, 136 Communication part, 18 Image forming part, 20, 28, 48, 84, 128, 138 Storage part 22, 50 user interface unit (UI unit) 24, 36, 52, 88, 130, 142 control unit, 30 device function information, 32 function purchase history information, 34 function execution unit, 38 purchase processing unit, 40 purchase history Management unit, 42, 90, 132, 144 identification unit, 46 cameras, 76, 78 devices, 80 servers, 86 cooperation function information, 140 device management information.

Claims (4)

Control means for changing the device when the device required to execute the cooperation function is specified and execution of the cooperation function fails during execution of the cooperation function,
An information processing apparatus having When changing the device, the device of the change destination is selected from devices capable of executing the cooperation function,
An information processing apparatus according to claim 1, characterized in that. When changing the device, the control means further notifies that only the device causing the failure of the cooperation function is to be changed.
The information processing apparatus according to claim 1 or 2, characterized in that: Computer,
Control means for changing the device when the device required to execute the cooperation function is specified and execution of the cooperation function fails during execution of the cooperation function,
A program to function as