JP6075503B1 - Information processing apparatus and program - Google Patents

Abstract

A plurality of devices used for a cooperation function can communicate with each other. The first device and the second device are identified by photographing the first device and the second device. Information including the address information of the first device or the second device is transmitted to the first device and the second device in the execution instruction of the cooperation function executed by linking the identified first device and the second device. The [Selection] Figure 14

Description

  The present invention relates to an information processing apparatus and a program.

  A system in which the apparatus main body and the user interface are separated may be used. For example, Patent Document 1 discloses a technique for displaying an image indicating operation guidance on an operation panel that is detachable from the apparatus main body and capable of wireless communication.

  Patent Document 2 discloses a portable terminal that generates an augmented reality image in which additional information including state information of an image forming apparatus is superimposed on a captured image captured by an imaging unit.

Patent No. 5737906 JP 2014-48672 A

  By the way, regarding the usage environment of a device such as an image forming apparatus, a usage environment in which a plurality of users use one device is generally assumed. On the other hand, a usage environment in which a plurality of users use a plurality of devices in the future can be assumed. In addition, it may be assumed that a user interface such as a touch panel is removed from the device, or the device is temporarily used at a user's destination. In such a use environment, it can be assumed that a cooperation function using a plurality of devices is executed.

  An object of the present invention is to enable a plurality of devices used for the cooperation function to communicate with each other.

  According to a first aspect of the present invention, there is provided a photographing unit that photographs the first device and the second device, an identifying unit that identifies the photographed first device and the second device, and the first unit identified by the identifying unit. Information including the address information of the first device or the second device is transmitted to the first device and the second device in the execution instruction of the cooperation function executed by linking one device with the second device. And an information processing apparatus.

  In the invention according to claim 2, the transmission means transmits information including the address information of the second device in the execution instruction to the first device, and includes the address information of the first device in the execution instruction. The information processing apparatus according to claim 1, wherein the information is transmitted to the second device.

  According to a third aspect of the present invention, the first device and the second device communicate with each other according to address information of other devices included in the information transmitted to the device, and execute the cooperation function. The information processing apparatus according to claim 1 or claim 2, wherein

  The invention according to claim 4 is the information processing apparatus according to any one of claims 1 to 3, wherein the first device and the second device have different functions.

  The invention according to claim 5 is the information processing apparatus according to any one of claims 1 to 4, wherein the cooperation function is a function that cannot be used before cooperation.

  According to a sixth aspect of the present invention, a display unit displays a first image associated with the first device and a second image associated with the second device, and the user operates the first image and the second image. 6. The information processing apparatus according to claim 1, further comprising a control unit configured to control display of the cooperation function to be executed.

  The invention according to claim 7 further includes user identification means for identifying the user, wherein the control means causes the display means to display information on the cooperation function that can be used by the identified user. An information processing apparatus according to claim 6, wherein

  According to an eighth aspect of the present invention, the identification unit identifies the first device by the identification information provided in the first device being photographed by the photographing unit, and the identification device is provided in the second device. The information processing apparatus according to claim 1, wherein the second device is identified by information being photographed by the photographing unit.

  In the invention according to claim 9, the identification unit identifies the first device by photographing the appearance of the first device by the photographing unit, and the appearance of the second device is photographed by the photographing unit. The information processing apparatus according to claim 1, wherein the second device is identified by identifying the second device.

  According to a tenth aspect of the present invention, the computer causes the first device and the second device identified by the identifying device to cooperate with the identifying device for identifying the first device and the second device photographed by the photographing device. Is a program that functions as a transmission unit that transmits information including the address information of the first device or the second device to the execution instruction of the cooperation function executed in step 1 to the first device and the second device.

  According to the first, second, third, and tenth aspects of the invention, a plurality of devices used for the cooperation function can communicate with each other.

  According to the invention which concerns on Claim 4, execution of a new function is attained by the some apparatus which has a mutually different function.

  According to the invention which concerns on Claim 5, it becomes possible to utilize the function which was not able to be utilized before cooperation.

  According to the invention which concerns on Claim 6, it becomes possible to specify the apparatus of cooperation object easily using the image tied with the apparatus.

  According to the invention which concerns on Claim 7, the information regarding the cooperation function which can be utilized by the user is provided.

  According to the invention according to claims 8 and 9, the device is identified by a simple operation.

1 is a block diagram illustrating an image forming system according to a first embodiment of the present invention. 1 is a block diagram illustrating an image forming apparatus according to a first embodiment. It is a block diagram which shows the server which concerns on 1st Embodiment. It is a block diagram which shows the terminal device which concerns on 1st Embodiment. 1 is a schematic diagram illustrating an appearance of an image forming apparatus. It is a figure which shows the function purchase screen displayed on a terminal device. It is a figure which shows the function display screen displayed on a terminal device. It is a figure which shows the function display screen displayed on a terminal device. It is a figure which shows the function display screen displayed on a terminal device. It is a sequence diagram which shows a function purchase process. It is a flowchart which shows the display process of a function display screen. It is a flowchart which shows the display process of a function display screen. It is a flowchart which shows the display process of a function display screen. It is a block diagram which shows the image forming system which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the server which concerns on 2nd Embodiment. It is a schematic diagram which shows the apparatus of cooperation object. It is a schematic diagram which shows the apparatus of cooperation object. It is a figure which shows the screen of the display part of a terminal device. It is a figure which shows the screen of the display part of a terminal device. It is a schematic diagram which shows each apparatus arrange | positioned in a search area. It is a sequence diagram which shows the process by the image forming system which concerns on 2nd Embodiment. It is a figure which shows the transition of the screen of a terminal device. It is a figure which shows the execution priority of a cooperation function. It is a block diagram which shows the server which concerns on 3rd Embodiment. It is a block diagram which shows the server which concerns on 4th Embodiment. It is a figure for demonstrating the process by the image forming system which concerns on 4th Embodiment. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of the search screen of a cooperation function. It is a figure which shows an example of a priority display. It is a figure which shows an example of a priority display. It is a figure which shows an example of a priority display. It is a figure which shows an example of a priority display.

[First Embodiment]
With reference to FIG. 1, an image forming system as an information processing system according to the first embodiment of the present invention will be described. FIG. 1 shows an example of an image forming system according to the first embodiment. The image forming system according to the first embodiment includes an image forming apparatus 10 as a device, a server 12, and a terminal device 14 as an information processing apparatus. The image forming apparatus 10, the server 12, and the terminal apparatus 14 are connected to each other via a communication path N such as a network. In the example illustrated in FIG. 1, one image forming apparatus 10, one server 12, and one terminal apparatus 14 are included in the image forming system, but a plurality of image forming apparatuses 10, a plurality of servers 12, and a plurality of terminals are included. The device 14 may be included in the image forming system.

  The image forming apparatus 10 is an apparatus having an image forming function. Specifically, the image forming apparatus 10 is an apparatus having at least one of a scan function, a print function, a copy function, and a facsimile function. Further, the image forming apparatus 10 has a function of transmitting / receiving data to / from other apparatuses.

  The server 12 is a device that manages functions available to the user for each user. For example, a function purchased by a user is a function that can be used by the user, and the server 12 manages a function purchase history for each user. Of course, there are functions that can be used free of charge, additional updater functions, functions that are specially managed by the administrator, and the like, so the server 12 not only manages functions depending on whether or not there is a purchase. The function purchase process is performed by the server 12, for example. The server 12 is a device that executes a specific function. The specific function executed by the server 12 is, for example, a function related to image processing. The functions managed by the server 12 are, for example, functions executed using the image forming apparatus 10 and functions executed by the server 12. The management of the function purchase history and the execution of the specific function may be performed by different servers 12 or may be performed by the same server 12. The server 12 has a function of transmitting / receiving data to / from other devices.

  The terminal device 14 is a device such as a PC (personal computer), a tablet PC, a smartphone, or a mobile phone, and has a function of transmitting and receiving data to and from other devices. The terminal device 14 functions as a user interface unit (UI unit) of the image forming apparatus 10 when the image forming apparatus 10 is used.

  In the image forming system according to the first embodiment, by using the terminal device 14, a function is purchased by a user, and the purchase history is managed by the server 12 as a function purchase history. The function purchased by the user is executed by, for example, the image forming apparatus 10 or the server 12.

  Hereinafter, the configuration of the image forming apparatus 10 will be described in detail with reference to FIG. FIG. 2 shows the configuration of the image forming apparatus 10.

  The communication unit 16 is a communication interface and has a function of transmitting data to another device and a function of receiving data from another device via the communication path N. The communication unit 16 may be a communication interface having a wireless communication function or a communication interface having a wired communication function.

  The image forming unit 18 executes functions related to image formation. Specifically, the image forming unit 18 executes at least one of a scan function, a print function, a copy function, and a facsimile function. By executing the scan function, the document is read and scan data (image data) is generated. By executing the print function, an image is printed on a recording medium such as paper. By executing the copy function, the document is read and printed on the recording medium. By executing the facsimile function, the image data is transmitted or received by facsimile. Further, a function combining a plurality of functions may be executed. For example, a scan transfer function that combines a scan function and a transmission function (transfer function) may be executed. By executing the scan transfer function, the original is read to generate scan data (image data), and the scan data is transmitted to a transmission destination (for example, an external device such as the terminal device 14). Of course, this combination function is only an example, and another combination function may be executed.

  The storage unit 20 is a storage device such as a hard disk. The storage unit 20 includes information (for example, job information) indicating an image formation command, image data to be printed, scan data generated by executing a scan function, various control data, various programs, Etc. are memorized. Of course, they may be stored in separate storage devices or may be stored in one storage device.

  The UI unit 22 is a user interface unit and includes a display unit and an operation unit. The display unit is a display device such as a liquid crystal display. The operation unit is an input device such as a touch panel or a keyboard. Note that the image forming apparatus 10 may not include the UI unit 22 or may include a hardware user interface unit (hardware UI unit) as hardware without including a display unit. The hardware UI unit is, for example, a hardware key (for example, a numeric keypad) specialized for numerical input, a hardware key (for example, a direction indication key) specialized for direction indication, and the like.

  The control unit 24 controls the operation of each unit of the image forming apparatus 10.

  Hereinafter, the configuration of the server 12 will be described in detail with reference to FIG. FIG. 3 shows the configuration of the server 12.

  The communication unit 26 is a communication interface and has a function of transmitting data to another device and a function of receiving data from another device via the communication path N. The communication unit 26 may be a communication interface having a wireless communication function or a communication interface having a wired communication function.

  The storage unit 28 is a storage device such as a hard disk. The storage unit 28 stores device function information 30, function purchase history information 32, a program for executing a specific function, and the like. Of course, they may be stored in separate storage devices or may be stored in one storage device. Here, the device function information 30 and the function purchase history information 32 will be described.

  The device function information 30 is information indicating a function group included in each image forming apparatus 10 included in the image forming system. For example, device identification information for identifying the image forming apparatus 10 for each image forming apparatus 10; This is information indicating correspondence with function identification information for identifying each function of the image forming apparatus 10. The device identification information is, for example, a device ID, a device name, a model number, position information, and the like. The function identification information is, for example, a function ID or a function name. For example, when an image forming apparatus 10 has a scan function, a print function, a copy function, and a scan transfer function, the device identification information of the image forming apparatus 10 includes function identification information indicating a scan function and a print function. The function identification information indicating the function, the function identification information indicating the copy function, and the function identification information indicating the scan transfer function are associated with each other. By referring to the device function information 30, a function group included in each image forming apparatus 10 is specified.

  The function purchase history information 32 is information indicating a function purchase history by each user, that is, information indicating a function purchased by each user. For example, user identification information for identifying a user for each user, It is information which shows matching with the function identification information which shows the function purchased by the said user. The user identification information is user account information such as a user ID and a name, for example. The function purchased by the user is a function that can be used by the user. By referring to the function purchase history information 32, a function purchased by each user, that is, a function that can be used by the user is specified for each user. The function purchase history information 32 is updated every time a user purchases a function, for example.

  The function execution unit 34 executes a specific function. For example, when the user designates a specific function using the terminal device 14 and instructs the execution of the function, the function execution unit 34 executes the function designated by the user. The function execution unit 34 executes functions related to image processing such as a character recognition function, a translation function, an image processing function, and an image forming function. Of course, the function execution unit 34 may execute a function related to processing other than image processing. By executing the character recognition function, a character in the image is recognized and character data indicating the character is generated. By executing the translation function, the characters in the image are translated into characters expressed in a specific language, and character data indicating the translated characters is generated. The image is processed by executing the image processing function. For example, the function execution unit 34 receives scan data generated by executing the scan function from the image forming apparatus 10, and targets the scan data as an image such as a character recognition function, a translation function, and an image processing function. Perform processing functions. The function execution unit 34 may receive image data from the terminal device 14 and execute each function for the image data. The character data and image data generated by the function execution unit 34 are transmitted from the server 12 to the terminal device 14, for example.

  The control unit 36 controls the operation of each unit of the server 12. The control unit 36 includes a purchase processing unit 38, a purchase history management unit 40, and a specifying unit 42.

  The purchase processing unit 38 executes a function purchase process. For example, when a paid function is purchased by a user, the purchase processing unit 38 applies a charging process to the user. Functions purchased by the user can be used by the user, and functions not purchased by the user cannot be used by the user.

  The purchase history management unit 40 manages the purchase history of the function by the user for each user, and generates the function purchase history information 32 indicating the purchase history. For example, the purchase history management unit 40 updates the function purchase history information 32 every time a function is purchased by the user. The information included in the function purchase history information 32 is displayed on the terminal device 14 as a function purchase screen, for example, when the user purchases a function or confirms a purchased function. The function purchase screen will be described in detail later with reference to FIG.

  The specifying unit 42 receives device identification information for identifying the image forming apparatus 10 to be used, and the function of each function associated with the device identification information in the device function information 30 stored in the storage unit 28. Identify identification information. Thereby, a function group included in the image forming apparatus 10 to be used is specified (recognized). For example, device identification information is transmitted from the terminal device 14 to the server 12, and the function identification information of each function associated with the device identification information is specified by the specifying unit 42. The function identification information (for example, information indicating the name of each function) of each function is transmitted from the server 12 to the terminal device 14 and displayed on the terminal device 14, for example. Thereby, the function identification information of each function of the image forming apparatus 10 specified by the device identification information is displayed on the terminal device 14.

  Further, the specifying unit 42 receives user identification information for identifying the user, and obtains function identification information of each function associated with the user identification information in the function purchase history information 32 stored in the storage unit 28. Identify. Thus, a function group purchased by the user, that is, a function group that can be used by the user is specified (recognized). For example, user identification information is transmitted from the terminal device 14 to the server 12, and the function identification information of each function associated with the user identification information is specified by the specifying unit 42. The function identification information (for example, information indicating the name of each function) of each function is transmitted from the server 12 to the terminal device 14 and displayed on the terminal device 14, for example. Thereby, the function identification information of each function that can be used by the user specified by the user identification information is displayed on the terminal device 14.

  For example, the specifying unit 42 receives the device identification information and the user identification information, specifies the function identification information of each function associated with the device identification information in the device function information 30, and the function purchase history information 32 The function identification information of each function associated with the user identification information is specified. As a result, a function group included in the image forming apparatus 10 specified by the device identification information and usable by the user specified by the user identification information is specified (recognized). For example, function identification information of each function that the image forming apparatus 10 has and that can be used by the user is transmitted from the server 12 to the terminal device 14 and displayed on the terminal device 14. Thereby, function identification information of each function that the image forming apparatus 10 has and that can be used by the user is displayed on the terminal device 14.

  The function identification information of each function included in the image forming apparatus 10 to be used and the function identification information of each function that can be used by the user are displayed on the terminal device 14 as a function display screen, for example. The function display screen will be described in detail later with reference to FIG.

  In this embodiment, as an example, by applying AR (Augmented Reality) technology (augmented reality technology), device identification information is acquired and the image forming apparatus 10 to be used is specified (recognized). A known AR technique is used as the AR technique. For example, a marker type AR technique using a marker such as a two-dimensional barcode, a markerless type AR technique using an image recognition technique, a position information AR technique using position information, and the like are used. Of course, the device identification information may be acquired without applying the AR technology, and the image forming apparatus 10 to be used may be specified.

  Hereinafter, the configuration of the terminal device 14 will be described in detail with reference to FIG. FIG. 4 shows the configuration of the terminal device 14.

  The communication unit 44 is a communication interface, and has a function of transmitting data to another device and a function of receiving data from another device via the communication path N. The communication unit 44 may be a communication interface having a wireless communication function or a communication interface having a wired communication function. A camera 46 serving as an imaging unit generates image data (for example, still image data or moving image data) by imaging an imaging target. The storage unit 48 is a storage device such as a hard disk or an SSD, and various programs, various data, address information of the server 12, address information of each device (for example, address information of each image forming apparatus 10), and the identified link target. Stores information on devices, information on linkage functions, and the like. The UI unit 50 is a user interface unit and includes a display unit and an operation unit. The display unit is a display device such as a liquid crystal display. The operation unit is, for example, an input device such as a touch panel, a keyboard, and a mouse. The control unit 52 controls the operation of each unit of the terminal device 14. The control unit 52 functions as a display control unit, for example, and displays a function purchase screen or a function display screen on the display unit of the UI unit 50.

  The device function information 30 described above may be stored in the storage unit 48 of the terminal device 14. In this case, the device function information 30 may not be stored in the storage unit 28 of the server 12. Similarly, the function purchase history information 32 described above may be stored in the storage unit 48 of the terminal device 14. In this case, the function purchase history information 32 may not be stored in the storage unit 28 of the server 12. In addition, the control unit 52 of the terminal device 14 may include the purchase history management unit 40 described above, and may manage a function purchase history by a user who uses the terminal device 14. In this case, the server 12 may not include the purchase history management unit 40. The control unit 52 of the terminal device 14 includes the specifying unit 42 described above, specifies the image forming apparatus 10 based on the device identification information, and specifies functions that can be used by the user based on the user identification information. May be. In this case, the server 12 may not include the specifying unit 42.

  Hereinafter, the device identification information acquisition process of the image forming apparatus 10 will be described in detail with reference to FIG. FIG. 5 schematically shows the appearance of the image forming apparatus 10. Here, a process for acquiring the device identification information by applying the marker AR technology will be described. The casing of the image forming apparatus 10 is provided with a marker 54 such as a two-dimensional barcode. The marker 54 is information in which device identification information of the image forming apparatus 10 is encoded. The user activates the camera 46 of the terminal device 14 and photographs the marker 54 provided on the image forming apparatus 10 to be used by the camera 46. Thereby, image data representing the marker 54 is generated. The image data is transmitted from the terminal device 14 to the server 12, for example. In the server 12, the control unit 36 extracts device identification information by applying a decoding process to the marker image represented in the image data. Thereby, the image forming apparatus 10 to be used (the image forming apparatus 10 having the photographed marker 54) is specified (recognized). The specifying unit 42 of the server 12 specifies function identification information of each function associated with the extracted device identification information in the device function information 30. As a result, the functions of the image forming apparatus 10 to be used are specified.

  The control unit 52 of the terminal device 14 may extract the device identification information by applying a decoding process to the image data representing the marker 54. In this case, the extracted device identification information is transmitted from the terminal device 14 to the server 12. The specifying unit 42 of the server 12 specifies function identification information of each function associated with the device identification information transmitted from the terminal device 14 in the device function information 30. When the device function information 30 is stored in the storage unit 48 of the terminal device 14, the control unit 52 of the terminal device 14 is associated with the device identification information extracted by the control unit 52 in the device function information 30. The function identification information of each function may be specified.

  The marker 54 may include coded function identification information of each function of the image forming apparatus 10. In this case, by applying the decoding process to the image data representing the marker 54, the device identification information of the image forming apparatus 10 is extracted, and the function identification information of each function of the image forming apparatus 10 is also extracted. . As a result, the image forming apparatus 10 is specified and each function of the image forming apparatus 10 is specified. This decoding process may be performed by the server 12 or the terminal device 14.

  When acquiring the device identification information by applying the markerless AR technology, for example, the user captures all or part of the appearance of the image forming apparatus 10 to be used by the camera 46 of the terminal device 14. Of course, it is useful to obtain information for identifying the device such as the name of the device to be used (for example, product name) and the model number from the appearance. Appearance image data representing all or part of the appearance of the image forming apparatus 10 to be used is generated by photographing. The appearance image data is transmitted from the terminal device 14 to the server 12, for example. In the server 12, the control unit 36 specifies the image forming apparatus 10 to be used based on the appearance image data. For example, the storage unit 28 of the server 12 associates, for each image forming apparatus 10, appearance image data representing all or part of the appearance of the image forming apparatus 10 with device identification information of the image forming apparatus 10. Is stored. For example, the control unit 36 compares the appearance image data transmitted from the terminal device 14 with each appearance image data included in the appearance image association information, and, based on the comparison result, the image forming apparatus 10 to be used. Specify device identification information. For example, the control unit 36 extracts the appearance features of the image forming apparatus 10 to be used from the appearance image data transmitted from the terminal device 14, and the appearance information is included in the appearance image data group included in the appearance image association information. Appearance image data representing a feature that is the same as or similar to the feature is specified, and device identification information associated with the appearance image data is specified. Thereby, the image forming apparatus 10 to be used (the image forming apparatus 10 photographed by the camera 46) is specified (recognized). As another example, when a name (for example, a product name) or a model number of the image forming apparatus 10 is photographed and appearance image data representing the name or the model number is generated, the name or model represented in the appearance image data is displayed. Based on the number, the image forming apparatus 10 to be used may be specified. The specifying unit 42 of the server 12 specifies function identification information of each function associated with the specified device identification information in the device function information 30. Thereby, the function of the image forming apparatus 10 to be used is specified (recognized).

  The control unit 52 of the terminal device 14 compares the appearance image data representing all or part of the appearance of the image forming apparatus 10 to be used with each appearance image data included in the appearance image association information, and compares the comparison. Based on the result, the device identification information of the image forming apparatus 10 to be used may be specified. The appearance image association information may be stored in the storage unit 48 of the terminal device 14. In this case, the control unit 52 of the terminal device 14 specifies the device identification information of the image forming apparatus 10 to be used by referring to the appearance image association information stored in the storage unit 48 of the terminal device 14. Alternatively, the control unit 52 of the terminal device 14 acquires the external image association information from the server 12 and specifies the device identification information of the image forming apparatus 10 to be used by referring to the external image association information. Also good.

  When the device identification information is acquired by applying the position information AR technology, for example, position information indicating the position where the image forming apparatus 10 is installed is acquired by using a GPS (Global Positioning System) function. For example, each image forming apparatus 10 has a GPS function, and acquires device position information indicating the position of the image forming apparatus 10 itself. The terminal device 14 outputs information indicating an acquisition request for device position information to the image forming apparatus 10 to be used, and device position information of the image forming apparatus 10 from the image forming apparatus 10 as a response to the acquisition request. Receive. The device position information is transmitted from the terminal device 14 to the server 12, for example. In the server 12, the control unit 36 specifies the image forming apparatus 10 to be used based on the device position information. For example, the storage unit 28 of the server 12 associates, for each image forming apparatus 10, device position information indicating the position where the image forming apparatus 10 is installed with device identification information of the image forming apparatus 10. The position association information shown is stored. The control unit 36 specifies device identification information associated with the device position information transmitted from the terminal device 14 in the position association information. Thereby, the image forming apparatus 10 to be used is specified (recognized). The specifying unit 42 of the server 12 specifies function identification information of each function associated with the specified device identification information in the device function information 30. Thereby, the function of the image forming apparatus 10 to be used is specified (recognized).

  Note that the control unit 52 of the terminal device 14 may specify the device identification information associated with the position information of the image forming apparatus 10 to be used in the position association information. The position association information may be stored in the storage unit 48 of the terminal device 14. In this case, the control unit 52 of the terminal device 14 specifies the device identification information of the image forming apparatus 10 to be used by referring to the position association information stored in the storage unit 48 of the terminal device 14. Alternatively, the control unit 52 of the terminal device 14 may acquire the position association information from the server 12 and specify the device identification information of the image forming apparatus 10 to be used by referring to the position association information. .

  Hereinafter, the screen displayed on the terminal device 14 will be described in detail. First, with reference to FIG. 6, a function purchase screen displayed when purchasing a function or confirming a purchased function will be described. FIG. 6 shows an example of the function purchase screen.

  For example, when a user accesses the server 12 using the terminal device 14, user identification information (user account information) of the user is transmitted from the terminal device 14 to the server 12. In the server 12, the specifying unit 42 specifies function identification information of each function associated with the user identification information in the function purchase history information 32. Thereby, a function group purchased by the user, that is, a function group that can be used by the user is specified (recognized). For example, function purchase screen information including function identification information indicating each function to be sold and function identification information indicating each function usable by the user is transmitted from the server 12 to the terminal device 14. . The control unit 52 of the terminal device 14 displays a function purchase screen based on the function purchase screen information on the display unit of the UI unit 50 of the terminal device 14. For example, the control unit 52 of the terminal device 14 displays each function identification information on the display unit of the UI unit 50 and displays information indicating the purchase status of each function on the display unit.

  In the function purchase screens 56 and 58 shown in FIG. 6A and FIG. 6B, a list of information indicating the functions to be sold is displayed. Purchase status information indicating purchase is displayed in association with each other. A function associated with purchase status information indicating purchase is a function purchased by a user, that is, a function that can be used by the user. The function associated with purchase status information indicating unpurchased is a function that has not been purchased by the user, that is, a function that cannot be used by the user (a function that is prohibited from being used).

  In the example shown in FIG. 6A, the function purchase screen 56 is a screen showing a function purchase history for the user A. As an example, the function purchase screen 56 displays a list of function purchase histories. Functions A and C are functions that have been purchased by user A and can be used by user A. Functions B, D, and E are not purchased by user A and cannot be used by user A. A function is purchased via the function purchase screen 56. For example, when the user A designates an unpurchased function B using the terminal device 14 and gives a purchase instruction, the function identification information indicating the function B and the information indicating the purchase instruction are transmitted from the terminal device 14 to the server 12. Is done. In the server 12, the purchase processing unit 38 executes the function B purchase process. When the function B is charged, the purchase processing unit 38 performs a charging process. Further, the purchase history management unit 40 updates the function purchase history information for the user A. That is, the purchase history management unit 40 associates the function identification information indicating the function B with the user identification information of the user A in the function purchase history information. Thereby, the use of the function B by the user A becomes possible. On the function purchase screen 56, the purchase status of the function B is changed from “not purchased” to “purchase”. Note that a corresponding device for each function may be displayed. By doing so, the device corresponding to the function that the user wants to use can be easily understood. For example, the functions A, B, and C are associated with a device α that can execute the functions A, B, and C, and information indicating the device α is displayed in association with the functions A, B, and C. ing. Further, the functions D and E are associated with a device β capable of executing the functions D and E, and information indicating the device β is displayed in association with the functions D and E. The description of the executable device may be displayed by the name of the device group, or may be a display that lists each device. Further, as in the function purchase screen 58 shown in FIG. 6B, the function and the device that can execute the function may be associated with each other and the executable device may be displayed in another column. For example, the models of devices that can execute the function A are the models a, b, c, and d, and the models of devices that can execute the function B are the model Z group. The model Z group includes models a, b, e, and f.

  For example, a Web browser program is stored in the terminal device 14, and access from the terminal device 14 to the server 12 is realized by using the Web browser. When the user accesses the server 12 using a Web browser, a Web page representing the function purchase screen 56 or the function purchase screen 58 is displayed on the display unit of the UI unit 50 of the terminal device 14, and functions via the Web page are displayed. Is purchased.

  Next, the function display screen will be described in detail with reference to FIG. The function display screen is displayed on the display unit of the UI unit 50 of the terminal device 14 when the image forming apparatus 10 is used. FIG. 7 shows an example of a function display screen.

  For example, by using any one of the above-described marker type AR technology, markerless type AR technology, or position information AR technology, device identification information of the image forming apparatus 10 to be used is acquired, and the device identification information is included in the device identification information. Function identification information indicating each associated function, that is, function identification information indicating each function included in the image forming apparatus 10 to be used is specified (recognized). Further, function identification information indicating each function associated with user identification information of a target user who uses the target image forming apparatus 10, that is, function identification information indicating each function usable by the target user is specified. (Recognized). Such information is displayed on the display unit of the UI unit 50 of the terminal device 14 as a function display screen. Further, since the function group of the image forming apparatus 10 to be used is specified, the function group that the image forming apparatus 10 to be used does not have is specified from the function groups to be sold. Function identification information indicating each function that the image forming apparatus 10 to be used does not have may be displayed on the function display screen.

  On the function display screen 60 shown in FIG. 7, a button image 62 representing the function A, a button image 64 representing the function B, and a button image 66 representing the function C are displayed as an example of the function identification information. ing. The function A is a function that the image forming apparatus 10 to be used has, and is a function that can be used by the target user, that is, a function purchased by the target user. The function B is a function that the image forming apparatus 10 to be used has and is a function that cannot be used by the target user, that is, a function that is not purchased by the target user. When the target user purchases the function B, the function B can be used by the target user. The function C is a function that the use target image forming apparatus 10 does not have, that is, a function that the use target image forming apparatus 10 does not support. The control unit 52 of the terminal device 14 changes the display mode of the button image according to whether or not the function represented by the button image is a function that the image forming apparatus 10 to be used has. Further, the control unit 52 changes the display mode of the button image according to whether or not the function represented by the button image is a function that can be used by the target user. For example, the control unit 52 changes the color and shape of the button image. In the example illustrated in FIG. 7, the control unit 52 displays the button images 62, 64, 66 on the display unit while distinguishing them from each other. For example, the control unit 52 changes the colors of the button images 62, 64, and 66. As an example, a button image (for example, the button image 62 representing the function A) representing the function that the image forming apparatus 10 to be used has and that can be used by the target user is displayed in blue. A button image (a button image 64 representing the function B) representing a function that the target image forming apparatus 10 has and cannot be used by the target user is displayed in yellow. A button image representing a function that the image forming apparatus 10 to be used does not have (for example, a button image 66 representing the function C) is displayed in gray. As another example, the control unit 52 may change the shape of the button images 62, 64, 66, or may change the font of the function display name. Of course, the display mode may be changed by another method. This informs the user whether or not each function can be used with high visibility.

  For example, when the target user designates the button image 62 representing the function A using the terminal device 14 and gives an instruction to execute the function A, execution instruction information indicating the execution instruction of the function A is displayed from the terminal device 14 as an image. It is transmitted to the forming apparatus 10. The execution instruction information includes control data for executing function A, data such as image data to which processing by function A is applied, and the like. When receiving the execution instruction information, the image forming apparatus 10 executes the function A according to the execution instruction information. For example, when the function A is a scan transfer function, the scan function is executed by the image forming unit 18 of the image forming apparatus 10, thereby generating scan data (image data). The scan data is transmitted from the image forming apparatus 10 to a transmission destination (for example, the terminal device 14) set. When the function A is a function realized by the cooperation of the image forming apparatus 10 and the server 12, a part of the function A is executed by the image forming apparatus 10, and other functions of the function A are performed. It is executed by the server 12. For example, the scan function is executed by the image forming unit 18 of the image forming apparatus 10 to generate scan data, the scan data is transmitted from the image forming apparatus 10 to the server 12, and the function executing unit 34 of the server 12 performs character By executing the recognition function, character data is extracted from the scan data. The character data is transmitted from the server 12 to a transmission destination (for example, the terminal device 14) set.

  When the target user designates the button image 64 representing the function B using the terminal device 14 and instructs the purchase of the function B, the terminal device 14 accesses the server 12. Accordingly, a screen (for example, a website) for purchasing the function B is displayed on the UI unit 50 of the terminal device 14 as information for enabling the target user to use the function B. By performing the purchase procedure on this screen, the use of the function B by the target user is permitted, and the function B is executed when the target user instructs the execution of the function B. As another example, the UI unit 50 includes a use permission request screen (for example, a website) for requesting the administrator or the like to use the function B as information for enabling the target user to use the function B. May be displayed. When the administrator or the like is requested to permit the use of the function B via the use permission request screen and the permission is obtained, the function B can be used by the target user.

  Another display mode may be adopted as the display mode of the function display screen. For example, the housing of the image forming apparatus 10 is provided with an installation location where the terminal device 14 is installed, and the function screen display mode (in accordance with the installation mode of the terminal device 14 installed at the installation location ( (Display design) may be changed. For example, the housing of the image forming apparatus 10 has a shape corresponding to the shape of the terminal device 14 and is formed with a recess that is used as the installation location of the terminal device 14. The shape of the dent is vertically long or horizontally long. By installing the terminal device 14 in the vertically long depression, the terminal device 14 is arranged vertically with respect to the casing of the image forming apparatus 10, and by installing the terminal device 14 in the horizontally long depression, the terminal device 14 is arranged. Is arranged beside the housing of the image forming apparatus 10. The display mode of the function display screen is changed according to the arrangement state.

  FIG. 8 shows a function display screen 68 when the terminal device 14 is arranged vertically with respect to the casing of the image forming apparatus 10, and FIG. 9 shows the terminal device 14 of the image forming apparatus 10. A function display screen 72 is shown when it is arranged horizontally with respect to the housing.

  In the case of the vertical arrangement, as shown in FIG. 8, the control unit 52 of the terminal device 14 arranges the button images 62, 64, 66 in the vertical direction and displays them on the display unit of the UI unit 50. That is, the control unit 52 arranges the button images 62, 64, 66 along the longitudinal direction of the terminal device 14 arranged vertically and displays it on the display unit of the UI unit 50. In addition, the control unit 52 may display the strip-shaped images 70 provided along the longitudinal direction of the terminal device 14 on both sides in the longitudinal direction of the function display screen 68.

  In the case of the horizontal arrangement, as shown in FIG. 9, the control unit 52 of the terminal device 14 arranges the button images 62, 64, 66 on the horizontal side and displays them on the display unit of the UI unit 50. That is, the control unit 52 arranges the button images 62, 64, 66 along the longitudinal direction of the terminal device 14 arranged horizontally and displays the button images 62, 64, 66 on the display unit of the UI unit 50. In addition, the control unit 52 may display a strip-shaped image 74 provided along the longitudinal direction of the terminal device 14 on both sides in the longitudinal direction of the function display screen 72. The image 74 has a different color and design from the image 70.

  As described above, by changing the display mode (display design) of the function display screen according to the installation mode of the terminal device 14, the information displayed on the function display screen is easier to see than when the display mode is fixed. Can be.

  Hereinafter, processing by the image forming system according to the first embodiment will be described in detail. First, the function purchase process will be described with reference to FIG. FIG. 10 is a sequence diagram showing the function purchase process.

  First, a target user who desires to purchase a function uses the terminal device 14 to instruct activation of an application (program) for function purchase processing. The control unit 52 of the terminal device 14 activates the application according to the instruction (S01). The application may be stored in advance in the storage unit 48 of the terminal device 14, or may be downloaded from the server 12 or the like.

  Next, the control unit 52 of the terminal device 14 reads the user account information (user identification information) of the target user (S02). For example, the user account information is stored in advance in the storage unit 48 of the terminal device 14, and the control unit 52 of the terminal device 14 functions as an example of a user identification unit, and stores the user account information of the target user from the storage unit 48. Read to identify the target user. When the user account information of a plurality of users is stored in the storage unit 48, the target user uses the terminal device 14 to specify his / her user account information. Thereby, the user account information of the target user is read and the target user is identified. Alternatively, the control unit 52 may identify the target user by reading the user account information of the user who has logged into the terminal device 14. As another example, when only one user account information is stored in the same terminal device 14, the control unit 52 may identify the target user by reading the user account information. If no user account is set and user account information is not created, initial setting is performed, thereby creating user account information.

  Next, the terminal device 14 accesses the server 12 via the communication path N (S03). At this time, the terminal device 14 transmits the user account information (user identification information) of the target user to the server 12.

  In the server 12, the specifying unit 42 reads the function purchase history of the target user having the user account information (S04). Specifically, the specifying unit 42 specifies the function identification information of each function associated with the user account information (user identification information) in the function purchase history information 32 stored in the storage unit 28 of the server 12. To do. Thereby, the function group purchased by the target user, that is, the function group that can be used by the user is specified.

  Next, the server 12 includes function identification information that indicates each function that is to be sold, and function identification information that indicates each function that can be used by the target user (function identification information that indicates each function purchased by the target user. ) Is sent to the terminal device 14 via the communication path N (S05).

  In the terminal device 14, the control unit 52 displays a function purchase screen based on the function purchase screen information transmitted from the server 12 on the display unit of the UI unit 50 of the terminal device 14 (S 06). For example, the function purchase screen 56 shown in FIG. 6A or the function purchase screen 58 shown in FIG. 6B is displayed. The function purchase screens 56 and 58 may display information indicating the setting contents of the purchased function.

  The target user uses the terminal device 14 to select a function to be purchased on the function purchase screen 56 (S07). Alternatively, the target user may change the setting contents of the purchased function on the function purchase screen 56. For example, the target user uses the terminal device 14 to select a function to be changed, and changes the setting content of the function.

  When the purchase target function is selected by the target user, the control unit 52 of the terminal device 14 displays a confirmation screen on the display unit of the UI unit 50 (S08). When a purchase instruction is given by the target user on the confirmation screen, the terminal device 14 transmits purchase instruction information indicating the purchase instruction to the server 12 via the communication path N (S09). The purchase instruction information includes function identification information indicating the function to be purchased. Note that the display of the confirmation screen may be omitted. In this case, when a purchase instruction is given after the purchase target function is selected in step S07, the purchase instruction information is transmitted from the terminal device 14 to the server 12. When the setting content of the function is changed by the target user, the terminal device 14 transmits information indicating the changed setting content to the server 12 via the communication path N.

  In the server 12, a purchase process is executed (S10). When the function to be purchased is charged, the purchase processing unit 38 performs billing processing. Further, the purchase history management unit 40 updates the function purchase history information 32 for the target user. That is, in the function purchase history information 32, the purchase history management unit 40 associates the function identification information indicating the purchase target function with the user identification information (user account information) of the target user. Thereby, the use of the function to be purchased is permitted. When the setting content of the function is changed by the target user, the purchase history management unit 40 changes the setting content of the function.

  When the purchase process is completed, the server 12 transmits purchase completion information indicating that the purchase process is completed to the terminal device 14 via the communication path N (S11). As a result, information indicating that the purchase procedure has been completed is displayed on the display unit of the UI unit 50 of the terminal device 14 (S12). Next, function identification information indicating the functions that have become available through the current purchase is displayed on the display unit of the UI unit 50 of the terminal device 14 (S13). Alternatively, the function purchase screen is displayed on the display unit of the UI unit 50, and the function purchase screen can be used from the display mode indicating that the function that has become available by the current purchase cannot be used. The display mode is changed to indicate that For example, the color or shape of the button image indicating the function is changed. When the setting content of the function is changed, the server 12 transmits procedure completion information indicating that the change process is completed to the terminal device 14 via the communication path N. As a result, information indicating that the change procedure has been completed is displayed on the display unit of the UI unit 50 of the terminal device 14.

  Hereinafter, the display process of the function display screen will be described with reference to FIG. FIG. 11 is a flowchart showing the processing. As an example, a case where the image forming apparatus 10 is recognized using the marker AR technology will be described.

  First, a target user who desires to display a function display screen uses the terminal device 14 to instruct activation of an application (program) for display processing of the function display screen. The control unit 52 of the terminal device 14 activates the application according to the instruction (S20). The application may be stored in advance in the storage unit 48 of the terminal device 14, or may be downloaded from the server 12 or the like.

  Next, the control unit 52 of the terminal device 14 reads the user account information (user identification information) of the target user (S21). This reading process is the same as the process in step S02 described above.

  Next, the target user instructs activation of the camera 46 using the terminal device 14. The control unit 52 of the terminal device 14 activates the camera 46 according to the instruction (S22). The target user uses the camera 46 to photograph the marker 54 provided in the image forming apparatus 10 to be used (S23). Thereby, image data representing the marker 54 is generated.

  Next, a function group included in the image forming apparatus 10 to be used is specified (S24). For example, image data representing the marker 54 is transmitted from the terminal device 14 to the server 12, and decoding processing is applied to the image data in the server 12. Thereby, device identification information indicating the image forming apparatus 10 to be used is extracted. In addition, after the device identification information is extracted by the terminal device 14, the usable function group is displayed on the UI unit 50 without additionally receiving an operation input for specifying a device (image forming apparatus 10) to be used by the user. be able to. This simplifies the work step of registering the device to be used by the user's operation input, and shortens the set time. As another example, a decoding process may be applied to image data in the terminal device 14, and thereby device identification information may be extracted. In this case, the device identification information extracted by the terminal device 14 is transmitted from the terminal device 14 to the server 12. In the server 12, the specifying unit 42 specifies function identification information of each function associated with the device identification information in the device function information 30. Thereby, a function group included in the image forming apparatus 10 to be used is specified (recognized).

  In addition, a function group that can be used by the target user is specified (S25). For example, user account information (user identification information) of the target user is transmitted from the terminal device 14 to the server 12. In the server 12, the specifying unit 42 specifies function identification information of each function associated with the user account information in the function purchase history information 32. Thereby, a function group purchased by the target user, that is, a function group that can be used by the target user is specified (recognized).

  In addition, the process of step S24 and step S25 may be performed simultaneously, and the process of step S25 may be performed before step S24.

  In the server 12, the control unit 36 generates function display screen information indicating a function display screen for displaying a function group of the image forming apparatus 10 to be used and a function group that can be used by the target user. The function display screen information is transmitted from the server 12 to the terminal device 14. Thereby, a function display screen is displayed on the display unit of the UI unit 50 of the terminal device 14 (S26). On the function display screen, function identification information of each function included in the image forming apparatus 10 to be used and function identification information of each function that can be used by the target user are displayed. In addition, function identification information indicating each function that is a sale target and that the use target image forming apparatus 10 does not have may be displayed on the function display screen. For example, the function display screen 60 shown in FIG. 7 is displayed on the display unit of the UI unit 50.

  When an unpurchased function is selected by the target user and a purchase instruction is given on the function display screen 60 (S27, Yes), purchase processing for the selected function is executed (S28). Thereby, the purchased function can be used. If no purchase instruction is given (S27, No), the process proceeds to step S29.

  When the target user selects a function (purchased function) that can be used by the target user and is provided by the target user (S29, Yes), the function selected by the target image forming apparatus 10 is selected. Is executed (S30). When the selected function is executed by the image forming apparatus 10, execution instruction information indicating an instruction to execute the function is transmitted from the terminal device 14 to the image forming apparatus 10, and the function is executed by the image forming apparatus 10. . Further, when the selected function is realized by the cooperation of the image forming apparatus 10 and the server 12, a part of the selected function is executed by the image forming apparatus 10, and other functions of the selected function are executed. The part is executed by the server 12. At this time, control data, data to be processed, and the like are transmitted and received among the image forming apparatus 10, the server 12, and the terminal apparatus 14 in order to execute the selected function.

  When the function execution instruction is not given by the target user (S29, No), the process returns to step S27.

  Hereinafter, another display process of the function display screen will be described with reference to FIG. FIG. 12 is a flowchart showing the processing. As an example, a case where the image forming apparatus 10 is recognized using the markerless AR technology will be described.

  First, in the terminal device 14, an application for display processing of the function display screen is activated (S40), user account information (user identification information) of a target user who desires to display the function display screen is read (S41), and the camera 46 starts (S42).

  Next, the target user photographs all or part of the appearance of the image forming apparatus 10 to be used by the camera 46 (S43). Thereby, appearance image data representing all or part of the appearance of the image forming apparatus 10 to be used is generated.

  Next, the image forming apparatus 10 to be used is specified (S44). For example, the appearance image data is transmitted from the terminal device 14 to the server 12. In the server 12, the appearance image data of each image forming apparatus 10 included in the appearance image association information and the appearance image data transmitted from the terminal device 14. And the device identification information of the image forming apparatus 10 to be used is specified.

  As a result of the above comparison processing, when a plurality of image forming apparatuses 10 are not specified and one image forming apparatus 10 is specified (S45, No), the process proceeds to step S24 shown in FIG. .

  On the other hand, if a plurality of image forming apparatuses 10 are specified as a result of the comparison process (S45, Yes), the target user selects the image forming apparatus 10 to be used from among the plurality of image forming apparatuses 10. (S46). For example, the identified device identification information of each image forming apparatus 10 is transmitted from the server 12 to the terminal apparatus 14 and displayed on the UI unit 50 of the terminal apparatus 14. The target user uses the terminal device 14 to select device identification information of the image forming apparatus 10 to be used from among a plurality of device identification information. The device identification information selected by the target user is transmitted from the terminal device 14 to the server 12. Next, the process proceeds to step S24 shown in FIG.

  Since the process after step S24 is the same as the content demonstrated with reference to FIG. 11, the description is abbreviate | omitted.

  Hereinafter, another display process of the function display screen will be described with reference to FIG. FIG. 13 is a flowchart showing the processing. As an example, a case where the image forming apparatus 10 is recognized using the position information AR technique will be described.

  First, in the terminal device 14, an application for displaying the function display screen is activated (S50), and the user account information (user identification information) of the target user who desires to display the function display screen is read (S51).

  Next, the terminal device 14 acquires position information of the image forming apparatus 10 to be used (S52). For example, each image forming apparatus 10 has a GPS function, and acquires position information indicating the position of the image forming apparatus 10 itself. The terminal device 14 transmits information indicating an acquisition request for position information to the image forming apparatus 10 to be used, and receives position information of the image forming apparatus 10 from the image forming apparatus 10 as a response.

  Next, the image forming apparatus 10 to be used is specified (S53). For example, the position information of the image forming apparatus 10 to be used is transmitted from the terminal apparatus 14 to the server 12, and the server 12 transmits the position information of each image forming apparatus 10 included in the position association information and the terminal apparatus 14. The device identification information of the image forming apparatus 10 to be used is specified by comparing the position information thus obtained.

  As a result of the above comparison processing, when a plurality of image forming apparatuses 10 are not specified and one image forming apparatus 10 is specified (S54, No), the process proceeds to step S24 shown in FIG. .

  On the other hand, when a plurality of image forming apparatuses 10 are specified as a result of the above comparison processing (S54, Yes), the target image forming apparatus 10 is selected from the plurality of image forming apparatuses 10 by the target user. (S55). The device identification information of the image forming apparatus 10 selected by the target user is transmitted from the terminal device 14 to the server 12. Next, the process proceeds to step S24 shown in FIG.

  Since the process after step S24 is the same as the content demonstrated with reference to FIG. 11, the description is abbreviate | omitted.

  As described above, according to the first embodiment, the target image forming apparatus 10 is specified by applying the AR technology, and the function identification information indicating the function group of the image forming apparatus 10 is used by the target user. Function identification information indicating a possible function group is displayed on the terminal device 14. Thus, even if the function of the image forming apparatus 10 to be used is unknown from the appearance, what function the image forming apparatus 10 to be used has in the target user. This makes it easy to understand whether the image forming apparatus 10 has a function that can be used by the target user.

  Further, according to the first embodiment, for example, in an environment where a plurality of users use a plurality of devices (for example, a plurality of image forming apparatuses 10), information on functions is provided for each user for each terminal device 14 of each user. Is displayed properly. For example, even when a user interface such as a touch panel is removed from a device such as the image forming apparatus 10, the terminal device 14 is used as the user interface, and information regarding functions according to each user is stored in each user's terminal. Appropriately displayed on the device 14. As another scene, for example, when a device is temporarily used at a user's destination, a user interface suitable for the user, that is, a user interface representing a function that can be used by the user is displayed on the terminal device 14. Realized.

  In the examples shown in FIGS. 11, 12, and 13, the user target information (image forming apparatus 10) is identified after the user account information is read and the user is identified. After the device 10) is identified, user account information may be read to identify the user. When the marker type AR technology or the markerless AR technology is applied, the device cannot be identified until the user goes to the device (image forming apparatus 10) and photographs the device with the camera. In such a case, it is possible to efficiently execute the process by identifying the user first and identifying the device to be used later.

  Hereinafter, modified examples of the first embodiment will be described.

  When the execution target function is selected in advance by the target user, the control unit 52 of the terminal device 14 may display the device identification information of the image forming apparatus 10 having the execution target function on the display unit of the UI unit 50. Good. For example, the control unit 52 of the terminal device 14 acquires the function purchase history information 32 for the target user from the server 12 in response to an instruction from the target user, and indicates function identification information indicating the function purchased by the target user. That is, function identification information indicating each function that can be used by the target user is displayed on the display unit of the UI unit 50. For example, a button image representing each function that can be used by the target user is displayed on the display unit as function identification information. Next, the target user selects a function to be executed from a function group that can be used by the target user. For example, the target user selects function identification information (button image) indicating a function to be executed from a function identification information group (for example, a button image group) displayed on the display unit. Thereby, the function identification information selected by the target user is transmitted from the terminal device 14 to the server 12. In the server 12, the specifying unit 42 specifies device identification information associated with the function identification information selected by the target user in the device function information 30. Thereby, the image forming apparatus 10 having the function selected by the target user is specified. At this time, one or a plurality of image forming apparatuses 10 are specified. The device identification information specified by the specifying unit 42 is transmitted from the server 12 to the terminal device 14 and displayed on the display unit of the UI unit 50 of the terminal device 14. This makes it easy for the target user to understand which image forming apparatus 10 has the function to be executed.

  Further, the position information of the image forming apparatus 10 having the function to be executed may be transmitted from the server 12 to the terminal device 14 and displayed on the display unit of the UI unit 50 of the terminal device 14. For example, the control unit 52 of the terminal device 14 displays a map on the display unit of the UI unit 50, and superimposes information (for example, an image such as a mark) indicating the image forming apparatus 10 having a function to be executed on the map. Then, it may be displayed on the display unit. This makes it easy for the target user to know where the image forming apparatus 10 having the function to be executed is installed.

  As another modified example, when the execution target function is selected in advance by the target user and the image forming apparatus 10 to be used has the execution target function, the control unit 52 of the terminal device 14 includes: The function to be executed may be executed by the image forming apparatus 10 to be used. In this case, the control unit 52 functions as an example of an execution control unit. For example, as described in the above example, the control unit 52 of the terminal device 14 displays function identification information (for example, a button image) indicating each function that can be used by the target user on the display unit of the UI unit 50. Next, the target user selects function identification information (button image) indicating the function to be executed from the function identification information group (button image group) displayed on the display unit. On the other hand, the image forming apparatus 10 to be used is specified by applying the AR technology, and function identification information indicating each function of the image forming apparatus 10 to be used is transmitted from the server 12 to the terminal device 14. When the function identification information indicating each function included in the image forming apparatus 10 to be used includes the function identification information indicating the function to be executed, that is, the function to be executed by the image forming apparatus 10 to be used. , The control unit 52 of the terminal device 14 transmits information indicating an execution instruction for the function to be executed to the image forming apparatus 10 to be used. At this time, control data and the like for executing the function to be executed is transmitted from the terminal device 14 to the image forming apparatus 10. Upon receiving the information indicating the execution instruction, the image forming apparatus 10 executes the execution target function. Thereby, compared with the case where a function that can be used by the target user and selected as a function to be executed is selected from the function group of the target image forming apparatus 10, it is possible to save time and effort for selection by the target user.

  As yet another modification, the display unit of the UI unit 50 of the terminal device 14 may expand and display information related to the UI unit 22 of the image forming apparatus 10. For example, the control unit 24 of the terminal device 14 changes information displayed on the UI unit 50 in accordance with an operation on the UI unit 22 of the image forming apparatus 10. For example, by cooperation of a hardware user interface unit (hardware UI unit) included in the image forming apparatus 10 to be used and a software user interface unit (software UI unit) realized by the UI unit 50 of the terminal device 14. A user interface unit for the image forming apparatus 10 to be used is realized. As described above, the hardware UI unit of the image forming apparatus 10 is, for example, a numeric keypad or a direction instruction key. Further, by displaying function identification information indicating each function of the image forming apparatus 10 to be used and function identification information indicating each function permitted to be used by the target user on the UI unit 50 of the terminal device 14. A software UI unit is realized. For example, when the terminal device 14 transmits information indicating a connection request to the image forming apparatus 10, communication is established between the terminal apparatus 14 and the image forming apparatus 10. In this state, information indicating an instruction given using the software UI unit of the terminal device 14 is transmitted from the terminal device 14 to the image forming apparatus 10 to be used, and the hardware UI of the image forming apparatus 10 to be used. Information indicating an instruction given using the copy unit is transmitted from the image forming apparatus 10 to be used to the terminal apparatus 14. For example, when the target user operates a numeric keypad or direction instruction key constituting the hardware UI unit, information indicating the operation content is transmitted from the image forming apparatus 10 to be used to the terminal device 14. The control unit 52 of the terminal device 14 functions as an example of an operation control unit, thereby realizing the operation content on the software UI unit. Accordingly, the software UI unit is operated using the hardware UI unit. For example, when the target user operates the hardware UI unit, selects function identification information (for example, a button image) displayed on the software UI unit and gives an execution instruction, information indicating the execution instruction of the function is displayed on the terminal. The function is transmitted from the apparatus 14 to the image forming apparatus 10 to be used, and the function is executed. As described above, the hardware UI unit provided in the image forming apparatus 10 and the software UI unit displayed on the terminal device 14 cooperate to realize the UI unit of the image forming apparatus 10. Compared with the case where only the user interface of the device, for example, the user interface of the image forming apparatus 10 or the terminal device 14 is used, the operability of the UI unit can be improved. Further, a fax number or the like may be input using the hardware UI unit, or a preview screen of image data may be displayed on the software UI unit.

  As yet another modification, individual setting information for each user is not stored in the image forming apparatus 10, and may be stored in an external device other than the image forming apparatus 10 (for example, the terminal device 14 or the server 12). Good. The individual setting information includes, for example, the user's name, address, telephone number, fax number, e-mail address, terminal device 14 address, fax destination managed by the user, e-mail address book, and the like. As an example, it is assumed that setting information is stored in the terminal device 14. When the function is executed using the setting information in the image forming apparatus 10 to be used, the setting information is transmitted from the terminal device 14 instructed to execute the function to the image forming apparatus 10 to be used. For example, when a facsimile transmission is performed in the image forming apparatus 10 to be used, information indicating a fax number used in the facsimile transmission is transmitted from the terminal device 14 instructing the facsimile transmission to the image forming apparatus 10 to be used. Sent. The image forming apparatus 10 to be used performs facsimile transmission using the facsimile number transmitted from the terminal apparatus 14. As another example, when executing the scan transfer function, the terminal device 14 transmits address information indicating the transmission destination of the image data to the image forming apparatus 10 to be used. The image forming apparatus 10 generates image data by executing a scan function, and transmits the image data to a transmission destination indicated by the address information. Thus, by not storing the setting information in the image forming apparatus 10, the leakage of the setting information from the image forming apparatus 10 is prevented or suppressed. Thereby, the security for the setting information in the image forming apparatus 10 is improved as compared with the case where the setting information is stored in the image forming apparatus 10. In the above example, the setting information is stored in the terminal device 14, but the setting information may be stored in the server 12. In this case, the terminal device 14 may access the server 12 and acquire setting information, or the image forming apparatus 10 may access the server 12 and acquire setting information.

[Second Embodiment]
Hereinafter, an image forming system as an information processing system according to the second embodiment of the present invention will be described with reference to FIG. FIG. 14 shows an example of an image forming system according to the second embodiment. The image forming system according to the second embodiment includes a plurality of devices (for example, devices 76 and 78), a server 80, and a terminal device 14. The devices 76 and 78, the server 80, and the terminal device 14 are connected to each other via a communication path N such as a network. In the example shown in FIG. 14, two devices (devices 76 and 78) are included in the image forming system, but three or more devices may be included in the image forming system. A plurality of servers 80 and a plurality of terminal devices 14 may be included in the image forming system.

  The devices 76 and 78 are devices having specific functions. For example, the devices such as the image forming apparatus 10 according to the first embodiment, a display device such as a personal computer (PC), a projector, a telephone, a clock, and a surveillance camera. It is. The devices 76 and 78 have a function of transmitting and receiving data to and from other devices.

  The server 80 is a device that manages a cooperation function executed by linking a plurality of devices. The server 80 has a function of transmitting / receiving data to / from other devices.

  The terminal device 14 has the same configuration as the terminal device 14 according to the first embodiment, and functions as a user interface unit (UI unit) of the device when the device is used, for example.

  In the image forming system according to the second embodiment, a plurality of devices are specified as devices to be linked, and the linkage function is specified by the linkage of the plurality of devices.

  Hereinafter, the configuration of the server 80 will be described in detail with reference to FIG. FIG. 15 shows the configuration of the server 80.

  The communication unit 82 is a communication interface and has a function of transmitting data to another device and a function of receiving data from another device via the communication path N. The communication unit 82 may be a communication interface having a wireless communication function or a communication interface having a wired communication function.

  The storage unit 84 is a storage device such as a hard disk or an SSD. The storage unit 84 stores linkage function information 86, various data, various programs, and the like. Of course, they may be stored in separate storage devices or may be stored in one storage device. Moreover, you may make it provide the cooperation function information 86 memorize | stored in the memory | storage part 84 regularly at the terminal device 14. FIG. By doing so, the information stored in the storage unit 48 of the terminal device 14 may be updated. Here, the cooperation function information 86 will be described.

  The cooperation function information 86 is information indicating a cooperation function that is executed by linking a plurality of devices. For example, for each cooperation function, the cooperation function information 86 is used to identify each device that is cooperated to execute the cooperation function. This is information indicating the association between a combination of device identification information and cooperative function identification information for identifying the cooperative function. The device identification information is, for example, a device ID, a device name, information indicating a device type, a model number, position information, and the like, similar to the device identification information according to the first embodiment. The cooperation function identification information is, for example, a cooperation function ID or a cooperation function name. The linkage function may be a function executed by linking a plurality of devices having different functions, or may be a function executed by linking a plurality of devices having the same function. . For example, the cooperation function is a function that could not be used before cooperation. The function that could not be used before the cooperation may be a function that can be used by using the same function among the functions of the cooperation target device, or among the functions of the cooperation target device, It may be a function that can be used by combining different functions. For example, the copy function is executed by linking a device (printer) having a print function and a device (scanner) having a scan function. That is, the copy function is executed by linking the print function and the scan function. In this case, the copy function is associated with the combination of the print function and the scan function. In the cooperation function information 86, cooperation function identification information for identifying a copy function as a cooperation function, device identification information for identifying a device having a print function, and device identification for identifying a device having a scan function. A combination of information is associated. By referring to the cooperation function information 86, a plurality of devices for executing the cooperation function are specified.

  The control unit 88 controls the operation of each unit of the server 80. The control unit 88 includes a specifying unit 90.

  The specifying unit 90 receives device identification information for identifying each device to be linked, and the linkage of linkage functions associated with a combination of each piece of device identification information in the linkage function information 86 stored in the storage unit 84. Specify function identification information. Thereby, the cooperation function performed by making each cooperation object apparatus cooperate is pinpointed (recognized). For example, a plurality of device identification information is transmitted from the terminal device 14 to the server 80, and the identification unit 90 identifies the cooperation function identification information of the cooperation function associated with the plurality of device identification information. The cooperative function identification information (for example, information indicating the name of the cooperative function) of the cooperative function is transmitted from the server 80 to the terminal device 14 and displayed on the terminal device 14, for example. Thereby, the cooperation function identification information of the cooperation function executed by the plurality of devices specified by the plurality of device identification information is displayed on the terminal device 14.

  Note that the cooperation function information 86 described above may be stored in the storage unit 48 of the terminal device 14. In this case, the cooperation function information 86 may not be stored in the storage unit 84 of the server 80. In addition, the control unit 52 of the terminal device 14 may include the specifying unit 90 described above, and specify the cooperation function based on a plurality of device identification information. In this case, the server 80 may not include the specifying unit 90.

  In the second embodiment, as an example, by applying the AR technology, device identification information of a cooperation target device is acquired, and the cooperation target device is specified (recognized). As the AR technology, a marker type AR technology, a markerless type AR technology, a position information AR technology, and the like are used as in the first embodiment.

  When the marker type AR technology is used, the camera 46 of the terminal device 14 captures a marker such as a two-dimensional barcode provided on the cooperation target device (for example, the marker 54 provided on the image forming apparatus 10). Then, image data representing the marker (for example, image data representing the marker 54) is generated. The image data is transmitted from the terminal device 14 to the server 80, for example. In the server 80, the control unit 88 extracts device identification information by applying a decoding process to the marker image represented in the image data. Thereby, the device identification information of the cooperation target device is acquired. By photographing the marker of each device to be linked, the device identification information of each device is acquired, and thereby the linkage function is specified. Note that the control unit 52 of the terminal device 14 may extract the device identification information by applying a decoding process.

  When the markerless AR technology is used, the camera 46 of the terminal device 14 captures all or a part of the appearance of the cooperation target device. Of course, it is useful to capture information for identifying the device such as the name of the device to be linked (for example, the product name) and the model number from the appearance. Appearance image data representing all or part of the appearance of the device to be linked is generated by photographing. The appearance image data is transmitted from the terminal device 14 to the server 80, for example. In the server 80, similarly to the first embodiment, the control unit 88 compares the appearance image data transmitted from the terminal device 14 with each appearance image data included in the appearance image association information, and the comparison result is obtained. Based on this, the device identification information of the device to be linked is specified. Thereby, the cooperation target apparatus is specified. As another example, when a device name (for example, a product name) or model number is photographed and appearance image data representing the name or model number is generated, it is based on the name or model number represented in the appearance image data. Thus, the cooperation target device may be specified. By photographing the appearance of each device to be linked, the device identification information of each device is acquired, and thereby the linkage function is specified. In addition, the control part 52 of the terminal device 14 may specify the device identification information of the cooperation target apparatus by applying the markerless AR technology.

  When the position information AR technology is used, for example, device position information indicating the position where the cooperation target device is installed is acquired by using the GPS function. Similarly to the first embodiment, the terminal device 14 acquires the device position information of the cooperation target device. The device position information is transmitted from the terminal device 14 to the server 80, for example. In the server 80, the control unit 88 specifies the device identification information of the cooperation target device by referring to the position association information, as in the first embodiment. Thereby, the cooperation target apparatus is specified. By acquiring the device position information of each device to be linked, the device identification information of each device is obtained, and thereby the linkage function is specified. Note that the control unit 52 of the terminal device 14 may specify the device identification information of the device to be linked by applying the position information AR technology.

  Hereinafter, a method of linking a plurality of devices by applying the AR technology will be described.

  With reference to FIG. 16, a method of linking a plurality of devices by applying the marker AR technology or the markerless AR technology will be described. FIG. 16 shows an example of a cooperation target device. As an example, the image forming apparatus 10 according to the first embodiment is used as the cooperation target device 76, and the PC 92 is used as the cooperation target device 78. For example, a marker 54 such as a two-dimensional barcode is provided on the casing of the image forming apparatus 10, and a marker 94 such as a two-dimensional barcode is provided on the casing of the PC 92. The marker 94 is information in which device identification information of the PC 92 is encoded. When acquiring the device identification information of the image forming apparatus 10 and the PC 92 by the marker type AR technique or the markerless type AR technique, the user photographs the image forming apparatus 10 and the PC 92 to be linked by the camera 46 of the terminal device 14. In the example illustrated in FIG. 16, the image forming apparatus 10 and the PC 92 are photographed together in a state where both the image forming apparatus 10 and the PC 92 are included in the photographing region of the camera 46. Thereby, image data representing the markers 54 and 94 is generated and transmitted from the terminal device 14 to the server 80. In the server 80, the control unit 88 extracts the device identification information of the image forming apparatus 10 and the device identification information of the PC 92 by applying a decoding process to the image data. Alternatively, appearance image data representing the appearance of both the image forming apparatus 10 and the PC 92 may be generated and transmitted from the terminal device 14 to the server 80. In this case, in the server 80, the control unit 88 specifies the device identification information of the image forming apparatus 10 and the device identification information of the PC 92 by referring to the appearance image association information. When the device identification information is specified, the specifying unit 90 specifies the cooperation function identification information associated with the combination of the device identification information of the image forming apparatus 10 and the device identification information of the PC 92 in the cooperation function information 86. As a result, the cooperation function executed by linking the image forming apparatus 10 and the PC 92 is specified. The cooperative function identification information indicating the cooperative function is transmitted from the server 80 to the terminal device 14 and displayed on the UI unit 50 of the terminal device 14. When the user instructs the execution of the cooperation function using the terminal device 14, the cooperation function is executed. The device identification information specifying process and the cooperation function specifying process may be performed by the terminal device 14.

  A device to be linked may be designated by a user operation. For example, by photographing the image forming apparatus 10 and the PC 92 with the camera 46, as shown in FIG. 16, a device image 98 representing the image forming apparatus 10 and a device representing the PC 92 are displayed on the screen 96 of the display unit of the terminal device 14. An image 100 is displayed. Note that the image data associated with the identified device displayed on the terminal device 14 when designating the device to be linked may be an image of the device photographed by the camera 46 (an image having a size at the time of photographing). Or an enlarged or reduced image), or appearance image data prepared in advance associated with the identified device (a schematic image, not an image itself obtained by photographing). It may be. For example, when using image data obtained by shooting a device, the appearance of the current device itself (for example, an appearance that reflects scratches, notes, stickers attached to the device, etc.) is reflected in the image. As a result, the user can visually understand the difference from other similar devices. When the user designates the device images 98 and 100 on the screen 96, the image forming apparatus 10 and the PC 92 are designated as cooperation target devices. For example, when the user designates the device image 98, the marker type AR technology or the markerless type AR technology is applied to the device image 98, whereby the device identification information of the image forming apparatus 10 is specified. Similarly, when the user designates the device image 100, the device identification information of the PC 92 is specified by applying the marker type AR technology or the markerless type AR technology to the device image 100. Thereby, the cooperation function executed by the image forming apparatus 10 and the PC 92 is specified, and the cooperation function identification information indicating the cooperation function is displayed on the UI unit 50 of the terminal device 14.

  In addition, as indicated by an arrow in FIG. 16, the user touches the device image 98 on the screen 96 and operates the screen up to 100 with a finger or the like (for example, tracing with the finger), and the user is ahead of the operation. The device images 98 and 100 may be designated, and the image forming apparatus 10 and the PC 92 may be designated as devices to be linked. The order in which the device images 98 and 100 are touched and the direction of tracing may be opposite to those in the above example. Of course, screen contact instruction means other than a finger such as a pen for tracing the screen 96 may be used. In addition to simply tracing, the device to be linked may be specified by a drawing operation that adds a circle, or the device image of the device to be linked is touched within a preset time to identify the device. You may do it. When canceling the cooperation, the user may specify a device to be canceled on the screen 96 or may press a cooperation cancellation button. When a device that is not a cooperation target is reflected in the image, the device may be excluded from the device group to be linked by designating the device on the screen 96. The operation for designating the device to be released may function in a predetermined operation that gives a cross mark.

  For example, when the image forming apparatus 10 has a scan function, the scan transfer function is executed as the cooperation function by linking the image forming apparatus 10 and the PC 92. When the scan transfer function is executed, scan data (image data) is generated by the scan function of the image forming apparatus 10, and the scan data is transmitted from the image forming apparatus 10 to the PC 92. As another example, when the image forming apparatus 10 has a print function, document data to be printed is transmitted from the PC 92 to the image forming apparatus 10. May be printed.

  FIG. 17 shows another example of devices to be linked. As an example, it is assumed that the printer 102 is used as the cooperation target device 76 and the scanner 104 is used as the cooperation target device 78. The printer 102 is an apparatus having only a print function as an image forming function, and the scanner 104 is an apparatus having only a scan function as an image forming function. For example, a marker 106 such as a two-dimensional barcode is provided on the housing of the printer 102, and a marker 108 such as a two-dimensional barcode is provided on the housing of the scanner 104. The marker 106 is information obtained by decoding the device identification information of the printer 102, and the marker 108 is information obtained by decoding the device identification information of the scanner 104. Similar to the example shown in FIG. 16, the printer 102 and the scanner 104 are photographed together in a state where both the printer 102 and the scanner 104 are included in the photographing region of the camera 46. By applying the marker type AR technology or the markerless type AR to the image data generated by the photographing, the device identification information of the printer 102 and the device identification information of the scanner 104 are specified, and the printer 102 and the scanner 104 are linked. The cooperation function to be executed is specified. The device identification information specifying process and the cooperation function specifying process may be performed by the server 80 or the terminal device 14.

  Similar to the example shown in FIG. 16, a device image 110 representing the printer 102 and a device image 112 representing the scanner 104 are displayed on the screen 96 of the display unit of the terminal device 14. , 112 may be used to specify the printer 102 and the scanner 104 as devices to be linked. Thereby, the cooperation function identification information indicating the copy function as the cooperation function is displayed on the UI unit 50 of the terminal device 14.

  By cooperating the printer 102 and the scanner 104, a copy function is executed. In this case, the document is read by the scanning function of the scanner 104, and scan data (image data) representing the document is generated. The scan data is transmitted from the scanner 104 to the printer 102, and an image based on the scan data is printed on a sheet by the print function of the printer 102. As described above, even when the device to be used does not have a copy function, the copy function as the link function is executed by linking the printer 102 and the scanner 104 together.

  Hereinafter, with reference to FIG. 18 and FIG. 19, another method of linking a plurality of devices by applying the marker type AR technology or the markerless type AR technology will be described. 18 and 19 show screens of the display unit of the terminal device 14. As an example, it is assumed that the image forming apparatus 10 is used as the cooperation target device 76 and the PC 92 is used as the cooperation target device 78. In this example, the image forming apparatus 10 and the PC 92 are photographed separately. This is because the devices to be linked are not necessarily arranged in the vicinity. Of course, the angle of the shooting area of the shooting means can be changed or the shooting area can be expanded or reduced, but if it can not be handled by such an operation, shooting by the shooting means can be divided into multiple times, Identifying devices to be linked can also be a solution. When the photographing by the photographing means is divided into a plurality of times, the device identification information identified at each photographing time is stored in the terminal device 14 or the storage unit of the server 80. For example, as shown in FIG. 18, the image forming apparatus 10 is photographed with the image forming apparatus 10 included in the photographing area of the camera 46, and as shown in FIG. The PC 92 is photographed. Thereby, image data representing the image forming apparatus 10 and image data representing the PC 92 are generated. By applying the marker type AR technology or the markerless type AR technology to each image data, the device identification information of the image forming apparatus 10 and the device identification information of the PC 92 are specified, and the cooperation function is specified.

  As yet another method, a device to be linked may be set in advance as a basic linked device. For example, it is assumed that the image forming apparatus 10 is preset as a basic cooperation device. The device identification information indicating the basic cooperation device may be stored in advance in the storage unit 48 of the terminal device 14, or may be stored in advance in the storage unit 84 of the server 80. Alternatively, the user may specify the basic cooperation device using the terminal device 14. When the basic cooperation device is set, the user uses the camera 46 of the terminal device 14 to photograph a device to be linked other than the basic cooperation device. For example, when the PC 92 is used as a cooperation target, the user photographs the PC 92 with the camera 46 as shown in FIG. As a result, the device identification information of the PC 92 is specified, and the cooperation function to be executed by the cooperation of the image forming apparatus 10 and the PC 92 is specified.

  Next, with reference to FIG. 20, a method of coordinating a plurality of devices by applying the position information AR technology will be described. FIG. 20 shows each device arranged in the search area. For example, the terminal device 14 has a GPS function, acquires terminal location information indicating the location of the terminal device 14 itself, and transmits the terminal location information to the server 80. The control unit 88 of the server 80 refers to the position association information indicating the association between the device position information indicating the installation position of the device and the device identification information, and is set in advance with reference to the position of the terminal device 14. Each device arranged in is identified as a cooperation candidate device. For example, as illustrated in FIG. 20, it is assumed that the image forming apparatus 10, the PC 92, the printer 102, and the scanner 104 are arranged in a range 114 set in advance with reference to the terminal device 14. In this case, the image forming apparatus 10, the PC 92, the printer 102, and the scanner 104 are specified as cooperation candidate devices. The device identification information of each cooperation candidate device is transmitted from the server 80 to the terminal device 14 and displayed on the UI unit 50 of the terminal device 14. As device identification information, an image of a cooperation candidate device may be displayed, or a character string such as a device ID may be displayed. The user designates a cooperation target device from among the cooperation candidate devices displayed on the UI unit 50. The device identification information of the device to be linked specified by the user is transmitted from the terminal device 14 to the server 80, and the server 80 identifies the linkage function based on the device identification information of the device to be linked. The UI unit 50 of the terminal device 14 displays cooperative function identification information indicating the cooperative function. In addition, the identification process of a cooperation candidate apparatus and the identification process of a cooperation function may be performed by the terminal device 14.

  Hereinafter, processing performed by the image forming system according to the second embodiment will be described with reference to FIG. FIG. 21 is a sequence diagram showing the processing.

  First, the user uses the terminal device 14 to instruct activation of an application (program) for executing the cooperation function. The control unit 52 of the terminal device 14 activates the application according to the instruction (S60). The application may be stored in advance in the storage unit 48 of the terminal device 14, or may be downloaded from the server 80 or the like.

  Next, the control unit 52 of the terminal device 14 reads the user account information (user identification information) of the user (S61). This reading process is the same as the process of step S02 according to the first embodiment.

  In addition, the utilization log | history of a cooperation function may be managed for every user, and the information which shows the cooperation function utilized in the past by the user which the read user account information shows may be displayed on UI part 50 of the terminal device 14. FIG. Information indicating the usage history may be stored in the storage unit 48 of the terminal device 14 or may be stored in the storage unit 84 of the server 80. Moreover, the information which shows the cooperation function currently used with the frequency more than the preset usage frequency may be displayed. By providing such a shortcut function, the time and effort of the user's operation regarding the cooperation function is reduced.

  Next, by applying the marker type AR technology, the markerless type AR technology, or the position information AR technology, a device to be linked is specified (S62). When applying the marker type AR technology or the markerless type AR technology, the user photographs the device to be linked by the camera 46 of the terminal device 14. As an example, when the devices 76 and 78 are used as linked devices, the user captures the devices 76 and 78 with the camera 46. Thereby, image data representing the devices 76 and 78 is generated, and the device identification information of the devices 76 and 78 is specified by applying the marker AR technology or the markerless AR technology. When the position information AR technology is applied, the device position information of each of the devices 76 and 78 is acquired, and the device identification information of each of the devices 76 and 78 is specified based on the device position information.

  Next, the terminal device 14 transmits information indicating a connection request to the devices 76 and 78 to be linked (S63). For example, when the address information indicating the addresses of the devices 76 and 78 is stored in the server 80, the terminal device 14 acquires the address information of the devices 76 and 78 from the server 80. When the address information is included in the device identification information, the terminal device 14 may acquire the address information of each of the devices 76 and 78 from the device identification information of each of the devices 76 and 78. Alternatively, the address information of each of the devices 76 and 78 may be stored in the terminal device 14. Of course, the terminal device 14 may acquire the address information of the devices 76 and 78 by another method. The terminal device 14 transmits information indicating a connection request to the devices 76 and 78 using the address information of the devices 76 and 78.

  The devices 76 and 78 permit or do not permit connection with the terminal device 14 (S64). For example, when the devices 76 and 78 correspond to devices that are not permitted to be connected, or when the number of terminal devices requesting connection exceeds the upper limit, the connection is not permitted. In addition, when the connection from the terminal device 14 is permitted, the change operation may be prohibited so that setting information unique to the devices 76 and 78 is not changed from the terminal device 14. For example, it is prohibited to change the color parameters of the image forming apparatus or the set time when shifting to power saving. Thereby, the security with respect to the apparatuses 76 and 78 improves. Alternatively, when the devices 76 and 78 are linked, the change of the setting information may be limited as compared to the case where the devices 76 and 78 are used independently without being linked. For example, fewer setting item changes may be permitted as compared to the case where the devices 76 and 78 are used alone. In addition, browsing of personal information of other users such as operation history may be prohibited. Thereby, the security with respect to a user's personal information improves.

  Result information indicating permission or non-permission of the connection is transmitted from the devices 76 and 78 to the terminal device 14 (S65). When the connection to the devices 76 and 78 is permitted, communication is established between the terminal device 14 and the devices 76 and 78.

  When the connection to the devices 76 and 78 is permitted, the UI unit 50 of the terminal device 14 displays cooperation function identification information indicating a cooperation function executed by linking the devices 76 and 78 (S66). . As described above, the cooperation function executed by linking the devices 76 and 78 is specified by the device identification information of the devices 76 and 78, and the cooperation function identification information indicating the cooperation function is displayed on the terminal device 14. Is done. The specifying process may be performed by the server 80 or the terminal device 14.

  Next, the user uses the terminal device 14 to instruct execution of the cooperation function (S67). In response to this instruction, execution instruction information indicating an instruction to execute the cooperation function is transmitted from the terminal device 14 to the devices 76 and 78 (S68). The execution instruction information transmitted to the device 76 includes information (for example, job information) indicating processing executed by the device 76, and the execution instruction information transmitted to the device 78 is executed by the device 78. Information (for example, job information) is included.

  The devices 76 and 78 that have received the execution instruction information execute the function according to the execution instruction information (S69). For example, when the linkage function includes processing for transmitting and receiving data between the devices 76 and 78, such as a scan transfer function for transferring scan data from the image forming apparatus 10 to the PC 92, the devices 76 and 78 are included. Communication is established between the two. In this case, for example, the execution instruction information transmitted to the device 76 includes the address information of the device 78, and the execution instruction information transmitted to the device 76 includes the address information of the device 78. Using this, communication is established between the devices 76 and 78.

  When the execution of the cooperation function ends, result information indicating completion of execution of the cooperation function is transmitted from the devices 76 and 78 to the terminal device 14 (S70). The display unit of the UI unit 50 of the terminal device 14 displays that the cooperation function has been executed (S71). Note that, when the execution completion is not displayed even after a preset time has elapsed since the execution instruction is given, the control unit 52 of the terminal device 14 displays information indicating the error on the display unit of the UI unit 50. The information may be displayed and the execution instruction information or the information indicating the connection request may be transmitted to the devices 76 and 78 again.

  Next, the user confirms whether or not to cancel the cooperation state of the devices 76 and 78 (S72), and processing according to the presence or absence of the cancellation is executed (S73). When canceling the cooperation state, the user uses the terminal device 14 to give a cancellation instruction. Thereby, the communication between the terminal device 14 and the devices 76 and 78 is released. Similarly, communication with the devices 76 and 78 is also released. If the cooperation state is not canceled, an execution instruction may be given continuously.

  Moreover, you may increase the apparatus of cooperation object. For example, the device identification information of the third device is acquired, and the cooperation function executed by linking three devices including the devices 76 and 78 is specified. Information indicating that the devices 76 and 78 have already been specified is stored in the terminal device 14 or the server 80.

  The device identification information of each of the cooperation target devices 76 and 78 and the cooperation function identification information indicating the executed cooperation function may be stored in the terminal device 14 or the server 80. For example, for each user, history information is created in which user account information (user identification information), device identification information of each device to be linked, and linkage function identification information indicating an executed linkage function are associated with each other. And stored in the terminal device 14 or the server 80. The history information may be created by the terminal device 14 or may be created by the server 80. By referring to the history information, it is specified which cooperation function is executed using which device group.

  The devices 76 and 78 may store, as history information, user account information indicating a user who has requested connection and terminal identification information indicating the terminal device 14. By referring to the history information, the user who uses the devices 76 and 78 is specified. For example, when specifying a user when the devices 76 and 78 are broken, or when charging a consumable or the like, the user may be specified using history information. The history information may be stored in the server 80 or the terminal device 14, or may be stored in another device.

  Next, transition of screens displayed on the UI unit 50 of the terminal device 14 from when a device to be linked is recognized to when the linkage function is executed will be described with reference to FIG.

  As an example, as shown in FIG. 16, a case where an image forming apparatus 10 and a PC 92 are used as devices to be linked will be described. In the example illustrated in FIG. 22, the image forming apparatus 10 includes at least a scan function, a print function, and a copy function as image forming functions, and functions as a so-called multifunction device.

  First, as shown in FIG. 16, the user takes an image of the image forming apparatus 10 (multifunction machine) to be linked and the PC 92 with the camera 46 of the terminal device 14. As a result, as shown in FIG. 22A, the device image 98 representing the image forming apparatus 10 and the device image 100 representing the PC 92 are displayed on the screen 96 of the UI unit 50 of the terminal device 14.

  As an example, the image forming apparatus 10 and the PC 92 are recognized by applying the marker type AR technology or the markerless type AR technology, and the recognition device screen 116 is displayed on the UI unit 50 as shown in FIG. On the recognition device screen 116, device identification information of the image forming apparatus 10 and device identification information of the PC 92 are displayed. On the recognition device screen 116, for example, (1) a character string indicating a multifunction peripheral is displayed as device identification information of the image forming apparatus 10, and (2) a character string indicating a personal computer is displayed as device identification information of the PC 92. In addition, the names and product names of the image forming apparatus 10 and the PC 92 may be displayed.

  When the device identification information of the image forming apparatus 10 and the device identification information of the PC 92 are specified, a cooperation function to be executed by linking the image forming apparatus 10 and the PC 92 is specified. As shown in FIG. A cooperation function selection screen 118 is displayed on the UI unit 50. The link function selection screen 118 includes, for example, (1) information indicating a scan data transfer function (scan transfer function) to a personal computer and (2) information indicating a function of printing document data stored in the personal computer. Are displayed as linkage function information. When the execution of the cooperation function (1) is instructed, the document is read by the scan function of the image forming apparatus 10 (multifunction machine) to generate scan data, and the scan data is transferred from the image forming apparatus 10 to the PC 92. The When the execution of the cooperation function (2) is instructed, document data stored in the PC 92 is transmitted from the PC 92 to the image forming apparatus 10, and a document based on the document data is placed on a sheet by the print function of the image forming apparatus 10. Printed. Note that the cooperation function executed when the device group selected by the user on the recognition device screen 116 shown in FIG. 22B is used as the device to be linked and the device group selected by the user is linked. May be displayed on the cooperation function selection screen 118.

  Another display form may be adopted as the display form of the cooperation function information. For example, the control unit 52 of the terminal device 14 displays information (for example, a button image group) indicating a function group including the cooperation function on the display unit of the UI unit 50, and a plurality of devices that execute the cooperation function in cooperation with each other. Is not specified (recognized), the cooperative function information (for example, an image of a button) is displayed on the display unit in a state where the cooperative function cannot be used. When device identification information of a plurality of devices that execute the cooperation function in cooperation is acquired and the plurality of devices are recognized, the control unit 52 displays the cooperation function information in a state where the cooperation function can be used. Display on the screen. For example, the control unit 52 displays information (for example, a button image group) indicating, for example, a print function, a scan function, a copy function, and a scan transfer function as a cooperation function on the UI unit 50. Let When a plurality of devices that execute the scan transfer function in cooperation are not recognized, the control unit 52 causes the display unit to display the link function information in a state where the scan transfer function cannot be used. For example, the control unit 52 does not accept an instruction to execute the scan transfer function. As a result, even when the user designates cooperative function information (for example, an image of a button) indicating the scan transfer function and gives an execution instruction, the scan transfer function is not executed. When a plurality of devices that execute the scan transfer function in cooperation are recognized, the control unit 52 causes the display unit to display link function information (for example, an image of a button) in a state where the scan transfer function can be used. When the execution instruction of the scan transfer function is given by the user, the control unit 52 receives the execution instruction and transmits execution instruction information indicating the execution instruction of the scan transfer function to the cooperation target device group.

  For example, when the user designates (1) the scan transfer function, a confirmation screen 120 is displayed on the UI unit 50 as shown in FIG. When the user presses the “NO” button on the confirmation screen 120, the screen returns to the previous cooperative function selection screen 118. When the user presses the “YES” button, the scan transfer function is executed. When execution of the scan transfer function is completed, an execution completion screen 122 indicating completion of execution of the cooperation function is displayed on the UI unit 50, as shown in FIG. Further, the execution completion screen 122 displays information for the user to instruct whether to release the connection of the cooperation target device. On the execution completion screen 122, when the user gives an instruction to disconnect the device, the connection between the terminal device 14, the image forming apparatus 10, and the PC 92 is released. If the user does not instruct the device connection release, the screen returns to the cooperation function selection screen 118.

  As described above, according to the second embodiment, by applying the AR technology, the cooperation function executed by linking the cooperation target device group is specified, and the cooperation function identification information indicating the cooperation function is displayed on the terminal. It is displayed on the device 14. This makes it easy for the user to understand what kind of linkage function can be executed even if it is not clear from the appearance what kind of linkage function can be executed depending on the device group to be linked. . In addition, functions that cannot be executed individually by individual devices can be executed by linking a plurality of devices, which is convenient. In addition, since the cooperation function can be used simply by recognizing the device to be linked by applying the AR technology, it is simpler than the case where the user performs settings for executing the cooperation function manually. The cooperation function can be used in the work, and the labor of the user is reduced.

  Further, according to the second embodiment, for example, in an environment in which a plurality of users use a plurality of devices, information regarding the cooperation function is appropriately displayed on the terminal device 14 of each user. For example, even when a user interface such as a touch panel is removed from a device, the terminal device 14 is used as the user interface, and information regarding a cooperation function executed by linking a plurality of devices is displayed for each user. Appropriately displayed on the terminal device 14. As another scene, for example, when a plurality of devices are temporarily used at a user's destination, etc., a user interface suitable for the user, that is, executed by linking a plurality of devices specified by the user A user interface representing the cooperation function to be performed is realized in the terminal device 14.

  Hereinafter, a specific example of the cooperation function will be described.

(Specific example 1)
The cooperation function according to the first specific example is a cooperation function executed by combining the image forming apparatus 10 as a multifunction peripheral and a display device such as a projector. This cooperation function is a function for printing the contents of a screen displayed on a display device such as a projector by the multifunction peripheral (image forming apparatus 10). As an example, it is assumed that the device 76 is a multifunction peripheral and the device 78 is a display device such as a projector. In the first specific example, by applying the AR technology, the device identification information of each of the multifunction peripheral and the display device is acquired, and based on the device identification information, the cooperation that is executed by linking the multifunction peripheral and the display device. Function is specified. Cooperation function identification information indicating the cooperation function is displayed on the terminal device 14. When the user uses the terminal device 14 to instruct execution of the cooperation function, the terminal device 14 transmits execution instruction information to the multifunction peripheral and the display device. In response to this, the display device transmits information (screen information) displayed on the screen to the multifunction device, and the multifunction device prints the screen information transmitted from the display device on paper. According to the first specific example, simply recognizing the multifunction peripheral and the display device by the AR technology, information that allows the user to understand what functions are performed by the multifunction peripheral and the display device is provided and displayed on the display device. The contents of the displayed screen are printed by the multifunction device. Therefore, compared with the case where the user performs print settings and the like by manual operation, the user's trouble is reduced.

(Specific example 2)
The cooperation function according to the second specific example is a cooperation function executed by combining the image forming apparatus 10 as a multifunction peripheral and a telephone. This linkage function includes a function A for printing contents (call contents) that a user is conversing using a telephone by a multifunction peripheral (image forming apparatus 10), electronic document data indicating the call contents, and a preset mail. It is at least one of a function B for transmitting by e-mail to an address and a function C for transmitting the electronic document data by facsimile to a fax number associated with the telephone number of the other party. As an example, it is assumed that the device 76 is a multifunction peripheral and the device 78 is a telephone. In the second specific example, by applying the AR technology, device identification information of each of the multifunction device and the telephone is acquired, and based on the device identification information, a cooperation function group ( Functions A, B, C) are identified. Cooperation function identification information indicating the functions A, B, and C as the cooperation function is displayed on the terminal device 14. When the user uses the terminal device 14 to select an execution target function from the functions A, B, and C and instruct the execution of the linkage function, the terminal device 14 transmits execution instruction information to the multifunction peripheral and the telephone. In response to this, the telephone transmits data indicating the content of the call to the multi-function peripheral. When execution of the function A is designated, the multi-function peripheral prints a character string indicating the content of the call on paper. When the execution of the function B is instructed, the multi-function peripheral transmits electronic document data indicating the contents of the call by e-mail to a preset e-mail address (for example, to the e-mail address of the call partner). When execution of the function C is instructed, the multi-function peripheral transmits the electronic document data by facsimile to the fax number associated with the telephone number of the other party. When a plurality of functions are specified from the functions A, B, and C and an execution instruction is given, the plurality of functions may be executed. According to the second specific example, simply recognizing a multifunction device and a telephone by AR technology, information that allows the user to understand what functions are performed by the multifunction device and the telephone is provided, and the contents of the telephone call are complex. At least one of printing, e-mail transmission and facsimile transmission is executed by the machine. Therefore, compared with the case where the user performs print settings and the like by manual operation, the user's trouble is reduced.

(Specific example 3)
The cooperation function according to the specific example 3 is a cooperation function executed by combining the image forming apparatus 10 as a multifunction peripheral and a clock. This linkage function is a function for adding a timer function to the multifunction peripheral. As an example, it is assumed that the device 76 is a multifunction peripheral and the device 78 is a clock. In specific example 3, by applying the AR technology, the device identification information of each of the multifunction device and the watch is acquired, and the cooperation function executed by linking the multifunction device and the watch is identified based on the device identification information. Is done. Cooperation function identification information indicating the cooperation function is displayed on the terminal device 14. When the user uses the terminal device 14 to instruct execution of the cooperation function, image formation using the timer function is executed. For example, the multifunction device executes image formation such as printing at a time designated by the user. According to the third specific example, information that allows the user to know what function is executed by the multifunction device and the clock only by recognizing the multifunction device and the clock by the AR technology is provided. Is granted. Therefore, even when a multi-function peripheral that does not have a timer function is used, it is possible to perform image formation using the timer function.

(Specific example 4)
The cooperation function according to the fourth specific example is a cooperation function that is executed by combining the image forming apparatus 10 as a multifunction peripheral and the monitoring camera. This cooperation function is a function for deleting specific information (for example, job information, image data, etc.) stored in the multifunction peripheral according to the content captured by the monitoring camera. As an example, it is assumed that the device 76 is a multifunction peripheral and the device 78 is a surveillance camera. In specific example 4, by applying the AR technology, the device identification information of each of the multifunction device and the monitoring camera is acquired, and the cooperation function executed by linking the multifunction device and the monitoring camera based on the device identification information Is identified. Cooperation function identification information indicating the cooperation function is displayed on the terminal device 14. When the user uses the terminal device 14 to instruct the execution of the cooperation function, the terminal device 14 transmits execution instruction information to the multifunction peripheral and the monitoring camera. In response to this, the surveillance camera analyzes the captured content and transmits an information deletion command to the multifunction device when a specific event occurs. For example, when a person suspected of being a suspicious person is photographed by the surveillance camera outside the business hours, the surveillance camera transmits an information deletion command to the multifunction peripheral. The multi-function peripheral deletes job information and image data stored in the multi-function peripheral in accordance with the information deletion instruction. This improves the security of the multifunction device. According to specific example 4, simply recognizing a multifunction peripheral and a surveillance camera by AR technology provides information that allows the user to understand what functions are performed by the multifunction peripheral and the surveillance camera. Thus, monitoring of the multifunction device is executed. Therefore, compared with the case where the user manually performs monitoring setting or the like, the labor of the user is reduced.

  In addition to the examples described above, by coordinating the image forming apparatus and the translation apparatus, the characters included in the document that was to be printed by the image forming apparatus are translated into a parallel translation of a language that can be translated by the translation apparatus, and then used on paper. A combination of output linkage functions is also conceivable.

(Specific example 5)
The cooperation function according to the above-described example is a cooperation function executed by combining a plurality of devices having different functions, but the cooperation function may be executed by combining a plurality of devices having the same function. . In this case, the plurality of devices execute the same function to share the processing. As an example, the cooperation function according to Specific Example 5 is a cooperation function that is executed by combining a plurality of image forming apparatuses 10 as a multifunction peripheral. The linkage function is, for example, an image forming function such as a print function, a copy function, or a scan function. In specific example 5, by applying the AR technology, device identification information of each of a plurality of multifunction peripherals is acquired, and a cooperation function (for example, executed by linking a plurality of multifunction peripherals based on the device identification information) Image forming function) is specified. Cooperation function identification information indicating the cooperation function is displayed on the terminal device 14. When the user uses the terminal device 14 to instruct execution of the cooperation function, the terminal device 14 transmits execution instruction information to a plurality of multifunction peripherals to be linked. The terminal device 14 divides the processing content (for example, job) according to the number of multifunction devices to be linked, assigns the divided processing content (job) to each multifunction device, and indicates an execution instruction indicating the processing content after the division Send information to each MFP. In response to this, each MFP executes a process (job) assigned to itself. For example, the terminal device 14 divides one job related to printing in accordance with the number of multifunction devices to be linked, assigns the divided print job to each multifunction device, and executes execution instruction information indicating the divided print job. Send to each MFP. In response to this, each multi-function peripheral executes a print job by executing a print function. Further, the terminal device 14 may assign the processing content after the division to each device according to the performance of each device to be linked. For example, a multi-function peripheral having a color printing function may be assigned a job set for color printing, and a multi-function peripheral not having a color printing function may be assigned a job set for monochrome printing.

  As yet another specific example, a high-speed printing mode or a preliminary printing mode (a mode for creating a plurality of copies of the same content) may be executed as a cooperation function by linking a plurality of devices having the same function.

  Hereinafter, a modification according to the second embodiment will be described with reference to FIG. FIG. 23 shows the execution priority of the cooperation function. In a modified example, when a plurality of terminal devices 14 are simultaneously requesting connection to the same device, connection is permitted according to a preset execution priority. As shown in FIG. 23, in the case of a connection request in an emergency situation (hurry), the degree of influence on priority is “extra large”. In the case of a connection request from the owner of the device, the influence level is “large”. As for the class on the organization, the degree of influence on the priority is “medium”, and the higher the connection request from a user with a higher class, the higher the priority. As for the scheduled completion time of the job (image forming process), the degree of influence on the priority is “small”, and the priority becomes higher as the connection request has a shorter estimated completion time of the job. For example, when a plurality of terminal devices 14 are simultaneously requesting connection to the same device, the terminal device 14 that has made a connection request including information indicating an emergency situation is connected to the device with the highest priority. Is done. When there is no terminal device 14 that has made an emergency connection request among the plurality of terminal devices 14, the terminal device 14 of the owner of the device is connected to the device with the highest priority. When there is no terminal device 14 that has made an emergency connection request among the plurality of terminal devices 14 and there is no terminal device 14 of the owner of the device, the terminal device of a user with a high organizational class 14 is preferentially connected to the device. If the terminal device 14 that made the emergency connection request does not exist among the plurality of terminal devices 14, the terminal device 14 of the owner of the device does not exist, and the class of each user is the same, the process is completed The terminal device 14 that has instructed execution of a job with a short expected time is preferentially connected to the device. It should be noted that, in the emergency situation, the equipment owner, the organizational class, and the scheduled completion time of the job, which item is important may be arbitrarily set by the administrator of the cooperation target equipment. . For example, the administrator may arbitrarily change the degree of influence of each item, or some items may not be used for determining the priority order. In addition, the priority order of device use may be displayed on the UI unit 50 of the terminal device 14 according to the attribute information of each user. The attribute information includes, for example, an urgency level, whether or not the device is an owner, an organizational class, a scheduled job completion time, and the like. By determining the execution priority of the cooperation function as described above, when there is a connection request for the same device at the same time, a user with higher importance is preferentially connected to the device.

  As another modification, when a plurality of terminal devices 14 are simultaneously requesting connection to the same device, the terminal devices 14 may notify the interruption. For example, each terminal device 14 may acquire the address information of the other terminal device 14 through the same device, or the address information of the other terminal device 14 may be obtained by using a process such as broadcast. You may get it. For example, when a user instructs an interrupt request using the terminal device 14, the terminal device 14 transmits an interrupt notification to another terminal device 14 that is simultaneously requesting connection to the same device. Information indicating an interrupt notification is displayed on the UI unit 50 of another terminal device 14. For example, when a user of another terminal device 14 cancels the connection request to the device in response to the interrupt notification, communication is established between the terminal device 14 that made the interrupt request and the device. Alternatively, when a user of another terminal device 14 permits interrupt processing, the other terminal device 14 may transmit information indicating the permission to the terminal device 14 that has made the interrupt request. In this case, the terminal device 14 that has issued the interrupt request may transmit information indicating the permission to the device, so that the terminal device 14 is preferentially connected to the device. By performing the interrupt notification as described above, it is possible to cope with the case where the cooperation function is to be executed in a hurry.

[Third Embodiment]
Hereinafter, an image forming system as an information processing system according to a third embodiment of the present invention will be described. FIG. 24 shows a server 124 according to the third embodiment. The image forming system according to the third embodiment is a system in which the image forming system according to the first embodiment and the image forming system according to the second embodiment are combined, and the server 124 instead of the server 80 according to the second embodiment. including. The configuration other than the server 124 is the same as the configuration of the image forming system according to the second embodiment shown in FIG.

  Similarly to the server 12 according to the first embodiment, the server 124 manages functions available for each user for each user, and links a plurality of devices in the same manner as the server 80 according to the second embodiment. It is a device that manages the linkage function executed by. The server 124 is a device that executes a specific function, like the server 12 according to the first embodiment. The specific function executed by the server 124 is, for example, a function related to image processing. The functions managed by the server 124 are, for example, functions executed using the devices 76 and 78, functions executed by the server 124, and the like. Management of functions that can be used by the user, management of linkage functions, and execution of specific functions may be performed by separate servers, or may be performed by the same server. The server 124 has a function of transmitting / receiving data to / from other devices.

  In the image forming system according to the third embodiment, by using the terminal device 14, a function is purchased by the user, and the purchase history is managed by the server 124 as a function purchase history. The function purchased by the user is executed by the devices 76 and 78 and the server 124, for example. Moreover, when a cooperation function is purchased, the said cooperation function is performed by making a some apparatus cooperate.

  Hereinafter, the configuration of the server 124 will be described in detail.

  The communication unit 126 is a communication interface and has a function of transmitting data to another device and a function of receiving data from another device via the communication path N. The communication unit 126 may be a communication interface having a wireless communication function or a communication interface having a wired communication function.

  The storage unit 128 is a storage device such as a hard disk. The storage unit 128 stores device function information 30, function purchase history information 32, linkage function information 86, various data, various programs, and the like. Of course, they may be stored in separate storage devices or may be stored in one storage device. The device function information 30 and the function purchase history information 32 are the same as the device function information 30 and the function purchase history information 32 according to the first embodiment, and the cooperation function information 86 is the same as the cooperation function information 86 according to the second embodiment. The same.

  The function execution unit 34 of the server 124 is the same as the function execution unit 34 of the server 12 according to the first embodiment. Note that, similarly to the second embodiment, the server 124 may not include the function execution unit 34.

  The control unit 130 controls the operation of each unit of the server 124. The control unit 130 includes a purchase processing unit 38, a purchase history management unit 40, and a specifying unit 132.

  The purchase processing unit 38 and the purchase history management unit 40 of the server 124 are the same as the purchase processing unit 38 and the purchase history management unit 40 of the server 12 according to the first embodiment.

  Similar to the specifying unit 42 of the server 12 according to the first embodiment, the specifying unit 132 receives device identification information for identifying a device to be used, and device function information 30 stored in the storage unit 128. By referring to, the function group of the device to be used is specified. Similarly to the specifying unit 42 according to the first embodiment, the specifying unit 132 refers to the function purchase history information 32 stored in the storage unit 128 when receiving the user identification information for identifying the target user. By doing this, a function group that can be used by the target user is specified. Similar to the first embodiment, for example, when receiving the device identification information of the target device and the user identification information of the target user, the specifying unit 132 is a function of the target device and is used by the target user. Identify possible functions.

  Further, as with the specifying unit 90 of the server 80 according to the second embodiment, the specifying unit 132 receives the device identification information for identifying each device to be linked, and stores the linkage stored in the storage unit 128. By referring to the function information 86, the cooperation function to be executed by linking the devices to be linked is specified.

  Furthermore, in 3rd Embodiment, the specific | specification part 132 is a cooperation function performed by making a cooperation target apparatus group cooperate, Comprising: The cooperation function which can be utilized by the object user is specified. For example, the function purchase history information 32 includes, for each user, information indicating a cooperation function that can be used by the user, that is, information indicating a cooperation function purchased by the user. The cooperation function purchase process is the same as the function purchase process in the first embodiment. The specifying unit 132 receives the device identification information for identifying each device to be linked, and refers to the link function information 86 stored in the storage unit 128 to link each device to be linked. Identify linked functions. Further, the specifying unit 132 receives user identification information for identifying the target user, and refers to the function purchase history information 32 stored in the storage unit 128, whereby the cooperation function purchased by the target user, that is, The cooperation function that can be used by the target user is specified. The identification unit 132 identifies a cooperation function that is executed by linking the device group to be linked through the above processing and that can be used by the target user. The cooperative function identification information indicating the cooperative function is transmitted from the server 124 to the terminal device 14 and displayed on the UI unit 50 of the terminal device 14. Thereby, it becomes easy to understand what cooperation function is available to the target user. When the execution instruction of the cooperation function is given by the target user, the cooperation function is executed by the cooperation target device group as in the second embodiment.

  The control unit 52 of the terminal device 14 causes the display unit of the UI unit 50 to display the cooperation function identification information indicating each cooperation function executed by linking the cooperation target device group, and can be used by the target user. The link function identification information indicating the function may be displayed on the display unit of the UI unit 50 separately from the link function identification information indicating the link function that cannot be used by the target user. As a result, it becomes easy for the target user to understand what kind of cooperation function can be executed depending on the device group to be linked, and what kind of cooperation function can be used by the target user. .

  As another example, the specifying unit 132 specifies a plurality of functions that can be used by the target user by referring to the function purchase history information 32, and specifies a cooperation function that is executed by linking the plurality of functions. May be. For example, when the scan function and the print function can be used by the target user as individual functions, the copy function executed by linking the scan function and the print function can be used by the target user as the link function. . Further, the specifying unit 132 refers to the link function information 86 to specify a link function group that is executed by connecting a plurality of devices to be linked. Through the above processing, the specifying unit 132 may specify a cooperation function that is executed by linking a plurality of devices to be linked, and can be used by the target user.

  Also in the third embodiment, the device identification information of the device is acquired by applying the AR technology. Of course, the device identification information of the device may be acquired without applying the AR technology. User operations and processing when a plurality of devices to be linked are linked are the same as the operations and processing in the second embodiment. Similarly to the first and second embodiments, the device function information 30, the function purchase history information 32, and the link function information 86 are stored in the storage unit 48 of the terminal device 14, and the purchase history management unit 40 and the specifying unit 132 are stored. Are provided in the control unit 52 of the terminal device 14, and processing using these may be executed by the terminal device 14.

  According to the third embodiment, when a user wants to know an individual function that can be used by an individual device, information indicating the function that can be used by recognizing the device to be used by AR technology. Is displayed on the terminal device 14. In addition, when a user wants to know a cooperation function that can be used by a user by linking a plurality of devices to be linked, the AR technology recognizes the devices to be linked. As a result, information indicating the available cooperation functions is displayed on the terminal device 14. In this way, information related to available functions is displayed on the terminal device 14 in accordance with the usage mode of the device.

[Fourth Embodiment]
Hereinafter, with reference to FIG. 25, an image forming system as an information processing system according to the fourth embodiment of the present invention will be described. FIG. 25 shows a server 134 according to the fourth embodiment. The image forming system according to the fourth embodiment includes a server 134 instead of the server 80 according to the second embodiment. The configuration other than the server 134 is the same as the configuration of the image forming system according to the second embodiment shown in FIG.

  The server 134 is a device that manages a device group to be connected according to a function to be used, that is, a device group to be connected to execute the function to be used. The function to be used is, for example, a cooperation function executed by linking a plurality of devices (for example, the devices 76 and 78), and the server 134 selects a device group to be linked that can be executed in cooperation with the cooperation function. to manage. Of course, the function to be used may be a function that can be executed independently by each device. The server 134 has a function of transmitting / receiving data to / from other devices.

  In the image forming system according to the fourth embodiment, by using the terminal device 14, a function to be used (for example, a function that the user wants to use) is specified, and the user should be connected to execute the function to be used. Information indicating the device group is displayed on the terminal device 14.

  Hereinafter, the configuration of the server 134 will be described in detail.

  The communication unit 136 is a communication interface, and has a function of transmitting data to another device and a function of receiving data from another device via the communication path N. The communication unit 136 may be a communication interface having a wireless communication function or a communication interface having a wired communication function.

  The storage unit 138 is a storage device such as a hard disk. The storage unit 138 stores linkage function information 86, device management information 140, various data, various programs, and the like. Of course, they may be stored in separate storage devices or may be stored in one storage device. The cooperation function information 86 is the same as the cooperation function information 86 according to the second embodiment.

  The device management information 140 is information for managing information related to a device. For example, for each device, the device identification information of the device, at least one information among device position information, performance information, and usage status information, It is the information which shows matching of. The device position information is information indicating the position where the device is installed, the performance information is information indicating the performance (spec) of the device, and the usage status information indicates the current usage status (status) of the device. Information. For example, device position information and performance information are acquired in advance and registered in the device management information 140. The device position information of each device is acquired by using, for example, a GPS device. In addition, usage status information is transmitted from each device to the server 134 and registered in the device management information 140. For example, usage status information is transmitted from the device to the server 134 at a preset time interval, every preset time interval, or whenever the usage status changes. Of course, the usage status information may be acquired and registered in the device management information 140 at another timing.

  The control unit 142 controls the operation of each unit of the server 134. For example, the control unit 142 manages the usage status of each device, and updates the device management information 140 every time the usage status information of each device is acquired. In addition, the control unit 142 includes a specifying unit 144.

  The specifying unit 144 specifies a device group to be connected according to the function to be used. For example, the specifying unit 144 receives cooperation function identification information indicating a cooperation function as a function to be used, and a plurality of pieces of association function information 86 stored in the storage unit 138 are associated with the cooperation function identification information. Specify device identification information. Thereby, a device group to be connected to execute the function to be used, that is, a device group that can be executed in cooperation with the cooperation function is specified (recognized). For example, the cooperation function identification information is transmitted from the terminal device 14 to the server 134, and the identification unit 144 identifies the device identification information of each device associated with the cooperation function identification information. For example, the device identification information of each device is transmitted from the server 134 to the terminal device 14 and displayed on the terminal device 14. Accordingly, information indicating a device group to be connected to execute the function to be used (for example, a cooperation function), that is, a device group that can be executed in cooperation with the function to be used is displayed on the terminal device 14. .

  When the device group to be connected is specified, the specifying unit 144 specifies, in the device management information 140, device position information, performance information, and usage associated with the device identification information for each device to be connected. Identify at least one of the situation information. Information such as device position information is transmitted from the server 134 to the terminal device 14 and displayed on the terminal device 14, for example.

  The function to be used may be a function that can be executed independently by each device. In this case, the specifying unit 144 specifies a single device to be connected to execute the function to be used, that is, a device that can execute the function to be used independently. Information indicating the device is transmitted from the server 134 to the terminal device 14 and displayed on the terminal device 14.

  The device management information 140 may be stored in the storage unit 48 of the terminal device 14. In this case, the device management information 140 may not be stored in the storage unit 138 of the server 134. Moreover, the control part 52 of the terminal device 14 may be provided with the specific | specification part 144, and may specify the apparatus group which should be connected. In this case, the server 134 may not include the specifying unit 144.

  Hereinafter, with reference to FIG. 26, the processing by the image forming system according to the fourth embodiment will be described in detail.

  For example, the control unit 52 of the terminal device 14 displays a list of functions on the UI unit 50, and the user selects a function (a function to be used) that the user wants to use from the list. As an example, as indicated by reference numeral 146 in FIG. 26, it is assumed that the function “print conversation contents on the telephone” is selected as the function to be used. This function is a linkage function executed by linking a telephone and a device having a print function (for example, a printer or a multifunction device). As indicated by reference numerals 148 and 150, a group of devices to be connected (requires connection). Group of devices) is a telephone and a printer. Of course, instead of a printer, a multifunction machine having a print function may be used as a device to be connected.

  Cooperation function identification information indicating the cooperation function selected by the user is transmitted from the terminal device 14 to the server 134. In the server 134, the specifying unit 144 specifies a plurality of device identification information associated with the cooperation function identification information in the cooperation function information 86 stored in the storage unit 138. Thereby, a device group to be connected to execute the cooperation function, that is, a device group that can execute the cooperation function in cooperation is specified (recognized). In the example shown in FIG. 26, as indicated by reference numerals 152, 154, and 156, the telephones A and B and the printer A are recognized as devices to be connected in order to execute the function of “printing conversation contents on the telephone”. Has been. The telephones A and B and the printer A are devices included in the image forming system, like the devices 76 and 78.

  At this stage, the device identification information of each of the telephones A and B and the printer A is transmitted from the server 134 to the terminal device 14 as information related to the device group to be connected, and displayed on the UI unit 50 of the terminal device 14. Good. As a result, the user is provided with information indicating a group of devices to be connected to execute the function to be used.

  When the device group to be connected is specified, the specifying unit 144 may acquire information about each of the telephones A and B and the printer A by referring to the device management information 140. For example, the specifying unit 144 acquires performance information indicating the performance (spec) of each of the telephones A and B and the printer A. In the example shown in FIG. 26, the performance indicated by reference numeral 158 is the performance of telephone A, the performance indicated by reference numeral 160 is the performance of telephone B, and the performance indicated by reference numeral 162 is the performance of printer A. As the performance of the telephones A and B, a compatible frequency band is defined. The telephone A is a telephone compatible with overseas, and the telephone B is a telephone compatible only in Japan. The resolution is defined as the performance of the printer A. The printer A is a printer compatible with color printing. The performance information of each of the telephones A and B and the printer A is transmitted from the server 134 to the terminal device 14 as information related to a device group to be connected, and displayed on the UI unit 50 of the terminal device 14. Thus, useful information is provided to the user in selecting a device suitable for the function to be used. For example, when the user desires color printing, referring to the performance information displayed on the UI unit 50 makes it easy to find a device (color printing compatible printer) that meets the request.

  In addition, screen transitions of the UI unit 50 of the terminal device 14 will be described with reference to FIGS. 27A to 27N as examples of devices necessary for executing the cooperation function and applications that request connection. The user is identified by starting the application and logging in to the account. Of course, it is possible to omit the login, but if security or a specific function is to be executed or managed by each user, this can be done by requesting an account login. FIG. 27A shows a screen that allows the user to specify a cooperation function to be executed. The user input unit shown in FIG. 27A is a place where the user inputs text or voice, or a place where a cooperation function desired to be used is input from a pull-down menu. In accordance with the contents of the cooperation function input here, device identification processing necessary for the subsequent cooperation function is performed. If there is no problem with the input linkage function, the screen changes to the next screen by pressing the OK button. FIG. 27B shows a result of automatically specifying necessary devices according to the contents of the cooperation function input by the user input unit. As an example, since the linkage function is a function of “printing conversation contents on the telephone”, a telephone and a printer are displayed as necessary devices.

  FIG. 27C and FIG. 27E show, among the specified necessary devices, devices in the same field that the user can identify and use in the past, or devices that are identified and extracted from the network that the user can newly use. Etc. are shown. A list of telephones is displayed on the screen shown in FIG. 27C, and a list of printers is displayed on the screen shown in FIG. 27E. The user touches and specifies a device name to be used from this list.

  27D and 27F show devices selected by the user from the device candidates necessary for executing the cooperation function shown in FIGS. 27C and 27E. As shown in FIG. 27D, the telephone set B is selected, and as shown in FIG. 27F, the printer B is selected. If a device is specified by mistake, it is possible to return to the selection screen by selecting “NO” on this confirmation screen. When “YES” is selected, the screen shifts to the next device selection screen.

  FIG. 27G shows a confirmation screen that is displayed after all the devices necessary for executing the cooperation function have been specified. When “NO” is selected on this confirmation screen, the screen returns to the selection screen of each device. Here, when “YES” is selected, the screen changes to a screen for sending a connection request to the selected device. FIG. 27H shows the screen.

  As shown in FIG. 27I, when the environment in which the cooperation function can be executed is prepared (for example, when the network connection or the execution function previously performed by each device is completed), the user is immediately asked whether the cooperation function is to be executed. A message is displayed. If “YES” is selected here, the cooperation function is immediately executed, and if “NO” is selected, the connection state is maintained for a preset time, and the user can wait for the cooperation function to be executed immediately. .

  The contents displayed on the screen can be switched between when the execution of the linkage function is successful and when it cannot be successful and fails. When the execution of the cooperation function is successful, the screen transitions in the order of the screens illustrated in FIGS. 27J, 27L, and 27N. On the other hand, when the execution of the cooperation function fails, the screen transitions in the order of the screens shown in FIGS. 27K, 27M, and 27N. In the screen shown in FIG. 27N, it is possible to instruct execution of the same cooperation function, execution of another cooperation function, or termination of the application again. When the same linkage function is executed, the connection setting process is omitted. However, if the reason for the failure to execute the cooperation function is a problem specific to the cooperation function, and there are other devices that can be selected, if “Execute the same cooperation function” is selected on the screen shown in FIG. 27N Even so, only the selection of the device causing the error may be changed. When “execute another link function” is selected, the screen changes to the screen shown in FIG. 27A. When “Exit Application” is selected, this application is terminated.

  As described above, the user can easily perform the settings necessary for executing the cooperation function only by installing the device necessary for executing the cooperation function and the application requesting connection to the terminal device 14. Is possible.

  Further, the performance information of each device to be connected may be displayed according to the priority condition. The priority condition is set by the user, for example. For example, when high-quality printing is designated by the user, the specifying unit 144 sets the priority of a printer that supports color printing or the priority of a printer that has a higher resolution than other printers over the priority of other printers. Set high. The control unit 52 of the terminal device 14 causes the UI unit 50 to display the device identification information of the printer supporting color printing or the high resolution printer on the UI unit 50 in preference to the device identification information of other printers according to the priority. As another example, when a call to overseas is designated by the user, the specifying unit 144 sets the priority of the overseas compatible telephone higher than the priority of the telephone compatible only in Japan. In accordance with the priority, the control unit 52 causes the UI unit 50 to display the device identification information of the overseas compatible telephone with priority over the device identification information of the domestic compatible telephone. In addition, when there are a plurality of printers as candidates for other devices to be connected, a printer closer to the user's position may be preferentially displayed on the UI unit 50. For example, the control unit 52 compares the device identification information of a device with high priority with the device identification information of another device and arranges the device identification information at a position that is easily noticeable, such as the center or the upper portion of the UI unit 50. As another example, a device with high priority may be arranged and displayed in a specific area where devices with high priority predetermined by the user are arranged. As another example, information indicating the recommendation is added to the device identification information of a high priority device, the information of a high priority device is enlarged, or the font, color and display format of characters are changed. It may be displayed on the UI unit 50. This facilitates selection of a device suitable for the function to be used, compared to a case where device identification information of a device group to be connected is displayed at random.

  FIGS. 28 to 31 show an example of priority display of devices. For example, as illustrated in FIG. 28, the character string indicating the device is displayed on the UI unit 50 of the terminal device 14 by changing the size, color, or font of the character string according to the priority. In addition, a position where a character string indicating a high-priority device (for example, overseas compatible telephone A) is more noticeable than a low-priority apparatus (for example, domestic compatible telephones B and C) (for example, the upper left position of the screen) It is arranged and displayed. As another example, as shown in FIG. 29, the shape of the image or mark indicating the device is changed according to the priority. In the example shown in FIG. 29, an image or mark indicating a high-priority device (for example, a printer C compatible with color printing) is higher than an image or mark indicating a device with a low priority (for example, printer D compatible with monochrome printing). Displayed in a conspicuous shape. As another example, as shown in FIG. 30, a character string indicating a high-priority device (for example, overseas compatible telephone A) is compared with a low-priority apparatus (for example, domestic compatible telephones B and C). The UI unit 50 is arranged and displayed at the center. As another example, as shown in FIG. 31, a character string indicating a high-priority device (for example, printer C for color printing) in a specific area 170 (priority-arrangement region) where a high-priority device is disposed. Is displayed, and in a region other than the specific region 170, a character string indicating a low-priority device (for example, a printer D compatible with monochrome printing) is displayed. The specific area 170 may be an area designated by the user or may be a preset area. By performing priority display as described above, it is easy to see a character string indicating a device with high priority, and it is easy to select a device suitable for the function to be used.

  Further, the specifying unit 144 may specify the current states of the telephones A and B and the printer A by referring to the device management information 140. For example, the specifying unit 144 acquires device position information of the telephones A and B and the printer A from the device management information 140. Further, the specifying unit 144 acquires user position information indicating the position of the user or the terminal device 14. The specifying unit 144 compares the position indicated by the device position information of the device with the position indicated by the user position information for each device to be connected, and the relative positional relationship between the user and the device for each device. Is identified. In the example shown in FIG. 26, the telephone A is installed at a position relatively close to the user or the terminal device 14 as indicated by reference numeral 164, and the telephone B and the printer A are indicated as indicated by reference numerals 166 and 168. Is installed at a position relatively far from the user or the terminal device 14. Information indicating the relative positional relationship is transmitted from the server 134 to the terminal device 14 as information related to a device group to be connected, and is displayed on the UI unit 50 of the terminal device 14. Thereby, in selecting a device to be used, useful information is provided to the user regarding the moving distance and the like.

  The user position information may be transmitted to the server 134 acquired by the terminal device 14 or may be acquired by another method. For example, the user position information is acquired by using the GPS function and transmitted to the server 134. As another example, the user position information may be position information registered in advance in the terminal device 14 or device position information of the apparatus registered in advance in the apparatus. For example, when the user uses the image forming system at or near the position of the device, since the position of the device can be regarded as the user's position, the device position information of the device can be used as the position information of the user. Good. In this case, the specifying unit 144 acquires device identification information from the device as user identification information. The device position information may also be registered in advance in the device.

  Further, the specifying unit 144 may specify the current usage status of each of the telephones A and B and the printer A by referring to the device management information 140. For example, the specifying unit 144 acquires the usage status information of each of the telephones A and B and the printer A. In the example shown in FIG. 26, as indicated by reference numerals 164 and 168, the telephone A and the printer A are available immediately, and the telephone B is currently unavailable. For example, the device can be used when it is not used by another user or when a failure has not occurred. On the other hand, when the device is used by another user or when a failure occurs, the device cannot be used. Usage status information indicating the current usage status is transmitted from the server 134 to the terminal device 14 as information related to a device group to be connected, and is displayed on the UI unit 50 of the terminal device 14. Thereby, in selecting a device to be used, useful information is provided to the user regarding use timing and the like.

  A reservation process for preferentially using a device to be connected may be performed. For example, when the user designates a function to be used using the terminal device 14, the control unit 52 of the terminal device 14 reserves for preferential use of a device to be connected to execute the function to be used. Information is transmitted to the server 134. In the server 134, the control unit 142 sets a reservation for a reservation target device, that is, a device to be connected. As an example, when there is a device that cannot be used because it is currently used by another user in the device group to be connected, reservation processing for next use may be performed. For example, when the user uses the terminal device 14 to designate a device that cannot be used (for example, the telephone B) and instruct the reservation, the control unit 52 of the terminal device 14 includes the device identification information of the designated device. Reservation information indicating a reservation for the next use of the device is transmitted to the server 134. In the server 134, the control unit 142 sets a reservation for a reservation target device (for example, the telephone set B). This makes it possible to use the reservation target device after the use of another user. For example, the control unit 142 issues a reservation number for using the reservation target device when it becomes available, and the reservation number is used as the device identification information of the reservation target device in the device management information 140. Associate. In the reservation state, use of the device is permitted by using the reservation number, and use of the device is not permitted when the reservation number is not used. Information indicating the reservation number is transmitted from the server 134 to the terminal device 14 and displayed on the UI unit 50 of the terminal device 14. When the reservation target device becomes available, the user uses the reservation target device using the reservation number. For example, the reservation target is permitted to be used by inputting the reservation number to the reservation target device or transmitting the reservation number to the server 134 using the terminal device 14. Note that when a preset time has elapsed since the reservation start time, the reservation state is canceled, and even a user who has not made a reservation may be permitted to use. Further, when it is desired to interrupt and use a device that has already been reserved, an interrupt notification process may be executed in the same manner as in the modification according to the second embodiment.

  Further, when a plurality of users request to use the same device, the connection may be permitted according to the execution priority as in the modification according to the second embodiment, and the priority is determined by the terminal. It may be displayed on the UI unit 50 of the device 14.

  When using a device, for example, as described with reference to FIG. 21, information indicating a connection request is transmitted from the terminal device 14 to a device to be used, and communication is established between the terminal device 14 and the device. Is done. For example, when the telephone A and the printer A are used as devices to be linked, information indicating a connection request is transmitted from the terminal device 14 to the telephone A and the printer A, whereby the terminal device 14, the telephone A, and the printer are transmitted. Communication is established between A and information indicating the content of the conversation on the telephone A is printed by the printer A.

  As described above, according to the fourth embodiment, information indicating a device group to be connected according to a function to be used is displayed on the terminal device 14. As a result, information is provided to the user so that the device group capable of executing the function to be used can be identified. In addition, since the functions to be used vary depending on the devices that can be used for each user and the functions that can be used among the functions that are installed in the devices, there is a limit on the search for linked functions displayed on the terminal device 14 for each user. Or the linkage functions that can be executed may be limited. By doing so, for example, if there is an electronic document that cannot be decrypted and decrypted without executing a specific link function (a link function that uses a specific function of a specific device), security is improved. It becomes possible to improve.

  Note that the control unit 52 of the terminal device 14 may cause the UI unit 50 to display information related to a device to which the terminal device 14 is to be newly connected, except for information related to a device to which the terminal device 14 has already been connected. . For example, in the case where the telephone A and the printer A are used as devices to be linked, communication has already been established between the terminal device 14 and the telephone A, and communication between the terminal device 14 and the printer A is possible. Is not established, the control unit 52 causes the UI unit 50 to display the device identification information of the printer A without displaying the device identification information and device management information of the telephone set A on the UI unit 50. The control unit 52 may display device management information related to the printer A on the UI unit 50. Information about devices that do not need to be connected after connection is completed is not displayed, but information about devices that are not connected and need connection work is displayed. Compared to the case where information is also displayed, it is easier to determine whether or not each device to be used is a device that requires connection work.

  In addition, the control unit 52 of the terminal device 14 may cause the UI unit 50 to display information indicating connection means corresponding to the device to be connected. The connection means is the above-described marker type AR technology, markerless type AR technology, position information AR technology, network connection, and the like. For example, in the device management information 140, device identification information and connection means information indicating connection means suitable for the equipment are associated with each equipment. A device provided with a mark such as a two-dimensional barcode in which device identification information is encoded is a device suitable for the marker type AR technology. The device identification information of the device includes a marker type as connection means information. Information indicating the AR technology is associated. When appearance image data of a device is generated and included in the above-described appearance image association information, the device is a device suitable for the markerless AR technology, and the device identification information of the device includes connection means. Information indicating the markerless AR technology is associated with the information. When the position information of the device is acquired and included in the above-described position association information, the device is a device suitable for the position information AR technology, and the device identification information of the device includes connection means information. , Information indicating the position information AR technique is associated. When the device group to be connected is specified, the specifying unit 144 of the server 134 refers to the device management information 140 and specifies a connection unit for each device to be connected. Information indicating the connection means is transmitted from the server 134 to the terminal device 14 and displayed on the UI unit 50 of the terminal device 14. For example, information indicating connection means is displayed for each device to be connected. To explain with a specific example, when the telephone A as a device to be connected is a device suitable for the marker-type AR technology, the UI unit 50 of the terminal device 14 has a marker-type AR technology as a connection means for the telephone A. Is displayed. Note that, regardless of which connection means is used, if it is predetermined that a connection request is not permitted for a user who has requested connection, the device may not be displayed. As a result, it is convenient to know how to connect to the device to be connected.

  Note that the first embodiment and the fourth embodiment may be combined. For example, the UI unit 50 of the terminal device 14 displays a function group purchased by the user, that is, a function group that can be used by the user, and when a specific function is selected from the function group by the user, Information indicating a device or a device group to be connected to execute the function is displayed on the UI unit 50. When a link function is selected, information indicating the device group to be linked that can be executed by linking the link function is displayed. When a function that can be executed by a single device is selected, the device that can execute the function Is displayed.

  Each of the image forming apparatus 10, the servers 12, 80, 124, 134, the terminal device 14, and the devices 76, 78 is realized by cooperation of hardware resources and software as an example. Specifically, each of the image forming apparatus 10, the servers 12, 80, 124, and 134, the terminal device 14, and the devices 76 and 78 include one or more processors such as a CPU (not shown). The one or more processors read and execute a program stored in a storage device (not shown), so that each unit of the image forming apparatus 10, the servers 12, 80, 124, 134, the terminal device 14, and the devices 76, 78 Function is realized. The program is stored in the storage device via a recording medium such as a CD or DVD, or via a communication path such as a network. Alternatively, each unit of the image forming apparatus 10, the servers 12, 80, 124, 134, the terminal device 14, and the devices 76, 78 may be realized by hardware resources such as a processor and an electronic circuit. In the realization, a device such as a memory may be used. As another example, each part of the image forming apparatus 10, the servers 12, 80, 124, and 134, the terminal device 14, and the devices 76 and 78 is realized by a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or the like. May be.

  10 image forming apparatus, 12, 80, 124, 134 server, 14 terminal apparatus, 16, 26, 44, 82, 126, 136 communication unit, 18 image forming unit, 20, 28, 48, 84, 128, 138 storage unit , 22, 50 User interface unit (UI unit), 24, 36, 52, 88, 130, 142 Control unit, 30 Device function information, 32 Function purchase history information, 34 Function execution unit, 38 Purchase processing unit, 40 Purchase history Management unit, 42, 90, 132, 144 identification unit, 46 cameras, 76, 78 devices, 80 servers, 86 linkage function information, 140 device management information.

Claims (10)

Photographing means for photographing the first device and the second device;
Identification means for identifying the photographed first device and the second device;
Information including address information of the first device or the second device is included in an instruction to execute a cooperation function executed by linking the first device and the second device identified by the identification unit. Transmitting means for transmitting to the device and the second device;
An information processing apparatus. The transmission means transmits information including the address information of the second device in the execution instruction to the first device, and transmits information including the address information of the first device to the second device in the execution instruction. Send,
The information processing apparatus according to claim 1. The first device and the second device communicate with each other according to address information of other devices included in the information transmitted to the device, and execute the cooperation function.
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. The first device and the second device have different functions;
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. The cooperation function is a function that could not be used before cooperation.
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. The linkage function that is executed when a first image associated with the first device and a second image associated with the second device are displayed on a display unit, and the user operates the first image and the second image. A control means for controlling the display of
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. User identification means for identifying the user;
The control means causes the display means to display information on the cooperation function that can be used by the identified user.
The information processing apparatus according to claim 6. The identification unit identifies the first device by photographing the identification information provided in the first device by the photographing unit, and the identification information provided in the second device is photographed by the photographing unit. Identifying the second device by
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. The identification unit identifies the first device by photographing the appearance of the first device by the photographing unit, and identifies the second device by photographing the appearance of the second device by the photographing unit. Identify,
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. Computer
Identification means for identifying the first device and the second device photographed by the photographing means;
Information including address information of the first device or the second device is included in an instruction to execute a cooperation function executed by linking the first device and the second device identified by the identification unit. Transmitting means for transmitting to the device and the second device;
Program to function as.