JP2017173927A - Information processing device, information processing system, service processing execution control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device, an information processing system, a service processing execution control method, and a program capable of shortening time required for executing service processing by an object terminal of the service processing.SOLUTION: The information processing device comprises: a terminal identification information storage part 305 for storing terminal identification information for identifying terminal equipment 100 and a face image of a user who carries the terminal equipment 100 in association; an image acquisition part 307 for acquiring pickup images obtained by imaging a plurality of users each of whom has the terminal equipment 100 in time series; a recognition part 309 for recognizing a first gesture on the basis of the pickup images acquired in time series, and for extracting the face image of the user who has performed the first gesture from the pickup images; a selection part 311 for selecting the terminal equipment 100 indicated by the terminal identification information associated with the face image similar to the extracted face image as an object terminal of service processing; and an execution control part 313 for communicating with the selected terminal equipment 100, and for controlling the execution of the service processing.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an information processing apparatus, an information processing system, a service process execution control method, and a program.

  Conventionally, a system in which an information processing device such as a projector and a plurality of terminal devices are connected via a network is known (see, for example, Patent Document 1). Examples of the terminal device include smart terminals such as tablets and smartphones, PCs (Personal Computers), and the like.

  In such a system, the user selects a terminal device that outputs a projection target image to the information processing device from among the plurality of terminal devices, and the information processing device outputs the projection target image output from the selected terminal device. Is projected to execute service processing (image projection).

  However, in the conventional technology as described above, in order to select a service processing target terminal, the user needs to find the service processing target terminal from a list of a plurality of terminal devices. It took time to execute the service process.

  The present invention has been made in view of the above circumstances, and an information processing apparatus, an information processing system, a service process execution control method, and a service process execution control method capable of reducing the time required for executing a service process on a target terminal of the service process The purpose is to provide a program.

  In order to solve the above-described problems and achieve the object, an information processing apparatus according to an aspect of the present invention corresponds to terminal identification information for identifying a terminal device and a face image of a user who owns the terminal device. A terminal identification information storage unit that is stored in addition, an image acquisition unit that acquires, in time series, captured images obtained by capturing a plurality of users each possessing a terminal device, and a first image based on the captured images acquired in the time series. A recognition unit for recognizing one gesture and extracting the face image of the user who performed the first gesture from the captured image, and terminal identification information associated with a face image similar to the extracted face image. A selection unit that selects a terminal device to be indicated as a target terminal for service processing, and an execution control unit that communicates with the selected terminal device and controls execution of the service processing.

  According to the present invention, there is an effect that it is possible to reduce the time required for executing the service process on the target terminal of the service process.

FIG. 1 is a block diagram illustrating an example of the configuration of the information processing system according to the present embodiment. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the terminal device according to the present embodiment. FIG. 3 is a block diagram illustrating an example of a hardware configuration of the imaging apparatus according to the present embodiment. FIG. 4 is a block diagram illustrating an example of a hardware configuration of the information processing apparatus according to the present embodiment. FIG. 5 is a block diagram illustrating an example of a functional configuration of the terminal device, the imaging device, and the information processing device of the present embodiment. FIG. 6 is an explanatory diagram illustrating an example of a recognition method according to the present embodiment. FIG. 7 is an explanatory diagram illustrating an example of a recognition method according to the present embodiment. FIG. 8 is an explanatory diagram illustrating an example of a recognition method according to the present embodiment. FIG. 9 is an explanatory diagram illustrating an example of a recognition method according to the present embodiment. FIG. 10 is an explanatory diagram illustrating an example of a recognition method according to the present embodiment. FIG. 11 is an explanatory diagram illustrating an example of a recognition method according to the present embodiment. FIG. 12 is a diagram illustrating an example of information stored in the terminal identification information storage unit of the present embodiment. FIG. 13 is an explanatory diagram illustrating an example of a face image similar to the extracted face image of the present embodiment. FIG. 14 is an explanatory diagram illustrating an example of a face image similar to the extracted face image of the present embodiment. FIG. 15 is a diagram illustrating an example of a recognition accuracy setting screen according to the present embodiment. FIG. 16 is a sequence diagram illustrating an example of processing executed in the information processing system of this embodiment. FIG. 17 is a flowchart illustrating an example of processing executed by the recognition unit of the present embodiment. FIG. 18 is an explanatory diagram of the first modification. FIG. 19 is a diagram illustrating an example of information stored in the terminal identification information storage unit according to the first modification. FIG. 20 is a diagram illustrating an example of information stored in the gesture image storage unit according to the second modification. FIG. 21 is a diagram illustrating an example of information stored in the gesture image storage unit according to the third modification. FIG. 22 is a block diagram illustrating an example of functional configurations of a terminal device, an imaging device, and an information processing device according to the fourth modification. FIG. 23 is a diagram illustrating an example of information stored in the operation availability information storage unit of the fourth modification.

  Hereinafter, embodiments of an information processing apparatus, an information processing system, a service process execution control method, and a program according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating an example of the configuration of the information processing system 1 according to the present embodiment. As illustrated in FIG. 1, the information processing system 1 includes terminal devices 100-1 to 100 -N (N is a natural number of 2 or more), an imaging device 200, and an information processing device 300.

  The terminal devices 100-1 to 100-N, the imaging device 200, and the information processing device 300 are connected via the network 2. The network 2 can be realized by, for example, a LAN (Local Area Network) or the Internet. Note that the imaging apparatus 200 and the information processing apparatus 300 may be directly connected via a communication cable or the like instead of the network 2.

  In the following description, the terminal devices 100-1 to 100-N are simply referred to as the terminal device 100 when it is not necessary to distinguish each of the terminal devices 100-1 to 100-N.

  Examples of the terminal device 100 include smart terminals such as tablets and smartphones, and PCs (Personal Computers). In the present embodiment, the case where the terminal device 100 is a tablet will be described as an example, but the present invention is not limited to this.

  FIG. 2 is a block diagram illustrating an example of a hardware configuration of the terminal device 100 according to the present embodiment. As shown in FIG. 2, the terminal device 100 includes a control device 11 such as a CPU (Central Processing Unit), a main storage device 12 such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and an HDD (Hard Disk). Drive) and an auxiliary storage device 13 such as an SSD (Solid State Drive), a display device 14 such as a touch panel and an input device 15, a communication device 16 such as a communication interface, and an imaging device 17 such as an image sensor. The hardware configuration uses a normal computer.

  Examples of the imaging device 200 include a video camera. FIG. 3 is a block diagram illustrating an example of a hardware configuration of the imaging apparatus 200 according to the present embodiment. As illustrated in FIG. 3, the imaging apparatus 200 includes a control device 21 such as a CPU, a main storage device 22 such as a ROM and a RAM, an auxiliary storage device 23 such as an HDD and an SSD, a display device 24 such as a touch panel, and an input. A device 25, a communication device 26 such as a communication interface, and an imaging device 27 such as an image sensor are provided, and a hardware configuration using a normal computer is provided.

  The information processing apparatus 300 is, for example, an image forming apparatus such as a projector, a camera, an air conditioner, a refrigerator, a fluorescent lamp, a vending machine, a handheld terminal, and various other electronic devices, a printing apparatus, a copier, a multifunction machine, a scanner apparatus, and a facsimile apparatus. Examples include devices, smart terminals, and PCs. The multifunction peripheral has at least two functions among a copying function, a printing function, a scanner function, and a facsimile function. In the present embodiment, the case where the information processing apparatus 300 is a projector will be described as an example, but the present invention is not limited to this.

  FIG. 4 is a block diagram illustrating an example of a hardware configuration of the information processing apparatus 300 according to the present embodiment. As illustrated in FIG. 4, the information processing apparatus 300 performs control processing 31 such as a CPU, a main storage device 32 such as a ROM and a RAM, an auxiliary storage device 33 such as an HDD and an SSD, and image processing related to image projection. An engine 34 to be performed, a display device 35 such as a touch panel and an input device 36, and a communication device 37 such as a communication interface are provided, and a hardware configuration using a normal computer is provided.

  In the present embodiment, the case where the information processing system 1 is used for a lesson at school will be described as an example. However, the present invention is not limited to this. Specifically, a teacher and a plurality of students in the classroom each have the terminal device 100, and the imaging device 200 images the classroom in time series so that all of the teacher and the plurality of students can be accommodated, The captured time-series images are output to the information processing apparatus 300. Then, the information processing apparatus 300 identifies the teacher or student who performed the predetermined gesture from the time-series images output from the imaging apparatus 200, and displays the terminal device 100 possessed by the identified teacher or student as the projection target image. The terminal to be output to the information processing apparatus 300 is selected, and the projection target image output from the selected terminal apparatus 100 is projected.

  As described above, in the information processing system 1 according to the present embodiment, the selection of the terminal device 100 that outputs the projection target image to the information processing device 300 is automated by recognizing the gesture of the teacher or student (for example, a raised hand). Therefore, it does not take time to select the terminal device 100, the projection target image of the terminal device 100 can be projected immediately from the information processing device 300, and the smooth progress of the lesson can be supported without hindering the progression of the lesson. . In the following description, teachers and students may be referred to as users.

  FIG. 5 is a block diagram illustrating an example of functional configurations of the terminal device 100, the imaging device 200, and the information processing device 300 according to the present embodiment.

  As illustrated in FIG. 5, the terminal device 100 includes an imaging unit 101, a registration request unit 103, and an execution control unit 105. The imaging unit 101 can be realized by the imaging device 17 or the like, for example. The registration request unit 103 and the execution control unit 105 can be realized by the control device 11, the main storage device 12, the communication device 16, and the like, for example.

  As illustrated in FIG. 5, the imaging apparatus 200 includes an imaging unit 201 and an image transmission unit 203. The imaging unit 201 can be realized by the imaging device 27, for example. The image transmission unit 203 can be realized by the control device 21, the main storage device 22, the communication device 26, and the like, for example.

  As illustrated in FIG. 5, the information processing apparatus 300 includes a registration unit 301, a gesture image storage unit 303, a terminal identification information storage unit 305, an image acquisition unit 307, a recognition unit 309, a selection unit 311, An execution control unit 313.

  The registration unit 301, the image acquisition unit 307, and the execution control unit 313 can be realized by the control device 31, the main storage device 32, the communication device 37, and the like, for example. The gesture image storage unit 303 and the terminal identification information storage unit 305 can be realized by the auxiliary storage device 33, for example. The recognition unit 309 and the selection unit 311 can be realized by the control device 31 and the main storage device 32, for example.

  The imaging unit 101 captures a gesture image indicating a gesture to be recognized by the recognition unit 309, or captures a face image of a user (teacher or student) who owns the terminal device 100. In the present embodiment, the case where the recognition target gesture of the recognition unit 309 is a hand is described as an example, but the present invention is not limited to this.

  The registration request unit 103 identifies, to the information processing device 300, a gesture image captured by the image capturing unit 101, a face image of a user possessing the terminal device 100 captured by the image capturing unit 101, and the terminal device 100. Request registration of pairs with information. In the present embodiment, the case where the terminal identification information is the IP address of the terminal device 100 will be described as an example, but the present invention is not limited to this. The terminal identification information is stored in the auxiliary storage device 13 or the like, for example.

  In the present embodiment, it is assumed that the registration request for the gesture image and the registration request for the set of the face image and the terminal identification information are performed when the terminal device 100 and the information processing device 300 are connected. It is not limited to. The gesture image registration request may be made by any terminal device 100, and need not be made by all the terminal devices 100. On the other hand, a registration request for a set of a face image and terminal identification information needs to be made by all the terminal devices 100.

  Upon receiving a gesture image registration request from the terminal device 100, the registration unit 301 registers the gesture image in the gesture image storage unit 303. When the registration unit 301 receives a registration request for a set of a face image and terminal identification information from the terminal device 100, the registration unit 301 registers the set of the face image and terminal identification information in the terminal identification information storage unit 305. Note that the registration unit 301 deletes the set of the face image and the terminal identification information of the terminal device 100 from the terminal identification information storage unit 305 when the terminal device 100 and the information processing device 300 are disconnected.

  The imaging unit 201 images the classroom in time series so that all of a plurality of users (teachers and a plurality of students) each possessing the terminal device 100 can be accommodated. Note that the time-series captured images captured by the imaging unit 201 may be moving images or may be images obtained by periodically capturing still images.

  The image transmission unit 203 transmits the time-series captured images captured by the imaging unit 201 to the information processing apparatus 300.

  The image acquisition unit 307 acquires captured images obtained by capturing a plurality of users (teachers and students) each possessing the terminal device 100 from the imaging device 200 in time series.

  The recognition unit 309 recognizes the first gesture based on the captured image acquired in time series by the image acquisition unit 307, and extracts the face image of the user who performed the first gesture from the captured image. In the present embodiment, the case where the first gesture is a left hand raising hand will be described as an example, but the present invention is not limited to this.

  Specifically, the recognition unit 309 includes the previous captured image (for example, the image illustrated in FIG. 6) acquired by the image acquisition unit 307 and the latest captured image (for example, FIG. 7) acquired by the image acquisition unit 307. And the difference image (for example, the image shown in FIG. 8) indicating the extracted difference is generated. In the image shown in FIG. 6, no one raises his hand, but in the image shown in FIG. 7, the left user raises his left hand, so the difference image shown in FIG. 8 is raised by the left user. Has left arm.

  Then, the recognition unit 309 performs template matching on the difference image using the gesture image stored in the gesture image storage unit 303 as a template, and searches for a gesture indicated by the gesture image. For example, when the gesture image 501 raising the left arm shown in FIG. 9 is stored in the gesture image storage unit 303, the recognition unit 309 uses the gesture image 501 as a template as shown in FIG. Perform template matching on images. As a result, as shown in FIG. 10, the position of the left arm template-matched on the difference image shown in FIG. 8 (specifically, the upper left coordinates (X, Y) of the rectangle surrounding the left arm) is specified.

  Further, the recognition unit 309 raises the left hand from the latest captured image (image shown in FIG. 7) acquired in time series by the image acquisition unit 307 based on the specified position (X, Y) of the left arm. The face image of (user on the left) is cut out. In the present embodiment, since the recognized first gesture is the left hand raising hand, the recognition unit 309 uses (XZ, Y), with the upper left of the image shown in FIG. A rectangle having apexes (X, Y), (X−Z, Y + Z), and (X, Y + Z) is cut out as a face image of the user who raised his left hand (left user). Note that the position of the face image to be cut out is not limited to this, and varies depending on the pattern of the first gesture.

  The selection unit 311 selects the terminal device 100 indicated by the terminal identification information associated with the face image similar to the face image extracted by the recognition unit 309 as the target terminal for service processing. Specifically, as shown in FIG. 12, terminal identification information storage unit 305 stores terminal identification information for identifying terminal device 100 and a face image of the user who owns terminal device 100 in association with each other. Therefore, the selection unit 311 acquires terminal identification information associated with a face image similar to the face image extracted by the recognition unit 309, and uses the terminal device 100 indicated by the acquired terminal identification information for service processing. Select the target terminal.

  Note that the face image similar to the face image extracted by the recognition unit 309 is the face most similar to the face image extracted by the recognition unit 309 among the face images stored in the terminal identification information storage unit 305, for example. It can be an image. For example, in the case of the example illustrated in FIG. 13, the face image 514 having the highest degree of similarity with the face image 511 extracted by the recognition unit 309 among the face images 512 to 515 stored in the terminal identification information storage unit 305. The face image is similar to the face image 511 extracted by the recognition unit 309. In the example illustrated in FIG. 13, the similarity is represented by a numerical value of 0 to 1, and the similarity increases as the value approaches 1.

  The face image similar to the face image extracted by the recognition unit 309 may be a condition that the similarity is equal to or higher than a threshold value. In this way, even if an image that is not a face is erroneously detected as a face image, the terminal device 100 of the user who has not raised his hand is erroneously selected as a terminal that outputs the projection target image to the information processing device 300 Can be prevented. For example, in the case of the example shown in FIG. 14, the similarity between the face image 522 to 525 stored in the terminal identification information storage unit 305 and the image 521 erroneously detected by the recognition unit 309 is greater than or equal to the threshold value. There is nothing. Therefore, it is possible to prevent the user's terminal device 100 of the face image 523 having the highest similarity with the image 521 from being erroneously selected as a terminal that outputs the projection target image to the information processing device 300.

  Note that the selection unit 311 may adjust the threshold value. For example, the selection unit 311 displays a recognition accuracy setting screen as illustrated in FIG. 15 on the display device 35, and sets the level of recognition accuracy based on an operation input from the input device 36 by the user, thereby setting a threshold value. The value of may be adjusted. In this way, an optimum threshold value can be set according to the use environment (brightness etc.).

  The execution control unit 313 communicates with the terminal device 100 selected by the selection unit 311 and controls the execution of service processing. Specifically, the execution control unit 313 communicates with the terminal device 100 selected by the selection unit 311 and controls the execution of service processing in cooperation with the execution control unit 105 of the terminal device 100.

  In the present embodiment, the service process is a projection of an image to be projected, and is an output of content held by the terminal device 100 by the information processing device 300. Therefore, the execution control unit 313 has the terminal device 100 selected by the selection unit 311. To request a projection target image. The execution control unit 105 captures the screen displayed on the display device 14 of the terminal device 100, sets it as a projection target image, and outputs it to the information processing device 300. Then, the execution control unit 313 projects the projection target image output from the terminal device 100.

  Note that the content output of the terminal device 100 by the information processing apparatus 300 is not limited to the projection of the projection target image, and may be printing of the print target image. In addition, the service processing may be distribution of content of the information processing device 300 to the terminal device 100, device control (door unlocking) according to the security level of the terminal device 100 by the information processing device 300, or the like. .

  FIG. 16 is a sequence diagram illustrating an example of processing executed in the information processing system 1 of the present embodiment.

  First, the imaging unit 101 captures a gesture image indicating a gesture to be recognized by the recognition unit 309 (step S101).

  Subsequently, the registration request unit 103 requests the information processing apparatus 300 to register the gesture image captured by the imaging unit 101 (step S103).

  Subsequently, when receiving a gesture image registration request from the terminal device 100, the registration unit 301 registers the gesture image in the gesture image storage unit 303 (step S105).

  Subsequently, the imaging unit 101 captures a face image of the user (teacher or student) who owns the terminal device 100 (step S107).

  Subsequently, the registration request unit 103 requests the information processing device 300 to register a set of a face image of the user who has the terminal device 100 captured by the imaging unit 101 and terminal identification information for identifying the terminal device 100. (Step S109).

  Subsequently, when the registration unit 301 receives a registration request for a set of a face image and terminal identification information from the terminal device 100, the registration unit 301 registers the set of the face image and terminal identification information in the terminal identification information storage unit 305 (Step S1). S111).

  Subsequently, the imaging unit 201 images a space such as a classroom in time series so that all of a plurality of users (teacher and a plurality of students) each possessing the terminal device 100 can be accommodated (step S121).

  Subsequently, the image transmission unit 203 transmits the time-series captured images captured by the imaging unit 201 to the information processing apparatus 300, and the image acquisition unit 307 acquires the captured images from the imaging apparatus 200 in time series ( Step S123).

  Subsequently, the recognition unit 309 recognizes the first gesture based on the captured image acquired in time series by the image acquisition unit 307, and extracts the face image of the user who performed the first gesture from the captured image. (Step S125).

  Subsequently, the selection unit 311 selects the terminal device 100 indicated by the terminal identification information associated with the face image similar to the face image extracted by the recognition unit 309 as the target terminal (projection target terminal device) for service processing. (Step S127).

  Subsequently, the execution control unit 313 requests a projection target image from the terminal device 100 selected by the selection unit 311 (step S129).

  Subsequently, the execution control unit 105 captures the screen displayed on the display device 14 of the terminal device 100 as a projection target image (step S131), and outputs it to the information processing device 300 (step S133).

  Subsequently, the execution control unit 313 projects the projection target image output from the terminal device 100 (step S135).

  FIG. 17 is a flowchart illustrating an example of processing executed by the recognition unit 309 according to the present embodiment.

  First, the recognition unit 309 acquires the latest captured image (for example, the image shown in FIG. 7) acquired by the image acquisition unit 307 (step S201).

  Subsequently, the recognition unit 309 extracts a difference between the latest captured image (for example, the image illustrated in FIG. 7) and the previous captured image (for example, the image illustrated in FIG. 6) acquired by the image acquisition unit 307. Then, a difference image (for example, the image shown in FIG. 8) indicating the extracted difference is generated (step S203).

  Subsequently, the recognition unit 309 performs template matching on the difference image using the gesture image stored in the gesture image storage unit 303 as a template, and searches for a predetermined gesture indicated by the gesture image (step S205).

  When the gesture is searched (Yes in step S207), the recognition unit 309 detects the partial image near the searched gesture from the latest captured image (the image shown in FIG. 7), and the face of the user who performed the predetermined gesture Cut out as an image (step S209).

  If no gesture is found (No in step S207), the process in step S209 is not executed.

  As described above, according to the present embodiment, the selection of the terminal device 100 that outputs the projection target image to the information processing device 300 is automated by recognizing the gesture (for example, the raising hand) of the user (teacher or student). Therefore, it does not take time to select the terminal device 100, the projection target image of the terminal device 100 can be immediately projected from the information processing device 300, and the smooth progress of the lesson is supported without impeding the progression of the lesson. it can.

(Modification 1)
In the above embodiment, the recognition unit 309 may recognize the first gesture based on a captured image acquired in time series during a certain time. In this way, as shown in FIG. 18, even if different users raise their hands for a certain period of time, they raise their hands second instead of raising their hands first. It is also possible to select the terminal device 100 possessed by.

  In the example illustrated in FIG. 18, when gesture recognition is performed using the captured image 602 as the latest captured image and the captured image 601 as the previous captured image, the left user's hand is recognized, and the captured image 603 is captured as the latest captured image. When gesture recognition is performed using the image and the captured image 602 as the previous captured image, the hand raised by the right user is recognized.

  Moreover, about a fixed time, you may enable it for a user to set time like the threshold value demonstrated in the said embodiment.

  When the recognition unit 309 recognizes a plurality of first gestures, the recognition unit 309 extracts face images of a plurality of users who have performed the first gesture.

  In Modification 1, as shown in FIG. 19, the terminal identification information storage unit 305 stores terminal identification information for identifying the terminal device 100, a face image of the user who owns the terminal device 100, and the terminal device. The priority of the user who owns 100 is stored in association with each other. The priority is higher as the value is smaller, but is not limited to this.

  Then, the selection unit 311 selects the terminal device 100 indicated by the terminal identification information associated with the face image having the highest priority among the face images similar to each of the plurality of face images extracted by the recognition unit 309 as the target of service processing. Select to the terminal.

  As described above, when a plurality of users raise their hands within a certain time, the terminal device 100 possessed by the user with the highest priority is selected as a target terminal for service processing. For example, if the priority of the teacher is set higher than that of the student, when the student and the teacher raise their hands, the terminal device 100 possessed by the teacher is preferentially selected as the target terminal for service processing.

(Modification 2)
In the above embodiment, each of the plurality of types of gestures may be the first gesture. Specifically, the imaging unit 101 captures a plurality of types of gesture images indicating gestures to be recognized by the recognition unit 309, and the registration request unit 103 captures the information processing apparatus 300 from the plurality of types captured by the imaging unit 101. When the registration unit 301 receives a registration request for a plurality of types of gesture images from the terminal device 100, the registration unit 301 stores the plurality of types of gesture images in the gesture image storage unit 303 as illustrated in FIG. You may register.

  In this way, since gestures are recognized using a plurality of types of gesture images, it is possible to prepare various gesture patterns for causing a service processing target terminal to select.

(Modification 3)
In the second modification, an operation corresponding to the gesture may be executed in the service process for each type of gesture. For example, as shown in FIG. 21, information that associates a gesture image with an operation example corresponding to the gesture indicated by the gesture image may be registered in the gesture image storage unit 303. In this case, the registration request unit 103 may make a registration request including a pair of a gesture image and information indicating an operation example corresponding to the gesture indicated by the gesture image.

  In the third modification, it is assumed that the first gesture is a start gesture for starting a service process. Specifically, the first gesture is a gesture of raising the right hand whose ID is 1 in FIG. 21 and is a gesture for starting projection of the projection target image. In addition, the second gesture is an end gesture for ending service processing. Specifically, the second gesture is a gesture that raises the left hand whose ID is 2 in FIG. 21 and is a gesture that ends the projection of the projection target image.

  In this case, the operations of the recognition unit 309, the selection unit 311, and the execution control unit 313 when the recognition unit 309 recognizes the first gesture are the same as those in the embodiment. Note that the execution control unit 313 communicates with the terminal device 100 selected as the service processing target terminal, and starts executing the service processing.

  On the other hand, when the recognizing unit 309 recognizes the second gesture, the execution control unit 313 communicates with the terminal device 100 in which the service process is started, and causes the service process to execute an operation corresponding to the second gesture. Specifically, the execution control unit 313 communicates with the terminal device 100 where the service process is started, and ends the service process.

  For example, the execution control unit 313 communicates with the terminal device 100 where the projection of the projection target image is started, and ends the projection of the projection target image. Specifically, the execution control unit 313 requests the terminal device 100 that has started projecting the projection target image to stop outputting the projection target image, and the execution control unit 105 displays the display on the display device 14 of the terminal device 100. The screen capture and the output of the projection target image to the information processing apparatus 300 are stopped, and the execution control unit 313 stops the projection of the projection target image.

  In this way, various operations can be executed in service processing using a plurality of types of gestures.

(Modification 4)
In the third modification, whether or not to execute the operation according to the second gesture may be controlled based on whether or not the user who performed the second gesture is the same person as the user who performed the first gesture.

  FIG. 22 is a block diagram illustrating an example of functional configurations of the terminal device 100, the imaging device 200, and the information processing device 1300 according to the fourth modification. As illustrated in FIG. 22, the information processing device 1300 further includes an operation availability information storage unit 1315.

  The operation availability information storage unit 1315 is information in which the target person is associated with the operation availability. In the example illustrated in FIG. 23, if the target person is the same person (the same person), the operation is permitted, and the target person is If it is not the person (other person), the operation is not permitted.

  In the fourth modification, the recognition unit 309 extracts the face image of the user who performed the second gesture from the captured image, and the selection unit 311 selects a face image similar to the extracted face image.

  Then, the execution control unit 313 matches the face image similar to the face image of the user who performed the first gesture with the face image similar to the face image of the user who performed the second gesture, and operation availability information In the storage unit 1315, when the principal operation availability information that defines whether or not an operation can be performed by the person who started the service process (operation availability when the target person is the person) indicates that the operation is possible, the terminal device 100 that has started the service process To finish the service processing.

  Further, the execution control unit 313 does not match the face image similar to the face image of the user who performed the first gesture and the face image similar to the face image of the user who performed the second gesture, and whether or not the operation can be performed. In the information storage unit 1315, the service process is started when the other person operation availability information that defines whether or not an operation other than the person who started the service process is permitted (operation availability when the target person is not the person) indicates the operation is possible. It communicates with the terminal device 100 and ends the service process.

  In this way, it is possible to prevent an operation related to service processing by a person other than the person who started the service processing.

  In the example shown in FIG. 23, if the target person is the person (same person), the operation is not permitted, and if the target person is other than the person (other person), the operation may be permitted. In this way, the service process started by the student can be terminated by the teacher.

(program)
A program executed by the terminal device 100, the imaging device 200, and the information processing device 300 (hereinafter referred to as “each device of the above-described embodiment and each modified example”) of the above-described embodiment is an installable format or executable The format file is provided by being stored in a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or flexible disk (FD).

  Further, the program executed by each device of the above-described embodiment and each modified example may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. In addition, the program executed by each device of the above embodiment and each modification may be provided or distributed via a network such as the Internet. Further, the program executed by each device of the above-described embodiment and each modification may be provided by being incorporated in advance in a ROM or the like.

  The program executed by each device of the embodiment and each modification has a module configuration for realizing the above-described units on a computer. As actual hardware, for example, the CPU reads out a program from the ROM to the RAM and executes the program, so that each functional unit is realized on the computer.

  The above embodiment is presented as an example, and is not intended to limit the scope of the invention. The above-described novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 Information processing system 2 Network 100-1 to 100-N (100) Terminal device 101 Imaging part 103 Registration request part 105 Execution control part 200 Imaging apparatus 201 Imaging part 203 Image transmission part 300, 1300 Information processing apparatus 301 Registration part 303 Gesture Image storage unit 305 Terminal identification information storage unit 307 Image acquisition unit 309 Recognition unit 311 Selection unit 313 Execution control unit 1315 Operation availability information storage unit

JP 2014-044738 A

Claims (13)

A terminal identification information storage unit for storing terminal identification information for identifying a terminal device and a face image of a user possessing the terminal device in association with each other;
An image acquisition unit that acquires, in time series, captured images obtained by imaging a plurality of users each possessing a terminal device;
A recognition unit that recognizes a first gesture based on the captured image acquired in time series, and extracts a face image of the user who performed the first gesture from the captured image;
A selection unit that selects, as a target terminal for service processing, a terminal device indicated by terminal identification information associated with a face image similar to the extracted face image;
An execution control unit that communicates with the selected terminal device and controls execution of the service process;
An information processing apparatus comprising:   The said selection part selects the terminal device which the terminal identification information matched with the face image whose similarity with the said extracted face image is more than a threshold value and which is the highest as a target terminal of the said service process. Information processing device.   The information processing apparatus according to claim 2, wherein the selection unit adjusts the threshold value. The terminal identification information storage unit further stores the priority in association with each other,
The recognizing unit recognizes the first gesture based on the captured images acquired in time series during a predetermined time, and performs the first gesture when a plurality of the first gestures are recognized. Multiple face images of each user
The selection unit selects, as a target terminal for the service process, a terminal device indicated by terminal identification information associated with a face image having the highest priority among face images similar to each of the extracted face images. The information processing apparatus according to any one of claims 1 to 3.   The information processing apparatus according to claim 1, wherein each of a plurality of types of gestures is the first gesture. The first gesture is a start gesture for starting the service process,
The execution control unit communicates with a terminal device selected as a target terminal for the service process, and starts executing the service process,
The recognition unit recognizes a second gesture based on the captured image acquired in the time series,
The said execution control part communicates with the terminal device from which the said service process was started, when the said 2nd gesture is recognized, and performs the operation | movement according to the said 2nd gesture in the said service process. 5. The information processing apparatus according to any one of 4. The second gesture is an end gesture for ending the service process,
The information processing apparatus according to claim 6, wherein when the second gesture is recognized, the execution control unit communicates with a terminal device that has started the service process, and ends the service process. The recognizing unit extracts a face image of a user who has performed the second gesture from the captured image;
The selection unit selects a face image similar to the extracted face image;
The execution control unit matches the face image similar to the face image of the user who performed the first gesture and the face image similar to the face image of the user who performed the second gesture, and the service The information processing according to claim 7, wherein if the principal operation availability information that defines whether or not an operation can be performed by the person who has started the processing indicates that the operation is possible, the information processing apparatus communicates with the terminal device that has started the service processing and ends the service processing. apparatus. The recognizing unit extracts a face image of a user who has performed the second gesture from the captured image;
The selection unit selects a face image similar to the extracted face image;
The execution control unit does not match a face image similar to the face image of the user who performed the first gesture and a face image similar to the face image of the user who performed the second gesture, and The apparatus according to claim 7 or 8, wherein when other person operation availability information defining whether or not an operation by a person other than the person who started the service process is operable indicates communication is possible, the service process is terminated by communicating with the terminal device from which the service process has been started. The information processing apparatus described.   The service processing includes: output of content of the terminal device by the information processing device; distribution of content of the information processing device to the terminal device; and equipment according to the security level of the terminal device by the information processing device The information processing apparatus according to claim 1, wherein the information processing apparatus is at least one of controls. A terminal identification information storage unit for storing terminal identification information for identifying a terminal device and a face image of a user possessing the terminal device in association with each other;
An imaging unit that images a plurality of users each possessing a terminal device in time series,
An image acquisition unit that acquires captured images captured by the imaging unit in time series;
A recognition unit that recognizes a first gesture based on the captured image acquired in time series, and extracts a face image of the user who performed the first gesture from the captured image;
A selection unit that selects, as a target terminal for service processing, a terminal device indicated by terminal identification information associated with a face image similar to the extracted face image;
An information processing system comprising: an execution control unit that communicates with the selected terminal device and controls execution of the service process. A terminal identification information storage step for storing the terminal identification information for identifying the terminal device and the face image of the user possessing the terminal device in association with each other;
An image acquisition step of acquiring, in time series, captured images obtained by imaging a plurality of users each possessing a terminal device;
A recognition step of recognizing a first gesture based on the captured image acquired in time series, and extracting a face image of the user who performed the first gesture from the captured image;
A selection step of selecting a terminal device indicated by terminal identification information associated with a face image similar to the extracted face image as a target terminal for service processing;
An execution control step of communicating with the selected terminal device and controlling execution of the service process;
Service processing execution control method including: A terminal identification information storage step for storing the terminal identification information for identifying the terminal device and the face image of the user possessing the terminal device in association with each other;
An image acquisition step of acquiring, in time series, captured images obtained by imaging a plurality of users each possessing a terminal device;
A recognition step of recognizing a first gesture based on the captured image acquired in time series, and extracting a face image of the user who performed the first gesture from the captured image;
A selection step of selecting a terminal device indicated by terminal identification information associated with a face image similar to the extracted face image as a target terminal for service processing;
An execution control step of communicating with the selected terminal device and controlling execution of the service process;
A program that causes a computer to execute.

Images