JP2008287691A - Information processing device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processing device and a control program for reducing user's operation burden. <P>SOLUTION: A PC 1 is connected to a projector 4 that projects an annotation image input from a PC 2 onto a projection area including an object and a background, and is connected to a video camera that photographs the projection area including the object and the background. A control part 101 of the PC 1 detects movement of an object 8 on the basis of the photographic image photographed by the video camera and extracts a changed region on the photographic image, which is caused by the movement of the object 8, and erases an annotation image (note image) when the changed region includes the annotation image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an information processing apparatus and a program.

  Conventionally, a remote instruction system including a server (for example, a computer) to which a video camera and a projector are connected and a remote client (for example, a computer) connected to the server via a network is known (for example, Patent Documents 1 and 2).

In this remote instruction system, a server transmits a captured image of an object captured by a video camera connected to the server to a client via a network. The client displays a captured image including a three-dimensional object on a display screen of a display device connected to the client, and when an annotation image (line, figure, character) or the like is written on the display screen, the annotation image is displayed on the network. To the server via The server sends the received annotation image to the projector, and the projector sends the received annotation image to an appropriate position of the projection area including the object (this is the position where the client wrote the annotation image within the captured image range). To the corresponding).
US Patent Publication No. 2004/0070674 JP 2005-33756 A

  However, in the above remote instruction system, if an annotation image is written on the object displayed on the display screen by the user of the client and then the object is moved, the annotation image becomes meaningless. Become. For this reason, the client user deletes the annotation image that has become meaningless from the display screen, but this operation takes time and effort for the client user.

  In the remote instruction system, it is desirable to appropriately store an image of an object on which an annotation image is projected in order to create a minutes of a meeting using the remote instruction system. However, since it is difficult for the user to determine an appropriate timing for saving an image, there is a possibility that an appropriate image cannot be saved. Moreover, since the user must measure an appropriate timing for image storage during the meeting, this operation may be a burden on the user.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an information processing apparatus and a program capable of reducing a user's operation burden.

  In order to achieve the above object, an information processing apparatus according to claim 1 is a projection apparatus that projects an annotation image input from an external terminal onto a projection area including an object and a background, and a projection area including the object and the background. An information processing device connected to a photographing device for photographing a detection means for detecting movement of the object based on a photographed image photographed by the photographing device; An extraction unit that extracts a change area on the photographed image; and a processing unit that executes processing on at least one of the photographed image and the annotation image when the annotation image is included in the change area. To do.

  The information processing apparatus according to claim 2 is the information processing apparatus according to claim 1, wherein the detection unit compares a captured image captured by the imaging apparatus with a captured image captured immediately after the captured image. Thus, the movement of the object is detected.

  The information processing apparatus according to claim 3 is the information processing apparatus according to claim 1 or 2, wherein the detection unit includes a photographed image photographed by the photographing apparatus and a photographed image photographed immediately after the photographed image. The movement of the object is detected by dividing the object into a plurality of small areas and comparing the divided small areas corresponding to each other.

  The information processing device according to claim 4 is the information processing device according to any one of claims 1 to 3, wherein the extraction unit includes the change area generated during the input of the annotation image from the external terminal or the The region of the annotation image in the display region is ignored based on the coordinate information of the annotation image from an external terminal.

  The information processing apparatus according to claim 5 is the information processing apparatus according to any one of claims 1 to 4, wherein the processing executed by the processing unit includes erasure of the annotation image. To do.

  The information processing apparatus according to claim 6 is the information processing apparatus according to claim 5, further comprising transmission means for transmitting information of the annotation image to be erased by the processing means to the external terminal, wherein the external terminal An image display for displaying a photographed image by the photographing apparatus and displaying an instruction unit capable of instructing the processing means to cancel the erasure of the annotation image according to the information of the annotation image to be erased from the transmission means. Means are provided.

  The information processing device according to claim 7 is the information processing device according to claim 6, wherein the image display means blinks the annotation image for a certain period of time in accordance with the information of the annotation image to be deleted from the transmission means. It is characterized by doing.

  The information processing device according to claim 8 is the information processing device according to any one of claims 5 to 7, wherein the processing unit includes the annotation when the change area includes a part of the annotation image. All the images are erased.

  The information processing apparatus according to claim 9 is the information processing apparatus according to any one of claims 5 to 8, wherein the processing unit also erases an annotation image within a range that is widened by a predetermined distance from the change area. It is characterized by that.

  The information processing apparatus according to claim 10 is the information processing apparatus according to any one of claims 5 to 9, wherein the processing unit treats a plurality of annotation images input from the external terminal as one group, When erasing at least one annotation image, another annotation image belonging to the same group is also erased.

  The information processing apparatus according to claim 11 is the information processing apparatus according to any one of claims 5 to 10, wherein the annotation image and information about a region associated with the annotation image are acquired from the external terminal. Means for erasing the annotation image when the change area includes an area associated with the annotation image.

  The information processing device according to claim 12 is the information processing device according to any one of claims 1 to 11, wherein the processing executed by the processing unit is performed before the annotation image is included in the change area. Storage of the photographed image at the time point is included.

  The information processing device according to claim 13 is the information processing device according to claim 12, wherein when the change area includes at least a part of the annotation image, the photographed image at the previous time point Is stored.

  The information processing apparatus according to claim 14 is the information processing apparatus according to claim 12 or 13, wherein the processing unit includes at least a part of the annotation image within a range that is widened by a predetermined distance from the change area. Also, the photographed image at the previous time point is stored.

  The information processing apparatus according to claim 15 is caused by the movement of the target object, a display unit that displays an annotation image input from an external terminal in a display area, a detection unit that detects the movement of the target object, and the target object. And an extraction means for extracting a change area in the display area, and a processing means for executing a process for the display in the display area when the annotation image is included in the change area.

  The information processing device according to claim 16 is the information processing device according to claim 15, wherein the extraction unit is configured to input the annotation region from the external terminal or the annotation image from the external terminal. The region of the annotation image in the display region is ignored based on the coordinate information.

  The information processing apparatus according to claim 17 is the information processing apparatus according to claim 15 or 16, wherein the processing executed by the processing means includes erasure of the annotation image.

  The information processing device according to claim 18 is the information processing device according to claim 17, wherein when the change area includes a part of the annotation image, the processing unit deletes all of the annotation image. It is characterized by.

  The information processing device according to claim 19 is the information processing device according to claim 17 or 18, characterized in that the processing means also deletes an annotation image within a range extended by a predetermined distance from the change region. .

  The information processing device according to claim 20 is the information processing device according to any one of claims 17 to 19, wherein the processing unit treats a plurality of annotation images input from the external terminal as one group, When erasing at least one annotation image, another annotation image belonging to the same group is also erased.

  The information processing device according to claim 21 is the information processing device according to any one of claims 17 to 20, wherein the annotation image and information about a region associated with the annotation image are acquired from the external terminal. Means for erasing the annotation image when the change area includes an area associated with the annotation image.

  The information processing apparatus according to claim 22 is the information processing apparatus according to any one of claims 15 to 21, wherein the processing executed by the processing unit is performed before the annotation image is included in the change area. Storage of the display in the display area at the time is included.

  23. The information processing apparatus according to claim 23, wherein the processing unit includes the display area at a previous time when the change area includes at least a part of the annotation image. It is characterized by memorizing the display inside.

  The information processing device according to claim 24 is the information processing device according to claim 22 or 23, wherein the processing means includes at least a part of the annotation image within a range extended by a predetermined distance from the change region. Also, the display in the display area at the previous time is stored.

  The information processing apparatus according to claim 25 is connected to a projection apparatus that projects an annotation image input from an external terminal onto a projection area including an object and a background, and an imaging apparatus that captures the projection area including the object and the background. An information processing apparatus for detecting a movement of the object based on a photographed image photographed by the photographing apparatus; and a change area on the photographed image caused by the movement of the object. The importance of the captured image is calculated based on at least one of extraction means for extracting, information on the change area, and information on the annotation image included in the change area, and the imaging according to the importance Storage means for executing image storage.

  The information processing apparatus according to claim 26 is caused by the movement of the target object and a display means for displaying an annotation image input from an external terminal in a display area, a detection means for detecting the movement of the target object, and the movement of the target object. The importance of display in the display area based on at least one of extraction means for extracting a change area in the display area, information on the change area, and information on the annotation image included in the change area And storage means for executing storage of the display in accordance with the degree of importance.

  An information processing apparatus according to a twenty-seventh aspect is the information processing apparatus according to the twenty-fifth or twenty-sixth aspect, wherein the storage unit executes the storage when the importance is equal to or greater than a predetermined value.

  The information processing device according to claim 28 is the information processing device according to any one of claims 25 to 27, wherein the storage unit stores the data of the importance level when executing the storage. Features.

  The information processing device according to claim 29 is the information processing device according to any one of claims 25 to 28, wherein the information on the change region used for calculating the importance includes a size of the change region, At least one of the time until the change area is generated is included, and the annotation image information included in the change area includes the number of the annotation images included in the change area, and a plurality of annotations in the change area. When an image is included, at least one of a distance between each annotation image and a time interval at which each annotation image is drawn is included.

  A program according to a thirty-third aspect is connected to a projection device that projects an annotation image input from an external terminal onto a projection region including an object and a background, and an imaging device that photographs the projection region including the object and the background. Detecting means for detecting movement of the object based on a photographed image photographed by the photographing device; extraction means for extracting a change area on the photographed image caused by movement of the object; and When the annotation image is included in the change area, it is made to function as processing means for executing processing for at least one of the photographed image and the annotation image.

  The program according to claim 31 is generated by a computer, a display unit that displays an annotation image input from an object and an external terminal in a display area, a detection unit that detects movement of the object, and the movement of the object. It is characterized by functioning as extraction means for extracting a change area in the display area, and processing means for executing processing for display in the display area when the annotation image is included in the change area.

  A program according to a thirty-second aspect is connected to a projection apparatus that projects an annotation image input from an external terminal onto a projection area including an object and a background, and an imaging apparatus that images the projection area including the object and the background. Detecting means for detecting movement of the object based on a photographed image photographed by the photographing device; extraction means for extracting a change area on the photographed image caused by movement of the object; and Storage means for calculating the importance of the captured image based on at least one of the information on the change area and the information on the annotation image included in the change area, and storing the captured image according to the importance It is made to function as.

  The program according to claim 33 is caused by a movement of the object, display means for displaying an annotation image input from the object and an external terminal in a display area, detection means for detecting movement of the object, and the computer. Based on at least one of extraction means for extracting a change area in the display area and information on the change area and information on the annotation image included in the change area, the importance of display in the display area is determined. It is calculated and functions as a storage unit for storing the display according to the importance.

  According to the first and fifteenth aspects of the invention, when the positional relationship between the object and the annotation image changes due to the movement of the object, the processing for the image or the display is automatically performed, so the operation burden on the user is reduced. can do.

  According to the second aspect of the present invention, the amount of calculation required to detect the movement of the object can be reduced as compared with the case where the process of tracking the movement of the object is executed.

  According to the invention of claim 3, it is possible to accurately detect the movement of the object without being affected by noise or the like of the photographed image.

  According to the inventions of claims 4 and 16, it is possible to ensure the input of a necessary annotation image.

  According to the fifth and 17th aspects of the present invention, it is possible to automatically delete an annotation image that is no longer necessary due to the movement of the object, thereby reducing the operation burden on the user.

  According to the invention of claim 6, the user of the external terminal can instruct to cancel the deletion of the annotation image.

  According to the invention of claim 7, it is possible to notify the user of the external terminal of the annotation image to be deleted.

  According to the eighth and 18th aspects of the present invention, it is possible to erase all the annotation images that are no longer necessary due to the movement of the object, and to reduce the operation burden on the user.

  According to the ninth and nineteenth aspects of the present invention, it is possible to effectively erase an annotation image that is no longer necessary, and to reduce the operation burden on the user.

  According to the tenth and twentieth inventions, a plurality of annotation images can be erased at once, and the operation burden on the user can be reduced.

  According to the eleventh and twenty-first aspects of the present invention, when an area associated with an annotation image is included in the change area, the annotation image can be automatically deleted, and the operation burden on the user can be reduced. Can do.

  According to the inventions of claims 12 and 22, when the positional relationship between the object and the annotation image changes due to the movement of the object, the immediately preceding image or display is automatically stored. Can be reduced.

  According to the inventions of claims 13 and 23, in all cases where the positional relationship between the object and the annotation image has changed due to the movement of the object, the immediately preceding image or display is automatically stored. The burden can be reduced.

  According to the fourteenth and twenty-fourth aspects of the present invention, it is possible to store images effectively and to reduce the operation burden on the user.

  According to the inventions of claims 25 and 26, when the positional relationship between the object and the annotation image changes due to the movement of the object, storage according to the importance of the captured image or display is automatically executed. Appropriate storage can be performed in a state where the operation burden on the user is reduced.

  According to the twenty-seventh aspect of the present invention, it is possible to store only images or displays with high importance.

  According to the invention of claim 28, since the importance is stored together with the image or display, the user can use the image or display according to the importance.

  According to the invention of claim 29, the importance can be determined quantitatively.

  According to the inventions of claims 30 and 31, when the positional relationship between the object and the annotation image changes due to the movement of the object, the processing for the image or the display can be automatically performed. Can be reduced.

  According to the inventions of claims 32 and 33, when the positional relationship between the object and the annotation image changes due to the movement of the object, storage according to the importance of the captured image or the display is automatically executed. Appropriate storage can be performed in a state where the operation burden on the user is reduced.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a diagram showing a configuration of a remote instruction system including an information processing apparatus according to an embodiment of the present invention.

  1 includes a personal computer (PC) 1 (information processing apparatus) that functions as a server and a PC 2 (external terminal) that functions as a client, which are connected to each other via a network 3. Yes. A projector 4 (projection device) and a video camera 5 (imaging device) are connected to the PC 1. Based on the control command from the PC 1, the projector 4 irradiates the object 8 and the screen 10 with a simple light beam through the half mirror 6 or projects an annotation image (annotation image) or the like. Note that the annotation image includes images of all aspects such as lines, characters, symbols, figures, colors, fonts (typefaces), and the like.

  The video camera 5 captures a reflected image of the screen 10 including the object 8 via the half mirror 6 and outputs the captured image to the PC 1.

  The PC 1 outputs the captured image captured by the video camera 5 to the PC 2 via the network 3. The PC 2 is connected to a display unit 205, and the display unit 205 displays a display area 12 for a captured image and a user interface (UI) 14. Note that the PC 2 may be configured as an integrated personal computer including the display unit 205.

  The UI 14 includes a group of buttons such as pen, text, and erase, and icons of line types and color types. In the display area 12, a captured image captured by the video camera 5 is displayed. In FIG. 1, a photographed image obtained by photographing the screen 10 including the object 8 with the video camera 5 is displayed in the display area 12.

  For example, when a pen button on the UI 14 is pressed and a figure or the like is written on the object 8 in the display area 12, information on the figure (specifically, coordinates (in the display area 12 indicating the figure) ( x, y)) are output from PC2 to PC1. The PC 1 converts the graphic information into the coordinate information of the projector 4 and outputs the information to the projector 4. The projector 4 projects the figure on the object 8 based on the converted figure information. Since the captured image is displayed in the display area 12 of the display unit 205, the coordinates (x, y) in the captured image coincide with the coordinates (x, y) in the display area 12.

  Further, the PC 2 outputs a control command to the PC 1, and the operations of the projector 4 and the video camera 5 (for example, the shooting angle of the video camera 5, the brightness of the captured image, and the brightness of the projection image of the projector 4). Can be controlled.

  In the figure, one client PC 2 is shown, but the remote instruction system may include a plurality of clients (PCs). Further, the projector 4 may be connected to the PC 1 and the video camera 5 may be connected to another PC (not shown). In this case, the captured image of the video camera 5 is output to the PC 2 via the other PC.

  FIG. 2 is a block diagram illustrating a functional configuration of the PCs 1 and 2.

  The PC 1 controls the entire apparatus and controls the operations of the projector 4, the video camera 5, and the three-dimensional measurement apparatus 7, and the PC 2 via the network 3 and the control unit 101 (detection means, extraction means, processing means, acquisition means). A transmission / reception unit 102 (transmission means) that transmits and receives information and data, a storage unit 103 that stores control programs, data, information, and the like, and an interface (IF) unit 104 for connection to the projector 4 and the video camera 5. And. The control unit 101 is connected to the transmission / reception unit 102, the storage unit 103, and the IF unit 104, and is further connected to the projector 4 and the video camera 5 via the IF unit 104.

  The PC 1 may further include an operation unit 105 configured with a mouse, a keyboard, and the like, and a display unit 106 (display unit) that displays a captured image.

  The PC 2 includes a control unit 201 that controls the entire apparatus, a transmission / reception unit 202 that transmits / receives information and data to / from the PC 1 via the network 3, a storage unit 203 that stores control programs, data, information, and the like, a mouse and a keyboard And the like, and a display unit 205 (image display means). The control unit 201 is connected to the transmission / reception unit 202, the storage unit 203, the operation unit 204, and the display unit 205.

  FIG. 3 is a block diagram illustrating a hardware configuration of the PC 1.

  The PC 1 is connected to a CPU 21 for controlling the entire apparatus, a ROM 22 having a control program, a RAM 23 functioning as a working area, a hard disk drive (HDD) 24 having various information and programs, a mouse and keyboard 25, and other computers. A network interface 26, a display 27 composed of a liquid crystal monitor or CRT, and a USB (universal serial bus) interface 28 for connecting to a USB device (not shown) are provided. The CPU 21 is connected to a ROM 22, a RAM 23, a hard disk drive (HDD) 24, a mouse and keyboard 25, a network interface 26, a display 27, and a USB interface 28 via a system bus 29.

  The control unit 101 corresponds to the CPU 21 that executes various processes according to the control program. The transmission / reception unit 102 corresponds to the network interface 26, and the storage unit 103 corresponds to the hard disk drive (HDD) 24. The IF unit 104 corresponds to the USB interface 28. The operation unit 105 corresponds to the mouse and keyboard 25, and the display unit 106 corresponds to the display 27.

  The PC 2 has the configuration shown in FIG. 3 like the PC 1.

  FIG. 4 is a flowchart showing processing executed by the PC 1.

  The control unit 101 of the PC 1 receives the captured image from the video camera 5 and stores it in the storage unit 103 (step S1). This photographed image is sent from the video camera 5 as a frame image every 0.5 seconds, for example.

  The control unit 101 compares the captured image stored in the storage unit 103 with the captured image just received from the video camera 5 for each pixel (step S2). Specifically, the control unit 101 compares the frame image immediately after being received from the video camera 5 and the frame image stored in the storage unit 103 immediately before the frame image for each pixel.

  The control unit 101 determines whether or not the object 8 has been moved based on the comparison result of step S2 (step S3). In step S <b> 3, the control unit 101 determines that the object 8 has moved when there is a difference between the two captured images (frame images).

  Note that the control unit 101 does not detect a change in the captured image (frame image) due to the projection of the annotation image. That is, the control unit 101 ignores the difference between the frame images before and after the projection of the annotation image. Thereby, the input of a required annotation image can be ensured.

  Next, the control unit 101 determines whether or not the movement of the object 8 has ended (step S4). For example, if there is no change in 50 consecutive frame images, the control unit 101 determines that the movement of the object 8 has ended.

  If NO in step S4, the determination process is repeated. On the other hand, in the case of YES in step S4, the control unit 101 includes the changed area (that is, the object 8 included in the captured image) included in the captured image (frame image) immediately after being received from the video camera 5. (Regions before and after movement) (hereinafter referred to as change regions) are extracted (step S5).

  When an annotation image is written in the display area 12 of the display unit 205, coordinate information indicating the annotation image is transmitted from the PC 2 to the PC 1. Therefore, the control unit 101 may exclude the region of the annotation image from the regions extracted in step S5 in FIG. 4 based on the coordinate information indicating the annotation image. Thereby, the input of a required annotation image can be ensured.

  FIG. 5A is a diagram illustrating an example of a captured image stored in the storage unit 103, and FIG. 5B is a diagram illustrating an example of a captured image immediately after being received from the video camera 5.

  In FIG. 5A, the objects 8a and 8b are displayed in the display area 12, and the annotation image 9 is projected on the object 8a. In FIG. 5B, the object 8a has moved from the state of FIG. Reference numeral 30 in the figure indicates a change area. In step S5, the change area 30 is extracted. The control unit 101 holds coordinate information on the captured image corresponding to the change area 30.

  Returning to FIG. 4, the control unit 101 determines whether or not an annotation image is included in the change area 30 (step S6). When an annotation image is written in the display area 12 of the display unit 205, coordinate information indicating the annotation image is transmitted from the PC 2 to the PC 1. Therefore, the determination in step S <b> 6 is performed by determining whether or not the coordinate information indicating the annotation image is included in the coordinate information corresponding to the change area 30.

  If NO in step S6, the process returns to step S1. On the other hand, if YES in step S6, the control unit 101 deletes the annotation image included in the change area 30 from the captured image (step S7). Since the control unit 101 has received the coordinate information indicating the annotation image from the PC 2, the coordinate information indicating the annotation image is discarded, or the coordinate information indicating the annotation image is not output to the projector 4. The annotation image included in the change area 30 can be deleted.

  Thereafter, the control unit 101 transmits the coordinate information of the annotation image to be erased to the PC 2 (step S8), and returns to step S1. Note that the control unit 201 of the PC 2 receives the coordinate information of the annotation image to be deleted from the PC 1 and deletes the annotation image from the display area 12. FIG. 5C shows an example of a captured image after the annotation image 9 is deleted.

  In step S7, the annotation image included in the change area 30 is deleted from the captured image. However, even when the change area 30 includes a part of the annotation image, the control unit 101 deletes the annotation image. It may be. This is because such an annotation image does not make sense when the object moves.

  In addition, the control unit 101 may delete an annotation image that is within a range widened by a predetermined distance from the change area 30 (for example, a range widened 30 pixels vertically and horizontally from the change area 30). Even when a change in the captured image is not detected by the light that projects the annotation image, the annotation image that is no longer needed can be erased.

  In addition, the control unit 101 may treat a plurality of annotation images as one group, and when deleting at least one annotation image, may also delete other annotation images belonging to the same group. Thereby, a plurality of annotation images can be erased at a time, and the operation burden on the user can be reduced. For example, as shown in FIG. 6A, when the annotation images 31a and 31b are handled as one group, the control unit 101 deletes the annotation image 31b at the same time when deleting the annotation image 31a. A plurality of annotation images belonging to the same group are, for example, a plurality of annotation images written in the display area 12 within a certain time, a plurality of annotation images written within a certain distance, or within a certain time and a certain distance. There are multiple annotation images written within the range.

  Furthermore, the annotation image may be associated with a predetermined area. In this case, when an annotation image is written in the display area 12 of the display unit 205, coordinate information indicating the annotation image and coordinate information of the area associated with the annotation image are transmitted from the PC 2 to the PC 1. The control unit 101 acquires coordinate information indicating the annotation image and coordinate information of an area associated with the annotation image via the transmission / reception unit 102. For example, as shown in FIG. 6B, a rectangular annotation image 31a is associated with a region 32a therein. The arrow annotation image 31b is associated with a circular region 32b at the tip of the other mark. When the region 32a or the region 32b is included in the change region 30 described above, the control unit 101 deletes the rectangular annotation image 31a or the arrow annotation image 31b.

  Depending on the shape or the like of the annotation image to be deleted, the annotation image may not be included in the change area 30, but when the change area 30 includes an area associated with the annotation image, the annotation image is automatically selected. Can be deleted automatically, and the operation burden on the user can be reduced.

  As described above in detail, according to the present embodiment, the control unit 101 detects the movement of the object 8 based on the captured image captured by the video camera 5, and is caused by the movement of the object 8. If the change area on the captured image is extracted and the annotation image is included in the change area, the annotation image is deleted, so that the annotation image that is no longer needed due to the movement of the target object can be automatically deleted. Can reduce the operation burden.

  The control unit 101 detects the movement of the object 8 by comparing the captured image captured by the video camera 5 with the captured image captured immediately after the captured image. Compared with the case where the process of tracking movement is executed, the amount of calculation required to detect the movement of the object 8 can be reduced.

(Second Embodiment)
This embodiment is different from the first embodiment in that the “deletion stop button” is displayed on the display unit 205 and the user's intention is confirmed before the control unit 101 deletes the annotation image.

  The remote instruction system according to the present embodiment has the same configuration as that of the remote instruction system of FIG.

  FIG. 7 is a flowchart showing processing executed by the PC 1. The same steps as those in FIG. 4 are denoted by the same step numbers and description thereof is omitted.

  In the case of YES in step S6, the control unit 101 transmits to the PC 2 an instruction for blinking display of the annotation image included in the change area 30 and display of the erasure stop button (step S11).

  FIG. 8 is a diagram illustrating an example of the display area 12 that displays a captured image.

  When the control unit 201 of the PC 2 receives an instruction to blink the annotation image included in the change area 30 and the display of the erasure stop button from the PC 1 via the transmission / reception unit 202, the control unit 201 causes the display unit 205 to blink the annotation image. Display the cancel button. In FIG. 8, the annotation image 9 is blinked in the display area 12 and an erasure cancel button 40 is displayed. The erasure cancel button 40 is displayed for a predetermined time (for example, 1 minute), and is automatically deleted when the predetermined time elapses. When the user presses the erasure cancel button 40 within a certain time via the operation unit 204, the control unit 201 transmits an erasure cancel instruction to the control unit 101 of the PC 1 via the transmission / reception unit 202.

  After the process of step S11, the control unit 101 determines whether or not the erasure stop button has been pressed within a certain time (step S12). Here, the control unit 101 determines that the erasure cancellation button has been pressed when it receives an erasure cancellation instruction from the PC 2 within a certain time, and does not receive the erasure cancellation instruction from the PC 2 within a certain time. In addition, it is determined that the delete cancel button has not been pressed.

  If YES in step S12, erasure of the annotation image included in the change area 30 is stopped, and the process returns to step S1. On the other hand, if NO in step S12, the process proceeds to step S7 in order to delete the annotation image included in the change area 30 from the captured image.

  In the example of FIG. 8, the erasure stop button 40 is displayed on the display unit 205, but the erasure button may be displayed. In this case, when the delete button is pressed, the control unit 101 deletes the annotation image included in the change area 30.

  Further, the control unit 101 may treat a plurality of annotation images as one group, and cancel the deletion of all annotation images belonging to the same group when the deletion cancellation button 40 is pressed. In this case, only one deletion cancel button 40 is displayed. Further, the control unit 101 may handle a plurality of annotation images blinking within a certain time as one group. In this case, blinking of a plurality of annotation images need not be started simultaneously. Furthermore, the control unit 101 may handle a plurality of annotation images whose display positions are close to each other (for example, only 20 pixels or less) as one group.

  As described above, according to the present embodiment, the display unit 205 of the PC 2 displays a photographed image by the video camera 5 and according to the information (coordinate information) of the annotation image to be deleted from the PC 1. Since the deletion cancel button 40 that can instruct the control unit 101 of the PC 1 to cancel the deletion of the annotation image is displayed, the user of the PC 2 can instruct the cancellation of the deletion of the annotation image.

  Further, the display unit 205 blinks and displays the annotation image for a certain period of time according to the information of the annotation image to be deleted from the PC 1, so that the user of the PC 2 can be notified of the annotation image to be deleted.

  In the first and second embodiments, the control unit 101 compares the captured image stored in the storage unit 103 with the captured image just received from the video camera 5 for each pixel, so that the object is It is determined whether or not it has been moved (steps S2 and S3). On the other hand, as shown in FIG. 9, the control unit 101 divides the photographed image stored in the storage unit 103 and the photographed image just received from the video camera 5 into a plurality of small areas, and corresponds to each other. It may be determined whether or not the object has been moved by comparing the divided small areas. Thereby, it is possible to determine whether or not the object is accurately moved without being influenced by noise or the like of the captured image.

  In addition, the control unit 101 may delete the annotation image when a change occurs in a certain number of small areas around the annotation image (for example, five or more small areas). Thereby, it is possible to avoid the influence of noise or the like of the photographed image and erase the annotation image that has become unnecessary.

  In the first and second embodiments, it is assumed that the object 8 (8a, 8b) has moved. However, for example, when the projector 4 or the video camera 5 is moved, the entire captured image changes. Since this is an area, the control unit 101 may delete all annotation images in the captured image.

  In the first and second embodiments, the projector 4 and the video camera 5 are connected to the PC 1, but as shown in FIG. 10, the same object is displayed on the display unit 106 of the PC 1 and the display unit 205 of the PC 2. 8 and the same annotation image 9 may be displayed. In the system of FIG. 10, the object 8 is a virtual object. Also in the system of FIG. 10, the PC 1 can execute the processing of the flowcharts of FIGS.

  In this case, the display unit 106 of the PC 1 displays the annotation image input from the object 8 and the PC 1 in the display area of the display unit 106. Then, when the control unit 101 detects the movement of the object 8, extracts a change area in the display area caused by the movement of the object 8, and the annotation image is included in the change area, the annotation image Erase. As a result, annotation images that have become unnecessary due to the movement of the object can be automatically deleted, and the operation burden on the user can be reduced.

(Third embodiment)
Next, the 3rd Embodiment of this invention is described based on FIGS. 11-12C. The third embodiment is characterized in that when an annotation image is included in the change area on the captured image extracted by the control unit 101, the immediately preceding image is stored. The remote instruction system according to the third embodiment has the same configuration as the remote instruction system of FIG.

  FIG. 11 is a flowchart showing processing executed by the PC 1. As shown in FIG. 11, the control unit 101 of the PC 1 receives (acquires) a captured image (referred to as “captured image Pf”) from the video camera 5 (step S101). This photographed image is sent from the video camera 5 as a frame image every 0.5 seconds, for example.

  The control unit 101 compares the captured image Pf acquired in step S101 with the captured image immediately after received from the video camera 5 (referred to as “captured image Pa”) for each pixel (step S102). Specifically, the control unit 101 compares the frame image immediately after being received from the video camera 5 and the frame image stored in the storage unit 103 immediately before the frame image for each pixel.

  The control unit 101 determines whether or not the object 8 has been moved based on the comparison result of step S102 (step S103). In step S103, the control unit 101 determines that the object 8 has moved if there is a difference between the two captured images (frame images). Note that the control unit 101 does not detect a change in the captured image (frame image) due to the projection of the annotation image. That is, the control unit 101 ignores the difference between the frame images before and after the projection of the annotation image in the annotation image area. Thereby, the input of a required annotation image can be ensured.

  If the determination in step S103 is negative (ie, the object has not moved), the captured image Pf is stored in the storage unit 103 (step S104), and the process returns to step S101.

  On the other hand, when the determination in step S103 is affirmative (that is, when the object has moved), the control unit 101 determines whether or not the movement of the object 8 has ended (step S105). For example, if there is no change in 50 consecutive frame images, the control unit 101 determines that the movement of the object 8 has ended.

  If the determination in step S105 is negative, the determination process is repeated, and when the determination in step S105 is affirmed, the control unit 101 is included in the captured image (frame image) immediately after received from the video camera 5. The changed area (change area) is extracted (step S106).

  When an annotation image is written in the display area 12 of the display unit 205, coordinate information indicating the annotation image is transmitted from the PC 2 to the PC 1. Therefore, the control unit 101 may exclude the region of the annotation image from the regions extracted in step S106 in FIG. 11 based on the coordinate information indicating the annotation image. Thereby, the input of a required annotation image can be ensured.

  In FIG. 12A, the objects 8a and 8b are displayed in the display area 12, and a state where the annotation image 9 is projected on the object 8a (captured image Pf) is shown. FIG. 12B shows a state (captured image Pa) in which the object 8a has moved from the state of FIG. FIG. 12C shows a state where a changed area is extracted (step S106). In FIG. 12C, areas indicated by reference numerals 130 and 130 'are change areas. The control unit 101 holds coordinate information on the captured image corresponding to the change areas 130 and 130 '.

  Returning to FIG. 11, the control unit 101 determines whether or not an annotation image is included in the change areas 130 and 130 '(step S107). When an annotation image is written in the display area 12 of the display unit 205, coordinate information indicating the annotation image is transmitted from the PC 2 to the PC 1. Therefore, the determination in step S107 is performed by the control unit 101 determining whether the coordinate information indicating the annotation image is included in the coordinate information corresponding to the change area 30.

  If the determination in step S107 is negative, the process returns to step S101. On the other hand, if the determination in step S107 is affirmative, the control unit 101 adds (saves) the captured image Pf (the captured image before the movement of the object) to the captured image list in the storage unit 103 ( Step S108).

  In step S107, the case where the captured image Pf is stored when all the annotation images are included in the change areas 130 and 130 ′ has been described. However, the present invention is not limited to this, and the change areas 130 and 130 ′ include the change images 130 and 130 ′. The captured image Pf may be stored even when a part of the annotation image is included.

  In addition, the control unit 101 includes at least a part of the annotation image within a range widened by a predetermined distance from the change areas 130 and 130 ′ (for example, a range widened 30 pixels vertically and horizontally from the change areas 130 and 130 ′). If it is, the captured image Pf may be stored. In this way, even when no change in the captured image is detected by the light of the projector that projected the annotation image, the captured image can be stored at an appropriate timing.

  As described above, the processing and determination in steps S101 to S108 are sequentially repeated, so that the captured images are accumulated in the captured image list of the storage unit 103 at an appropriate timing. The user can create minutes of a conference between remote locations held by using the remote instruction system according to the present embodiment by using these stored captured images.

  As described above in detail, according to the third embodiment, the control unit 101 detects the movement of the object 8 based on the captured image captured by the video camera 5 and moves the object 8. When the change area on the photographed image generated by the above is extracted and an annotation image is included in the change area, the photographed image Pf before the change is added (stored) to the photographed image list in the storage unit 103. Thus, the captured image can be automatically stored, and the operation burden on the user can be reduced. Moreover, it is possible to create appropriate minutes and the like by using captured images stored at appropriate timing.

  In the third embodiment, the case where the captured image Pf is stored in the storage unit 103 has been described. However, the present invention is not limited to this, and for example, a device (PC) different from the PC 1 existing on the network. In addition, the captured image Pf may be stored.

(Fourth embodiment)
Next, the 4th Embodiment of this invention is described based on FIGS. 13-14F. The fourth embodiment is different from the third embodiment described above in the method of storing a photographed image when an annotation image is included in the change area on the photographed image extracted by the control unit 101. Therefore, in the following, this point will be mainly described.

  FIG. 13 is a flowchart showing processing executed by the PC 1. As can be seen by comparing FIG. 13 with FIG. 11 (the process of the third embodiment described above), steps S201 to S206 in FIG. 13 and steps S101 to S106 in FIG. It has become. Therefore, the description of steps S201 to S206 is omitted here.

After extracting the changed area in step S206, the control unit 101 calculates the importance of the photographed image Pf before the change (step S207). Here, as parameters for determining the importance, in the present embodiment, parameters related to the change region (parameters related to the size of the change region (β ₁ ), parameters related to the time until the change region occurs (β ₂ ),... Etc.), parameters related to annotation images included in the change region (parameters related to the number of annotation images included in the change region (β _n-2 )), and between annotation images when multiple annotation images are included in the change region A parameter (β _n-1 ) related to the distance and a parameter (β _n )... Related to the time interval at which each annotation image is drawn are adopted.

  Here, the importance will be specifically described. For example, as shown in FIGS. 14A and 14B, when the number of annotation images included in the change regions 130 and 130 ′ is different, The greater the number of annotation images, the higher the importance of the captured image Pf (ie, “importance in FIG. 14A <importance in FIG. 14B”). Further, as shown in FIGS. 14C and 14D, when there are a plurality of annotation images in the change areas 130 and 130 ′, the distance between the annotation images is smaller (the degree of approach). It is assumed that the importance of the captured image Pf is higher (so that “importance in FIG. 14A <importance in FIG. 14B”). Further, as shown in FIGS. 14E and 14F, when the size of the change area is different, the importance of the captured image Pf is increased as the change area is larger (see FIG. 14 (E) <importance in FIG. 14 (F) ”.

  Although not shown in the figure, for example, when the time taken from the time when a captured image was previously stored until it is determined whether to store the next captured image is short or long, Since it is assumed that the longer one has sufficiently discussed the image, the longer the time is, the higher the importance of the captured image Pf is handled.

  Furthermore, when the drawing interval (time interval) of the annotation image drawn when determining the importance is short or long, it is assumed that the shorter one has a lively discussion about the image. Therefore, the shorter the time is, the higher the importance of the captured image Pf is handled.

Therefore, the control unit 101 uses the above parameters (β _{1 to} β _n ) and the weighting coefficients (α _{1 to} α _n ) for weighting these parameters to determine the importance (γ) of the following equation (1). Thus, the calculation is performed.
γ = α ₁ β ₁ + α ₂ β ₂ + ... + α _n β _n (1)

  Returning to FIG. 13, the control unit 101 compares the importance level (γ) calculated as described above with a predetermined importance level threshold value (γs). It is determined whether or not (γs) or more (step S208).

  If the determination in step S208 is affirmative, it means that the importance of the captured image Pf is determined to be relatively high, and the control unit 101 places the image in the captured image list of the storage unit 103. Add (memorize).

  On the other hand, if the determination in step S208 is negative, it means that the importance of the captured image Pf is determined to be not so high, so that the control unit 101 does not store the image and does not store the image. Return to S201.

  As described above, the processes and determinations in steps S201 to S209 are sequentially repeated, so that captured images are accumulated in the captured image list of the storage unit 103 at appropriate timing. The user can create minutes of a conference between remote locations held by using the remote instruction system according to the present embodiment by using these stored captured images.

  As described above in detail, according to the fourth embodiment, the control unit 101 detects the movement of the object 8 based on the captured image captured by the video camera 5 and moves the object 8. The change area on the captured image generated by the above is extracted, the importance (γ) of the captured image before the change is calculated, and the captured image is automatically only when the importance (γ) is equal to or greater than the threshold (γs). Therefore, only high-importance images can be stored among the captured images. Moreover, it is possible to create an appropriate minutes etc. by creating a minutes etc. using such a highly important photographed image.

In the fourth embodiment, the importance level is calculated using the equation (1). However, the present invention is not limited to this, and the importance level is calculated using other statistical formulas. It is also possible to do this. For example, a standard deviation can be used as the statistical calculation expression. That is, with respect to all the data of the time interval at which the annotation image was drawn, the standard deviation δ _n of the time interval at which each annotation image was drawn,
γ = α ₁ δ ₁ + α ₂ δ ₂ + ... + α _n δ _n (2)
It can be expressed as. In addition, it is also possible to employ | adopt expressions other than said Formula (2) as a formula using a standard deviation.

  In the fourth embodiment, the captured image is stored when the importance (γ) is equal to or greater than the threshold (γs). However, the captured image is added to the captured image list in the storage unit 103 ( As shown in FIG. 15, the importance may be described in a table (a table for managing file names, storage dates and the like) created at the time of storage. By doing in this way, the user can create the minutes etc., referring to the importance of the captured image.

  In the fourth embodiment, it is determined in step S208 in FIG. 13 whether or not the importance is equal to or higher than the threshold. However, the present invention is not limited to this. For example, the importance is equal to or higher than the threshold. It is also possible to add (store) all of the images for which importance has been calculated to the photographed image list in the storage unit 103 together with the value of importance, without determining whether or not the image is. Even in this way, the user can create the minutes and the like while referring to the importance of the captured image.

  In the third and fourth embodiments, the control unit 101 compares the captured image stored in the storage unit 103 with the captured image immediately after received from the video camera 5 for each pixel. Although it is determined whether or not an object has been moved, the present invention is not limited thereto, and the captured image stored in the storage unit 103 and the captured image immediately after being received from the video camera 5 are each divided into a plurality of small areas, and You may make it discriminate | determine whether the target object is moved by comparing the corresponding divided | segmented small area | regions (refer FIG. 9). Thereby, it is possible to determine whether or not the object is accurately moved without being influenced by noise or the like of the captured image.

  In the third and fourth embodiments, as in the case of the first and second embodiments described above, the display unit 106 of the PC 1 and the display unit 205 of the PC 2 are arranged as shown in FIG. The same object (virtual object) 8 and the same annotation image 9 may be displayed. In the system of FIG. 10 as well, the PC 1 can execute processing according to the flowcharts of FIGS. 11 and 13.

  The first or second embodiment and the third or fourth embodiment can be used in combination. That is, for example, when an annotation image is included in the change area, the control unit 101 stores the captured image immediately before the change in the storage unit 103 (captured image list) and is included in the change area. It is also possible to perform processing for deleting the annotation image.

  In each of the above embodiments, the case where the captured image is directly transmitted from the PC 1 to the PC 2 via the network has been described. However, the present invention is not limited to this, and for example, another hardware is provided in the vicinity of the PC 1. The captured image may be transmitted from the other hardware that has received the captured image from the PC 1 to the PC 2 via the network.

  The above-mentioned implementation can also be realized by supplying a recording medium in which a software program for realizing the functions of PC1 to PC2 is recorded to each PC, and the CPU of each PC reads and executes the program stored in the storage medium. The same effect as that of the embodiment is produced. Examples of the storage medium for supplying the program include a CD-ROM, a DVD, or an SD card.

  Further, the same effects as those of the above-described embodiment can also be obtained when the CPU of each PC executes a software program for realizing the function of each PC.

  In addition, this invention is not limited to each embodiment mentioned above, In the range which does not deviate from the summary, it can change and implement variously.

It is a figure which shows the structure of the remote instruction | indication system containing the information processing apparatus which concerns on embodiment of this invention. It is a block diagram which shows the function structure of PC1-2. It is a block diagram which shows the hardware constitutions of PC1. It is a flowchart which shows the process performed by PC1. (A) is a figure which shows an example of the picked-up image memorize | stored in the memory | storage part, (B) is a figure which shows an example of the picked-up image immediately after receiving from a video camera, (C) is an annotation image. An example of the photographed image after deletion is shown. (A) is a figure which shows an example of the picked-up image in the case of handling two annotation images as one group, (B) is an example of the picked-up image containing an annotation image and the area | region linked | related with the said annotation image. FIG. It is a flowchart which shows the process performed with PC1 which concerns on 2nd Embodiment. It is a figure which shows an example of the display area which displays a picked-up image. It is a figure which shows an example of the picked-up image divided | segmented into the several small area | region. It is a figure which shows the modification of a structure of the remote instruction | indication system containing an information processing apparatus. It is a flowchart which shows the process performed by PC1 which concerns on 3rd Embodiment. 12A is a diagram illustrating an example of a captured image stored in the storage unit, and FIG. 12B is a diagram illustrating an example of a captured image immediately after being received from the video camera. C) is a diagram illustrating an example of a change region. It is a flowchart which shows the process performed by PC1 which concerns on 4th Embodiment. FIG. 14A to FIG. 14F are diagrams for explaining the importance. It is a figure which shows an example of the table used in order to manage data.

Explanation of symbols

1 PC (information processing equipment)
2 PC (external terminal)
3 Network 4 Projector (Projector)
5 Video camera (photographing device)
6 half mirror 101 control unit (detection means, extraction means, processing means, acquisition means)
102 Transmission / reception unit (transmission means)
103 Storage Unit 104 IF Unit 105 Operation Unit 106 Display Unit (Display Unit)
201 Control Unit 202 Transmission / Reception Unit 203 Storage Unit 204 Operation Unit 205 Display Unit (Image Display Unit)

Claims (33)

An information processing apparatus connected to a projection apparatus that projects an annotation image input from an external terminal onto a projection area including an object and a background, and an imaging apparatus that captures the projection area including the object and the background,
Detection means for detecting movement of the object based on a photographed image photographed by the photographing device;
Extraction means for extracting a change area on the captured image caused by movement of the object;
An information processing apparatus comprising: processing means for executing processing on at least one of the captured image and the annotation image when the annotation image is included in the change area. The detection unit detects movement of the object by comparing a captured image captured by the imaging device with a captured image captured immediately after the captured image. The information processing apparatus described. The detection unit divides a photographed image photographed by the photographing device and a photographed image photographed immediately after the photographed image into a plurality of small regions, and compares the divided small regions with each other. The information processing apparatus according to claim 1, wherein movement of the object is detected. The extraction means ignores the region of the annotation image in the display region based on the change region generated during the input of the annotation image from the external terminal or the coordinate information of the annotation image from the external terminal. The information processing apparatus according to any one of claims 1 to 3. The information processing apparatus according to claim 1, wherein the processing executed by the processing unit includes erasure of the annotation image. Further comprising transmission means for transmitting information of the annotation image to be erased by the processing means to the external terminal,
The external terminal displays a photographed image by the photographing device and can instruct the processing means to cancel the erasure of the annotation image according to the information of the annotation image to be erased from the transmission means. 6. The information processing apparatus according to claim 5, further comprising image display means for displaying a part. The information processing apparatus according to claim 6, wherein the image display unit displays the annotation image in a blinking manner for a certain period of time in accordance with information on the deleted annotation image from the transmission unit. 8. The information processing apparatus according to claim 5, wherein when the change area includes a part of the annotation image, the processing unit deletes all of the annotation image. 9. . 9. The information processing apparatus according to claim 5, wherein the processing unit also deletes an annotation image within a range extended by a predetermined distance from the change area. 10. The processing means treats a plurality of annotation images input from the external terminal as one group, and when deleting at least one annotation image, deletes other annotation images belonging to the same group. The information processing apparatus according to any one of claims 5 to 9. An acquisition means for acquiring information on the annotation image and an area associated with the annotation image from the external terminal;
11. The information processing according to claim 5, wherein the processing unit deletes the annotation image when the change region includes a region associated with the annotation image. apparatus. 12. The process executed by the processing unit includes storing the captured image at a time point before the annotation image is included in the change area. The information processing apparatus described. The information processing apparatus according to claim 12, wherein when the change area includes at least a part of the annotation image, the processing unit stores the captured image at a previous time point. The processing unit stores the photographed image at a previous time point even when at least a part of the annotation image is included in a range that is widened by a predetermined distance from the change region. 13. The information processing apparatus according to 13. Display means for displaying an annotation image input from an object and an external terminal in a display area;
Detecting means for detecting movement of the object;
Extraction means for extracting a change area in the display area caused by the movement of the object;
An information processing apparatus comprising: processing means for executing processing for display in the display area when the annotation area is included in the change area. The extraction means ignores the region of the annotation image in the display region based on the change region generated during the input of the annotation image from the external terminal or the coordinate information of the annotation image from the external terminal. The information processing apparatus according to claim 15. The information processing apparatus according to claim 15, wherein the processing executed by the processing unit includes erasure of the annotation image. The information processing apparatus according to claim 17, wherein when the change area includes a part of the annotation image, the processing unit deletes all of the annotation image. The information processing apparatus according to claim 17 or 18, wherein the processing unit also deletes an annotation image within a range that is widened by a predetermined distance from the change area. The processing means treats a plurality of annotation images input from the external terminal as one group, and when deleting at least one annotation image, deletes other annotation images belonging to the same group. The information processing apparatus according to any one of claims 17 to 19. An acquisition means for acquiring information on the annotation image and an area associated with the annotation image from the external terminal;
The information processing according to any one of claims 17 to 20, wherein the processing unit deletes the annotation image when the region associated with the annotation image is included in the change region. apparatus. The process executed by the processing means includes storing the display in the display area at a time before the annotation area is included in the change area. The information processing apparatus according to item 1. The information processing apparatus according to claim 22, wherein when the change area includes at least a part of the annotation image, the processing unit stores a display in the display area at a previous time point. . The processing means stores a display in the display area at a previous time point even when at least a part of the annotation image is included in a range extended by a predetermined distance from the change area. Item 24. The information processing apparatus according to Item 22 or 23. An information processing apparatus connected to a projection apparatus that projects an annotation image input from an external terminal onto a projection area including an object and a background, and an imaging apparatus that captures the projection area including the object and the background,
Detection means for detecting movement of the object based on a photographed image photographed by the photographing device;
Extraction means for extracting a change area on the captured image caused by movement of the object;
The importance of the captured image is calculated based on at least one of the information on the change area and the information on the annotation image included in the change area, and the captured image is stored according to the importance. An information processing apparatus comprising storage means. Display means for displaying an annotation image input from an object and an external terminal in a display area;
Detecting means for detecting movement of the object;
Extraction means for extracting a change area in the display area caused by the movement of the object;
Based on at least one of the information on the change area and the information on the annotation image included in the change area, the importance of the display in the display area is calculated, and the display is stored according to the importance. An information processing apparatus comprising storage means for executing. 27. The information processing apparatus according to claim 25, wherein the storage unit executes the storage when the importance is equal to or greater than a predetermined value. The information processing apparatus according to any one of claims 25 to 27, wherein the storage unit also stores the data of the importance when the storage is executed. The information of the change area used for the calculation of the importance includes at least one of the size of the change area and the time until the change area occurs,
The information on the annotation image included in the change area includes the number of the annotation images included in the change area, the distance between the annotation images in the case where a plurality of annotation images are included in the change area, and the annotation images. The information processing apparatus according to any one of claims 25 to 28, wherein at least one of time intervals in which is drawn is included. A projection device that projects an annotation image input from an external terminal onto a projection area including an object and a background, and a computer connected to an imaging apparatus that captures the projection area including the object and the background;
Detecting means for detecting movement of the object based on a photographed image photographed by the photographing device;
Extraction means for extracting a change area on the photographed image caused by the movement of the object, and processing for at least one of the photographed image and the annotation image when the annotation image is included in the change area A program characterized by functioning as a processing means. Computer
Display means for displaying an annotation image input from an object and an external terminal in a display area;
Detecting means for detecting movement of the object;
Extracting means for extracting a change area in the display area caused by movement of the object, and processing means for executing processing for display in the display area when the annotation image is included in the change area A program characterized by functioning. A projection device that projects an annotation image input from an external terminal onto a projection area including an object and a background, and a computer connected to an imaging apparatus that captures the projection area including the object and the background;
Detecting means for detecting movement of the object based on a photographed image photographed by the photographing device;
Extraction means for extracting a change area on the photographed image caused by the movement of the object, and the photographing based on at least one of information on the change area and information on the annotation image included in the change area. A program that calculates an importance level of an image and functions as a storage unit that stores the captured image according to the importance level. Computer
Display means for displaying an annotation image input from an object and an external terminal in a display area;
Detecting means for detecting movement of the object;
Based on at least one of extraction means for extracting a change area in the display area caused by movement of the object, information on the change area, and information on the annotation image included in the change area, A program which calculates the importance of display in a display area and functions as storage means for executing storage of the display according to the importance.

Images