JP5012644B2 - Presentation recording apparatus, presentation playback apparatus, and program - Google Patents

Description

  The present invention relates to a presentation recording device, a presentation playback device, and a program.

  It has become common to give presentations by sequentially displaying slide images created in advance using a computer. As a technique for recording and reproducing a presentation, for example, there are those described in Patent Document 1 and Patent Document 2 below.

Patent Document 1 discloses a technique for detecting user's operation information and playing back recorded audio in accordance with the slide switching time. Patent Document 2 discloses a displayed slide image. Is disclosed in which audio data is divided and stored in accordance with the detected change timing, and a slide image and audio are reproduced in synchronization during reproduction.
JP 2003-58901 A JP 2006-127518 A

  However, in the conventional technique, when explanation is given while displaying one slide for a long time, it is impossible to quickly find explanation portions corresponding to a plurality of topics included in the slide.

  The present invention has been made in view of the above problems, and one of the objects of the present invention is to provide a presentation recording apparatus and a presentation that can easily find a portion to be viewed from the description using recorded slides. To provide a playback device and a program.

  In order to achieve the above object, the invention of the presentation recording apparatus according to claim 1 is a presentation recording apparatus for recording a presentation that is performed while indicating a displayed image by an instruction means. Specifying means for specifying the indicated area of the image, generating means for generating a reference image representing the specified area of the image specified by the specifying means in a predetermined form, and a reference generated by the generating means The image processing apparatus includes a storage unit that stores the image and the image and sound related to the presentation performed when the instruction area specified by the specifying unit is instructed by the instruction unit.

  According to a second aspect of the present invention, in the presentation recording apparatus according to the first aspect, the means for acquiring the displayed image and a partial area for setting one or more partial areas in the acquired image Setting means, wherein the specifying means specifies the indication area from the partial areas set by the partial area setting means based on the position indicated by the indication means. To do.

  According to a third aspect of the present invention, in the presentation recording apparatus according to the second aspect, the partial area setting means analyzes the layout of the acquired image, and based on the analyzed layout, A partial region is set.

  According to a fourth aspect of the present invention, in the presentation recording apparatus according to the second or third aspect, the generation unit continuously specifies the indication area specified by the specifying unit over a predetermined time. A reference image representing the designated area in a predetermined manner is generated.

  Further, the invention according to claim 5 is the presentation recording apparatus according to claim 1, wherein the specifying unit is configured to determine the position at each time based on a distribution of positions instructed by the instruction unit every predetermined time. An instruction area specified by the instruction means is specified.

  According to a sixth aspect of the present invention, in the presentation recording apparatus according to any one of the first to fifth aspects, the storage unit is instructed by the instruction unit together with the image and sound relating to the presentation. This data is further stored.

  According to a seventh aspect of the present invention, there is provided a presentation playback apparatus comprising: means for displaying one or more reference images stored in the storage means by the presentation recording apparatus according to any one of the first to sixth aspects; Replay means for replaying a presentation recorded based on at least an image and sound associated with the selected reference image when a reference image is selected from one or more of the displayed reference images; Including.

  Further, the invention of the program according to claim 8 is directed to a computer for recording a presentation performed while indicating the displayed image by the instruction means, and a specifying means for specifying an instruction area of the image indicated by the instruction means. Generating means for generating a reference image representing the specified area of the image specified by the specifying means in a predetermined form, a reference image generated by the generating means, and an indicated area specified by the specifying means Is functioned as a storage means for storing the image and sound related to the presentation performed when the instruction means is instructed.

  According to the first aspect of the present invention, it is possible to record a presentation (explanation) using an image (slide) that is displayed so that a part that the user wants to view can be easily found.

  According to the second aspect of the present invention, it is possible to set a plurality of areas in the image, and to identify and record the explanation made for each area.

  According to the third aspect of the present invention, the description can be identified and recorded for each group of slides by setting the partial area to be set in the image based on the layout.

  According to the fourth aspect of the present invention, it is possible to identify and record the explanations made for the area of the designated time exceeding the threshold.

  According to the fifth aspect of the present invention, the explanation given every time can be identified and recorded together with the indicated area.

  According to the sixth aspect of the present invention, the presentation can be recorded so as to reproduce the display image (slide), sound, and the position indicated by the pointing means such as the pointer.

  According to the seventh aspect of the present invention, it is possible to reproduce the presentation from a location selected by the user from the reference images corresponding to the respective parts obtained by dividing the explanation made using one image.

  According to the invention described in claim 8, it is possible to make the computer function so as to record the description (presentation) using the displayed image (slide) so that the part to be viewed can be easily found.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments (hereinafter referred to as embodiments) for carrying out the invention will be described with reference to the drawings.

  FIG. 1 shows a functional block diagram of a presentation recording / reproducing apparatus 10 according to the present embodiment. As shown in FIG. 1, the presentation recording / reproducing apparatus 10 includes a storage unit 12, a control unit 14, a display unit 16, a sound acquisition unit 18, an image acquisition unit 20, an indicated position information acquisition unit 22, a partial region setting unit 24, An instruction area specifying unit 26, an instruction area change detection unit 28, a dwell time measurement unit 30, a thumbnail generation determination unit 32, an index data generation unit 34, a thumbnail generation unit 36, a presentation playback unit 38, and a thumbnail selection unit 40 are included.

  The function of each part is that a program stored in a computer-readable information storage medium is read into a presentation recording / reproducing apparatus 10 which is a computer system using a medium reading apparatus (not shown) and executed by the presentation recording / reproducing apparatus 10. It may be realized by. Here, the program is supplied to the presentation recording / reproducing apparatus 10 by the information storage medium, but the program may be downloaded to the presentation recording / reproducing apparatus 10 from a remote place via a data communication network such as the Internet.

  The storage unit 12 includes a storage device such as a memory and a hard disk, and stores data and programs. The storage unit 12 stores a slide display program for sequentially displaying slide data related to the presentation on the display unit 16 (liquid crystal display or the like) in accordance with the operation of the user of the presentation recording / reproducing apparatus 10, and includes a CPU. 14 writes graphic data in the VRAM based on the stored slide display program and slide data, and displays a slide image on the display unit 16 based on the graphic data written in the VRAM. At this time, a user who makes a presentation makes a presentation (explanation) based on the contents of the slide image while pointing the slide image displayed on the display unit 16 with a pointer that moves according to the operation of the mouse or the like.

  The voice acquisition unit 18 is connected to a microphone and collects voice of a user who makes a presentation using the microphone to acquire voice data. The acquired voice data is stored in the storage unit 12.

  The image acquisition unit 20 acquires the image data (slide image data) of the slide image displayed on the display unit 16 and stores it in the storage unit 12. The format of the acquired slide image data may be bitmap data, or may be compressed image data according to various formats (such as JPEG) in which the bitmap data is compressed.

  The designated position information acquisition unit 22 sequentially acquires position information of pointers pointing to a part of the displayed slide image at predetermined time intervals and stores them in the storage unit 12. When the position of the pointer is controlled in accordance with an operation input of an input device such as a mouse connected to the presentation recording / reproducing apparatus 10, the position of the designated coordinates on the slide screen of the pointer based on the input from the input device May be acquired sequentially.

  The partial area setting unit 24 performs a process based on a predetermined rule for the slide image acquired by the image acquisition unit 20, for example, a layout analysis process on the slide image, and a plurality of slide images are added to the slide image based on the analyzed layout. Set the partial area. For example, a known technique used in the OCR process may be employed for the above-described layout analysis.

  FIG. 2 shows an example of a partial area set in the slide image by the partial area setting unit 24. As shown in FIG. 2, the captured slide image 50 is subjected to layout analysis processing, and partial areas 52, 54, 56, 58 and 60 are set for the title and each explanation item. Each of the partial areas may be set as follows, for example. That is, the partial area setting unit 24 performs OCR processing on the captured slide image 50, extracts a character string from the slide image 50, and character strings in the vicinity of the extracted character string constitute the same area. Integrate as you do. Then, the partial area setting unit 24 acquires coordinate information for each integrated area. For example, if the partial area is a rectangle, the coordinate information may be represented by coordinate values for two points, the upper left corner and the lower right corner of the rectangle.

  Based on the position coordinates indicated by the pointer acquired by the specified position information acquisition unit 22, the instruction region specifying unit 26 indicates any partial region of the partial region set by the partial region setting unit 24. To identify.

  The designated area change detection unit 28 detects a change in the designated area of the pointer specified by the specified area specifying unit 26. The indication area may be changed when the position coordinates of the pointer come out of the partial area last specified by the indication area specifying unit 26, or from the last specified partial area to another partial area. The indication area may be changed when entering.

  The staying time measuring unit 30 measures the time until the pointing area change detecting unit 28 detects that the pointing area of the pointer has changed as the staying time of the pointer in the pointing area.

  Data representing the designated area designated by the pointer is acquired at predetermined time intervals, and these data are sequentially stored in, for example, the designated area history table shown in FIG.

  FIG. 3 shows an example of an instruction area history table that sequentially stores pointer instruction areas acquired when the predetermined time interval is 10 seconds. As shown in FIG. 3, the instruction area history table sequentially records records storing slide ID, pointer position information, time, area ID, IN, OUT, and residence time values. Here, the slide ID is an identifier assigned to each displayed slide, and the pointer position information is data representing the position of the pointer, and may be represented by the coordinate position of the pointer, for example. The time is the time from the start of recording the presentation when the position of the pointer is recorded, the area ID is an identifier for identifying each set partial area, and IN and OUT are the time when the partial area is entered and The residence time represents the time spent in the partial area, that is, the time from IN to OUT. For example, in FIG. 3, it is recorded that the pointer pointing area at the start of presentation (time 0:00) is in the background (Bg), and then the area ID is located in the area R1 and stayed for 50 seconds. .

  When the residence time measured by the residence time measurement unit 30 exceeds a predetermined threshold, the thumbnail generation determination unit 32 determines to generate a thumbnail image in which the designated area is highlighted. The thumbnail generation determination unit 32 reads the pointer dwell time for each area from, for example, the designated area history table shown in FIG. 3, and when the value exceeds a predetermined time interval (for example, 30 seconds), that portion It is determined that a thumbnail image in which the area is highlighted is generated. A thumbnail ID determined to be generated by the thumbnail generation determination unit 32 is assigned a thumbnail ID, and is stored in association with the partial area and the time range. Details of the data held in this way will be described below.

  The index data generation unit 34 generates each data acquired by the voice acquisition unit 18, the image acquisition unit 20, and the specified position information acquisition unit 22 when an instruction area determined to generate a thumbnail image is specified, and the generation Index table data that associates the ID (thumbnail ID) of the thumbnail image to be generated is generated. Here, the index table data generated by the index data generation unit 34 will be specifically described below.

  FIG. 4 shows an example of index table data. As shown in FIG. 4, the index table data includes one record storing data corresponding to each item of slide ID, area ID, instruction start time, instruction end time, thumbnail ID, audio data ID, and pointer data ID. Or it is table data comprised by two or more. Here, the thumbnail ID is an identifier given to the thumbnail image in which the corresponding area ID is highlighted, and the audio data ID is the period during which the area is instructed, that is, between the instruction start time and the instruction end time. It is an identifier assigned to the acquired audio data. The pointer data ID is pointer position data acquired while the area is designated. The pointer data ID may be represented, for example, by a set of pointer position data stored in the designated area history table shown in FIG.

  The thumbnail generation unit 36 generates a thumbnail image that highlights the instruction area determined by the thumbnail generation determination unit 32 based on the captured slide image. The thumbnail generation unit 36 reads the thumbnail ID and also reads the slide ID and area ID associated with the thumbnail ID. Then, a highlight image is applied to the area indicated by the area ID in the captured image associated with the read slide ID to generate a reference image. Then, the generated reference image is reduced to generate a thumbnail image. The generated thumbnail image is stored in the storage unit 12 in association with the thumbnail ID. The above process is performed for all thumbnail IDs. The highlighting may be any form as long as the area can be distinguished from other areas, for example, by enclosing the outside of the designated area with a frame.

  Next, a process for reproducing the presentation recorded by the above process will be described.

  The presentation playback unit 38 generates a moving image of the presentation by synchronizing the data based on the audio data, slide image data, and pointer data recorded on the presentation stored in the storage unit 12, and displays them on the display unit 16.

  FIG. 5 shows an example of a screen 70 (user interface) displayed on the display unit 16 when a computer application (presentation reproduction program) in which the presentation reproduction unit 38 is installed is executed. Here, the screen 70 shown in FIG. 5 includes a moving image display unit 72 that displays moving images and a thumbnail display unit 74 that displays thumbnail images 76, 78, and 80. The thumbnail display unit 74 displays a list of all or part of the stored thumbnail images. For example, the thumbnail display unit 74 may selectively display a thumbnail image generated in association with a slide image displayed and reproduced by the moving image display unit 72. The displayed thumbnail image can be selected in response to a user operation (for example, a mouse click with the cursor placed on the thumbnail image).

  The thumbnail selection unit 40 selects a thumbnail image from the thumbnail images displayed on the thumbnail display unit 74 in accordance with the user's operation, and sends the ID (thumbnail ID) of the selected thumbnail image to the presentation playback unit 38. Output.

  The presentation playback unit 38 refers to the index data table based on the thumbnail ID input from the thumbnail selection unit 40, specifies the audio data, slide image data, and pointer data to be played back, and based on the specified data. Then, a moving image of the presentation is displayed on the moving image display unit 72 and reproduced.

  Next, refer to the flowcharts shown in FIGS. 6 and 7 for the respective flows of the presentation recording process and the presentation playback process based on the recorded presentation data performed by the presentation recording / reproducing apparatus 10 according to the present embodiment. While explaining.

  First, presentation recording processing will be described with reference to the flowchart shown in FIG. As shown in FIG. 6, the presentation recording / reproducing apparatus 10 starts recording a presentation in response to a user operation (S101), and a slide image (or a screen including a slide image) displayed on the display unit 16 is displayed. In addition, the sound produced by the user who makes the presentation is recorded, and the position of the pointer pointing to the slide operated by the user is sequentially recorded every predetermined time (S102). With respect to the data thus recorded, the presentation recording / reproducing apparatus 10 analyzes the layout of the captured slide image, divides the slide image into regions (S103), and determines the position of the pointer based on the position on the slide image indicated by the acquired pointer. Is included in which of the divided areas (S104). Then, the presentation recording / reproducing apparatus 10 determines whether or not the time during which the pointer stays in the specified area exceeds a predetermined threshold (S105). If it is determined that the time exceeds (S105: Y), the specification is performed. In addition, a reference image (for example, a reduced image such as a thumbnail image) with a highlighted area is generated (S106), and audio data, slide image data, and pointer pointing position data are associated with the generated reference image and indexed. Store in the data table (S107). If it is determined in S105 that the residence time does not exceed the predetermined threshold (S105: N), the reference image is not generated, and the above processing is executed until the processing is completed for all the regions.

  Next, the presentation reproduction process performed based on the recorded presentation data will be described with reference to the flowchart shown in FIG. As shown in FIG. 7, when the presentation recording / playback apparatus 10 starts a presentation playback program for playing back the presentation data recorded in the storage unit 12 (S201), the index data table is referred to along with the first slide image. A playback screen displaying a list of all or part of the thumbnail images stored in association with the slide image is displayed on the thumbnail image display unit (S202). Here, when one of the thumbnail images displayed on the playback screen is selected by the user (S203), the presentation recording / playback apparatus 10 stores the thumbnail image in association with the ID of the thumbnail image selected from the index data table. The audio data, slide image data, and pointer position data are read (S204), and the video of the presentation is displayed on the moving image display unit 72 and reproduced based on the data (S205).

  According to the presentation recording / reproducing apparatus 10 according to the present embodiment described above, the presentation (description) using the displayed image (slide) is recorded and recorded so that the item desired to be viewed can be easily found. You can play your presentation. In the above embodiment, the recording device and the playback device are configured as the same device, but the recording device and the playback device may of course be configured as devices in separate housings.

  In the above embodiment, the obtained slide image is divided into a plurality of areas based on the layout, and the designated area is specified depending on which of the divided areas the designated position of the pointer is. The identification of the area is not limited to this method. For example, as in the second embodiment described below, a position distribution on the slide image of the pointer at predetermined time intervals is acquired, and a reference image is specified for the pointer indication area specified from the acquired position distribution. It may be generated.

  FIG. 8 shows a functional block diagram of the presentation recording / reproducing apparatus 11 according to the second embodiment. As shown in FIG. 8, the presentation recording / reproducing apparatus 11 includes a storage unit 12, a control unit 14, a display unit 16, a sound acquisition unit 18, an image acquisition unit 20, an indicated position information acquisition unit 22, an index data generation unit 34, A thumbnail generation unit 36, a presentation playback unit 38, a thumbnail selection unit 40, a time range setting unit 42, an indicated position distribution analysis unit 44, and an indicated area specifying unit 46 are included. In addition, since the functional block to which the same code | symbol as the functional block which concerns on 1st Embodiment shown by FIG. 1 was provided has the same function, description is abbreviate | omitted.

  The time range setting unit 42 sets a time range for analyzing the position distribution of the pointer. The time range setting unit 42 may set the time range to, for example, a predetermined value (every 2 minutes or the like), and the predetermined value may be appropriately changed by the user.

  The designated position distribution analysis unit 44 divides the slide image into meshes of a predetermined size, and calculates the pointer position distribution based on the mesh present in the pointer slide image for each time range set by the time range setting unit 42. To analyze. The designated position distribution analysis unit 44 may analyze the position distribution by measuring the number of times of designation and the designated time for each mesh, for example.

  The pointing area specifying unit 46 specifies the pointer pointing area in each time range based on the pointer position distribution analyzed by the pointing position distribution analysis unit 44. The instruction area specifying unit 46 may specify, as the instruction area of the pointer, a mesh that exceeds the threshold value set for each of the measured instruction count and the instruction time in the position distribution analyzed by the instruction position distribution analyzing unit 44. .

  In the presentation recording / reproducing apparatus 11, the captured slide image is divided into meshes as shown in FIG. FIG. 9B shows an example of the transition of the area specified as the pointer pointing position by the above method. In FIG. 9B, (1) to (5) are respectively the elapsed times from the start of recording of the presentation, for example, 0 to 2 minutes, 2 to 4 minutes, 4 to 6 minutes, 6 to 8 minutes, 8 It shows the area designated by the pointer at 10 minutes. Then, for each of the areas thus identified, a thumbnail image highlighting the area may be generated and stored in association with the image, sound, and pointer data of the presentation performed at each time.

  In the above embodiment, the presentation recording / playback apparatuses 10 and 11 capture the graphic image output to the projector such as a display or a projector to obtain the image data of the display slide. However, the present invention is not limited to this. Alternatively, the displayed slide image may be captured by a camera or the like to obtain slide image data. Also, when obtaining the position of the pointer, it may be obtained by calculating the position on the slide image based on an operation input of the mouse, or the obtained slide image is subjected to image processing and the difference from the background is used. Thus, the position of the pointer may be detected.

  In addition, this invention is not limited to said embodiment, Of course, a various change, deformation | transformation, or substitution is possible by those skilled in the art who have the normal knowledge of this field | area.

It is a functional block diagram of the presentation recording / reproducing apparatus 10 which concerns on this embodiment. It is a figure which shows an example of the partial area | region set to a slide image. It is a figure which shows an example of an instruction | indication area | region log | history table. It is a figure which shows an example of index table data. It is the figure which showed an example of the display screen of a presentation reproduction program. It is a flowchart of the recording process of a presentation. It is a flowchart of the reproduction | regeneration processing of a presentation. It is a functional block diagram of the presentation recording / reproducing apparatus 11 which concerns on 2nd Embodiment. It is a figure explaining the specification of the instruction | indication area | region which concerns on 2nd Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10,11 Presentation recording / reproducing apparatus, 12 Memory | storage part, 14 Control part, 16 Display part, 18 Audio | voice acquisition part, 20 Image acquisition part, 22 Instruction position information acquisition part, 24 Partial area setting part, 26 Instruction area specification part, 28 Indication area change detection unit, 30 dwell time measurement unit, 32 thumbnail generation determination unit, 34 index data generation unit, 36 thumbnail generation unit, 38 presentation playback unit, 40 thumbnail selection unit, 42 time range setting unit, 44 indication position distribution analysis 46, designated area specifying section, 50 slide image, 52, 54, 56, 58, 60 partial area, 70 screen, 72 moving image display section, 74 thumbnail display section, 76, 78, 80 thumbnail image.

Claims (8)

A presentation recording apparatus for recording a presentation performed while instructing a displayed image by an instruction unit,
Specifying means for specifying an instruction area of the image instructed by the instruction means;
Generating means for generating a reference image representing the designated area of the image specified by the specifying means in a predetermined manner;
A storage unit that stores the reference image generated by the generation unit in association with the image and sound related to the presentation performed when the instruction unit specified by the specifying unit is instructed by the instruction unit;
A presentation recording apparatus comprising: Means for obtaining the displayed image;
A partial area setting means for setting one or a plurality of partial areas in the acquired image;
The specifying means specifies the indication area from the partial areas set by the partial area setting means based on the position indicated by the indication means.
The presentation recording apparatus according to claim 1. The partial area setting means analyzes the layout of the acquired image and sets the partial areas based on the analyzed layout.
The presentation recording apparatus according to claim 2. The generating unit generates a reference image representing the designated area in a predetermined manner when the designated area identified by the identifying unit is continuously identified over a predetermined time;
The presentation recording apparatus according to claim 2 or 3, The specifying means specifies an indication area indicated by the indication means at each time based on a distribution of positions indicated by the indication means every predetermined time;
The presentation recording apparatus according to claim 1. In addition to the image and sound relating to the presentation, the storage means further stores data of the indicated position indicated by the instruction means.
The presentation recording apparatus according to claim 1, wherein the presentation recording apparatus is provided. Means for displaying one or more reference images stored in the storage means by the presentation recording apparatus according to any one of claims 1 to 6;
Replay means for replaying a presentation recorded based on at least an image and sound associated with the selected reference image when a reference image is selected from the displayed one or more reference images; including,
A presentation playback apparatus characterized by the above. A computer that records a presentation that is performed while instructing the displayed image by an instruction means,
Specifying means for specifying an instruction area of the image instructed by the instruction means;
Generating means for generating a reference image representing the designated area specified by the specifying means of the image in a predetermined manner;
The reference image generated by the generating unit, and the storage unit that stores the image and sound related to the presentation performed when the instruction unit specified by the specifying unit is instructed by the instruction unit. A program characterized by that.

Images