JP6476560B2 - Display device, display system, program, and image generation method - Google Patents

Description

  The present invention relates to a display device, a display system, a program, and an image generation method.

  Conventionally, in a meeting or the like, a display device that displays an image on a display screen is used, and a presentation is performed by switching an image or performing various user operations on the display screen. In recent years, display screens tend to increase in size.

  When the display screen becomes large, in order to assist user operations performed on the display screen, a reduced image obtained by reducing the original image is synthesized and displayed on the original image to be displayed, and a user operation on the reduced image is detected. A technique for reflecting in an original image is known (for example, see Patent Document 1).

  However, when a reduced image obtained by reducing the original image is combined and displayed on a large display screen, the reduced image is displayed on the displayed original image. Therefore, there is a problem that important display information of the original image is hidden by the reduced image, and the progress of the presentation is hindered.

  An object of one embodiment of the present invention is to provide a display device that arranges a reduced image so as not to hinder the display of important display information of the original image when displaying a display image in which the reduced image is arranged on a large display surface. It is to provide.

In view of the above problems, in one embodiment of the display device of the present invention,
A display device that displays a display image obtained by combining a display source image and a reduced image obtained by reducing the display source image in a display area of a display surface,
A storage unit having image information of the display source image and information on an area where the reduced image can be arranged;
A reduced image generation unit that generates a reduced image obtained by reducing the display source image stored in the storage unit;
A reduced image position determination unit that determines an arrangement position of the reduced image generated by the reduced image generation unit based on information about an area where the reduced image can be arranged;
A display image generation unit that generates a display image obtained by combining the display source image and the reduced image arranged at the determined arrangement position;
A display unit for displaying the display image generated by the display image generation unit on the display surface;
Have
For each type of display information of the display source image, the reduced image generation unit generates a reduced image in which only the necessary types are arranged according to setting information set by a user using the display device .

  According to one embodiment of the present invention, when displaying a display image in which a reduced image is arranged on a large display surface, a display device that arranges the reduced image so as not to hinder the display of important display information of the original image. Can be provided.

It is a functional block diagram which shows the structural example of the display apparatus which concerns on 1st Embodiment. It is a figure which shows an example of the data memorize | stored in the image information storage part. It is a flowchart which shows an example of the image generation process flow of the display apparatus which concerns on 1st Embodiment. It is a figure which shows an example of the reduction image arrange | positioned by the image generation process flow of FIG. It is a figure which shows the other example of the reduction image arrange | positioned by the image generation process flow of FIG. It is a flowchart which shows an example of the image generation process flow of the display apparatus which concerns on 2nd Embodiment. It is a figure which shows an example of the reduction image arrange | positioned by the image generation process flow of FIG. 10 is a flowchart illustrating an example of an image generation process flow of a display device according to a third embodiment. It is a figure which shows an example of the display screen which selects the determination of a moving direction in the image generation processing flow of FIG. It is a flowchart which shows an example of the image generation processing flow of the display apparatus which concerns on 4th Embodiment. It is a figure which shows an example of the reduced image produced | generated in the image production | generation process flow of FIG. It is a flowchart which shows an example of the image generation processing flow of the display apparatus which concerns on 5th Embodiment. It is a figure which shows an example of the display state of the display source image displayed on the display area. It is a figure which shows the other example of the display state of the display source image displayed on the display area. It is a figure which shows the other example of the display state of the display source image displayed on the display area. It is a figure which shows the other example of the display state of the display source image displayed on the display area. It is a functional block diagram which shows the structural example of the display system which concerns on 7th Embodiment.

  Hereinafter, embodiments of a display device, a display system, a program, and an image generation method according to the present invention will be described with reference to the accompanying drawings. In addition, this invention is not limited by the following embodiment. Moreover, each embodiment can be combined suitably as long as the content is not contradicted.

[First Embodiment]
<Configuration>
FIG. 1 is a functional block diagram illustrating a configuration example of the display device according to the first embodiment. The display device 1 of the present embodiment is preferably implemented when a presentation is performed by displaying display information on a large display surface.
In the present embodiment, the projection apparatus 1 will be described as an example of a display apparatus. Therefore, the following description will be given as the projection apparatus 1. In addition, the description of “display” in the claims or the specification is understood as “projection”. For example, the display source image is a projection source image, and the display image is a projection image. However, in some cases, it is simply a display.

  As shown in FIG. 1, the projection apparatus 1 includes a storage unit 10 and a control unit 20. The storage unit 10 stores data used in various processes by the control unit 20 and various processing results by the control unit 20. Specifically, it has an operation information storage unit 11 and an image information storage unit 12. The storage unit 10 is a semiconductor memory device such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory (Flash Memory), or a storage device such as an HDD (Hard Disk Drive) or an optical disk. .

  The operation information storage unit 11 stores user position information in a monitoring area that can be monitored and information related to user operations. The position information and the information related to the motion are information acquired from a human motion detection unit (not shown) connected to the projection device 1. The person motion detection means is, for example, an infrared sensor, a camera, an IR transmitter, an IR receiver, or the like, and detects position information and motion information of a user (operation instruction person). As will be described later, when the motion recognition unit 24 of the control unit 20 recognizes the presence of the user based on the detection information acquired from the human motion detection unit, the position information of the user and information related to the motion are stored in the motion information storage unit 11.

  The information obtained from the infrared sensor is position information specified by a signal that detects reflection of infrared rays when a user (operation instruction person) enters a predetermined area. The information obtained from the camera is the position information of the user identified by image recognition from the captured image obtained by imaging the display surface as a monitoring area by the camera. Also, operation information can be obtained from the locus of the user's hand. In addition, it is also preferable that an area of a reduced image is captured as a monitoring area, a user operation on the reduced image is detected, and stored in the operation information storage unit 11.

  When using an IR transmitter and an IR receiver, for example, an IR transmitter is built in a pointer or key holder of an operation instructor, and an IR receiver is provided in the vicinity of the projection device, so that the user's position information and Operation information is detected.

  The image information storage unit 12 stores an original image DB 12A that stores information related to a projection original image projected on the display surface, a reduced image DB 12B that stores information related to a reduced image obtained by reducing the projection original image, and a projection original image and a reduced image. A projection image DB 12C that stores information relating to the synthesized projection image (corresponding to an example of a display image in the scope of claims);

  The original image DB 12A has image data of the projection original image, page numbers, information on areas, and the like. As shown in FIG. 2, the information related to the area includes information related to size (size) and position information such as coordinates. Furthermore, it also has information on the type of display information, information on the area for each type of display information, necessity information for setting whether to display for each type of display information, and information on an empty area without display information. . As information about the area, vertical and horizontal sizes, start point coordinates (X, Y), and the like are described. Here, the starting point coordinates refer to the upper left coordinate position with respect to the facing view of the display surface, but are not limited thereto. The type of display information refers to elements such as a table, a graph, and a text sentence. The necessity information is information that is selected and determined in advance by the user.

  Further, as shown in FIG. 2, the original image DB 12A also has position information such as size (vertical and horizontal) and coordinates related to the display area. Here, the display area is an area within a rectangle having four sides. Further, when a projection source image is projected in the display area, if there is a blank area without display information outside the projection source image, information such as the start point coordinates and size of the blank area is also stored. The empty area without the display information may be recognized as an area where a reduced image can be arranged as described later.

  Further, the original image DB 12A also has necessity information on whether or not to project (display) a reduced image selected in advance by the user. As will be described later, when the size or image data of the projection source image is corrected by the control unit 20, information regarding the corrected projection source image is also stored. At this time, information on the free area in the display area newly formed by the correction is also stored (updated).

  The information regarding the area of the original image DB 12A is calculated as a result of image analysis of the original image by the projection image generation unit 25 described later. In addition, a method of including the original image file in advance is also conceivable.

  The reduced image DB 12B stores image data generated by reducing the projection source image, page number, area information (size information, position information such as coordinates), information about operation icons to be displayed in the reduced image, and the like. Has been. As the information related to the operation icon, for example, operation icon data for switching pages and position information thereof are stored. In addition, display information indicating whether or not a reduced image is displayed, and display control information such as a minimum setting of the reduced image are also stored. The display control information is composed of the above-mentioned information and is a group of information necessary for displaying the reduced screen.

  Although described later, the projection image DB 12C stores information related to a projection image obtained by combining a projection source image and a reduced image determined and generated by each unit of the control unit 20. For example, information on size, image data, page number, area, and the like.

  The control unit 20 has an internal memory for storing a control program, a program defining various processing procedures, and required data. The control unit 20 includes a reduced image generation unit 21, a reduced image position determination unit 22, an original image correction unit 23 (an example of a display source image correction unit), an action recognition unit 24 (an example of a recognition unit), and a projection image generation unit 25 ( An example of a display image generation unit) and a projection unit 26 (an example of a display unit).

  The control unit 20 is, for example, an integrated circuit such as AAIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), or an electronic circuit such as CPU or MPU. In addition, about each said part, these part or all may be software (program) and above-mentioned hardware circuit.

  The reduced image generation unit 21 has a function of generating a reduced image obtained by reducing the projection source image to a predetermined size. The predetermined size is set according to the size of the display surface, and has a size (vertical and horizontal) that is easy for a user (operation instruction person) to operate. Specifically, the reduced image generation unit 21 acquires information about the projection source image to be projected from the original image DB 12A, reduces the acquired projection source image to a predetermined size, and reduces the reduced image. Generate.

  The reduced image generation unit 21 does not only reduce the image data of the projection source image as it is and generate a reduced image. The reduced image generation unit 21 displays only the display information of the type selected by the user as “necessary” based on the display necessity information stored in the original image DB 12A and determined in advance by the user according to the type of display information. A compact reduced image with a minimum size is generated. Therefore, the space required for the arrangement of the reduced image can be minimized by making the size compact. This point will be described later.

Further, when the motion recognition unit 24 recognizes that there are two users in the monitoring area, the reduced image generation unit 21 may generate two reduced images according to the distance information of the two people. As appropriate. The number of reduced images is appropriately changed according to the number of users.
After generating the reduced image as described above, the reduced image generation unit 21 outputs the generated reduced image to the projection image generation unit 25 and stores it in the reduced image DB 12B.

  The reduced image position determination unit 22 has a function of determining and determining in which area (arrangement position) of the display area of the display surface the reduced image generated by the reduced image generation unit 21 is displayed. Further, information on the projection source image is acquired from the original image DB 12A, and it is determined whether or not there is an empty area without display information in which the reduced image can be displayed (or arranged) in the projection source image. When there is a vacant area that can be arranged, it is determined to display (place) the reduced image in the vacant area in the projection source image. Then, the display position is determined in consideration of the height position where the hand can reach in the vicinity of the user (operation instruction person) as the position for displaying the reduced image. Therefore, the reduced image is projected (displayed) in a free area that is close to the user and reachable by the user and that has no display information.

  Further, when there is no empty area sufficient to display (place) the reduced image in the projection source image, the projection source image can generate an empty area in the display area on the display surface to the original image correction unit 23 described later. Ask to fix.

When requested by the reduced image position determining unit 22, the original image correcting unit 23 has a function of correcting the projection original image and generating an empty area in the display area on the display surface.
For example, when there is no empty area in the projection source image where the reduced image can be placed, the projection source image can be corrected by reducing the size of the projection source image by independent scaling, The image is corrected to the corner or upper right corner (see FIG. 5). Therefore, an empty area is generated in the display area 100 on the display surface. The projection source image to be corrected is reduced by independent scaling, with the upper left corner or the upper right corner as the base coordinate, and in the illustrated example, an empty area in which the reduced image can be arranged is generated below the right end of the display area 100.

  As another method for correcting the projection source image, an empty area is generated in the display area 100 of the display surface by correcting the position of the projection source image to an image that has been moved (slided) upward or to the left and right. .

  Specifically, the projection source image is corrected to a moving image that is slid leftward or rightward, and an empty area in which a reduced image can be placed on the right end or the left end is generated (see FIG. 7). Whether to slide in the left or right direction is corrected according to the position information of the user recognized by the motion recognition unit 24 so that an empty area is formed near the user. Further, the projection source image can be moved upward by a similar method so that an empty area can be formed on the lower side.

With this correction, a part of the display information of the projection source image cannot be seen only in the moved area S. Therefore, it is preferable to let the user determine the movement in the upward or left-right direction. At this time, it is possible to slide the display information in the projection source image in a direction in which display information with low importance is not displayed. Then, it is possible to generate a free space where a reduced image can be arranged in a way that does not hinder the display of important display contents.
In the above-described correction, for example, there is no free space in the projection source image that is sufficient to arrange the reduced image. However, the present invention can be suitably implemented when there is an empty area that is not sufficient to place a reduced image on one or two of the four sides of the display area 100 outside the projection source image (FIGS. 15 and 15). 16).

  Information on the modified projection original image is stored (updated) in the original image DB 12A, and information on an empty area in the display area newly formed by the modification (information on an area where the reduced image can be arranged) is also included. Remembered.

The motion recognition unit 24 recognizes the position and operation of the user. Specifically, when the motion recognition unit 24 obtains detection information related to the presence (position) of a person and a user's motion from a human motion detection unit (not shown), the motion recognition unit 24 recognizes the user's location and the user's motion, and information about them. Is recorded in the operation information storage unit 11. The user position information can be recognized by a signal from an infrared sensor or the like.
Such user actions are sometimes referred to as gesture actions. Further, examples of user operations include an operation of moving a pointer and an operation of switching a page of a projected image (using an operation icon). In this specification, it may be described as a user operation.

For example, the user (operation instructor) moves his / her hand to move the pointer or switch the page of the projected image in a reduced image projected onto the display surface, for example. In other words, the motion recognition unit 24 uses the detection information from the camera, the IR transmitter, and the IR receiver, in particular, the user motion to move the pointer from the locus of the user's hand in the reduced image, and the projection image page. Recognize user action to switch.
Further, the motion recognition unit 24 calculates the coordinates of the position of the user's hand with respect to the captured image using the captured image of the camera that has recognized the user's motion to move the pointer, and stores the calculated coordinates in the motion information storage unit 11. Store.

  Then, the motion recognition unit 24 indicates that the user has recognized that the user is in the monitoring area, has recognized that the user has moved the pointer, and has recognized that the user has switched the projected image page. It outputs to the determination part 22 and the projection image generation part 25.

  The projection image generation unit 25 has a function of generating a projection image obtained by appropriately combining the projection source image and the reduced image. More specifically, the projection image generation unit 25 acquires information related to the projection source image from the original image DB 12A. When the reduced image position determination unit 22 determines that there is an empty area for displaying the reduced image in the projection source image, the image data of the original projection source image is acquired. . When the reduced image position determination unit 22 determines to correct the projection source image, a corrected projection source image corrected to a size or image data that can generate a free area in the display area is acquired. The projection image generation unit 25 acquires a reduced image from the reduced image DB 12B. When display of an operation icon is set in the reduced image, a reduced image in which the operation icon is arranged is acquired.

  When the projection source image acquired as described above and the reduced image are combined, a combining process corresponding to the first to third cases is performed.

  (1) The first case is a case where there is an empty area without display information in which a reduced image can be arranged in the projection source image. In that case, an uncorrected projection original image is acquired from the original image DB 12A of the image information storage unit 12, and a reduced image is acquired from the reduced image DB 12B. At this time, position information of the projection source image and the reduced image is also acquired, and a projection image in which the reduced image 102 is arranged in the empty area R in the projection source image 101 is generated as shown in FIG. The area where the reduced image can be arranged is an area where the user's hand can reach close to the user. Therefore, the reduced image is arranged in an area as close as possible to the user position according to the user position detected by the motion recognition unit 24.

  (2) The second case is a case where there is no empty area without display information in which a reduced image can be arranged in the projection source image. In that case, a reduced projection original image is acquired from the original image DB 12A of the image information storage unit 12, and a reduced image is acquired from the reduced image DB 12B. At that time, the position information of the projection source image and the reduced image is also acquired, and as shown in FIG. 5, the reduced projection source image 101 and the empty area R2 generated in the display area 100 of the display surface, A projection image obtained by arranging and reducing the reduced image 102 is generated.

  (3) The third case is also a case where there is no empty area without display information in which a reduced image can be arranged in the projection source image. In this case, a projection source image moved in the vertical or horizontal direction is acquired from the original image DB 12A of the image information storage unit 12, and a reduced image is acquired from the reduced image DB 12B. At this time, the positional information of the projection source image and the reduced image is also acquired, and the reduced image 102 is displayed in the projection source image 101 moved upward or left and right and the empty area (R3) generated in the display area 100 of the display surface. To generate a projected image synthesized (see, for example, FIG. 7).

  The projection images generated as described above are stored in the projection image DB 12C of the image information storage unit 12, and the projection images of appropriate pages are sequentially output to the projection unit 26.

  When the motion recognition unit 24 recognizes the movement of the pointer in the reduced image, the projection image generation unit 25 calculates the projection source image based on the position information (coordinates) of the pointer stored in the motion information storage unit 11. An image reflecting the position of the pointer is generated and displayed. When the motion recognition unit 24 recognizes the operation of switching pages on the operation icon, the projection image generation unit 25 has a function of outputting the projection image of the next page to the projection unit 26.

  The projection unit 26 projects a projection image on the display surface. More specifically, the projection unit 26 projects the projection image generated by the projection image generation unit 25 onto a display surface such as a display screen. Incidentally, in the case of a display device that displays display information on a display screen, the projection unit is, for example, a touch panel display device.

  Although not shown in the figure, the projection apparatus 1 may have an input unit for inputting information related to selection and setting from the user.

<Processing>
FIG. 3 is a flowchart illustrating an example of an image generation process flow of the projection apparatus according to the first embodiment.

  First, in A1, a camera or infrared sensor, which is a human motion detection means connected to the projection apparatus 1, detects that the user has entered the monitoring area (A1). Then, the human motion detection means recognizes that there is a user and a user motion (A2). Of course, the presence of the user may be detected by an IR transmitter, an IR receiver, or the like as the human motion detection means.

  The motion recognition unit 24 of the projection device 1 confirms whether the human motion detection means has detected and recognized the presence of the user and the user's motion (A3). If the human motion detection means does not recognize (NO), the process returns to A1.

When the motion recognition unit 24 receives the user position information or motion information from the human motion detection means in A3, the presence of the user and the motion of the user are recognized (YES), and the motion recognition unit 24 analyzes the person position information and A place is specified (A4). When a camera is used as the person motion detection means, the position of the person is specified from the images photographed by the camera. When infrared rays are used, the position of the person is specified from the position of the light receiving unit that receives the infrared rays. In A4, the motion recognition unit 24 further interprets the motion performed by the person, and determines whether the performed motion means an operation on the projection image (whether it is an operation by a gesture). As operations for the projected image, there are an instruction to switch the projected image, an instruction with a pointer, and the like.
The motion recognition unit 24 records the user position information and motion information acquired from the human motion detection means in the motion information storage unit 11. Further, the reduced image position determination unit 22 and the projected image indicate that the user has recognized that the user is in the monitoring area, has recognized the user action to move the pointer, and has recognized the user action to switch the page of the projected image. The data is output to the generation unit 25.

Next, in A5, the reduced image position determination unit 22 confirms whether a reduced image is being displayed in the display area 100 of the display surface. If the displayed information stored in the reduced image DB 12B is displaying a reduced image (YES), the process proceeds to A8.
When the displayed information stored in the reduced image DB 12B does not display the reduced image (NO), the process proceeds to A6.

  In A8, the reduced image position determining unit 22 obtains the display position information of the projection source image stored in the original image DB 12A of the image information storage unit 12 and the display position information of the reduced image stored in the reduced image DB 12B. read out.

In A9, the reduced image position determination unit 22 checks whether the reduced image hides the display information of the projection source image (there is no overlapping portion). That is, the determination is made by collating the display position and size of the read reduced image and the display position and size of the projection source image. The size includes not only the overall size but also a display information area, an empty area, and the like. If it is not hidden (not overlapped) (NO), the process proceeds to A14. In A14, it is confirmed whether the reduced image is displayed near the user. This is confirmed by referring to the user position information recognized by the motion recognition unit 24 and the position information of the reduced image read in A8. At this time, it is also confirmed whether the reduced image is located within the reach of the user's hand. When the reduced image is displayed in the vicinity of the user and at a height that can be reached by the user, the position determination processing flow is ended. When the reduced image is not displayed near the user (NO), the process proceeds to A10.
In A9, if the reduced image hides the display information of the projection source image (YES), the process proceeds to A10.

  In A10, the reduced image position determination unit 22 confirms whether or not there is an empty area without display information near the user in the projection source image with reference to the original image DB 12A.

  If there is an empty area (YES), the reduced image position determination unit 22 determines whether the reduced image can be arranged in an (empty) area without display information (A11). In FIG. 4, the portion indicated by the symbol R is an empty area. If there is not enough free space to place the reduced image (NO), the process proceeds to A15.

  In A11, when there is an empty area that can arrange a reduced image (YES), the reduced image position determination unit 22 displays the reduced image in the empty area (A12). That is, the projection image generation unit 25 is requested to generate a projection image synthesized by arranging the reduced image at the determined position in the projection source image.

  In A12, the projection image generation unit 25 acquires a projection source image from the original image DB 12A and a reduced image from the reduced image DB 12B. Then, as illustrated in FIG. 4, a projection image in which the reduced image 102 is arranged in the empty area R without the display information of the projection source image 101 is generated, and the projection image is projected to the display area 100 to the projection unit 26. Ask. The projection unit 26 displays (projects) a projection image in which the reduced image 102 is arranged in the empty area R without display information in the projection source image 101. The projection source image 101 shown in FIG. 4 is substantially the same as the display area 100 on the display surface, and shows a state where there is no empty area outside the projection source image 101.

  In A13, the reduced image position determination unit 22 records information on the display position of the reduced image determined in A11 in the reduced image DB 12B.

  Here, returning to A5, when the reduced image is not displayed in the display area 100 of the display surface (NO), the processing flow continued from A6 will be described.

  In A6, the reduced image position determination unit 22 confirms from the “necessity information” of the original image DB 12A whether the user has selected to display the reduced image. If the user has selected not to display the reduced image (NO), the position determination process flow is terminated.

  If the user has selected to display a reduced image (YES), the reduced image position determination unit 22 requests the reduced image generation unit 21 to generate a reduced image. Then, the reduced image generation unit 21 acquires information related to the projection source image including the image data from the original image DB 12A, and generates a reduced image (A7). Then, the reduced image generation unit 21 outputs the generated reduced image to the projection image generation unit 25.

  Next, in A10, when the reduced image position determination unit 22 determines that there is no free area near the user in the projection source image (NO), or in A11, the reduced image position determination unit 22 reduces to a free area near the user in the projection source image. A case where the image cannot be displayed (NO) will be described. The fact that the reduced image cannot be displayed means that there is no free space for displaying the reduced image. In that case, it progresses to A15.

  In A15, the reduced image position determination unit 22 causes the original image correction unit 23 to reduce the projection original image by independent scaling and to correct the image position to the corner of the display area. The original image correction unit 23 reads display control information stored in the reduced image DB 12B. In the display control information, the minimum size setting of the reduced image is used. First, the reduction ratio with respect to the width (horizontal) and height (vertical) is calculated by the following formula.

Reduction ratio: width = (width of projection source image− (minimum size of reduced image + left and right margins)) / width of projection source image × 100
Reduction ratio: Height = (height of projection source image− (minimum size of reduced image + upper and lower margins)) / height of projection source image × 100
The margins ((left and right) width, (up and down) height) are set to appropriate values on the system.
In the reduction correction of the present embodiment, independent scaling is performed to avoid distortion of the figure and the table. Therefore, it is preferable to perform reduction so as to maintain the aspect ratio of the projection source image.

  By using the reduction ratios of the width and height obtained by the following calculation formula, independent scaling is performed. That is, the reduction ratio: width and the reduction ratio: height obtained by the following calculation formulas are compared, and the larger value is defined as the reduction ratio ′ (common to width and height).

(Selection of reduction ratio)
'Reduction ratio' (common to width and height) =
It is assumed that (reduction ratio: width) <(reduction ratio: height) (reduction ratio: height).
It is assumed that (reduction ratio: width) ≧ (reduction ratio: height) (reduction ratio: width).

  In A15, the original image correcting unit 23 reduces and corrects the projection original image at the reduction ratio, outputs the projection original image to the projection image generating unit 25, and stores it in the original image DB 12A. At this time, the original image DB 12A stores not only information related to the reduced projection original image but also information related to an empty area (area where the reduced image can be arranged) in the display area newly formed by the reduction. The

  In A12, the projection image generation unit 25 acquires a reduced image from the reduced image DB 12B. Then, as shown in FIG. 5, a projection image in which the reduced image 102 is arranged in the reduced projection source image 101 and the empty area R2 of the display area 100 generated by the reduction is generated. Then, the projection unit 26 is requested to project the generated projection image onto the display area 100.

  The projection unit 26 displays (projects) the reduced projection original image 101 and the projection image in which the reduced image 102 is arranged in the empty area R2 outside the projection original image 101 and having no display information in the display area 100. To do. As described above, the reduced image 102 is disposed at the lower left position where the hand can reach in the vicinity of the user.

  Therefore, the projection apparatus 1 according to the first embodiment arranges a reduced image in an empty area without display information in the projection source image, and allows the smooth presentation to proceed without hiding the display information of the projection source image. Contribute. Further, even when there is no empty area in the projection source image where the reduced image can be arranged, the projection source image is appropriately reduced to create an empty area where the reduced image can be arranged in the display area 100 of the display surface. As a result, it is possible to prevent the display information of the projection source image from being hidden.

[Second Embodiment]
FIG. 6 is a flowchart illustrating an example of an image generation process flow of the projection apparatus according to the second embodiment. The configuration of the projection apparatus 1 is as shown in FIG.

Since the processing of B1 to B15 shown in FIG. 6 is the same as the processing of A1 to A15 shown in FIG. 3 described above, detailed description thereof will be omitted, and only the differences will be described.
In particular, if the reduced image position determination unit 22 determines in B10 that there is no empty area near the user in the projection source image (NO), or reduces to an empty area near the user in the projection source image in B11. A case where the image cannot be displayed (NO) will be described. In that case, it progresses to B21. The fact that the reduced image cannot be displayed means that there is no free space for displaying the reduced image.

  In B21, the reduced image position determination unit 22 determines whether to reduce the projection source image. When the projection source image is reduced (YES), the process proceeds to B15. B15 is the same as A15 described in FIG.

  The case where the projection source image is not reduced (NO) is, for example, a case where the display information in the projection source image is fine and small data, and the display information becomes difficult to see when reduced. This is a case where a reduced image is not sufficient to be placed in the display area 100 of the display surface, but there is an empty area without display information outside the projection source image (R62 in FIGS. 15 and 16). To R64). This determination is made, for example, when the size of the display information in the original image DB 12A is greater than or equal to a predetermined threshold, or when the user sets in advance, a determination is made not to reduce the projection original image.

  When the projection source image is not reduced (NO), the reduced image position determination unit 22 causes the original image correction unit 23 to correct the projection source image to an image that has been moved (slided) in the upward or left-right direction (B22). This is effective when a portion to be noted in the presentation is a part and there is no problem even if other display information (display information in the region S in FIG. 7) is not visible. If it is difficult to be invisible due to movement, it may be selected to reduce the projection source image in B21.

  In B22, the original image correction unit 23 reads display control information stored in the original image DB 12A. In the display control information, the minimum size setting of the reduced image is used. The amount of movement is calculated using the following formula.

<Calculation of travel distance>
Movement amount = Minimum size of reduced image + Blank In other words, the image is moved by the amount of the reduced image in which the minimum size and the margin are secured. This is because, as shown in FIG. 7, a part of the projection source image (the left side net portion in the drawing) is hidden when moving, so that it is necessary to set the movement amount to a minimum. The moving direction is moved toward the screen edge farther from the user position.

  The original image correction unit 23 records information such as image data and region information (position information) of the projection original image moved by the above method in the original image DB 12A and outputs the information to the projection image generation unit 25. At this time, the original image DB 12A stores not only information related to the moved projection original image but also information related to an empty area (area where a reduced image can be arranged) in the display area newly formed by the movement. Is done.

  In B12, the projection image generation unit 25 acquires a reduced image from the reduced image DB 12B. Then, as shown in FIG. 7, a projection image in which the reduced image 102 is arranged in the moved projection source image 101 and the empty area R3 of the display area 100 generated by the movement is generated, and the projection image is displayed in the display area. The projection unit 26 is requested to project to 100. The projection unit 26 displays (projects) the reduced projection original image 101 and the projection image in which the reduced image 102 is arranged in the empty area R3 in the display area 100. As described above, the reduced image 102 is disposed at the lower left position where the hand can reach in the vicinity of the user.

  The projection apparatus 1 according to the second embodiment displays the important display information of the projection source image and is far from the user position even when there is no empty area in the projection source image where the reduced image can be arranged. It is possible to create a vacant area for displaying a reduced image by moving toward the screen edge. Therefore, smooth the progress of the presentation.

[Third Embodiment]
FIG. 8 is a flowchart illustrating an example of an image generation process flow of the projection apparatus according to the third embodiment. The configuration of the projection apparatus 1 is as shown in FIG.

  The processes of C1 to C15, C21, and C22 shown in FIG. 8 are the same as the processes of A1 to A15 shown in FIG. 3 and B21 and B22 shown in FIG. 6, and a specific description thereof is omitted here. Only the differences will be described.

  In particular, in C21, the reduced image position determination unit 22 determines whether or not to reduce the projection source image, and the difference in subsequent processing will be described when the projection source image is not reduced (NO).

  In C21, when the projection source image is not reduced (NO), the original image correction unit 23 is corrected to an image obtained by moving (sliding) the projection source image.

At that time, the original image correcting unit 23 causes the user to select the moving direction of the projection original image (C31).
That is, as shown in FIG. 9, the selection icon K (an example of an operation icon) for selecting whether to move the projection source image upward or in the left-right direction is displayed on the user, and the user selects it. The user makes a selection by operating the pointer or the hand, and the above-described person motion detection means detects the user operation, and the motion recognition unit 24 recognizes the user operation and outputs it to the original image correction unit 23. .

  According to the above selection, the user can slide and hide a portion that may not be visible according to the position of important display information in the presentation explanation, and can create a free area in the display area 100, and proceed as desired. It can be carried out. This is when there is a lot of display information of the projection source image, when the content of the display information (characters, etc.) is small, or when important display information is concentrated on either the left or right edge or upper and lower ends of the projection source image It is effective for.

  In C22, the original image correcting unit 23 corrects the projection original image to an image moved according to the selected moving direction. Since the correction method of the original image correction part 23 is as above-mentioned, description is abbreviate | omitted. Further, a projection image in which the reduced image 102 is combined with the moved projection source image 101 and the empty area generated in the display area 100 by the movement is generated and projected by the projection unit 26. In addition, as described above, the above-described information is recorded in each DB of the image information storage unit 12.

  The projection apparatus 1 according to the third embodiment can move the projection source image in the direction desired by the user even when there is no empty area in the projection source image where the reduced image can be placed. Therefore, smooth presentation progress can be achieved by effectively creating an empty area for displaying a reduced image without hiding important display information.

[Fourth Embodiment]
FIG. 10 is a flowchart illustrating an example of an image generation process flow of the projection apparatus according to the fourth embodiment. The configuration of the projection apparatus 1 is as shown in FIG.

  The processing of D1 to D5 and D8 to D15 shown in FIG. 10 is the same as the processing of A1 to A5 and A8 to A15 shown in FIG. 3 described above. explain.

  In particular, in D6, the reduced image position determination unit 22 confirms whether or not the user has selected to display the reduced image from the original image DB 12A, and when the user has selected to display the reduced image (YES), Differences regarding these steps will be described.

  In the case of D6 (YES), the reduced image position determination unit 22 requests the reduced image generation unit 21 to generate a reduced image. Then, the reduced image generation unit 21 checks whether or not the user has selected whether or not to display the reduced image from the original image DB 12A according to the type of display information (D41).

  In D42, when the user has not selected necessity for each type or when “required” is selected for all types of display information, the reduced image generation unit 21 performs A7 in FIG. As described, a reduced image is generated by reducing the projection original image. However, for example, when “required” and “rejected” are selected for each type in the “necessity” column shown in FIG. 2, only the types for which “required” is selected are displayed as shown in FIG. The arranged reduced image 103 is generated. Then, the reduced image generation unit 21 outputs information about the generated reduced image to the projection image generation unit 25 and records it in the reduced image DB 12B. The subsequent processing is as described with reference to FIG. 3, and the reduced image is arranged and projected without impeding the display of important display information of the projection source image.

  Therefore, the projection apparatus 1 according to the fourth embodiment generates a compact reduced image composed of only necessary types of display information of the projection source image according to the necessity selection of the user, and reduces the size of the reduced image. Can be minimized. Further, since the size of the reduced image is reduced, the free area to be generated in the display area of the display surface by correcting the projection source image is also reduced. Therefore, the range for correcting the projection source image is reduced, problems such as difficulty in viewing can be reduced, and the operability for the user is improved, leading to a smooth presentation.

[Fifth Embodiment]
FIG. 12 is a flowchart illustrating an example of an image generation process flow of the projection apparatus according to the fifth embodiment. The configuration of the projection apparatus 1 is as shown in FIG.

  The processes E1 to E15 shown in FIG. 12 are the same as the processes A1 to A15 shown in FIG. 3 described above, and a specific description thereof will be omitted. Only differences will be described.

  In particular, in E6, the reduced image position determination unit 22 confirms whether or not the user has selected to display the reduced image from the original image DB 12A, and when the user has selected to display the reduced image (YES), Differences regarding the subsequent steps will be described.

  In the case of E6 (YES), the reduced image position determination unit 22 confirms with the motion recognition unit 24 whether there are two users in the monitoring area (E51). In E51, when there are not two persons, that is, there is only one person (NO), the process proceeds to E7. E7 is the same as A7 in FIG. 3 (or D42 in FIG. 10), and description thereof is omitted.

  In E51, when there are two users in the monitoring area (YES), the reduced image position determination unit 22 determines whether the distance between the two is close based on the user position information acquired by the motion recognition unit 24. (E52). The determination of whether the distance is near or far varies depending on the size of the display area 100 on the display surface. Therefore, a threshold value may be set with reference to the size of the display area of the display surface stored in the original image DB 12A, and determination may be made based on the threshold value. For example, if the threshold is 3 m, it is determined that the distance between the two is 3 m or more. In E52, for example, if it is determined that the distance between the two persons is 2 m and is close (YES), the process proceeds to E7.

If it is determined that the distance between the two persons is 3.5 m and the distance is long (NO), the reduced image position determination unit 22 requests the reduced image generation unit 21 to generate two reduced images. Then, the reduced image generation unit 21 acquires information regarding the projection source image including the image data from the original image DB 12A, and generates two reduced images (E53). At this time, the display information displayed on the two reduced images may be the same or may be displayed separately for each type.
Each of the above steps has been described by taking as an example the case where there are two users, but the same processing flow is also performed when there are two or more users.

  Therefore, in the projection apparatus 1 according to the fifth embodiment, even when a presentation is performed by a plurality of users, a dedicated reduced image is displayed according to the distance between the two people, so that the user operability is improved. Can give a smooth presentation. Even when two reduced images are displayed, the important display information of the projection source image is displayed so as not to be hidden, thereby preventing the presentation from being hindered.

[Sixth Embodiment]
The first to fifth embodiments described above can be appropriately and appropriately implemented according to the display state of the projection source image in the display area 100. In addition, the first to fifth embodiments can be appropriately combined to perform a good presentation. This selection can be performed by an input unit not shown. Of course, a selection icon may be displayed on the projection image for selection.

Next, as examples of the display status of the projection source image 101 in the display area 100, examples of FIGS. However, this is not the case with selection examples.
FIG. 13 shows a display state in which there is an empty area R60 in which the reduced image 102 (hereinafter also including the reduced image 103) can be placed near the user (lower position) in the projection source image 101 displayed in the display area 100. Indicated. In this case, the first embodiment can be suitably implemented. In particular, it is preferable to carry out the processing steps from A1 to A14.

  FIG. 14 shows a display state in which there is no empty area where the reduced image 102 can be placed in the vicinity of the user in the projection source image 101 displayed in the display area 100. There is an empty area R61 above, but it is an area that cannot be reached by the user and does not fit in the reduced image. Also in this case, the first embodiment can be suitably implemented. In particular, it is preferable that the process step A15 is performed and the process of reducing the projection source image 101 shown in FIG. 5 is performed.

  FIG. 15 shows a display state in which an empty area R62 having a size that is insufficient to display the reduced image 102 exists outside the projection source image 101 displayed in the display area 100. In the illustrated example, a free area R62 exists on the right side of the display area 100 having four sides. In this case, the second embodiment can be suitably implemented. In particular, the processing steps B21 to B22 are performed, and it is preferable to perform a process of moving (sliding) the projection source image 101 shown in FIG. 7 in the left-right direction.

  FIG. 16 shows a display situation in which a right empty area R63 and an upper empty area R64 that are not large enough to display the reduced image 102 exist outside the projection source image 101 displayed in the display area 100. showed that. In this case, the third embodiment can be suitably implemented. In particular, the processing steps C31 to C22 are performed, and it is preferable to perform processing for allowing the user to select whether the projection source image 101 shown in FIG.

  Therefore, according to the display status of the projection source image 101 displayed in the display area 100, the reduced image 102 is arranged at a display position that is optimal for the progress of the presentation, and the user's operability and smooth presentation are performed. Can do.

[Seventh Embodiment]
In the above embodiment, based on the data detected by the human motion detection means such as a camera or an infrared sensor, the user motion such as the presence of the user, location information, operation information is recognized, and according to the recognized user motion, The projection apparatus 1 that performs processing such as generation of a reduced image has been described. However, the projection apparatus 1 can be implemented as a projection system that is functionally or physically distributed or integrated in arbitrary units.

FIG. 17 is a functional block diagram showing an example of the device configuration of a display system (hereinafter referred to as a projection system) according to the seventh embodiment.
As illustrated in FIG. 17, the projection system includes a human motion detection unit 30, a projection device 40, and an information processing device 50. The person motion detection means 30 is a camera or an infrared sensor having a detection unit 31. The detection unit 31 transmits data related to the user operation to the information processing apparatus 50. The projection device 40 is a projector having a projection unit 41. The projection unit 41 has a function of projecting the projection image generated by the information processing apparatus 50 onto the display area 100 of the display surface.

  The information processing apparatus 50 includes a storage unit 510 and a control unit 520. The storage unit 510 has the same functions and information as each storage unit and DB of the storage unit 10 illustrated in FIG. 1. That is, it has an operation information storage unit 511 and an image information storage unit 512, and the image information storage unit 512 includes an original image DB 512A, a reduced image DB 512B, and a projection image DB 512C.

  The control unit 520 also has the same function as each functional unit of the control unit 20 shown in FIG. That is, it includes a reduced image generation unit 521, a reduced image position determination unit 522, an original image correction unit 523, an operation recognition unit 524, and a projection image generation unit 525. The difference is that the projection unit is mounted in the projection device 40. For each functional unit described above, the same processing as that of the functional unit having the same name as that of the projection apparatus 1 is executed, and thus detailed description thereof is omitted. In addition to that shown in FIG. 17, the human motion detection means 30, the projection device 40, and the information processing device 50 may be realized as an integrated device, and the combination thereof can be changed.

  Similarly, it goes without saying that the information processing apparatus 50 can be implemented as a distributed or integrated display system. Incidentally, in the case of a display system, the projector 40 can be implemented as a display device and a display unit.

  Information including processing procedures, specific names, various data, parameters, and the like shown in the document and drawings can be arbitrarily changed unless otherwise specified. Each element configuration of each illustrated apparatus is functionally conceptual, and does not necessarily have to be physically combined as illustrated. That is, the specific form of distribution or integration of each device is not limited to the one shown in the figure, and all or a part thereof is functionally or physically distributed or arbitrarily distributed in arbitrary units according to various burdens or usage conditions. Can be integrated. That is, the projection device 1 or the display device 1 described in the above-described embodiment may be configured by being incorporated in various systems according to the application and purpose.

<Program>
The program executed by the projection device 1 or the information processing device 50 is, as one form, a file in an installable format or an executable format, such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD. For example, the program is recorded on a computer-readable storage medium. The program executed by the projection apparatus 1 or the information processing apparatus 50 may be configured to be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. The program executed by the projection device 1 or the information processing device 50 may be provided or distributed via a network such as the Internet. The program may be provided by being incorporated in advance in a ROM or the like.

  The program executed by the information processing apparatus 50 has a module configuration including each functional unit in the control unit 520 described above. As actual hardware, a CPU (processor) reads a program from a storage medium and executes it, whereby the above-described functional units are loaded onto the main storage device, and the functional units within the control unit 520 are loaded onto the main storage device. It is to be generated.

  The embodiments of the present invention have been described in detail with reference to the drawings. In addition, said description is for understanding embodiment and does not limit the range of embodiment. Further, the plurality of embodiments described above are not mutually exclusive. Therefore, it is intended to combine the elements of different embodiments as long as no contradiction arises, and various modifications and changes are possible within the scope of the technical gist of the disclosure disclosed in the claims. .

DESCRIPTION OF SYMBOLS 1 Projection apparatus 10 Storage part 11 Motion information storage part 12 Image information storage part 12A Original image DB
12B reduced image DB
12C projection image DB
20 Control unit 21 Reduced image generation unit 22 Reduced image position determination unit 23 Original image correction unit 24 Motion recognition unit 25 Projected image generation unit 26 Projection unit 100 Display area 101 on display surface Projected source images 102 and 103 Reduced images R and R2, R3, R60 to R64 Free space

JP 2009-282737 A

Claims (9)

A display device that displays a display image obtained by combining a display source image and a reduced image obtained by reducing the display source image in a display area of a display surface,
A storage unit having image information of the display source image and information on an area where the reduced image can be arranged;
A reduced image generation unit that generates a reduced image obtained by reducing the display source image stored in the storage unit;
A reduced image position determination unit that determines an arrangement position of the reduced image generated by the reduced image generation unit based on information about an area where the reduced image can be arranged;
A display image generation unit that generates a display image obtained by combining the display source image and the reduced image arranged at the determined arrangement position;
A display unit for displaying the display image generated by the display image generation unit on the display surface;
Have
The reduced image generation unit generates a reduced image in which only the necessary types are arranged in accordance with setting information set by a user using the display device for each type of display information of the display source image. Display device. A recognition unit for recognizing the position of the user in the monitoring area;
The reduced image position determination unit
The display device according to claim 1, wherein an arrangement position of the reduced image is determined according to the position of the user recognized by the recognition unit.   A display source image correction unit that corrects the display source image so that a free area in which the reduced image can be arranged is generated in the display region, and stores the corrected display source image in a storage unit; The display device according to claim 1 or 2. The display source image correction unit
4. The display device according to claim 3, wherein a size of the display source image is reduced by independent scaling and corrected to an image brought close to a corner of the display area. The display source image correction unit
The display device according to claim 3, wherein the display source image is corrected to an image that is moved upward or leftward and rightward by an amount necessary for displaying the reduced image. The storage unit
It has information on the type of display information of the display source image and information on necessity of display on the reduced image for each type of display information,
The reduced image generation unit
2. The display device according to claim 1, wherein a reduced image in which only the types for which the key is selected is arranged is generated based on the necessity information of the storage unit corresponding to the setting information by the user. . A display system having an information processing device and a display device,
The information processing apparatus includes:
A storage unit having image information of a display source image and information on an area where a reduced image can be arranged;
A reduced image generation unit that generates a reduced image obtained by reducing the display source image stored in the storage unit;
A reduced image position determination unit that determines an arrangement position of the reduced image generated by the reduced image generation unit based on information about an area where the reduced image can be arranged;
A display image generation unit that generates a display image obtained by combining the display source image and the reduced image arranged at the determined arrangement position;
The display device
A display unit that displays the display image generated by the display image generation unit on a display surface;
The reduced image generation unit generates a reduced image in which only the necessary types are arranged in accordance with setting information set by a user using the display device for each type of display information of the display source image. Display system. A display device that displays a display image obtained by combining a display source image and a reduced image obtained by reducing the display source image in a display area of a display surface;
A storage unit having image information of the display source image and information on an area where the reduced image can be arranged;
A reduced image generation unit that generates a reduced image obtained by reducing the display source image stored in the storage unit;
A reduced image position determining unit that determines an arrangement position of the reduced image generated by the reduced image generation unit based on information relating to an area where the reduced image can be arranged;
A display image generation unit that generates a display image by combining the display source image and the reduced image arranged at the determined arrangement position;
Causing the display image generated by the display image generation unit to function as a display unit for displaying on the display surface;
The reduced image generation unit is a program for generating a reduced image in which only the necessary types are arranged according to the setting information set by a user using the display device for each type of display information of the display source image. An image generation method executed in a display device that displays a display image obtained by combining a display source image and a reduced image obtained by reducing the display source image in a display area of a display surface,
A storage step having image information of the display source image and information on an area where the reduced image can be arranged;
A reduced image generation step of generating a reduced image obtained by reducing the display source image stored in the storage step;
A reduced image position determining step for determining an arrangement position of the reduced image generated by the reduced image generation step based on information on an area where the reduced image can be arranged;
A display image generation step of generating a display image obtained by synthesizing the display source image and the reduced image arranged at the determined arrangement position;
A display step of displaying the display image generated by the display image generation step on the display surface;
Have
In the reduced image generation step, for each type of display information of the display source image, a reduced image in which only the necessary types are arranged is generated according to setting information setting by a user using the display device. Image generation method.

Images