WO2019039416A1

WO2019039416A1 - Display device and program

Info

Publication number: WO2019039416A1
Application number: PCT/JP2018/030604
Authority: WO
Inventors: 長谷川　洋
Original assignee: シャープ株式会社
Priority date: 2017-08-24
Filing date: 2018-08-20
Publication date: 2019-02-28

Abstract

This display device obtains image data for displaying one or more objects included in content as three-dimensional images, and also obtains input area information representing an input area Pb for the objects in a virtual space in which the three-dimensional images Pa of the objects are to be displayed. The display device displays the three-dimensional images Pa on the basis of the acquired image data, and detects the location in three-dimensional space of an operation element for subjecting the displayed three-dimensional images Pa to an input operation. The display device sets the input area Pb in the virtual space for the three-dimensional images on the basis of the input area information and the display location of the three-dimensional images Pa to be displayed, determines whether or not the operation element is included in the input area Pb on the basis of the location of the detected operation element, and executes prescribed processing when the operation element is included in the input area Pb.

Description

Display device and program

The following invention relates to a display device and a program.

2. Description of the Related Art Conventionally, there has been proposed a technique for displaying a three-dimensional image of a game character, receiving an input operation on a three-dimensional space by a user, and executing predetermined processing. Japanese Patent Application Laid-Open No. 2013-167938 discloses an information input device that receives such an input operation in a three-dimensional space. The information input device sets a plurality of rectangular parallelepiped space areas obtained by dividing the virtual space in front of the display screen (user side) as areas to receive input from the user, and input options are assigned to the respective areas. . The information input device detects three-dimensional coordinates of the user's finger, determines which rectangular solid area in the virtual space the detected three-dimensional coordinate is included in, and an area including the position of the user's finger Perform the corresponding processing.

In the case of Japanese Patent Application Laid-Open No. 2013-167938, since the input area in the virtual space where the user can input is fixed, 3D image content such as a game is restricted, or input is performed at an appropriate position according to the progress of the game. It becomes difficult to accept the operation.

The invention disclosed below aims to provide a technology that enables an input operation at an appropriate position according to 3D image content.

A display apparatus according to an embodiment includes image data for displaying at least one object included in content as a stereoscopic image, and input area information indicating an input area for the object in a virtual space in which a stereoscopic image of the object is displayed. Detection unit for acquiring an image, a display unit for displaying the stereoscopic image based on the image data, and detection for detecting a position in an three-dimensional space of an operating body for performing an input operation on the stereoscopic image to be displayed The input area of the virtual space for the stereoscopic image is set based on the unit, the display position of the stereoscopic image to be displayed, and the input area information, and the operation is performed based on the detected position of the operating body A determination unit that determines whether or not a body is included in the input area; and the operation body is included in the input area If it is determined, and a execution unit for executing predetermined processing, the input area is a predetermined range overlapping at least a portion of the stereoscopic image in the virtual space.

According to the above configuration, it is possible to receive an input operation at an appropriate position according to the 3D image content.

FIG. 1 is an image diagram for explaining a stereoscopic image and an input area in the embodiment. FIG. 2 is a functional block diagram of the display device in the embodiment. FIG. 3 is an image diagram for explaining the input area of the modification (1). FIG. 4 is an image diagram for explaining an input area different from that in FIG. FIG. 5 is an image diagram for explaining an input area different from those in FIGS. 3 and 4. FIG. 6 is an image diagram for explaining an input area different from those in FIGS. 3 to 5.

A display apparatus according to an embodiment of the present invention is an input indicating image data for displaying at least one object included in content as a stereoscopic image, and an input area for the object in a virtual space where the stereoscopic image of the object is displayed. A position in an three-dimensional space of an operating tool for performing an input operation on the displayed three-dimensional image and a display unit for displaying the three-dimensional image based on the image data and an acquisition unit for acquiring area information The input area of the virtual space for the stereoscopic image is set based on the detecting unit, the display position of the stereoscopic image to be displayed, and the input area information, and the detected position of the operating body is determined. A determination unit that determines whether the operating body is included in the input area; and the operating body is in the input area If it is determined that Murrell, and a execution unit for executing predetermined processing, the input area is a predetermined range overlapping at least a portion of the stereoscopic image in the virtual space (a first configuration).

According to the first configuration, the input area is set for the stereoscopic image of the object included in the content. The input area is a predetermined range overlapping with at least a part of the three-dimensional image in the virtual space, and when the position of the operating body in the three-dimensional space is included in the input area, predetermined processing is performed. Therefore, it is possible to receive an input operation at an appropriate position according to the object in the content without the content being restricted.

In the first configuration, the input area information may include information indicating a range of each input area for each of a plurality of portions in the three-dimensional image of the virtual space (second configuration). According to the second configuration, since a plurality of input areas can be set for an object, the degree of freedom of input operation can be improved.

In the first configuration, the acquisition unit acquires the plurality of image data for displaying each of a plurality of objects as a stereoscopic image, and the input area information for the stereoscopic image in the virtual space of each of the plurality of objects. The display unit displays a plurality of stereoscopic images on the same screen based on the plurality of image data, and the determination unit determines that the detected position of the operating body is the stereoscopic image of the plurality of objects. It may be determined whether it is included in any one of the input areas (third configuration). According to the third configuration, it is possible to receive an input operation for each of stereoscopic images of a plurality of objects.

In any one of the first to third configurations, the determination unit may reset the input area for the stereoscopic image according to a change in a display position of the stereoscopic image (fourth configuration). According to the fourth configuration, the input area can be set at an appropriate position according to the display position of the object.

A program according to an embodiment of the present invention is an image data for displaying at least one object included in content as a stereoscopic image on the display unit on a computer of a display device having the display unit, and a stereoscopic image of the object Obtaining an input area information indicating an input area for the object in the virtual space in which the image is displayed, a display control step of causing the display unit to display the stereoscopic image based on the acquired image data, and Detecting the position in the three-dimensional space of the operating body for performing an input operation on the three-dimensional image, and based on the display position of the three-dimensional image to be displayed and the input area information, Set the input area of the virtual space for the A determination step of determining whether the operation body is included in the input area, and execution means for executing a predetermined process when it is determined that the operation body is included in the input area; And the input area is a predetermined range overlapping with at least a part of the three-dimensional image in the virtual space (first program).

According to the first program, an input area is set for a stereoscopic image of an object included in the content. The input area is a predetermined range overlapping with at least a part of the three-dimensional image in the virtual space, and when the position of the operating body in the three-dimensional space is included in the input area, predetermined processing is performed. Therefore, it is possible to receive an input operation at an appropriate position according to the object in the content without the content being restricted.

Hereinafter, embodiments of the invention will be described in detail with reference to the drawings. The same or corresponding parts in the drawings have the same reference characters allotted and description thereof will not be repeated. In order to make the description easy to understand, in the drawings referred to in the following, the configuration is simplified or schematically shown, or some constituent members are omitted. Also, the dimensional ratios among the components shown in the drawings do not necessarily indicate the actual dimensional ratios.

(Overview of Display Device)
The display device in the present embodiment displays, for example, a stereoscopic image (3D image) of an object representing a character such as a game. The display device receives an input operation from the user for the displayed 3D image, and when the input operation is performed on an input area preset for the displayed 3D image, the input Execute processing according to the operation.

The 3D image is composed of display images for the right and left eyes that represent the same object. An object is stereoscopically viewed by utilizing the parallax generated by shifting the positions of the display image for the right eye and the display image for the left eye.

Here, FIG. 1 shows a 3D image displayed on the display device in the present embodiment and an image diagram showing an input area set for the 3D image. In this example, the 3D image Pa of the object displayed on the display surface D is viewed at a position where the display surface D protrudes to the user side as viewed from the user (X-axis positive direction side).

In the present embodiment, in the virtual space where the 3D image Pa is displayed on the user side of the display surface D, a rectangular parallelepiped input area Pb for receiving an input operation is set. The input area Pb is invisible to the user. When the user performs an input operation on the input area Pb, the display device executes a process according to the input operation. In the present embodiment, it is preferable that the 3D image can be stereoscopically viewed by the user with the naked eye. Here, although the case where the shape of the input area Pb is a rectangular parallelepiped shape will be described as an example, the shape of the input area Pb is not limited to this as long as it is a three-dimensional shape.

(Configuration of display device)
Hereinafter, the configuration of the display device of the present embodiment will be specifically described. FIG. 1 is a block diagram showing a functional configuration of the display device 1 in the present embodiment. The display device 1 includes a control unit 10, a position detection unit 11, a display unit 12, and a storage unit 13.

The position detection unit 11 detects the position (coordinates) in the three-dimensional space of the operating body used for the input operation as needed, and outputs position information indicating the detection result to the control unit 10. The operating body may be the user's finger.

The position detection unit 11 can use, for example, a three-dimensional position sensor. A three-dimensional position sensor is attached to the user's hand, and position information detected by the three-dimensional position sensor is output to the input determination unit 111 by wire or wirelessly. The position detection of the operation body is not limited to the three-dimensional position sensor, and an imaging unit such as a camera may be used. When using a camera, for example, one camera is disposed on each side of the display surface D, and the user's finger is photographed by each camera, and the user's finger is photographed based on the photographed image using triangulation method or the like. Three-dimensional position may be detected.

The display unit 12 includes, for example, a liquid crystal display panel or an organic EL display panel, and displays each image of an object in the game program under the control of the control unit 10.

The storage unit 13 is realized by, for example, a non-volatile memory such as a mask ROM (Read-Only Memory), a flash memory, and an EEPROM (Electrically Erasable Programmable Read-Only Memory). The storage unit 13 stores scenario data including data such as an object according to the progress of the game. Specifically, scenario data stores input area information including the position and size of the input area Pb for each object in time series, as well as storing 3D image data for displaying the object as a 3D image in time series Remember. The scenario data stores execution processing information that defines the display mode of the object with respect to the input area Pb, such as changing the expression or posture of the object, for example, when an input operation is performed on the input area Pb.

The 3D image data includes image data for right eye and left eye for each time-series object, and information indicating display positions of image data for right eye and left eye.

The apparent position of the 3D image Pa in the depth direction with respect to the display surface D, that is, the position in the virtual space is defined according to the display positions of the image data for the right eye and the left eye of the object. The input area Pb is set at a position overlapping with a predetermined part such as the head and body of the object (3D image Pa) in the virtual space. That is, in the present embodiment, a predetermined range based on a predetermined position (coordinates) on the 3D image Pa in the virtual space is stored in advance as the input area information of the input area Pb.

The control unit 10 includes functional units of an input determination unit 111, a 3D image display control unit 112, and a process execution unit 113. The control unit 10 has a CPU and a memory (ROM and RAM), and the CPU executes a game program stored in the ROM to execute game processing to realize the functions of the above-described units. The game program may be stored in the storage unit 13 together with the scenario data.

The 3D image display control unit 112 displays the 3D image Pa by causing the display unit 12 to display the image data for the right eye and the left eye of the object based on the scenario data in the storage unit 13. Further, the 3D image display control unit 112 performs a process of changing the display mode of the object in accordance with an instruction from the process execution unit 113 described later. The 3D image display control unit 112 may be realized by a GPU (Graphics Processing Unit).

The input determination unit 111 reads input area information of the input area Pb for the object displayed by the 3D image display control unit 112 from the storage unit 13, and sets the coordinate range of the input area Pb for the object. Note that the input determination unit 111 resets the coordinate range of the input area Pb along with the object based on the display position of the time-series object that changes as the game processing progresses.

Further, the input determination unit 111 acquires position information of the operating tool from the position detection unit 11 as needed, converts the acquired position information into coordinates in the virtual space using a predetermined coordinate conversion table, and converts the converted coordinates into virtual The three-dimensional coordinates indicate the position of the operating body in space. The input determination unit 111 processes the determination result indicating that the input operation has been performed on the input area Pb when the three-dimensional coordinates indicating the position of the operating body in the virtual space are included in the input area Pb. Output to 113.

When the process execution unit 113 acquires from the input determination unit 111 the determination result indicating that the input operation has been performed on the input area Pb, the process execution unit 113 refers to the execution process information of the scenario data in the storage unit 13 and The 3D image display control unit 112 is instructed to perform processing according to the request.

In the above embodiment, the input area Pb is set for each object at a position overlapping the predetermined portion of the 3D image Pa of the object, and the input area Pb is set again according to the display position of the object. Therefore, the input area Pb can be set in conjunction with the movement or state of the object, compared to the case where the range in which the input operation can be received is fixed, and the input operation can be performed at an appropriate position according to the progress of the game. It becomes possible to receive. Further, in the above embodiment, since the user can stereoscopically view the 3D image with the naked eye and operate on the 3D image in real space, when operating on the 3D image using an HMD (Head Mounted Display) Compared to, you can get a more realistic game feeling.

As mentioned above, although an example of a display was explained, a display is not limited to composition of an embodiment mentioned above, but can be considered as various modification composition.

<Modification>
(1) In the embodiment described above, an example in which one input area Pb is set for an object has been described, but as shown in FIG. 3, two input areas Pb1 and Pb2 for a 3D image Pa of an object It may be set. The point is that the number of input areas Pb set for one object may be at least one or more.

Further, as shown in FIG. 4, input areas Pb11 and Pb12 may be set for each of 3D images Pa1 and Pa2 of two objects displayed on the same screen.

Also, for example, in a certain scene, as shown in FIG. 5A, the input area Pb is set for the 3D image Pa of the object. In the next scene, as shown in FIG. 5 (b), an input area Pb 'at a position different from the input area Pb on the 3D image Pa shown in FIG. 5 (a) may be set. Also, for example, in a certain scene, as shown in FIG. 6A, input areas Pb31 and Pb41 are set for the 3D image Pa of the object. In the next scene, as shown in FIG. 6B, input areas Pb32 and Pb42 of different sizes from the input areas Pb31 and Pb41 on the 3D image Pa shown in FIG. 6A may be set. That is, at least one of the size and the position of the input area Pb with respect to the object may be changed according to a scene change or the like.

(2) In the embodiment described above, an example in which 3D image data and input area information are stored in the display device 1 has been described. However, 3D image data and input area information are stored in a storage device external to the display device 1 May be stored. For example, the external storage device may be connected to the display device 1 via a wired or wireless communication line, or the external storage device may be a semiconductor memory or the like that can be attached to or detached from the display device 1. Good. In this case, the display device 1 includes an acquisition unit that acquires 3D image data and input area information from an external storage device, displays a 3D image based on the acquired 3D image data on the display unit 12, and acquires the input area Based on the information, set an input area for the 3D image.

(3) In the embodiment described above, it is preferable that the user stereoscopically view the 3D image displayed on the display device 1 with the naked eye and perform an input operation on the 3D image, but displays the 3D image using the HMD, An input operation may be received on the 3D image.

Claims

Image data for displaying at least one object included in the content as a stereoscopic image, and an acquiring unit for acquiring input area information indicating an input area for the object in a virtual space where the stereoscopic image of the object is displayed;
A display unit for displaying the three-dimensional image based on the image data;
A detection unit configured to detect a position of an operating body in a three-dimensional space for performing an input operation on the displayed stereoscopic image;
The input area of the virtual space for the three-dimensional image is set based on the display position of the three-dimensional image to be displayed and the input area information, and the operating body is the one based on the detected position of the operating body. A determination unit that determines whether or not it is included in the input area;
An execution unit configured to execute a predetermined process when it is determined that the operation body is included in the input area;
The display device, wherein the input area is a predetermined range overlapping at least a part of the three-dimensional image in the virtual space.
The display device according to claim 1, wherein the input area information includes information indicating a range of each input area with respect to each of a plurality of portions in the stereoscopic image of the virtual space.
The acquisition unit acquires a plurality of image data for displaying each of a plurality of objects as a stereoscopic image, and the input area information for a stereoscopic image in the virtual space of each of the plurality of objects.
The display unit displays a plurality of stereoscopic images on the same screen based on the plurality of image data,
The display according to claim 1, wherein the determination unit determines whether or not the detected position of the operating body is included in the input area of any one of the stereoscopic images of the plurality of objects. apparatus.
The display device according to any one of claims 1 to 3, wherein the determination unit resets the input area for the stereoscopic image according to a change in a display position of the stereoscopic image.
In a computer of a display device having a display unit,
Acquisition to obtain image data for displaying at least one object included in content as a stereoscopic image on the display unit, and input area information indicating an input area for the object in a virtual space where the stereoscopic image of the object is displayed Step and
A display control step of causing the display unit to display the stereoscopic image based on the acquired image data;
Detecting the position of the operating body in the three-dimensional space for performing an input operation on the displayed three-dimensional image;
The input area of the virtual space for the three-dimensional image is set based on the display position of the three-dimensional image to be displayed and the input area information, and the operating body is the one based on the detected position of the operating body. A determination step of determining whether or not included in the input area;
An execution unit configured to execute a predetermined process when it is determined that the operation body is included in the input area;
The program, wherein the input area is a predetermined range overlapping at least a part of the stereoscopic image in the virtual space.