JP2000348215A - Virtual sense of reality multiwindow system and spreadsheet application software - Google Patents

Abstract

PROBLEM TO BE SOLVED: To operate inherent information processing in an optional window among a plurality of relevant picture windows for information processing while displaying the windows as a list in one multiwindow image display device. SOLUTION: This virtual sense of reality multiwindow system has a window object generating means 1 which generates a three-dimensional CG window object from the display content of a picture window for information processing, a virtual reality space controlling means 2 which arranges the window object in a three-dimensional space and a three-dimensional CG image displaying means 3 which shows the window object in the three-dimensional space from a given viewpoint. Thus, it is possible to perform an inherent information processing operation in an optional window of a plurality of relevant picture windows for information processing while displaying the windows as a list on one three- dimensional CG image display device 3.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multi-window system or a multi-computer system window display technique in which screen windows of related application software such as a personal computer and a workstation are used in a multiplex manner.

[0002]

2. Description of the Related Art With the advance of information processing technology, an information processing environment in an office has been greatly improved by a large-capacity personal computer, workstation, and its software having a small size but high-speed processing performance.
Currently, in office work and system development work, multiple related application software are multiplexed on one computer, multiple PCs are placed on a desk, and multiple computers connected by lines etc. More and more, he works while looking at multiple related screen windows.

On the other hand, the size of a normal computer display screen is determined in consideration of an easy gaze movement angle of about 10 ° with respect to a human gaze viewing angle (about 3 °). Specifically, a screen with a diagonal of 10 to 20 inches is frequently used. Then, XGA (10
24 × 768 pixels) or SXGA (1280 × 102
(4 pixels). When a window that secures the character size and the number of characters per line that can be easily seen by humans is arranged on this screen, one window occupies the entire screen, and even in a multi-window system, only one active window is in front. Inactive windows are often hidden behind. Also,
When performing related tasks on multiple computers,
Since a plurality of screen display devices are viewed side by side, the movement of the line of sight when comparing the screens is large, and the work efficiency is deteriorated.

On the other hand, the progress of panel display technology using LCD (liquid crystal) and plasma display is remarkable,
At present, as an example, 2560x called QSXGA
A prototype of a high-definition 16-inch LCD panel with 2048 pixels has been manufactured (see Nikkei Microdevices Supplement “Flat Panel Display”, p190-191, 1999, Nikkei BP). Such a high-definition screen has the ability to display a plurality of windows therein without overlapping and to provide a sufficiently visible work environment. However, it has not been possible to guide a plurality of computer outputs to the single high-definition screen and display the list. Further, as a multi-window screen of spreadsheet application software of a single computer, such a high-definition panel cannot display a plurality of spreadsheets in a list in a plurality of non-overlapping windows.

Also, virtual reality (Virtua) utilizing three-dimensional computer graphics (hereinafter referred to as CG) technology.
A device capable of experiencing a virtual image of a virtual space with a wide field of view has been developed by using technology (hereinafter, referred to as VR). VRML (Virtual Reality Modeling Language) that describes VR objects created by such VR technology in an extended language of HTML and exchanges them among a plurality of computers.
e) VR object drawing technology using publicly known (for example, the book “VRML 2: Moving 3D Graphics” by Shigeru Nakayama, published in August 1997, ISBN4-7655 by Gihodo Shuppan)
-3319-0). However, a technique for arranging information processing screen windows of a plurality of computers as VR objects in the same VR space has not yet been disclosed.

[0006]

As described above, in the conventional information processing environment, a plurality of information processing screen windows are displayed in a list on a single screen display device, and an arbitrary one of the windows is displayed. There has been a problem that the unique information processing operation cannot be performed.

[0007]

The object of the present invention is to provide a window object generating means 1 for generating a three-dimensional CG window object from the display contents of an information processing screen window, as shown in FIG. The three-dimensional CG image display means 3 displays the window object in the three-dimensional space from a given viewpoint.

That is, the window object generating means 1 is software arranged in the information processing apparatus 10 on which the information processing application software 1011 operates, and stores the information processing screen window created by the information processing application software 1011 in the frame memory 103.
3D CG which reads out from this and gives a 3D CG image
It is generated as a window object, and this object is sent to the virtual reality generation device 20.

On the other hand, the virtual reality space control means 2 operates in the virtual reality generation device 20 to define the three-dimensional space for displaying the window object in the three-dimensional space sent from the window object generation means 1 and the viewpoint. Determine. Also, the three-dimensional CG image display means 3 in the virtual reality generation device 20
Reads the window object in the three-dimensional space,
This is expanded in the display memory 200, and the three-dimensional CG viewed from this viewpoint is referred by referring to the separately provided viewpoint parameter E.
An image is generated and this is used as the video display signal V1.
It is sent to the dimensional CG image display device 23.

The above operation is repeated in accordance with the plurality of information processing screen windows created by the information processing application software 1011 of the information processing apparatus 10, thereby overlapping the plurality of information processing screen windows created by the information processing application software 1011. Instead, the three-dimensional CG image arranged can be given to the three-dimensional CG image display device 23. The operator uses the information processing keyboard / mouse device 12 of the information processing device 10 to write information in one of the information processing screen windows while viewing the three-dimensional CG image of the three-dimensional CG image display device 23.

In the virtual reality multi-window system according to a second aspect of the present invention, the operator wears the three-dimensional CG image display device 23 on the head, and
A gaze detecting means 22 is provided, the positional relation of which is linked with the gaze detecting means 22. The virtual reality space control means 2 in the virtual reality generating device 20 receives the gaze information S from the gaze detecting means 22, and then sets the viewpoint in the three-dimensional space. Since the parameter E is calculated, the three-dimensional C
The G image display means 3 is a three-dimensional C that is always linked to the operator's line of sight
A G image can be provided.

Further, in the virtual reality multi-window system according to the first and second aspects of the present invention, the window object generating means 1 operates to activate the window object in the three-dimensional space every time the screen input by the information processing keyboard / mouse device 12 is interrupted. To the virtual reality generator 20
Since the control is returned to the information processing application software 1011 after the completion of the transmission, it takes some time to reflect the input to the three-dimensional CG image on the three-dimensional CG image display device 23, and there is a fear that a poor response may be given. is there.

In order to improve this, in the virtual reality multi-window system according to the third aspect of the present invention, the display switching means 4 is provided in the virtual reality generating device 20 so that the information processing screen window of the information processing device 10 is changed. The video display signal V2 to be provided is also provided to the three-dimensional CG image display device 23. That is, the display switching means 4 receives from the virtual reality space control means 2 as a switching signal X that the operator designates the window object in the three-dimensional space to be watched through the display control input unit 21 or the line-of-sight detection means 22. The video display signal V0 output to the three-dimensional CG image display device 23 by the three-dimensional CG image display means 3 is output as a three-dimensional image.
Is switched to a video display signal V2 for providing an information processing screen window, and a screen corresponding to the information processing display device 11 is also displayed. With the above operation, the three-dimensional CG image display device 23 is equivalent to a dedicated display device of the information processing device 10, and the screen input update operation is performed with a comfortable response. Also, three-dimensional CG image display means 3
If the operator wants to return to the original video display signal V1 that is output as a three-dimensional image, the operator again operates the display control input unit.
What is necessary is just to give an instruction | indication using 21 or the gaze detection means 22.

The spreadsheet application software according to claim 4 of the present invention is different from the above, and as shown in FIG. 2, the spreadsheet application software is located at the same position as the information processing application software 1011 arranged in a single information processing apparatus 10. 6. Sheet window generating means
The sheet window 611 generated by 61 is displayed on the information processing display device 11 at a position designated by the operator. By appropriately arranging the sizes and positions of the sheet windows, the operator can display a plurality of sheets in a list on the information processing display device 11 without overlapping.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a virtual reality multi-window system will be described with reference to FIG. 1 and FIGS. An embodiment of spreadsheet application software will be described with reference to FIG. Note that the computer processing according to the present invention is executed on a main storage device of the computer by a computer program, and the provision form of the computer program includes an auxiliary storage device connected to the computer and a floppy (registered trademark).
It is provided by being stored in a recording medium such as a portable storage device such as a disk or a CD-ROM or a main storage device and an auxiliary storage device of another computer connected to a network. It is loaded on the main storage of the computer and executed.

FIG. 1 shows the configuration of an embodiment of a virtual reality multi-window system. The virtual reality multi-window system according to claims 1 to 3 of the present invention includes a case where there is one information processing device 10 and a case where there are a plurality of information processing devices, but this embodiment shows a case where there are a plurality of information processing devices. On the plurality of information processing apparatuses 10, information processing application software 1011, for example, Microsoft Excel, Word, Visual Basic, or the like is operating. Conventionally, such information processing application software 1011 displays the window screen on each information processing display device 11 when these screens are related to the same business, and the operator operates each information processing keyboard / mouse device 12. I was doing business using it. In the present invention, these window screens are displayed on a single three-dimensional CG image display device 23, and the operator looks at the window screens and uses the information processing keyboard / mouse device 12 corresponding to each window to perform a task. Is what you do.

The window object generating means 1 of the present invention is located in the OS on each information processing apparatus 10, and linked to the generation and update of the screen window of each information processing application software 1011, corresponding to the window. Generate and update a window object in the dimensional space.
In this embodiment, the configuration and operation of the window object generating means 1 of one information processing apparatus 10 having Microsoft Windows 95 as an OS will be described.

Here, the window object in the three-dimensional space is a flat object for displaying a general screen window displayed on the information processing display device 11 as a three-dimensional CG image as shown in FIG. is there. The window object generating means 1 corresponds to the window object in the three-dimensional space corresponding to all the windows which have been generated on each information processing display device 11 and which have not disappeared (that is, include active windows as well as inactive windows). Generate

The name of this object is
Machine id (mid), the process id (pid) of the process that created the window, the window id (wid) that is the serial number of the window created in this process,
W-mid-pid-wid. Therefore, all windows managed by the virtual reality generation device 20 can be uniquely identified.

The three-dimensional space in which the objects are arranged has a coordinate system having a unique unit vector length e, while each object has an intra-object coordinate system. here,
Each window object in the three-dimensional space is a flat plate of thickness e, and the window size is e times the original screen size. Therefore, as shown in FIG. 3, the window has a window size of ww * e × wh * e, Window thickness e
(Represented by a Box node in the VRML geometry field). And the frame memory 103
(Corresponding to the file name W-mid-pid-wid.
gif) is attached.

FIG. 4 shows an example in which the above-mentioned window object in the three-dimensional space is described by VRML code. The window object generating means 1 sends such a window object in the three-dimensional space to the three-dimensional CG image display means 3. The method will be described below.

First, FIG. 5 shows an outline of the overall operation of the virtual reality multi-window system of the present invention. FIG. 5A is an overall operation flow. When the power is turned on, the virtual reality generation device 20 sets a default three-dimensional space as an initial setting and displays it on the three-dimensional CG image display device 23 (step S).
501). In this case, the viewpoint position is set to the initial setting value,
Also, no screen of information processing device 10 is displayed. 3
At the bottom of the screen of the dimensional CG image display device 23, only the dialogue buttons generated by the virtual reality generation device 20 are displayed as shown in FIG.

The virtual reality generating device 20 executes step S50.
2. As shown in step S504, "the information processing apparatus 10
Windows changed in Windows? ], “Is the space and display changed by the interactive screen? ], And the steps S503 and S505 are dealt with by these events.

The change of the window screen of the information processing apparatus 10 is effected by a user's operation using the information processing keyboard / mouse device 12.
The window object generating means 1 is informed from the window control function of S (Windows 95 in this embodiment), and the window object generating means 1 transmits the information to the virtual reality generating device 20 based on this. Step S503 shows such an operation of the window object generating means 1, and details thereof will be described later with reference to FIG.

On the other hand, while viewing the multi-window image on the three-dimensional CG image display device 23, the user uses the interactive buttons at the bottom of the screen shown in FIG. You can make changes.
In FIG. 5 (b), a portion indicated by a triangle is a button for requesting a change as described below, and the current mode is indicated by coloring.

Look: Switching between viewpoint input mode / arrangement change mode When this button is turned on by clicking, the viewpoint input mode is set and viewpoint movement can be input by dragging the cursor. When the button is turned off by clicking the button, the mode changes to the arrangement change mode, and the arrangement of the object in the space can be changed by placing the cursor on the object and dragging.

Icon: Iconization When this button is clicked to light up, it enters iconification mode, and when an object is clicked, it is iconified and moves to a predetermined icon area.

Window: windowing When this button is clicked to light up, the windowing mode is set. When the icon in the icon area is clicked, the window becomes the original window and is displayed at the original arrangement location.

HMD: Switching of visual line input enable / disable.
The line of sight input mode from 22 is entered, and the cursor can be moved according to the movement of the line of sight. At this time, only the buttons of the mouse of the display control input unit 21 are enabled, and the cursor is moved with the line of sight.

TVinTV: Original screen insertion mode When this button is turned on by clicking this button, the original screen insertion mode is set, and the original information processing display device 11 corresponding to the clicked object is displayed on the three-dimensional CG image display device.
It is superimposed and displayed in a predetermined area on 23.

The operation of the above step S505 is performed by the virtual reality space control means 2 and the three-dimensional CG in the virtual reality generation device 20.
This is realized by the interlocking operation of the image display means 3 and the display switching means 4. The details will be described later in detail with reference to FIG.

The information processing application software 1011 in the information processing apparatus 10 uses many windows. These windows are generated / deleted / updated by the application requesting the OS. Windows95
In the application, multi-process and multi-thread processing are performed. Application is CreateProces
Ask the OS to activate the process with s (). The process is split into multiple threads, the main thread of which is started first. The OS assigns thread information, virtual input queue, and window to each thread.
Create an NTEXT structure. Then, window control is performed using a thread RIT (Raw Input Thread) for transmitting a KB mouse input to the current active window (active thread).

The relationship between the operation of the OS and the window object generating means 1 will be described with reference to FIG. When a KB mouse input is performed using the information processing keyboard / mouse device 12, the device driver firstly inputs the input to the RIT queue, a Z order (screen depth), an active state, a mouse capture state, a mouse cursor shape, The display / non-display of the mouse cursor is determined. The window object generating means 1 is activated as a hard event at each time. This is to inform the window object generating means 1 that the display contents of the active window are changed by the character input or the cursor position change each time.

In this way, the refresh unit of the window object generating means 1 is activated, and as shown in FIG. 6B, the bitmap image of the active window is pasted from the frame memory 103 to the corresponding window object in the three-dimensional space. Texture file W-
Copy to mid-pid-wid.gif. Then, the window event sending section of the window object generating means 1 issues a window event to notify the virtual reality generation device 20 that the texture file has been refreshed, and stores the file name (URL) in the virtual reality generation device 20.
To send to.

Depending on the value of the KB mouse input, a new window is created, deleted, an existing window is resized,
In addition, there may be an instruction to change the display of the window. The RIT determines these, and if it is necessary to update the window, sends an update instruction to SendMessage ()
To tell the window procedure. The window object generation unit of the window object generation means 1 is started at this timing, and uses this information to execute the processing shown in FIG.
As shown in (c), a corresponding window object in the three-dimensional space is generated or updated. FIG. 4 shows a description example of the generated window object in the three-dimensional space. This description conforms to VRML 2.0, but the part with * is the field and route control newly extended for the present invention. These will be described later with reference to the operation flow of the three-dimensional CG image display means in FIG. In FIG. 4, the portions marked with ◎ are portions that convey the correspondence with the screen of the information processing display device 11 and will be described below.

◎ 1 is a translation field of a Transform node, which designates an arrangement (amount of parallel movement) of this object in a three-dimensional space. xp yp Z gives the window coordinates and depth of the original screen as initial values when the window is created. If this value is too large, the window will be out of view, giving the effect of disappearing.

◎ 2 is the url field of the ImageTexture node, the image file name of the texture image W-mid-pid-wid.gi
f is specified. This corresponds to the window image of the original screen, and its contents are refreshed every time there is a change as described in FIG. 6 (b).

◎ 3 is the size field of the Box node, which specifies the width, height and thickness of this flat object by ww * e wh * ee. This is the original screen window size ww × wh
It corresponds to.

The route control portion indicated by △ is a portion for realizing the object arrangement change by the user, and causes a parallel movement in a three-dimensional space when the corresponding window object is dragged (FIG. 5). Step S
505 (a) realization).

Next, FIG. 7 shows a configuration example of the virtual reality generation device 20. The virtual reality generation device 20 includes a virtual reality space control unit 2, a three-dimensional CG image display unit 3, and a display switching unit 4. FIG. 5 of step S505
(b) to (f) are controlled by the virtual reality space control means 2 by the operation shown in FIG. The interactive screen of FIG. 5B described above is displayed in step S801. At the same time, the space is set in step S501 in FIG. The mode setting section 202 of the virtual reality space control means 2 shown in FIG.
It monitors click events and drag events from 21. In step S802, it is checked whether or not there is a click event at the position of the "look" button. If there is (step S803), the mode is switched to the viewpoint input mode or the arrangement change mode, and the corresponding lighting / extinguishing is performed.

In the viewpoint input mode, the input switching unit 203 guides the drag event of the display control input unit 21 exclusively to the viewpoint movement amount calculation unit 204, and enters the mode in which the viewpoint parameter E is sent to the three-dimensional CG image display means 3. On the other hand, in the arrangement change mode, the mode setting unit 202 guides the drag event exclusively to the event management unit 303 of the three-dimensional CG image display unit 3 to set the environment mode in which the above-described object movement indicated by the symbol in FIG. 4 is executed.

In step S804, it is checked whether or not there is a click event at the icon button position. If there is (step S805), the process of clicking on the object is changed. That is, in the iconification mode, the mode setting unit 202 receives a click event on the object, sends an instruction I to the iconification processing unit 205, and the iconization processing unit 205 executes a change Ix for iconifying the window object in the three-dimensional space. It is sent to the three-dimensional CG image display means 3. In the case of a window object in a three-dimensional space, the iconification is simply a size change and a position change, and it is only necessary to change the numerical values of the translation field and the size field described above. In this embodiment, these values are fixed in the system.

In step S806, it is checked whether or not there is a click event at the position of the window button. If there is (step S807), the process of clicking the icon area is changed. That is, in the windowing mode, the mode setting unit 202 receives the click event of the icon area, sends an instruction W to the windowing processing unit 206, and the windowing processing unit 206 changes the icon back to the original window object in the three-dimensional space. Wx is sent to the three-dimensional CG image display means 3. This processing is also simply a size change and a position change, and merely restores the numerical values of the translation field and the size field described above.

In step S808, it is checked whether or not there is a click event at the position of the HMD button. If there is (step S809), the mode setting unit 202 instructs the input switching unit 203 to switch. In other words, switching is performed between the normal mode in which the viewpoint movement in the three-dimensional space is performed by the drag event of the display control input unit 21 and the visual line mode performed by the visual line information S from the visual line detection unit 22. Input switching unit 203
Selects these inputs in accordance with the switching between the normal mode and the line-of-sight mode, and transmits these inputs to the viewpoint movement amount calculation unit 204 only in the viewpoint input mode described above. The configuration and operation of the visual line detection means 22 will be described later with reference to FIG.

Finally, in step S810, TVinTV
Check if there was a click event at the button position,
If there is (step S811), the mode setting unit 202 instructs the display switching means 4 to switch. That is,
In the original screen insertion mode, the mode setting unit 202 obtains a machine id designating the information processing apparatus 10 from a click on the window object in the three-dimensional space, and transmits it to the display switching means 4 as a switching signal X. In response to this signal, the display switching means 4 obtains the video display signal V2 from the information processing device 10 and outputs it to the three-dimensional CG image display means 3.
A video signal V0 synthesized with a predetermined part of the video signal V1 is output to the three-dimensional CG image display device 23. By this combination, a screen equivalent to the information processing display device 11 is created in the three-dimensional CG image display device 23. On the other hand, when the mode is switched to the normal mode, the display switching means 4 supplies the three-dimensional CG image display device 23 with the video display signal V1 as the video display signal V0.

Next, the operation of the three-dimensional CG image display means 3 will be described with reference to FIG.
It will be explained by. The configuration of the three-dimensional CG image display means 3 is shown in FIG. The drawing unit 304 of the three-dimensional CG image display means 3 is VR
It has an execution function of the ML browser, and generates a video display signal V1 for displaying the window object in the three-dimensional space supplied from the display memory 200 in the three-dimensional space by the viewpoint parameter E.

In step S901 of FIG. 9, only the initial three-dimensional space is drawn. As described above, when the information processing apparatus 10 is started, the information processing application software 10
11 creates various windows. Each time, the window object generating means 1 generates a corresponding window object in the three-dimensional space and sends it to the three-dimensional CG image display means 3. At this time, the window event synchronized with SendMessage () is received by the display temporary memory generation unit 301 of the three-dimensional CG image display means 3, and is sent to step S9 in FIG.
02 or in step S904.

If the VRML code file is a designated window event (step S 903), the display temporary memory generation unit 301 reads the specified file and generates the window object in the three-dimensional space on the display memory 200. In the case of a window event designating the change of the existing code (change of the translation and size field values) (step S905), the display temporary memory generation unit 301 changes the field value in the display memory 200.
The drawing unit 304 performs drawing reflecting this change, so that a corresponding window is generated on the three-dimensional CG image display device 23,
They disappear and resize.

The hardware event corresponding to the input from the information processing keyboard / mouse device 12 activates the route control unit 302 of the three-dimensional CG image display means 3 (step S90).
7). The route control unit 302 is a high-speed routine created especially for a hard event, and stores an image file provided by the window object generation unit 1 in the display memory 200.
Transfer to * 1, 2, and 3 in Fig. 4 indicate this route control unit.
This is a route control code that specifies 302, which is a normal VRM
Not implemented in L 2.0.

* 1 adds a refresh field to the ImageTexture node, and instructs the route control unit 302 to reread the file indicated by the url field each time this field value becomes TRUE. * 2 is a window event,
A new sensor node, WinIRTSensor, that receives a hard event in association with an object.
Define the sensed field which becomes * 3 is a route control in which the sensed value is set to refresh in the event, and the route control unit 302 receives this designation and refreshes the texture of the display memory 200.

In step S908, the virtual reality generation device 20
The event management unit 303 refreshes the texture of the display memory 200 by performing the refresh operation specified in FIGS. 4 * 1 and 4 by the timer event (step S909). . Step S910 is the same as step S910 in FIG.
When the arrangement change mode described in 803 is ON, it is checked that the drag event of the display control input unit 21 interrupts the event management unit 303. This operation is executed by the normal VRML route control function shown by the mark in FIG. 4 (step S911).

Finally, the configuration and operation of the visual line detecting means 22 will be described with reference to FIG. The line of sight detecting means 22 is mounted on the head of the user, and has a head tracker 2201 for detecting the direction of the head and an eye tracker 2202 for detecting the direction of the eyeball. Head tracker 22
Reference numeral 01 denotes three angular velocity sensors for detecting angular velocity of rotation about three orthogonal axes. The outputs of these sensors are integrated to determine the amount of rotation, and the direction of the head is detected. This output is the relative movement amount (xh, yh,
zh). On the other hand, the eye tracker 2202 is composed of a light emitting unit that emits light of a wavelength that cannot be sensed by the human eye and its light receiving unit, receives light from the light emitting unit reflected by the eyeball at the light receiving unit, and appears in the image The direction of the eyeball is detected from the position of the pupil. This output is obtained by a vector amount (xe, ye) as a relative movement amount on a coordinate system fixed to the head. Therefore, the output of the line-of-sight detecting means 22 is obtained as (xh + xe, yh + ye, zh) obtained by combining these. The gaze detecting means 22 outputs (xh + xe, yh + ye, zh) as the gaze information S. This is the input format of the amount of movement of the viewpoint by the mouse of the display control input unit 21 (for example, horizontal drag, vertical drag,
The oblique drag) is transmitted in one operation, and the operability of moving the viewpoint is greatly improved.

[0053]

As is apparent from the above description, according to the present invention, a plurality of information processing screen windows are displayed in a list on a single multi-window screen display device, and an arbitrary one of the windows is displayed. There is an effect that the unique information processing operation can be performed. That is, in the method using the virtual reality space control means, which is the first solution, a plurality of information processing screen windows related to one three-dimensional CG image display device as a plurality of three-dimensional space window objects are listed. While displaying the information, the user can perform the unique information processing operation in any of the windows. Further, as a second solution, by providing a spreadsheet application software with a sheet window generating means, a plurality of spreadsheets can be displayed on a multi-window screen in a non-overlapping window, and the spreadsheet can be displayed in any of the windows. A unique information processing operation can be performed.

[Brief description of the drawings]

FIG. 1 is an embodiment of a virtual reality multi-window system.

FIG. 2 is a configuration example of a spreadsheet application software of the present invention.

FIG. 3 shows components of a window object in a three-dimensional space.

FIG. 4 is a description example of a window object in a three-dimensional space.

FIG. 5: Overall operation of a virtual reality multi-window system

FIG. 6 shows a configuration example and an operation example of a window object generation unit.

FIG. 7 is an embodiment of a virtual reality generation apparatus;

FIG. 8 is an operation flow of the virtual reality space control means.

FIG. 9 is an operation flow of the three-dimensional CG image display means.

FIG. 10 is a configuration example of a gaze detection unit.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Window object generation means 2 Virtual reality space control means 3 Three-dimensional CG image display means 4 Display switching means 6 Spreadsheet application software 21 Display control input part 22 Eye-gaze detection means 23 Three-dimensional CG image display device 61 Sheet window generation means 201 Dialogue Screen display unit 202 Mode setting unit 203 Input switching unit 204 Viewpoint movement amount calculation unit 301 Temporary memory generation unit for display 302 Route control unit 303 Event management unit 304 Drawing unit 611 Sheet window

 ──────────────────────────────────────────────────の Continued on the front page F term (reference) CC17 EA05 EA07 EA10 EA11 EA34 EB01 EB05 FA02 FA03 FA04 FA06 FA14 FA23 FA24 FA27 FB04 FB22 FB34

Claims (5)

[Claims] 1. A window object generating means for generating a three-dimensional CG window object from display contents of an information processing screen window; a virtual reality space control means for arranging the window object in a three-dimensional space; A three-dimensional CG image display means for displaying the inner window object from a given viewpoint. 2. A three-dimensional CG image display device that forms an image of an output of the three-dimensional CG image display means and forms a virtual image of the three-dimensional space on an operator's retina by an optical system, and a position of a line of sight of the operator. The gaze detection means to be detected is attached to the operator's head, and the virtual reality space control means receives gaze information from the gaze detection means and determines a viewpoint in the three-dimensional space based on the gaze information. The virtual reality multi-window system according to claim 1, wherein: 3. A display switching means for at least partially switching and outputting a video display signal for providing the information processing screen window and a video display signal output by the three-dimensional CG image display means as a three-dimensional image. The virtual reality multi-window system according to claim 1 or 2, wherein: 4. Spreadsheet application software having a function of displaying a plurality of spreadsheets on a multi-window screen in non-overlapping windows. 5. A window object generating means for generating a three-dimensional CG window object from display contents of an information processing screen window; a virtual reality space control means for arranging the window object in a three-dimensional space; A computer-readable recording medium recording a program for realizing a three-dimensional CG image display unit for displaying an inner window object from a given viewpoint.