EP1425653A2

EP1425653A2 - Desktop manager

Info

Publication number: EP1425653A2
Application number: EP02777079A
Authority: EP
Inventors: Bernd 3Dconnexion GmbH GOMBERT; Bernhard Von Prittwitz
Original assignee: 3DConnexion GmbH
Current assignee: 3DConnexion GmbH
Priority date: 2001-09-13
Filing date: 2002-09-12
Publication date: 2004-06-09
Also published as: WO2003023592A3; WO2003023592B1; WO2003023592A2; US20040046799A1

Abstract

The invention relates to a desktop manager program that allows extension of the graphical user interface (3) of conventional screens and PCs by the free positioning of the displayed section of the user interface by way of a 3D input device (1, 1') in such a manner that the user can thereby determine himself the visible part of a user interface (3) of a screen (6) and a PC (4). The selection of this visible part, a kind of virtual window (2), can be made using an input device (1, 1') with at least three degrees of freedom. Two degrees of freedom are used to navigate a virtual window (2) on the user surface (3). Another degree of freedom is used for the adjustment of an enlargement/reduction ratio with respect to the objects on the user surface (3) within the virtual window (2). The virtual window can thus be defined only as a part of the entire display surface of the screen (6). If the user interface (3) is displayed on the display of the screen (6), the virtual window can be navigated across the user interface (3) by means of the input device as a kind of magnifying glass with an adjustable enlargement ratio.

Description

desktop Manager

The present invention relates to a method for managing user interfaces, to a computer software program for implementing such a method and to the use of a force / moment sensor for such a method.

The general background of the present invention is the management of graphical user interfaces on which symbols are arranged, the arrangement being generally freely selectable by the user. According to a definition, "desktop" is the designation for the visible work surface of the graphical user interface of, for example, Microsoft Windows or OS / 2. "Desktop" is normally a work area on the screen that contains symbols and menus, around the surface of a To simulate desks. A desktop, for example, is characteristic of window-oriented programs such as Microsoft Windows. The purpose of such a desktop is the intuitive operation of a computer, since the user can move the pictures of objects and start and end tasks in almost the same way as he is used to from a real desk.

Since, according to one aspect of the invention, a force / torque sensor is used as an input device for such a desktop program, the state of the art with regard to Force / torque sensors are explained.

Force / torque sensors, the output signals with respect to a force / torque vector acting on them and thus output signals with respect to mutually independent different Providing degrees of freedom (for example, three translational and three rotational degrees of freedom) are known from the prior art. Additional degrees of freedom can be provided by switches, rotary wheels, etc., which are permanently assigned to the force / torque sensor.

DE 199 52 560 AI discloses a method for adjusting and / or adjusting a seat of a motor vehicle using a multifunctional, manually operated input device with a force / moment sensor. Such a force / torque sensor is shown in FIG. 6 of DE 199 52 560 AI. In this regard, the technical details of such a sensor refer to this figure and the associated description for DE 199 52 560 AI. In DE 199 52 560 AI, the input device has a user interface on which a number of areas are provided for entering at least one pressure pulse. The input device has a device for evaluating and recognizing a pressure pulse detected by means of the force / moment sensor and converted into a force and moment vector pair. After such a selection, for example of a seat or seat part of a motor vehicle to be controlled, the selected device can then be controlled linearly by means of an analog signal from the force / torque sensor. According to this prior art, the selection of a function and the subsequent activation are thus separated into two processes that are separated from one another in time.

From DE 199 37 307 AI it is known to use such a force / moment sensor for controlling operating elements of a real or virtual mixing or control desk, for example in order to create and design novel coloring, light and / or tone compositions , Here, the intuitive spatial control in three translational and three rotational degrees of freedom can be advantageously transferred to a continuous spatial mixing or control of a large number of optical and / or acoustic parameters. For control purposes, the user interface of the A pressure is exerted on the input device and thereby a pulse is generated which is detected with the aid of the force / moment sensor and converted into a vector pair consisting of a force and a moment vector. If certain characteristic impulse specifications are met, an object-specific control operation and / or a technical function can be triggered by switching to an activation state or terminated by switching to a deactivation state.

Starting from the above-mentioned prior art relating to force / moment sensors and desktop programs, it is the object of the present invention to further develop the desktop technology in such a way that the management of user interfaces (desktop surfaces) can be made even more intuitive.

This object is achieved according to the invention by the features of the independent claims. The dependent claims further develop the central idea of the invention in a particularly advantageous manner.

The central finding of the invention is that a user of a real desk arranges various documents on the desk surface in accordance with an intuitive, user-specific working behavior. This aspect has already been taken into account in classic desktop technology, i.e. translated into the world of the graphical user interface.

According to the present invention, however, it is possible for the first time to navigate a virtual window - similar to microfiche technology (microfilm with microcopies arranged in rows) - relative to a user interface. In order to stay with the microfiche analogy, the user interface under the virtual window can be moved in three dimensions, for example. In a desktop manager program according to the invention, it is thus possible for the first time to extend the graphical user interface of conventional monitors by freely positioning the user interface in relation to the virtual window by means of a 3D input device such that the user can use it to see the visible part of a monitor user interface and / or determine its display scale.

It should be pointed out once again that the following definitions are used in the context of the present description:

"User interface": The total of the (virtual) area available to the user for the arrangement of symbols

"Desktop", "virtual window": definable section of the user interface displayed on the monitor

The user interface can therefore be larger than the desktop, depending on the desktop. In this case, the entire user interface is not displayed on the monitor. However, it is also possible to equate the size of the desktop with the entire user interface.

A further finding in the present invention is that the user first takes a certain distance (“lean back”) in order to obtain an overview of the workplace. After recognizing desired documents etc. by means of this overview, the focus is then turned to interesting working documents This is reproduced by the invention in that the enlargement / reduction factor of a virtual window can be changed, which essentially corresponds to a zoom effect with regard to the objects located within the window of the viewer are gradually directed towards certain screen objects (working documents, icons, etc.).

This effect is more precisely achieved according to the invention in that objects are first arranged on a user interface, for example by the user. As is known as such, the user can add, delete or move objects and scale the display size of the objects.

This step corresponds to arranging documents on a desk, for example. Then, according to the invention, a virtual window with an adjustable enlargement / reduction factor can be navigated with respect to the user interface, which corresponds to a focus that can be changed in terms of position and viewing angle.

It is particularly advantageous if an input device is used which provides control signals in at least three mutually independent degrees of freedom. It is thus possible to navigate three-dimensionally with respect to the user interface, with control signals in two degrees of freedom for the

Positioning and the further control signal for the

Setting the enlargement / reduction factor

(corresponding to a change in the viewing angle of the focus).

More specifically, according to the present invention, a method for managing objects on a graphical user interface is provided. First, objects are arranged on the user interface by the user. Finally, a virtual window can be navigated with respect to the overall user interface configured in this way, the content of the window being displayed on the screen in each case.

As already explained above, it can be particularly advantageous to use an input device that generates control signals in at least three degrees of freedom. Control signals in Two degrees of freedom are used for the positioning of the virtual window with respect to the user interface and that

Control signal in the third degrees of freedom for the

Enlargement / reduction function used.

The input device can provide control signals in at least three translational and / or rotational degrees of freedom. This input device can in particular be a force / moment sensor.

Alternatively, an input device can also be used Navigation (for example a computer mouse) can be used, which is physically assigned an element for generating a control signal in a third degree of freedom. This element can be, for example, an additional switch, a rotary wheel or a button.

The virtual window can correspond to the entire display area of a screen. Thus, when the zoom function is carried out, the size of all objects on the total user surface changes to the same extent.

Alternatively, however, it is also possible to define the virtual window only as part of the total display area of the screen. If the entire user interface is then displayed on the display area of the screen, the input device can be used to navigate the virtual window as a type of “magnifying glass” with an adjustable magnification factor with respect to the user interface, so that the user interface can be moved under the “magnifying glass”, so to speak.

The software programs to be managed can in particular be office applications, such as word processing or spreadsheets. In this case, the objects on the user interface can be windows of files which can be changed with regard to their display size. These files can be active, ie immediately callable and executable state are displayed. It is therefore not necessary to start an application program after activating such an object.

The objects can be displayed on the user interface in a pseudo 3D view.

When exercising the enlargement / reduction function (zoom function) of the object area, no navigation control of the pointer mark is required.

According to a further aspect of the present invention, a computer software program is provided which implements a method of the type mentioned above when it runs on a computer.

Finally, the invention proposes the use of a force / moment sensor for a method according to one of the above-mentioned types.

Further features, advantages and properties of the present invention will now be explained on the basis of exemplary embodiments and with reference to the figures of the accompanying drawings

1 shows a system having a 3D input device and a computer with a desktop surface, and

Fig. 2 shows a modification of

1, wherein at the same time a screen object is shown in the enlarged state (zoomed state),

3 to 5 show a further embodiment in which a virtual window has been defined as the entire screen,

6 shows a schematic flow diagram of a sequence for carrying out the present invention, and

FIG. 7 shows the evaluation step S3 from FIG. 6 in detail.

As can be seen in FIG. 1, for example a PC 4 is used to implement the invention. This PC 4 has a monitor 6 on which a desktop 3, that is to say a section of the user interface, is displayed. Several graphic objects 5, 10 are arranged on this displayed section of the user interface.

A 3D input device 1 has an operating part 7 which can be manipulated by the fingers or the hand of a user and which is movably mounted, for example, in three mutually independent rotational and three translational degrees of freedom with respect to a base part 8. A relative movement between the operating part 7 and the base part 8 is evaluated and the result of the evaluation is transmitted to the computer 4 in the form of control signals.

It should be noted that the input device 1 can of course still output control signals with respect to further degrees of freedom by physically assigning it further rotary dials, buttons or switches on the operating part 7 or on the base plate 8.

One aspect of the present invention is that the input device 1 can be used to navigate a virtual window with an adjustable size with respect to the total area of the user interface. The display scale of objects within of the virtual window, can be optionally selected in a particularly advantageous embodiment within certain limits by means of the input device 1.

More specifically, control signals are used in two degrees of freedom of the input device 1 for navigating the virtual window with respect to the user interface 3 (up / down or left / right). Finally, a control signal in a third degree of freedom of the input device 1 is provided - if this option is provided - for real-time setting of an enlargement / reduction factor for the objects lying within the virtual window.

This enlargement / reduction factor can be changed continuously with the appropriate pixel scaling or discretely, for example in the case of defined font size levels.

For example, increasing the enlargement / reduction factor within the virtual window can be as

Answer to a push (translation) or a tilt

(Rotation) of the control unit 7 of the input device 1 to the front. An intuitive hand / eye coupling thus takes place, since this forward movement corresponds to an approximation of the virtual window to the user interface 3, the

Approximation according to the screen objects are shown larger and the section of the user interface 3 shown on the screen is reduced.

Such a virtual window is designated by the reference symbol 2 in FIG. As can be seen, the size of this window 2 is set such that it occupies only part of the display area of the screen 6. Accordingly, it can be navigated selectively, for example, as shown, via the object 10, so that the object 10 lies within the window area. If now by means of the input device 1, the enlargement / reduction factor of the virtual window 2 is increased, which can be done in steps or continuously, results in the enlarged representation 10 ′ of the object 10, which is shown schematically in FIG.

FIGS. 3 to 5, on the other hand, show the case in which the virtual window 2 is set such that it corresponds to the entire display area of the screen 6. When the virtual window 2 is navigated, the user interface 3 is thus moved with respect to the desktop.

In the preferred embodiment, in which an enlargement / reduction factor can be selected for the virtual window, the display size of all objects represented on the display area changes when the enlargement / reduction factor changes. If the user has arranged a group 11 on the user interface 3, he can enlarge the display of it continuously (pixel scaling) or step by step until, for example (see FIG. 5), only the document 12 from this group 11 is legibly displayed. This corresponds to zooming in on the user interface 3.

In contrast to FIG. 1, a computer mouse 1 'is symbolically provided in FIG. 2 as an input device. This computer mouse 1 ', which can actually only provide control signals in two degrees of freedom (x-y axis), is physically assigned a further element 9 which can generate a control signal in at least one further degree of freedom. In the case shown, this further element is a rotary wheel 9, which is arranged on the top of the computer mouse 1 '. By rotating this wheel 9 to the front, the display area of a screen object 10, 10 'can also be enlarged (selective focus) or all screen objects 5, 10 can be shown enlarged (general focus).

The reduction function can accordingly by rotating the wheel 9 in the reverse direction (in the three-dimensional Input device by pressing or tilting the control panel 7 backwards), which intuitively corresponds to leaning back of the user in order to get a better overview of the objects 5, 10 on the user interface 3.

In the event that objects 5, 10 on the user interface 3 display files from application programs, such as word processing or spreadsheets, these file objects can be actively displayed. This means that when the corresponding object is enlarged / reduced, not only is an icon enlarged or reduced as a symbol for the corresponding application program, but rather the document / spreadsheet itself can be enlarged or reduced. Accordingly, several screen objects can be actively displayed on the user interface 3 at the same time, their respective display scale being freely selectable. The user can thus, for example, arrange documents of any size and at any position on the screen surface 3.

Figure 6 shows schematically the sequence in the implementation of the present invention. Output signals of the force / moment sensor are generated in a step S1. These are then fed (step S2) to the data input of an EDP system. This can be done for example by means of a so-called USB interface. USB (Universal Serial Bus) is a connection (port) for peripheral devices (such as a mouse, modem, printer, keyboard, scanner, etc.) to a computer. The transfer rate of USB version 1.1 is already 12 Mbit / s.

In a step S3, the signals input by the force / moment sensor are evaluated. This step S3 is explained in detail below with reference to FIG. 7. Depending on the evaluation in step S3, the control of the graphical user interface (GUI) is then carried out in a step S4 before the data are evaluated again by the force / torque sensor.

With reference to FIG. 7, step S3 of the sequence of FIG. 6 will now be explained in more detail. As can be seen in FIG. 7, data in three different degrees of freedom x, y and z are evaluated, for example, to determine whether the corresponding signal is in the positive or negative range. With regard to the degree of freedom “z”, a positive signal can be used to enlarge and a negative signal to reduce the virtual window with respect to the entirety of the graphical user interface.

With regard to the degree of freedom “y”, a positive signal can shift the virtual window to the left and a negative signal can shift the virtual window to the right (always with respect to the entirety of the graphical user interface).

This is of course equivalent to the inverse shift of the user interface "under" the virtual window. For example, the virtual window can therefore be designed as a fixed marking bar "under" the user interface is navigated. Objects that come under the virtual window are automatically marked "highlight") and preselected for a possible subsequent click or other activation. This procedure is particularly advantageous if a directory structure (directory tree) is navigated under the fixed window, under directories located in the window can be selected automatically, so that in principle, you can navigate in infinitely large structures without the user's hand having to leave the input device. A "grasp" to change the image section as soon as the cursor in the case of known techniques on the edge of the screen is no longer required.

Finally, with regard to the degree of freedom "x", a positive signal can move the window upwards and a negative signal can move the window downwards. This can also be seen analogously as an inverse movement of the user interface "under" the virtual window.

In the following, the advantages of the invention compared to the prior art will be briefly mentioned. Today's desktop programs, on the other hand, only offer a work surface that is defined by the screen size and window size of the corresponding application. Accordingly, the only degree of freedom of today's desktop programs is to be able to arrange so-called icons as links to documents, execute programs and other content freely on the desktop.

In the present invention, however, the display size or the document size can be freely selected on the user interface. With the help of a single device, such as a 3D input device or a 2D input device with additional elements, the arrangement and the size of the screen objects on the desktop surface can be freely selected. Correspondingly, the recognition value of freely arranged areas is significantly larger, since optical recognition features and not just purely memory features apply here. Thus, according to the present invention, a real intuitive working behavior is largely reproduced. The real working behavior is usually the fact that the user works in the workplace with the inclusion of the optically perceptible sector. The focus on a working document that leans back to gain an overview are a natural part of processing real objects. But the present invention does it now for the first time possible to transfer such intuitive behavior to virtual objects, namely objects that are displayed on a user interface.

With a desktop manager program, it is thus possible to expand the graphical user interface 3 of conventional monitors and PCs by freely positioning the displayed section of the user interface 3 by means of a 3D input device 1, 1 'in such a way that the user can use it to view the visible part ( "Virtual window") of the user interface 3 of a monitor 6 and a PC 4 can determine itself.

Claims

Expectations .

1. A method for managing a graphical user interface (3), on which an input device (1, 1 ') can be used to navigate, the method comprising the following steps:

- arranging graphic objects (5) on the user interface (3),

- Navigating a virtual window (2) with respect to the user interface (3), the navigation being carried out by means of control signals from the input device (1, 1 '), and

- Display of the section of the user interface (3) located in the virtual window (2).

2. The method according to claim 1, characterized in that an enlargement / reduction factor for objects located within the virtual window (2) can be set by means of the input device (1, 1 ').

3. The method according to claim 1 or 2, characterized in that the navigation and possibly the setting of

Enlargement / reduction factor takes place essentially in real time.

4. The method according to any one of the preceding claims, characterized in that control signals are generated in at least three degrees of freedom by means of the input device (1, 1 '), wherein Control signals in two degrees of freedom for the navigation of the virtual window (2) with respect to the user interface

(3) and the control signal in the third degree of freedom may be used for setting the enlargement / reduction factor.

5. The method according to claim 4, characterized in that the input device (1) provides control signals in at least three translational and / or rotational degrees of freedom.

6. The method according to claim 5, characterized in that the input device is a force / torque sensor (1).

7. The method according to claim 1 or 2, characterized in that an input device (1) is used for two-dimensional navigation, such as. A computer mouse, which is physically assigned an element (9) for generating a control signal in a third degree of freedom.

8. The method according to any one of the preceding claims, characterized in that the size of the virtual window (2) is adjustable.

9. The method according to claim 7, characterized in that the virtual window (2) is defined as a part of the total display area of the screen.

10. The method according to any one of claims 1 to 9, characterized in that the virtual window (2) of the entire display area corresponds to a screen.

11. The method according to claim 9, characterized in that by means of the input device (1, 1 ') the virtual window can be navigated as a type of "magnifying glass" with an adjustable enlargement / reduction factor via the user interface (3).

12. The method according to any one of the preceding claims, characterized in that the software programs are office applications, such as word processing or spreadsheets, and the objects on the user interface (3) windows (5, 10, 10 ') with respect to their display size are changeable files.

13. The method according to claim 12, characterized in that the files are active, i.e. be displayed in an immediately executable state.

14. The method according to any one of the preceding claims, characterized in that the objects on the user interface (3) are displayed in a pseudo 3D view.

15. The method according to any one of the preceding claims, characterized in that the enlargement / reduction of an object is carried out in the manner of a zoom effect.

16. A method for managing a desktop, characterized in that the graphical user interface (3) of a monitor (6) is expanded by the free positioning of the user interface (3) by means of a 3D input device (1, 1 '). that the user can thus determine the visible partial section of a user interface (3) of the monitor (6) by actuating the 3D input device (1, 1 ').

17. Computer software program, characterized in that it implements a method according to one of the preceding claims when running on a processor-controlled device (4).

18. Use of a force / moment sensor for a method according to one of claims 1 to 16.