KR102539045B1 - Dashboard control apparatus and method for wearable augmented reality device - Google Patents

Abstract

The present invention relates to a dashboard control device for a wearable augmented reality device, and more particularly, to a dashboard control device for a wearable augmented reality device, wherein an image of a wearer's hand obtained from a camera of the wearable augmented reality device is provided. a hand recognition unit recognizing a side view of the wearer's first hand clenched into a fist from the physical user interface (PUI); a menu display unit displaying and providing a dashboard menu as an augmented reality image around the recognized PUI when the recognition of the PUI by the hand recognition unit is completed; and a control unit that performs an interaction according to selection of the dashboard menu by a wearer's second hand and recognition of a hand gesture of the second hand.
In addition, the present invention relates to a dashboard control method for a wearable augmented reality device, and more specifically, as a dashboard control method for a wearable augmented reality device, the side view of the first hand clenched into a fist of both hands of the wearer Each step is performed by the wearable augmented reality device including a hand recognition unit that recognizes a PUI and recognizes a hand gesture of a second hand for selecting a dashboard menu displayed as an augmented reality image around the PUI, (1) acquiring a hand image of the wearer from a camera of the wearable augmented reality device; (2) recognizing a side view of the wearer's first hand clenched into a fist from the hand image by the hand recognition unit as a PUI; (3) if the recognition of the PUI is completed in step (2), displaying and providing a dashboard menu as an augmented reality image around the PUI; and (4) selecting the dashboard menu by the wearer's second hand and performing an interaction according to hand motion recognition of the second hand.
According to the dashboard control device and dashboard control method for a wearable augmented reality device proposed in the present invention, a side view of a wearer's first hand clenched into a fist is recognized, and a dashboard menu is displayed around the fist of the first hand. By displaying and providing an augmented reality image, it is possible to comfortably use motions in terms of body structure even though it is a gesture of a shape that is not normally used, and it is possible to minimize recognition errors by calling up the dashboard menu without the influence of accessories or clothing.

Description

Dashboard control device and dashboard control method for wearable augmented reality device {DASHBOARD CONTROL APPARATUS AND METHOD FOR WEARABLE AUGMENTED REALITY DEVICE}

The present invention relates to a wearable augmented reality device, and more particularly, to a dashboard control device and dashboard control method for a wearable augmented reality device.

Various wearable devices are being developed according to the trend of light weight and miniaturization of digital devices. A head mounted display, which is a kind of wearable device, refers to various devices that can be worn on a user's head to receive multimedia contents and the like. Here, the head mounted display (HMD) is worn on the user's body and provides images to the user in various environments as the user moves. Head-mounted displays (HMDs) are classified into see-through and see-closed types. The see-through type is mainly used for Augmented Reality (AR), and the closed type is mainly used for virtual reality (VR). reality, VR).

Since it is difficult to use an input method such as a conventional touch screen in a head-mounted display, an input method optimized for user interaction is required. As an input method that can be used in the augmented reality of the head-mounted display, there may be buttons provided in the HMD, a separate input device connected to the HMD, gesture recognition, voice recognition, and the like.

However, since gesture recognition is usually based on finger recognition in the existing user interface (UI) of a head-mounted display for augmented reality, a special method for inputting a command to output a dashboard is required.

On the other hand, as a prior art related to the present invention, Patent Registration No. 10-1700569 (Title of Invention: HMD device capable of gesture-based user authentication and gesture-based user authentication method of the HMD device, registration date: January 2017 23), etc. have been disclosed.

The present invention has been proposed to solve the above problems of the previously proposed methods, recognizing the side view of the wearer's first hand clenched into a fist, and displaying a dashboard menu around the fist of the first hand as an augmented reality image. A wearable augmented reality device that can be used by displaying and providing gestures of shapes that are not normally used, but can be used by taking motion comfortably due to the body structure, and can minimize recognition errors by calling up the dashboard menu without the influence of accessories or clothing Its purpose is to provide a dashboard control device and dashboard control method for

Dashboard control device for a wearable augmented reality device according to the features of the present invention for achieving the above object,

As a dashboard control device for a wearable augmented reality device,

a hand recognition unit recognizing a side view of the wearer's first hand clenched into a fist from the image of the wearer's hand acquired from the camera of the wearable augmented reality device as a Physical User Interface (PUI);

a menu display unit displaying and providing a dashboard menu as an augmented reality image around the recognized PUI when the recognition of the PUI by the hand recognition unit is completed; and

It is characterized in that it includes a control unit that performs interaction according to selection of the dashboard menu by the wearer's second hand and recognition of the hand gesture of the second hand.

Preferably, the hand recognition unit,

An infrared hand image may be used from an infrared image camera of the wearable augmented reality device.

Preferably, the hand recognition unit,

A side view of the wearer's first hand clenched into a fist may be recognized as a PUI, and a hand gesture of the second hand for selecting a dashboard menu displayed as an augmented reality image around the PUI may be recognized.

Preferably, the hand recognition unit,

The PUI may be recognized by extracting a shape or feature point of the first hand of the wearer from the hand image.

Preferably, the menu display unit,

It may interwork with the hand recognition unit, and display and provide a loading image representing a recognition state of the PUI by the hand recognition unit as an augmented reality image on a side view of the wearer's first hand making a fist.

Preferably, the hand recognition unit,

When the first hand of the wearer maintains a side view with a clenched fist for a predetermined time, completion of recognizing the PUI may be transmitted to the menu display unit.

Preferably,

Further comprising a gaze tracking unit for tracking the wearer's gaze,

The hand recognition unit,

When the wearer's gaze tracked by the gaze tracking unit gazes at a side view of the wearer's first hand clenched into a fist for a predetermined time, completion of PUI recognition may be transmitted to the menu display unit.

In addition, the dashboard control method for the wearable augmented reality device according to the features of the present invention for achieving the above object,

A dashboard control method for a wearable augmented reality device,

The wearable augmentation including a hand recognition unit recognizing a side view of a first hand clenched into a fist among both hands of the wearer as a PUI and recognizing a hand gesture of a second hand for selecting a dashboard menu displayed as an augmented reality image around the PUI. Each step is performed by a reality device,

(1) acquiring a hand image of the wearer from the camera of the wearable augmented reality device;

(2) recognizing a side view of the wearer's first hand clenched into a fist from the hand image by the hand recognition unit as a PUI;

(3) if the recognition of the PUI is completed in step (2), displaying and providing a dashboard menu as an augmented reality image around the PUI; and

(4) Selecting the dashboard menu by the wearer's second hand and performing an interaction according to hand gesture recognition of the second hand is a feature of the configuration.

According to the dashboard control device and dashboard control method for a wearable augmented reality device proposed in the present invention, a side view of a wearer's first hand clenched into a fist is recognized, and a dashboard menu is displayed around the fist of the first hand. By displaying and providing an augmented reality image, it is possible to comfortably use motions in terms of body structure even though it is a gesture of a shape that is not normally used, and it is possible to minimize recognition errors by calling up the dashboard menu without the influence of accessories or clothing.

1 is a diagram showing the configuration of a dashboard control device for a wearable augmented reality device according to an embodiment of the present invention.
2 is a diagram showing, for example, a state in which a dashboard is provided in a dashboard control device for a wearable augmented reality device according to an embodiment of the present invention.
3 and 4 are diagrams showing, for example, a recognition method of a hand recognition unit in a dashboard control device for a wearable augmented reality device according to an embodiment of the present invention.
5 is a view showing, for example, a state in which a loading image is displayed in a dashboard control device for a wearable augmented reality device according to an embodiment of the present invention.
6 is a diagram showing, for example, a dashboard menu displayed in a dashboard control device for a wearable augmented reality device according to an embodiment of the present invention.
7 is a view showing, for example, how an interaction according to a second hand gesture is performed in a dashboard control device for a wearable augmented reality device according to an embodiment of the present invention.
8 is a view showing the overall configuration of a wearable augmented reality device including a dashboard control device for a wearable augmented reality device according to an embodiment of the present invention.
9 is a flowchart illustrating a dashboard control method for a wearable augmented reality device according to an embodiment of the present invention.

Hereinafter, preferred embodiments will be described in detail so that those skilled in the art can easily practice the present invention with reference to the accompanying drawings. However, in describing a preferred embodiment of the present invention in detail, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the same reference numerals are used throughout the drawings for parts having similar functions and actions.

In addition, throughout the specification, when a part is said to be 'connected' to another part, this is not only the case where it is 'directly connected', but also the case where it is 'indirectly connected' with another element in between. include In addition, 'including' a certain component means that other components may be further included, rather than excluding other components unless otherwise specified.

1 is a diagram showing the configuration of a dashboard control device 100 for a wearable augmented reality device 10 according to an embodiment of the present invention. As shown in FIG. 1, the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention includes a hand recognition unit 110, a menu display unit 120, and a control unit ( 130), and may further include a gaze tracking unit 140.

That is, the dashboard control device 100 may provide a dashboard interface of the wearable augmented reality device 10 based on augmented reality. More specifically, the wearable augmented reality device 10 includes a dashboard control device 100 including a hand recognition unit 110, a menu display unit 120, and a control unit 130, and controls the dashboard. The device 100 is one of the components of the wearable augmented reality device 10, and provides a dashboard menu using hand images collected by the camera unit 600, provided by the dashboard control device 100 Through the dashboard, the controller 400 may receive various input signals and perform functions of the wearable augmented reality device 10 .

Here, the dashboard is a user interface (UI) function that allows centralized management and finding of various information on one screen. , media files, etc. can be managed on one screen. For example, settings, default launcher, home screen, etc. can be dashboards, and users can interact with them by selecting desired functions on the dashboard.

On the other hand, the wearable augmented reality device 10 is worn on the head or head of the user, and the real world and the display in which the user wearing the wearable augmented reality device 10 is transmitted through the optical system are output to the wearer through the optical system. It may be a device that allows users to experience augmented reality by receiving image information delivered to the pupil of the child. Here, the wearable augmented reality device 10 may be of various types such as glass type and goggle type, and if it is worn on the user's body to experience augmented reality through the wearer's field of view, the present invention regardless of its specific form or type It can serve as a wearable augmented reality device 10 of.

FIG. 2 is a view showing, for example, a state in which a dashboard is provided in the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention. As shown in FIG. 2 , the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention displays a side view of the wearer's first hand 1 clenched into a fist through a PUI , it is possible to provide an interface by displaying the dashboard menu as an augmented reality image around the fist of the first hand (1). The wearer can select one of the displayed dashboard menus using the first hand (1) used as a PUI for displaying the dashboard menu and the other hand, that is, the second hand (2) among both hands (in FIG. 2, the camera function selection), the controller 130 may perform an interaction according to the hand motion of the second hand 2, such as executing the selected function.

In the wearable augmented reality device 10 such as AR glasses, since finger recognition is mostly used as a physical user interface (PUI) for operation control, it is difficult to utilize finger gestures to call a dashboard. Therefore, it can be used as a PUI for dashboard control by specifying the hand shape and hand shape of the range that is not generally used. For example, Microsoft's HoloLens calls up the settings screen through recognition of the wrist. However, errors may occur in the recognition process of the wrist part depending on the wearing of accessories such as watches and bracelets, or worn clothes such as clothes designed to cover the wrist or long sleeved clothes.

As shown in FIG. 2 , in the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention, in a state where the first hand 1 is clenched into a fist, the thumb and index finger side By using the side view of the PUI as a PUI, it is a hand motion that is different from finger recognition generally used for operation control, but corresponds to a comfortable motion in terms of the body structure, so both the convenience of motion and recognition accuracy can be increased. Unlike , recognition errors can be minimized because hand gestures that are not influenced by accessories or clothing are used.

Meanwhile, FIG. 2 shows that the left hand serves as the first hand 1 and the right hand serves as the second hand 2, but the first hand 1 is a hand used as a PUI for displaying the dashboard menu and The second hand (2) is a hand that performs input by selecting the displayed menu icon, etc. The first hand (1) and the second hand (2) can be changed depending on the situation, and the dashboard menu is displayed around the fist of the right hand. (First hand (1)) It is also possible to select a menu with the left hand (second hand (2)).

Hereinafter, each component of the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 and 2 .

The hand recognizer 110 may recognize a side view of the wearer's first hand 1 clenched into a fist from the wearer's hand image obtained from the camera of the wearable augmented reality device 10 as a Physical User Interface (PUI). there is. More specifically, the hand recognition unit 110 recognizes a side view of the first hand 1 of the wearer's hands making a fist as a PUI, and the second hand selecting a dashboard menu displayed as an augmented reality image around the PUI The hand gesture of (2) can be recognized.

As shown in FIG. 1 , in the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention, the camera unit 600 of the wearable augmented reality device 10 The captured hand image is received, and hand shapes and hand gestures of the first hand 1 and the second hand 2 of the wearer may be recognized from the hand image. That is, the hand recognizer 110 may estimate the shape or feature point of the hand using vision recognition technology based on image processing and artificial intelligence, and recognize the shape and motion of the hand using the estimated hand shape or feature point. can

3 and 4 are diagrams showing, for example, a recognition method of the hand recognizer 110 in the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention. . 3 and 4, the hand recognition unit 110 of the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention, The PUI can be recognized by extracting the shape or feature point of the first hand 1 . That is, as shown in FIG. 3, the bent shapes of the thumb and index finger of the first hand 1 are recognized, or as shown in FIG. 4, feature points are extracted from the hand image and determined based on the positional relationship between feature points. 1 The side view of the hand (1) clenched into a fist can be recognized.

Depending on the embodiment, the hand recognizer 110 may use a hand image obtained from an RGB camera, an infrared image camera, a depth sensor, or the like of the wearable augmented reality device 10 . More specifically, the hand recognizer 110 may use an infrared hand image obtained from an infrared image camera of the wearable augmented reality device 10 .

The infrared image is a single-channel image, and hand recognition using the infrared hand image can reduce computational costs used in image processing, so hand recognition can be effectively performed even in the embedded environment of the wearable augmented reality device 10 . In particular, when the weight of the wearable augmented reality device 10 worn on the head is heavy, it is difficult to wear it for a long time because the neck and shoulders are burdened and the head movement is restricted, so it is important to reduce the weight and size. Therefore, since many calculation resources cannot be loaded into the wearable augmented reality device 10, calculation cost can be reduced so that it can be used in an embedded environment using an infrared image, which is a single-channel image.

When PUI recognition is completed in the hand recognition unit 110, the menu display unit 120 may display and provide the dashboard menu as an augmented reality image around the recognized PUI. That is, as shown in FIG. 2 , the menu display unit 120 may provide a user interface by displaying a dashboard menu in augmented reality around the recognized PUI.

Meanwhile, the menu display unit 120 interlocks with the hand recognition unit 110, and displays a loading image representing the PUI recognition state of the hand recognition unit 110 on the side of the wearer's first hand 1 clenched into a fist. It can be displayed and provided as an augmented reality image.

FIG. 5 is a diagram showing, for example, a state in which a loading image is displayed in the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention. As shown in FIG. 5 , the menu display unit 120 of the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention provides a loading image indicating completion of recognition. 1 It can indicate that the PUI has been recognized by displaying it on the hand (1). At this time, the loading image may be displayed centering on the tip of the index finger in the side view of the clenched fist, or may be displayed at the center of the fist.

Here, the loading image may indicate a recognition state such as recognition in progress or recognition completion, and may be an image that changes according to the recognition state. In addition, the loading image may be an animated image, and its color, brightness, etc. may be changed. It may be animated or its color, brightness, etc. may be changed according to a recognition state.

FIG. 6 is a view showing, for example, a dashboard menu displayed in the dashboard control device 100 for the wearable augmented reality device 10 according to one embodiment of the present invention. As shown in FIG. 6 , the menu display unit 120 of the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention is When the recognition of the side view is completed and the creation of the dashboard menu to be displayed is completed, the loading image may be removed and the dashboard menu may be displayed in augmented reality.

The dashboard menu may include a plurality of items, and each item may be displayed as an icon or the like. In this case, the dashboard menu may be displayed at a predetermined distance from the center of the fist. 6, the dashboard menu is displayed in the form of icons arranged on a semicircular arc centered on a fist, but according to the embodiment, icons are arranged in a circle, or icons are arranged on several arcs or circles with the same or different centers. You can also provide a dashboard menu.

The hand recognition unit 110 may transmit PUI recognition completion to the menu display unit 120 when the wearer's first hand 1 maintains a cleaved fist for a predetermined time. The menu display unit 120 may receive the recognition completion signal and control the dashboard menu to be displayed as an augmented reality image from the shape shown in FIG. 5 to the shape shown in FIG. 6 . Here, the predetermined time may be 1.5 seconds or more, and the predetermined time may be set by the wearer or the user.

The gaze tracking unit 140 may track the wearer's gaze. The gaze tracker 140 may detect a pupil position from the pupil image of the wearer in the wearable augmented reality device 10 and track the gaze based on the detected pupil position. The hand recognition unit 110, when the wearer's gaze tracked by the gaze tracking unit 140 gazes at the side of the wearer's first hand 1 clenched into a fist for a predetermined time, completes PUI recognition through the menu display unit (120). Here, the predetermined time may be 1.5 seconds or more, but the predetermined time may be set by the wearer or the user.

That is, the wearer maintains the posture of the fisted side view of the first hand 1 within the field of view for a predetermined time, or the wearer maintains the fisted side view of the first hand 1 in advance. If you gaze for a predetermined time, the hand recognition unit 110 recognizes the shape of the first hand 1 as a PUI, and when the recognition is completed, the menu display unit 120 can output the dashboard menu as an augmented reality image. .

The controller 130 may perform interaction according to selection of a dashboard menu by the wearer's second hand 2 and recognition of a hand gesture of the second hand 2 . Here, the hand gesture of the second hand 2 may be an action of selecting a dashboard menu displayed with a finger.

FIG. 7 illustrates, for example, a state in which an interaction according to a motion of a second hand 2 is performed in the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention. it is a drawing As shown in FIG. 7 , the control unit 130 of the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention uses the fingers of the second hand 2 to camera A function may be selected, and the selection information may be transmitted to the control unit 400 of the wearable augmented reality device 10 and the camera function may be activated.

Meanwhile, the controller 130 may perform an interaction for controlling the dashboard menu through a hand motion of the second hand 2 . For example, an interaction in which the dashboard menu is scrolled may be performed through a hand motion in which the second hand 2 scrolls the dashboard menu displayed in a semicircular shape clockwise or counterclockwise. In addition, the icon may be moved through a hand motion of the second hand 2 of dragging and dropping a specific icon.

8 is a diagram showing the overall configuration of the wearable augmented reality device 10 including the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention. As shown in FIG. 8 , the wearable augmented reality device 10 including the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention is a dashboard control device. 100 and a control unit 400, and may include an HMD frame 200, an optical display unit 300, a GPS module 500, a camera unit 600, a sensor unit 700, and a communication unit 800. ) may be further included.

The HMD frame 200 is a frame configuration of the wearable augmented reality device 10 that can be worn on the head or head by the user. The HMD frame 200 may be configured in the form of a helmet or goggles having a frame structure through which light may enter while worn on the head by a user. Here, when the HMD frame 200 is formed in the form of a helmet, it may be composed of a helmet (not shown) worn on the user's head and a display frame (not shown) disposed in front of the helmet. In addition, when the HMD frame 200 is configured in the form of goggles, it may be configured with a band frame (not shown) that can be worn on the user's head and a goggle frame (not shown) fastened and fixed to the band frame.

The optical display unit 300 may be disposed in front of both eyes of the wearer to provide augmented reality or mixed reality to the wearer. That is, the optical display unit 300 is a component corresponding to the AR, VR or MR glasses of the wearable augmented reality device 10, and displays and wears an OLED that outputs image light so that image information can be provided to the wearer. It may be configured to include an optical system that is disposed in front of both eyes of the wearer wearing the type augmented reality device 10 and provides augmented reality by combining real world light and image light. In addition, the optical system is composed of a plurality of lenses and mirrors and may be implemented in various ways, such as an optical diffraction method, a beam splitter method, a pin mirror method, and the like.

The GPS module 500 may provide location information of the wearer, and the camera unit 600 may capture an image in the direction of the wearer's gaze. In particular, the camera unit 600 may capture a hand image and transmit it to the hand recognizing unit 110 to recognize a hand motion and hand knuckles based on vision recognition.

The control unit 400 generates image information to be provided to the wearer based on information collected by the GPS module 500, the camera unit 600, and other various sensors, and controls the image information to be transmitted to the display, so that the wearer can enjoy augmented reality. Through this, it is possible to receive additional information about the external environment through an optimized screen.

The sensor unit 700 may include at least one or more sensors, and more specifically, an iris recognition sensor or the like. According to the embodiment, when the wearer wears the wearable augmented reality device 10 and authenticates the user, the control unit 400 sets various user-specific settings such as illumination, IPD (Inter-Pupillary Distance), and interface stored according to each wearer. Content can be applied automatically. In this case, user authentication may use an iris recognition sensor included in the sensor unit 700 .

The communication unit 800 is installed on one side of the wearable augmented reality device 10 and can transmit and receive various signals and data to and from other wearable augmented reality devices 10 or servers. Here, the network used by the communication unit 800 is a wired network or a mobile communication network such as a Local Area Network (LAN), a Wide Area Network (WAN), or a Value Added Network (VAN). radio communication network), satellite communication network, Bluetooth, Wibro (Wireless Broadband Internet), HSDPA (High Speed Downlink Packet Access), LTE (Long Term Evolution), 3/4/5G (3/4/5th Generation Mobile Telecommunication ) can be implemented in all types of wireless networks, such as

9 is a flow diagram illustrating a dashboard control method for the wearable augmented reality device 10 according to an embodiment of the present invention. As shown in FIG. 9 , in the dashboard control method for the wearable augmented reality device 10 according to an embodiment of the present invention, the side view of the first hand 1 of the wearer's both hands clenched into a fist is converted into a PUI. Each step is performed by the wearable augmented reality device 10 including the hand recognizer 110 that recognizes the hand motion of the second hand 2 for selecting the dashboard menu displayed as an augmented reality image around the PUI. Acquiring an image of the wearer's hand (S110), recognizing the side view of the wearer's first hand (1) clenched into a fist as a PUI (S120), displaying the dashboard menu as an augmented reality image around the PUI and providing (S130), selecting a dashboard menu by the second hand (2) of the wearer and performing an interaction according to hand gesture recognition of the second hand (2) (S140). .

Since the details related to each step have been sufficiently described in relation to the dashboard control device 100 for the wearable augmented reality device 10 according to an embodiment of the present invention, detailed descriptions will be omitted. .

As described above, according to the dashboard control device 100 and the dashboard control method for the wearable augmented reality device 10 proposed in the present invention, the side view of the wearer's first hand 1 clenched into a fist By recognizing and providing the dashboard menu displayed as an augmented reality image around the fist of the first hand (1), it is possible to use it comfortably by taking a motion due to the body structure even though it is a gesture of a shape that is not generally used, and the influence of accessories or clothing Since the dashboard menu can be called up without the need, recognition errors can be minimized.

Meanwhile, the present invention may include a computer-readable medium including program instructions for performing operations implemented in various communication terminals. For example, computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD_ROMs and DVDs, and floptical disks. It may include hardware devices specially configured to store and execute program instructions, such as magneto-optical media and ROM, RAM, flash memory, and the like.

Such computer-readable media may include program instructions, data files, data structures, etc. alone or in combination. At this time, program instructions recorded on a computer-readable medium may be specially designed and configured to implement the present invention, or may be known and usable to those skilled in computer software. For example, it may include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes generated by a compiler.

The present invention described above can be variously modified or applied by those skilled in the art to which the present invention belongs, and the scope of the technical idea according to the present invention should be defined by the claims below.

1: first hand
2: second hand
10: wearable augmented reality device
100: dashboard control device
110: hand recognition unit
120: menu display unit
130: control unit
140: gaze tracking unit
200: HMD frame
300: optical display unit
400: control unit
500: GPS module
600: camera unit
700: sensor unit
800: communication department
S110: Acquiring an image of the wearer's hand
S120: Recognizing a side view of the wearer's first hand clenched into a fist as PUI
S130: Step of displaying and providing the dashboard menu as an augmented reality image around the PUI
S140: Performing an interaction according to the selection of the dashboard menu by the wearer's second hand and recognition of the gesture of the second hand

Claims (8)

As a dashboard control device 100 for a wearable augmented reality device 10,
a hand recognizer 110 recognizing a side view of the wearer's first hand 1 clenched into a fist from the wearer's hand image acquired from the camera of the wearable augmented reality device 10 as a Physical User Interface (PUI);
When the hand recognition unit 110 completes recognizing the PUI, a menu display unit 120 that displays and provides a dashboard menu as an augmented reality image around the recognized PUI; and
It includes a controller 130 that performs an interaction according to the selection of the dashboard menu by the second hand 2 of the wearer and the recognition of the hand gesture of the second hand 2,
The dashboard menu,
It consists of a plurality of items displayed in a semicircular arc or circle at a predetermined distance around the central position of the fist of the first hand 1, which is the recognized PUI,
The controller 130,
The wearable augmented reality device (10), characterized in that the second hand (2) performs an interaction in which the dashboard menu is scrolled through a hand motion of scrolling the dashboard menu clockwise or counterclockwise. Dashboard control device (100) for.
The method of claim 1, wherein the hand recognition unit 110,
The dashboard control device 100 for the wearable augmented reality device 10, characterized in that using an infrared hand image from the infrared image camera of the wearable augmented reality device 10.
The method of claim 1, wherein the hand recognition unit 110,
Recognizes a side view of the first hand (1) of the wearer's hands (1) clenched into a fist as a PUI, and recognizes a hand gesture of the second hand (2) for selecting a dashboard menu displayed as an augmented reality image around the PUI. To, the dashboard control device 100 for the wearable augmented reality device 10.
The method of claim 1, wherein the hand recognition unit 110,
The dashboard control device 100 for the wearable augmented reality device 10, characterized in that the PUI is recognized by extracting the shape or feature point of the first hand 1 of the wearer from the hand image.
The method of claim 1, wherein the menu display unit 120,
It interworks with the hand recognition unit 110, and displays a loading image representing the PUI recognition state of the hand recognition unit 110 as an augmented reality image on the side of the wearer's first hand 1 clenched into a fist. Characterized in that it provides, the dashboard control device 100 for the wearable augmented reality device 10.
The method of claim 1, wherein the hand recognition unit 110,
When the wearer's first hand (1) maintains a fist-clenched side view for a predetermined time, the PUI recognition completion is transmitted to the menu display unit (120), wearable augmented reality device (10) Dashboard control device (100) for.
According to claim 1,
It further includes a gaze tracking unit 140 that tracks the wearer's gaze,
The hand recognition unit 110,
When the wearer's gaze tracked by the gaze tracking unit 140 gazes at the side of the wearer's first hand 1 making a fist for a predetermined time, completion of PUI recognition is displayed on the menu display unit 120. Dashboard control device 100 for the wearable augmented reality device 10, characterized in that it transmits.
As a dashboard control method for the wearable augmented reality device 10,
A hand recognition unit that recognizes a side view of the first hand (1) of the wearer's hands (1) clenched into a fist as a PUI, and recognizes a hand gesture of the second hand (2) selecting a dashboard menu displayed as an augmented reality image around the PUI Each step is performed by the wearable augmented reality device 10 including 110,
(1) acquiring a hand image of the wearer from the camera of the wearable augmented reality device 10;
(2) recognizing, in the hand recognition unit 110, a side view of the wearer's first hand 1 clenched into a fist from the hand image as a PUI;
(3) if the recognition of the PUI is completed in step (2), displaying and providing a dashboard menu as an augmented reality image around the PUI; and
(4) selecting the dashboard menu by the wearer's second hand 2 and performing an interaction according to hand gesture recognition of the second hand 2;
The dashboard menu,
It consists of a plurality of items displayed in a semicircular arc or circle at a predetermined distance around the central position of the fist of the first hand 1, which is the recognized PUI,
In the step (4),
The wearable augmented reality device (10), characterized in that the second hand (2) performs an interaction in which the dashboard menu is scrolled through a hand motion of scrolling the dashboard menu clockwise or counterclockwise. Dashboard control method for .

Images