JP2022102681A - Eyeglass-type devices, programs, and control methods - Google Patents

Abstract

The present invention provides an eyeglass-type device, a program, and a control method for controlling display depending on whether the eyeglass-type device is worn or not. [Solution] In a glasses-type device. The control unit 200 includes a wearing state determination unit that determines whether or not the glasses-type device is being worn by a user, and a wear-state determination unit that determines whether the glasses-type device is being worn by the user, and when the glasses-type device is being worn, the user who is wearing the glasses-type device a display control unit that displays information toward the outside when the glasses-type device is not worn, a communication unit that communicates with another glasses-type device, and a wearing state determination unit. When it is determined that the glasses-type device is not worn, the communication unit sends notification information to the other glasses-type device of the communication partner to notify that the user is not wearing the glasses-type device. and a notification control unit that controls the communication unit to transmit data to the type device. [Selection diagram] Figure 7

Description

The present invention relates to eyeglass-type devices, programs, and control methods.

Patent Document 1 includes an AR (Augmented) having a display unit capable of superimposing and displaying an AR virtual object on a landscape in a real space by displaying an image such as a text or a figure in a transparent or semi-transparent state. Reality) Glass is described.
[Prior Art Document]
[Patent Document]
[Patent Document 1] Japanese Unexamined Patent Publication No. 2019-114078

According to one embodiment of the present invention, a spectacle-type device is provided. The spectacle-type device should include a wearing state determination unit for determining whether or not the spectacle-type device is in the wearing state worn by the user. The spectacle-type device displays information to the user wearing the spectacle-type device when the spectacle-type device is worn, and displays information to the outside when the spectacle-type device is not in the wearing state. It may be provided with a display control unit for controlling the eyeglasses.

The display control unit displays information inward of the lens unit of the spectacle-type device when the spectacle-type device is in the mounted state, and outwards of the lens unit when the spectacle-type device is not in the mounted state. It may be controlled to display information. The display control unit controls so that when the spectacle-type device is not in the wearing state, the information is displayed outward in the lens unit in a larger size than when the spectacle-type device is in the wearing state. good. When the spectacle-type device receives new information, the display control unit displays notification information for notifying that the spectacle-type device has been received outward from the lens unit if the spectacle-type device is not in the wearing state. It may be controlled to do so. The display control unit displays the notification information outward of the lens unit, and then displays the new arrival information outward of the lens unit in response to detecting a predetermined operation by the user. It may be controlled as follows.

The display control unit displays information inward in the lens portion of the spectacle-type device when the spectacle-type device is in the wearing state, and when the spectacle-type device is not in the wearing state, the spectacle-type device is in the wearing state. The information may be controlled to be displayed inward in the lens portion of the spectacle-type device with a size larger than that in the case of. The display control unit displays information on the first layer of the lens unit of the spectacle-type device when the spectacle-type device is in the wearing state, and when the spectacle-type device is not in the wearing state, the display control unit displays information on the first layer of the lens unit of the spectacle-type device. Information may be controlled to be displayed on one layer and a second layer outside the first layer. The display control unit may control the display control unit to display information on the first layer inside the opaque layer and the second layer outside the opaque layer when the spectacle-type device is not in the wearing state.

The spectacle-type device is a user who determines whether the user is located on the front side of the spectacle-type device or on the wearing side of the spectacle-type device when the spectacle-type device is not in the wearing state. A determination unit may be provided, and the display control unit is outside the lens unit of the spectacle-type device when the user is located on the front side of the spectacle-type device when the spectacle-type device is not in the wearing state. Information may be displayed in the orientation, and when the user is located on the wearing side of the spectacle-type device, the information may be controlled to be displayed inward in the lens portion of the spectacle-type device. The spectacle-type device may include an irradiation unit that projects information toward the outside, and the display control unit causes the image irradiation unit of the spectacle-type device to project information when the spectacle-type device is not in the wearing state. You may control it. The irradiation unit may have a first irradiation unit that projects information in the front direction of the spectacle-type device and a second irradiation unit that projects information in the wearing direction of the spectacle-type device. May selectively project the above information onto the first irradiation unit and the second irradiation unit when the eyeglass-type device is not in the wearing state. The spectacle-type device determines whether the user is located on the front side of the spectacle-type device or on the wearing direction side of the spectacle-type device when the spectacle-type device is not in the wearing state. A user determination unit may be provided, and the display control unit transfers information to the first irradiation unit when the user is located on the front side of the spectacle-type device when the spectacle-type device is not in the wearing state. When the user is positioned on the wearing direction side of the spectacle-type device, the information may be projected onto the second irradiation unit. The spectacle-type device is a communication unit that communicates with another spectacle-type device, and when the wearing state determination unit determines that the spectacle-type device is not in the wearing state, the other communication partner of the communication unit is the other. The spectacle-type device is provided with a notification control unit that controls the communication unit to transmit notification information notifying that the user is not wearing the spectacle-type device to the other spectacle-type device. good.

According to one embodiment of the present invention, a spectacle-type device is provided. The spectacle-type device may include a communication unit that communicates with other spectacle-type devices. The spectacle-type device may include a wearing state determination unit that determines whether or not the spectacle-type device is in the wearing state worn by the user. When the spectacle-type device is determined by the wearing state determination unit that the spectacle-type device is not in the wearing state, the user does not wear the spectacle-type device on another spectacle-type device of the communication partner of the communication unit. A notification control unit that controls the communication unit to transmit the notification information to be notified to other eyeglass-type devices may be provided.

According to one embodiment of the present invention, a program for making a spectacle-type device function as the spectacle-type device is provided.

According to one embodiment of the invention, a control method performed by a spectacle-type device is provided.

According to one embodiment of the invention, a control method performed by a spectacle-type device is provided.

The outline of the above invention does not list all the necessary features of the present invention. A subcombination of these feature groups can also be an invention.

An example of the system 10 is shown schematically. An example of the configuration of the spectacle-type device 100 is schematically shown. A display example of the notification screen 400 by the spectacle-type device 100 is schematically shown. A display example of the notification screen 400 by the spectacle-type device 100 is schematically shown. A display example of the notification screen 400 by the spectacle-type device 100 is schematically shown. An example of the processing flow by the spectacle-type device 100 is schematically shown. An example of the functional configuration of the control device 200 is schematically shown. An example of the hardware configuration of the computer 1200 that functions as the control device 200 is schematically shown.

As the spread of spectacle-type devices such as AR (Augmented Reality) glasses progresses, it is expected that the number of forms in which various communications are executed only by the spectacle-type devices without having a smartphone will increase. If you wear a glasses-type device, you can immediately notice new information in e-mail, SNS, etc. and check it immediately. However, there is a possibility that new information may be overlooked in a non-wearing state such as when the eyeglass-type device is temporarily removed. In addition, communication loss may occur frequently by overlooking the notification from the communication partner. The spectacle-type device 100 according to the present embodiment has a function of easily confirming information without wearing the spectacle-type device 100 even in a non-wearing state, and a function of notifying a communication partner of the wearing state.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention to which the claims are made. Also, not all combinations of features described in the embodiments are essential to the means of solving the invention.

FIG. 1 schematically shows an example of the system 10. The system 10 includes a plurality of eyeglass-type devices 100 and a management server 300.

The spectacle-type device 100 may be any spectacle-type device as long as it has a wireless communication function and a display function for displaying various information on the display unit. The spectacle-type device 100 may be, for example, an AR glass.

Examples of the information displayed by the spectacle-type device 100 include information such as time and date and time, information such as e-mail and SNS, information on applications, and contents for AR. Not limited to these, the spectacle-type device 100 may be capable of displaying arbitrary information.

The eyeglass-type device 100 may receive the content for AR from the management server 300 and display it. The content managed by the management server 300 may be in any data format. For example, the content is 3D CG (Computer Graphics) data. For example, the content is a still image. Also, for example, the content is a moving image. For example, the content is text. The content may include these in combination.

The spectacle-type device 100 may be a device capable of giving the user 102 the feeling that the content is arranged in the real space by displaying the content on a transparent or translucent glass. That is, the spectacle-type device 100 may be a so-called transmissive type (see-through type) device. The glasses-type device 100 may be a video pass-through type device. In this case, the spectacle-type device 100 may display a captured image capturing a range corresponding to the field of view of the user 102 on a non-transparent display, and superimpose and display the content.

The management server 300 manages the content to be delivered to the eyeglass-type device 100. The management server 300 registers the content in association with the object in the real world, for example. Objects in the real world may be real-world positions, areas, objects, and the like. The content may be registered by any person authorized by the administrator of the management server 300 or the like. The management server 300 may accept registration of contents by a smartphone, a tablet terminal, a PC (Personal Computer), an AR glass, or the like.

The management server 300 may receive, for example, the image pickup range information indicating the image pickup range of the eyeglass type device 100 from the eyeglass type device 100, and transmit the content corresponding to the object included in the image pickup range to the eyeglass type device 100. .. The imaging range of the spectacle-type device 100 may be the imaging range of the camera of the spectacle-type device 100. The imaging range of the spectacle-type device 100 by the camera may be a range corresponding to the field of view of the user 102. The imaging range of the camera of the spectacle-type device 100 may be wider than the field of view of the user 102, and may be substantially the same as the field of view of the user 102.

The eyeglass-type device 100 and the management server 300 may communicate with each other via the network 20. The network 20 includes a cloud. The network 20 may include the Internet. The network 20 may include a mobile communication network. The mobile communication network may comply with any of the LTE (Long Term Evolution) communication method, the 3G (3rd Generation) communication method, the 5G (5th Generation) communication method, and the 6G (6th Generation) communication method or later. good. The network 20 may be compatible with the Wi-Fi (registered trademark) communication method. The network 20 may be accessible via a Wi-Fi access point. The network 20 may be accessible by using other wireless communication in general. The management server 300 may be connected to the network 20 by wire or wirelessly. The spectacle-type device 100 may be connected to the network 20 by mobile communication, Wi-Fi communication, and other wireless communication in general.

FIG. 2 schematically shows an example of the configuration of the spectacle-type device 100. The spectacle-type device 100 includes a frame 110 having a rim 112 and a temple 118. The spectacle-type device 100 includes a control device 200 that executes various controls in the spectacle-type device 100.

The arrangement of the control device 200 in FIG. 2 is an example, and the control device 200 may be arranged at another position of the frame 110. For example, the control device 200 may be arranged outside the frame 110. Further, for example, the control device 200 may be arranged in the temple 118. As a specific example, the control device 200 may be arranged at the tip of the temple 118, that is, in a modern manner. The control device 200 may include a battery. The control device 200 and the battery may be separately arranged at different positions on the frame 110.

The spectacle-type device 100 may include a camera 120. The imaging range of the camera 120 may be a range corresponding to the field of view of the user 102 wearing the spectacle-type device 100. The arrangement of the camera 120 in FIG. 2 is an example, and the camera 120 may be arranged at another position.

The spectacle-type device 100 may include a sensor 122. The sensor 122 may be a sensor for identifying the environment around the spectacle-type device 100. The sensor 122 may include LiDAR (Light Detection and Ringing). The sensor 122 may include a ranging sensor. Although FIG. 2 illustrates the case where the sensor 122 is arranged on the bridge 114, the arrangement of the sensor 122 in FIG. 2 is an example, and the sensor 122 may be arranged at another position.

The spectacle-type device 100 may include a sensor 124. The sensor 124 detects, for example, the line of sight of the user 102 who is the wearer of the spectacle-type device 100. The sensor 124 detects the line of sight of the user 102, for example, by imaging and monitoring the eyes of the user 102. The arrangement of the sensor 124 in FIG. 2 is an example, and the sensor 124 may be arranged at another position.

The spectacle-type device 100 may include a microphone 126. The microphone 126 is arranged at a position where the utterance of the user 102 who is the wearer of the spectacle-type device 100 can be acquired. The arrangement of the microphone 126 in FIG. 2 is an example, and the microphone 126 may be arranged at another position.

The spectacle-type device 100 may include a speaker 128. The speaker 128 outputs sound to the outside of the spectacle-type device 100. The arrangement of the speaker 128 in FIG. 2 is an example, and the speaker 128 may be arranged at another position.

The spectacle-type device 100 may include a light emitting unit 130. The light emitting unit 130 may be, for example, an LED (Light Emitting Diode). The arrangement of the light emitting unit 130 in FIG. 2 is an example, and the light emitting unit 130 may be arranged at another position.

The spectacle-type device 100 may include a sensor 132. The sensor 132 may be a sensor that detects contact with a place where the sensor 132 is installed. The output of the sensor 132 may be used to determine whether the spectacle-type device 100 is attached to the user 102.

The spectacle-type device 100 may include a sensor 134. The sensor 134 may be a sensor that detects contact with a place where the sensor 134 is installed. The output of the sensor 134 may be used to determine if the spectacle-type device 100 is located somewhere.

The spectacle-type device 100 may include a sensor 136. The sensor 136 may be a sensor that detects contact with a place where the sensor 136 is installed. The output of the sensor 136 may be used to determine whether the spectacle-type device 100 is placed upside down.

The spectacle-type device 100 may include a sensor 138. The sensor 138 may be a sensor that detects contact with a place where the sensor 138 is installed.

The spectacle-type device 100 may include an irradiation unit 140. The irradiation unit 140 projects an image. The irradiation unit 140 may be a so-called projector.

The control device 200 may accept various inputs. The control device 200 may accept various inputs from the user 102 via an operation unit arranged at an arbitrary position of the spectacle-type device 100. The control device 200 receives various inputs from the user 102, for example, via an input unit such as a button arranged at an arbitrary position of the frame 110.

The control device 200 may accept voice input by the user 102. The control device 200 may accept the voice input by the user 102 by recognizing the utterance of the user 102 acquired by the microphone 126.

The control device 200 may accept input by the movement of the hand of the user 102. The control device 200 may accept input by the user 102, such as selection of content, by recognizing the movement of the user 102's hand captured by the camera 120. The control device 200 may accept the gesture input by the user 102.

The control device 200 may include various sensors. The control device 200 includes, for example, a positioning sensor. For example, the control device 200 includes a GNSS (Global Navigation Satellite System) sensor such as a GPS (Global Positioning System) sensor. The control device 200 may include an acceleration sensor. The control device 200 may include a gyro sensor. The output of the gyro sensor may be used to determine whether the spectacle-type device 100 is placed upside down. The control device 200 may include other sensors.

The control device 200 may generate image pickup range information indicating the image pickup range of the camera 120. The control device 200 may generate image pickup range information based on the position, orientation, and orientation of the spectacle-type device 100. The control device 200 may specify the position, orientation, and posture of the spectacle-type device 100 by the outputs of various sensors.

The control device 200 may have a display function on the lens unit 116. The lens unit 116 may be an example of a display unit of the spectacle-type device 100. The lens unit 116 may be a transparent or translucent display, and the control device 200 may cause the lens unit 116 to execute various displays. That is, the spectacle-type device 100 may be a transparent or translucent display-type device. Further, the control device 200 may have a function of projecting a display onto the lens unit 116. That is, the spectacle-type device 100 may be a projection-type device.

The control device 200 may have a function of executing communication via the network 20. The control device 200 may access the network 20 by at least one of mobile communication, Wi-Fi communication, and other wireless communication in general. The control device 200 may communicate with the management server 300 via the network 20.

The control device 200 may determine whether or not the spectacle-type device 100 is in the wearing state worn by the user 102. For example, when the sensor 132 detects contact, the control device 200 determines that the spectacle-type device 100 is in the wearing state, and when the sensor 132 does not detect contact, the spectacle-type device 100 wears the spectacle-type device 100. It is determined that the device is not in the state (not attached).

The control device 200 may determine whether or not the spectacle-type device 100 is placed somewhere when the spectacle-type device 100 is not worn. When the sensor 134 detects the contact, the control device 200 may determine that the spectacle-type device 100 is placed. When the sensor 136 detects the contact, the control device 200 may determine that the spectacle-type device 100 is placed upside down. The control device 200 may determine whether the spectacle-type device 100 is placed in the vertical forward direction or the vertical direction upside down based on the output of the gyro sensor.

The control device 200 may determine whether or not the spectacle-type device 100 is folded. The control device 200 determines whether or not the spectacle-type device 100 is folded by the output of a sensor arranged on the hinge of the spectacle-type device 100, for example. The sensor may be built in the frame 110, for example, and detects a state such as a hinge.

When the spectacle-type device 100 is in the wearing state, the control device 200 displays various information on the lens unit 116 according to the user 102 who wears the spectacle-type device 100. For example, the spectacle-type device 100 displays information inward from the lens unit 116. For example, the eyeglass-type device 100 displays various information in the vertical positive direction. Further, for example, the spectacle-type device 100 displays various information on the lens unit 116 in a size suitable for the eyes of the user 102 located in front of the lens unit 116.

The control device 200 controls the lens unit 116 to display information to the outside when the spectacle-type device 100 is not in the wearing state. For example, the control device 200 displays information outward from the lens unit 116. Further, for example, the control device 200 displays information in the vertical positive direction when the spectacle-type device 100 is placed in the vertical positive direction, and when the spectacle-type device 100 is placed in the vertical vertical direction. Displays information upside down. Further, for example, the control device 200 displays information in a larger size than when the spectacle-type device 100 is in the wearing state. As a result, the information can be displayed so that the user 102 who does not wear the spectacle-type device 100 can easily see the information.

For example, when the control device 200 receives new information in e-mail, SNS, or the like, if the eyeglass-type device 100 is not in the wearing state, the control device 200 displays notification information to the outside to notify that the new information has been received. do. The control device 200 displays, for example, the notification information outward of the lens unit 116 in a relatively large size. As a result, even when the user 102 has removed the spectacle-type device 100, it is possible to notify the user 102 that new information has been received by the notification information displayed to the outside. When the control device 200 receives new information, it may notify the outside by light emission or voice output. For example, the control device 200 causes the light emitting unit 130 to emit light, or causes the speaker 128 to output notification information by voice.

After displaying the notification information, the control device 200 may display the detailed information by a simple operation by the user 102. The detailed information may indicate the details of the new arrival information. The control device 200 displays detailed information, for example, when the sensor 138 detects a tap by the user 102 after displaying the notification information.

The control device 200 may set in advance the content to be displayed when the spectacle-type device 100 is not in the wearing state, and display the information to the outside according to the setting. For example, when the time is set to be displayed, the control device 200 displays the time to the outside in response to the non-wearing state of the spectacle-type device 100.

The control device 200 may display the information according to the setting to the outside in response to the detection of the predetermined operation of the user 102 when the spectacle-type device 100 is not in the wearing state. For example, when the frame tap is set as a predetermined operation and the time is set as information, the control device 200 detects the tap by the sensor 138 after the spectacle-type device 100 is not worn. The time is displayed to the outside.

FIG. 3 schematically shows a display example of the notification screen 400 by the eyeglass-type device 100. The notification screen 400 shown in FIG. 3 may be an example of notification information for notifying a new message.

When the control device 200 receives a new message when the spectacle-type device 100 is not in the wearing state, the control device 200 displays the notification screen 400 outward of the lens unit 116 in a relatively large size, as illustrated in FIG. You can do it. As a result, the notification screen 400 can be displayed in an easy-to-see manner for the user 102 who does not wear the spectacle-type device 100.

After displaying the notification screen 400 outward of the lens unit 116, the control device 200 displays the notification screen 400 from the outward direction of the lens unit 116 in response to detecting a predetermined operation of the user 102. You may switch inward. Thereby, for example, when the user 102 with the spectacle-type device 100 removed is located on the wearing side of the spectacle-type device 100, it is possible to provide a function of making the notification screen 400 easy to see by a simple operation.

Although FIG. 3 illustrates a case where the notification screen 400 is displayed on only one of the two lens units 116, the notification screen 400 may be displayed on both of the two lens units 116. When displaying the notification screen 400 on the two lens units 116, the control device 200 may display the same information on the two lens units 116, or may display different information.

By displaying the same information on the two lens units 116, the user 102 can confirm the content of the notification screen 400 by looking at only one of the two lens units 116. The control device 200 may display the notification screen 400 on one of the two lens units 116 facing outward and the other facing inward. This makes it easy for the user 102 to confirm the contents of the notification screen 400 regardless of whether the user 102 is located on the front side of the spectacle-type device 100 or on the wearing side of the spectacle-type device 100. The front side of the spectacle-type device 100 may be the front side of the lens portion 116 of the spectacle-type device 100. The wearing side of the spectacle-type device 100 may be the rear side of the lens portion 116 of the spectacle-type device 100.

For example, the control device 200 divides the content to be notified to the user 102 into two and displays it on each of the two lens units 116. As a result, the amount of information to be displayed can be increased as compared with the case where the same content is displayed on the two lens units 116.

FIG. 4 schematically shows a display example of the notification screen 400 by the eyeglass-type device 100. The notification screen 400 shown in FIG. 4 may be an example of notification information for notifying a new message.

As illustrated in FIG. 4, the spectacle-type device 100 may project the notification screen 400 in the front direction of the spectacle-type device 100 by the irradiation unit 140. As a result, for example, the notification screen 400 can be displayed on an area larger than the lens unit 116, and the visibility of the user 102 may be improved.

The projection direction by the irradiation unit 140 may be changeable. For example, the spectacle-type device 100 may change the projection direction of the notification screen 400 depending on whether the spectacle-type device 100 is arranged in the vertical forward direction or upside down.

The spectacle-type device 100 may accept the hand tracking of the user 102 for the projected image. The eyeglass-type device 100 recognizes, for example, the movement of the user 102's hand with respect to the notification screen 400, and changes the position of the notification screen 400 or the size of the notification screen 400 according to the movement of the hand. You can do it.

FIG. 5 schematically shows a display example of the notification screen 400 by the eyeglass-type device 100. Here, the differences from FIG. 4 will be mainly described.

The spectacle-type device 100 may include an irradiation unit 142 that projects an image in the wearing direction of the spectacle-type device 100, and the irradiation unit 142 may project a notification screen 400 in the wearing direction of the spectacle-type device 100.

The spectacle-type device 100 may include both an irradiation unit 140 and an irradiation unit 142. Then, the spectacle-type device 100 may selectively project the notification screen 400 by the irradiation unit 140 and the irradiation unit 142.

For example, the spectacle-type device 100 determines whether the user 102 from which the spectacle-type device 100 is removed is located on the front side or the wearing side of the spectacle-type device 100 based on the outputs of various sensors.

Then, when the user 102 determines that the user 102 is located on the front side of the spectacle-type device 100, the spectacle-type device 100 projects the notification screen 400 by the irradiation unit 140 and is located on the wearing side of the spectacle-type device 100. If it is determined that the eyeglasses are present, the irradiation unit 142 projects the notification screen 400. As a result, the notification screen 400 can be presented to the user 102 more easily.

FIG. 6 schematically shows an example of the processing flow by the spectacle-type device 100. Here, the flow of processing when the user 102 removes the spectacle-type device 100 while the spectacle-type device 100 of the user 102 and the spectacle-type device 160 of the user 106 are communicating will be described. The communication between the spectacle-type device 100 and the spectacle-type device 160 may be P2P (Peer-to-Peer) communication or communication via a server or the like.

In step 102 (the step may be abbreviated as S) 102, it is detected that the spectacle-type device 100 is in the non-wearing state. In S104, the spectacle-type device 160 selects the spectacle-type device 100 as a chat target or an e-mail transmission target according to the instruction of the user 106, and notifies the spectacle-type device 100 to that effect.

In S106, the spectacle-type device 100 transmits the wearing state status to the spectacle-type device 160. In this example, the spectacle-type device 100 transmits a status indicating that the spectacle-type device 100 is not worn to the spectacle-type device 160.

In S108, the spectacle-type device 160 notifies the user 106 of the wearing state of the spectacle-type device 100, which is the communication partner, indicated by the status received in S106. The spectacle-type device 160, for example, displays and outputs a wearing state and outputs a voice. As a specific example, the spectacle-type device 160 presents the user 106 with a message indicating that the user 102 is not wearing the spectacle-type device 100 and that it may take a long time to respond.

When the user 102 does not wear the spectacle-type device 100, even if the user 106 sends an e-mail or a message by the spectacle-type device 160, the user 102 may not immediately notice or may not be able to reply immediately. However, if the user 106 does not know the state, communication may be hindered without knowing why the reply does not come. On the other hand, according to the spectacle-type device 100 according to the present embodiment, since the wearing state is notified to the communication partner, the occurrence of such a problem can be suppressed.

FIG. 7 schematically shows an example of the functional configuration of the control device 200. The control device 200 includes a storage unit 202, an information acquisition unit 204, a communication unit 206, a display control unit 208, a mounting state determination unit 210, a user determination unit 212, and a notification control unit 214.

The information acquisition unit 204 acquires various types of information. The information acquisition unit 204 may store the acquired information in the storage unit 202.

The information acquisition unit 204 acquires, for example, an image captured by the camera 120 from the camera 120. The information acquisition unit 204 acquires, for example, the information output by various sensors. The information acquisition unit 204 may acquire the information output by the sensor 122. The information acquisition unit 204 may acquire the information output by the sensor 124. The information acquisition unit 204 acquires, for example, the line-of-sight information of the user 102 output by the sensor 124. The information acquisition unit 204 may acquire the information output by the sensor 132. The information acquisition unit 204 may acquire the information output by the sensor 134. The information acquisition unit 204 may acquire the information output by the sensor 136.

The information acquisition unit 204 may acquire the information output by the positioning sensor. The information acquisition unit 204 may acquire the information output by the acceleration sensor. The information acquisition unit 204 may acquire the information output by the gyro sensor. The information acquisition unit 204 may acquire information output by other sensors.

The information acquisition unit 204 acquires, for example, the utterance of the user 102 acquired by the microphone 126. The information acquisition unit 204 may acquire the utterance content by voice-recognizing the utterance of the user 102 acquired by the microphone 126.

The information acquisition unit 204 may acquire various inputs from the user 102. The information acquisition unit 204 may acquire the input information input by the user 102 via the operation unit. The information acquisition unit 204 may acquire the voice input content of the user 102. The information acquisition unit 204 may acquire input information by the movement of the hand of the user 102.

The information acquisition unit 204 may generate image pickup range information indicating the image pickup range of the camera 120. The information acquisition unit 204 may generate image pickup range information based on the position, orientation, and posture of the spectacle-type device 100. The information acquisition unit 204 may specify the position, orientation, and posture of the spectacle-type device 100 by the outputs of various sensors.

The communication unit 206 executes various communications. The communication unit 206 may communicate with various devices connected to the network 20.

The communication unit 206 communicates with, for example, a mail server, an SNS server, an application server, and the like. The communication unit 206 may receive various information such as mails and messages from the mail server, the SNS server, and the application server.

Further, the communication unit 206 communicates with, for example, the management server 300. The communication unit 206 transmits, for example, various information acquired by the information acquisition unit 204 to the management server 300. For example, the communication unit 206 transmits the image pickup range information to the management server 300, and receives the content corresponding to the target within the image pickup range indicated by the image pickup range information from the management server 300.

Further, the communication unit 206 communicates with, for example, another eyeglass-type device 100. The communication unit 206 may communicate with another spectacle-type device 100 without going through the network 20. The communication unit 206 communicates with another eyeglass-type device 100 by using a short-range wireless communication method such as Wi-Fi (registered trademark), Bluetooth (registered trademark) and Zigbee (registered trademark). The communication unit 206 transmits / receives a message to / from, for example, another eyeglass-type device 100.

The display control unit 208 controls to display various information. The display control unit 208 may display various information on the lens unit 116. The display control unit 208 may display various information received by the communication unit 206 on the lens unit 116. The display control unit 208 causes the lens unit 116 to display, for example, an e-mail, a message, or the like. Further, the display control unit 208 causes the lens unit 116 to display, for example, the content corresponding to the object included in the imaging range of the spectacle-type device 100.

The wearing state determination unit 210 determines whether or not the eyeglass-type device 100 is in the wearing state worn by the user 102. For example, when the sensor 132 detects contact, the wearing state determination unit 210 determines that the spectacle-type device 100 is in the wearing state, and when the sensor 132 does not detect contact, the spectacle-type device 100 determines. Is not in the mounted state (not in the mounted state). Further, when the wearing state determination unit 210 determines that the spectacle-type device 100 is moving, for example, by the output of the gyro sensor, the acceleration sensor, or the like, it determines that the spectacle-type device 100 is in the wearing state and is moving. When it is determined that there is no spectacle type device 100, it is determined that the spectacle-type device 100 is not in the wearing state.

The display control unit 208 displays information to the user 102 who wears the spectacle-type device 100 when the spectacle-type device 100 is in the wearing state, and outwards when the spectacle-type device 100 is not in the wearing state. Control to display information.

For example, the display control unit 208 displays information inward of the lens unit 116 of the spectacle-type device 100 when the spectacle-type device 100 is in the wearing state, and the display control unit 208 displays information inward when the spectacle-type device 100 is not in the wearing state. It may be controlled to display the information outward. For example, when the spectacle-type device 100 is not in the wearing state, the display control unit 208 may control the information to be displayed outward in the lens unit in a larger size than in the case where the spectacle-type device 100 is in the wearing state. ..

The display control unit 208 may display information on only one of the two lens units 116 or may display the information on both of the two lens units 116 when the spectacle-type device 100 is not in the wearing state. When the display control unit 208 displays information on the two lens units 116, the same information may be displayed on the two lens units 116, or different information may be displayed. When the display control unit 208 displays information on the two lens units 116, both of the two lens units 116 may be displayed outward. Further, when the display control unit 208 displays information on the two lens units 116, one of the two lens units 116 may display the information outward and the other may display the information inward.

The display control unit 208 displays information outward of the lens unit 116 when the spectacle-type device 100 is not in the wearing state, and then displays the information in response to detecting a predetermined operation by the user 102. The lens unit 116 may be switched from outward to inward. The display control unit 208 may switch the display of information between the outward direction and the inward direction of the lens unit 116 each time the display control unit 208 detects a predetermined operation by the user 102.

When the spectacle-type device 100 receives new information, the display control unit 208 displays the notification information for notifying that the spectacle-type device 100 has been received outward of the lens unit 116 if the spectacle-type device 100 is not in the wearing state. You may control it. The display control unit 208 controls to display the new arrival information outward of the lens unit 116 in response to detecting a predetermined operation by the user 102 after displaying the notification information outward of the lens unit 116. You can do it. Examples of the predetermined operation include tapping the frame 110, contact with the frame 110, and the like, but any operation may be adopted.

The display control unit 208 displays information inward of the lens unit 116 of the spectacle-type device 100 when the spectacle-type device 100 is in the wearing state, and the spectacle-type device 100 is mounted when the spectacle-type device 100 is not in the wearing state. Information may be controlled to be displayed inwardly in the lens unit 116 of the spectacle-type device 100 with a size larger than that in the state.

The display control unit 208 displays information on the first layer of the lens unit 116 of the spectacle-type device 100 when the spectacle-type device 100 is in the wearing state, and the display control unit 208 displays information on the first layer of the lens unit 116 of the spectacle-type device 100. Information may be controlled to be displayed in one layer and a second layer outside the first layer. The display control unit 208 controls, for example, to display information on the first layer inside the opaque layer and the second layer outside the opaque layer when the spectacle-type device 100 is not in the wearing state. As a result, information can be displayed on both the front side of the spectacle-type device 100 and the wearing side of the spectacle-type device 100, and the information can be displayed to the user 102 regardless of whether the user 102 is located on the front side or the wearing side. It can be presented in an easy-to-see manner.

The user determination unit 212 determines whether the user 102 is located on the front side of the spectacle-type device 100 or on the wearing side of the spectacle-type device 100 when the spectacle-type device 100 is not in the wearing state. The user determination unit 212 may determine whether the user 102 is located on the front side or the wearing side of the spectacle-type device 100 based on the outputs of various sensors. When the user determination unit 212 captures the user 102 on the front side of the spectacle-type device 100 by, for example, the captured image captured by the camera 120 and the output by the sensor 122, the user 102 is positioned on the front side of the spectacle-type device 100. If the user 102 is not captured on the front side, it is determined that the user 102 is located on the wearing side of the spectacle-type device 100.

The display control unit 208 displays information outward of the lens unit 116 when the user 102 is located on the front side of the spectacle-type device 100 when the spectacle-type device 100 is not in the wearing state, and the user 102 displays the spectacles. When it is located on the mounting side of the mold device 100, it may be controlled to display information inward of the lens unit 116.

The display control unit 208 may control the irradiation unit 140 or the irradiation unit 142 to project information when the eyeglass-type device 100 is not in the wearing state. The display control unit 208 may control the irradiation unit 140 and the irradiation unit 142 to project information when the eyeglass-type device 100 is not in the wearing state.

The display control unit 208 may selectively project information onto the irradiation unit 140 and the irradiation unit 142 when the eyeglass-type device 100 is not in the wearing state. For example, the display control unit 208 projects information onto the irradiation unit 140 when the user 102 is located on the front side of the spectacle-type device 100 when the spectacle-type device 100 is not in the wearing state, and the user 102 uses the spectacle-type device 100. When it is located on the mounting side of the eyeglasses, the information is projected onto the irradiation unit 142.

When the wearing state determination unit 210 determines that the spectacle-type device 100 is not in the wearing state, the notification control unit 214 tells the other spectacle-type device 100 of the communication partner of the communication unit 206 that the user 102 uses the spectacle-type device 100. The communication unit 206 is controlled so as to transmit the notification information notifying that the glasses are not attached.

FIG. 8 schematically shows an example of a hardware configuration of a computer 1200 that functions as a control device 200. A program installed on the computer 1200 causes the computer 1200 to function as one or more "parts" of the apparatus according to the present embodiment, or causes the computer 1200 to perform an operation associated with the apparatus according to the present embodiment or the one or the like. A plurality of "parts" can be executed and / or a computer 1200 can be made to execute a process according to the present embodiment or a stage of the process. Such a program may be run by the CPU 1212 to cause the computer 1200 to perform certain operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphic controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes an input / output unit such as a communication interface 1222, a storage device 1224, and an IC card drive, which are connected to the host controller 1210 via the input / output controller 1220. The storage device 1224 may be a hard disk drive, a solid state drive, or the like. The computer 1200 also includes an input / output unit such as a ROM 1230 and a keyboard or touch panel, which are connected to the input / output controller 1220 via an input / output chip 1240.

The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires the image data generated by the CPU 1212 in a frame buffer or the like provided in the RAM 1214 or itself so that the image data is displayed on the display device 1218.

The communication interface 1222 communicates with other electronic devices via the network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The IC card drive reads the program and data from the IC card and / or writes the program and data to the IC card.

The ROM 1230 stores in it a boot program or the like executed by the computer 1200 at the time of activation, and / or a program depending on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

The program may be provided by a computer-readable storage medium such as an IC card. The program is read from a computer-readable storage medium, installed in a storage device 1224, RAM 1214, or ROM 1230, which is also an example of a computer-readable storage medium, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200 and provides a link between the program and the various types of hardware resources described above. The device or method may be configured to implement the operation or processing of information in accordance with the use of the computer 1200.

For example, when communication is executed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded in the RAM 1214, and performs communication processing with respect to the communication interface 1222 based on the processing described in the communication program. You may order. Under the control of the CPU 1212, the communication interface 1222 reads the transmission data stored in the transmission buffer area provided in the recording medium such as the RAM 1214, the storage device 1224, or the IC card, and transfers the read transmission data to the network. The received data transmitted or received from the network is written in the reception buffer area or the like provided on the recording medium.

Further, the CPU 1212 makes the RAM 1214 read all or necessary parts of the file or the database stored in the external recording medium such as the storage device 1224 and the IC card, and various types of processing for the data on the RAM 1214. May be executed. The CPU 1212 may then write back the processed data to an external recording medium.

Various types of information such as various types of programs, data, tables, and databases may be stored in recording media and processed. The CPU 1212 describes various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, and information retrieval described in various parts of the present disclosure with respect to the data read from the RAM 1214. Various types of processing may be performed, including / replacement, etc., and the results are written back to the RAM 1214. Further, the CPU 1212 may search for information in a file, a database, or the like in the recording medium. For example, when a plurality of entries each having an attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 is the first of the plurality of entries. The attribute value of the attribute of is searched for the entry that matches the specified condition, the attribute value of the second attribute stored in the entry is read, and the attribute value of the second attribute is changed to the first attribute that satisfies the predetermined condition. You may get the attribute value of the associated second attribute.

The program or software module described above may be stored on a computer 1200 or in a computer readable storage medium near the computer 1200. Also, a recording medium such as a hard disk or RAM provided in a dedicated communication network or a server system connected to the Internet can be used as a computer readable storage medium, thereby allowing the program to be transferred to the computer 1200 via the network. offer.

The blocks in the flowcharts and block diagrams of this embodiment may represent the stage of the process in which the operation is performed or the "part" of the device responsible for performing the operation. Specific steps and "parts" are supplied with a dedicated circuit, a programmable circuit supplied with computer-readable instructions stored on a computer-readable storage medium, and / or with computer-readable instructions stored on a computer-readable storage medium. It may be implemented by the processor. Dedicated circuits may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. Programmable circuits include logical products, ORs, exclusive ORs, Negative ORs, Negative ORs, and other logical operations, such as, for example, Field Programmable Gate Arrays (FPGAs) and Programmable Logic Arrays (PLAs). , Flip-flops, registers, and reconfigurable hardware circuits including memory elements.

The computer readable storage medium may include any tangible device capable of storing instructions executed by the appropriate device, so that the computer readable storage medium having the instructions stored therein may be in a flow chart or block diagram. It will be equipped with a product that contains instructions that can be executed to create means for performing the specified operation. Examples of the computer-readable storage medium may include an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, and the like. More specific examples of computer-readable storage media include floppy (registered trademark) disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), and erasable programmable read-only memory (EPROM or flash memory). , Electrically Erasable Programmable Read Only Memory (EEPROM), Static Random Access Memory (SRAM), Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD), Blu-ray® Disc, Memory Stick , Integrated circuit cards and the like may be included.

Computer-readable instructions include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcodes, firmware instructions, state-setting data, or Smalltalk®, JAVA®, C ++, etc. Object-oriented programming languages, and either source code or object code written in any combination of one or more programming languages, including traditional procedural programming languages such as the "C" programming language or similar programming languages. May include.

Computer-readable instructions are used to generate means for a general-purpose computer, a special-purpose computer, or the processor of another programmable data processing device, or a programmable circuit, to perform an operation specified in a flowchart or block diagram. General purpose computers, special purpose computers, or other programmable data processing locally or via a local area network (LAN), a wide area network (WAN) such as the Internet, etc. to execute such computer-readable instructions. It may be provided to the processor of the device or a programmable circuit. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers and the like.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the description of the claims that the form with such changes or improvements may be included in the technical scope of the present invention.

The order of execution of each process such as operation, procedure, step, and step in the apparatus, system, program, and method shown in the claims, specification, and drawings is particularly "before" and "prior to". It should be noted that it can be realized in any order unless the output of the previous process is used in the subsequent process. Even if the scope of claims, the specification, and the operation flow in the drawings are explained using "first", "next", etc. for convenience, it means that it is essential to carry out in this order. It's not a thing.

10 systems, 20 networks, 100 eyeglass devices, 102 users, 106 users, 110 frames, 112 rims, 114 bridges, 116 lenses, 118 temples, 120 cameras, 122 sensors, 124 sensors, 126 microphones, 128 speakers, 132 sensors. , 134 sensor, 136 sensor, 140 irradiation unit, 142 irradiation unit, 160 eyeglass type device, 200 control device, 202 storage unit, 204 information acquisition unit, 206 communication unit, 208 display control unit, 210 mounting status determination unit, 212 user Judgment unit, 214 notification control unit, 300 management server, 400 notification screen, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 I / O chip

Claims (17)

It ’s a glasses-type device,
A wearing state determination unit that determines whether or not the spectacle-type device is worn by the user, and a wearing state determination unit.
When the spectacle-type device is in the wearing state, information is displayed to the user who wears the spectacle-type device, and when the spectacle-type device is not in the wearing state, information is displayed to the outside. A glasses-type device with a display control unit to control. The display control unit displays information inward of the lens unit of the spectacle-type device when the spectacle-type device is in the mounted state, and outwards of the lens unit when the spectacle-type device is not in the mounted state. The spectacle-type device according to claim 1, which controls to display information. When the spectacle-type device is not in the wearing state, the display control unit controls to display the information outward of the lens unit in a larger size than when the spectacle-type device is in the wearing state. The spectacle-type device according to claim 2. When the spectacle-type device receives new information, if the spectacle-type device is not in the wearing state, the display control unit displays notification information for notifying that the new arrival information has been received outward of the lens unit. The spectacle-type device according to claim 2 or 3, wherein the spectacle-type device is controlled so as to be used. The display control unit displays the notification information outward of the lens unit, and then displays the new arrival information outward of the lens unit in response to detecting a predetermined operation by the user. The spectacle-type device according to claim 4, which is controlled in such a manner. The display control unit displays information inward in the lens portion of the spectacle-type device when the spectacle-type device is in the wearing state, and when the spectacle-type device is not in the wearing state, the spectacle-type device is in the wearing state. The spectacle-type device according to claim 1, wherein the spectacle-type device is controlled to display information inward in the lens portion of the spectacle-type device with a size larger than that in the case of. The display control unit displays information on the first layer of the lens unit of the spectacle-type device when the spectacle-type device is in the wearing state, and when the spectacle-type device is not in the wearing state, the first layer of the lens unit. The spectacle-type device according to claim 1, wherein the information is controlled to be displayed on one layer and a second layer outside the first layer. 7. The display control unit controls to display information on the first layer inside the opaque layer and the second layer outside the opaque layer when the spectacle-shaped device is not in the wearing state. Eyeglass-type device described in. A user determination unit for determining whether the user is located on the front side of the spectacle-type device or on the wearing side of the spectacle-type device when the spectacle-type device is not in the wearing state is provided.
The display control unit displays information outward from the lens unit of the spectacle-type device when the user is located on the front side of the spectacle-type device when the spectacle-type device is not in the wearing state. The spectacle-type device according to claim 1, wherein when the user is located on the wearing side of the spectacle-type device, the user controls to display information inward in the lens portion of the spectacle-type device. Equipped with an irradiation unit that projects information to the outside
The spectacle-type device according to claim 1, wherein the display control unit controls to project information on the irradiation unit when the spectacle-type device is not in the wearing state. The irradiation unit has a first irradiation unit that projects information in the front direction of the eyeglass-type device, and a second irradiation unit that projects information in the mounting direction of the eyeglass-type device.
The spectacle-type device according to claim 10, wherein the display control unit selectively projects the information onto the first irradiation unit and the second irradiation unit when the spectacle-type device is not in the wearing state. A user determination unit for determining whether the user is located on the front side of the spectacle-type device or on the wearing side of the spectacle-type device when the spectacle-type device is not in the wearing state is provided.
The display control unit projects information onto the first irradiation unit when the user is located on the front side of the spectacle-type device when the spectacle-type device is not worn, and the user can use the spectacle-type device. The spectacle-type device according to claim 11, wherein the information is projected onto the second irradiation unit when the device is located on the wearing side of the device. With a communication unit that communicates with other eyeglass-type devices,
When the wearing state determination unit determines that the spectacle-type device is not in the wearing state, the user does not wear the spectacle-type device on the other spectacle-type device of the communication partner of the communication unit. The spectacle-type device according to any one of claims 1 to 12, further comprising a notification control unit that controls the communication unit to transmit notification information. It ’s a glasses-type device,
With a communication unit that communicates with other eyeglass-type devices,
A wearing state determination unit that determines whether or not the spectacle-type device is worn by the user, and a wearing state determination unit.
When the wearing state determination unit determines that the spectacle-type device is not in the wearing state, the user does not wear the spectacle-type device on the other spectacle-type device of the communication partner of the communication unit. A spectacle-type device including a notification control unit that controls the communication unit so as to transmit the notification information for notifying the spectacle-type device to the other spectacle-type device. A program for making a spectacle-type device function as the spectacle-type device according to any one of claims 1 to 14. A control method performed by a spectacle-type device,
A wearing state determination step for determining whether or not the eyeglass-type device is worn by the user, and
When the spectacle-type device is in the wearing state, information is displayed to the user who wears the spectacle-type device, and when the spectacle-type device is not in the wearing state, information is displayed to the outside. A control method with a display control stage to control. A control method performed by a spectacle-type device,
The communication stage to communicate with other eyeglass-type devices,
A wearing state determination step for determining whether or not the eyeglass-type device is worn by the user, and
Notification to notify the other spectacle-type device of the communication partner that the user is not wearing the spectacle-type device when it is determined in the wearing state determination stage that the spectacle-type device is not in the wearing state. A control method comprising a notification step of transmitting information to the other eyeglass-type device.

Images