CN116386486A - Virtual window self-adaptive adjusting method and device of intelligent glasses and intelligent glasses - Google Patents
Virtual window self-adaptive adjusting method and device of intelligent glasses and intelligent glasses Download PDFInfo
- Publication number
- CN116386486A CN116386486A CN202211549327.5A CN202211549327A CN116386486A CN 116386486 A CN116386486 A CN 116386486A CN 202211549327 A CN202211549327 A CN 202211549327A CN 116386486 A CN116386486 A CN 116386486A
- Authority
- CN
- China
- Prior art keywords
- virtual window
- camera
- display screen
- target object
- intelligent glasses
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000011521 glass Substances 0.000 title claims abstract description 85
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000009434 installation Methods 0.000 claims abstract description 55
- 230000003044 adaptive effect Effects 0.000 claims description 24
- 239000004984 smart glass Substances 0.000 claims description 23
- 238000006243 chemical reaction Methods 0.000 claims description 18
- 238000003384 imaging method Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 7
- 230000006978 adaptation Effects 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 10
- 230000003190 augmentative effect Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/001—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/14—Display of multiple viewports
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The disclosure relates to a virtual window self-adaptive adjustment method and device for intelligent glasses. The method comprises the following steps: acquiring a real scene image, and determining an initial virtual window based on a target object in the real scene image; acquiring a relative position relation between a display screen of the intelligent glasses and a camera, and adjusting the initial virtual window according to the relative position relation to acquire a first virtual window; determining a relative included angle relation between the display screen installation plane and the camera installation plane based on the camera installation plane, and adjusting the first virtual window according to the relative included angle relation to obtain a second virtual window; and acquiring the real-time distance between the target object and the camera in the actual scene, adjusting the second virtual window according to the real-time distance between the target object and the camera to acquire a target virtual window, and displaying the target virtual window on the display screen. The target virtual window can be accurately overlapped with a target object in the real world, so that the identification accuracy of the target object is improved, and the user experience is improved.
Description
Technical Field
The disclosure relates to the technical field of intelligent glasses, and in particular relates to a virtual window self-adaptive adjustment method and device of intelligent glasses, intelligent glasses and a computer readable storage medium.
Background
Augmented reality (Augmented Reality, abbreviated as AR), which is a newer technology content that facilitates integration between real world information and virtual world information content, implements simulated simulation processing on the basis of scientific technology such as a computer on the basis of which entity information that is otherwise difficult to experience in a spatial range of the real world is superimposed, effectively applies virtual information content in the real world, and can be perceived by human senses in the process, thereby realizing a sensory experience of augmented reality. In the prior art, when a user wearing the AR glasses experiences an AR application recognition scene, such as face recognition, license plate recognition and object recognition, the phenomenon that the virtual-real superposition window and the real object are overlapped and misplaced, which is observed through the optical lens, often occurs, so that the user experience is poor.
Disclosure of Invention
To overcome the problems in the related art, the present disclosure provides a virtual window adaptive adjustment method and apparatus for smart glasses, and a computer-readable storage medium. When an installation included angle exists between the camera and the display screen of the intelligent glasses, the virtual window self-adaptive adjustment method of the intelligent glasses can carry out self-adaptive adjustment on the virtual window according to the installation included angle between the camera and the display screen and the real-time distance between the camera and the target object, so that the virtual window and the target object in the real world are accurately overlapped, the recognition accuracy of the target object is improved, and the user experience is improved.
According to a first aspect of embodiments of the present disclosure, a virtual window adaptive adjustment method of an intelligent glasses is provided, and the virtual window adaptive adjustment method is applied to the intelligent glasses, where the intelligent glasses include a camera and a display screen, the camera and the display screen are mounted on different planes, and a mounting plane of the display screen and a mounting plane of the camera are at a preset included angle;
the self-adaptive adjustment method for the virtual window comprises the following steps:
photographing a real scene to obtain a real scene image, and determining an initial virtual window based on a target object in the real scene image;
acquiring a relative position relation between a display screen and a camera of the intelligent glasses, and adjusting the initial virtual window according to the relative position relation to acquire a first virtual window;
determining a relative included angle relation between the display screen installation plane and the camera installation plane based on the camera installation plane, and adjusting the first virtual window according to the relative included angle relation to obtain a second virtual window;
and acquiring a real-time distance between a target object and a camera in an actual scene, adjusting the second virtual window according to the real-time distance between the target object and the camera to acquire a target virtual window, and displaying the target virtual window on the display screen.
According to a second aspect of embodiments of the present disclosure, there is provided a virtual window adaptive adjustment apparatus, including:
the initial virtual window determining module is used for photographing the real scene to obtain a real scene image, and determining an initial virtual window based on a target object in the real scene image;
the first adjusting module is used for acquiring the relative position relation between the display screen of the intelligent glasses and the camera, and adjusting the initial virtual window according to the relative position relation to acquire a first virtual window;
the second adjusting module is used for determining the relative included angle relation between the display screen installation plane and the camera installation plane based on the camera installation plane, and adjusting the first virtual window according to the relative included angle relation to obtain a second virtual window;
and the third adjusting module is used for acquiring the real-time distance between the target object and the camera in the actual scene, adjusting the second virtual window according to the real-time distance between the target object and the camera to acquire a target virtual window, and displaying the target virtual window on the display screen.
According to a third aspect of embodiments of the present disclosure, there is provided an intelligent glasses, the intelligent glasses including a camera and a display screen, the camera and the display screen being mounted on different planes, the display screen mounting plane and the camera mounting plane being a preset included angle;
the smart glasses further comprise at least one processor and at least one memory;
the memory is electrically connected with the processor;
wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method for adaptive adjustment of a virtual window as described in any of the embodiments above.
According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the virtual window adaptive adjustment method according to any one of the embodiments above.
By applying the technical scheme, when an installation included angle exists between the camera and the display screen of the intelligent glasses, the virtual window self-adaptive adjustment method determines an initial virtual window according to the target object in the real scene image obtained through shooting, adjusts the initial virtual window for a plurality of times according to the relative position relation between the display screen and the camera and the relative included angle relation between the display screen installation plane and the camera installation plane, and then adjusts the virtual window in a self-adaptive manner according to the real-time distance between the target object and the camera in the actual scene, so that the virtual window and the real-world target object are accurately overlapped, the identification accuracy of the target object is improved, and the user experience is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
For a better understanding and implementation, the present disclosure is described in detail below with reference to the drawings.
Drawings
Fig. 1 is a schematic structural diagram of smart glasses according to an embodiment of the present disclosure;
fig. 2 is a flow chart of a virtual window adaptive adjustment method according to an embodiment of the disclosure;
fig. 3 is a flowchart of step S2 of a virtual window adaptive adjustment method according to an embodiment of the disclosure;
fig. 4 is a flowchart of step S3 of a virtual window adaptive adjustment method according to an embodiment of the disclosure;
fig. 5 is a flowchart of step S4 of a virtual window adaptive adjustment method according to an embodiment of the disclosure;
fig. 6 is a schematic diagram of a first application of a face recognition scenario according to an embodiment of the disclosure;
fig. 7 is a schematic diagram of a second application of the face recognition scenario shown in the embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of a virtual window adaptive adjustment device according to an embodiment of the disclosure;
fig. 9 is a schematic diagram of smart glasses according to an alternative embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.
The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if"/"if" as used herein may be interpreted as "at … …" or "at … …" or "in response to a determination", depending on the context.
The virtual window self-adaptive adjustment method in the embodiment of the disclosure is applied to intelligent glasses, and is particularly suitable for AR glasses. Referring to fig. 1, fig. 1 is a schematic structural diagram of smart glasses according to an embodiment of the disclosure.
The intelligent glasses comprise a glasses frame 101, and a camera 102 and a display screen 103 which are arranged on the glasses frame 101, wherein the display screen 103 is a lens 103 of the intelligent glasses, the lens 103 and the camera 102 are arranged on different planes, namely, the display screen 103 and the camera 102 are arranged on different planes, the installation plane of the display screen and the installation plane of the camera form a certain preset included angle, the determination mode and the specific numerical value of the preset included angle are not limited, and the preset can be carried out according to an experience value.
The smart glasses may be binocular glasses or monocular glasses, in fig. 1, the smart glasses are binocular glasses having two lenses, which are transparent lenses, virtual windows can be displayed on the lenses, and a user can see the real world through the lenses when wearing the glasses. In other embodiments, the smart glasses may also be monocular glasses, i.e. having only one lens as a display screen.
The smart glasses may be VR (Virtual Reality) glasses, AR (Augmented Reality, AR, augmented Reality) glasses, or MR (Mixed Reality) glasses, which have perspective functions.
The AR glasses are AR head displays and augmented reality head-mounted display devices. The augmented reality (Augmented Reality, AR) technology is a technology for skillfully fusing virtual information with a real world, and widely uses various technical means such as multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, sensing and the like, and applies virtual information such as characters, images, three-dimensional models, music, videos and the like generated by a computer to the real world after simulation, wherein the two kinds of information are mutually complemented, so that the enhancement of the real world is realized.
The virtual-real superposition window is an important application in the AR glasses, a user can draw a virtual window on the glasses when looking at the real world through the glasses of the intelligent glasses, and the virtual window is superposed on a target object of the real world in a dislocation-free manner in a view angle of the user. The virtual-real superposition window can be applied to various recognition scenes, for example, the face of the person is recognized through intelligent glasses or the license plate is recognized. Because the camera and the lens of the general intelligent glasses are arranged on the same plane, when the virtual window is drawn and adjusted, the virtual window can be adjusted according to the position relationship between the camera and the lens of the intelligent glasses, so that the virtual window is overlapped with a target object in the real world.
The applicant finds in the research process that if a certain installation included angle exists between the camera and the lens of the intelligent glasses, namely, the camera and the lens are not located on the same plane, on one hand, the design of the appearance of the intelligent glasses is facilitated, and on the other hand, a user obtains more comfortable wearing experience. However, when an installation included angle exists between the camera and the lens of the intelligent glasses, if the virtual window drawing and adjusting method is still adopted, the problem that the virtual window and the real world target object cannot be accurately overlapped may exist, and the virtual window relatively linearly deviates after imaging. In the recognition scene, the positions of the user and the target object are changed in real time, if the virtual window and the target object cannot be accurately overlapped, poor experience can be brought to the user, and the accuracy of recognition information can be influenced to a certain extent.
The method for adaptively adjusting the virtual window is described in detail below through specific embodiments.
According to a first aspect of an embodiment of the present disclosure, a virtual window adaptive adjustment method of an intelligent glasses is disclosed, which is applied to the intelligent glasses. Referring to fig. 2, fig. 2 is a flow chart illustrating a method for adaptively adjusting a virtual window of an intelligent glasses according to an embodiment of the disclosure.
The virtual window self-adaptive adjusting method of the intelligent glasses comprises the following steps:
step S1: and photographing the real scene to obtain a real scene image, and determining an initial virtual window based on a target object in the real scene image.
The real scene image may be an image obtained by photographing a current real scene with a camera on the smart glasses, and the resolution of the real scene image is an imaging resolution of the camera.
If the smart glasses are applied to the recognition scene, the target object may be an object to be recognized, such as a face to be recognized or a license plate to be recognized, which is a target entity in the real world.
In this embodiment, the step S1 may be implemented by the following method: and photographing the real scene to obtain a real scene image, determining a target object in the real scene image, and determining an initial virtual window based on the edge of the target object. If the target object does not exist in the real scene image, deleting the image so as to prevent the shot image from occupying the memory of the intelligent glasses excessively.
Alternatively, step S1 may be implemented by the following method: according to the characteristics of the object to be identified, searching and judging the object to be identified in the real scene in advance, photographing the real scene with the target object to obtain a real scene image, and determining an initial virtual window based on the edge of the target object in the real scene image.
Step S2: and acquiring a relative position relation between a display screen of the intelligent glasses and the camera, and adjusting the initial virtual window according to the relative position relation to acquire a first virtual window.
Since the initial virtual window is determined based on the target object on the real scene image, and the display screen and the camera imaging are located in different coordinate systems, the coordinate systems need to be unified, and coordinate conversion needs to be performed on the initial virtual window according to the relative positional relationship between the display screen and the camera.
For example, a group of camera imaging data can be collected, the face model is fixed at the same position, and the camera moves back and forth for 1m,2m,3m and ETC equidistance to take pictures and image; then calculating the offset of the horizontal position of the face frame in the photo; assuming that the fore-and-aft translation distance is x, there is x1=offset_1; x2=offset_2; x3=offset_3, etc., a quantized linear relationship can be obtained, so that this linear relationship can be added in the course of coordinate transformation, and can be corrected.
The relative positional relationship between the display screen and the camera includes: the physical positional relationship between the cameras of the display and the scaling relationship between the display resolution and the camera imaging resolution.
Optionally, referring to fig. 3, fig. 3 is a flow chart of step S2 in the embodiment of the disclosure.
Step S2 may be implemented by the following steps:
s21: and acquiring a physical position relation between a display screen and a camera of the intelligent glasses, and performing plane coordinate conversion on the initial virtual window based on the physical position relation.
In this embodiment, step S21 includes: acquiring external parameters of a camera and the installation position of a display screen relative to the center of the intelligent glasses; determining a physical position relation between the display screen and the camera according to the camera external parameters and the installation position of the display screen relative to the center of the intelligent glasses; and performing plane coordinate conversion on the initial virtual window based on the physical position relation between the display screen and the camera.
Wherein the off-camera parameters are parameters in the world coordinate system, such as the position, rotation direction, etc. of the camera determined based on the center of the smart glasses. And the installation parameters of the display screen are also determined based on the center of the intelligent glasses, so that the physical position relationship between the display screen and the camera can be determined.
S22: and obtaining a scaling relation between the resolution of the display screen and the imaging resolution of the camera, and scaling the initial virtual window after the plane coordinate conversion according to the scaling relation to obtain a first virtual window.
In this embodiment, step S22 includes: acquiring resolution of a display screen and parameters in a camera; determining a scaling relationship between a virtual scene and a real scene image according to the resolution of the display screen and the parameters in the camera; and scaling the initial virtual window after the plane coordinate conversion according to the scaling relation to obtain a first virtual window.
The in-camera parameters are parameters related to the characteristics of the camera itself, such as the focal length of the camera, the pixel size, and the like.
If the intelligent glasses are provided with a left-eye camera and a right-eye camera, the internal parameters of the left-eye camera and the right-eye camera can be the same as those of the right-eye camera or can be adjusted according to actual conditions.
And step S3, determining a relative included angle relation between the display screen installation plane and the camera installation plane based on the camera installation plane, and adjusting the first virtual window according to the relative included angle relation to obtain a second virtual window.
In an alternative embodiment, referring to fig. 4, fig. 4 is a schematic flow chart of step S3 in an embodiment of the disclosure.
Step S3 may be implemented by the following steps, step S3 including:
s31: determining a camera mounting plane and a display screen mounting plane;
s32: based on the camera mounting plane, a relative angular relationship between the display screen mounting plane and the camera mounting plane is determined.
Typically, the display screen may determine its mounting plane based on the smart glasses center and the camera may determine its mounting plane based on the smart glasses center. Because the virtual window is displayed on the display screen, the relative included angle relation between the two installation planes can be determined by taking the installation plane of the display screen as a reference.
S33: and according to the relative included angle relation, performing space coordinate conversion on the first virtual window to obtain a second virtual window.
If an included angle exists between the camera and the display screen, the camera installation plane and the display screen installation plane are different planes, so that the relative included angle relation between the two planes is required to be acquired, and the virtual window is adjusted according to the relative included angle relation so as to unify the virtual window on the display screen installation plane.
And S4, acquiring a real-time distance between a target object and a camera in an actual scene, adjusting the second virtual window according to the real-time distance between the target object and the camera to acquire a target virtual window, and displaying the target virtual window on the display screen.
In an alternative embodiment, please refer to fig. 5, fig. 5 is a flowchart illustrating step S4 in an embodiment of the disclosure.
Step S4 may be implemented by the following steps, step S4 including:
s41: the real-time distance between the target object and the camera is obtained according to the imaging principle of the camera.
Specifically, a plurality of real scene images are obtained, face imaging sizes in the real scene images are measured, actual distances between the corresponding cameras and the target objects are obtained, association between the face imaging sizes and the actual distances is estimated through analysis of the face imaging sizes and the corresponding actual distances, and then the actual distances are reversely calculated according to the face imaging sizes.
For example, when the target object is a face, the size of the face in the image is 3 square centimeters when the distance between the camera and the face is 10 meters, the size of the face in the image is 2.8 square centimeters when the distance between the camera and the face is 9 meters, the size of the face in the image is 2.5 square centimeters when the distance between the camera and the face is 8 meters, the correlation between the imaging size of the face and the actual distance is calculated according to the data, and then the actual distance is calculated according to the imaging size of the face.
S42: and carrying out linearization operation on the basis of a predetermined distance linearization conversion relation and the real-time distance between the target object and the camera to obtain the linear distance between the target object and the camera.
Wherein, the predetermined distance linearization conversion relation comprises:
a plurality of distance values between a target object and a camera are obtained in advance;
based on a plurality of distance values, calculating the outline size of a target object and the size of a virtual window on a corresponding real scene image;
and determining a distance linear conversion relation according to a plurality of distance values, the corresponding outline dimension of the target object and the virtual window dimension.
S43: and adjusting the second virtual window according to the linear distance between the target object and the camera to obtain a target virtual window.
The second virtual window is adjusted through the linear distance between the target object and the camera, so that the operation amount can be reduced, and the delay can be reduced.
S44: and drawing the target virtual window and displaying the target virtual window on a display screen.
Referring to fig. 6 and fig. 7, fig. 6 is a schematic diagram of a first application of a face recognition scenario shown in an embodiment of the disclosure; fig. 7 is a second application schematic diagram of a face recognition scenario shown in an embodiment of the disclosure.
The virtual window self-adaptive adjusting method can be applied to face recognition scenes or license plate recognition scenes and other recognition scenes, and is wide in application range. Alternatively, the face recognition can be realized through the intelligent glasses or through a face recognition system.
In an alternative embodiment, the intelligent glasses may be networked with the face recognition system, and when the face of the real scene is photographed, the photograph may be transmitted to the face recognition system, and then the face recognition system returns the face recognition information.
In fig. 6, the virtual window adaptive adjustment method is applied to a face recognition scene of the intelligent glasses, the circle is a face to be recognized, the dashed box may be a target virtual window in the embodiment, and the information in the solid box is information obtained by recognizing the face in the virtual box, such as the gender corresponding to the face, and/or the name of the person, and/or the identification card number, and/or the geographical location of the person, and other related information.
In fig. 7, an image is shown on a computer, the image is virtual, and in reality, the face recognition system does not record the related information of the image, so that the face in the image cannot be recognized, and then a word of "Unknown" and some image information are displayed below.
By applying the technical scheme, when an installation included angle exists between the camera and the display screen of the intelligent glasses, the virtual window self-adaptive adjustment method determines an initial virtual window according to the target object in the real scene image obtained through shooting, adjusts the initial virtual window for a plurality of times according to the relative position relation between the display screen and the camera and the relative included angle relation between the display screen installation plane and the camera installation plane, and then adjusts the virtual window in a self-adaptive manner according to the real-time distance between the target object and the camera in the actual scene, so that the virtual window and the real-world target object are accurately overlapped, the identification accuracy of the target object is improved, and the user experience is improved.
According to a second aspect of the disclosed embodiments, a virtual window adaptive adjustment device is disclosed, which may be used to execute the virtual window adaptive adjustment method of the intelligent glasses according to the corresponding embodiments of the present application, and has corresponding functions and beneficial effects. For details not disclosed in the embodiments of the adaptive adjustment device for a virtual window in the present application, please refer to the content of the adaptive adjustment method for a virtual window in the above embodiments of the present application.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a virtual window adaptive adjustment device according to an embodiment of the disclosure.
A virtual window adaptive adjustment device, comprising:
an initial virtual window determining module 801, configured to take a photograph of a real scene to obtain a real scene image, and determine an initial virtual window based on a target object in the real scene image;
the first adjustment module 802 is configured to obtain a relative positional relationship between a display screen of the smart glasses and the camera, and adjust the initial virtual window according to the relative positional relationship to obtain a first virtual window;
a second adjusting module 803, configured to determine a relative angle relationship between the display screen installation plane and the camera installation plane based on the camera installation plane, and adjust the first virtual window according to the relative angle relationship to obtain a second virtual window;
the third adjustment module 804 is configured to obtain a real-time distance between a target object and a camera in an actual scene, adjust the second virtual window according to the real-time distance between the target object and the camera to obtain a target virtual window, and display the target virtual window on the display screen.
According to the virtual window self-adaptive adjusting device, when an installation included angle exists between the camera and the display screen of the intelligent glasses, the virtual window self-adaptive adjusting method determines an initial virtual window according to a target object in a real scene image obtained through shooting, adjusts the initial virtual window for a plurality of times according to the relative position relation between the display screen and the camera and the relative included angle relation between the display screen installation plane and the camera installation plane, and then adjusts the virtual window in a self-adaptive mode according to the real-time distance between the target object and the camera in an actual scene, so that the virtual window and the target object in the real world are accurately overlapped, recognition accuracy of the target object is improved, and user experience is improved.
It should be noted that, when executing the virtual window adaptive adjustment method of the smart glasses, the virtual window adaptive adjustment device provided in the foregoing embodiment is only exemplified by the division of the foregoing functional modules, and in practical application, the foregoing functional allocation may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the device and the embodiment of the adaptive adjustment method for the virtual window provided in the foregoing embodiments belong to the same concept, and the detailed implementation process of the device and the embodiment of the adaptive adjustment method for the virtual window are shown in the embodiments and are not repeated here.
According to a third aspect of the embodiments of the present disclosure, there is provided a pair of smart glasses, which may be VR glasses, AR glasses, or MR glasses, and the present embodiment is not limited thereto.
Referring to fig. 1 and 9, fig. 1 is a schematic structural diagram of a pair of smart glasses according to an embodiment of the disclosure, and fig. 9 is a schematic block diagram of a pair of smart glasses according to an embodiment of the disclosure.
The intelligent glasses are AR glasses, the intelligent glasses comprise a camera and a display screen, the camera and the display screen are arranged on different planes, and a preset included angle is formed between the display screen installation plane and the camera installation plane.
The smart glasses include at least one processor 110, at least one network interface 120, a user interface 130, a memory 140, and at least one communication bus 150.
Wherein the communication bus 150 is used to enable connected communication between these components.
The user interface 130 may include an interface for connecting to a display screen, and the optional user interface 130 may also include a standard wired interface, a wireless interface, among others.
Wherein the processor 110 may include one or more processing cores. The processor 110 utilizes various interfaces and lines to connect various portions of the overall electronic device 100, perform various functions of the electronic device 100, and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the processor 110, and invoking data stored in the memory 140. Alternatively, the processor 110 may be implemented in hardware in at least one of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable logic arrays, PLA). The processor 110 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 110 and may be implemented by a single chip.
The Memory 140 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 140 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 140 may be used to store instructions, programs, code sets, or instruction sets. The memory 140 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-described various method embodiments, etc.; the storage data area may store data or the like referred to in the above respective method embodiments. Memory 140 may also optionally be at least one storage device located remotely from the aforementioned processor 110. As shown in fig. 9, an operating system, a network communication module, a user interface module, and an operating application of the smart device may be included in the memory 140, which is one type of computer storage medium.
A memory 140 stores a computer program adapted to be loaded by said processor 110 and to perform the virtual window adaptation method according to any of the embodiments described above.
The intelligent glasses can be used for executing the content of the virtual window self-adaptive adjusting method according to the corresponding embodiment of the application, and have corresponding functions and beneficial effects.
According to a fourth aspect of embodiments of the present application, there is provided a computer-readable storage medium, on which a computer program is stored, the computer program being adapted to be loaded by a processor and to perform the relevant operations of the virtual window adaptation method as described in the above embodiments, and having corresponding functions and advantageous effects.
Where computer-readable storage media includes both non-transitory and non-transitory, removable and non-removable media, information storage may be implemented by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable storage media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.
Note that the above is only a preferred embodiment of the present application and the technical principle applied. Those skilled in the art will appreciate that the present application is not limited to the particular embodiments described herein, but is capable of numerous obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the present application. Therefore, while the present application has been described in connection with the above embodiments, the present application is not limited to the above embodiments, but may include many other equivalent embodiments without departing from the spirit of the present application, the scope of which is defined by the scope of the appended claims.
Claims (11)
1. The virtual window self-adaptive adjusting method of the intelligent glasses is applied to the intelligent glasses, the intelligent glasses comprise cameras and display screens, the cameras and the display screens are arranged on different planes, and the installation plane of the display screen and the installation plane of the camera form a preset included angle; it is characterized in that the method comprises the steps of,
the self-adaptive adjustment method for the virtual window comprises the following steps:
photographing a real scene to obtain a real scene image, and determining an initial virtual window based on a target object in the real scene image;
acquiring a relative position relation between a display screen and a camera of the intelligent glasses, and adjusting the initial virtual window according to the relative position relation to acquire a first virtual window;
determining a relative included angle relation between the display screen installation plane and the camera installation plane based on the camera installation plane, and adjusting the first virtual window according to the relative included angle relation to obtain a second virtual window;
and acquiring a real-time distance between a target object and a camera in an actual scene, adjusting the second virtual window according to the real-time distance between the target object and the camera to acquire a target virtual window, and displaying the target virtual window on the display screen.
2. The method for adaptively adjusting a virtual window of a pair of smart glasses according to claim 1, wherein the relative positional relationship between the display and the camera comprises: a physical positional relationship between cameras of the display screen and a scaling relationship between the display screen resolution and the camera imaging resolution;
the obtaining the relative position relationship between the display screen and the camera of the intelligent glasses, and adjusting the initial virtual window according to the relative position relationship to obtain a first virtual window comprises the following steps:
acquiring a physical position relation between a display screen and a camera of the intelligent glasses, and performing plane coordinate conversion on the initial virtual window based on the physical position relation;
and obtaining a scaling relation between the resolution of the display screen and the imaging resolution of the camera, and scaling the initial virtual window after the plane coordinate conversion according to the scaling relation to obtain a first virtual window.
3. The method for adaptively adjusting a virtual window of an intelligent glasses according to claim 2, wherein the obtaining a physical positional relationship between a display screen and a camera of the intelligent glasses, and performing planar coordinate transformation on the initial virtual window based on the physical positional relationship, comprises:
acquiring external parameters of a camera and the installation position of a display screen relative to the center of the intelligent glasses;
determining a physical position relation between the display screen and the camera according to the camera external parameters and the installation position of the display screen relative to the center of the intelligent glasses;
and performing plane coordinate conversion on the initial virtual window based on the physical position relation between the display screen and the camera.
4. The method for adaptively adjusting a virtual window of an intelligent glasses according to claim 2, wherein the obtaining a scaling relationship between a resolution of a display screen and an imaging resolution of a camera, scaling an initial virtual window after plane coordinate conversion according to the scaling relationship to obtain a first virtual window, comprises:
acquiring resolution of a display screen and parameters in a camera;
determining a scaling relationship between a virtual scene and a real scene image according to the resolution of the display screen and the parameters in the camera;
and scaling the initial virtual window after the plane coordinate conversion according to the scaling relation to obtain a first virtual window.
5. The method for adaptively adjusting a virtual window of an intelligent glasses according to claim 1, wherein determining a relative angle relation between a display screen installation plane and a camera installation plane based on the camera installation plane, and adjusting the first virtual window according to the relative angle relation to obtain a second virtual window comprises:
determining a camera mounting plane and a display screen mounting plane;
determining a relative angle relationship between the display screen mounting plane and the camera mounting plane based on the camera mounting plane;
and according to the relative included angle relation, performing space coordinate conversion on the first virtual window to obtain a second virtual window.
6. The method for adaptively adjusting a virtual window of an intelligent glasses according to claim 1, wherein the obtaining the real-time distance between the target object and the camera in the actual scene, adjusting the second virtual window according to the real-time distance between the target object and the camera to obtain a target virtual window, and displaying the target virtual window on a display screen, comprises:
obtaining a real-time distance between a target object and a camera according to an imaging principle of the camera;
performing linearization operation based on a predetermined distance linearization conversion relation and the real-time distance between the target object and the camera to obtain a linear distance between the target object and the camera;
adjusting the second virtual window according to the linear distance between the target object and the camera to obtain a target virtual window;
and drawing the target virtual window and displaying the target virtual window on a display screen.
7. The method for adaptively adjusting a virtual window of a pair of smart glasses according to claim 6, wherein the predetermined distance linearizes a conversion relationship, comprising:
a plurality of distance values between a target object and a camera are obtained in advance;
based on a plurality of distance values, calculating the outline size of a target object and the size of a virtual window on a corresponding real scene image;
and determining a distance linear conversion relation according to a plurality of distance values, the corresponding outline dimension of the target object and the virtual window dimension.
8. A virtual window adaptive adjustment apparatus, comprising:
the initial virtual window determining module is used for photographing the real scene to obtain a real scene image, and determining an initial virtual window based on a target object in the real scene image;
the first adjusting module is used for acquiring the relative position relation between the display screen of the intelligent glasses and the camera, and adjusting the initial virtual window according to the relative position relation to acquire a first virtual window;
the second adjusting module is used for determining the relative included angle relation between the display screen installation plane and the camera installation plane based on the camera installation plane, and adjusting the first virtual window according to the relative included angle relation to obtain a second virtual window;
and the third adjusting module is used for acquiring the real-time distance between the target object and the camera in the actual scene, adjusting the second virtual window according to the real-time distance between the target object and the camera to acquire a target virtual window, and displaying the target virtual window on the display screen.
9. The intelligent glasses are characterized by comprising cameras and display screens, wherein the cameras and the display screens are arranged on different planes, and the installation plane of the display screen and the installation plane of the camera form a preset included angle;
the smart glasses further comprise at least one processor and at least one memory;
the memory is electrically connected with the processor;
wherein the memory stores a computer program adapted to be loaded by the processor and to perform the virtual window adaptation method of any one of claims 1 to 7.
10. The smart glasses according to claim 9, wherein the smart glasses are AR glasses.
11. A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, implements the virtual window adaptation method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211549327.5A CN116386486A (en) | 2022-12-05 | 2022-12-05 | Virtual window self-adaptive adjusting method and device of intelligent glasses and intelligent glasses |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211549327.5A CN116386486A (en) | 2022-12-05 | 2022-12-05 | Virtual window self-adaptive adjusting method and device of intelligent glasses and intelligent glasses |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116386486A true CN116386486A (en) | 2023-07-04 |
Family
ID=86964362
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211549327.5A Pending CN116386486A (en) | 2022-12-05 | 2022-12-05 | Virtual window self-adaptive adjusting method and device of intelligent glasses and intelligent glasses |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116386486A (en) |
-
2022
- 2022-12-05 CN CN202211549327.5A patent/CN116386486A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10362296B2 (en) | Localized depth map generation | |
| JP6431245B1 (en) | Edge recognition bidirectional image processing | |
| CN109887003B (en) | Method and equipment for carrying out three-dimensional tracking initialization | |
| CN110276317B (en) | Object size detection method, object size detection device and mobile terminal | |
| CN113689578B (en) | Human body data set generation method and device | |
| TW202013149A (en) | Augmented reality image display method, device and equipment | |
| JP2017022694A (en) | Method and apparatus for displaying light field based image on user's device, and corresponding computer program product | |
| US9813693B1 (en) | Accounting for perspective effects in images | |
| US20210041945A1 (en) | Machine learning based gaze estimation with confidence | |
| CN114742703A (en) | Method, device and equipment for generating binocular stereoscopic panoramic image and storage medium | |
| CN114066999A (en) | Target positioning system and method based on three-dimensional modeling | |
| EP3599588A1 (en) | Rendering an object | |
| CN109978945B (en) | Augmented reality information processing method and device | |
| CN115239815B (en) | Camera calibration method and device | |
| CN112017242B (en) | Display method and device, equipment and storage medium | |
| WO2021151380A1 (en) | Method for rendering virtual object based on illumination estimation, method for training neural network, and related products | |
| CN116188349A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN116386486A (en) | Virtual window self-adaptive adjusting method and device of intelligent glasses and intelligent glasses | |
| CN117726747A (en) | Three-dimensional reconstruction method, device, storage medium and equipment for complementing weak texture scene | |
| CN116450002A (en) | VR image processing method and device, electronic device and readable storage medium | |
| WO2023183041A1 (en) | Perspective-dependent display of surrounding environment | |
| US20180108173A1 (en) | Method for improving occluded edge quality in augmented reality based on depth camera | |
| CN112114659B (en) | Method and system for determining a fine gaze point of a user | |
| CN116883770A (en) | Training method and device of depth estimation model, electronic equipment and storage medium | |
| CN115205974A (en) | Gesture recognition method and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |