CN107765840A - A kind of Eye-controlling focus method equipment of the general headset equipment based on binocular measurement - Google Patents
A kind of Eye-controlling focus method equipment of the general headset equipment based on binocular measurement Download PDFInfo
- Publication number
- CN107765840A CN107765840A CN201610696362.8A CN201610696362A CN107765840A CN 107765840 A CN107765840 A CN 107765840A CN 201610696362 A CN201610696362 A CN 201610696362A CN 107765840 A CN107765840 A CN 107765840A
- Authority
- CN
- China
- Prior art keywords
- infrared
- eyeglass
- display screen
- eye
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/193—Preprocessing; Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
Abstract
The invention discloses a kind of Eye-controlling focus method, system and the equipment of the general headset equipment based on binocular measurement, wherein binocular measuring apparatus includes imaging display device, Eye-controlling focus device and infrared light generating means, and the visual focusing point tracking device includes infrared external reflection eyeglass, miniature infrared camera；Method comprises the following steps, pre-processing image data；Measuring apparatus off-line calibration；Measuring apparatus on-line proving；Direction of visual lines measures and fixation point measurement.What the present invention can realize has the technical effect that：Stated accuracy is lifted, reduces the nominal time, and then improves experience effect of the user in the interaction of general headset equipment.
Description
Technical field
The invention belongs to general wear-type display technology field, and in particular to a kind of side of the Eye-controlling focus based on binocular measurement
Method, system and equipment.
Background technology
In virtual reality (VR) experience of the prior art, some is employed sets the mode of button real on glasses
Now selection confirms operation, and a part realizes interaction by the way of handle, and some catches limb action using camera
Mode realize interaction, also some use Eye-controlling focus mode.
However, button, handle, the mode of motion capture are undesirable for the operating efficiency of user, existing Eye-controlling focus
The eye tracking that mode presently, there are is divided into two categories below：Two-dimensional map is calculated, calculates three-dimensional sight；The minimal configuration of the first kind
It is that a camera adds an infrared index point, but needs user to stare 9 calibration points online, process is relatively complicated；Second class
Method belongs to three-dimensional measurement, and minimal configuration is that a camera adds two infrared index points, it is necessary to the human-eye model of complexity.
The content of the invention
The technical problem to be solved in the present invention, there is provided a kind of method of Eye-controlling focus based on binocular measurement, system and set
It is standby, it is possible to reduce the nominal time that user uses for the first time, to lift the precision of Eye-controlling focus, and then lift Consumer's Experience.
To solve the above problems, the present invention adopts the following technical scheme that：
A kind of equipment of the Eye-controlling focus based on binocular measurement, including：Imaging display device, visual focusing follow-up mechanism and
Housing.
Preferably, the imaging display device is positioned in user's visual field, including positioned at described in housing front wall observable
Viewing eyeglass, the display screen of enclosure interior, the display screen are arranged on the enclosure interior along observer's direction of visual lines.
Preferably, the visual focusing point tracking device includes infrared external reflection eyeglass, miniature infrared camera, infrared photoproduction
Into device and mainboard, the infrared external reflection eyeglass is between user's human eye and imaging display device, the infrared external reflection eyeglass
It is obliquely installed with imaging display device, the miniature infrared camera, which is provided towards looking squarely sight angle with user, to be 10 ° and arrive
Between 120 °, the miniature infrared camera towards and infrared external reflection eyeglass angle be 30 ° to 85 ° between；The infrared light
Generating means pacify the infrared light generating means and infrared camera whole installation, or with the miniature infrared camera split
Set, between viewing eyeglass and display screen, or the infrared light generating means be located at viewing eyeglass and eyes of user it
Between, fixed along viewing lens edge position.
Preferably, the viewing eyeglass includes left eye viewing eyeglass and right eye viewing eyeglass, and the display screen includes left eye
Display screen and right eye display screen, baffle plate, one end of the baffle plate and shell are provided between the left eye display screen and right eye display screen
It is connected in the middle part of body antetheca.
Preferably, the infrared external reflection eyeglass is between left eye viewing eyeglass and left eye display screen, the infrared external reflection
Eyeglass is fixed on the housing front wall with left eye viewing eyeglass, left eye display screen into Z-shaped, described miniature infrared camera one end
On, the miniature infrared camera other end is fixed on the baffle plate.
Preferably, the infrared external reflection eyeglass is between left eye viewing eyeglass and left eye display screen, the infrared external reflection
Eyeglass and left eye viewing eyeglass, left eye display screen are into anti-Z-shaped, before described miniature infrared camera one end is fixed on the housing
Wall left end, the miniature infrared camera other end are fixed in the housing left wall.
Preferably, the infrared external reflection eyeglass is between right eye viewing eyeglass and right eye display screen, the infrared external reflection
Eyeglass is fixed on the housing front wall with right eye viewing eyeglass, right eye display screen into Z-shaped, described miniature infrared camera one end
Right-hand member, the miniature infrared camera other end are fixed in the housing right wall.
Preferably, the infrared external reflection eyeglass is between right eye viewing eyeglass and right eye display screen, the infrared external reflection
Eyeglass is fixed on the housing with right eye viewing eyeglass, right eye display screen into anti-Z-shaped, described miniature infrared camera one end
On antetheca, the miniature infrared camera other end is fixed on the baffle plate.
Preferably, described miniature infrared camera at least two forms one group of binocular camera, the folder between two cameras
Angle is 0 to 45 degree, the distance between two be 0 to the left arm of housing, right arm, baffle plate maximum between described miniature infrared take the photograph
As head group by bolt or industrial glue hydropexis in housing.
Preferably, after the follow-up mechanism mainboard is located at display screen, it is connected by winding displacement with miniature infrared camera, it is described
Host mainboard is additionally provided with housing, after the host mainboard is located at display screen, the follow-up mechanism mainboard passes through USB interface, string
One kind or combination in mouth, SDIO, MIPI are connected with the host mainboard, by the mainboard of follow-up mechanism and virtual reality main frame
Mainboard is separately set up, and visual focusing point tracking device is kept relative independence, convenient dimension with virtual reality imaging device
Repair.
A kind of method of the Eye-controlling focus based on binocular measurement, comprises the following steps：
Step 1：Measuring apparatus off-line calibration；
Step 2：Pre-processing image data；
Step 3：Measuring apparatus on-line proving；
Step 4：Real-time visual line measurement.
S1 measuring apparatus off-line calibration is by the miniature infrared camera in visual focusing point tracking device, imaging display device
In flat-faced screen, infrared external reflection eyeglass and infrared light generating means are all unified arrives camera measurement coordinate system.
Preferably, binocular camera is fixed on housing, and the demarcation of binocular camera is by calibration element.
Preferably, flat-faced screen demarcation by control display screen display indicate or by scaling board, be printed on mark
Attachment film measures to obtain the position and direction of flat-faced screen at screen using Stereometric device.
Preferably, Stereometric device uses the above-mentioned binocular camera being fixed on housing.
Preferably, the demarcation of infrared external reflection eyeglass is filled by placing mark on infrared external reflection eyeglass using measurement in space
Put to obtain the position relationship for the screen that is all-trans.Mark is scaling board or the pad pasting for being printed on mark, is fitted in infrared external reflection eyeglass
Anterior or rear portion.Stereometric device uses the above-mentioned binocular camera being fixed on housing.
Preferably, infrared generator uses the array of infrared lamp bead composition, demarcates the position of lamp bead in lamp bead array.Lamp
The location position of pearl is needed by extra Stereometric device and calibration element.The Stereometric device is that photo taking type three-dimensional is swept
Retouch one or more combination in instrument, laser 3 d scanner, three dimensional coordinate measuring machine, self-control binocular camera；Described calibration element
For standard calibration plate or other marks.
Preferably, the Stereometric device selection self-control binocular camera, self-control binocular camera are complete using standard calibration plate
Into after demarcation, it is fixed on and ensures that the relative position of two cameras in self-control binocular camera is constant on a rigid objects, but make by oneself double
The integral position of mesh camera is adjustable.
Preferably, the calibration element is standard calibration plate.
Preferably, the demarcation of lamp bead needs to adjust self-control binocular camera, calibration element and is fixed on binocular camera on housing
Relative position, making self-control binocular camera and the binocular camera that is fixed on housing, totally four cameras can see the same of calibration element simultaneously
One region.
Preferably, the demarcation of lamp bead needs to adjust self-control binocular camera, calibration element and is fixed on binocular camera on housing
Relative position so that make by oneself binocular camera two cameras see public domain A, two of the binocular camera being fixed on housing
Camera sees public domain B, and public domain A and B are in same calibration element, and region A and region B relative position can
Survey.
S2 image preprocessings are divided into the extraction of pupil elliptic contour and the extraction of infrared mark.
Preferably, the extraction of infrared mark：Significance detection first is carried out to artwork, then in significance testing result figure
Upper progress light blob detection, obtain the sub-pixel location of bright spot.
Preferably, the extraction of pupil elliptic contour is divided into rough extraction and accurate extraction process.Rough extraction uses threshold value
The method of segmentation, different regions is obtained, ellipse fitting is carried out to different regions, removes ineligible region, and then
Obtain elliptic contour and center；The center that accurate extraction is obtained using extracting roughly carries out algorithm of region growing, and in view of upper
The influence that flushing OM outer marker thing is calculated elliptic contour and central point, it is again accurate to calculate oval profile and central point.
Preferably, the extraction of infrared mark and pupil elliptic contour can on the basis of previous frame processing result image,
The computing of full figure is no longer carried out, reduces operand.
Preferably, for eye closing situation, Kalman filter can be used to realize that the continuously smooth of pupil position is estimated.
The on-line proving of S3 measuring apparatus is, it is necessary to which user in the case of wearing spectacles, stares screen, by being fixed on shell
Binocular camera on body shoots the picture of now eyes, S2 pretreatment is carried out to picture, and then complete on-line proving.
Preferably, user needs to stare in screen to a little less, the position at screen midpoint for the center of screen or other
Position；Demarcate drift angle of the sight with eyeball optical axis.
Preferably, the position of the cornea centre of sphere is obtained by computing by obtained in S2 at least two infrared marks, then
By the position of the pupil center obtained in S2, and then determine drift angle of the sight with eyeball optical axis.
Preferably, infrared mark selection pul drink spot or other significant characteristics.
The real-time visual line measurements of S4, the image for the binocular camera collection being fixed on housing is reflected by infrared external reflection eyeglass
Picture, therefore in measurement process by infrared external reflection eyeglass reflective relation introduce.
Preferably, the process of the visual line measurement in real time is：The image photographed in real time to camera carries out S2 pretreatment,
Real-time pupil center and the cornea centre of sphere obtained in S3 are obtained, calculates the drift angle of real-time direction of visual lines and eye optical axis.
Preferably, the process of the visual line measurement in real time is：The image photographed in real time to camera carries out S2 pretreatment,
Real-time pupil center and the cornea centre of sphere are obtained, calculates the drift angle of real-time direction of visual lines and eye optical axis.
Preferably, the picture number of shooting per second at least 10, to reduce tracking delay, the real-time of system is improved.
Preferably, the direction of visual lines obtained according to the photo of captured in real-time, then the sight with being obtained during on-line proving and eye
Ball optical axis direction carries out computing, and then obtains the position of screen currently watched attentively.
Preferably, the direction of visual lines and eyeball optical axis direction obtained according to the photo of captured in real-time, then during with on-line proving
Obtained sight and eyeball optical axis direction carries out computing, and then obtains the position of screen currently watched attentively.
What the present invention can realize has the technical effect that：Demarcation when can reduce user's use takes, and improves stated accuracy,
And then lift the experience of user.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is the schematic diagram of the first embodiment of the present invention；
Fig. 2 is the front view of the first embodiment of the present invention；
Fig. 3 is the schematic diagram of second of embodiment of the present invention；
Fig. 4 is the schematic diagram of the third embodiment of the present invention；
Fig. 5 is the schematic diagram of the 4th kind of embodiment of the present invention；
Fig. 6 is the schematic flow sheet of a kind of method of the Eye-controlling focus based on binocular measurement of the present invention, system and equipment.
Brief description of the drawings：Housing 1, viewing eyeglass 2, left eye viewing eyeglass 21, right eye viewing eyeglass 22, left eye display screen 31,
Right eye display screen 32, infrared external reflection eyeglass 51, miniature infrared camera 521, miniature infrared camera 522, follow-up mechanism mainboard
53, direction and the observer of miniature infrared camera look squarely sight angle 6, miniature infrared camera direction and infrared external reflection eyeglass
Angle 7, host mainboard 8.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, below in conjunction with accompanying drawing, the present invention is entered
Row is further described.It should be appreciated that embodiment described herein is not used to only to explain the present invention
Limit the present invention.
As shown in Figure 1 and Figure 2, a kind of virtual reality imaging device with infrared light generating means, including housing 1, be located at
Viewing eyeglass 2, display screen inside the antetheca observable housing 1 of housing 1, display screen are arranged on the shell along observer's direction of visual lines
Inside body 1, viewing eyeglass 2 includes left eye viewing eyeglass 21 and right eye viewing eyeglass 22, and display screen includes the He of left eye display screen 31
Right eye display screen 32, is provided with baffle plate 4 between left eye display screen 31 and right eye display screen 32, one end of baffle plate 4 with the antetheca of housing 1
Portion is connected；1 visual focusing point tracking device is provided with housing 1, visual focusing point tracking device includes infrared external reflection eyeglass
51st, miniature infrared camera 521 and 522 (diagram only draws one), infrared external reflection eyeglass 51 is located at the He of left eye viewing eyeglass 21
Between left eye display screen 31, infrared external reflection eyeglass 51 and left eye viewing eyeglass 21, left eye display screen 31 are z-shape, miniature infrared to take the photograph
As first 521 and 522 one end are fixed on the antetheca of housing 1, the miniature other end of infrared camera 521 and 522 is fixed on described
On baffle plate 4, direction and the observer of miniature infrared camera look squarely sight angle 6 as 40 °, miniature infrared camera towards with it is red
The angle 7 of external reflectance eyeglass is 70 °, and angle is 0 to 45 degree between two infrared cameras, apart from the width for 0 to baffle plate 4.
In preferred embodiment, visual focusing point tracking device also includes follow-up mechanism mainboard 53, follow-up mechanism mainboard 53
After display screen, follow-up mechanism mainboard 53 is connected by winding displacement (diagram is not drawn into) with miniature infrared camera 521,522, shell
Host mainboard 8 is additionally provided with body 1, after host mainboard 8 is located at display screen, (diagram is not by USB connecting lines for follow-up mechanism mainboard 53
Draw) it is connected with host mainboard 8.
In preferred embodiment, (diagram is not by bolt for miniature infrared camera 521 and 522 (diagram only draws one)
Draw) it is fixed in housing 1.
Another preferred scheme, miniature infrared camera 521,522 can also be viscous by industrial glue (diagram is not drawn into)
Patch is fixed in housing.
It should be noted that in preferred embodiment, as shown in figure 3, infrared external reflection eyeglass 51 can also be used positioned at a left side
Observe between shadow eyeglass 21 and left eye display screen 31, infrared external reflection eyeglass 51 and left eye viewing eyeglass 21, left eye display screen 31 into
Anti- Z-shaped, miniature infrared camera 521 and 522 (diagram only draws one) one end is fixed on the antetheca left end of housing 1, miniature
The other end of infrared camera 521 and 522 is fixed in the left wall of institute's housing 1.Angle is 0 to 45 degree between two infrared cameras,
Distance is 0 to housing left arm width.
In preferred embodiment, as shown in figure 4, the He of right eye viewing eyeglass 22 can also be located at using infrared external reflection eyeglass 51
Between right eye display screen 32, infrared external reflection eyeglass 51 and right eye viewing eyeglass 22, right eye display screen 32 are z-shape, miniature infrared to take the photograph
Picture first 521 and 522 (diagram only draws one) one end are fixed on the antetheca right-hand member of housing 1, and miniature infrared camera 521 and 522 is another
One end is fixed in the right wall of housing 1.Angle is 0 to 45 degree between two infrared cameras, and distance is 0 to housing right arm width.
In preferred embodiment, as shown in figure 5, the He of right eye viewing eyeglass 22 can also be located at using infrared external reflection eyeglass 51
Between right eye display screen 32, infrared external reflection eyeglass 51 and right eye viewing eyeglass 22, right eye display screen 32 are miniature infrared into anti-Z-shaped
Camera 521 and 522 (diagram only draws one) one end is fixed on the antetheca of housing 1, and miniature infrared camera 521 and 522 is another
One end is fixed on baffle plate 4.Angle is 0 to 45 degree between two infrared cameras, and distance is 0 to the width of baffle 4.
In preferred scheme, at least one infrared light generating means, infrared light generating means are located at viewing eyeglass and user's eye
Between eyeball (diagram is not drawn into), the infrared light that more than 700 nanometers of generation is projected on infrared external reflection eyeglass 51.
Another preferred scheme, at least one infrared light generating means, infrared light generating means and miniature infrared camera
52 turn into a whole installation.
Another preferred scheme, at least one infrared light generating means, infrared light generating means can also with it is described miniature
Infrared camera split settings, between viewing eyeglass and display screen；The infrared light generating means be located at viewing eyeglass with
Between eyes of user, fixed (diagram is not drawn into) along the peripheral position of viewing lens edge, the infrared light that more than 700 nanometers of generation
It is projected on infrared external reflection eyeglass 51.
The present invention realizes Eye-controlling focus by following steps：
Step 1：Measuring apparatus off-line calibration；
Step 2：Pre-processing image data；
Step 3：Measuring apparatus on-line proving；
Step 4：Real-time visual line measurement.
S1 measuring apparatus off-line calibration is by the miniature infrared camera in visual focusing point tracking device, imaging display device
In flat-faced screen, infrared light generating means and infrared external reflection eyeglass, whole unified measuring coordinate systems to binocular camera.
(1) demarcation of camera
Under pin-hole model, it is real to pass through three steps from three-dimensional world coordinate to the conversion two-dimentional computer picture coordinate
It is existing：World coordinate system is to camera coordinates, camera coordinates to image plane coordinate, image plane coordinate to computer picture coordinate.
A point M (X under world coordinate systemw, Yw, Zw) can be with table to the mathematics transformational relation between the picture point m (u, v) on the plane of delineation
It is shown as：
Wherein, matrixRespectively photo coordinate system is with calculating
The transition matrix between transition matrix, camera coordinate system and photo coordinate system, world coordinate system between machine image coordinate
Transition matrix between camera coordinate system.In transfer process between world coordinate system and camera coordinate system, if
Rigid body translation, then only consider rotation transformation R and translation transformation t, wherein, R be 3x3 orthogonal matrices, t for D translation to
Amount.dx, dyFor the physical size of unit pixel in x-axis and Y direction, f is the focal length of video camera.
P is 3x4 matrixes, referred to as projection matrix, wherein P in formula (1)1Completely by ax, ay, u0, v0Determine.Due to ax,
ay, u0, v0It is only relevant with video camera internal structure, therefore referred to as intrinsic parameters of the camera；P2Completely by video camera relative to generation
The outside orientation decision of boundary's coordinate system, referred to as video camera external parameter.
The demarcation of stero and the difference of single camera calibration are also to need to measure the relative of twin camera by demarcating
Position relationship.During twin camera relative position is calculated, we are assuming that single camera has completed the base of demarcation
On plinth, the position relationship between video camera is asked by the inside and outside parameter of two single cameras.
In solution procedure, the homogeneous coordinates that we arrange picture point in the image plane of left and right are used respectivelyRepresent, space
Point M secondly coordinate is usedRepresent, the projection equation of left and right camera can be written as：
Wherein Pl1、Pr1Left and right projection matrix P is represented respectivelyl、PrThe 3x3 parts on the middle left side, Pl、PrProjection square is represented respectively
The 3x1 parts on the right in battle array.Following relation can be obtained through operation transform：
It can be drawn by (3)：As long as knowing the projection matrix of left and right camera respectively, then the relative position of twin camera and pole
Line equation monarch can be obtained by two projection matrixes.
In the present embodiment, the machine of binocular phase 1 is fixed on housing, binocular camera 2 be fixed on one it is moveable straight
On rigid board, the demarcation of inside and outside ginseng and the position relationship of binocular camera 1 and 2 of camera are completed by scaling board and classical method
Demarcation.
(2) imaging display device is demarcated
The demarcation of imaging display device uses convergence type stereoscopic model, under this vision mode, calculates target point M
Three-dimensional coordinate, can be solved by projective transformation matrix using least square method.Assuming that two
The projection matrix of individual video camera is respectively Pl、Pr, and, it is known that then have：
Simultaneous equations (4), (5) eliminate zcl、zcrAfterwards, X is obtainedw、Yw、ZwFour linear equations：
Therefore, equations simultaneousness above can be obtained to spatial point M coordinate (Xw, Yw, Zw).But in actual applications, due to
Data are always noisy, and spatial point M three-dimensional coordinate (X is generally obtained using least square methodw, Yw, Zw)。
The specific implementation of imaging display device demarcation utilizes above method, and scaling board or patch are placed on flat-faced screen
Mark is put, scaling board is irradiated using infrared light supply, the position of flat-faced screen is demarcated by binocular camera 1.
(3) infrared external reflection eyeglass is demarcated
Preferably, with (2) principle, fitting placement scaling board, uses the binocular camera 1 demarcated on infrared external reflection eyeglass
Demarcate infrared external reflection eyeglass.
Preferably, with (2) principle, fitting mark is demarcated using the binocular camera 1 demarcated on infrared external reflection eyeglass
Infrared external reflection eyeglass.
(4) infrared generator is demarcated
With (2) principle, infrared generator uses the array of infrared lamp bead composition, positioned at viewing eyeglass and user's glasses it
Between, the location position of lamp bead is needed by the extra binocular camera 2 demarcated.Infrared lamp bead position passes through in infrared external reflection eyeglass
It is upper to place scaling board and the scaling board is extended to the position that binocular camera 2 sees infrared lamp bead and the scaling board simultaneously, lead to
Cross binocular camera 1, binocular camera 2 while shoot scaling board and obtain the position of infrared lamp bead.
By stereoscopic camera to (calibrated, referred to as camera ensures that infrared lamp bead can to 1) being positioned in front of infrared lamp bead
See, while stereoscopic camera in the connected infrared camera of same glasses to, to the rear of (camera is to 2), can now measure infrared lamp
Position of the pearl in camera is to 1；Then plane reference part is positioned over camera to 1 front, can now measures plane reference seat
Mark system arrives camera to 1 and camera to the rotation and translation of 2 coordinate systems respectively, can obtain infrared mark by Coordinate Conversion and exist
Position of the infrared camera to coordinate system.
S2 image preprocessings are divided into the extraction of pupil elliptic contour and the extraction of infrared mark.The extraction of infrared mark：First
Significance detection is carried out to artwork, light blob detection is then carried out in significance testing result figure, obtains the sub-pix position of bright spot
Put.The extraction of pupil elliptic contour is divided into rough extraction and accurate extraction process.Rough extraction is using adaptive threshold fuzziness
Method, elliptic contour is obtained, then elliptical center is determined by the method rejecting useless region of fitted ellipse；Accurate extraction is using thick
The center for slightly extracting to obtain carries out algorithm of region growing, and elliptic contour and central point are calculated in view of infrared mark above
Influence, it is again accurate to calculate oval profile and central point.For eye closing situation, pupil can be realized using Kalman filter
The continuously smooth estimation of hole site.
The on-line proving of S3 measuring apparatus completes the mark of user, it is necessary to user in the case of wearing spectacles, stares screen
It is fixed.Now user needs to stare extremely a little less in screen, and the position at screen midpoint is center or the other positions of screen.Pass through
The image preprocessing that S2 is carried out to the picture shot in binocular camera 1 obtains pupil center and at least two infrared index point positions
Put, the centre of sphere of cornea obtained by the two infrared index points, by the center of the cornea centre of sphere and pupil determine direction of visual lines and
Eye optical axis direction.
The real-time visual line measurements of S4, the image of infrared camera collection is the picture reflected by infrared external reflection eyeglass, therefore is measured
During by plane mirror reflective relation introduce.
Preferably, the picture number of shooting per second at least 30, tracking delay is reduced.
Preferably, the pretreatment for S2 being carried out according to the photo of captured in real-time obtains pupil center and at least two infrared marks
Point position, obtains the centre of sphere of cornea by the two infrared index points, sight side is determined by the center of the cornea centre of sphere and pupil
Computing is carried out to eye optical axis direction, then the sight with being obtained during on-line proving and eyeball optical axis direction, and then is obtained current
The position for the screen watched attentively.
Its operation principle is：By the picture at thermal camera high-speed capture eye position, by picture while shooting
It is transmitted, identifies the position of pupil and infrared index point in picture, cornea is obtained by least two infrared index points
The centre of sphere, and then obtain the direction of direction of visual lines and eyeball optical axis, by the direction of visual lines during user's use and during on-line proving
Computing is carried out with the direction of eyeball optical axis, the watching area corresponded in real time on screen, realizes visual pursuit.And because high speed is clapped
Major class picture (each second is up to 120) can be generated by taking the photograph, therefore a large amount of invalid datas can be generated in computing.The present invention by
Threshold segmentation is carried out to shooting picture before visual pursuit, the invalid background image outside pupil watching area is deleted, remains with
Region is imitated, and according to former frame result, carrying out computing in its certain scope replaces Zone Full, considerably reduces meter
The data volume of calculation machine processing, improves reaction speed.
What the present invention can realize has the technical effect that：Time and the complexity of user's on-line proving can be reduced, raising regards
Feel the precision for focusing on point tracking, experience effect of the lifting user in general headset equipment interactive experience.
The foregoing is only a specific embodiment of the invention, but protection scope of the present invention is not limited thereto, any
The change or replacement expected without creative work, it should all be included within the scope of the present invention.Therefore, it is of the invention
Protection domain should be determined by the scope of protection defined in the claims.
Claims (22)
1. a kind of general headset equipment based on binocular measurement, including：Imaging display device, visual focusing point tracking device and shell
Body；Characterized in that, the visual focusing point tracking device includes infrared light generating means, infrared external reflection eyeglass, miniature infrared
Camera and follow-up mechanism mainboard, the miniature infrared camera at least two form one group.
A kind of 2. general headset equipment based on binocular measurement according to claim 1, it is characterised in that：The imaging is aobvious
Showing device is fixed in user's visual field, includes the viewing eyeglass and display screen positioned at housing front wall observable enclosure interior；Institute
State and at least one visual focusing point tracking device is provided with housing.
A kind of 3. general headset equipment based on binocular measurement according to claim 2, it is characterised in that：The viewing mirror
Piece includes left eye viewing eyeglass and right eye viewing eyeglass；The display screen is arranged in the housing along observer's direction of visual lines
Portion, the display screen include left eye display screen and right eye display screen, and gear is provided between the left eye display screen and right eye display screen
Plate, one end of the baffle plate in the middle part of housing front wall with being connected.
A kind of 4. general headset equipment based on binocular measurement according to claim 1, it is characterised in that：The infrared light
Generating means can produce infrared light of the wave band more than 700 nanometers；The infrared light generating means form lamp bead battle array by lamp bead
Row, or area source is formed by reflector；The infrared light generating means and infrared camera whole installation, or with it is described micro-
Type infrared camera split settings, between viewing eyeglass and display screen, or the infrared light generating means are located at viewing
Between eyeglass and eyes of user, fixed along the peripheral position of viewing lens edge.
A kind of 5. general headset equipment based on binocular measurement according to claim 2, it is characterised in that：It is described infrared anti-
Eyeglass is penetrated between user's human eye and imaging display device, the infrared external reflection eyeglass is obliquely installed with imaging display device,
The miniature infrared camera is provided towards looking squarely between sight angle is 10 ° to 120 ° with user, described miniature infrared to take the photograph
Between being 30 ° to 85 ° as the angle of head direction and infrared external reflection eyeglass.
A kind of 6. general headset equipment based on binocular measurement according to claim 5, it is characterised in that：It is described infrared anti-
Eyeglass is penetrated between left eye viewing eyeglass and left eye display screen, the infrared external reflection eyeglass shows with left eye viewing eyeglass, left eye
Display screen is fixed on the housing into Z-shaped, the miniature infrared camera at least two, each miniature infrared photography one end
On antetheca, the other end is fixed on the baffle plate of housing.
A kind of 7. general headset equipment based on binocular measurement according to claim 5, it is characterised in that：It is described infrared anti-
Eyeglass is penetrated between left eye viewing eyeglass and left eye display screen, the infrared external reflection eyeglass shows with left eye viewing eyeglass, left eye
Display screen is fixed on the housing front wall left end, the miniature infrared camera into anti-Z-shaped, described miniature infrared camera one end
The other end is fixed in the housing left wall.
A kind of 8. general headset equipment based on binocular measurement according to claim 5, it is characterised in that：It is described infrared anti-
Eyeglass is penetrated between right eye viewing eyeglass and right eye display screen, the infrared external reflection eyeglass shows with right eye viewing eyeglass, right eye
Display screen is fixed on the housing front wall right-hand member into Z-shaped, described miniature infrared camera one end, and the miniature infrared camera is another
One end is fixed in the housing right wall.
A kind of 9. general headset equipment based on binocular measurement according to claim 5, it is characterised in that：It is described infrared anti-
Eyeglass is penetrated between right eye viewing eyeglass and right eye display screen, the infrared external reflection eyeglass shows with right eye viewing eyeglass, right eye
Display screen is fixed on the housing front wall into anti-Z-shaped, described miniature infrared camera one end, and the miniature infrared camera is another
One end is fixed on the baffle plate of housing.
A kind of 10. general headset equipment based on binocular measurement according to claim 5, it is characterised in that：It is described miniature
The angle between two miniature infrared cameras in every group of infrared camera is 0 to 45 degree, two miniature infrared cameras it
Between distance for 0 between the left wall of housing, right wall, the maximum of baffle plate.
A kind of 11. general headset equipment based on binocular measurement according to claim 2, it is characterised in that：The tracking
After device mainboard is located at display screen, the follow-up mechanism mainboard is connected by winding displacement with miniature infrared camera；In the housing
Be additionally provided with host mainboard, after the host mainboard is located at display screen, the follow-up mechanism mainboard by USB connecting lines or other connect
Mouth is connected with the host mainboard.
A kind of 12. Eye-controlling focus side of general headset equipment based on binocular measurement according to claim any one of 1-11
Method, it is characterised in that realized by following steps：
Step 1：Measuring apparatus off-line calibration；
Step 2：Pre-processing image data；
Step 3：Measuring apparatus on-line proving；
Step 4：Real-time visual line measurement.
13. Eye-controlling focus method according to claim 12, it is characterised in that：Off-line calibration described in step 1 is by binocular
The general headset equipment of measurement is unified into camera measurement coordinate system, including：It is miniature infrared in visual focusing point tracking device
Camera, imaging display device, infrared external reflection eyeglass and infrared light generating means.
14. Eye-controlling focus method according to claim 13, it is characterised in that：The imaging display device shows including plane
Display screen, the demarcation of the full-screen panel, show that demarcation is put with pattern or on flat-faced screen by control plane display screen
Demarcation thing is put to realize；The demarcation uses gridiron pattern or other patterns with pattern.
15. Eye-controlling focus method according to claim 13, it is characterised in that：The infrared external reflection eyeglass demarcation, passes through
Plane reference part or mark are measured to the position and direction of infrared external reflection eyeglass, plane reference part or mark and institute
The fitting of infrared external reflection eyeglass is stated to place.
16. Eye-controlling focus method according to claim 13, it is characterised in that：The position mark of the infrared light generating means
Surely completed using three-dimensional measuring apparatus.
17. Eye-controlling focus method according to claim 16, it is characterised in that：The position mark of the infrared light generating means
Surely the three-dimensional measuring apparatus used is photo taking type spatial digitizer, laser 3 d scanner, three dimensional coordinate measuring machine, self-control binocular
One or more combination in camera；Marked by being placed in other known coordinate systems the infrared lamp bead position of infrared light generating means
Will thing computing obtains.
18. Eye-controlling focus method according to claim 12, it is characterised in that：Pre-processing image data includes in step 2
Extraction and the extraction of infrared mark to user's pupil elliptic contour.
19. Eye-controlling focus method according to claim 18, it is characterised in that：Described image data prediction first passes through original
Figure carries out significance detection, and light blob detection is then carried out in significance testing result figure, obtains the sub-pixel location of bright spot.
20. Eye-controlling focus method according to claim 19, it is characterised in that：When infrared bright spot appears in pupil contour line
When neighbouring, extracted with reference to bright spot, reject the profile point that bright spot influences, improve ellipse fitting precision；For eye closing situation, card is used
Thalmann filter realizes the continuously smooth estimation of pupil position.
21. Eye-controlling focus method according to claim 12, it is characterised in that：On-line proving in step 3, Yong Huning
Depending at least 1 point in imaging display device, drift angle of the demarcation sight with eyeball optical axis.
22. Eye-controlling focus method according to claim 12, it is characterised in that：Real-time visual line measurement in step 4, lead to
Introducing level crossing reflective relation is crossed, compared with the on-line proving result in step 3, obtains the direction of current gaze.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610696362.8A CN107765840A (en) | 2016-08-22 | 2016-08-22 | A kind of Eye-controlling focus method equipment of the general headset equipment based on binocular measurement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610696362.8A CN107765840A (en) | 2016-08-22 | 2016-08-22 | A kind of Eye-controlling focus method equipment of the general headset equipment based on binocular measurement |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107765840A true CN107765840A (en) | 2018-03-06 |
Family
ID=61263316
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610696362.8A Pending CN107765840A (en) | 2016-08-22 | 2016-08-22 | A kind of Eye-controlling focus method equipment of the general headset equipment based on binocular measurement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107765840A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109725714A (en) * | 2018-11-14 | 2019-05-07 | 北京七鑫易维信息技术有限公司 | Sight determines method, apparatus, system and wear-type eye movement equipment |
| CN109990756A (en) * | 2019-04-11 | 2019-07-09 | 武汉纺织大学 | A kind of binocular distance measuring method and system |
| CN114022946A (en) * | 2022-01-06 | 2022-02-08 | 深圳佑驾创新科技有限公司 | Sight line measuring method and device based on binocular camera |
| CN114688987A (en) * | 2022-03-29 | 2022-07-01 | 北华航天工业学院 | In-situ measurement device and method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101716068A (en) * | 2009-11-13 | 2010-06-02 | 深圳大学 | Device and method for objectively testing stereoscopic vision |
| CN101901485A (en) * | 2010-08-11 | 2010-12-01 | 华中科技大学 | 3D free head moving type gaze tracking system |
| CN103593051A (en) * | 2013-11-11 | 2014-02-19 | 百度在线网络技术（北京）有限公司 | Head-mounted type display equipment |
| US20150323990A1 (en) * | 2010-07-23 | 2015-11-12 | Telepatheye Inc. | Eye-wearable device user interface and method |
| CN105718046A (en) * | 2014-12-23 | 2016-06-29 | 联发科技股份有限公司 | Head-Mount Display for Eye Tracking based on Mobile Device |
-
2016
- 2016-08-22 CN CN201610696362.8A patent/CN107765840A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101716068A (en) * | 2009-11-13 | 2010-06-02 | 深圳大学 | Device and method for objectively testing stereoscopic vision |
| US20150323990A1 (en) * | 2010-07-23 | 2015-11-12 | Telepatheye Inc. | Eye-wearable device user interface and method |
| CN101901485A (en) * | 2010-08-11 | 2010-12-01 | 华中科技大学 | 3D free head moving type gaze tracking system |
| CN103593051A (en) * | 2013-11-11 | 2014-02-19 | 百度在线网络技术（北京）有限公司 | Head-mounted type display equipment |
| CN105718046A (en) * | 2014-12-23 | 2016-06-29 | 联发科技股份有限公司 | Head-Mount Display for Eye Tracking based on Mobile Device |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109725714A (en) * | 2018-11-14 | 2019-05-07 | 北京七鑫易维信息技术有限公司 | Sight determines method, apparatus, system and wear-type eye movement equipment |
| CN109725714B (en) * | 2018-11-14 | 2022-06-14 | 北京七鑫易维信息技术有限公司 | Sight line determining method, device and system and head-mounted eye movement equipment |
| CN109990756A (en) * | 2019-04-11 | 2019-07-09 | 武汉纺织大学 | A kind of binocular distance measuring method and system |
| CN109990756B (en) * | 2019-04-11 | 2021-07-27 | 武汉纺织大学 | Binocular ranging method and system |
| CN114022946A (en) * | 2022-01-06 | 2022-02-08 | 深圳佑驾创新科技有限公司 | Sight line measuring method and device based on binocular camera |
| CN114688987A (en) * | 2022-03-29 | 2022-07-01 | 北华航天工业学院 | In-situ measurement device and method |
| CN114688987B (en) * | 2022-03-29 | 2023-10-20 | 北华航天工业学院 | In-situ measurement device and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6902075B2 (en) | Line-of-sight tracking using structured light | |
| CN106168853B (en) | A kind of free space wear-type gaze tracking system | |
| Plopski et al. | Corneal-imaging calibration for optical see-through head-mounted displays | |
| CN105094337B (en) | A kind of three-dimensional gaze estimation method based on iris and pupil | |
| Shih et al. | A calibration-free gaze tracking technique | |
| Nishino et al. | The world in an eye [eye image interpretation] | |
| US20140218281A1 (en) | Systems and methods for eye gaze determination | |
| TWI486631B (en) | Head mounted display and control method thereof | |
| US20160173864A1 (en) | Pickup of objects in three-dimensional display | |
| CN106056092A (en) | Gaze estimation method for head-mounted device based on iris and pupil | |
| JP2010259605A (en) | Visual line measuring device and visual line measuring program | |
| CN108354585B (en) | Computer-implemented method for detecting corneal vertex | |
| US20090214086A1 (en) | Vision Correction Aid System | |
| CN107765840A (en) | A kind of Eye-controlling focus method equipment of the general headset equipment based on binocular measurement | |
| CN108259887A (en) | Watch point calibration method and device, blinkpunkt scaling method and device attentively | |
| JP2018099174A (en) | Pupil detector and pupil detection method | |
| CN206696529U (en) | Imaing projector and its pupil position follow-up mechanism with pupil tracing function | |
| CN114973392A (en) | Human eye motion tracking system and method | |
| JPH0351407B2 (en) | ||
| CN108446011A (en) | A kind of medical householder method and equipment based on augmented reality | |
| CN108985291A (en) | A kind of eyes tracing system based on single camera | |
| JP2022133133A (en) | Generation device, generation method, system, and program | |
| US20220395176A1 (en) | System and method for digital optician measurements | |
| CN114004880B (en) | Point cloud and strong reflection target real-time positioning method of binocular camera | |
| JP3711053B2 (en) | Line-of-sight measurement device and method, line-of-sight measurement program, and recording medium recording the program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180306 |
|
| WD01 | Invention patent application deemed withdrawn after publication |