CN107102736A - The method for realizing augmented reality - Google Patents
The method for realizing augmented reality Download PDFInfo
- Publication number
- CN107102736A CN107102736A CN201710277012.2A CN201710277012A CN107102736A CN 107102736 A CN107102736 A CN 107102736A CN 201710277012 A CN201710277012 A CN 201710277012A CN 107102736 A CN107102736 A CN 107102736A
- Authority
- CN
- China
- Prior art keywords
- projector
- projection surface
- augmented reality
- image
- external equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B21/00—Projectors or projection-type viewers; Accessories therefor
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The invention provides a kind of method for realizing augmented reality, comprise the following steps：The changeless projector of relative position and external equipment are provided；Obtain the image on the projection surface of projector；Real-world object in described image is extracted；The real-world object is identified；Result to the identification carries out interactive processing, and generates an image to be projected according to the result of interactive processing；The image to be projected is adjusted according to the spatial relation between the projector and the projection surface, and the image transmitting to be projected after adjustment is projected on projection surface to the projector.The method that what the present invention was provided realize augmented reality, so that common projector can also have the function of augmented reality, and without external computer, it is not necessary to complicated and expensive motion capture system, the cost for realizing augmented reality is reduced, Consumer's Experience is greatly improved.
Description
Technical field
The present invention relates to technical field of image processing, more particularly to a kind of method for realizing augmented reality.
Background technology
Projector, is called projecting apparatus, be it is a kind of can be by the equipment on image or VIDEO PROJECTION to curtain, its extensive use
In family, office, school and various public places of entertainment.Existing projector is divided into common projector and intelligent projector, its area
Not with whether carrying the intelligent operating system opened to user.Augmented reality, augmented reality, it is a kind of by true generation
Boundary's information and the integrated new technology of virtual world information " seamless ", be script in the certain time spatial dimension of real world
It is difficult to the entity information (visual information, sound, taste, tactile etc.) experienced, by science and technology such as computers, after analog simulation
It is superimposed again, virtual Information application to real world is perceived by human sensory, so as to reaches the sense organ body of exceeding reality
Test.It is a kind of brand-new human-computer interaction technology, using this technology, can simulate real live landscape, user is not only
The verisimilitude of " on the spot in person " that is undergone in the objective physical world can be experienced by virtual reality system, and can
Break through space, time and other objective limitations, experience in real world can not personal experience experience.
But, either common projector or intelligent projector, do not possess augmented reality function substantially.At present, have
The optical projection system of standby augmented reality function only has a small amount of application in research laboratory and military training field, and those systems are all valencys
Lattice are extremely expensive, the professional augmented reality system of structure extremely bulky complex, do not possess extensive civilian feasibility.Though
So there are indivedual manufacturers of projector to have been developed that interaction function, but be all based on the interaction mode of infrared track touch-control, with very big
Limitation, can not all be used in many occasions, and these projecting function components be typically all combined with projector,
So that the substantial appreciation of prices of projector, to obtain higher commercial interest.Therefore, how to cause common projector that there is augmented reality
Function, strengthens human-computer interaction, improves the use enjoyment of people, is technical problem urgently to be resolved hurrily at present.
The content of the invention
The present invention provides a kind of method for realizing augmented reality, and augmented reality work(can not be realized to solve common projector
The problem of energy, to strengthen human-computer interaction, improve the use enjoyment of people.
In order to solve the above problems, the invention provides a kind of method for realizing augmented reality, comprise the following steps：There is provided
The changeless projector of relative position and external equipment；Obtain the image on the projection surface of projector；To in described image
Real-world object is extracted；The real-world object is identified；Result to the identification carries out interactive processing, and according to mutual
The result of dynamic processing generates an image to be projected；According to the spatial relation pair between the projector and the projection surface
The image to be projected is adjusted, and the image transmitting to be projected after adjustment is projected into projection to the projector
On face.
It is preferred that, also comprise the following steps after the offer changeless projector of relative position and external equipment：
The external equipment exports the first reference picture to the projector, and is projected on projection surface；The external equipment obtains institute
State the first reference picture shown on projection surface；According to being calculated the first reference picture shown on the projection surface of acquisition
Mutual spatial relation and geometric projection relation between projector, the external equipment, the projection surface three.
It is preferred that, the changeless projector of relative position and the specific steps of external equipment of providing includes：Using
One connecting elements connects the projector and external equipment；The external equipment transmits the second reference picture to the projector,
And project on projection surface；According to position of second reference picture on projection surface, connecting elements is adjusted so that the projector
Difference with the angle of external equipment and the off-axis angle of projector is in a preset range.
It is preferred that, also comprise the following steps before the image on the projection surface for obtaining projector：One infrared benefit is provided
Light lamp, an infrared switch；Open the infrared light compensating lamp and infrared switch.
It is preferred that, also comprise the following steps before the image on the projection surface for obtaining projector：Obtain real-world object
Various states image；Deep learning is carried out to the various states image；According to learning outcome rock mechanism identification database,
The pattern recognition data base is used to real-world object is identified.
It is preferred that, also comprise the following steps after the offer changeless projector of relative position and external equipment：
The external equipment exports the 3rd reference picture to the projector, and is projected on projection surface；Acquisition is shown in the projection
The 3rd reference picture on face；The natural light field parameters in place where obtaining projector；Show according on the projection surface of acquisition
The 3rd reference picture and the natural light field parameters shown, calculates projective parameter；The projection is adjusted according to the projective parameter
The inner parameter of machine.
It is preferred that, the shape of the projection surface is tabular, spherical, curved or polyhedron.
It is preferred that, the material of the projection surface is light tight material, the semi-transparent material of energy scatter projection light or can dissipated
Penetrate the high light-transmitting materials of projection ray.
It is preferred that, the angle α between the projection ray of the projection surface and the projector meets 0 ° of 180 ° of ＜ α ＜.
It is preferred that, the method for realizing augmented reality also comprises the following steps：The external equipment sends control instruction
To the projector；The projector receives the control instruction, and performs corresponding operation according to the control instruction.
The method that what the present invention was provided realize augmented reality so that common projector can also have the work(of augmented reality
Can, after being optimized by using embedded system, key algorithm and system architecture, it is not necessary to external computer again, it is not necessary to multiple
Miscellaneous and expensive motion capture system, user's operating procedure is simple, reduces the cost for realizing augmented reality, enhances man-machine mutual
It is dynamic, and the augmented reality device volume is small, low in energy consumption, greatly improves Consumer's Experience.
Brief description of the drawings
Accompanying drawing 1 is the method flow diagram for realizing augmented reality of the specific embodiment of the invention；
Accompanying drawing 2 is the structural representation of the augmented reality go of the embodiment of the present invention 1；
Accompanying drawing 3 is the structural representation of the augmented reality freely falling body elastic pellet of the embodiment of the present invention 2；
Accompanying drawing 4 is the structural representation of the augmented reality four fundamental rules hybrid operation of the embodiment of the present invention 3.
Embodiment
The embodiment of the method for realizing augmented reality provided below in conjunction with the accompanying drawings the present invention elaborates.
The invention provides a kind of method for realizing augmented reality, accompanying drawing 1 is that the realization of the specific embodiment of the invention increases
The method flow diagram of strong reality.As shown in figure 1, the method for the present invention for realizing augmented reality specifically includes following step
Suddenly：
There is provided the changeless projector of relative position and external equipment by step S11.Wherein, the external equipment volume
It is compact, it is easy to carry.The external equipment include taking module, communication module, vision processing module, pattern recognition module, mutually
In dynamic functional module, pervasive projection module, wireless same panel module, projector's Premium Features control module and intelligent operating platform
It is one or more of.Wherein, the taking module includes at least one camera；The communication module is mainly used in the outside and set
Standby internal or between the external equipment and the projector data transfer；The vision processing module is regarded including a computer
Feel processing routine, the image for being shot to the taking module carries out computer vision processing；The pattern recognition module bag
A pattern recognition program is included, is mainly used in carrying out pattern-recognition to image；The interaction function module is included at an interaction function
Reason program, is mainly used in carrying out interaction function processing according to the result of pattern-recognition；It is pervasive that the pervasive projection module includes one
Projection program, for being adjusted to image to be projected；It is described it is wireless include with panel module it is one wireless with screen program, for
Image transmitting is carried out between the external equipment and the projector；Projector's Premium Features control module and intelligent operation
Platform is used to send projector control instruction, and control projector performs corresponding operation.In this embodiment, can be with
The projector is fixedly connected with external equipment using a connecting elements, the two ends of various connection cables respectively with positioned at described
On interface, externally-located equipment on projector main frame interface connection, wherein, the connection cables can be HDMI cable,
One or more in ethernet line or USB cable.After the projector and the external equipment are powered, both internal softwares
Operation is automatically begun to, self-inspection is carried out by USB cable, ethernet line or HDMI cable.
In order to obtain preferably Projection Display effect and improve the precision of interactive identification, it is preferred that the offer phase
Specific steps to the changeless projector in position and external equipment include：
(1) projector and external equipment are connected using a connecting elements.Wherein, the two ends of the connecting elements and institute
It is all to be flexibly connected to state projector and external equipment, i.e., user can according to actual needs translation or rotate the projector and/
Or the external equipment, to adjust the relative position relation between the projector and the external equipment.
(2) external equipment transmits the second reference picture to projector, and projects on projection surface.The external equipment
Second reference picture is exported to the projector by HDMI cable, and controls the projector to throw second reference picture
It is incident upon on the projection surface.Wherein, second reference picture is that have particular meaning in computer graphics art field
Picture, those skilled in the art can select the content of the second reference picture according to actual needs, and present embodiment is to this
It is not construed as limiting.
(3) position according to the second reference picture on projection surface, adjusts connecting elements so that the projector and outside
The difference of the angle of equipment and the off-axis angle of projector is in a preset range.It is artificial to be thrown according to the second reference picture described
The position penetrated on face, adjusts the connecting elements, for example, rotates the projector and/or the external equipment so that the throwing
The difference of the off-axis angle of the angle and projector of shadow machine and the external equipment is in a preset range.Wherein, it is described default
Scope those skilled in the art can be set according to actual needs, as long as ensureing the projector and the external equipment
Angle is roughly the same with the off-axis angle of projector, that is to say, that only at the visual angle of the projected picture and camera
Difference than it is larger when, such as the angle formed between the visual angle of described projected picture and camera is more than 10 degree, just needs
Adjustment, and also only need substantially to adjust, such as described preset range can be 0-10 degree.So, the outside is greatly reduced
Equipment and the assembly difficulty of the projector.
Step S12, obtains the image on the projection surface of projector.Due to the general work in the environment of dark of projector
Make, in order to also obtain the image on the projection screen in the environment of dark, it is preferred that the acquisition is thrown
Also comprise the following steps before image on the projection surface of shadow machine：One infrared light compensating lamp, an infrared switch are provided；Open described
Infrared light compensating lamp and infrared switch.Taking module on the external equipment includes a camera, the general work of camera
Make under natural light state, during the augmented reality is realized, the camera continuously acquires the projection surface of projector
On image.In the case where light is brighter, the infrared switch and the infrared light compensating lamp are closed, and the camera is caught
Picture under natural light；In the case of dark, the infrared light compensating lamp and the infrared switch are opened, and described red
Outer switch only allows infrared light.So, also can be to the image on the projection surface in the case of dark
Accurately caught.Wherein, the camera described in present embodiment is common monocular or binocular camera.This
It is the important difference with other projection interactive devices.Existing projection interactive device has to special camera, or necessary
Using special auxiliary equipment (such as infrared pen, depth perception sensor), the normal work of mnemonic mark (such as Quick Response Code) ability
Make.And the augmented reality device of present embodiment greatly simplify system architecture, manufacturing cost is reduced.
Step S13, is extracted to the real-world object in described image.In order to improve the degree of accuracy of image information extraction,
Program inside the external equipment can also be adjusted to the taking module automatically, the taking module is obtained
To the higher image of definition.In order to improve the degree of accuracy of image information extraction, the real-world object in described image is carried
The specific steps taken include：Obtain the spatial relation between the external equipment and the projection surface；To in described image
Real-world object extracted；Entered according to the spatial relation between the external equipment and the projection surface to extracting result
Row adjustment.
In order to obtain mutual alignment relation and the throwing between the external equipment, the projector, the projection surface three
Shadow relation, to improve the accuracy of follow-up mode identification, it is preferred that the changeless projector of offer relative position and outer
Also comprise the following steps after portion's equipment：The external equipment exports the first reference picture to the projector, and is projected to
Penetrate on face；The external equipment obtains the first reference picture shown on the projection surface；According on the projection surface of acquisition
First reference picture of display calculates the mutual space bit between the projector, the external equipment, the projection surface three
Put relation and geometric projection relation.Wherein, first reference picture is that have special contain in computer graphics art field
The picture of justice, those skilled in the art can select the content of the first reference picture, present embodiment according to actual needs
This is not construed as limiting.The external equipment is handled the first reference picture on the projection surface that gets, according to place
The result of reason calculates accurate spatial relation and geometry between the projector, the external equipment, the projection surface
Projection relation, and be converted into projecting control parameter.Wherein, the external equipment is to the first ginseng on the projection surface that gets
Examine the process that picture is handled, mainly the first reference picture of acquisition is carried out computer vision processing, pattern-recognition and
The adjustment of recognition result.The step for passing through, the projector is formed a spatial relationship with the external equipment and fixed
And it is overall known to parameter, the projector is realized together with the external equipment firm " binding ", it is ensured that follow-up
Be not in distortion in projection and augmented reality interactive process.
In order to further improve the degree of accuracy of image recognition during follow-up augmented reality, it is preferred that the external equipment
The first reference picture is exported to the projector and before being projected to the step on projection surface also to comprise the following steps：The outside
Equipment catches the second reference picture being projected on projection surface by taking module thereon；Caught according to the taking module
Second reference picture extracts the projection surface relative to the projector and/or the outside position set.So, it is that next step is true
The spatial relation determined between projector, external equipment and projection surface three is ready, and has been reached and has more accurately been known three
The purpose of position relationship between person.
Step S14, the real-world object is identified.The real-world object is accurately identified in order to realize, preferably
, also comprise the following steps before the image on the projection surface for obtaining projector：Obtain the various states figure of real-world object
Picture；Deep learning is carried out to the various states image；According to learning outcome rock mechanism identification database, the pattern-recognition
Database is used to real-world object is identified.The pattern recognition program carried in the external equipment is beforehand through accurate identification
Algorithm and/or fuzzy recognition algorithm study object identification.I.e. before the real-world object in described image is accurately identified,
Need in advance to enter the image of all kinds of authentic items such as chess piece, card, operation panel and projection with a variety of intelligent algorithms
Row deep learning so that the external equipment can accurately recognize the various forms and real-world object of real-world object with it is virtual
The differentiation of image, removes all kinds of obstacles simultaneously.Specifically, the taking module that the external equipment is first passed through thereon obtains described
The various states image of real-world object, and deep learning is carried out to the status image, and mould is formed according to the learning outcome
Formula identification database.The various states image of the real-world object includes the real-world object in various positions, posture, illumination bar
Image under part, if the real-world object has the change in terms of color, size, combination, the status image is also wrapped
Include image of the real-world object under various change state.The deep learning algorithm of the external equipment mainly has two major classes：
Accurate recognizer and fuzzy recognition algorithm.Accurate recognizer is used to accurately judge target, such as is card " 6 " or card
Board " 9 " or card " b ", this needs to carry out deep learning to a large amount of models in advance；Fuzzy recognition algorithm is then used for fuzzy judgement
Target, such as target are dark strip objects that can be interactive, light circular object that still cannot be interactive.Fuzzy diagnosis
The core of algorithm be its efficiently, have classification targetedly algorithm, such as color, for profile, and set with the outside
The standby interaction function processing routine carried has substantial connection to the demand of identification.In this embodiment, two kinds of algorithms can
Concurrently to run, while two recognition results (i.e. fuzzy diagnosis result and accurate recognition result) are exported, and the interaction
Function treatment program can select to use wherein one or two result as needed.In order to ensure the general of deep learning result
Property, reject error caused by hardware or environment, it is preferred to use the respective independent carry out deep learning of many set learning systems.In addition,
When being connected with each other in order to ensure the projector and the external equipment, institute also can be normally recognized under uncontrollable posture state
Real-world object is stated, the deep learning also includes the deep learning to different visual angles under several given pose state.Above-mentioned
Deep learning process is only carried out in the research and development of the external equipment and test phase, the generally applicable pattern-recognition number after volume production
According to storehouse, it is not necessary to individually learnt again.
Real-world object described in present embodiment is not limited only to the object of volume, such as chess piece, building block, card
Deng or the real-world object without volume such as scribble, person's handwriting, color lump, printed matter.Enhancing is realized described in present embodiment
The method of reality can be simultaneously to having volume and real-world object without volume to be extracted, recognized.This and existing image procossing
Technology is essentially different：Existing identification technology can only recognize and track infrared light spot, for there is the object of volume can not
Extract, recognize, can not extract, recognize for objects without volume such as non-infrared person's handwriting, scribbles.And present embodiment
The method of described augmented reality can be extracted to any real-world object, recognized, be expanded described in present embodiment
Realize the application field of the method for augmented reality.
Step S15, the result to the identification carries out interactive processing, and to be projected according to the generation one of the result of interactive processing
Image.
Step S16, according to the spatial relation between the projector and the projection surface to the figure to be projected
As being adjusted, and the image transmitting to be projected after adjustment is projected on projection surface to the projector.In order to accurate
, can also be according to pre- by the precision image to be projected for being shown on the projection surface, improving augmented reality after interactive processing
Wait to throw described in spatial relation and/or the adjustment of geometric projection relation between the projector first obtained and the projection surface
Shape, brightness of the image of shadow etc..The image to be projected can be transmitted to the projector by HDMI cable, be incident upon
On projection surface.
In order to improve the degree of accuracy of image recognition, and obtain preferably Projection Display effect, it is preferred that the outside is set
For by also comprising the following steps before camera collection image thereon：
(a) external equipment exports the 3rd reference picture to the projector, and is projected on projection surface.Wherein, institute
It is the picture with light and shade change, tone variations to state the 3rd reference picture.
(b) the 3rd reference picture being shown on the projection surface is obtained.
(c) the natural light field parameters in place where obtaining projector.Wherein, the projection light field produced with the projector is not
Together, the natural light field includes the synthesis light field that the Nature light source and artificial light source are produced.
(d) according to the 3rd reference picture and the natural light field parameters shown on the projection surface of acquisition, calculate and throw
Shadow parameter.Computer vision processing is carried out to the 3rd reference picture shown on the projection surface that captures, and to computer vision
Result after processing is calculated with natural light field parameters, draws the best projection parameter of suitable projection and visual identity.
(e) inner parameter of the projector is adjusted according to the projective parameter.The external equipment passes through data transfer
Passage, according to the obtained projective parameter of calculating, is adjusted to the inner parameter of the projector, sets.Wherein, the projection
The inner parameter of machine includes the one or more in brightness, contrast, colour temperature, resolution ratio.
Moreover, in order that the demand of user can more be adapted to by obtaining drop shadow effect, user can also adjust institute manually
State every inner parameter of projector.In order to save cost, it can also be not provided between the external equipment and the projector
Data transmission channel, the external equipment can be obtained by the way that the 3rd reference picture is identified according to recognition result
One correction parameter, using the projection error of image to be projected described in the correction parameter adjustment, such as projector's brightness is too high,
Then pervasive projection software module reduces picture overall brightness to be projected, conversely, then lifting picture overall brightness.
Present embodiment is not construed as limiting to the concrete shape of projection surface, and those skilled in the art can be according to actual need
Set, in order to obtain more interactive functions, improve the use enjoyment of user, it is preferred that the shape of the projection surface
It is tabular, spherical, curved or polyhedron.In addition, the material of the projection surface is light tight material, energy scatter projection light
Semi-transparent material or can scatter projection light high light-transmitting materials, those skilled in the art can be selected according to actual needs
Select, present embodiment is not construed as limiting to this.Because present embodiment can be to projection surface, projector and external equipment
Between spatial relation carry out Intelligent Recognition and carry out pervasive projection, as long as the display surface of projection surface is not parallel to projection
Light, the method for realizing augmented reality described in present embodiment can be stablized, normal operation, it is preferred that described to throw
The angle α penetrated between the projection ray of face and the projector meets 0 ° of 180 ° of ＜ α ＜.
It is preferred that, the projection surface is true operation plate or pseudo operation plate, wherein, pseudo operation plate includes ground, wall
Face or desktop.Projection surface is present as the bearing apparatus of interactive operation and the projection carrier of projected picture, due to projection
The characteristics of technology, what it was necessarily present, but it is not necessarily real operation panel or virtual operation panel, such as table
Face, metope or ground, so as to expand the application scenario of the method for realizing augmented reality.
In order to strengthen the interest of interaction, it is preferred that the method for realizing augmented reality also comprises the following steps：There is provided
At least one augmented reality accessory；The external equipment recognizes the augmented reality accessory, and carries out interactive processing；According to interaction
Result generation one is image to be projected, and transmits to projector；The projector, will be described according to the projection control parameter
It is image to be projected to be incident upon on projection surface.Wherein, can to include but is not limited to chess piece, card etc. all kinds of for the augmented reality accessory
Physical item, those skilled in the art can be matched according to actual needs.Augmented reality operation accessory almost can be with
It is all objects in nature, but they need to have following characteristics：Need to have under visible or infrared light and be available for knowing
Another characteristic point, such as specific profile, specific color, specific indicia patterns.By using these additional enhancings
Real accessory, interactive limitation can only be carried out using infrared operating pen and simple gesture by having broken away from projector, add use
The enjoyment that family is used.
When the projector is intelligent projector, for the ease of being controlled to the projector, so as to be made us
Satisfied projected picture, it is preferred that the method for realizing augmented reality also comprises the following steps：The external equipment sends control
System is instructed to the projector；The projector receives the control instruction, and performs corresponding behaviour according to the control instruction
Make.It is furthermore preferred that the control instruction includes one kind or several in function computing, interactive processing, projector's inner parameter adjustment
Kind.Wherein, projector's inner parameter adjustment include focusing parameter adjustment, the adjustment of camera lens offset parameter, luminance parameter adjust,
One or more in contrast level parameter adjustment.I.e. described external equipment can shift the hardware device of part or software program
Carried out on to projector, realize sharing out the work and help one another for the projector and the external equipment, improve image processing efficiency, quickening
Response speed during augmented reality, improves Consumer's Experience.Pass through the mutual of the external equipment and the projector
Control, coordinated takes full advantage of both process resources, further increases the treatment effeciency of augmented reality.
The method that what the present invention was provided realize augmented reality so that common projector can also have the work(of augmented reality
Can, after being optimized by using embedded system, key algorithm and system architecture, it is not necessary to external computer again, it is no longer necessary to
External computer, it is not necessary to which complicated and expensive motion capture system, user's operating procedure is simple, reduces and realizes augmented reality
Cost, enhance human-computer interaction, greatly improve Consumer's Experience.
Embodiment 1
Present embodiments provide a kind of augmented reality go.Accompanying drawing 2 is the knot of the augmented reality go of the embodiment of the present invention 1
Structure schematic diagram.As shown in Fig. 2 during the augmented reality go described in the present embodiment is realized, projector is on projection surface
Chessboard 41 and the display information frame 44 on right side are projected out, the display information frame 44 includes prompt message 442, virtual interacting button
441.Wherein, the prompt message 442 include patch mesh during playing Weiqi, allow son, son, black used time, white square used time, in advance
Estimate the one or more in information etc.；The virtual interacting button 441 includes new office, sets, retracts a false move, discs, point out, give up
In one or more.User holds the dark sub (real go chess piece), and augmented reality projection go system is tackled with white chessman, so that
Realize augmented reality go function.
During concrete implementation, projector is projected out chessboard 41 and the display information frame 44 on right side, institute in projection surface
Stating display information frame 44 includes prompt message 442, interactive button 441.Between taking module in the external equipment is begun to not
Disconnected ground is shot to the projection surface, and the image of shooting is carried out into computer vision processing.When user hold the dark son begin after,
The external equipment carries out computer vision processing by the image to acquisition, finds that the image on the projection surface occurs at once
Change, then extracts the part changed from entire image, and according between the external equipment and projection surface
Spatial relation and/or geometric projection relation pair shape are corrected, because due to aerial prospective relation, the shooting
It is most likely not round, but flat that module, which photographed chess piece, and the real-world object in the modified-image after correction finally is carried out into mould
Formula is recognized.The described pair of image changed carries out the process of pattern-recognition, is specifically that the image changed is accurately recognized
(premise accurately recognized is to carry out deep learning to go chess piece in advance), is that user puts down to judge
Black mole, or interference figure picture.If the black mole that user is put, then position of the black mole on chessboard is further accurately recognized
Put.The interaction function processing routine in the present embodiment is specifically go artificial intelligence software's program.The artificial intelligence of go
Energy software program provides the point position that next step is moved according to the recognition result (position of the i.e. described black mole on chessboard), and
The site moved according to the next step generates an image to be projected.The external equipment is thrown according to the projector with described
The spatial relation and geometric projection relation penetrated between face, predeformation is carried out to the image to be projected that obtains, and by after deformation
Image to be projected by data communication interface or it is wireless with screen functional module by the image transmitting to be projected through deformation extremely
The projector, and be incident upon on the projection surface of the projector.Said process goes round and begins again, that is, realizes augmented reality projection
Go function.Moreover, user touches the virtual interacting button 441 with hand, and the taking module 11 photographs user and touched
Touch the image of the virtual interacting button 441；The external equipment is extracted to the human hand in described image；To human hand and its
Touch location is identified；The virtual interacting button 441 touched according to user performs corresponding operation；Generated according to the operation
One image to be projected；Treated according to the spatial relation between the projector and projection surface and geometric projection relation pair
The image of projection is adjusted, and the image transmitting to be projected after adjustment is projected on projection surface to the projector.
Embodiment 2
A kind of augmented reality freely falling body elastic pellet is present embodiments provided, accompanying drawing 3 is the enhancing of the embodiment of the present invention 2
The structural representation of real freely falling body elastic pellet.As shown in figure 3, realizing that the augmented reality described in the present embodiment freely falls
During body elasticity bead, projector is fallen in some beads 52 that projection surface 51 is projected out in the way of freely falling body from top
Under, variously-shaped, color building block 53 can be placed in projection surface by user, if bead 52 is encountered during whereabouts
Building block 53, then can rebound；If running into building block 53 again during bounce-back, then it can rebound again, so circulation, realization has
The spring game of interest.
During concrete implementation, the projector projects the bead of freely falling body in the projection surface, described outer
Taking module in portion's equipment is continual to the projection surface to be shot, and the image of shooting is carried out at computer vision
Reason.When the building block 53 is placed into the projection surface 51 by user, the external equipment is entered by the image to acquisition
The processing of row computer vision, finds that the image on the projection surface is changed, is then extracted from entire image at once
The part changed, and according to the spatial relation between the external equipment and projection surface and/or geometric projection relation
The part that changes is carried out shape correction (because, due to aerial prospective relation, the institute that the taking module is photographed
The shape for stating building block 53 is not its true shape), the real-world object in the modified-image after correction is finally entered into row mode
Identification.Mode identification procedure described in the present embodiment is to carry out fuzzy diagnosis (because the present embodiment to the image changed
In building block not necessarily rectangle or circle, triangle, setting is irregular shape, therefore is not suitable for
Accurate recognizer, can only use fuzzy recognition algorithm), it is the building block that user puts down with judgement, or interference figure picture.It is worth one
It is mentioned that, if the result of fuzzy diagnosis is the figure that " can not judge " or the external equipment can not be shot from the taking module
The part clearly changed is extracted as in, then the external equipment automatically can also open infrared light compensating lamp, and open red
Outer switch, in the case of infrared ray floor light, is shot to the image on projection surface, is extracted and pattern knowledge again
Not.If recognition result is the building block of user side, further accurately recognize it in the position of projection surface and exact extension.This
The external equipment in embodiment stores the position of each building block, scope in software program, and according in physics
The law of conservation of momentum and free fall law control the movement locus of each bead 52.The interaction function in the present embodiment
Pellet free falling rigid collision program during processing routine.If the movement locus of the bead 52 touches the building blocks
Interaction function processing routine inside the edge of block 53, the external equipment just can be at once according to the physical law of Rigid Body Collision
The bead 52 and the track after the building block 53 " collision " are calculated, and is treated according to the Track Pick-up one after " collision "
The image of projection.The external equipment is according to the spatial relation between the projector and the projection surface and geometric projection
Relation, carries out predeformation to the image to be projected, and by the image to be projected after deformation by data communication interface or
It is wirelessly same to shield functional module by the image transmitting to be projected through deformation to the projector, and it is incident upon the throwing of the projector
Penetrate on face.Said process goes round and begins again, and has higher requirement to real-time, thus realizes augmented reality freely falling body
Elastic pellet function.
Embodiment 3
A kind of augmented reality four fundamental rules hybrid operation is present embodiments provided, accompanying drawing 4 is the augmented reality of the embodiment of the present invention 3
The structural representation of four fundamental rules hybrid operation.As shown in figure 4, realizing the augmented reality four fundamental rules hybrid operation described in the present embodiment
During, user is put into the card 62 for being printed on numeral or operator, the augmented reality in the projection surface 61 of the projector
Card 62 described in device Real time identification, and neat four fundamental rules hybrid operation formula is launched in the bottom of the projection surface 61, finally
Result of calculation is projected being printed on behind the card of equal sign.
During concrete implementation, the projector in the present embodiment does not project mutual after being initialized in the projection surface
Motion video, waits user to prevent from being printed on the card of numeral or operator.The taking module of the external equipment is continual to institute
State projection surface to be shot, and the image of shooting is subjected to computer vision processing.When user will be printed on digital or operator
When card is placed in the projection surface, the external equipment carries out computer vision processing by the image to acquisition, at once
It was found that the image on the projection surface is changed, the part changed is then extracted from entire image, and according to
Spatial relation between the external equipment and projection surface the part changed is carried out shape correction (because,
Due to aerial prospective relation, the shape for the card 62 that the taking module 11 is photographed not is its true shape), finally
Real-world object in modified-image after correction is subjected to pattern-recognition.The mode identification procedure is that the image changed is entered
(premise accurately recognized is that the external equipment has all carried out too deep to every card in advance to the accurate identification of row
Degree study), to judge being card 62 that user puts down, or interfering object.If the card 62 that user puts down, then further
The content of position and card of the card in the projection surface is accurately recognized, the content of the card is included in card 62
The numeral or operator of print, and recognition result is subjected to interaction function processing.Interaction function processing journey in the present embodiment
Sequence is four fundamental rules hybrid operation software program.The four fundamental rules hybrid operation software program recognize first the card 62 content whether
It is "=" (equal sign), if it is above will carries out computing by whole four fundamental rules hybrid operation formula, and be generated according to the result after computing
One image to be projected；, whereas if the four fundamental rules hybrid operation software program identifies the content printed in the card 62 not
It is "=" (equal sign), then an image to be projected is generated according to the content of the card identified.The external equipment is according to described
Spatial relation and geometric projection relation between projector and the projection surface, to described to be projected that image is become in advance
Shape, and by the image to be projected after deformation by data communication interface or wirelessly with screen functional module by through the to be projected of deformation
Image transmitting to the projector, and the bottom section being incident upon on the projection surface.Thus achieve augmented reality four fundamental rules
Hybrid operation function.
Described above is only the preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
Member, under the premise without departing from the principles of the invention, can also make some improvements and modifications, these improvements and modifications also should be regarded as
Protection scope of the present invention.
Claims (10)
1. a kind of method for realizing augmented reality, it is characterised in that comprise the following steps：
The changeless projector of relative position and external equipment are provided；
Obtain the image on the projection surface of projector；
Real-world object in described image is extracted；
The real-world object is identified；
Result to the identification carries out interactive processing, and generates an image to be projected according to the result of interactive processing；
The image to be projected is adjusted according to the spatial relation between the projector and the projection surface, and
By the image transmitting to be projected after adjustment to the projector, it is projected on projection surface.
2. the method for augmented reality according to claim 1, it is characterised in that the offer relative position is changeless
Also comprise the following steps after projector and external equipment：
The external equipment exports the first reference picture to the projector, and is projected on projection surface；
The external equipment obtains the first reference picture shown on the projection surface；
The projector is calculated according to the first reference picture for being shown on the projection surface of acquisition, it is the external equipment, described
Mutual spatial relation and geometric projection relation between projection surface three.
3. the method according to claim 1 for realizing augmented reality, it is characterised in that the offer relative position is fixed not
The projector of change and the specific steps of external equipment include：
The projector and external equipment are connected using a connecting elements；
The external equipment transmits the second reference picture to the projector, and projects on projection surface；
According to position of second reference picture on projection surface, connecting elements is adjusted so that the projector and external equipment
The difference of the off-axis angle of angle and projector is in a preset range.
4. the method according to claim 1 for realizing augmented reality, it is characterised in that the projection surface of the acquisition projector
On image before also comprise the following steps：
One infrared light compensating lamp, an infrared switch are provided；
Open the infrared light compensating lamp and infrared switch.
5. the method according to claim 1 for realizing augmented reality, it is characterised in that the projection surface of the acquisition projector
On image before also comprise the following steps：
Obtain the various states image of real-world object；
Deep learning is carried out to the various states image；
According to learning outcome rock mechanism identification database, the pattern recognition data base is used to real-world object is identified.
6. the method according to claim 1 for realizing augmented reality, it is characterised in that the offer relative position is fixed not
Also comprise the following steps after the projector of change and external equipment：
The external equipment exports the 3rd reference picture to the projector, and is projected on projection surface；
Obtain the 3rd reference picture being shown on the projection surface；
The natural light field parameters in place where obtaining projector；
According to the 3rd reference picture and the natural light field parameters shown on the projection surface of acquisition, projective parameter is calculated；
The inner parameter of the projector is adjusted according to the projective parameter.
7. the method according to claim 1 for realizing augmented reality, it is characterised in that the shape of the projection surface is flat board
Shape, spherical, curved or polyhedron.
8. the method according to claim 1 for realizing augmented reality, it is characterised in that the material of the projection surface is impermeable
The high light-transmitting materials of finish matter, the semi-transparent material of energy scatter projection light or energy scatter projection light.
9. the method according to claim 1 for realizing augmented reality, it is characterised in that the projection surface and the projector
Projection ray between angle α meet 0 ° of 180 ° of ＜ α ＜.
10. the method according to claim 1 for realizing augmented reality, it is characterised in that the side for realizing augmented reality
Method also comprises the following steps：
The external equipment sends control instruction to the projector；
The projector receives the control instruction, and performs corresponding operation according to the control instruction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710277012.2A CN107102736A (en) | 2017-04-25 | 2017-04-25 | The method for realizing augmented reality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710277012.2A CN107102736A (en) | 2017-04-25 | 2017-04-25 | The method for realizing augmented reality |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107102736A true CN107102736A (en) | 2017-08-29 |
Family
ID=59657744
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710277012.2A Pending CN107102736A (en) | 2017-04-25 | 2017-04-25 | The method for realizing augmented reality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107102736A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108735052A (en) * | 2018-05-09 | 2018-11-02 | 北京航空航天大学青岛研究院 | A kind of augmented reality experiment with falling objects method based on SLAM |
| CN108805917A (en) * | 2018-05-25 | 2018-11-13 | 网易（杭州）网络有限公司 | Sterically defined method, medium, device and computing device |
| CN109587188A (en) * | 2017-09-28 | 2019-04-05 | 阿里巴巴集团控股有限公司 | Determine the method, apparatus and electronic equipment of relative positional relationship between terminal device |
| CN110109538A (en) * | 2019-04-01 | 2019-08-09 | 广东芬莱信息科技有限公司 | A kind of scene reminding method, device and storage medium based on virtual reality |
| CN110263885A (en) * | 2019-05-23 | 2019-09-20 | 深圳光维科技有限公司 | Check method, apparatus, terminal device and the storage medium of projector's failure |
| CN110392251A (en) * | 2018-04-18 | 2019-10-29 | 广景视睿科技（深圳）有限公司 | A kind of dynamic projection method and system based on virtual reality |
| WO2020155995A1 (en) * | 2019-01-31 | 2020-08-06 | 广景视睿科技（深圳）有限公司 | Projection method and projection device |
| WO2021164387A1 (en) * | 2020-02-20 | 2021-08-26 | 华为技术有限公司 | Early warning method and apparatus for target object, and electronic device |
| CN114563900A (en) * | 2022-02-23 | 2022-05-31 | 宫智勇 | Lens tracking type store projection advertisement device with infrared body temperature sensing probe |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140160162A1 (en) * | 2012-12-12 | 2014-06-12 | Dhanushan Balachandreswaran | Surface projection device for augmented reality |
| CN104656890A (en) * | 2014-12-10 | 2015-05-27 | 杭州凌手科技有限公司 | Virtual realistic intelligent projection gesture interaction all-in-one machine |
| CN105451007A (en) * | 2015-11-16 | 2016-03-30 | 上海尚镜信息科技有限公司 | Interactive projection system and method |
| CN106131536A (en) * | 2016-08-15 | 2016-11-16 | 万象三维视觉科技（北京）有限公司 | A kind of bore hole 3D augmented reality interactive exhibition system and methods of exhibiting thereof |
-
2017
- 2017-04-25 CN CN201710277012.2A patent/CN107102736A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140160162A1 (en) * | 2012-12-12 | 2014-06-12 | Dhanushan Balachandreswaran | Surface projection device for augmented reality |
| CN104656890A (en) * | 2014-12-10 | 2015-05-27 | 杭州凌手科技有限公司 | Virtual realistic intelligent projection gesture interaction all-in-one machine |
| CN105451007A (en) * | 2015-11-16 | 2016-03-30 | 上海尚镜信息科技有限公司 | Interactive projection system and method |
| CN106131536A (en) * | 2016-08-15 | 2016-11-16 | 万象三维视觉科技（北京）有限公司 | A kind of bore hole 3D augmented reality interactive exhibition system and methods of exhibiting thereof |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109587188B (en) * | 2017-09-28 | 2021-10-22 | 阿里巴巴集团控股有限公司 | Method and device for determining relative position relationship between terminal devices and electronic device |
| CN109587188A (en) * | 2017-09-28 | 2019-04-05 | 阿里巴巴集团控股有限公司 | Determine the method, apparatus and electronic equipment of relative positional relationship between terminal device |
| CN110392251A (en) * | 2018-04-18 | 2019-10-29 | 广景视睿科技（深圳）有限公司 | A kind of dynamic projection method and system based on virtual reality |
| CN110392251B (en) * | 2018-04-18 | 2021-07-16 | 广景视睿科技（深圳）有限公司 | Dynamic projection method and system based on virtual reality |
| CN108735052A (en) * | 2018-05-09 | 2018-11-02 | 北京航空航天大学青岛研究院 | A kind of augmented reality experiment with falling objects method based on SLAM |
| CN108805917A (en) * | 2018-05-25 | 2018-11-13 | 网易（杭州）网络有限公司 | Sterically defined method, medium, device and computing device |
| CN108805917B (en) * | 2018-05-25 | 2021-02-23 | 杭州易现先进科技有限公司 | Method, medium, apparatus and computing device for spatial localization |
| US11886107B2 (en) | 2019-01-31 | 2024-01-30 | Iview Displays (Shenzhen) Company Ltd. | Projection method and projection device |
| WO2020155995A1 (en) * | 2019-01-31 | 2020-08-06 | 广景视睿科技（深圳）有限公司 | Projection method and projection device |
| CN110109538A (en) * | 2019-04-01 | 2019-08-09 | 广东芬莱信息科技有限公司 | A kind of scene reminding method, device and storage medium based on virtual reality |
| CN110263885B (en) * | 2019-05-23 | 2022-08-05 | 深圳光维科技有限公司 | Method and device for checking projector fault, terminal equipment and storage medium |
| CN110263885A (en) * | 2019-05-23 | 2019-09-20 | 深圳光维科技有限公司 | Check method, apparatus, terminal device and the storage medium of projector's failure |
| WO2021164387A1 (en) * | 2020-02-20 | 2021-08-26 | 华为技术有限公司 | Early warning method and apparatus for target object, and electronic device |
| CN114563900A (en) * | 2022-02-23 | 2022-05-31 | 宫智勇 | Lens tracking type store projection advertisement device with infrared body temperature sensing probe |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107102736A (en) | The method for realizing augmented reality | |
| US10990189B2 (en) | Processing of gesture-based user interaction using volumetric zones | |
| US20230093612A1 (en) | Touchless photo capture in response to detected hand gestures | |
| US11762478B2 (en) | Virtual or augmediated topological sculpting, manipulation, creation, or interaction with devices, objects, materials, or other entities | |
| CN110045816A (en) | Near-eye display and system | |
| CN106201173B (en) | A kind of interaction control method and system of user's interactive icons based on projection | |
| CN106062862A (en) | System and method for immersive and interactive multimedia generation | |
| CN106030495A (en) | Multi-modal gesture based interactive system and method using one single sensing system | |
| CN114489347A (en) | Dynamic sand table and demonstration method and system thereof | |
| CN107122731A (en) | Augmented reality device | |
| CN109642788A (en) | Information processing system, information processing method and program | |
| KR102173608B1 (en) | System and method for controlling gesture based light dimming effect using natural user interface | |
| CN107015650A (en) | Alternative projection method, device and system | |
| US11863963B2 (en) | Augmented reality spatial audio experience | |
| Candela et al. | HumanTop: A multi-object tracking tabletop | |
| JP2011092657A (en) | Game system for performing operation by using a plurality of light sources | |
| Park et al. | Interactive display of image details using a camera-coupled mobile projector | |
| CN105636656B (en) | Light modulator for moving simulated environment | |
| Voglhuber | Hand simulation for virtual climbing | |
| CN102375541B (en) | User movement is converted into the response of multiple object | |
| CN113918014A (en) | Virtual object control method, device, equipment and storage medium | |
| CN113870438A (en) | Virtual object control method, device, equipment and storage medium | |
| Gropengießer et al. | A gesture controlled game for a media facade | |
| Oliveira et al. | Interaction in an immersive multi-projection system based on light wand like devices | |
| TWM453888U (en) | Augmented reality projection apparatus using wireless motion sensing device for determining posture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170829 |