Description
We discussed the current shape of the API at the Immersive Web Working Group Face to Face.
There’s a growing recognition in the working group that a lack of camera access has held back adoption of WebXR compared to alternatives like getUserMedia. The ironic effect of having an artificially limited api is that it pushes people toward less privacy sensitive alternatives. The goal should be to have user consent that is potentially improved from existing flows (whatever that entails) but is powerful enough to empower the gamut of valuable developer use cases.
In order to make the current proposal fully compatible with headsets, the following changes might be needed
- frames are not coupled to XR frames, but come on a callback or some other mechanism
- frames are timestamped
- camera to viewer transform
- field of view for camera is provided
Here are some examples of reasons developers choose use alternatives to WebXR because this API is lacking:
- Custom Visual Effects: https://diamonds.samsmithworld.com/ renders diamonds with custom refraction of the camera feed.
- High quality reflections: https://www.8thwall.com/alivenow/freefire renders a metallic paint car with the users' environment reflected.
- Responsive Scale: https://www.8thwall.com/beemwebar/hearst-philips shows a hologram discussing body positivity that can be viewed comfortably at room scale or table scale.
- Photo Sharing: https://www.8thwall.com/aircards/dune-avatar enables photo sharing of a custom Dune avatar, with a Dune watermark embedded in the photo.
- Video Sharing: https://www.8thwall.com/alivenow/cadbury-fc-webar has users record and share a video with audio and embedded images painting their face with colors of their favorite soccer teams.
- Image Target Effects: https://www.8thwall.com/toolofnorthamerica/pizza-hut-pacman is a Pac Man game on top of a pizza box.
- Curved Image Target Effects: https://www.8thwall.com/tactic/white-castle animates comic on White Castle cups.
- Face Effects: https://www.jlab.com/pages/fitting-room lets users try on various audio accessories before purchasing.
- QR Code Scanning: https://8th.io/scan is a web assembly QR code scanner.
- Custom TensorFlow Models: https://www.campaignasia.com/article/heineken-asks-singapore-to-race-an-f1-car-in-ar/453689 detects when two Heineken bottles "cheers".
- Hand Effects: https://blog.tensorflow.org/2021/11/3D-handpose.html allows pages to detect hand position.
- Foot Effects: https://app2.vyking.io/vyking-web-ar/#/web lets users try on Air Jordans.
- Body Effects: https://www.auganix.org/geenee-announces-webar-full-body-tracking-sdk-for-3d-augmented-reality-try-on-and-avatars/ enables 3D WebAR try-on.