A web-based AR experience with an AI-powered dog that responds to your interactions. Place a virtual dog in your real environment, talk to it, pet it, and play with it - all powered by large language models.
- Places a 3D dog in your real world using AR
- Dog responds to your voice/text using AI
- Pet the dog and throw a ball to play with it
- Works directly in your browser - no app download needed
- Node.js
- API key from OpenAI or Anthropic (Claude)
- Smartphone with AR support (modern iOS/Android)
- Clone this repo
- Install dependencies:
npm install - Create a
.envfile:LLM_PROVIDER=openai # or "anthropic" OPENAI_API_KEY=your_key_here ANTHROPIC_API_KEY=your_key_here PORT=3000 - Make sure you have a 3D dog model:
- Place a GLB/GLTF model file in the
/assetsfolder - Default path is
/assets/corgy.glb
- Place a GLB/GLTF model file in the
- Start the server:
node server.js - Access on your computer:
http://localhost:3000
WebXR requires HTTPS. Use ngrok to create a secure URL:
- Install ngrok if needed:
npm install -g ngrok - Create a tunnel (while your server is running):
ngrok http 3000 - Open the HTTPS URL (provided by ngrok) on your phone
- Grant camera permissions when prompted
- Click "Start AR"
- Move your phone to scan for surfaces
- Place your dog on a detected surface
- Interact using the buttons:
- Talk: Say something to your dog
- Pet: Give your dog affection
- Throw Ball: Play fetch
- Frontend: HTML/JS with Three.js and WebXR
- Backend: Express.js server
- AI: OpenAI/ChatGPT or Anthropic/Claude
- 3D: GLTF/GLB model with animations