Just a simple example using the non-official 3DGS extension (j3soon/omni-3dgs-extension) with kit-app-template and web-viewer-sample for web streaming.
We assume you have already ran the Embedded Web Viewer example on your machine and confirm that it works.
Preferably, you should have also ran the Omniverse 3DGS Extension to test the 3DGS renderer.
-
Clone the repo:
git clone --recursive https://github.com/j3soon/omni-3dgs-streaming.git cd omni-3dgs-streaming -
Set up 3DGS extension.
Download sample 3DGS assets by following the instructions in the 3DGS extension repo. Specifically:
cd omni-3dgs-extension wget https://github.com/j3soon/omni-3dgs-extension/releases/download/v0.0.2/assets.zip unzip assets.zip # change the DATE_TIME to the name of the placeholder DATE_TIME=2025-02-19_105311 cp -r ./assets/exports/poster/splatfacto/$DATE_TIME ./assets/exports/poster/splatfacto/DATE_TIME DATE_TIME=2025-02-19_121606 cp -r ./assets/exports/poster/nerfacto/$DATE_TIME ./assets/exports/poster/nerfacto/DATE_TIME ls ./assets/exports/poster/splatfacto/DATE_TIME/splat/splat.ply ls ./assets/exports/poster/nerfacto/DATE_TIME/tsdf/mesh.obj
-
Run the VanillaGS renderer:
cd omni-3dgs-extension xhost +local:docker docker compose build docker compose up # in a new terminal docker exec -it vanillags-renderer bash -ic "python /src/main.py"
Wait for the renderer to start (seeing the
Gaussian Splatting renderer ready for requests...message), and then run the following on host:sudo chown -R $(id -u):$(id -g) /tmp/omni-3dgs-extension
-
Build and run the Omniverse Kit app (backend):
cd kit-app-template # Link extension ln -s "${PWD}/../omni-3dgs-extension/extension/exts/omni.gsplat.viewport" "${PWD}/source/extensions/omni.gsplat.viewport" ./repo.sh build mkdir -p _build/linux-x86_64/release/samples # Link assets ln -s "${PWD}/../omni-3dgs-extension/assets" "${PWD}/_build/linux-x86_64/release/samples/gsplat_assets" # Run the app ./repo.sh launch -- --no-window # and press down to select `my_company.my_usd_viewer_streaming.kit` and press enter
-
Build and run the web viewer (frontend):
(Optional) If you want to run the web viewer on a different machine within the same local network, you can modify the
"server": "127.0.0.1"line in the web-viewer-sample/stream.config.json to the IP address of the machine running the Omniverse Kit app.cd web-viewer-sample npm install npm run dev -- --hostOr use docker if you don't want to install
npmon host.docker run --rm -it --network=host -v $PWD:/workspace node bash cd /workspace npm install npm run dev -- --host
Open http://localhost:5173/ and connect to the viewer.
The Replicator still reports some errors, which breaks the alpha blending feature, but I haven't have time to investigate them yet. The error message is:
KeyError: '/Render/PostProcess/SDGPipeline/Replicator_DistanceToCameraSDbuffPtr'
which could be observed by removing the try-except block in the omni.gsplat.viewport/omni/gsplat/viewport/extension.py.
cd isaacsim-app-template
./repo.sh build
uv venv --python 3.10
source .venv/bin/activate
uv pip install pyzmq
./repo.sh launch -- --ext-folder /src/exts --enable omni.gsplat.viewportEnable the Script Editor extension and run the following python code:
from omni.gsplat.viewport import OmniGSplatViewportExtension
OmniGSplatViewportExtension._instance.init_streaming()To ensure reproducibility, we have pinned specific versions of dependencies. The licenses, versions, and commit hashes of these dependencies are listed below:
-
NVIDIA-Omniverse/kit-app-template (at commit fa73fe6) is released under the NVIDIA Software License Agreement.
This is basically Omniverse Kit (106.5), which is the Omniverse rendering and streaming backend. This version corresponds to the Kit version used in Isaac Sim 4.5.0.
-
(Unused) Isaac Sim App Template (at commit 08187a4) is released under the NVIDIA Software License Agreement.
This is basically Isaac Sim 4.5.0. But isn't used in the instructions above.
-
NVIDIA-Omniverse/web-viewer-sample (at commit 433062f) is released under the NVIDIA Omniverse License Agreement.
This is a sample app (v1.4.1) built with React along with Vite, which is the frontend for web streaming.
Further changes based on the packages above are released under the Apache-2.0 License.