pakomm.demediapakommintroduction.mp4.mp4
For further Information visit the PaKOMM Website.
- Dgraph (23.1.1)
- Cloud Storage for Firebase (11.6.0) (optional) (for the setup see the App section)
The folder server contains PaKOMM's stack for the graph database to be deployed via Docker Swarm. In chosen in an agile development approach, might be replaced by geo database in a future development cycle.
- Create the necessary SSL certificates and provide them via a Docker Bind mount to the Nginx reverse proxy service.
- Set the Dgraph token
pakomm-stack.ymlfile in for the Dgraphalpha1service to protect the admin endpoint.
- Initialise the database with
curl 'https://xxx:8080/admin' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'Origin: altair://-' -H 'X-Dgraph-AuthToken: xxx' --data-binary '{"query":"mutation InitializeSchema {\n updateGQLSchema(\n input: {\n set: {\n schema: \"\"\"\n type EventSession @withSubscription {\n number: Int! @id\n name: String @search\n created_at: DateTime\n screenshotPath: String\n objects: [PlacedObject] @hasInverse(field: eventSession)\n brushAnnotation: [BrushAnnotation] @hasInverse(field: eventSession)\n logOperation: [LogOperation] @hasInverse(field: eventSession)\n deleted: Boolean\n locked: Boolean\n }\n\n type PlacedObject @withSubscription {\n id: ID!\n created_at: DateTime\n pX: Float\n pY: Float\n pZ: Float\n rX: Float\n rY: Float\n rZ: Float\n sX: Float\n sY: Float\n sZ: Float\n prefabName: String\n eventSession: EventSession\n modified_at: DateTime\n annotation: [Annotation] @hasInverse(field: placedobject)\n }\n\n type LogOperation {\n id: ID!\n number: Int\n created_at: DateTime\n eventSession: EventSession\n content: String\n objectid: String\n pX: Float\n pY: Float\n pZ: Float\n rX: Float\n rY: Float\n rZ: Float\n sX: Float\n sY: Float\n sZ: Float\n }\n\n\n type BrushAnnotation {\n id: ID!\n created_at: DateTime\n pX: Float\n pY: Float\n pZ: Float\n rX: Float\n rY: Float\n rZ: Float\n sX: Float\n sY: Float\n sZ: Float\n prefabName: String\n eventSession: EventSession\n modified_at: DateTime\n brushStrokeWidth: Float\n brushColor: BrushColor\n ribbons: [RibbonPoint]\n annotation: [Annotation] @hasInverse(field: brushAnnotation)\n }\n\n type RibbonPoint {\n id: ID!\n number: Int\n pos: PositionFormat\n rot: RotationFormat\n }\n\n type BrushColor {\n r: Float\n g: Float\n b: Float\n a: Float\n }\n\n type PositionFormat {\n x: Float\n y: Float\n z: Float\n }\n\n type RotationFormat {\n w: Float\n x: Float\n y: Float\n z: Float\n }\n\n type Annotation @withSubscription {\n id: ID!\n content: String\n created_at: DateTime\n placedobject: PlacedObject @hasInverse(field: annotation)\n brushAnnotation: BrushAnnotation @hasInverse(field: annotation)\n upvotes: Int\n downvotes: Int\n }\n \"\"\"\n }\n }\n ) {\n gqlSchema {\n id\n schema\n generatedSchema\n }\n }\n}","variables":{}}' --compressed- You might want to test the successful initialisation of the database with
curl 'https://xxx:8080/admin' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'Origin: altair://-' -H 'X-Dgraph-AuthToken: xxx' --data-binary '{"query":"query getSchema {\n getGQLSchema {\n id\n schema\n generatedSchema\n }\n}","variables":{}}' --compressedThe import pipeline for use of geodata in Unity has been extensively described by Keil et al. 2021. Make sure to choose the layer 15: Landscape for surfaces that should allow teleporation in VR.
Navid Kabir's SimpleGraphQL-For-Unity Unity package provides the GraphQL client in Unity. For more information and a setup guide see the README.md . (MIT license)
`https://github.com/ngoninteractive/SimpleGraphQL-For-Unity.git`
To use the client in Unity, ...
- add package from git URL to the UPM,
- past the Dgraph endpoint created in the server section (e. g. https://.../graphql) in the asset file (
GraphQLConfig) located in the folderAssets/Configs/, and - assign the configuration file to the
DgraphQueryscript assigned to theGlobalManagerGO (GameObject) in theBaseScene.
Normal's Normcore provides real-time collaboration between all participants (touch table, VR, and AR). To install and configure the package also see the Getting Started guide.
To use the Normcore, ...
- register for a Normcore account,
- create a Normcore application in the dashboard (in the project phase a free plan with limitations is offered)
- add the package via Unity's Asset Store,
- add Normcore's
Realtimescript to theNetworkManagerGO in theBaseScene, - assign the added script to the
NetworkManagerscript in theNetworkManager, and - past the
App Keycreated in step 2. Finally, add Normcore's RealtimeView and RealtimeTransform scripts to the assets as described in [3D Assets (Editable Objects)](#3D Assets (Editable Objects)).
TouchScript provides TUIO input functionality used for the marker input on the touch table. To install the library follow the Getting started section. (MIT license)
Once installed, ...
- add the
TouchManagerPrefab to theBaseSceneand - follow the instructions in the Building the Apps section to chose the appropriate settings when building the app.
Immersal provides a visual positioning approach for the AR functionality. To install and configure the library also see the SDK documentation guide.
To use Immersal, ...
- register for an Immersal Developer Portal,
- download the Immersal SDK 1.19.0 Core (.unitypackage) provided in the portal,
- download the immersal-sdk-samples Unity project to use it its
SampleSceneas a reference for the following steps (and to compile the scanning app instead of step 5, if needed), - add the
ImmersalSDKprefab and theARSpaceprefab containingARMapGOs. For the real-world registration, - use the app Immersal for scanning and download the processed results (.bytes and ...-metadata.json files) from the developer portal. Finally,
- import the files in the
Map Datafolder in the Unity project and assign the .bytes file to theMap Fileattribute in theARMapscript. (Use a separate GO with anARMapfor each scan.) - To align the scan with the geodata curated in a dedicated scene (see Geodata), load the latter and the
BaseScenetogether in the Unit's Hierarchy and change theTransformcomponent of theARMapGO until the latter's point cloud is properly overlapping with the geodata.
Cloud Storage for Firebase provides thumbnail functionality and is used to store the thumbnail screenshots users can take at the touch table or in AR. This package is optional and does not need to be installed if the thumbnail functionality is not required.
To utilize Cloud Storage for Firebase, ...
- create a Google account and create a new app in the Firebase Console and make sure you understand and setup the Security & Rules,
- download the FirebaseStorage_11.6.0.unitypackage, which also contains the SDK's dependencies,
- resolve the all needed plaform-specific libraries in the
Assets>External Dependency Managermenu before building, - download the
google-services.jsonandGoogleService-Info.plistfiles and copy them in theAssetsfolder. Finally, - assign a default thumbnail to the
NetworkManagerGO'sScreenshotHelperscript.
Import the 3D-Object to Unity and make sure that ...
- textures, materials and shaders are fine (make sure that the shader supports emission) and that
- all parts or the parent GO have an appropriate collider.
To use the assets in the scene as editable objects that can be manipulated by the users create a Unity prefab for each item. the prefabs must be located in the Resources folder or its child folders to be used with Normcore (2.6.1).
(1) For the miniature models in the VR Menu copy the prefabs into the folder associated with the desired category (e. g. for a MobilityHub Assets/Ressources/MobilityHub/).
(2) For the thumbnails in in the menus of the touch table and the iPads, create thumbnails with a transparent background and save the png files in the Assets/Ressources/Thumbnails/ folder. Categories are derived from the subfolders in (1).
(3) For the editable objects in the scene, copy the exact same models in the Assets/Ressources/ folder. These objects will be instantiated during runtime, so they need realtime components (in our case from normcore as described above). Fore the realtime synchronization via Normcore (2.6.1) add the latter's RealtimeView and RealtimeTransform scripts as components to the prefabs. Furthermore, add the DatabaseSync script as well to allow the saving edits in the Dgraph (23.1.1) database. (Do not add these three scripts to the prefabs copied to the category subfolders in the previous step!).
Notice: If the objects are not true to scale, it might be necessary to test and adjust their sizes iteratively. Test if you can add a new object to the scene during runtime. (If you encounter some issues, check your internet connection, your normcore setup, your database setup and if the objects in Resources folder have all necessary scripts attached. Make sure that all the files representing the same object (VR menu (1), thumbnails on the touch devices (2) and the objects placed in the scene (3)) have the same name in all folders.
If objects in the VR menu or their default size in the scene are too big or too small, adjust the item's scale individual in the relevant folder of (1) or (3).
To create and delete categories apply the necessary changes in the folder structure in (1) and adapt the VRMenuHolder prefab in the Resources folder (which is also used in the BaseScene) accordingly. Copy one of the category-buttons, place it as you wish, change icon and change path to category folder in Assets/Ressources/and assign the Category Item representing a the object category in the menu as well as the Category Transform Position describing the position in the menu. Finally, for the touch menus, search for Left Menu GO, adapt the category buttons, change the icon and assign it according to you wish.
To adapt the starting position on the touch devices, in VR, and in AR, apply the changes in the GlobalManager GO in the BaseScene.
Check Enable TUIO in the Touchscript settings (Windows → Touchscript → Settings).
Apply the following settings in the GlobalManager script:
- VR Mode
- VR Simulator
- TUIO Mode
- Drawing_Enabled
- Eraser_Enabled
- Annotations Visible
- Cam Controllers Enabled
Deactivate the Standard Input component in the TouchManager GameObject.
Uncheck Enable TUIO in the Touchscript settings (Windows → Touchscript → Settings).
Apply the following settings in the GlobalManager script:
- VR Mode
- VR Simulator
- TUIO Mode
- Drawing_Enabled
- Eraser_Enabled
- Annotations Visible
- Cam Controllers Enabled
- In build settings: switch platform to android
- check player settings and enable OpenXR, set Oculus Touch profile for android platform.
Activate the Standard Input component in the TouchManager GameObject.
Uncheck Enable TUIO in the Touchscript settings (Windows → Touchscript → Settings).
Apply the following settings in the GlobalManager script:
- VR Mode
- VR Simulator
- TUIO Mode
- Drawing_Enabled
- Eraser_Enabled
- Annotations Visible
- Cam Controllers Enabled
Activate the Standard Input component in the TouchManager GameObject.
You might need to install Firebase's plaform-specific libraries for iOS with the CocoaPods after exporting the Xcode project in the previous step. Install the pods using the given pod file in the Xcode project folder:
pod installOpen the workspace file in Xcode to build and install the project.