720p, with sound
Multiplayer AR Experience with Topics
Welcome to Overload, the AR interaction installation
Installation always talks topic
We are talking about trending topics in Taiwan’s online society (like PTT, Dcard or Facebook group/comment) and how Taiwanese discussing these topics.
lastly, are they always bullying or trolling, but never discussing.
To make this installation bold and funny, the challenges are:
- What VFX/graphics can show how people bullying or trolling;
- How interface/experience designs suiting with AR interactions;
- Make a multiplayer AR experience with the webserver and work with the installation.
When AR meets real-world model and how player spawn
Unity with Apple ARKit and Syncing
In this project, my primary job is to create a multiplayer AR environment:
- Getting the camera position in the AR world and spawn topics models correctly;
- Locating ARKit
WorldMapmakes players in the same AR coordinate;
- Serializing and syncing WorldMap via Node.js server and PUN;
- Using Unity to creating 3D visual effects and anchoring real-world’s planet;
- Setting RWD interfaces correctly for different sizes iOS device.
Self brainstorming with Unity, ARKit, Node.js and Photon developing
Issue #1 - Syncing
The primary technology issues are to sync and anchoring in the AR world. I was using
SerializeToByteArray() in Unity ARKit Plugin upload to suitable Node.js server and calling PUN’s
PunPRC to download and load serialized
WorldMap to every player's devices.
Issue #2 - Anchoring AR graphics to real-world
This installation, including 2-parts: the real world is showing place (the planet) and AR world’s graphics.
I try to make ARKit’s
ImageAnchor function matching these parts together. Once setup the ImageAnchor in Unity editor and export to Xcode project, I modifying size of the ImageAnchor's image in the real world. Lastly, ImageAnchor/synced
WorldMap, real-world/AR realm combines correctly.
The showing place with ImageAnchor QR code and the planet
UX Across Designer & Developer
Topics graphics and interfaces for Overload
When the designer becomes a developer
It’s my first time to entire becoming a developer and doing “hard” programming work. Our team included a developer, 2 UI/UX designers/product managers and a site/hardware supervisor.
When designers are outputting sketch interfaces or visual elements to me, I need to code and combine these elements with Unity’s UI system.
In this procedure, I can use UI/UX skills effectively to find what interaction part/way or interface image(s) have a problem(s), and get quick review and feedback to the designer for further discuss, or fix by myself: using UI/UX to producing ways as usual.
Player's flow and UI logic
I need keeping discuss with the supervisor, and frequently measure/scale the planet’s physical location in real-world and size with AR world’s
ImageAnchor to match two worlds together and bring to players best experience.
Trolling, Discussing and Topics-guiding
People enjoying our work
(Stage Supervisor) Wu Yukun