Overload (2019)

AR-based interaction installation with the interface

Overload’s 720p, with sound 14.5 MiB

The Challenges

Multiplayer AR Experience with Topics

Welcome to Overload, the AR interaction installation

Welcome to Overload, the AR interaction installation

Installation always talks topic

We are talking about trending topics in Taiwan’s online society (like PTT, Dcard or Facebook group/comment) and how Taiwanese discussing these topics.

lastly, are they always bullying or trolling, but never discussing.

To make this installation bold and funny, the challenges are:

  • What VFX/graphics can show how people bullying or trolling;
  • How interface/experience designs suiting with AR interactions;
  • Make a multiplayer AR experience with the webserver and work with the installation.

When AR meets real-world model and how player spawn 720p, silent 11.3 MiB

My Role

Unity with Apple ARKit and Syncing

In this project, my primary job is to create a multiplayer AR environment:

  • Getting the camera position in the AR world and spawn topics models correctly;
  • Locating ARKit WorldMap makes players in the same AR coordinate;
  • Serializing and syncing WorldMap via Node.js server and PUN;
  • Using Unity to creating 3D visual effects and anchoring real-world’s planet;
  • Setting RWD interfaces correctly for different sizes iOS device.
Self brainstorming with Unity, ARKit, Node.js and Photon developing

Self brainstorming with Unity, ARKit, Node.js and Photon developing

Issue #1 - Syncing WorldMap correctly

The primary technology issues are to sync and anchoring in the AR world. I was using SerializeToByteArray() in Unity ARKit Plugin upload to suitable Node.js server and calling PUN’s PunPRC to download and load serialized WorldMap to every player's devices.

How ImageAnchor works

How ImageAnchorworks

Issue #2 - Anchoring AR graphics to real-world

This installation, including 2-parts: the real world is showing place (the planet) and AR world’s graphics.

I try to make ARKit’s ImageAnchor function matching these parts together. Once setup the ImageAnchor in Unity editor and export to Xcode project, I modifying size of the ImageAnchor's image in the real world. Lastly, ImageAnchor/synced WorldMap, real-world/AR realm combines correctly.

The showing place with ImageAnchor QR code and the planet

The showing place with ImageAnchor QR code and the planet


UX Across Designer & Developer

Topics graphics and interfaces for Overload

Topics graphics and interfaces for Overload

When the designer becomes a developer

It’s my first time to entire becoming a developer and doing “hard” programming work. Our team included a developer, 2 UI/UX designers/product managers and a site/hardware supervisor.

When designers are outputting sketch interfaces or visual elements to me, I need to code and combine these elements with Unity’s UI system.

In this procedure, I can use UI/UX skills effectively to find what interaction part/way or interface image(s) have a problem(s), and get quick review and feedback to the designer for further discuss, or fix by myself: using UI/UX to producing ways as usual.

Player's flow and UI logic

Player's flow and UI logic

The stage

I need keeping discuss with the supervisor, and frequently measure/scale the planet’s physical location in real-world and size with AR world’s ImageAnchor to match two worlds together and bring to players best experience.

The Final

Trolling, Discussing and Topics-guiding

People enjoying our work

People enjoying our work

Credits and Refs
(UI, UX) Chi-Yun Tsou, Meng Yun Tsai,
(Stage Supervisor) Wu Yukun
Team Members
Chi-Yun Tsou - Overload (zh-tw)
Meng Yun Tsai - Overload
“Poooooof” the Graduation Exhibition, Dept of IxD, Taipei Tech (zh-tw)