Concept + Design, Environmental Modelling, Hardware Design, Coding in Unreal, Programming in TouchDesigner, 3D Printing, 3D Modelling, Assembly, Graphic Design.
Model support from Henry Beach, CAD design from Michael Armstrong, 3D Print support from Julian Silva, Electrical Engineering and Python Programming from Jessica Lai.
Documentation below sot by Paul Amendola.
Tasked with designing an interactive experience that showcased the architectural model of the MIX center at SXSW '23, I decided to create a virtual representation of the building, responsive to human input w/o hardware mediation. The landscape, built from textures centered on immersive technology, changes weather, models, and particle simulations depending on the emotions "seen" by an OAK-D device running an AI emotion recognition machine vision model. As users "fly" through the virtual landscape (using a Leap Motion for gesture control in a custom input setup), they witness the sky turning red with anger when they scowl, or the sun peaking out with a smile, and a small physical print of the MIX center changing lights in turn. Additionally users were able to seed the landscape with different models specific gestures, leaving permanent reflections of their emotional embodiment of the experience.
Though students enjoyed the MIX center, and found it an exciting place to be and learn, they also found the programs and community sometimes difficult or confusing to navigate. I wanted the instructions for the experience to reflect this, allowing a degree of confusion, play, and interpretation which forced the users to engage more fully to achieve navigation.
Instructions
Textures
