Share it
Building an Augmented Dance Application with Small Stage
This semester, a team of MDM students have formed an xR prototyping lab. Rather than being paired up with an industry client for the semester, they are being matched with 5 different clients on a series of 2-3 week rapid xR prototyping sprints.
The idea is to adapt existing rapid development processes in order to rapidly prototype VR, AR, MR and hybrid reality projects.
The initiative is meant to serve as a practical benefit to those considering development in the xR space and the documentation and research that comes out of the lab is intended to support xR initiatives in BC and the rest of Canada.
The team’s project manager Samantha Yueh is documenting their experiences throughout the summer. If you missed it, you can read Part 1 here.
__
Small Stage was our first client for the summer term. They are is a live dance producing company that combines art and technology through their performances.
The goal was to prototype an Augmented Reality (AR) pipeline to show one or more ways that virtual dance can be presented in a public space and to use AR markers to present a virtual dancer to complement a July performance.
With this in mind, we decided to build an AR dance performance app using motion capture data. To get motion capture data of each dancer, we collaborated with The Sawmill, which is a motion capture studio located in Vancouver.
Our Pipeline for Motion Capture
Rehearsals
Setup
Prior to visiting The Sawmill, there were rehearsal sessions held on campus with the four dancers from Small Stage. The team, assisted by the client, organized a space for the dancers to practice in. The space had a designated area that the dancer had to stay within during the motion capture session. The area was lit appropriately by photographic lights.
The primary purpose of the morning rehearsals was to make adjustments and help the dancers to finalise their choreography. Changes, such as the spacing, timing or physical gestures, were made to ensure that each dance would be optimized and suitable for the motion capture recording.
Rehearsing beforehand meant that, upon arrival at The Sawmill, each dancer was able to perform a routine that would work well with the motion capture area, usually only requiring some small tweaks before getting a perfect digital capture of the dance.
Motion Capture with The Sawmill
The Sawmill provided us with a learning experience, including the process of motion capture, data cleaning and tips for animating. When doing the the motion capture session, we rehearsed with the dancers from Small Stage, which allowed us to understand each dance performance. After the session, The Sawmill cleaned up the motion capture data and provided us with guidelines for the animation.
Calibrating the camera in Motive
Feedback and Testing
We then got some feedback from our client and provided A/B testing in order to give her possible solutions for the app. We also looked into particle effects in Unity, checked RADiCAL's motion capture app (a mobile app that turns the 2D videos into 3D human motion capture files using only a phone.) and started working on UX, UI and stage designs.
Conclusion
After the 2 week sprint, we finally finished our first project. We started from not knowing anything about an AR pipeline to creating a working AR prototype in just 2 weeks. The biggest takeaway from the team was "It doesn't have to be perfect, it’s a prototype. Also, every prototype is perfect! We are still learning."
Download the digital dance app.
Check out Small Stages' Summer Series.
Watch the Trailer
Takeaways
Personally, I learned so much working with this client. My two main takeaways were:
- I am not a native speaker, thus my English may cause some problems. Whenever I sent emails to the client, I needed to ask teammates who were native speakers to double check them. It was very stressful for me. I didn't want to be the one who ruined the client relationship. However, whenever I tried to do it better, the situations ended up getting worse. In the end, I had to ask for help from my teammates and supervisor. After I asked for some assistance, they were always willing to proofread my emails and suggested ways to improve what I was writing to the clients, to ensure we’d get the answers that we needed.
- My second takeaway from this project was actually something that I learned from the client - if you can't manage the client, find someone you and the client both trust to be the bridge, it will change the situation into something much better.
Overall, the first rapid prototype was surprisingly successful. One project in a two week pipeline, that was a little bit crazy and confusing in the beginning. However, we did it!
Next: Read Part 3 here.