For the past few years I've been leading teams of students in research and design projects. We create interactive stories with a twist: we're interested in physical objects and physical spaces, and a lot of the projects ask how we can incorporate non-digital objects into storytelling and design.
The goal of this mini asynchronous workshop is to pull back the curtain on some aspects of this research process. These projects are, admittedly, a little weird. The point is to get us thinking about what we design, and how we design.
Project 1: Tangible Virtual Reality
What do yo think about when you think about virtual reality? In each of my research projects, we're interested in storytelling but we’re also trying to ask broader questions about technology. For this project, as consumer VR began to release new controllers, which would become the primary means of interacting with virtual worlds, we asked what does that mean for our hands and our bodies in VR? What does it mean if these are the objects that mediate our interaction with the virtual?
To ask that question, we created the outline of a story in which you’re going from place to place, helping solve puzzles along the way with your friend the squirrel. In our first interaction, the squirrel tells you that the forest is in danger and the only thing that will save us is the forest song, which is trapped inside the magic cube. You have to memorize a sequence of lights and sounds, tapping the faces of the cube.
When you solve the puzzle, the forest animals congratulate you. As design researchers, the important thing for us is that you’re physically interacting with an object from the story. The cube is a cube, rather than a controller that represents a cube. And the fact that this version of the cube is literally made out of cardboard and sensors should show what kind of prototyping we’re doing — we’re more interested in the interactions, in asking questions, rather than what it looks like.
Then there’s the squirrel who talks to you in the story, and who is your main companion in the narrative. What would it mean to actually interact with that squirrel?
In case you're wondering, yes: it is weird to hold a stuffed animal that you filled with sensors and motors to give it a moving head and a heartbeat. We also gave it a wire skeleton to give its body shape and to give the feeling of a spine while you’re petting it. It is very weird. But it's also effective.
Reflection questions [no need to submit your responses these questions -- these are just to practice asking questions about the design process]: Compare these kinds of interactions to interactions with video game controllers. What do we gain with this kind of specificity? What kinds of materials did we use to ask our design questions?
Project 2: Sensory Virtual Reality
This next project goes a little further. Around this time, new research and new companies were coming out, proposing a lot of extra technology to create sensory experiences in VR.
We asked, why do we have to wear all this technology to feel something in VR? How might we create something like this with everyday objects and materials? To answer this question, we decided to design a low-cost experience in VR by focusing on the sensory qualities of a single moment.
Our first proof of concept is a moment at a virtual beach. The player sits in a lawn chair, puts on sunscreen for that embodied action as well as for the distinct smell, turns on a space heater, prepares a drink of choice, puts their feet into a plastic container of sand, puts on the headset for the visuals of the beach, and headphones for the sound of waves and the birds.
There's something a little bit funny about preparing all this to create a sensory experience, but it is relatively cheap, it doesn't require a lot of extra technology, and most importantly, it can easily be personalized. When we got someone from our lab to try this out, he sat back in the chair, put his feet in the sand and said, "All I need now is a drink." So we gave him one, and he just sat there with a smile on his face.
Other than the headset, we're not using any fancy technology, but in this moment there is smell, sound, taste, touch, visuals, the warmth of the heater, and that feeling of sitting in a deep lawn chair with your feet in the sand.
Reflection questions: Compare these kinds of interactions to interactions with mobile phones or computers or VR headsets. What kinds of materials did we use to ask our design questions? How might players choose to co-construct their own version of this experience?
Project 3: Tangible Online Narratives
When the pandemic pushed us all online, we still asked some of the same kinds of questions. What are our physical interactions now that we're all online? How can we bring a little bit of physicality to our online environments?
This question brings us to your bootcamp challenge. I would like you to organize and play a game prototype with members of your cohort.
Over first pandemic summer, I worked with three students (Stefan Grambart, Rodrigo Skazufka Bergel, and Carter Richmond) to design and create a prototype of a tangible video conferencing game. As a player, your first step is to choose three household objects:
Once you've chosen your three objects, download the PDF of your form-fillable playsheet, which will tell you how to play. Use Teams to connect with your buddy and reach out to another pair of students so that you can schedule a game session. You don't have to know each other beforehand -- you can even use this game as a way to get to know each other.
The game is an unfinished prototype, so it certainly isn't perfect, and you may have to work together to figure out what to do. Your task is to think like a player and like a design researcher. What is this game trying to accomplish? How might it be improved? How would you change this game to suit your own needs and preferences?
After you play the game (it should take about 20-30 minutes), take a screenshot of your team holding your objects and upload it to Teams. After chatting with your teammates about the game, share your thoughts about the experience in a short questionnaire.
As this character would say, "Good luck, Agents. Welcome to the multiverse."
Comments