This article documents my process over this project, which will be finalized by the end of September 2022.
What kind of story should I tell? / Conceptualization
September 1st 2022
Over the weekend, I journaled to find overall themes about the story I intended to tell. During that process, I began to think about what part of this story was best told as an experience. I thought back to what other students did last year, and recalled that many did not explicitly depict the story. Although I haven’t actually realized the project, I can foresee how this allows for a conceptual depth when designing an experience.
Through brainstorming with post-its (below, left) and reflection on paper (below, right), I explored the themes of gratitude, family, and reunion. I made questions on a post-it that may help people connect their own experiences to Kay Lee and John Nordeen’s story. Additionally, I created a table below that personally mapped our five senses to the aforementioned themes.
Although I haven’t decided the exact medium of this, I think that this experience could direct people to share their own thoughts about gratitude, family, and reunion. Just based off my own sense/feeling matrix, I can see that people’s feelings about these subjects can be both incredibly diverse and also very unifying. However, our experiences can also be very uncomfortably intimate. Just based on my own responses, many of them are totally unacceptable to show in a public setting. However, as this is just a time for conceptualization, I will leave this problem for three-days-in-the-future Bryce to solve.
How do I tell this story? / The problem that Bryce had to solve three days later
September 5-8th, 2022
In this phase, I storyboarded an experience that would rely on the removal of sight to draw attention to other senses. At the same time, a voice would guide you to your senses, and create an experience to bring gratitude to you. However, Daphne brought up a good point: gratitude is such a broad concept, which can be difficult to make something meaningful out of it.
After our conversation, I decided to focus more on the gratitude that one can feel about the people that they care more about. This requires adjustments to the conversational interface– for example, the experience could allow for users to jot down certain names or memories that they’re grateful for. At this point of ideation, Susan reminded that the user sees nothing but black, preventing the actual writing of something. It might be an interesting challenge to solve for this.
Additionally, to add sensory depth to the design of this experience, I thought about creating a physical space augmented by a headset where this experience would take place in. While my initial design, a experience contained entirely within a headset, aimed to create an experience where the user felt comfortable and less vulnerable, I think that a physical space will give me more control over emotion through touch and smell.
Further ideation and work
September 12th, 2022
In this iteration, I wanted to create a basic first iteration for the physical space that the user would sit in. I explored layouts through very rough sketching, shapes and form through Sketchup, and finally lighting in Unity. In this period of time, I made some major changes to the direction of the project. For example, instead of having a VR overlay to everything with flashing colors, I instead worked on focusing this experience to be entirely grounded within the space itself.
My first overarching goal was to create a relaxing space. To start, I created a space where people could lay down, surrounded by cushions. I tried to get an angle that seemed like a natural resting pose, somewhat like a psychotherapy couch. Meanwhile, I chose for a direct connection with nature– I hope my exhibit can be placed in a location with aromas of nature, which would be connected through a hole in the dome. This hole would also let in natural light and disperse it though the dome, creating soft lighting. I was inspired by my experience in the “Three Gems” skyspace by James Turrell (2005), which made me feel contemplative and calm.
I also tried to design for a human-friendly scale. This went along smoothly in drafting process within Sketchup, but once I was fiddling with import scales when importing to Unity this went out the window.
Writing the script and further iteration
September 15th, 2022
During this iteration, I fleshed out the a script that would record the overall narrative arc of this project. Now that the experience did not have any VR overlay, I focused on creating a good audio script that would communicate the experience well.
When researching this, I looked at a ton of existing stuff about guided meditation and gratitude. For context, gratitude-based therapy is frequently based on meditation. I’m not a psychologist. However, this really helped put me into a mental space that allowed me to have gratitude. I recall one of my high school teachers being really into gratitude. She went all out on the meditation experience, trying out things like using a tuning fork or natural sounds to enhance the experience. From a frame of reference to the “sensory” aspect of this project, this really helped– however, I recall that it was also super hard to keep your head in the game because everyone was sitting in very hard metal chairs.
For example, I had been trying out Headspace’s guided breathing meditations– I really loved the mood that I was put into by just controlling my breathing. This was really important, and became one of the first things that the visitor would do. Additionally, I watched meditation videos on youtube and tried out other apps.
Finally, I put together a script based on the overall arc identified in my previous ideation. I combined this with writing from existing gratitude scripts.
- “Breath in”, (wait 4 seconds) “breath out” (wait 4 seconds), repeat x 5- “Concentrate on what you smell. Does it remind you of anything?”
- “Listen to your heart beat. Think about how amazing it is that you’re present and breathing.”
- “Feel your surroundings around you”- “Think about a memory that made you happy. What was it?“
- “Write it down on the ground with your finger.”
- “What did your memory smell like? Feel like?”- “Think of someone or something present in that memory.”
- “Write it down on the ground with your finger.”
- “Feel gratitude for this memory, the people and things present, your own life, and the many gifts you have been blessed with.”- “Now bring your attention to how this gratitude feels in the area around your heart. With
each inhale, let this feeling grow outwards. Expanding to fill your chest, your arms and
hands, your legs and feet. With each inhale this feeling grows, filling you up.”
- “And now, even as you return your attention to your breath, let your body remember the
sensations of your gratitude.”
Beyond the script
Looking at these meditation resources, I saw how important it was to be in a good environment to meditate well. You have to be alone. You should have pillows and cushions. Your space should be well-ventilated. There’s all sorts of factors interacting with sight, sound, smell, and touch that can either make or break the experience.
If you read the script, you’ll also see that I had a couple lines telling the visitor to write down things on the ground. In class earlier, Daphne and I talked about how a visitor could try to remember something for later. We both agreed that maybe the visitor could write it down. But there was a problem. How could you write in the dark? Initially, we talked about having a tablet somewhere. But I felt that this took away from this dark, serene space that you had to yourself. So eventually, I thought about putting the text under the wood. What if you could write the text on the wood? I saw that this tech exists in a limited capacity, and I went with it.
To mock it up, I would take two directions: firstly, I would make a physical prototype with a wood veneer over the screen of an iPad, and I would also mock it up in Unity. It was hard. At first, I tried to create the effect in Photoshop, but I wasn’t able to get the right feeling. Eventually, I used p5.js to create a mockup where you could draw something on the screen with your finger. I then took a screenshot of the drawing and ported it into Unity.
Recording + integrating script into a dynamic scene
September 20
If you spend a lot of time on the internet, you may have heard of something called ASMR. It’s basically a sensory experience through sound that’s supposed to have a relaxing or even arousing effect. The interesting part about it is how the microphone is set up– it’s often very high quality, and picks up on everything.
I thought a lot about how I should record this. While ASMR sounds tend to calm people down, it can also be way too intimate and freak people out (which is what I feel when I listen to it). I decided to keep it simple and have a quiet recording.
Eventually, I settled on having Luca as the narrator because I felt that a deep voice would be calming. Putting it all together, I spliced together multiple clips of him reading the script and added a handful of seconds between every line. When it was all done, I composited a second layer of nature sound effects that would play in the background.
I also integrated this with a dynamic scene– I created shortcuts for dimming the sky. For example, if I pressed “1”, the lights would dim slowly over the span of 10 seconds. On top of that, I also added in projections of other visitor’s memories and people. My reasoning for this would be to help suggest to the viewer what they could write down. One concern would be that this would feel to exposing to the viewer. However, the examples use very brief descriptions, and they all seem to be relatively anonymous and non-identifying.
Scripting the scene, starting to make final touches
September 22
In this iteration, I made very small changes.
To start off, I modified the dynamic system so it would automatically change the environment as the sound played. This would give me consistent timings that would help me solidify the feeling of the experience, allow people to use the Unity scene as a demo without external intervention, and to have a more consistent timing for the video.
Additionally, I updated the UV mapping of the model. Because sketchup was incorrectly exporting UVs, I went into Blender and unwrapped the UVs again. This would allow for smoother lighting without strange artifacts, as the lightmap would be preserved. Additionally, this would let me add a concrete texture to the walls to present a better sense of the material.
As for the rest of the scene, I added in these pillows that would feel more comfortable. Two classes ago, Yash had recommended that I check out these squiggly intestine-shaped pillows at an exhibition. I instantly fell in love with the design– however, when implementing it, there were many moments when the pillows felt way too similar to an intestine. Eventually, I settled on using the same material as the cushions, which made the pillows somewhat less visceral.
In the previous class, Daphne and I talked about different ways for a visitor to take their gratitude home. It was important to have something that wouldn’t pressure the visitor to do anything. We decided on having a small business card-shaped card that would be printed and given to to the visitor. I thought about having a receipt printed, but a receipt is meant to be thrown away, while a card is mean to be kept for a longer time.
I also started the veneer prototype: eventually, I just printed out a texture of wood onto a piece of office paper and taped it over my iPad. This turned out to work fine, given if there wasn’t too much surrounding light to drown everything else out.
Final touches and presentation
September 27
The most significant change I made to this project during this iteration was that I modified the human model. I changed the viewpoint of the person so that they wouldn’t be facing the door, and I also adjusted the shader so that the transparency of the person wouldn’t overlap itself. Previously, I was having issues with either picking a solid color that distracted and hid parts of the environment, or having strange occlusion when rendering the model transparently. Eventually, I wrote HLSL that would just have a solid transparent model, kinda like turing the opacity of an entire layer to 50% in Illustrator.
Additionally, I re-recorded Luca’s script with minor changes. For example, he now says to “touch the floor” instead of “ground”. Additionally, the script also draws attention to “where the memory happened” instead of what exactly it was.
Once I had recorded the Unity scene, I moved onto designing the card that the visitor would receive. I used a very elegant and classic sans-serif as the primary font, which I felt communicated serenity and a timeless feel. I carried these design attributes to the presentation as well.
Reflection
Overall, this project felt a lot about restraint. I started wanting to do a AR project, but through the process of refinement with a razor-sharp focus on a goal, I just kept on whittling urges to add complexity and “cool factor” out. What came out of this was my project– something that used my skills only to advance its goal. In that case, I think I succeeded.
If I were to have more time on this project, I should explore some other senses. I never got the chance to explore smell. Additionally, I think I might have narrowed down the idea too quickly– I always wonder what other paths I could’ve went down besides the idea of gratitude, given the richness of the starting story I had to work with. Overall, this was an intriguing project that I took a lot of delight in doing.
