"Project Echo" is an immersive audiovisual experience created by sound artist and founder of POSTRED Beso Kacharava. It is an exciting project that aims to take visitors on a journey through the strange corridors of one's subconscious. Your Echo guides you through this experience. Echo is an amorphous and enigmatic companion and may remind you of otherworldly fictional encounters, such as Twin Peaks, Annihilation, or Arrival.
The concept for Project Echo has been around for a long time with Beso Kacharava. His idea was to offer visitors a unique opportunity to encounter and connect with their inner selves. The idea became significant after POSTRED partnered up with UNESCO's designation of Tbilisi as the World Book Capital 2021, Beso explained. “This opportunity was a catalyst that helped me transform the idea into a tangible project. I aimed to employ various art mediums to breathe life into this concept, focusing on creating an intimate space where individuals could connect with themselves.”
Project Echo utilizes a unique structure where visitors encounter a series of generative ‘rounds,’ each featuring a combination of sights and soundscapes—the ‘leads.’ We engineered each lead to resonate with the visitor, potentially evoking emotions, memories, or sensations.
Between each round, the journey loops back to the Echo's chamber, where the Echo, dwelling in the core, poses a question, a ‘riddle’ to the visitor. You select an answer on the tablet, which determines the course of the journey and leads the visitor to the next round. The environment restructures itself into a new scene.
In an unexpected twist at round Eight, the journey abruptly halts, and Echo reappears, this time metamorphosed into a darker entity, a reflection of the abyss within. This shadowy Echo is like a nod to Jung's shadow archetype, Lovecraftian darkness, or the suspense-filled works of Stephen King. Answering Echo's probing question takes the visitor off to round Nine.
Upon reaching this final stage, the visitor undergoes a symbolic rebirth, culminating in a return to the beginning. Having come full circle, the journey resets, and the next visitor is awaiting. This cyclical design encourages multiple visits and guarantees a unique and new experience with each run-through.
<iframe src="https://player.vimeo.com/video/935826596?badge=0&badge=0&autopause=0&player_id=0&app_id=58479" width="400" height="580" frameborder="0" allow="autoplay; fullscreen; picture-in-picture; clipboard-write" title="Project Echo - Trailer for UNESCO's World Book Capital 2021 in Tbilisi"></iframe> Impact
Since its launch, Project Echo has received an overwhelming response from visitors, with many describing the experience as transformative and emotionally intense. The sheer demand for the experience was evident as slots filled rapidly, leading to a fully booked calendar throughout the exhibition's scheduled timeframe. Such was the flow of visitors that the exhibition had to be extended by an additional month to accommodate the influx of eager attendees.
The duration of each visitor's stay within the installation varied significantly. While some spent a brief ten minutes, others were so engrossed that they lingered for up to forty minutes. To our surprise and quiet delight, a few attendees were so moved by Project Echo that they left with tears in their eyes.
Technical Execution
We wanted it to allow visitors to immerse themselves in the installation's meaning and purpose. But before we could start working, we needed to find a perfect setting for which we would later design the sounds and the visuals. So we installed huge mirrors in one of the rooms at Artarea, Tbilisi, Georgia, which would serve as Echo's chamber, where each visitor would sit on a very comfortable sofa surrounded by the circular 7-speaker audio system with a subwoofer. The mirrors were the starting point for designing the visuals, and we wanted them to enhance the visitors' feeling of depth and dimensionality. Hence, the mirrors had to fit and connect flawlessly. This was a very challenging task for the team who installed the mirrors.
The technical prototype of Project Echo, made by the sound designer George Murghulia with MaxMSP, was helpful in the early development of the technical layout, but we eventually shifted to Unity+Wwise. This combination offered more opportunities for creating more complex audiovisual systems.
This article is reduced in technical details. If you want to go in-depth, you can read the full article on the Audiokinetic blog. https://www.audiokinetic.com/en/blog/project-echo/
The visitor’s experience starts with the waiting mode. The sounds and visuals pulsate in sync, helping to prepare the visitor for what is about to come. Selecting the language on the tablet starts the experience. It leads the visitor to “Echo’s Chamber,” where Echo asks the visitor a question, which, upon answering, leads the visitor into the round corresponding to the question.
Each round had its own set of possible sounds in Wwise and visuals in Unity, which interact dynamically and generatively. The Hero's Journey inspired the progression of the experience between the rounds and the selection of feelings each round had to portray.
After going through each round and experiencing the rebirth, the program returns to the waiting mode, signaling the completion of the experience. However, visitors can experience it again without leaving the room.
"One of our challenges was to create smooth transitions between the different scenes without disrupting the visitors' immersion and highlighting the importance of their choice, which should not have felt right or wrong," said George Murgulia. Later, we will discuss the sounds controlling the visuals, which helped make impactful transitions between rounds.
The audio approach
Each sound had to transport visitors into a specific location and resonate with them. The challenge was designing and preparing a selection of sounds and layers that could work together compellingly while benefiting from a generative nature. “The sounds had to be specific enough to remind the audience of unique sounds they could encounter during their lifetime while also being ambiguous enough to fit people of diverse backgrounds,” said Beso Kacharava. The team settled on settings that included familiar environments like the seaside, a stadium, a plane, and a subway, while also having more designed and rare environments like a mine, a futuristic city, etc.
The objective was to create a dynamic and expressive atmosphere, so Beso and his team approached it like we design sounds for video game environments. We started with a sound structure that included "bed tracks" and hundreds of layers, reflecting specific sounds such as cars passing by, birds twittering, calls, shouts, etc.
Approach to music and designing a library
The music for "Project Echo" was composed by Zviad Mgebry. He aimed to produce music centered on timbres and emotions rather than being restricted to a particular genre. "The concept of freedom inspired me; I intended to create a musical form that, while being adaptive to visitors' decisions, would still feel musically natural," said Zviad.
For this project, we needed to create a musical language and timbres that blended seamlessly with sounds and visuals. To that end, Zviad worked with POSTRED's internal audio tools team to develop an Echo Toolkit virtual musical instrument.
Virtual Instrument - Echo Toolkit
To create the Echo Toolkit, we utilized the sampler Kontakt for the prototype but decided to stick with it until the end because of its strong scripting and modulation capabilities. "I wanted to transform traditional instruments into something new to create more xylophone-like instruments. Imagine instead of performing a single Taiko drum like a percussive instrument, you play a melody with it all over the keyboard," said Zviad.
My initial idea was to utilize algorithms that would adapt seamlessly to visitors' choices and produce a dynamic soundtrack. But you know how experimental art projects go when more important projects are under the ticking clock. Ultimately, we decided to use a safer approach that would be as effective but also work "out of the box" - with less risk.
Approach to dynamic visuals
Research and experiments for shaping Echo’s visuals
Luka Lebanidze tells the whole experience from his perspective: As the technical lead on Project Echo, I wanted to achieve a unique yet very natural experience in the execution. From the start, I knew I had to find an approach that would melt visual and audio together in an integral way.
Everyone in the audio world is used to seeing moving images that procedurally react to audio: curves, particles, meters, and waves. As sound artists, we see them everywhere: in music apps, plugins, hardware, and even video games. While these visuals are prevalent, they are still captivating and beautiful. This organic nature of waveforms has always captivated me; during the last few years, I had discovered the oscilloscope music genre for myself - music where sounds create images. Inspired by the exciting and crazy things the artists in that niche do with sounds, lasers, and oscilloscopes, I decided to try it as a geeky hobby, and I am glad I did.
Through playing, I observed something that has fascinated me since then. When you watch through an oscilloscope, combine two signals, and tune them in different combinations, you see their fluctuating interaction visually.
Sometimes, it is chaos, but sometimes, when you find "sweet spots," the image vibrates and dances interestingly. What you hear is what you see.
In the following video, you can see that signal #1, which draws a cube, is fixed to a frequency of 55 Hz, which corresponds to note A; when we start to slowly tune signal #2 up, which looks like a pyramid, the square image starts to break and reappear again in different "twisted" versions.
Using octaves and different musical intervals brings exciting visual results. So, for note A, the sweet spots would be 55*n frequencies. When cycling through different frequencies, images form exciting shapes in places where the audio sounds feel good (‘harmonious’, if we can use this word in this context). Again, what you see is what you hear. When you tune the signals to the frequencies that might not have any relation and don't sound good, the visual becomes a real mess - noise chaos.
Tuning signal #2 to 440 Hz, the A note creates a more stable figure because it is in octaves to signal #1 of 55 Hz, which becomes the fundamental frequency. The real magic happens when you slightly detune signal #2 to 440.37 Hz. The image starts to pulsate; an exciting texture emerges from the lines and rotates, which was not present before. The figures form into complex but beautiful Mandala-like patterns.
The kind of images you can see through oscilloscopes always look to me in the way ‘they meant’ to be in nature. I think there is something fundamental about the image of waves interacting with each other. Vibration is one of nature's fundamental phenomena and may even apply to everything.
People with experience working with hardware oscilloscopes know a pattern called Lissajous Figures. Lissajous figures are produced by the intersection of two sinusoidal curves, the axes at right angles. I wanted to explore in depth what Lissajous Patterns can do and find an approach that would work in a game engine for Echo.
The best tool I found for quickly experimenting with Lissajous Figures is the online Desmos Graphic Calculator.
By experimenting with parameters and trigonometric equations in the Desmos Graphic Calculator, I was able to build a unique logic for particle movement in Unity. I translated my Desmos equations to Unity VFX graph particle systems; we could generate satisfying and dynamic visuals by feeding in real-time parameters from Wwise. By adding multiplier variables (m, n, i, j, etc.) to equations, you can affect the symmetry and the twist of the figure; these variables are assigned to game parameters received from Echo's sounds to morph the particle figures.
This is an example of very complicated knots that you can achieve. I call them "elven figures." If you take a closer look at the images, which are chaotic and orderly, you can feel the same behavior I described in oscilloscope visuals.
Desmos calculator in action
On a side note, Desmos is handy for explaining simple audio and synthesis concepts to our academy sound and music students. It helps to demonstrate graphically how different sounds interact and what happens during physical processes like filtering, doppler, additive, and subtractive synthesis.
<iframe src="https://player.vimeo.com/video/944466612?h=9bb3ab7e30" width="400" height="580" frameborder="0" allow="autoplay; fullscreen; picture-in-picture; clipboard-write" title="Project Echo - Trailer for UNESCO's World Book Capital 2021 in Tbilisi"></iframe>
This is an example of particles animated through a similar equation created in Desmos and later translated into particle movement in Unity’s VFX graph.
Making Echo’s figure speak
Echo's entity The image of "Echo's" entity was inspired by one of Carl Jung's Archetypes: the shadow, which refers to the unconscious aspect of our psyche that contains repressed or hidden aspects of our personality.
I wanted to use a very intuitive shape with a concept fundamental to nature so the sphere idea resonated with me because a sphere is an object with all its edges equally spaced from the center. The sphere is an ideal object that cannot exist in real nature, so if we think of the concept of Project Echo, the Echo holds the pure answer, and we can never know the real essence of the answer; so the right thing to do seemed to make the sphere's surface fluctuate and to hide the sphere's shape in noise. Similar to the chorus/mandala effect we found in our experiments with an oscilloscope and Desmos graphing calculator. So, I fed the noise into the sphere's geometry and assigned the fluctuation multipliers to Echo's voice's amplitude and frequency spectrum. We recorded POSTRED’s managing partner Tina Babakishvili as Echo's voice. When Tina talked, Echo's Geometry responded and morphed.
Final thoughts
“When you sit in a dark room on a very comfortable sofa surrounded by seven speakers and see beautiful visuals in mirrors, which create a feeling of endless space in total darkness, you start to think about your existence. The feeling is unique: testing the experience every day before the project's launch was a great memory,” said Saba Baidoshvili, the developer who worked with me on this project.
We would be happy to do Echo II if there is a chance and time. It was a very interesting way to see what can be done with the same tools that we use every day but with a totally different twist.
