Frisson: An Immersive Symphony of Synesthesia
Frisson reimagines the concert experience through XR wearables that transform sound into light, texture, and
color allowing audiences to see and feel music the way synesthetes do.
Set in the iconic Hill Auditorium (Ann Arbor, 2030), this speculative concept explores how immersive technologies can enhance live performances while preserving their emotional and acoustic authenticity.
Client:
Hill Auditorium
My Role:
UX Researcher, Product & Experience Designer
Year:
2024
Service Provided:
UX/UI Design, Information Architecture
Problem Statement:
Traditional concerts rely solely on auditory engagement, excluding individuals with hearing or sensory differences.
The challenge was to rethink how audiences experience sound, can extended reality (XR) enhance inclusivity, immersion, and emotional connection without diluting the purity of live music?
It is a neurological phenomenon where the stimulation
of one sense automatically triggers another for example, seeing sounds as colors or feeling textures when hearing music. It transforms sensory boundaries into a blended experience, creating a unique perception of the world.
It is the physical and emotional “chill” or goosebumps people feel in response to powerful stimuli such as music, art, or storytelling. It’s a moment of deep emotional resonance the body’s way of reacting to awe, beauty, or connection.
Together, synesthesia and frisson form the emotional and sensory foundation of the Frisson XR concert experience, where sound, light, and emotion merge into a shared multisensory journey.
What is Frisson?
What is Synesthesia?
Audiences wear lightweight XR headsets which acts as an invisible interpreter, enhancing rather than interrupting the performance.
Visual Interpretation: Converts live sound frequencies into color, motion, and form in real time.
Interactive Experience: Allows users to isolate instruments, read lyrical translations, or explore composer stories with subtle hand gestures.
Holographic Memory Mode: During intermissions, the headset projects Hill’s historical performances as holographic overlays, revealing its cultural legacy.
Emotional Feedback Loop: As the concert progresses, the headset responds to audience-selected emotional states, creating personalized lightscapes that evolve with each performance.
Role of XR:
Its architectural form serves as a canvas for XR projections, transforming arches, ceilings, and stage depth into living storytelling surfaces that respond to music.
The symmetry of its design naturally draws the audience’s gaze toward the stage and upward to its glass-domed ceiling, creating both acoustic clarity and visual grandeur.
To maintain spatial accuracy and immersion, a designated “XR Zone” is created in the balcony section. This area provides the best spatial coverage for headset tracking, capturing the entire depth of the stage and ceiling while ensuring the visuals remain in sync with Hill’s geometry and acoustics.
Role of Hill Auditorium:
System Interaction Map:
A Journey to Frisson: A Synesthete’s Story
I’ve always seen sound. My mom’s laughter used to paint yellow light in the air, and the piano tasted like cold metal. For years, I thought everyone saw the world this way until they didn’t. That’s when I learned to keep it to myself.
Tonight, that changes. Sitting in Hill Auditorium, my mom and best friend slip on an XR headset and, for the first time, the world sees what I see. As the orchestra begins, color and sound merge gold ripples for violins, deep red for cellos, silver bursts for percussion. The hall glows alive, and my heart does too.
I glance at my mom, eyes wide and hand outstretched as if to touch the light. My best friend whispers, “I didn’t know it could be like this.” And for once, I don’t have to explain.
By the final note, the colors bloom into brilliant teal and soft violet the feeling of frisson, of being completely understood. I leave the hall with goosebumps on my skin and warmth in my chest. For the first time, I’m not alone in how I see the world everyone else has finally stepped inside it.
