Bridging Neuroscience & Art: The Story Behind “This Is What You Don’t See”

Debshree: Thank you! I’ve always been fascinated by how invisible struggles can be expressed through visual metaphor. Sound sensitivity is rarely acknowledged, yet it shapes how many neurodiverse individuals experience daily life. I wanted to explore how photography and AI together could give a voice to those silent battles.

Bodhisatta: My background is in AI research, especially in vision and multimodal models. For me, the challenge was: Can we design a pipeline where invisible auditory stressors—like a ticking clock or siren—translate into coherent visual metaphors? The idea of bridging neuroscience, sensory studies, and generative AI was irresistible.

Debshree: It came from the idea that sensory overload is invisible to outsiders. A room full of laughter may seem joyful, but to someone with sound sensitivity, it can feel like the world collapsing. The title captures that unseen dimension of lived experience.

Bodhisatta: It also reflects the AI’s role. The system isn’t visualising the sound itself but the internal state it provokes. The title acknowledges the gap between what’s externally heard and what’s internally felt.

Bodhisatta: We started from published clinical and neuroscience studies documenting which sounds are most distressing for people with autism or sound sensitivity. Each sound was mapped to associated descriptors like “fragmentation,” “overstimulation,” or “collapse.” We then used a language model to expand these descriptors into structured scene prompts, which fed into SDXL for image generation.

Debshree: My role was to refine those prompts artistically—choosing palettes, metaphors, and textures that resonate emotionally while staying faithful to the science.

Debshree: A lot. We didn’t settle for the first image the AI produced. We tested dozens of variants per sound, then reviewed them over days, asking: Does this really feel like a jackhammer or siren would feel inside the mind?

Bodhisatta: Technically, we controlled seeds for reproducibility, adjusted guidance scale and diffusion steps, and iterated on prompt phrasing to fine-tune emotional fidelity.

Debshree: The jackhammer became an image of earth splitting apart—because for someone hypersensitive, it feels like being torn open. The siren unfolded as a red vortex, swirling and overpowering.

Bodhisatta: We deliberately avoided literal depictions. The pipeline’s goal was metaphorical: the feeling of collapse rather than a picture of a jackhammer.

Bodhisatta: “Chaotic laughter” was tricky. It’s socially coded as positive, but for hypersensitive individuals, it’s overwhelming. Capturing that duality—something that should feel warm, yet reads as disorienting—was hard for the AI to balance.

Debshree: “Clock ticking” was subtle. It doesn’t explode like fireworks, but it gnaws away. We wanted visuals that communicated that quiet, inescapable tension.

Bodhisatta: It demonstrates how generative models can visualise subjective internal states that are otherwise invisible to sensors. That has applications in therapeutic visualisation, accessibility design, and even education. In research terms, it’s an experiment in aligning generative models with psychological and sensory data.

Debshree: And socially, it helps build empathy. If AI can translate invisible struggles into something visible, it becomes a bridge between lived experience and public understanding.

Debshree: I approached it as a storyteller. I’d imagine how a sound feels inside the body and suggest metaphors—shards, spirals, fractures.

Bodhisatta: I’d then operationalise that into prompts, embeddings, and model runs. The back-and-forth was essential: emotion without structure would drift, and pure code without intuition would fall flat.

Debshree: It’s validation that socially engaged AI art has a place beside traditional forms. It shows audiences are ready to see technology as a tool for empathy, not just novelty.

Bodhisatta: For me, it proves that technically rigorous pipelines, when combined with human sensitivity, can create art that matters. It bridges research and public impact.

Bodhisatta: Anchor your work in data and research. When AI art is rooted in real human phenomena—whether psychological, cultural, or scientific—it transcends aesthetics and contributes to understanding.

Debshree: And don’t forget the heart. AI can generate images endlessly, but it’s your vision, your empathy, that turns those images into a story.