Somatic Drawing Tool: 3D Body Sheet as Material for Articulating Synaesthetic Experiences

Mamoru Watanabe, Oussama Metatla, Atau Tanaka

We present a participatory co-design cycle that studied synaesthesia, a cross-sensory phenomenon where one stimulus automatically triggers another perception. We worked with a community of synaesthetes to develop and evaluate an interactive, immersive technology to depict and communicate synaesthetic experience. By working with qualitative methods to understand experience, we accommodate the highly individual nature and support otherwise limited sensory vocabulary and lack of shared understanding of the condition. The first study involved a series of participatory workshops that explored text, creative tools, interactive technologies, and sensory materials in conveying the synaesthetic experience. Insights from these workshops informed the development of the Somatic Drawing Tool, a VR drawing system that allows users to depict their experiences. Within the tool, we deployed the 3D Body Sheet, an immersive extension of techniques drawn from soma design. We engaged users in an initial evaluation of the system and took on board their feedback in a subsequent cycle of iterative development. This resulted in a bespoke system that could be personalised to diverse types of individual synaesthetic experience. This enables nuanced articulation, providing a supportive environment for externalising and reflecting sensations. These approaches for capturing the experiential nature of embodied sensory perception has potential to be applied in broader research in cross-modal and multisensory interaction.

 Overview  Programme