The RepairBot Framework: Touch-Aware Conversational Agent for Hands on Clothes Repair
Yifu Liu, Tao Bi, Chuang Yu, Lucie F Hernandez, Bruna Beatriz Petreca, Minna Nygren, Sharon Baurley, Youngjun Cho, Nadia Berthouze
Learning clothes repair is challenging for novices, who face interconnected procedural and embodied challenges, especially when learning alone. Existing tools fail to provide holistic support as interactive tutors and lack awareness of the embodied interactions of working with textiles. This paper presents a multi-phase study that investigates these challenges and explores the design space for a Human-Touch-Aware conversational agent (RepairBot). We began with an in-depth autoethnography to understand the novice experience, which informed the development of the RepairBot Conversation Framework (RBCF) together with a design implementation of a technology probe. Using the RepairBot prototype together with a Wizard-of-Oz approach to simulate Human-Touch-Awareness, we investigated how a conversational agent could support repair learning in novices as well as engage them with their own clothes-repairing projects. Subsequent lab and in-home studies with novice participants suggested specific conversational and embodied mechanisms that would facilitate novices’ holistic understanding of repair, increase their confidence, and elicit attentive touch and emotional reflection. We bring these mechanisms together in the framework presented in this paper.