Exploring Soundscapes in Robotic Storytelling: Effects on Engagement and Genre Variation

The study explores how adding sound effects and background music enhances robotic storytelling, with the most significant improvements seen in horror and romantic genres. While sound increased engagement and decreased fatigue, its effectiveness varied across different story types.

A study by researchers Sophia C. Steinhaeusser and Birgit Lugrin from the University of Würzburg explores the role of non-speech sounds like sound effects and background music in enhancing robotic storytelling. This series of four online studies examined how these audio additions impacted participants' storytelling experiences, emotional responses, and perceptions of the robot across different genres, including horror, detective, romantic, and humorous stories. Social robots, which combine the physical expressiveness of human storytellers with the technical capabilities of audiobooks, are increasingly used in a variety of fields, including education, healthcare, and entertainment. However, the addition of non-speech sounds to robotic storytelling is relatively unexplored. The researchers aimed to determine whether incorporating these elements could improve the overall storytelling experience, enhance emotional engagement, and reduce listener fatigue.

Sound Effects and Music Enhance Storytelling Experience

The findings of the research suggest that the integration of sound effects and background music into robotic storytelling can indeed enhance the experience, though the extent of this improvement varied depending on the story genre. In particular, romantic stories saw an increase in enjoyment when non-speech sounds were added, suggesting that sound effects and music helped to build the emotional atmosphere necessary for these types of stories. Horror stories appeared to benefit the most from the addition of sounds, with participants reporting a stronger emotional response and higher levels of engagement. This is perhaps not surprising, given that sound effects and music have long been used in horror films to heighten tension and induce fear. The study also found that the addition of non-speech sounds tended to reduce listener fatigue across all genres, indicating that sound effects and music may help maintain audience attention and energy throughout the storytelling experience.

Varying Impact of Sound in Different Genres

Interestingly, while romantic and horror stories showed clear benefits from the addition of sound, the results were more mixed for the detective and humorous genres. In the case of detective stories, participants in the sound-enhanced condition actually reported lower levels of surprise, a key emotional response in this genre. This suggests that the addition of sound may have undermined the buildup of suspense that is crucial to detective narratives. Similarly, in the humorous genre, the addition of sound effects and music did not significantly enhance participants’ enjoyment or emotional engagement. This may be due to the nature of the jokes used in the study, which were described as flat or simplistic. The researchers suggest that more complex or intelligent humor, as well as the use of robots capable of expressing facial emotions, could produce different results. Despite these mixed outcomes, the study provides valuable insights into how sound can be used to enhance robotic storytelling and highlights the need for further research into the role of genre in determining the effectiveness of non-speech sounds.

Robotic Perception Altered by Sound Effects

Another key finding from the study is the impact of sound on participants’ perceptions of the robot itself. In the horror story condition, the robot was perceived as more anthropomorphic, meaning it appeared more human-like to participants when sound effects and music were added. However, this increased anthropomorphism came at a cost, as participants also rated the robot as less safe when sound was included, possibly due to the fear-inducing nature of horror stories. This suggests that while sound effects and music can enhance the storytelling experience, they may also influence how the robot is perceived in ways that depend on the emotional tone of the story.

Challenges in Online Study Setup

One of the limitations of the study is that it was conducted online, which may have affected how participants experienced the robot and the storytelling process. The researchers acknowledge that in-person interactions with a robot may produce different results, particularly in terms of how the robot is perceived. Additionally, the sound effects and music were played through the robot, but participants may not have always associated these sounds with the robot itself, particularly in an online setting where the audio quality and volume could not be controlled. Future studies could explore these variables further by conducting in-person experiments and examining whether participants attribute the sound to the robot in the same way they would in a live setting.

Future Research in Robotic Storytelling

Overall, the study suggests that sound effects and background music hold significant potential for enhancing robotic storytelling, particularly in genres like horror and romance. However, the effectiveness of these non-speech sounds appears to depend heavily on the genre of the story being told, with some genres benefiting more than others. The researchers recommend further exploration into how different types of sounds can be used to enhance the storytelling experience across various genres, as well as the development of guidelines for selecting the most appropriate sound effects and music for specific types of stories. By doing so, future research could help unlock the full potential of robotic storytelling and provide new ways to engage audiences through the use of sound.