Exploring consumer engagement in a multisensory human-chatbot interaction

Abstract

This thesis examines the effects of incorporating multimodal inputs, including text, image, and audio-visual elements, on consumer engagement during interactions with OuluBot, a chatbot designed for the City of Oulu's services. The study utilized surveys to analyze cognitive, emotional, and behavioral engagement. Word clouds were created from OuluBot discussions, and participant-submitted drawings were qualitatively assessed. Different analyses were performed for the control and experiment (video) groups to see how multimodal inputs affected engagement.

Regarding the results, incorporating multimodal inputs such as text, images, and audio-visual elements into interactions with OuluBot enhances consumer engagement by improving comprehension and making interactions more enjoyable. Videos, in particular, deepen users' understanding of OuluBot's functionality and maintain high cognitive engagement through effective information delivery. Emotionally, videos make interactions more interesting, leading to deeper and more sustained engagement, though the chatbot function slightly challenges this emotional connection. Behaviorally, there is a higher willingness to recommend OuluBot among users exposed to video content, suggesting that videos highlight features and provide context that enhance perceived usefulness and value.

According to the managerial implications, this study offers critical insights for developers of OuluBot and similar chatbot systems aimed at enhancing public city services. Employing multimodal interaction methods, including visual and auditory cues in marketing videos, can enhance consumer engagement by making the chatbot appear more human-like. Features such as facial expressions and human-like speech contribute to a lifelike portrayal, increasing consumer interaction. However, it is essential for managers to balance user expectations with the chatbot's actual capabilities, ensure consistent experiences, and address privacy concerns to optimize their sense of involvement.