The Partner Modelling Questionnaire: A validated self-report measure of perceptions toward machines as dialogue partners
Doyle, Philip R.; Gessinger, Iona; Edwards, Justin; Clark, Leigh; Dumbleton, Odile; Garaialde, Diego; Rough, Daniel; Bleakley, Anna; Branigan, Holly P.; Cowan, Benjamin R. (2025-04-17)
ACM
17.04.2025
Philip R. Doyle, Iona Gessinger, Justin Edwards, Leigh Clark, Odile Dumbleton, Diego Garaialde, Daniel Rough, Anna Bleakley, Holly P. Branigan, and Benjamin R. Cowan. 2025. The Partner Modelling Questionnaire: A validated self-report measure of perceptions toward machines as dialogue partners. ACM Trans. Comput.-Hum. Interact. Just Accepted (April 2025). https://doi.org/10.1145/3729170
https://rightsstatements.org/vocab/InC/1.0/
© Owner/Author 2025. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM transactions on computer-human, http://dx.doi.org/10.1145/10.1145/3729170.
https://rightsstatements.org/vocab/InC/1.0/
© Owner/Author 2025. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM transactions on computer-human, http://dx.doi.org/10.1145/10.1145/3729170.
https://rightsstatements.org/vocab/InC/1.0/
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:oulu-202505093214
https://urn.fi/URN:NBN:fi:oulu-202505093214
Tiivistelmä
Abstract
Recent work has looked to understand user perceptions of speech agent capabilities as dialogue partners (termed partner models), and how this affects user interaction. Yet, partner model effects are currently inferred from language production as no metrics are available to quantify these subjective perceptions more directly. Through three phases of work, we develop and validate the Partner Modelling Questionnaire (PMQ): an 18-item self-report semantic differential scale designed to reliably measure people’s partner models of non-embodied speech interfaces. Through confirmatory factor analysis, we confirm that the PMQ scale consists of three factors: communicative competence and dependability, human-likeness in communication, and communicative flexibility. Our studies show that the measure consistently demonstrates good internal reliability, strong test-retest reliability over 4- and 12-week intervals, and predictable convergent/divergent validity. Based on our findings, we discuss the multidimensional nature of partner models, whilst identifying key future research avenues that the development of the PMQ facilitates. Notably, this includes the need to identify the activation, sensitivity, and dynamism of partner models in speech interface interaction.
Recent work has looked to understand user perceptions of speech agent capabilities as dialogue partners (termed partner models), and how this affects user interaction. Yet, partner model effects are currently inferred from language production as no metrics are available to quantify these subjective perceptions more directly. Through three phases of work, we develop and validate the Partner Modelling Questionnaire (PMQ): an 18-item self-report semantic differential scale designed to reliably measure people’s partner models of non-embodied speech interfaces. Through confirmatory factor analysis, we confirm that the PMQ scale consists of three factors: communicative competence and dependability, human-likeness in communication, and communicative flexibility. Our studies show that the measure consistently demonstrates good internal reliability, strong test-retest reliability over 4- and 12-week intervals, and predictable convergent/divergent validity. Based on our findings, we discuss the multidimensional nature of partner models, whilst identifying key future research avenues that the development of the PMQ facilitates. Notably, this includes the need to identify the activation, sensitivity, and dynamism of partner models in speech interface interaction.
Kokoelmat
- Avoin saatavuus [37957]