Disfluency as a Window into Cognitive Mediation: Psycholinguistic Metrics for Evaluating AI-Integrated Spoken Communication

Published: CHI Extended Abstracts 2026, Barcelona, Spain.

This paper proposes a psycholinguistic evaluation framework that uses speech disfluency patterns—filled pauses (um, uh), repetitions, repairs, and prolonged silences—as observable signatures of underlying cognitive processing during AI-integrated spoken conversation. While most evaluation metrics for conversational AI focus on output fluency or task success, disfluencies are systematic windows into speakers’ cognitive load, prediction failure, and repair work.

The paper maps four classes of disfluency to four speech-processing stages (perception, lexical access, content integration, production planning) and proposes how each can be measured to evaluate whether AI mediation alleviates or amplifies cognitive load at specific stages. This framework supports more nuanced evaluation of AI tools designed for non-native speakers and others who experience predictable cognitive bottlenecks during real-time conversation.

Recommended citation: He, W. P. (2026). "Disfluency as a Window into Cognitive Mediation: Psycholinguistic Metrics for Evaluating AI-Integrated Spoken Communication." Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems. Barcelona, Spain.
Download Paper