How You Ask Shapes What You Get: A Theory-Seeded Measurement of Articulation in Advice-Seeking LLM Conversations

Baek, J., S. Lee, & D. Shin

Findings of the Association for Computational Linguistics: EMNLP, 2026

Forthcoming

Abstract

Users articulate the same advice-seeking request in different ways: some specify detailed constraints, others gesture at a vague need. Prior work treats this variation as noise to be averaged away; we instead treat it as a stable, measurable structure in the input distribution.

We ask whether articulation, meaning how people ask, forms latent dimensions separable from topic, meaning what they ask about, and whether it is associated with how language models respond. We extract interpretable features from 16,447 advice-seeking prompts pooled from public chat corpora (WildChat, LMSYS, and ShareChat) and recover a small set of latent articulation factors that replicate across train/test splits and across corpora.

Why This Question

Variation in how people ask is usually discarded

Existing work treats prompt variation either as noise to be removed, as in the paraphrase robustness literature, or as a problem of model behavior. Both framings sit downstream of a more basic question that has not been answered for naturalistic chat: does articulation variation in the wild form a stable structure separable from topic?

The answer determines what an evaluation can see

If articulation collapses onto topic, then varying prompts across topics already covers it and nothing is missed. If it is a separate dimension, evaluations that sample only across topics leave a systematic part of real usage unexamined.

Approach

Data

16,447 advice-seeking prompts pooled from WildChat, LMSYS, and ShareChat
Real user conversations rather than constructed prompt sets

Measurement

Interpretable features rather than opaque embeddings, so the factors can be read
Latent articulation factors recovered and checked for replication across splits and corpora
Separability from topic tested directly rather than assumed

Keywords

Human-AI Interaction Prompt Articulation LLM Measurement Advice Seeking Chat Corpora Latent Factor Models