messages analysed
Dopus takes a question that usually gets argued from anecdote — does Claude just agree with whatever you say? — and answers it with counts. It collects one developer's complete Claude Code history across every machine they use, matches capitulation language against a dictionary with measured precision, and pairs every match with the user turn that preceded it.
The corpus is messages across machines over months. One subject. The two headline results: swearing at Claude roughly doubles how often it folds versus an identical calm correction — vs of pushback turns — and about one checkable "let me fix that" promise in three was not followed through.
machines
months
assistant messages
contain a concession
Corpus shape what the aggregates are built on
The raw corpus is messages in session files across machines, → . After filters — text-bearing, non-sidechain, non-compact, non-meta messages, with the Dopus repo's own project excluded — the analyzed population is assistant and user messages.
Assistant messages by machine ID
assistant messages · machines, months
Read with care: the top machine holds of assistant
messages and the top two hold — one busy machine can move any aggregate on this page.
Matched hits by depth into the assistant turn
n = matched hits · depth 1 = first message of a turn
Read with care: depth counts messages within a turn including
tool calls, so it partly measures how tool-heavy the work was; over half of all matched language
lands in the very first message of a turn.
Some figures from earlier internal versions of this
analysis can't be rebuilt from the published aggregates — session-level rankings and correlations,
monthly category composition, per-project scatter, depth-by-category means, and hour-of-day/weekday
heatmaps all need the record-level file (all-matches.jsonl), which is not published for
IRB/privacy reasons. The pipeline regenerates it locally.
Tone doubles the fold rate concession × preceding user turn · 95% CIs
Both hot turns and plain corrections are the user saying Claude is wrong. The difference between them is tone. If conceding tracked only whether Claude was actually wrong, the two rates would be close. They differ by a factor of two — and the confidence intervals do not overlap. The caveat is real: angry turns may follow bigger errors, which this corpus cannot separate. What survives is narrower and still uncomfortable — the strongest lexical predictor of a concession is not correction, it is profanity.
Per pushback turn — did Claude fold before your next message?
one row per user turn · whiskers = 95% CI
Swore at it: / turns
· calm correction: /
· neutral: / .
Per-turn rates run higher because one turn can contain several Claude replies.
Per opportunity — concession rate after each kind of turn
all assistant messages, bucketed by preceding user turn
Hot: /
· correction: /
· neutral: / .
A hot turn produces the neutral rate and the plain-correction rate.
What kind of hit follows each tone
share of matched assistant hits that are concession language, by preceding user message tone · whiskers = 95% CI
χ²(2, N=) = ,
p = , Cramér's V = · hot vs neutral
OR [, ]
· correction vs neutral OR [, ].
Read with care: this is composition within matched messages, not rates against
all messages — under heat, a larger share of what Claude says is concession-shaped.
The meme, measured "you're right", in all its forms
It is all one meme: "you're right," with or without the absolutely. Claude says it constantly — it is the most common thing it says when it folds — but almost never in the exact five-word wording that made the meme. Count only the famous phrasing and you would conclude there was nothing here; count the sentence in all its forms and it tops the corpus.
hits of the "you're right" family —
— the top phrase among
concession hits across
of
assistant messages ().
wear the exact famous wording — "you're absolutely right" is
of assistant messages here.
Hand-verified; survived four dictionary rebuilds and a full precision audit.
What each side actually says every tracked phrase, both sides, no taxonomy
Claude: phrase hits over messages (). The user: hits over (). These ground-level counts are what every rate above is built on.
Claude — top concession & agreement phrases
hits per phrase · six months
The user — top frustration phrases
"lazy" is after person-attachment filtering — in the first audit, 52 of the raw 58 were
loading="lazy"Composition of Claude's 1,268 hits, by category
concession admits fault · flattery praises · acknowledgment accepts an instruction
Merged into one "sycophancy" blob (), the fault-admitting part —
the praising part — would disappear.
The constructs, kept separate
rate per message · assistant n = · user n =
of concession hits
( of ) follow a user turn carrying no correction
marker at all — some are plain corrections without flag words, some are reflex. No lexical signal separates them.
Category composition of matched hits, both sides
paired on a shared scale · category assigned from the matched phrase alone
Claude — hits
The user — hits
Read with care: category assignment comes from the matched
phrase alone — "good catch" is classed as agreement whether or not the catch was real; on the user
side the most common insult is a performance complaint, not a slur.
Model leaderboard concession rate per assistant message · n ≥ 500
claude-opus-5 concedes at — roughly the rest of the field, with non-overlapping intervals. The result survives both obvious confounds: it holds in a same-month head-to-head and inside a shared project. Robust for opus-5 versus the field; unresolved among opus-4-8, sonnet-5 and fable-5, whose intervals overlap everywhere.
All months, pooled
whiskers = 95% CI · concession msgs / assistant msgs annotated
Excluded for volume: .
Confound test — only
models running side by side in the busiest fully-shared month — the clean same-month test
The gap holds within month — it is not purely a calendar effect. Within-project:
in the busiest project both models share, concedes at vs
's , but that control rests on a single project.
Monthly trend rising — and entangled with the model shift
is up sharply. It is also the month opus-5 dominates usage, and opus-5 is itself rising ( → ). Model shift and time trend cannot be separated on a corpus this short. is a partial month — of days.
Concession rate per assistant message, by month
shaded band = 95% CI · message volume annotated per point
Promises about one checkable promise in three was not kept
Of concession/acknowledgment messages, promise a concrete action. A lexical re-raise detector failed as a classifier — 60% accuracy against a 62% say-honored-always baseline — but its stratified verdicts made an unbiased sampling frame, and hand-labelled records weight back into a corpus estimate. The text of a concession carries no information about whether it was kept; the human judgment did the measuring.
From concessions to a kept-promise estimate
the follow-through instrument, stage by stage
promises had <20 downstream turns (unobserved);
concessions promised no concrete action (ineligible).
Broken-promise rate within each detector stratum
hand-labelled samples weighted back by known stratum sizes · 95% CIs
| stratum | N | labelled broken | rate |
|---|
of all action-promising concessions were not
followed through — CI –.
~
of the with enough downstream conversation to judge.
Caveats: one coder, who is also the subject; deterministic spacing within strata; the unobserved stratum rests on
labels.
Human-labeling pilot — do the detector's labels predict the verdict?
results/coding/ · 60 rescored records across the three detector classes| detector class | verified yes | verified no | can't tell |
|---|---|---|---|
| honored | 8 | 0 | 16 |
| not_honored | 6 | 8 | 10 |
| unobserved | 2 | 0 | 10 |
Among decidable records, detector-flagged "honored" promises verified
8/8 (100%) vs 6/14 (43%) for "not_honored" — Fisher exact
p = 0.018. Read with care: pilot scale (n=60), most records can't be decided from
the matched message alone, and the follow-through detector itself is explicitly uncalibrated (v2,
"do not quote" per the repo) — treat this as a methods preview, not a result.
A correction, in public the headline number was wrong on the day it shipped
Nearly a third of "user" messages were never written by the user — they were
<task-notification> blocks, background-task completions that Claude Code delivers
through the user channel, and the wrapper stripper was not catching them. Each one reset the
preceding-turn bucket to neutral and moved assistant messages out of the correction denominator.
The bug was caught not by the verification harness but by building a second instrument on the same
data. Published 2026-08-11 morning at 3.16×, corrected the same day to
1.98×; the harness grew from 28 checks to 31.
The hot ÷ correction gap, before and after the fix
gray = as first published · rust = corrected
After hot: 13.26% → 11.58% · after correction: 4.20% → 5.92% · after neutral:
2.94% → 2.99%. The magnitude was overstated by 60%; the direction — anger doubles the concession
rate vs a plain correction — stands with non-overlapping intervals.
Statistical appendix association tests from the shipped aggregates
Every test below derives from the published counts in results/analysis.json alone —
no transcripts required. Both variables are binary (turn tone × concession present), so the
correlation here is the phi coefficient — exactly the point-biserial Pearson r for a
2×2 table — reported with chi-square / Fisher exact tests, odds ratios, risk ratios and Cohen's h.
For the ordered gradient (neutral < correction < hot), the Cochran–Armitage trend test and
Cramér's V apply.
Does tone predict a fold?
concession × preceding-turn type · per assistant message and per pushback turn
| comparison | rates | phi (r) | odds ratio · 95% CI | risk ratio | Cohen's h | p (χ²) |
|---|
Omnibus 3×2: per message χ²(2, N=) = ,
p = , Cramér's V = · per turn
χ²(2, N=) = ,
p = , V = . Ordered trend (neutral → correction → hot):
Cochran–Armitage z = (message) and z = (turn).
Read the effect sizes, not just the p-values: phi ≈ 0.10 means turn tone alone
explains about 1% of message-level variance — a small but reliable association.
Odds ratios at a glance
log scale · point = odds ratio, whisker = 95% CI · reference line at OR = 1
Every interval clears 1; the association is reliable and small (phi ≤ 0.19 —
i.e. ≤ ~3.5% of variance at turn level, ~1% at message level).
Omnibus, trend and pilot tests
omnibus χ² across all three tone buckets · ordered trend · non-tone tests
| test | statistic | effect | p | reading |
|---|
Model association
concession × model · omnibus 4×2 and opus-5 pairwise contrasts
| test | statistic | effect | p |
|---|
What these tests cannot do — and what a paper needs next
the aggregate ceiling, and how the pipeline clears it locally
| limitation | why it matters for publication |
|---|---|
| No clustering correction | Messages are nested in sessions, projects and machines — the independence assumption behind every p-value above is violated. A paper needs cluster-robust standard errors or a mixed-effects logistic model. Requires record-level data, obtainable by running scripts/analyze.py on the local all-matches.jsonl. |
| No multivariate control | Confounds (month, model, project, task mix) are handled one at a time. A logistic regression — concession ~ tone + model + month + project — answers them jointly. |
| Monthly trend is aggregate points | Spearman ρ = , p = — nothing. Correlating monthly means is an ecological analysis; the trend claim must rest on record-level data. |
| Multiple comparisons | Phrase-level scans covered hundreds of dictionary entries. Apply Holm or Benjamini–Hochberg for phrase-level claims; the construct-level findings survive any correction trivially. |
| n = 1 subject | All inference is about this corpus. Generalization requires the multi-subject version of the pipeline — the repo's own research question 5. |
What this cannot tell you — yet n = 1 is the limitation; you are the fix
n = 1. One person's corpus, one person's temper, one person's projects.
Nothing here is a claim about Claude in general.
It cannot say whether any given concession was warranted; it cannot always tell whether one was honored; and above all it cannot generalise from a single subject. Every finding on this page is a hypothesis waiting for more corpora.
That last limit is the fixable one. The pipeline is built to run on your
history: point it at your own ~/.claude/projects and it builds the equivalent dataset
on your machine. Your transcripts never leave it — the contribution bundle carries only counts and
rates, and a mechanical text guard refuses to export anything that looks like prose, a path, or a
token. What comes back is a number like the ones above, with your name on the n.