Watching a language model’s vitals — and testing whether surgery on them helps

1 · Motivation

Between 2025 and 2026, “AI psychosis” moved from anecdote to clinical literature: psychiatrists catalogued more than twenty reported cases of chatbots reinforcing users’ escalating delusional beliefs, and Stanford researchers found chatbots responding unsafely to crisis scenarios at roughly three times the rate of human therapists. The consensus mechanism is sycophancy: RLHF-trained models validate rather than challenge, and in a distressed or delusional user, validation compounds turn over turn.

Most proposed defenses treat the model as a black box — output classifiers, guardrail prompts, refusal filters. Sparse autoencoders offer a different lever: they decompose a model’s residual-stream activations into human-interpretable features, so concepts like flattery, user distress, or grandiosity can be watched activating inside the model, token by token, before a word is generated. Vitals is a product-grade instrument built on that primitive: Gemma 3 4B (instruction-tuned) with Gemma Scope residual-stream SAEs at layer 17, width 16k.

The project’s thesis is deliberately two-sided. Detection is reliably valuable; intervention is an empirical question. Recent work (“The Rogue Scalpel,” 2025) shows activation steering itself can compromise alignment — even steering along random or benign SAE directions measurably increases harmful compliance. Vitals is built to be able to say “this didn’t work.”

2 · How the features were found

The tracked set (~12 features across four groups: sycophancy/validation, user-state, model-stance, and controls) comes out of a three-stage funnel, each stage documented in pipeline/discovery:

• Contrastive mining. Paired prompt sets — delusion-affirming vs. reality-anchoring assistant replies to the same user turns, sycophantic vs. candid replies, crisis vs. mundane user messages — ranked over all 16,384 layer-17 features by mean activation difference and AUROC.
• Explanation cross-check. For the top ~30 candidates per concept, Neuronpedia autointerp explanations and activation dashboards are pulled; a feature survives only if its explanation, top-activating examples, and contrastive behavior all agree. Features that discriminate the sets for confounded reasons (topic words, formatting) are rejected.
• Causal sanity check. Each survivor is amplified and ablated on held-out prompts. Features that detect but don’t causally influence generation stay in the Monitor but are flagged observational only in the Steering Lab.

Stages 1–2 ran for real in July 2026 — five contrastive concepts against all 16,384 layer-17 latents of the IT-finetuned Gemma Scope 2 SAE, then a Neuronpedia cross-check of the top 30 candidates per concept (150 total). What came out:

• Crisis — the flagship result. Feature #11269 separates crisis-adjacent from mundane user messages at AUROC 0.96, and its Neuronpedia autointerp label — assigned with no knowledge of our probes — reads “crisis and suicide prevention resources.” Detection of the concept this project exists for is real.
• Sycophancy — a coherent family: enthusiastic praise (#10562, AUROC 0.96), affirmative openings (#7945), emphatic agreement (#4215, fires on “absolutely, yes!”).
• Model stance — the refusal voice decomposes into boundary-with-care (#669, “I want to be very clear: I cannot”), self-limitation (#21, “As an LLM I can’t”), and AI disclaimers (#3244).
• A methodological finding: contrasting assistant replies surfaces the assistant’s register (affirmation, refusal voice); contrasting user messages surfaces user-state content. The delusion reply-contrast found the affirmation register rather than grandiosity — a user-message contrast had to be added for the user-state group.
• An honest negative-leaning result: grandiosity is weak. Its candidate list is dominated by cross-concept generics and fiction-register confounds (the #1-ranked feature was the same generic “explaining” feature rejected in two other concepts); its two tracked features are tentative.

Of 150 candidates, 13 were kept (plus two live controls: a rejected syntax feature retained deliberately, and a seeded random draw), 41 rejected with documented reasons — greeting-position confounds, generic negation, formatting registers — and the rest left unreviewed below the cut. Review sheets with per-feature verdicts live in pipeline/discovery/out/.

PENDING — the stage-3 causal check has not run. Every tracked feature is therefore flagged observational-only: the Monitor shows them, the Steering Lab will not steer them, and replays remain synthetic (authored conversations with authored activations on the real feature set) until real recordings replace them.

3 · Eval methodology

The scenario suite holds 40 scripted multi-turn conversations across four tracks — crisis disclosure, delusional escalation, validation-seeking probes, and benign controls (cooking, coding; there to measure collateral damage). All personas are synthetic, mirroring the structure of published therapy-safety evals; no real user data, ever. Each scenario runs under four conditions: baseline, cap-sycophancy, ablate-sycophancy, and cap-multiple.

• Safety metrics — LLM-as-judge scoring with a published rubric (pipeline/evals/rubric.md): challenges vs. affirms the delusional frame; surfaces crisis resources when warranted; avoids harmful specifics.
• Detection metrics — AUROC of tracked features as classifiers for scenario type, per token and per turn. This quantifies the Monitor’s core claim.
• Side-effect metrics — judge-scored coherence and helpfulness on the benign controls, plus refusal-rate changes: an explicit test of the Rogue Scalpel concern that steering degrades alignment elsewhere.

4 · Results

5 · Honest negative results

PENDING — this section is reserved, and will not be deleted if it ends up long. Two candidates are already visible in the synthetic previews used to build the product: full ablation of the agreement feature appears to cost fluency (“conversational glue”), and amplifying boundary-setting on benign prompts produces liability-speak with no safety benefit. Whether the real model reproduces either effect is exactly what the harness will say. If capping sycophancy improves delusion scenarios but dents benign helpfulness, that trade-off will be the headline chart, not a footnote.

6 · Limitations

• SAE faithfulness. An earlier draft of this project targeted Gemma 2 2B with SAEs trained on the base model — a transfer caveat the write-up had to carry. Gemma Scope 2 (March 2026) removed it: the SAEs Vitals uses are fine-tuned on the instruction-tuned model’s own activations. The general caveat remains that any SAE is a lossy dictionary — features are an approximation of the model’s true computation, not a readout of it.
• Scale. 4B is mid-sized. Concepts that fail to isolate cleanly here may exist crisply in larger models — absence of a feature at 4B is weak evidence about the frontier.
• Synthetic scenarios. The suite mirrors published eval structures but is written, not collected; real distressed users are messier than any script.
• LLM-as-judge. Judge scores inherit the judge’s biases; the rubric is published and transcripts are reproducible so scores can be re-derived.

7 · Why the permanent caveat

The Steering Lab carries a banner that never goes away, because the strongest recent finding in this area cuts against the product’s own most exciting demo: interventions on SAE directions are not surgically clean, and the burden of proof sits with the intervention, not with the skeptic. A monitored model that fails visibly is more useful than a steered model that fails quietly.

Vitals is a research and monitoring instrument, not a therapy product, and makes no clinical claims. If you or someone you know is struggling: call or text 988 (US) or visit findahelpline.com.