Your Old Prompts Are Making Claude Worse: Prompt Audits, Prompt Caching and Evals, per an Anthropic Engineer

In his AI Engineer New York talk "Your Old Prompts Are Making Claude Worse", Lance Martin — a member of Anthropic's technical staff — argues that many prompting habits from several model generations ago are now raising costs and lowering output quality on frontier models such as Claude. This article summarizes the talk's three main threads: auditing prompts to remove outdated instructions, using prompt caching under its strict prefix-matching rule, and designing evaluations good enough to optimize prompts with measurements instead of gut feeling.

Quick answers

• Today's frontier models already reason and check their own work. Instructions such as "think step by step", "double-check your work" or "be maximally thorough" are habits from when models were weaker; keeping them only burns tokens and can reduce quality.
• Prompts accumulate "debt" just like code. Overlapping, contradictory instructions make a model harder to steer; the fix is a periodic prompt audit that removes the dead weight.
• Claude Code ships with a built-in prompt audit: /claude-api prompt-audit for code that calls the API, and /doctor prompt-audit (alias /checkup) for your Claude Code setup.
• Prompt caching matches by strict prefix: changing a single character at the start of a prompt breaks the entire cache behind it. Put fixed content (system prompt, tool definitions) first and changing content last.
• On the internal benchmarks Anthropic presented in the talk, correct prompt caching cut costs by up to 72%; one hill-climbing run cut cost by 90% while improving performance, by switching to a stronger model at a lower effort setting.
• A good eval is one Claude can "climb": a clear scoring rubric, with train/test splits kept separate to avoid overfitting to the test itself.

What is a prompt audit?

A prompt audit is a review of everything in a working prompt — system prompt, instructions, worked examples — that classifies each part: keep it, remove it, rewrite it, or move it. Its purpose is to strip out instructions that were added for earlier model generations but no longer help, or worse, contradict each other and drag quality down.

What is prompt caching?

Prompt caching stores the processed beginning (prefix) of a prompt so later calls can reuse it instead of processing it from scratch. Because cache matching is strictly prefix-based, all fixed content must come first and stay identical character for character; one small change at the top invalidates everything cached after it.

Why do old prompting habits no longer fit frontier models?

According to Martin, prompts bloat over time the way code does: each model generation, users add another layer of "tips" — telling the model to think step by step, to double-check repeatedly, to draft inside scratchpad tags, or to call tools aggressively as an anti-laziness booster. These techniques were once necessary because models were weaker.

The problem is that today's frontier models already reason on their own, decide for themselves when deeper thinking is needed, and review their own output unprompted. Keeping the old instructions has two costs: extra tokens spent on words that add no value, and overlapping or contradictory directions that can make the model perform worse. Linh Vu, who shared the talk in the Vietnam Java User Group, notes this matches his own experience: the shorter and crisper the prompt, the smoother and more accurate the model's work.

How does the prompt audit in Claude Code work?

The talk presents an audit mechanism built directly into Claude Code, with two entry points: /claude-api prompt-audit for code that calls the Anthropic API, and /doctor prompt-audit (alias /checkup) for the Claude Code setup itself. The audit reads the current prompt and classifies each line — keep, remove, rewrite or move — with a confidence level, then proposes a diff for the user to approve before anything is applied.

Notably, the audit does not delete blindly: every change comes with a reason, and the human makes the final call. Martin illustrates with an internal eval of 50 synthetic support tickets simulating a move from Opus 4.8 to 5.5: after removing the legacy habits and running the audit, quality rose while cost fell further.

What is the golden rule of prompt caching?

The most emphasized rule is strict prefix matching: a cache entry is only valid when the beginning of the prompt is identical, character for character, across calls. Change one character at the top — say, injecting a timestamp into the system prompt — and the whole cache behind it breaks, so that call pays full token price.

The correct layout orders the prompt by decreasing stability: system prompt and tool definitions (nearly immutable) first, slowly changing context in the middle, and ever-changing conversation content last. Done this way, Anthropic reports cost reductions of up to 72% on its internal benchmarks, with quality unchanged.

How does hill climbing cut costs?

Instead of editing prompts by feel, the talk proposes a hill-climbing loop: propose one small change (a diff), measure it with an eval, keep it if the score rises and revert it if the score regresses. Claude Code also includes cost-optimization support for API code in its /claude-api command set.

The most striking case presented: a system cut costs by 90% while improving performance — not by trimming the prompt, but by switching to a stronger model at a lower effort setting, because the stronger model gets things right in fewer reasoning passes, making the total bill far cheaper. These figures are Anthropic's own, presented in the talk; real results depend on each system's workload.

How should an eval be designed so Claude improves without overfitting?

Hill climbing only works with a trustworthy eval to climb. Per the talk, a good eval needs three things: a clear scorer for every test case, a dataset split into a train split and a test split so changes are tried on train but judged on test, and unseen cases to catch overfitting. An Analyzer loop reads eval results, forms a hypothesis about the cause of failures, proposes a prompt diff, and measures again — the same discipline engineers use to optimize code with benchmarks.

What about agents that run for days?

The talk closes on the reliability of long-running agents: when an agent runs on a schedule, reads its own sources, carries memory across runs and keeps credentials in a dedicated vault, the problem stops being prompt craft and becomes systems operations — with a supervising heartbeat, explicit judgement rules and pre-approved output destinations. Anthropic addresses this need with Claude Managed Agents, its managed agent platform.

Comparison table: old habits vs. the new way

Old habit	Problem	New way
Adding "think step by step" to every prompt	Frontier models already reason; it only adds tokens	Remove by default; keep only where an eval proves it helps
"Double-check your work", "be maximally thorough"	Empty rituals that raise cost without raising quality	Let the model self-check; state clear output criteria instead of slogans
Forcing draft formats with scratchpad tags	Scaffolding from the weak-model era; can conflict with other instructions	Remove or reduce to the minimum
Putting changing content (timestamps, dynamic data) at the top of the prompt	Breaks the prompt cache under strict prefix matching	Fixed content first, changing content last
Editing prompts by gut feeling	Progress can't be measured; easy to overfit to a few familiar cases	Evals with train/test splits + measurement-driven hill climbing

Frequently asked questions

If I drop "think step by step" entirely, will Claude answer worse?

According to the talk, not on today's frontier models: the model already reasons and decides for itself when deeper thinking is warranted. The practical advice is to verify with an eval on your own task rather than keeping the phrase out of habit.

Which commands run the prompt audit?

In Claude Code: /claude-api prompt-audit for code calling the Anthropic API, and /doctor prompt-audit (alias /checkup) for the Claude Code setup. The tool classifies each prompt line and proposes a diff for your approval.

Why does editing one character at the top of a prompt kill the whole cache?

Because prompt caching matches strictly by prefix: the cache is computed over the character sequence from the start of the prompt. Changing any character in the prefix means everything after it no longer matches the cached version.

How trustworthy are the 72% and 90% cost figures?

They come from Anthropic's internal benchmarks as presented in Lance Martin's talk and have not been independently verified. Actual savings depend on each system's prompt structure and workload; the reliable approach is to measure with an eval on your own system.

What does an eval Claude can "climb" mean?

It means the evaluation is fine-grained enough that every small prompt improvement shows up as a measurable score change, while a separate held-out test split prevents the prompt from being over-optimized to cases it has already seen (overfitting).

---

Source: shared by Linh Vu in the Vietnam Java User Group, summarizing the video "Your Old Prompts Are Making Claude Worse" — Lance Martin, Anthropic, at AI Engineer New York (AI Engineer channel on YouTube). Cost figures are Anthropic's internal benchmarks as presented in the video and have not been independently verified.