
Living Spec: A Controlled Experiment Shows the Spec File Is 54x Cheaper — But a Stale Spec Is a Single Point of Failure
Vũ Khôi
Tác giả tại Wotech Electronics Blog

Living Spec: A Controlled Experiment Shows the Spec File Is 54x Cheaper — But a Stale Spec Is a Single Point of Failure
I just made my own AI prove a hot method, and the result made me revise my own understanding. That is the sharing of Cáo Nhăn Nhó in the Muse AI Việt Nam group (19/11/2026) about the Living Spec method — a controlled experiment the author designed and ran himself. All figures, procedures, and views in this article follow the post and have not been independently verified by Wotech.
Quick answer: According to the post, there is a shared method called Living Spec, roughly: don't make AI re-read a long chat history anymore; distill the project's state into a short file and pin it. It sounds very reasonable, but reasonable is not necessarily right, so the author made it run a proper controlled experiment. The three most valuable findings: one, on cost the method wins absolutely — reading the spec file costs only 970 input tokens, reading the whole history costs 52,767 tokens, about 54 times more for the same correct answer. Two, on accuracy it is not as magical as the original post says — the large model reading raw history still got 12 out of 12 right, but the small model dropped to 10 out of 12 in the dirtiest-history condition, wrong on exactly the two places where the final decision was only mentioned in passing; reading the spec stayed perfectly correct. Three, and this is the finding the author values most: the spec file is a single point of failure — he deliberately made 4 fields in the spec stale, and the results were wrong on exactly those 4 fields, unable to self-correct a single one; fresh spec means the method is right, stale spec means it propagates errors with absolute confidence.
A Living Spec is the method of distilling a project's state into a short file and pinning it, instead of making AI re-read a long chat history — per the post, the author ran a controlled experiment to test this method.
How was the experiment done?
According to the post, the method: build a fake project with 12 decisions, each with an old superseded version, add 10 other projects with identical structure as noise, chat history up to 52,767 tokens. Then have two models, one large and one small, answer 12 questions about the current state under four different conditions.
The three most valuable findings
According to the post, the results are:
| Finding | Experiment result, per the post |
|---|---|
| Cost | Reading the spec file costs only 970 input tokens, reading the whole history costs 52,767 tokens, about 54 times more for the same correct answer — the method wins absolutely |
| Accuracy | The large model reading raw history still got 12 out of 12 right; the small model dropped to 10 out of 12 in the dirtiest-history condition, wrong on exactly the two places where the final decision was only mentioned in passing; reading the spec stayed perfectly correct — not as magical as the original post says |
| Single point of failure | Deliberately making 4 fields in the spec stale made the results wrong on exactly those 4 fields, unable to self-correct a single one; fresh spec means the method is right, stale spec means it propagates errors with absolute confidence |
Apply now: use Living Spec because it is cheap — and keep the update discipline
According to the post, the author's conclusion: Living Spec is worth using, but not because it cures AI's memory loss. It is worth using because it is tens of times cheaper, it helps small models still do the job right, and it gives you a single source of truth to check. The price to pay is update discipline — forget to update and it is systematically wrong. The post's closing question for readers in the group: is anyone using this kind of state file for a real project, updating by hand or already automated?
Frequently asked questions
1. What is Living Spec? According to the post, Living Spec is the method of not making AI re-read a long chat history anymore, but distilling the project's state into a short file and pinning it.
2. How was the controlled experiment done? According to the post: build a fake project with 12 decisions, each with an old superseded version, add 10 other projects with identical structure as noise, chat history up to 52,767 tokens; then have two models, one large and one small, answer 12 questions about the current state under four different conditions.
3. How much cheaper is the spec file than reading the whole history? According to the post, reading the spec file costs only 970 input tokens, reading the whole history costs 52,767 tokens, about 54 times more for the same correct answer.
4. What about accuracy — do small models still get it right? According to the post, the large model reading raw history still got 12 out of 12 right; the small model dropped to 10 out of 12 in the dirtiest-history condition, wrong on exactly the two places where the final decision was only mentioned in passing; reading the spec stayed perfectly correct.
5. What is the biggest risk of Living Spec? According to the post, the spec file is a single point of failure: deliberately making 4 fields in the spec stale made the results wrong on exactly those 4 fields, unable to self-correct a single one; fresh spec means the method is right, stale spec means it propagates errors with absolute confidence. The price to pay is update discipline — forget to update and it is systematically wrong.
Source: post by Cáo Nhăn Nhó in the Muse AI Việt Nam group (19/11/2026) about the controlled experiment on the Living Spec method. All figures, procedures, and views in this article follow the post and have not been independently verified by Wotech.
Bài viết liên quan

20 tháng 11, 2026 · 7 phut doc
Muse Assistant Sues Meta Itself: Locked Out as a Suspected Hacker, Then It Drafted the $75,000 Lawsuit
A small business owner in South Carolina just used Meta's own Muse assistant to sue Meta, and this story is exactly the kind only the agent era could produce. That is the content of a post by Trần Mạ…

20 tháng 11, 2026 · 7 phut doc
Muse Connects to Google Calendar: A Calendar That Reports to AI, So Muse Knows About a Rescheduled Meeting Before You Do
One afternoon, a meeting with a supplier got rescheduled, and before you even opened your calendar to check, Muse already knew. Not because Muse guessed, but because your calendar is connected to it,…

20 tháng 11, 2026 · 7 phut doc
A 2-Person Team With 10+ Jobs Running Themselves Daily: Javis OS Replaces Repetitive Work, the Owner Just Reads Reports
Our team has exactly 2 people. But every day, more than 10 jobs run themselves, even while both of us are asleep. That is the sharing of Phạm Quảng Đại in the Sắc Màu Cuộc Sống Xa Quê group (20/11/20…