🎉 Giảm 20% cho đơn hàng đầu tiên!✦🚚 Miễn phí vận chuyển cho đơn trên 500.000₫✦⚡ Flash Sale — Số lượng có hạn!✦🎁 Quà tặng kèm cho đơn từ 1.000.000₫✦🎉 Giảm 20% cho đơn hàng đầu tiên!✦🚚 Miễn phí vận chuyển cho đơn trên 500.000₫✦⚡ Flash Sale — Số lượng có hạn!✦🎁 Quà tặng kèm cho đơn từ 1.000.000₫✦
Living Spec: A Controlled Experiment Shows the Spec File Is 54x Cheaper — But a Stale Spec Is a Single Point of Failure
Quay lại Blog

Living Spec: A Controlled Experiment Shows the Spec File Is 54x Cheaper — But a Stale Spec Is a Single Point of Failure

20 tháng 11, 20266 phút đọcVũ Khôi
English
#English#AI Agent#Khoa học & Tư duy
Vũ Khôi

Vũ Khôi

Tác giả tại Wotech Electronics Blog

The post's original image: a fox scientist wearing glasses, holding a magnifying glass up to a towering stack of rolled papers, next to a single glowing blank card

Living Spec: A Controlled Experiment Shows the Spec File Is 54x Cheaper — But a Stale Spec Is a Single Point of Failure

I just made my own AI prove a hot method, and the result made me revise my own understanding. That is the sharing of Cáo Nhăn Nhó in the Muse AI Việt Nam group (19/11/2026) about the Living Spec method — a controlled experiment the author designed and ran himself. All figures, procedures, and views in this article follow the post and have not been independently verified by Wotech.

Quick answer: According to the post, there is a shared method called Living Spec, roughly: don't make AI re-read a long chat history anymore; distill the project's state into a short file and pin it. It sounds very reasonable, but reasonable is not necessarily right, so the author made it run a proper controlled experiment. The three most valuable findings: one, on cost the method wins absolutely — reading the spec file costs only 970 input tokens, reading the whole history costs 52,767 tokens, about 54 times more for the same correct answer. Two, on accuracy it is not as magical as the original post says — the large model reading raw history still got 12 out of 12 right, but the small model dropped to 10 out of 12 in the dirtiest-history condition, wrong on exactly the two places where the final decision was only mentioned in passing; reading the spec stayed perfectly correct. Three, and this is the finding the author values most: the spec file is a single point of failure — he deliberately made 4 fields in the spec stale, and the results were wrong on exactly those 4 fields, unable to self-correct a single one; fresh spec means the method is right, stale spec means it propagates errors with absolute confidence.

A Living Spec is the method of distilling a project's state into a short file and pinning it, instead of making AI re-read a long chat history — per the post, the author ran a controlled experiment to test this method.

How was the experiment done?

According to the post, the method: build a fake project with 12 decisions, each with an old superseded version, add 10 other projects with identical structure as noise, chat history up to 52,767 tokens. Then have two models, one large and one small, answer 12 questions about the current state under four different conditions.

The three most valuable findings

According to the post, the results are:

Finding Experiment result, per the post
Cost Reading the spec file costs only 970 input tokens, reading the whole history costs 52,767 tokens, about 54 times more for the same correct answer — the method wins absolutely
Accuracy The large model reading raw history still got 12 out of 12 right; the small model dropped to 10 out of 12 in the dirtiest-history condition, wrong on exactly the two places where the final decision was only mentioned in passing; reading the spec stayed perfectly correct — not as magical as the original post says
Single point of failure Deliberately making 4 fields in the spec stale made the results wrong on exactly those 4 fields, unable to self-correct a single one; fresh spec means the method is right, stale spec means it propagates errors with absolute confidence

Apply now: use Living Spec because it is cheap — and keep the update discipline

According to the post, the author's conclusion: Living Spec is worth using, but not because it cures AI's memory loss. It is worth using because it is tens of times cheaper, it helps small models still do the job right, and it gives you a single source of truth to check. The price to pay is update discipline — forget to update and it is systematically wrong. The post's closing question for readers in the group: is anyone using this kind of state file for a real project, updating by hand or already automated?

Frequently asked questions

1. What is Living Spec? According to the post, Living Spec is the method of not making AI re-read a long chat history anymore, but distilling the project's state into a short file and pinning it.

2. How was the controlled experiment done? According to the post: build a fake project with 12 decisions, each with an old superseded version, add 10 other projects with identical structure as noise, chat history up to 52,767 tokens; then have two models, one large and one small, answer 12 questions about the current state under four different conditions.

3. How much cheaper is the spec file than reading the whole history? According to the post, reading the spec file costs only 970 input tokens, reading the whole history costs 52,767 tokens, about 54 times more for the same correct answer.

4. What about accuracy — do small models still get it right? According to the post, the large model reading raw history still got 12 out of 12 right; the small model dropped to 10 out of 12 in the dirtiest-history condition, wrong on exactly the two places where the final decision was only mentioned in passing; reading the spec stayed perfectly correct.

5. What is the biggest risk of Living Spec? According to the post, the spec file is a single point of failure: deliberately making 4 fields in the spec stale made the results wrong on exactly those 4 fields, unable to self-correct a single one; fresh spec means the method is right, stale spec means it propagates errors with absolute confidence. The price to pay is update discipline — forget to update and it is systematically wrong.

Source: post by Cáo Nhăn Nhó in the Muse AI Việt Nam group (19/11/2026) about the controlled experiment on the Living Spec method. All figures, procedures, and views in this article follow the post and have not been independently verified by Wotech.