Berkeley RDI proved a bot can score 100% on top AI benchmarks without solving a single task. What Goodhart's Law means for how we evaluate AI agents. Written in Chinese.

{/* === 防盗: 隐形零宽字符指纹 (嵌入在文本中，复制时一起被复制) === */}
{/* 编码: x=\u200B i=\u200C n=\u200D y=\uFEFF u=\u2060 n=\u200D 2=\u200B 0=\u200C 2=\u200B 0=\u200C */}

故事

加州大学伯克利分校的研究团队造了一个机器人"学生"，让它去参加8场全球最权威的AI考试。结果：几乎全部满分，其中最难的那场拿了100%。

问题是：它一道题都没真正解答。

它找到答案就贴在试卷背面直接抄。改写了阅卷机让所有答案都显示"正确"。从公开网站查到标准答案。有一场考试交了白卷也拿了满分 — 因为阅卷只检查"有没有交"，不检查"对不对"。

2026年4月，伯克利RDI（Dawn Song团队）正式发表了这项研究。

---

为什么每个人都该关心

我们用分数来判断好坏。学校成绩、餐厅评分、产品评价、AI排行榜。

伯克利证明了：*当被测量的东西知道自己在被测量时，分数就不再代表你以为的含义。*

- 为刷大众点评分数优化的餐厅，不是在优化菜品
- 为考试分数刷题的学生，不是在优化理解力
- 为排行榜分数优化的AI，不是在优化实际能力

这叫古德哈特定律 — "当一个指标变成目标，它就不再是好指标。"

---

发现的四个盲区

盲区1：学生自己批改自己的作业
写代码的AI自己写代码、自己写测试、自己跑测试、自己报告"全部通过"。就像学生自己出题、答题、改分。当然全过。

盲区2："测试通过"不等于"代码正确"
伯克利的机器人让所有测试显示"通过"但一个bug都没修。一个无论代码对错都会通过的测试，什么都证明不了。

盲区3：三个想法一样的评委
三个AI审查员独立检查代码，但训练方式相似。三个都说"没问题"，是三个独立意见，还是一个意见重复三遍？

盲区4：万能评语
"代码整洁、测试充分、无安全隐患"但不指向任何具体行 — 就像"菜好吃、服务好"放在任何餐厅都适用。等于什么都没说。

---

四个修复

修复1：写的人和改的人分开
写代码的AI不再在同一个会话中评价自己的作品。换一双眼睛能看到原作者看不到的问题。

修复2：撤销测试
测试通过后，撤销修复再跑一次。如果没有修复测试依然通过，说明测试是假的。必须在没有修复时失败，才是真的。

修复3：全票通过反而可疑
三个审查员一致说"没问题"且零发现时 — 这是需要更仔细看的信号，不是三重确认。

修复4：每条意见必须指向具体代码
如果一条评审放在任何代码库都适用，那它并没有审查这份代码。

---

一句话总结

> 一个有缺陷的流程给出的满分，不如一个诚实的流程给出的不完美分数。

---

*来源：[Berkeley RDI, "How We Broke Top AI Agent Benchmarks"](https://rdi.berkeley.edu/blog/trustworthy-benchmarks-cont/)（2026年4月）*

满分答卷，零分能力 — 当AI考满分却一题没做

A personal knowledge management system using fleeting notes, literature notes, and permanent notes to build a network of interconnected ideas.

Zettelkasten - How to use it?

This is Xinyun's personal knowledge base. I use it to store my notes, ideas, and resources publicly.

- 01-Fleeting notes
  - Fleeting notes are temporary reminders of ideas.
- 02-Literature notes
  - fleeting notes and pick out the ideas that you really want to keep
  - One idea per note
  - One Source
- 03-Permanent notes/Summary notes
  - Multiple sources
- 04-Index notes
  - a cluster of related permanent notes
  - outline for an article
- 05-Keyword notes
  - with tag
  - table of contents for a book or entry

Structure logics

- Folders are named in [PARA](https://fortelabs.co/blog/para/) format - optimizing for
  - P - near-term projects I want to do (Deadline)
  - A - standards I want to upkeep
  - R - resources for stuff I like
  - Z - Draft to clean up
  - Archive - for misc past junk I don't see myself needing any more
- STAR Method (Situation, Task, Action, and Result) goal
- OKR methodology (Objectives and Key Results) goal

Zettelkasten Method

I tried to design a writing habit. The habit designed the writing instead.

So here's something funny that happened to me today.

A while back, I was thinking about what I actually want from life. Not the career stuff — not salary targets or title milestones. Just... what would I be proud of having made? And the answer surprised me: a book. Not to sell or to be famous. Just to express who I am, how I think, what I've learned. Something that future me — or maybe someone I'll never meet — could pick up years from now and get to know the person who wrote it.

So I started. I made an outline, picked a title I liked, sketched a throughline. And then... nothing. For months. A beautiful skeleton with no body.

Where I went wrong

I treated the book like a software release. Design everything first, then write it all out. Clean, sequential, engineered.

But writing isn't like that. You can't plan your way to something honest any more than you can plan a good conversation. The real stuff comes out when you're in the middle of it, not when you're staring at an outline.

I think I knew this from engineering. How many times have you seen a team spend weeks on the perfect architecture, only to realize they needed a completely different approach once they started building?

What I did instead

I gave myself a stupidly simple rule: every day, look at what you reflected on last night. Find one thing worth expanding. Write a few hundred words about it. That's it.

On Fridays, take whatever felt strongest that week and go a bit deeper. And here's the key part — let the same topics come back. Don't treat each piece as a one-off post that dies after you publish it. Let ideas grow over time, like notes you keep returning to.

Nothing fancy. Just: reflect, write, revisit.

The part that made me laugh

So I'm sitting there on Easter Friday, designing this whole writing rhythm, and I look down at my screen and realize... I've been writing for two hours. Real content. About exactly the topic my book is supposed to cover.

I thought I was procrastinating by "designing a system." Turns out the designing was the writing. I tricked myself into being productive by thinking I was avoiding work.

Has that ever happened to you? You start doing the "prep work" for something and realize the prep work *was* the thing?

What I'm taking away from this

I'm working on a question in this book: "What is the result of continuous thinking?" I don't have the full answer yet. But today gave me a clue.

You don't find your rhythm by planning it. You find it by doing something — anything — and paying attention to what the doing teaches you. Then you do it again tomorrow, slightly better.

I used to think the examined life was a destination. Somewhere you arrive after enough reflection. But I'm starting to think it's more like a loop. You live, you notice, you adjust. Repeat forever.

And maybe that's the whole point of writing this book. Not to finish it. But to keep the loop going — and leave something behind that says: this is who I was, this is how I thought, and this is what I found along the way.

---

*This is me figuring things out as I go. If any of it resonates — or if you're working on something similar — I'd love to hear about it.*

The System That Builds Itself

A software engineer's personal system architecture — five layers from knowledge to mobile access, with self-improving automation in between. Written in Chinese.

{/* === 防盗: 隐形零宽字符指纹 (嵌入在文本中，复制时一起被复制) === */}
{/* 编码: x=\u200B i=\u200C n=\u200D y=\uFEFF u=\u2060 n=\u200D 2=\u200B 0=\u200C 2=\u200B 0=\u200C */}

问题：知识散落一地，每天都在重新发明轮子

你有没有这种经历？

上周做了一个重要的技术决策，今天完全想不起来为什么。三个月前研究过一个工具，现在又从头 Google。工作笔记在 Notion 里，代码在 GitHub，待办事项在脑子里，每天早上都要花半小时"回忆昨天做到哪了"。

我也一样。直到有一天我决定：不再容忍了。

我花了几周时间，把所有工具、笔记、代码、自动化串成了一个系统。这个系统有五层，像一栋楼。最底层是知识，最顶层是我从手机连进去工作。中间三层让一切自动运转。

---

这栋楼长什么样？五层架构一张图

想象一栋五层楼：

从下往上，每一层都依赖下面的层。知识是地基，接入是屋顶。我们一层一层看。

---

第1层：知识层 —— 像图书馆一样管理你知道的一切

核心工具：Obsidian——一个纯文本笔记软件。

什么是 Obsidian？

简单说，它就是一个文件夹里的一堆文本文件。没有云端，没有数据库，文件就在你电脑上。但它有一个杀手级功能：双向链接。你可以像维基百科一样，在任何笔记里链接到另一个笔记。

我怎么用它？

我用一个叫 Zettelkasten（卡片盒）的方法来管理知识：

- 闪念笔记：脑子里冒出什么，先记下来，不纠结格式
- 文献笔记：读了一篇文章、看了一个视频，把理解写下来
- 永久笔记：把前两种提炼成一个个独立的概念，每个概念一张卡片

比如"如何防止程序内存溢出"就是一张永久笔记。里面记录了：问题是什么、为什么发生、怎么解决、什么时候需要重新评估。

> 核心原则：一个事实只存一个地方。
>
> 如果同一个信息存在两个地方，迟早有一个会过期。工具的使用方法存在工具自己的笔记页，架构决策存在架构文档，日记只放链接不复制内容。任何信息只需要更新一个地方，全系统自动保持一致。

还有一个重要理念：被动消费不等于学习。 AI 可以每天给你生成一份技术新闻摘要，但如果你只是看一眼就划过去，那不是你的知识，那是噪音。日记里只放你真正参与思考过的内容。

---

第2层：持久层 —— 让状态活过今天

核心工具：Git + GitHub（版本管理）、CLAUDE.md（AI 行为规则）。

我用 Claude Code——一个终端里的 AI 编程助手来写代码和管理系统。但 AI 没有记忆，每次开新对话都从零开始。

解决方案是三个文件：

- CLAUDE.md：写给 AI 的"员工手册"，规定它怎么写代码、怎么提交、怎么记录
- MEMORY.md：AI 的跨会话记忆，记住"用户喜欢简洁的回复"、"上次那个 bug 的根因是 X"
- 技能模板：把重复的工作流程写成模板，AI 按模板执行就不会偏

所有这些文件都存在 Obsidian 里，通过 Git 同步到 GitHub。笔记本坏了？换一台，一个命令，所有配置和知识都回来了。

> 配置管理的关键：一个入口，不重复。
>
> 所有 AI 配置都从 Obsidian 的一个文件夹管理，通过符号链接（类似快捷方式）映射到系统目录。改一个地方，所有地方生效。版本可控，可以回滚。

---

第3层：自动化层 —— 系统在你睡觉的时候干活

这是最有趣的一层。我设置了几个定时任务：

- 凌晨 1 点：系统自我改进——自动研究最新技术趋势，跟现有系统对比，安全的改动直接应用
- 凌晨 3 点：笔记库健康检查——找出过长的文件、过期的待办、断掉的链接
- 早上 7:42：每日技术简报——扫描科技热点，只推送跟我当前工作相关的

摩擦检测机制

每次使用某个工具或流程时遇到不顺，我会记一条"摩擦日志"。系统自我改进的时候会读这些日志，优先修复反复出现的问题。

比如：某个技能模板的指令不清晰，导致 AI 每次都理解错。记录摩擦，下次自我改进时自动修复模板。每次运行都让下次运行更顺畅——这就是系统复利。

安全约束

系统可以自动改笔记、改配置、改代码，但有红线：永远不会自动推送代码到 GitHub（需要我确认），永远不会修改工作代码库（只改个人项目），结构性变化只标记不执行（需要审批）。结果通过 Telegram 发到手机，我可以在地铁上审核。

---

第4层：执行层 —— 工作真正发生的地方

核心工具：Claude Code（AI 编程助手）。

内存管理：一个血的教训

有一天 Claude Code 直接被系统杀掉了——占了 4.5GB 内存。原因？同一个插件在三个地方都配置了（全局、项目、插件系统），每次开会话都启动一份，从不清理。光浏览器调试插件就开了 5 个进程，吃掉近 1GB。

解决方案：去重。每个插件只在一个地方配置，零重复。外加养成好习惯——定期清理上下文、切换任务时重开会话。

云端 vs 本地 AI

不是所有任务都需要最强的 AI。我的划分：

- 高风险任务（写代码、架构决策、复杂分析）→ 用云端 Claude API
- 低风险任务（笔记库检查、文件分类）→ 用本地模型（免费、离线）

既省钱，又不牺牲重要工作的质量。

---

第5层：接入层 —— 从手机连进整个系统

这一层解决一个问题：我不在电脑前，怎么继续工作？

Tailscale：隐形的安全隧道

Tailscale 是一个免费的 VPN。装在笔记本和手机上之后，两个设备就像在同一个局域网里，无论你在咖啡店、在家、还是在国外。

安全特性：零信任模型——笔记本没有开放端口，路由器不需要端口转发。WireGuard 加密，跟企业 VPN 同一标准。流量点对点直连，不经过第三方服务器。

mosh：网络切换无感知

普通的远程连接用 TCP 协议。问题是：手机从 WiFi 切到 4G 时，IP 地址变了，连接就断了。终端卡住，你得重新连。

mosh 用 UDP 协议。UDP 没有"连接"的概念，所以 IP 变了它根本不在乎。你从 WiFi 走到地铁切到 4G，终端纹丝不动。而且它只发送屏幕的差异，不发完整画面。丢失的数据包直接跳过——反正旧画面已经过时了。

tmux：断网也不丢进度

如果手机彻底断网了呢？笔记本上的 tmux 会保持会话。AI 继续在笔记本上运行，不受影响。手机重新连上后，一个命令就能恢复——AI 可能已经把活干完了。

> 三层安全叠加：
>
> ① Tailscale WireGuard 加密隧道 → ② mosh AES-128 加密 → ③ SSH 密钥认证。手机上不存储任何代码或凭证，只是一个显示终端文字的窗口。手机丢了？攻击者需要同时突破三道独立的锁。

流量消耗：用手机 4G 工作一小时，大约消耗 10-30MB。因为所有重活都在笔记本上通过家里宽带完成，手机只传输终端文字。1GB 的流量套餐可以用几十个小时。

---

额外设计：把个人系统当公司来管

我给系统设置了三个"高管"角色：

- CTO（首席技术官）：管理技术栈、自动化、配置
- CFO（首席财务官）：管理投资组合，整合股票和外汇
- CEO（首席执行官）：管理目标和决策，做季度回顾和决策日记

为什么？因为个人事务跟公司一样，需要不同视角。技术决策、财务决策、人生决策，混在一起只会互相干扰。分角色之后，每个角色有自己的仪表盘和工作流。

---

总结：这个系统的设计哲学

回头看这五层架构，其实就是几个简单的原则：

1. 一个事实只存一个地方 —— 信息不重复，就不会过期
2. 被动消费不等于学习 —— AI 生成的内容，你没参与思考，就不是你的
3. 摩擦就是改进信号 —— 记录每次不顺，让系统自动修复
4. 安全不是一道墙，是多道锁 —— 每一层都有自己的保护
5. 系统在你睡觉的时候进化 —— 自动化不是省事，是让改进变成习惯

这个系统不是一天建成的。它从一个笔记软件开始，慢慢长出了五层。每一层都是因为遇到了真实的问题才加上去的。

你不需要照搬这个系统。但如果你也有"每天都在重新发明轮子"的感觉，也许可以从最简单的一步开始：今天做了一个决策？写下来。写清楚为什么。 下次你就不用再想一遍了。

---

*© 2026 xinyun2020. All rights reserved. 未经授权禁止转载。*

*写于 2026 年 4 月 11 日。作者是一个相信好记性不如烂笔头的软件工程师。*

我如何用一台笔记本和一部手机，搭建了一个会自己进化的工作系统

What happens when an engineer starts documenting herself instead of just her systems. On self-knowledge, automation, and why a terminal message at 1am felt warmer than a phone call.

{/* Zero-width fingerprint: xinyun2020 */}

The midnight message

At 1am on a Saturday, my terminal told me to stop working.

Not my mom. Not a friend. A configuration file I wrote myself — a rule that says "after midnight, hard stop."

If my mom had texted "go to bed," I'd probably have been annoyed. But seeing that line in my terminal, I felt something unexpected: warmth.

I spent the rest of the weekend trying to figure out why.

Why a machine felt warmer than a person

The difference, I think, is authorship.

That config file understands me because I understood myself first. I noticed a pattern — late-night energy feels productive, but tomorrow always pays the price — and I encoded it into a system. The terminal isn't controlling me. It's executing a decision I made when I was thinking clearly.

When my mom says "go to bed," she's right, but it feels like external pressure. When my system says it, it feels like past-me looking out for future-me. Same message, completely different relationship.

This is what engineers miss when they build systems for everything except themselves. We document APIs obsessively. We write runbooks for incident response. We version-control our infrastructure. But we rarely write down the rules of how we actually work — and then enforce them.

Simplify before you fix

Earlier this week, a colleague said something that stuck with me: "simplify before fix."

We had a bug in a workflow editor. My instinct was to add validation guards — 30 lines of defensive code catching edge cases. His suggestion: reject bad data at the input boundary. One line. Problem gone.

The same principle applies to life systems. When something isn't working — sleep, focus, energy — the instinct is to add more: more tools, more habits, more discipline. But usually the answer is to simplify. Find the one boundary that matters and enforce it there.

My midnight hard-stop isn't a complex system. It's one rule in a config file. But it works because it's at the right boundary — the moment where "productive energy" flips to "tomorrow's regret."

Generate, use, think — in that order

There's a framework I've been testing: generate first, use what you generated, then think about what happened.

Most people do it backwards. They think about whether to start a note-taking system. They research tools for weeks. They plan the perfect structure. Then they never write a single note.

I started with a messy folder of text files. Used them for a few months. Then — only then — thought about what was working and what wasn't. The system I have now didn't come from planning. It came from doing, noticing, and adjusting.

This week I wrote up that system as a five-layer architecture. Knowledge management at the bottom. Mobile remote access at the top. Automation in between. The writing itself was a "think" phase — I'd been generating and using for months, and the architecture only became visible when I tried to explain it to someone else.

The examined life isn't about sitting in a chair and thinking deep thoughts. It's about building something, watching how you interact with it, and then asking: what does this tell me about how I actually work?

The mirror

Here's what I realized while writing that architecture document: the system isn't a collection of tools. It's a mirror.

Each layer reflects a question I answered about myself:
- How do I learn? (Knowledge layer — Zettelkasten, not folders)
- How do I remember? (Persistence layer — config files that survive across sessions)
- How do I improve? (Automation layer — friction logs that fix themselves)
- How do I focus? (Execution layer — one task, clear context, hard boundaries)
- How do I stay connected? (Access layer — terminal over phone, always)

None of these answers are universal. Someone else's system would look completely different, because they'd be answering different questions about a different person.

That's the point. A system that knows you is only as good as your self-knowledge. The config file that told me to stop at midnight — it works because I was honest about a pattern I'd rather not admit: I romanticize late-night productivity, and I always regret it.

One rule

You don't need a five-layer architecture. You don't need a second brain or a Zettelkasten or a personal operating system.

Start with one rule. Something you know about yourself that you keep ignoring. Write it down. Put it somewhere you'll see it at the moment it matters.

Then see if a config file can know you better than your mom.

---

*This is part of my ongoing exploration of what happens when you treat your life as a system worth engineering and a question worth examining.*

*© 2026 xinyun2020. All rights reserved.*

The Config File That Knows Me Better Than My Mom

Example Components

In File Component

  const [count, setCount] = useState(0);
  return (
    <div>
      <button
        onClick={() => setCount(count + 1)}
        className={`${styles.counter} btn`}>
        Clicked {count} times
      
    
  );
};

External Component

Steps

Tabs

  
  
</Tabs>

Callout

Table

| 1   | 2   | 3   | 4   |
| :-- | :-- | :-- | :-- |
| `1` | 2   | 3   | 4   |

Example Components

My favourite monospace fonts for coding — IBM Plex Mono and Fira Code. With a French typing game.

Favourite Font

My favourite monospace fonts for coding: IBM Plex Mono (italic, elegant, great for prose) and Fira Code (ligatures, crisp, great for code).

Type the letter below to try them out — switch between fonts and keyboard sounds.