Skip to content
Back/Claude Code

Self-Improving Skills — Eval Loops and Memory

View in Graph
Updated 2026-11-12
1 min read
37 words

Self-Improving Skills — Eval Loops and Memory

『自我改进技能』的方法散落在 eval loop、/retro、plugin eval、单线程过拟合等碎片里,页面缺少一条主线:修改凭什么被信任——没有晋升、回归与淘汰机制,改进就是漂移。

主题导航

  • 同任务对照与 plugin eval 基线:多处来源收敛到同一验证方式:同一批任务分别在启用/不启用 skill 或 plugin 的条件下跑,用分数差决定规则去留,并把已认可的真实工作样例固化成小型回归集。小样本下的判定阈值与主观任务的 rubric 设计仍不可靠。
  • 单次纠正的晋升与过拟合风险:把聊天里的一次纠正直接写回 skill 容易对当前线程过拟合,因此需要晋升/降级标准、跨案例回归集,以及『编码前先审阅候选规则』这类门槛。规则该多长、多久复审没有共识。
  • 技能组合的瘦身与上下文成本:强模型会内化通用流程,使旧 skill 变成纯上下文负担,因此改进包含删除:按触发率、token 占用和冗余做组合级审计,只保留模型推不出的判断、安全边界与验收标准。哪些内容属于『方法论永生』部分仍是主观判断。

来源与原文

拆分前完整原文与来源

Sources

Synthesized from 29 sources

Evolution

1 event
  1. absorbed

    Derived from source material

    This page is currently synthesized from 29 sources.

    Sources: /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-09-03-02-03.md · /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-09-06-08-11.md · /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-09-06-17-40.md · /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-09-11-23-15.md · /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-09-12-08-04.md · /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-09-17-01-58.md · /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-09-17-08-05.md · /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-09-21-22-20.md · /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-10-02-22-57.md · /Users/jean/Documents/Obsidian Vault/raw/newsletters/Ben's Bites/2026-09-16 Good luck slowing this down.md · /Users/jean/Documents/Obsidian Vault/raw/newsletters/Every/2026-09-11 Evals for Everyone.md · /Users/jean/Documents/Obsidian Vault/raw/newsletters/Every/2026-09-16 You're Probably Sleeping On Computer Use.md · raw/briefing/AI Briefing/2026-08-30-08-10.md · raw/briefing/AI Briefing/2026-08-30-23-07.md · raw/briefing/AI Briefing/2026-09-03-02-03.md · raw/briefing/AI Briefing/2026-09-06-08-11.md · raw/briefing/AI Briefing/2026-09-06-17-40.md · raw/briefing/AI Briefing/2026-09-11-23-15.md · raw/briefing/AI Briefing/2026-09-12-08-04.md · raw/briefing/AI Briefing/2026-09-17-01-58.md · raw/briefing/AI Briefing/2026-09-17-08-05.md · raw/briefing/AI Briefing/2026-09-21-22-20.md · raw/briefing/AI Briefing/2026-10-02-22-57.md · raw/briefing/AI Briefing/2026-11-01-23-20.md · raw/briefing/AI Briefing/2026-11-04-08-03.md · raw/briefing/AI Builders Digest/2026-11-07.md · raw/newsletters/Ben's Bites/2026-09-16 Good luck slowing this down.md · raw/newsletters/Every/2026-09-11 Evals for Everyone.md · raw/to-learn/Full Tutorial Build Self-Improving Claude Skills in 20 Min (Eval + Memory).md

Linked from