Agent-Driven Code Maintenance
What it is
Agent-driven code maintenance uses coding agents for repeatable repository work such as dead-code removal, stale feature cleanup, test maintenance, and CI improvement. These tasks require evidence about call sites, runtime behavior, tests, and ownership. The agent's edit is only one part of the workflow.
Why it matters
Code generation can increase maintenance work as well as reduce it. Automation is useful when a task has a bounded target, available evidence, and a review path. The sources describe two complementary designs: agents investigate and plan multi-file changes, while deterministic tools and independent checks constrain risky edits.
Key points
- Begin with a narrow maintenance task and inspect its dependencies before changing code. The OpenAI report describes an evolution from static analysis and runtime logs toward agents gathering evidence across code, prior changes, and production data.
- Uber's AutoClean uses an agent to trace stale flags across configuration, call sites, tests, and branches. It delegates Boolean simplification to PolyglotPiranha rather than asking the model to perform every mechanical rewrite.
- CI checks the broad repository impact. Uber describes deterministic fixers for mechanical failures and narrowly scoped agent fixers for harder failures.
- A passing CI run does not establish that the requested cleanup is complete or preserves intent. Uber's CodeCourt checks separate properties including semantic preservation, no unrelated edits, no added code, and no agent artifacts before human review.
- The OpenAI report recommends expanding from small reviewed tasks to larger areas only after measuring useful findings, cost, and recurring error patterns. Both source reports describe company workflows and figures that have not been independently audited here.
Evidence across sources
| Source | Key claim | Relevance |
|---|---|---|
| Maintaining software with the GPT-6 family | OpenAI reports agent work on dead code, stale experiments, and CI, with human review and cost comparisons across models. | Evidence for bounded tasks, investigation quality, feedback from review, and model cost. |
| AI Briefing 2026-10-21 Afternoon | Uber describes AutoClean using deterministic rewrites, repository-wide CI, scoped repair agents, and property-specific validation before human review. | Adds an independent company case for separating agent reasoning from deterministic transformation and final validation. |
Open questions
- The reports do not give a comparable accounting of human review time, rejected diffs, or long-term maintenance cost.
- Which properties can be checked deterministically, and which require a human or a separate agent reviewer?
- How should teams measure whether a model change makes prior prompt corrections stale?
Prompts for witness
- Which repositories contain a small maintenance task with clear evidence and a low-risk rollback path?
- What evidence should an agent provide before proposing removal of a flag, test, or code path?
- Which review checks would catch an incomplete cleanup that still passes CI?
Counterpoints & Gaps
- The OpenAI and Uber figures are reports from the organizations that built the systems. The sources do not provide a shared benchmark or independent audit.
- Uber describes the validation architecture but does not quantify its false-positive rate, reviewer workload, or production incidents prevented.
- These cases do not establish that autonomous deletion is appropriate when ownership or runtime evidence is incomplete.
Related
- harness-engineering/agent-code-review-at-anthropic
- harness-engineering/agent-trust-verification
- harness-engineering/model-routing
- ai-ecosystem/openai-codex-superapp
2026-10-14:OpenAI GPT-6 维护案例的模型路由数据
来源:Maintaining software with the GPT-6 family(OpenAI Developers 博客,中文转述;本页前节已引用,此处补量化细节)
OpenAI Core Experimentation 团队 5 名工程师两周(兼职)合并 1,400+ 维护 PR、删除 80 万+ 行。同 prompt 同目录(6,000+ 文件)的三档模型对照提供了少见的成本数据:Astra Ultra $109 换 142 条发现($0.77/条),Sol Ultra $15 换 131 条(92% 的发现数、14% 的成本),Sol Medium $5 换 86 条。误差模式集中在"被其他 flag 依赖的常量 flag"——识别误差模式后改 prompt,让便宜模型逼近贵模型;换模型时需重审 prompt。可验证收益:容器镜像 3.31GB→1.72GB(-48%);CI 中位 10→4.5 分钟;一个 mock 修复让 63 个测试从 592.6s 降到 29.8s。
判断力案例:Astra 发现一行 Python 造成显著 CPU fan-out,但 flag 标记为 Permanent,未直接删除而是上报人工。"知道什么时候不删"是 agent 可信维护的标志。
方法四步:小任务起步 → 对比模型/努力度 → 小范围跑通并吸收 review 反馈进 prompt → 全库铺开;用 instrumentation(CPU 成本数据)给"找到太多"的发现排优先级。
Pending Jean review
- harness-engineering/agent-driven-code-maintenance —
ai_draft; suggested action: verify whether these validation properties fit a local maintenance pilot.
2026-10:Uber AutoClean 的确定性重写范式
Uber AutoClean:代码代理 + 确定性转换 + CI 修复 + 独立验证,清理大型代码库过期功能开关——「确定性重写 + 代理审查」的组合范式。