Skip to content
Back/Harness Engineering

Agent in Loop Methodology

View in Graph
Updated 2026-10-24
3 min read
579 words

Agent in Loop Methodology

What it is

Agent-in-loop engineering gives an agent the project context, tools, and feedback needed to complete a bounded task through a repeatable cycle: clarify the expected behavior, build, inspect the result, and feed recurring failures back into the harness.

Why it matters

A capable model cannot compensate for missing context or unverifiable expectations. A well-shaped loop can reduce handoffs while keeping the result accountable to evidence and human review at consequential boundaries.

Key points

  • Give the agent a usable harness. Provide relevant repository context and focused tools that make logs, documentation, and other evidence navigable. Elvis Sun’s reported one-shot turbo-cache fix illustrates the potential of a complete debugging environment; it is an example, not a reliability guarantee.
  • Specify before building. In one account, agents first discussed a system and produced a specification, then built it. The author estimated a week of work took one or two hours; this is a personal estimate, not a general benchmark.
  • Check independently after generation. The same account describes post-build checks that surfaced bugs, omissions, and dead code. Acceptance criteria and checks make those findings actionable instead of relying on the agent’s own completion claim.
  • Turn repeated findings into process. A small-team adoption proposal recommends treating feedback as candidate work, applying human approval where needed, and moving recurring issues into skills, workflows, lint, or tests.
  • Bound autonomy by the task. Start with a measurable workflow, named owner, known data and permissions, and an explicit acceptance check. Decide which steps can run unattended and where a person must review the result.

Evidence across sources

Source Key claim Relevance
AI Briefing 2026-04-11 Elvis Sun describes giving an agent a suitable harness and reports a one-shot turbo-cache fix. Supports the role of context and tools in an engineering loop.
AI Briefing 2026-10-22 Morning An account describes spec-first build and post-build checks; a second proposes human-reviewed feedback and encoding recurring failures into workflow assets. Adds explicit specification, verification, and feedback stages; the reported time saving remains anecdotal.

Open questions

  • Which repeated task in a real repository has a clear enough acceptance check to run as a bounded loop?
  • Which findings should become a test or lint rule, and which still require human judgment?
  • Does the loop reduce rework when measured across several tasks, rather than one reported example?

Prompts for witness

  • Which recurring task in your work loses the most time to missing context or repeated clarification?
  • What evidence would convince you that an agent completed that task correctly?
  • Where should the loop stop and ask for human review?

Pending Jean review

  • harness-engineering/agent-in-loop-methodology — ai_draft; suggested action: keep as a draft and evaluate the specification, independent-check, and feedback loop on a repeated repository task before treating the reported time saving as transferable.

2026-10-24 — FDE 案例把客户 Eval 与 Harness 变成可复制资产

来源:AI Briefing 2026-10-24 Morning;@Xudong07452910 转述 OpenAI FDE 负责人 Colin Jarvis 的工程案例。

该案例补充了“指定行为—构建—独立验证—反馈”的企业落地实例:半导体团队围绕客户流程建立 Eval 与 Harness;推文称定向测试生成节省约 48% 时间,首个团队部署约两个月、复制到第二个团队约一天,并在 Codex 能力增强后移除约 90% 定制代码,改用 Skills 和插件。以上均是推文对案例的转述,应回看原始工程博客后再作为可迁移基准。

Counterpoints & Gaps

  • 时间线未包含原始博客全文;百分比、时间和代码删减比例均未独立验证。
  • 复用速度取决于 Eval、连接器和客户流程是否相似,不能直接推定其他企业也能一天上线。

Sources (addendum)

Pending Jean review

Sources

Synthesized from 2 sources
  • AI Briefing 2026-04-11Supporting source listed by this page.Whole pagemediumbody
  • AI Briefing 2026-10-22 MorningSupporting source listed by this page.Whole pagemediumbody

Evolution

1 event
  1. absorbed

    Derived from source material

    This page is currently synthesized from 2 sources.

    From AI Briefing 2026-04-11, AI Briefing 2026-10-22 MorningTo Agent in Loop Methodology
    Sources: raw/briefing/AI Briefing/2026-04-11.md · raw/briefing/AI Briefing/2026-10-22-08-04.md

Linked from