Skip to content
Back/Harness Engineering

Computer Use Best Practices — Anthropic Official Guide

View in Graph
Updated 2026-09-23
5 min read
1,075 words

Computer Use Best Practices — Anthropic Official Guide

What it is

Anthropic 官方发布的 Claude Computer Use / Browser Use 生产部署指南,涵盖分辨率缩放、点击精度、思考力度调优、提示注入防御、上下文管理和实验性工具配置。

Why it matters

Computer use agents interact with untrusted content by design. Every screenshot and webpage could contain adversarial instructions. Without deliberate engineering around resolution scaling, context management, and safety, production deployments fail on basic click accuracy or security gaps.

Key points

Resolution and click accuracy

  • Pre-downscale screenshots before sending to API. The most common cause of poor click accuracy is sending native-resolution screenshots that exceed API limits and get silently downscaled.
  • Claude 4.6 limits: max long edge 1568px, max total pixels 1.15MP.
  • Opus 4.7 limits: max long edge 2576px, max total pixels 3.75MP.
  • Recommended default: 1280x720 for 4.6 family; 1080p for Opus 4.7.
  • Coordinate scaling is critical: scale API-returned coordinates back to native screen resolution before executing clicks.
  • Content ordering: place text instruction before the image in the messages array.
  • Model selection: Sonnet 4.6 tends to be more mechanically precise at clicking; Opus 4.7 narrows this gap with higher resolution budget.

Thinking effort tuning

  • Claude 4.6: medium is the sweet spot — close to highest success rate at roughly half the tokens of high. low is surprisingly strong for cost-sensitive workloads.
  • Claude Opus 4.7: high achieves near-maximum success rate while using roughly half the tokens of max.
  • Avoid max effort for computer use on 4.6 models — no accuracy benefit over high while increasing cost. UI tasks are perceptual, not deeply logical.

Prompt injection defense

  • Training-time robustness: RL builds resistance directly into Claude's capabilities.
  • Real-time classifiers: scan content entering context window and flag potential injection attempts.
  • Built-in classifiers run automatically when using official computer_20251124 tool type — zero additional latency, no extra cost.
  • Best practices regardless of classifier use: human-in-the-loop for high-stakes actions, scope agent permissions narrowly, monitor and log all actions, treat all web content as untrusted.

Context management for long-running agents

Screenshots accumulate fast: each consumes roughly 1,000–1,800 tokens. A 200k context window fills in well under 100 screenshots.

Three layers that compose cleanly:

  1. Cache breakpoints: one on stable prefix (system prompt), up to three on most recent tool results. Spreading breakpoints across recent positions gives graceful degradation.
  2. Cache-aware rolling buffer: keep most recent keep_n=3 screenshots; when total exceeds keep_n + interval=25, replace oldest interval screenshots with placeholders in a single pass. Prefix stays byte-identical between prunes.
  3. LLM-based compaction: summarize conversation history before discarding. Critical sections: user instructions (verbatim), task template, constraints, actions taken, errors and fixes, progress tracking, current state, next step.

Server-side compaction (beta): pass custom summarization prompt as instructions parameter in context_management. Set pause_after_compaction to attach most recent messages across events. Mirror server truncation on client to keep views aligned.

Experimental settings

  • Batch tools (computer_batch, browser_batch): execute multiple sub-actions in single tool call. Use when sub-actions are self-contained and don't depend on each other's visual outcomes. Avoid in exploratory navigation or error-recovery sequences.
  • Advisor tool: pairs executor model with higher-intelligence advisor model for strategic guidance mid-generation. Useful when most turns are mechanical but occasional planning moments need Opus-level reasoning. Cleanup orphaned advisor blocks when disabling the tool.
  • Teach Mode: record human performing a workflow (screenshots, actions, optional voice), then replay as context. Not strict replay — Claude adapts to UI changes. Supports strict, adaptive, and goal-oriented playback modes.

Competitive Audit with Computer Use(2026-06-11)

来源:Ben's Bites — New Claude model - Fable

Keshav 在 3–4 天内构建完全本地的语音转文本应用"Option AFK",展示了 Computer Use 的一种非传统但高效的用途:竞品功能审计。

流程:

  1. 让 Codex Computer Use 对付费工具进行逐屏审计,记录所有功能并截图。
  2. 在单次 $20 计划的会话中完成审计(仅一次触及 5 小时限制)。
  3. 用审计结果作为 spec,让编码 agent 构建替代方案。

关键发现:

  • 审计结果成为可靠的替代方案 spec,降低了从"我想做一个"到"具体怎么做"的摩擦。
  • 总成本低于 $50,包括 Codex Computer Use 审计、Opus 4.7/4.8 开发和 Fable 5 最终优化。

适用边界:

  • 适合 UI 流程明确、功能可枚举的付费工具。
  • 不适合依赖后端集成或数据网络效应的服务。
  • 审计本身需要沙盒环境,避免对真实账户或数据造成意外操作。

Open questions

  • How do batch tools affect error recovery when one sub-action in a batch misses its target?
  • Does the advisor tool pattern generalize beyond computer use to other long-horizon agent tasks?
  • What is the cost breakpoint where server-side compaction pays for itself vs rolling buffer alone?

Prompts for witness

  • What is the most frequent failure mode in your computer use integration, and which layer (resolution, thinking effort, context management, safety) would fix it?
  • If you had to set a single "taste standard" for agent-generated UI interactions, what would it be?

2026-09-16 evening: fallback use needs explicit permission accounting

来源:AI Briefing 2026-09-16 Evening

本次时间线中的 Slack 案例把 Computer Use 放在 API 认证失败后的读取 fallback 位置。Voxyz 的案例进一步说明,MCP 与浏览器可能对应不同的账户权限,因此 Agent 不能因为工具调用失败就自动切换到权限更大的页面操作。若写入超时或结果不清楚,也必须先核验是否已经生效。

Counterpoints & Gaps

  • 这些是个人经验和工程建议,没有提供不同访问路径的受控基准。
  • 浏览器路径可能更快,但速度不能替代权限审查和动作回执。
  • 本次没有复现 Slack、Supabase 或其他真实账户上的权限差异。

Sources (addendum)

Pending Jean review

2026-09-16:计算机使用从演示走向电脑杂务

来源:Every — You're Probably Sleeping On Computer Use

Every 记录了 Astra 从“可以用”变成日常主力的路径:Mike Taylor 先让它填写女儿学校表格,再把 Google Slides 更新、视频编辑和整本书 PDF 链接校验交给它。Every 员工还把日历维护、旧软件网页操作、Verizon 客服沟通、iPhone 照片整理和 Blender 角色修复交给 Computer Use。可复用的边界是先从低风险、结果可检查的杂务开始,并让任务标题或日志明确标记当前是否由 Agent 控制电脑。

Douglas Brundage 的幻灯片流程补充了“示例驱动”:先手工制作 2–3 张参考页,再让 Agent 在 Google Slides 中执行,人工修订后把重复错误写回 skill。计算机使用降低了执行摩擦,但没有取消权限、停止点和结果复核。

Counterpoints & Gaps

  • 这是 Every 的团队实践报道,没有提供跨任务成功率、漏检率或独立复现。
  • 文章没有定义统一的外部动作授权协议;发送、购买、提交和账户设置仍需单独审批。

Sources (addendum)

Pending Jean review

2026-09-16(补充):连接器与计算机使用是两种边界

同一来源的另一条边界:有 API 时优先用结构化接口,没有 API 或界面行为本身就是任务时(如 Blender 角色修复、旧软件网页操作)再使用 computer use,并限制权限。执行自动化越强,结果验证越重要:agent 操作电脑不等于拥有最终判断,截图、结果检查和回滚仍是工作流的一部分。Every 的观察是,agent 释放速度后瓶颈转向 review——人仍需确认按钮真的工作、界面是否一致。

Sources (addendum)

2026-09-20: Computer Use as product QA

来源:AI Briefing 2026-09-20 Evening

HiTw93 自述将 300 多款 Mac 软件分成 30 组,借助 computer use 逐批从官网下载、安装、检查 bundle 与自启动项,再卸载并查找残留;发现遗漏后补充清理规则,也写了自动化脚本。这为 computer use 增加了一个产品验证用途:让 agent 在应用生命周期中执行重复测试,并把失败案例回流到规则与验证脚本,连接到 harness-engineering/self-verification-loops。

Counterpoints & Gaps

  • 这是单个开发者的自述,没有给出覆盖率、误报率、失败样本或独立复现。
  • 安装与卸载会改变系统状态;应在隔离或可还原环境中测试,限制权限,并由人处理管理权限提示。

Sources (addendum)

Pending Jean review

2026-09-23:无障碍语义也是 Computer Use 的测试接口

来源:AI Briefing 2026-09-23 Evening

HiTw93 分享了一个重要连接:无障碍属性不仅服务视障用户,也能帮助 computer-use agent 更准确地定位按钮和控件。对产品团队而言,同一套语义界面可以同时支持无障碍检查、自动化业务流程测试和 agent 操作,减少依赖脆弱坐标。

Counterpoints & Gaps

  • 这是个人经验,尚未提供不同应用结构下的定位成功率对比。
  • 无障碍属性不能替代权限边界、动作确认和结果回读。

Pending Jean review

Sources

Synthesized from 11 sources

Evolution

1 event
  1. absorbed

    Derived from source material

    This page is currently synthesized from 11 sources.

    From Anthropic — Best Practices for Computer and Browser Use with Claude (EN), Ben's Bites 2026-06-12 — New Claude model - Fable, 2026-09-16-02-18, 2026-09-20-22-59, 2026-09-23-22-58To Computer Use Best Practices — Anthropic Official Guide
    Sources: raw/to-learn/best-practices-computer-browser-use-claude.md · raw/newsletters/Ben's Bites/2026-06-12 New Claude model - Fable.md · /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-09-16-02-18.md · /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-09-20-22-59.md · /Users/jean/Documents/Obsidian Vault/raw/briefing/AI Briefing/2026-09-23-22-58.md · /Users/jean/Documents/Obsidian Vault/raw/newsletters/Every/2026-09-16 You're Probably Sleeping On Computer Use.md · raw/briefing/AI Briefing/2026-09-16-02-18.md · raw/briefing/AI Briefing/2026-09-20-22-59.md · raw/briefing/AI Briefing/2026-09-23-22-58.md · raw/newsletters/Every/2026-09-16 You're Probably Sleeping On Computer Use.md · raw/to-learn/Best practices for computer and browser use with Claude.md

Linked from