Computer Use Production
What it is
Computer Use production is the set of engineering practices required to deploy Claude's computer/browser use capabilities beyond demos into repeatable, auditable workflows. It covers resolution scaling, click accuracy, thinking effort tuning, prompt injection defense, context management for long-running agents, and permission boundaries — the layer that turns "the model can click" into "the system works reliably at 2 AM."
Why it matters
Computer use agents interact with untrusted content by design. Every screenshot and webpage could contain adversarial instructions. Without deliberate engineering around resolution, context, and safety, production deployments fail on basic click accuracy or security gaps. The gap between "works in a demo" and "works in production" is where most computer use projects die.
Evidence across sources
| Source | Key Claim | Relevance |
|---|---|---|
| Anthropic — Best Practices for Computer and Browser Use | Resolution pre-scaling, thinking effort sweet spots, injection defense, three-layer context management | Official engineering guide |
| Every — You're Probably Sleeping On Computer Use | Astra evolved from "can use" to daily driver via low-risk, checkable chores first | Production adoption path |
| AI Briefing 2026-09-16 Evening | Browser fallback must preserve tool boundary; API vs browser permission paths differ | Permission boundary engineering |
| AI Briefing 2026-09-20 Evening | 300+ Mac apps tested via computer use; failures fed back into rules and verification scripts | QA and self-verification loop |
| AI Briefing 2026-09-23 Evening | Accessibility semantics as agent test interface; same semantic layer serves a11y, testing, and agent operation | Interface robustness |
Key production patterns
Resolution and click accuracy:
- Pre-downscale screenshots before sending to API; the most common cause of poor accuracy is silent server-side downscaling
- Coordinate scaling: scale API-returned coordinates back to native screen resolution
- Recommended default: 1280x720 for 4.6 family; 1080p for Opus 4.7
Thinking effort tuning:
- Claude 4.6:
mediumis the sweet spot — near-highest success rate at roughly half the tokens ofhigh - Avoid
maxfor computer use on 4.6 — no accuracy benefit overhighwhile increasing cost
Context management for long-running agents:
- Screenshots accumulate fast (~1,000–1,800 tokens each); a 200k context window fills in well under 100 screenshots
- Three layers: cache breakpoints, cache-aware rolling buffer, LLM-based compaction
Permission boundaries:
- Computer Use in API-auth-failure fallback position must not become a permission escalation path
- Browser fallback can be a read-task recovery strategy, but must not bypass MCP or sandbox denials
- Agent operating the computer does not equal agent having final judgment; screenshots, result checks, and rollback remain part of the workflow
Open questions
- How do batch tools affect error recovery when one sub-action misses its target?
- Does the advisor tool pattern generalize beyond computer use to other long-horizon agent tasks?
- What is the cost breakpoint where server-side compaction pays for itself vs. rolling buffer alone?
Related
- Browser Harness — Self-healing CDP browser agent architecture
- Computer Use Best Practices — Anthropic's official deployment guide
- Self-verification Loops — Failure feedback into rules
- Agent Trust Boundary Permissions — Permission scope design
Pending Jean review
- harness-engineering/computer-use-production —
ai_draft; suggested action: define one low-risk computer-use task with read-only fallback, explicit approval, and post-action verification.