Skip to content
Back/Harness Engineering

Computer Use Production

View in Graph
Updated 2026-10-25
3 min read
559 words

Computer Use Production

What it is

Computer Use production is the set of engineering practices required to deploy Claude's computer/browser use capabilities beyond demos into repeatable, auditable workflows. It covers resolution scaling, click accuracy, thinking effort tuning, prompt injection defense, context management for long-running agents, and permission boundaries — the layer that turns "the model can click" into "the system works reliably at 2 AM."

Why it matters

Computer use agents interact with untrusted content by design. Every screenshot and webpage could contain adversarial instructions. Without deliberate engineering around resolution, context, and safety, production deployments fail on basic click accuracy or security gaps. The gap between "works in a demo" and "works in production" is where most computer use projects die.

Evidence across sources

Source Key Claim Relevance
Anthropic — Best Practices for Computer and Browser Use Resolution pre-scaling, thinking effort sweet spots, injection defense, three-layer context management Official engineering guide
Every — You're Probably Sleeping On Computer Use Astra evolved from "can use" to daily driver via low-risk, checkable chores first Production adoption path
AI Briefing 2026-09-16 Evening Browser fallback must preserve tool boundary; API vs browser permission paths differ Permission boundary engineering
AI Briefing 2026-09-20 Evening 300+ Mac apps tested via computer use; failures fed back into rules and verification scripts QA and self-verification loop
AI Briefing 2026-09-23 Evening Accessibility semantics as agent test interface; same semantic layer serves a11y, testing, and agent operation Interface robustness

Key production patterns

Resolution and click accuracy:

  • Pre-downscale screenshots before sending to API; the most common cause of poor accuracy is silent server-side downscaling
  • Coordinate scaling: scale API-returned coordinates back to native screen resolution
  • Recommended default: 1280x720 for 4.6 family; 1080p for Opus 4.7

Thinking effort tuning:

  • Claude 4.6: medium is the sweet spot — near-highest success rate at roughly half the tokens of high
  • Avoid max for computer use on 4.6 — no accuracy benefit over high while increasing cost

Context management for long-running agents:

  • Screenshots accumulate fast (~1,000–1,800 tokens each); a 200k context window fills in well under 100 screenshots
  • Three layers: cache breakpoints, cache-aware rolling buffer, LLM-based compaction

Permission boundaries:

  • Computer Use in API-auth-failure fallback position must not become a permission escalation path
  • Browser fallback can be a read-task recovery strategy, but must not bypass MCP or sandbox denials
  • Agent operating the computer does not equal agent having final judgment; screenshots, result checks, and rollback remain part of the workflow

Open questions

  • How do batch tools affect error recovery when one sub-action misses its target?
  • Does the advisor tool pattern generalize beyond computer use to other long-horizon agent tasks?
  • What is the cost breakpoint where server-side compaction pays for itself vs. rolling buffer alone?

Pending Jean review

Sources

Synthesized from 5 sources
  • Anthropic — Best Practices for Computer and Browser UseSupporting source listed by this page.Whole pagemediumbody
  • Every — You're Probably Sleeping On Computer UseSupporting source listed by this page.Whole pagemediumbody
  • AI Briefing 2026-09-16 EveningSupporting source listed by this page.Whole pagemediumbody
  • AI Briefing 2026-09-20 EveningSupporting source listed by this page.Whole pagemediumbody
  • AI Briefing 2026-09-23 EveningSupporting source listed by this page.Whole pagemediumbody

Evolution

1 event
  1. absorbed

    Derived from source material

    This page is currently synthesized from 5 sources.

    From Anthropic — Best Practices for Computer and Browser Use, Every — You're Probably Sleeping On Computer Use, AI Briefing 2026-09-16 Evening, AI Briefing 2026-09-20 Evening, AI Briefing 2026-09-23 EveningTo Computer Use Production
    Sources: raw/to-learn/best-practices-computer-browser-use-claude.md · raw/newsletters/Every/2026-09-16 You're Probably Sleeping On Computer Use.md · raw/briefing/AI Briefing/2026-09-16-02-18.md · raw/briefing/AI Briefing/2026-09-20-22-59.md · raw/briefing/AI Briefing/2026-09-23-22-58.md

Linked from