AI Agent News/stories/browser-use-1-0-5-adds-a-judge-that-grades-every-agent-run.md

Frameworks

browser-use 1.0.5 adds a judge that grades every agent run

browser-use 1.0.5 sends each run through one extra low-effort model request that fills a judgement field on the result. The release also adds a judge_context option for custom criteria and a judge=False switch to turn the check off.

Source: github.com/browser-use/browser-use/releases/tag/1.0.5

The release, published by sarath-menon, adds a second model request to every agent run. That request compares the run with the task and fills result.judgement with success, failure_reason, description, prompt_injection_detected, bot_block_or_captcha_detected and gotchas. The judge reads the trace, the run's last screenshots and the final page, unless use_vision=False is set.

Developers can pass judge_context= to give the judge their own criteria, or set judge=False to disable it. The older use_judge and ground_truth options from the 0.x line work again. The gotchas field is filled whenever a site rejects input, errors, blocks or rate-limits the agent, even when the run succeeds.

The changelog reports a test on 13 real runs, each judged three times. Gotchas appeared on 15 of 15 verdicts where the site pushed back, up from 6 of 15 before the change, and on 2 of 24 verdicts for clean runs. Success verdicts were unchanged.

Why it matters if you build agents

If you build on browser-use, each run now makes one more model call and returns a judgement object, so factor that cost and latency into your budget and handle the new fields. Use judge=False if you do not want the extra request, and judge_context if your task has its own success rules.

The gotchas output can flag blocked, captcha-gated or rate-limited runs without parsing logs yourself, though the release does not describe how reliable the judge is beyond the test figures above.

Get stories like this every weekday

One short email with the day's agent tooling news.

Subscribe