A Puma experiment
Evals that stay on your machine.
Private suites run in this browser. The public board is a browser-agent harness, and the numbers on it are baselines.
Published baselines · heuristic-scripted 7/11 · naive-first-match 1/11
Private by design
Private evals tell you whether your prompts, tools, and agents hold up on the tasks you actually run. Your cases stay on your device.
Cases you keep
Capture the tickets, documents, and workflows your system must handle. Nothing is uploaded. The cases stay on this device.
Graders you control
Score with exact match, contains, regex, JSON field checks, and keyword lists. Start simple, then tighten the rubric as the work gets clearer.
Runs that stay local
Paste model outputs, label the system under test, and keep a history of pass rates as prompts and models change — all in this browser.
Browser agents, scored locally
11 deterministic tasks on fixture sites that stay on your machine: extraction, navigation, reasoning, forms, and a multi-step login. Checkers score every run. The numbers below are baselines from the open-source harness.
Baseline · heuristic-scripted
7/11
Scripted actions. Reads labels, fills quoted fields, and clicks named controls. No model and no API key.
Baseline · naive-first-match
1/11
Lower bound. No clicks. Answers with the first number or time on the start page.
Language-model adapter
No published run
The adapter is in the harness. A score appears here when a results file is committed and the site is rebuilt.
Open the browser-agent leaderboard
Two paths. One purpose.
getEvals sits next to Puma Browser and the PumaAI lab. Same purpose: AI that stays in your hands.
01 / Company
Puma.tech
AI, on your terms. A more personal browser, and research toward a future where AI stays in your hands.
02 / Research
PumaAI
A small lab studying private local inference, observable agent harnesses, and memory store.
03 / Product
Puma Browser
Local and cloud AI, right in your browser. Private by design. iOS and Android.
Start with the demo suite Browser agent baselines See the benches