About getEvals

Why this exists

People still have to decide whether a model, a prompt, or an agent harness is good enough for the task in front of them. getEvals is a small loop for that decision: private suites you run on your device, and one public harness whose tasks and checkers are in this repo.

We prefer small reproducible experiments, visible limitations, and a clear line between measurement and hypothesis — the same habits as the PumaAI lab.

What we build

The app is a local loop: cases from your domain, graders that encode what “good” means, and runs that catch regressions before they ship. Data stays in the browser. Nothing is uploaded.

The public result is the browser-agent harness: eleven tasks on local fixture sites, scored by checkers. The runs on that page are baselines. The language-model adapter has no published run.

How we work

PumaAI is a small research lab led by Yuriy Dybskiy, founder of Puma. The lab studies private local inference, observable agent harnesses, and memory store. getEvals is the evals experiment next to that work.

Puma’s values are optimism, kindness, curiosity, speed, and grit. We are not trying to become the measurement institution of a trillion-dollar market. We are trying to make useful evals that stay in your hands.

The Puma family

• Puma.tech — AI, on your terms.
• PumaAI — research lab. Private local inference, observable agents, memory store.
• Puma Browser — local and cloud AI, right in your browser.
• Puma OS — a private AI operating system for your company.