terminal-bench

Framework for evaluating and improving agents

A harness optimized to smaller LLMs

Long Horizon Terminal Benchmark with Dense Reward Grading

Taking the pi coding agent to Claude Code-level performance: 0.539 → 0.773 pass@1 on Terminal-Bench 2.1 with the same self-hosted 27B model, by re-engineering the agent.

A research framework for principled agent self-improvement under frozen evaluators and declared mutation boundaries, recording verifiable lineage to make it reproducible and auditable.

Open-source desktop coding agent where a lead model plans and cheaper models handle search, edits and review.

Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.

Official Implementation of "CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion"

Some tools help. Some tools assist. Villani Code intervenes.

Open-source AI agent that runs on your own computer and does your tasks. Any model (Claude, GPT, Gemini, Grok, Ollama), drives your real browser, shell, files, DOCX/XLSX.

Strands-based agents and harnesses for agentic benchmarks.

Simple Long Horizon Agent - A simple yet effective AI agent for learning, experimentation, and long horizon work.

A coding agent that builds it's own tools and mines it's own skills and always ships with receipts.

Spoox CLI - Terminal Agent - SPlit lOOp eXand agent

Fast, Multi-Cloud Sandbox Engine for AI Agents

Which host should serve your open model? Run your workload across pinned, verified providers and get one table: success, cost, latency, cache hits.

Evidence-first, crash-resumable self-evolution engine for DeepSeek Harness and Harbor.

Runs BOTH controls (nop + oracle) across EVERY task in a coding-agent benchmark, classifies the outcome, captures evidence, and lints leaks statically. One binary, zero LLM calls.

Trajectories for running OpenHands on Terminal Bench

Open-source desktop coding agent (Electron + React) with plan/execute modes, MCP, skills, and a terminal-bench 2.0 harness adapter