Top 100 Agent Benchmark Repositories β
Ranking β
| Ranking | Project Name | Stars | Forks | Language | Open Issues | Description | Last Commit |
|---|---|---|---|---|---|---|---|
| 1 | cua | 16,994 | 1,079 | HTML | 122 | Open-source infrastructure for Computer-Use Agents. Sandboxes, SDKs, and benchmarks to train and evaluate AI agents that can control full desktops (macOS, Linux, Windows). | 2026-05-21 |
| 2 | agentmemory | 16,132 | 1,327 | TypeScript | 73 | #1 Persistent memory for AI coding agents based on real-world benchmarks | 2026-05-21 |
| 3 | pyod | 9,853 | 1,475 | Python | 188 | A Python library for anomaly detection across tabular, time series, graph, text, and image data. 60+ detectors, benchmark-backed ADEngine orchestration, and an agentic workflow for AI agents. | 2026-05-19 |
| 4 | agentops | 5,572 | 582 | Python | 106 | Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and Ca... | 2026-03-19 |
| 5 | AgentBench | 3,444 | 255 | Python | 61 | A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24) | 2026-02-08 |
| 6 | OSWorld | 2,866 | 462 | Python | 147 | [NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments | 2026-05-21 |
| 7 | Awesome-LLM-Long-Context-Modeling | 2,095 | 93 | - | 0 | π° Must-read papers and blogs on LLM based Long Context Modeling π₯ | 2026-05-17 |
| 8 | mle-bench | 1,539 | 247 | Python | 2 | MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineering | 2026-04-24 |
| 9 | Introduction-to-Quantitative-Finance | 1,404 | 156 | Python | 0 | ε ₯ι¨θ΅ζζ΄ηοΌ1.ε€ε εθ‘η₯¨ιεζ‘ζΆεΌζΊζη¨ 2.ε¦ηεδΈηηη»ε Έθ΅ζζΆε½ 3.AI + ιθηηΈε ³ε·₯δ½οΌε ζ¬LLM, Agent, benchmark(evaluation), etc. | 2026-05-21 |
| 10 | smac | 1,352 | 237 | Python | 21 | SMAC: The StarCraft Multi-Agent Challenge | 2024-02-18 |
| 11 | little-coder | 1,290 | 78 | TypeScript | 1 | A coding agent optimized to smaller LLMs | 2026-05-19 |
| 12 | ClawGUI | 1,265 | 48 | Python | 1 | Build, Evaluate, and Deploy GUI Agents β online RL training, standardized benchmarks, and real-device deployment in one framework. | 2026-05-21 |
| 13 | tau2-bench | 1,214 | 315 | Python | 67 | Ο-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains | 2026-05-21 |
| 14 | skill | 1,195 | 132 | Python | 17 | PinchBench is a benchmarking system for evaluating LLM models as OpenClaw coding agents. Made with π¦ by the humans at https://kilo.ai | 2026-05-14 |
| 15 | smallcode | 1,146 | 81 | JavaScript | 0 | AI coding agent optimized for small LLMs. 87% benchmark with 4B-active model. | 2026-05-22 |
| 16 | Xcode-Build-Optimization-Agent-Skill | 1,064 | 39 | Python | 1 | An Agent Skill helping you to optimize Xcode incremental and clean builds by running benchmarks and optimizing build settings. | 2026-04-15 |
| 17 | Mind2Web | 991 | 124 | Jupyter Notebook | 7 | [NeurIPS'23 Spotlight] "Mind2Web: Towards a Generalist Agent for the Web" -- the first LLM-based web agent and benchmark for generalist web agents | 2025-11-05 |
| 18 | LuaN1aoAgent | 924 | 147 | Python | 4 | LuaN1aoAgent is a cognitive-driven AI hacker. It is a fully autonomous AI penetration testing agent. Using dual-graph reasoning, LuaN1ao achieves a success rate of over 90% on the XBOW Benchmark. | 2026-04-13 |
| 19 | token-savior | 899 | 76 | Python | 4 | The MCP server that turns Claude into the only coding agent hitting 100% on a real benchmark. -77% active tokens, -76% wall time, 0 losses across 96 tasks on Claude Opus 4.7. Structural code navigatio... | 2026-05-19 |
| 20 | WindowsAgentArena | 860 | 95 | Python | 30 | Windows Agent Arena (WAA) πͺ is a scalable OS platform for testing and benchmarking of multi-modal AI agents. | 2026-04-13 |
| 21 | AICGSecEval | 839 | 106 | Python | 5 | A.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team. | 2026-04-21 |
| 22 | ISC-Bench | 775 | 118 | Python | 3 | Internal Safety Collapse: Turning the LLM or an AI Agent into a sensitive data generator. | 2026-05-14 |
| 23 | android_world | 770 | 154 | Python | 37 | AndroidWorld is an environment and benchmark for autonomous agents | 2026-04-09 |
| 24 | OpenCUA | 766 | 99 | Python | 14 | [NeurIPS 2025 Spotlight] OpenCUA: Open Foundations for Computer-Use Agents | 2026-02-04 |
| 25 | deep_research_bench | 730 | 77 | Python | 19 | DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents | 2026-05-11 |
| 26 | TheAgentCompany | 710 | 116 | Python | 22 | An agent benchmark with tasks in a simulated software company. | 2025-11-17 |
| 27 | ClawProBench | 690 | 50 | Python | 0 | ClawProBench is a live-first benchmark harness for evaluating LLM agents in the OpenClaw runtime with deterministic grading and repeated-trial reliability. | 2026-05-19 |
| 28 | Rankify | 676 | 69 | Python | 1 | π₯ Rankify: A Comprehensive Python Toolkit for Retrieval, Re-Ranking, and Retrieval-Augmented Generation π₯. Our toolkit integrates 40 pre-retrieved benchmark datasets and supports 7+ retrieval techni... | 2026-05-06 |
| 29 | BenchMARL | 623 | 130 | Python | 28 | BenchMARL is a library for benchmarking Multi-Agent Reinforcement Learning (MARL). BenchMARL allows to quickly compare different MARL algorithms, tasks, and models while being systematically grounded ... | 2026-02-07 |
| 30 | MLGym | 600 | 58 | Python | 8 | MLGym A New Framework and Benchmark for Advancing AI Research Agents | 2025-08-10 |
| 31 | MCP-Universe | 588 | 82 | Python | 17 | MCP-Universe is a comprehensive framework designed for RL training, benchmarking, and developing AI agents for general tool-use. | 2026-05-18 |
| 32 | AgentLab | 583 | 112 | Python | 20 | AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility. | 2026-03-17 |
| 33 | agentdojo | 572 | 149 | Python | 17 | A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents. | 2026-03-30 |
| 34 | VectorizedMultiAgentSimulator | 569 | 110 | Python | 7 | VMAS is a vectorized differentiable simulator designed for efficient Multi-Agent Reinforcement Learning benchmarking. It is comprised of a vectorized 2D physics engine written in PyTorch and a set of ... | 2026-05-19 |
| 35 | awesome-agent-harness | 547 | 41 | Python | 0 | An awesome list of Agent Harness engineering resources, including GitHub projects, tools, benchmarks, and practical guides. | 2026-05-22 |
| 36 | crafter | 543 | 92 | Python | 9 | Benchmarking the Spectrum of Agent Capabilities | 2024-01-23 |
| 37 | TravelPlanner | 513 | 75 | Python | 0 | [ICML'24 Spotlight] "TravelPlanner: A Benchmark for Real-World Planning with Language Agents" | 2025-11-07 |
| 38 | PaperOrchestra | 512 | 72 | Python | 0 | An automated AI research-paper writer based off Google's PaperOrchestra paper's implementation through a skills - benchmark + autoraters using any coding agent (Claude Code, Cursor, Antigravity, Clin... | 2026-05-15 |
| 39 | bigcodebench | 501 | 72 | Python | 26 | [ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI | 2026-01-03 |
| 40 | cap-x | 498 | 77 | Python | 3 | A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation | 2026-04-09 |
| 41 | meta-agents-research-environments | 497 | 66 | Python | 14 | Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments w... | 2026-05-21 |
| 42 | mcp-bench | 484 | 60 | Python | 16 | MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers | 2025-10-07 |
| 43 | visualwebarena | 474 | 79 | Python | 21 | VisualWebArena is a benchmark for multimodal agents. | 2024-11-09 |
| 44 | safety-starter-agents | 463 | 111 | Python | 8 | Basic constrained RL agents used in experiments for the "Benchmarking Safe Exploration in Deep Reinforcement Learning" paper. | 2023-04-02 |
| 45 | ParseBench | 459 | 54 | Python | 2 | ParseBench - A Document Parsing Benchmark for AI Agents | 2026-05-15 |
| 46 | VLABench | 433 | 36 | Python | 33 | Official repo of VLABench, a large scale benchmark designed for fairly evaluating VLA, Embodied Agent, and VLMs. | 2025-11-11 |
| 47 | models | 426 | 17 | Rust | 0 | TUI and CLI for browsing AI models, benchmarks, coding agents, and statuses for AI providers. | 2026-05-22 |
| 48 | DAComp | 426 | 1 | Python | 1 | [ICLR 2026] DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle | 2026-01-04 |
| 49 | OmniSearch | 424 | 33 | Python | 11 | Repo for Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent | 2025-04-22 |
| 50 | Awesome-Agent-Memory | 423 | 38 | - | 0 | Curated systems, benchmarks, and papers etc. on memory for LLMs/MLLMs --- long-term context, retrieval, and reasoning. | 2026-05-21 |
| 51 | appworld | 423 | 68 | Python | 7 | π AppWorld: A Controllable World of Apps and People for Benchmarking Function Calling and Interactive Coding Agent, ACL'24 Best Resource Paper. | 2026-02-17 |
| 52 | crab | 421 | 61 | Python | 34 | π¦οΈ CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents. https://crab.camel-ai.org/ | 2026-05-15 |
| 53 | mcpmark | 420 | 36 | Python | 12 | MCPMark is a comprehensive, stress-testing MCP benchmark designed to evaluate model and agent capabilities in real-world MCP use. | 2026-05-13 |
| 54 | WildClawBench | 399 | 35 | Python | 2 | An in-the-wild benchmark for AI agents in the OpenClaw Environment. | 2026-05-19 |
| 55 | BoxPwnr | 398 | 48 | Python | 0 | A modular framework for benchmarking LLMs and agentic strategies on security challenges across HackTheBox, TryHackMe, PortSwigger Labs, Cybench, picoCTF and more. | 2026-05-20 |
| 56 | Agent_Memory_Techniques | 382 | 55 | Jupyter Notebook | 0 | Agent memory for LLMs: 30 runnable Jupyter notebooks covering conversation buffers, vector stores, knowledge graphs, episodic and semantic memory, MemGPT, Mem0, Letta, Zep, Graphiti, LoCoMo benchmarks... | 2026-05-15 |
| 57 | multiagent_mujoco | 372 | 35 | Python | 5 | Benchmark for Continuous Multi-Agent Robotic Control, based on OpenAI's Mujoco Gym environments. | 2023-03-16 |
| 58 | Toolathlon | 367 | 40 | Python | 7 | [ICLR 2026] The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution | 2026-05-18 |
| 59 | Agents_Failure_Attribution | 364 | 25 | Python | 2 | Benchmark for automated failure attributions in agentic systems (π ICML 2025 Spotlight) | 2026-02-11 |
| 60 | harvey-labs | 328 | 57 | Python | 2 | A benchmark built to evaluate and improve agent capabilities for supporting legal work. | 2026-05-21 |
| 61 | LAMM | 317 | 16 | Python | 9 | [NeurIPS 2023 Datasets and Benchmarks Track] LAMM: Multi-Modal Large Language Models and Applications as AI Agents | 2024-04-16 |
| 62 | ClawBench | 315 | 19 | Python | 42 | Open-source benchmark for browser AI agents on 153 everyday online tasks across 144 live websites. 5-layer recording + DOM-match + LLM judge. Top score 33.3%. | 2026-05-22 |
| 63 | AgentClinic | 307 | 51 | Python | 4 | Agent benchmark for medical diagnosis | 2024-12-31 |
| 64 | PaperGuru-Benchmark | 305 | 47 | TeX | 0 | Lifecycle-Aware Memory for long-horizon LLM agents β 66.05% on PaperBench, 94.66% on SurveyBench, 10 peer-reviewed acceptances at FSE/ICML/TOSEM/AEI/ICoGB | 2026-05-14 |
| 65 | EmbodiedBench | 305 | 36 | Python | 6 | [ICML 2025 Oral] Official repo of EmbodiedBench, a comprehensive benchmark designed to evaluate MLLMs as embodied agents. | 2026-04-28 |
| 66 | AEnvironment | 301 | 36 | Python | 9 | Standardized environment infrastructure for Agentic AI development. | 2026-05-22 |
| 67 | embodied-agent-interface | 290 | 27 | Python | 15 | Embodied Agent Interface (EAI): Benchmarking LLMs for Embodied Decision Making (NeurIPS D&B 2024 Oral) | 2025-03-06 |
| 68 | CyberStrike | 282 | 55 | TypeScript | 2 | AI-powered offensive security agent with 7,300+ actionable security skills. Autonomous pentesting powered by MITRE ATT&CK (2,000+ Atomic tests), CIS Benchmarks (1,500+ controls), OWASP, NIST. Lazy-loa... | 2026-05-21 |
| 69 | BrowseComp-Plus | 277 | 46 | Python | 5 | BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent (ACL 2026 Main) | 2025-12-11 |
| 70 | Auto-GPT-Benchmarks | 275 | 71 | Jupyter Notebook | 2 | A repo built for the purpose of benchmarking the performance of agents, regardless of how they are set up and how they work. | 2023-12-09 |
| 71 | DeepFund | 275 | 46 | Python | 0 | π₯[NeurIPS'25] DeepFund: Pilot for Your Next Fund Investment | 2026-03-18 |
| 72 | Awesome-GraphMemory | 273 | 15 | - | 1 | A survey of Graph-based Agent Memory | A curated list of resources (surveys, papers, benchmarks, and opensource projects) on graph-based agent memory. | 2026-04-01 |
| 73 | MedAgentBench | 270 | 59 | Python | 2 | MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents | 2025-11-21 |
| 74 | sweet_rl | 268 | 12 | Python | 7 | Benchmark and research code for the paper SWEET-RL Training Multi-Turn LLM Agents onCollaborative Reasoning Tasks | 2025-05-05 |
| 75 | VideoGen-Eval | 266 | 14 | - | 5 | VideoGen-Eval: Agent-based System for Video Generation Evaluation | 2025-12-16 |
| 76 | GitTaskBench | 255 | 20 | Python | 1 | Repo-level benchmark for real-world Code Agents: from repo understanding β env setup β incremental dev/bug-fixing β task delivery, with cost-aware Ξ± metric. | 2025-09-22 |
| 77 | BALROG | 254 | 43 | Python | 0 | Benchmarking Agentic LLM and VLM Reasoning On Games | 2026-04-09 |
| 78 | deadend-cli | 241 | 42 | Python | 6 | Agentic pentest tooling. Currently achieving 81% (KIMI K2.5) on XBOW's benchmark in full black-box. Completely Self-hosted. Every model available on LiteLLM (Ollama, anthropic, openai...) | 2026-05-16 |
| 79 | myclaw-bench | 237 | 39 | Python | 0 | The definitive benchmark for AI agents on OpenClaw. 45 tasks across 4 tiers. Powered by MyClaw.ai | 2026-03-09 |
| 80 | ts-bench | 234 | 11 | TypeScript | 5 | Benchmark CLI for comparing AI coding agents on TypeScript workloads. | 2026-04-07 |
| 81 | agent-studio | 232 | 30 | Python | 1 | [ICLR 2025] A trinity of environments, tools, and benchmarks for general virtual agents | 2025-06-16 |
| 82 | cve-bench | 225 | 45 | Python | 1 | CVE-Bench: A Benchmark for AI Agentsβ Ability to Exploit Real-World Web Application Vulnerabilities | 2026-01-14 |
| 83 | SanityHarness | 221 | 4 | Go | 0 | A lightweight harness designed to be simple to use, efficient, universially compatible with any coding agent to evaluate them over a broad set of coding tasks and languages. | 2026-04-17 |
| 84 | OpAgent | 219 | 23 | Python | 0 | The OpAgent framework is used to automate web browser operations, achieving State-of-the-Art (SOTA) performance on the WebArena benchmark. | 2026-05-11 |
| 85 | VEFX-Bench | 218 | 17 | Python | 1 | VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects | 2026-05-16 |
| 86 | MobileWorld | 209 | 33 | Python | 13 | Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments (ACL 2026) | 2026-05-11 |
| 87 | AgentJet | 208 | 19 | Python | 2 | Cutting-edge platform for LLM agent tuning. Deliver RL tuning with flexibility, reliability, speed, multi-agent optimization and realtime community benchmarking. | 2026-05-20 |
| 88 | ComfyBench | 202 | 10 | Python | 2 | Implementation for the paper "ComfyBench: Benchmarking LLM-based Agents in ComfyUI for Autonomously Designing Collaborative AI Systems". | 2025-12-24 |
| 89 | Awesome-Agent-Memory-Papers | 186 | 6 | - | 0 | Awesome Papers related to Agent Memory: methods, benchmarks and surveys. Website: https://yyyujintang.github.io/Awesome-Agent-Memory-Papers/ | 2026-04-25 |
| 90 | Awesome-Self-Evolving-Agents | 185 | 13 | - | 2 | A Survey of Self-Evolving Agents | A curated list of resources (surveys, papers, benchmarks, and opensource projects) on Self-Evolving Agents. | 2026-05-02 |
| 91 | gameworld | 179 | 6 | Python | 0 | GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents | 2026-05-13 |
| 92 | claw-bench | 170 | 18 | Python | 3 | The Definitive AI Agent Benchmark | 2026-05-21 |
| 93 | DeepPHY | 170 | 6 | Python | 0 | DeepPHY: Benchmarking Agentic VLMs on Physical Reasoning | 2025-11-16 |
| 94 | SWE-CI | 168 | 20 | Python | 0 | SWE-CI: Evaluating Agent Capabilities in Maintaining Codebases via Continuous Integration | 2026-04-23 |
| 95 | LLM-Agent-Benchmark-List | 168 | 11 | - | 3 | A banchmark list for evaluation of large language models. | 2026-05-12 |
| 96 | SGI-Bench | 167 | 4 | Python | 0 | Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows | 2026-05-13 |
| 97 | OSWorld-G | 166 | 7 | TypeScript | 11 | [NeurIPS 2025 Spotlight] Scaling Computer-Use Grounding via UI Decomposition and Synthesis | 2025-11-06 |
| 98 | weblinx | 160 | 17 | Python | 1 | WebLINX is a benchmark for building web navigation agents with conversational capabilities | 2025-02-11 |
| 99 | ai-coding-lang-bench | 157 | 66 | Ruby | 6 | Which programming language is best for AI coding agents? Benchmarking 13 languages with Claude Code. | 2026-03-05 |
| 100 | WorfBench | 151 | 8 | Python | 1 | [ICLR 2025] Benchmarking Agentic Workflow Generation | 2025-02-19 |