Crab
Framework for building LLM agent benchmark environments in a Python-centric way.
🛡️ AgentReady threat assessment
MAESTRO 7-layer threat model + OWASP AIVSS risk score for Crab, derived from its capabilities.
These scores are auto-generated from public information (the agent's own listing, docs, and repository) using the canonical OWASP AIVSS formula and the MAESTRO framework — an estimate for guidance, not a penetration test, audit, or certification. See the scoring methodology — every score is re-derived by the same automated method as an agent's public evidence changes.
Overview
Crab is a comprehensive framework designed by Camel AI for building and benchmarking environments tailored for large language model (LLM) agents. The platform supports the creation of cross-platform environments, allowing for deployment across in-memory systems, Docker-hosted environments, virtual machines, or distributed physical machines. Crab provides an easy-to-use Python-centric interface for defining agent environments and actions, making it flexible for various use cases. Additionally, it includes a novel benchmarking suite that provides fine-grained evaluation metrics.
Key features and capabilities
- Cross-platform & Multi-environment Deployment,
- Unified Interface for Environment Access,
- Python-native Configuration,
- Novel Benchmarking Suite,
- Fine-grained Graph Evaluator.
Use cases
- Benchmarking LLM Agents,
- Cross-environment Testing,
- Multimodal Data Handling,
- Agent Environment Simulation,
- Python-based Agent Development.