Prowl

@opcastil11

About Prowl

Open-source benchmark runner for the Prowl Agent Discovery Network

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "prowl-bench": {
      "command": "python",
      "args": [
        "-m",
        "venv",
        ".venv",
        "&&",
        "source",
        ".venv/bin/activate"
      ]
    }
  }
}

Tools

No tools detected

We auto-extract tools from the README. The maintainer can list them under a ## Tools heading to populate this section.

Overview

What is Prowl?

Prowl is an open-source benchmark runner that evaluates any API across 8 dimensions of agent-readiness using multi-LLM scoring. It measures how easily AI agents can discover, understand, and use an API, targeting developers who want to optimize their APIs for the agent economy.

How to use Prowl?

Install via pip install prowl-bench, set at least one LLM API key (Anthropic, OpenAI, or Google), then run prowl-bench run <url>. Optionally specify a template, credentials, output format (JSON), or a minimum score for CI. Prowl also provides a Python API for programmatic use.

Key features of Prowl

Multi-LLM scoring across Claude, GPT-4o, and Gemini
8 weighted dimensions measuring agent efficiency
4-phase pipeline: Analyze, Plan, Execute, Interpret
Auto-detected benchmark templates (API, platform, MCP, docs, DeFi, crypto)
CI integration with --min-score flag
Community submission to the Prowl network

Use cases of Prowl

Benchmark your own API to improve agent-friendliness
Evaluate third-party APIs before integrating them into agent workflows
Catch agent-efficiency regressions in CI/CD pipelines
Compare API documentation quality across providers
Score MCP servers or DeFi protocols for agent readiness

FAQ from Prowl

What does Prowl measure that traditional API testing tools don’t?

Traditional tools measure uptime and response time; Prowl measures whether an AI agent can actually use the API across dimensions like token efficiency, first‑try success, error clarity, and response parseability.

What dependencies or runtime requirements does Prowl have?

Prowl requires Python 3.10+ and no system dependencies. You need at least one LLM API key (Anthropic, OpenAI, or Google) for scoring; more keys produce more balanced results.

What data does Prowl send to third parties during a benchmark?

Prowl sends extracted API spec data and test results to the configured LLM providers for scoring. Execution phase HTTP requests go through a sandbox that blocks SSRF and prompt injection. No data is shared with Prowl unless you explicitly use the --submit or --vendor-submit flags.

How does multi-LLM scoring work?

Each available LLM (Claude, GPT-4o, Gemini) independently scores all 8 dimensions. Scores are then averaged to reduce single-model bias. The JSON output includes per-model breakdowns.

What templates are available and how are they chosen?

Six templates are auto-detected from service metadata: api_benchmark, platform_profile, mcp_compliance, docs_quality, defi_yield, and crypto_app. You can also force a template with the --template flag.

Frequently asked questions