Mcp Autotest

@strowk

About Mcp Autotest

Utility for autotesting MCP servers

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "mcp-autotest": {
      "command": "npx",
      "args": [
        "mcp-autotest",
        "run",
        "testdata",
        "--",
        "npx",
        "-y",
        "@modelcontextprotocol/server-postgres",
        "localhost:5432"
      ]
    }
  }
}

Tools

No tools detected

Fetch the live tool list by running this server in a temporary sandbox using the button above.

Overview

What is Mcp Autotest?

Mcp Autotest is a simple tool for testing MCP servers in a language-agnostic way. It uses MCP protocol by defining YAML files with requests and expected responses.

How to use Mcp Autotest?

Install via npm (npm install -g mcp-autotest), download prebuilt binaries from GitHub Releases, or build from source with Go. Run mcp-autotest [flags] run path/to/tests/folder [--] command-to-run-server [server-args]. Test case files must be multi-document YAML with _test.yaml suffix. By default stdio transport is used; for Streamable HTTP (POST only) add --url http://host:port/path.

Key features of Mcp Autotest

Language-agnostic MCP server testing via YAML definitions
Supports both stdio and Streamable HTTP transports
Dynamic matching with !!re tag for regular expressions
Embedded regex (!!ere) for partial string matching
Escaping forward slashes in embedded regex strings

Use cases of Mcp Autotest

Validate an MCP server’s responses against expected JSON-RPC messages
Automate regression testing for MCP server implementations
Test servers across different languages without writing test code
Verify dynamic responses (timestamps, UUIDs) using regex matching

FAQ from Mcp Autotest

How do I install Mcp Autotest?

Install globally via npm (npm install -g mcp-autotest), download a prebuilt binary from GitHub Releases, or build from source with go install github.com/strowk/mcp-autotest@latest.

What file naming convention is required for test cases?

Test case files must end with _test.yaml to be recognized.

Can I test an MCP server running over HTTP?

Yes, use the --url flag to specify the server URL. Only the POST method is currently supported for Streamable HTTP; GET support is planned.

How do I match dynamic values like timestamps in responses?

Use the !!re tag to treat a string as a regular expression for full match, or !!ere for embedded regex (parts between slashes / are treated as regex).

What transports does Mcp Autotest support?

It supports stdio transport by default and Streamable HTTP (via the --url flag). Only HTTP POST is implemented at this time.

Frequently asked questions

How do I install Mcp Autotest?

Install globally via npm (`npm install -g mcp-autotest`), download a prebuilt binary from GitHub Releases, or build from source with `go install github.com/strowk/mcp-autotest@latest`.

What file naming convention is required for test cases?

Test case files must end with `_test.yaml` to be recognized.

Can I test an MCP server running over HTTP?

Yes, use the `--url` flag to specify the server URL. Only the `POST` method is currently supported for Streamable HTTP; `GET` support is planned.

How do I match dynamic values like timestamps in responses?

Use the `!!re` tag to treat a string as a regular expression for full match, or `!!ere` for embedded regex (parts between slashes `/` are treated as regex).

What transports does Mcp Autotest support?

It supports stdio transport by default and Streamable HTTP (via the `--url` flag). Only HTTP POST is implemented at this time.

Comments

Basic information

More Other MCP servers

ghidraMCP

LaurieWired

MCP Server for Ghidra

9.4K

Mcp

browsermcp

Browser MCP is a Model Context Provider (MCP) server that allows AI applications to control your browser

6.8K

DeepMark

DeepMark helps teachers deliver rapid, consistent marking with meaningful feedback for every student — in a fraction of the time. What once took a week, now takes one free period.

IDA Pro MCP

mrexodia

AI-powered reverse engineering assistant that bridges IDA Pro with language models through MCP.

9.9K

Blender

ahujasid

Open-source MCP to use Blender with any LLM

23.6K