OpenChainBench

TUTORIAL

Submit a benchmark.

Anyone can publish on OpenChainBench. You write the harness, you host it, you keep your secrets. The project shares one Prometheus that scrapes your public /metrics endpoint, the only piece of common infrastructure.

  1. 01

    Open an issue

    Align on the metric.

  2. 02

    Write the spec

    One YAML file.

  3. 03

    Build the harness

    Expose /metrics.

  4. 04

    Host it

    Anywhere with HTTPS.

  5. 05

    Wire the scrape

    One block in prometheus.yml.

  6. 06

    Open a PR

    The page renders itself.

HOW IT ACTUALLY WORKS

A federation, not a platform.

Every benchmark on OpenChainBench is run by its author. You pick the providers, write the harness, host it on Fly, Railway, Cloud Run or your own VPS. There is no shared runtime, no shared credentials, no central queue. The harness exposes a public /metrics endpoint over HTTPS, the entire integration surface.

One Prometheus scrapes every published harness. To wire yours in, add one block to scrape_configs inside infrastructure/prometheus/prometheus.yml and open a PR. Maintainers reload Prometheus, the site picks up the new spec, your benchmark renders itself.

No team can gate-keep a metric, no outage on one harness affects another, and the cost of running a benchmark stays with the person who cares about it.

OPTIONAL

Drafting with AI

A single source-of-truth skill walks any agent through the six steps with the exact conventions a reviewer will check. Pick the surface that matches your tooling.

any LLM

Web brief

Plain markdown. Paste into ChatGPT, Claude, Cursor, Aider, Codex, Continue or any chat-based agent.

View raw

Claude Code

ClawHub skill

One-line install with the ClawHub CLI. The skill auto-loads as a slash command and applies the conventions on every reply.

$ openclaw skills install openchainbench-contributor
ClawHub listing ↗

If you have already cloned the repo, Claude Code picks up the same skill from .claude/skills/contribute-benchmark/ without any install.

WALK-THROUGH

What each step really involves.

Roughly three to four hours of focused work, spread across a couple of days.

  1. 01

    Open an issue

    Describe the metric and the providers you intend to compare. Maintainers reply within a day to align on methodology: what counts as a sample, the cadence, which percentiles matter, and how to label cohorts. Output: a one-paragraph spec you both agree on.

  2. 02

    Write the spec

    One YAML file at benchmarks/<slug>.yml with title, description, the Prometheus query, the dimensions (chain / region / asset), and the providers it ranks. The site reads it at build time and renders the page; you never touch React.

  3. 03

    Build the harness

    Any language. Only contract: a public /metrics endpoint in the Prometheus format. Hold your own credentials, emit one labelled time-series per observation. Reference harnesses in mobula-api/openchainbench-app and miniapps/.

  4. 04

    Host it

    Anywhere with HTTPS: Fly, Railway, Cloud Run, your VPS, a Pi. Cost stays with the person who cares. Verify the endpoint answers a 200 OK from the public internet before moving on.

  5. 05

    Wire the scrape

    One block under scrape_configs in infrastructure/prometheus/prometheus.yml pointing at your endpoint, with the labels your spec expects. Only file in shared infrastructure you ever touch.

  6. 06

    Open a PR

    Spec + scrape config in one PR. Maintainers review, merge, and reload Prometheus. The page picks up the new series within 60 s. From that moment on the benchmark is yours.