Cross-oracle price deviation across 10 USD pairs
Maximum pairwise price disagreement in basis points across Chainlink, Pyth, Binance and Coinbase, polled every 30 s.
Read this carefully
Three caveats. (1) Lower is better, but a non-zero floor (~5-20 bps) is normal, ticker last-traded prices have micro-jitter and Chainlink updates only on deviation/heartbeat. (2) XRP, ADA, DOGE measured on 3 sources because Chainlink deprecated those USD feeds on Ethereum mainnet, tag flags it. (3) POL/USD is the post-migration symbol for MATIC; we point at POLUSDT (not the frozen legacy MATICUSDT).
This benchmark answers the question every protocol designer picking an oracle asks. how aligned are the major price feeds in reality, not in marketing, on the exact pair I'm settling at this exact second. Most "oracle comparison" articles cite vendor whitepapers ("99.9% accurate", "sub-second latency") without publishing the diff. We poll four oracles every 30 seconds for ten USD-quoted blue chips, compute the pairwise deviation `|a-b|/avg * 100` for every source pair, and publish the per-asset maximum as a Prometheus gauge in basis points. Coverage. Chainlink AggregatorV3 (eth_call against the canonical Ethereum mainnet contracts), Pyth Hermes REST (batched latest_price_feeds), Binance REST ticker (USDT-quoted, treated as ≈ USD), Coinbase REST ticker (USD-quoted). Pairs. BTC, ETH, SOL, BNB, AVAX, LINK, POL (4 oracles each) and XRP, ADA, DOGE (3 oracles, Chainlink mainnet feeds for these pairs are deprecated, documented per provider). Honest about the SOL case. SOL/USD currently shows the widest disagreement (~1.2% live) not because the bench is broken but because the Chainlink on-chain SOL/USD feed is updated on a 0.5% deviation trigger that frequently lags the CEX/Pyth quote during fast moves, exactly the kind of oracle-health signal a protocol team needs to see before settling perp liquidations on it.
Methodology
We measure live oracle disagreement by polling four price oracles every 30 seconds for ten USD-quoted blue chips, then computing the full pairwise deviation matrix per pair. Deviation between two sources is `|price_a, price_b| / ((price_a + price_b) / 2) * 100` in percent; the bench surfaces it both per source-pair (`ocb_oracle_deviation_pct{source_a, source_b}`) and as the per-pair maximum across all source pairs (`ocb_oracle_max_deviation_pct`). The headline leaderboard ranks the 10 pairs by p99 of the max deviation over 24 h (lower = better cross-oracle alignment, i.e. more agreement on what the asset actually trades at). A stale-price guard skips any sample older than `2 * pollInterval` (60 s) so a dead poller doesn't artificially flatten the deviation. Three pairs (XRP, ADA, DOGE) are measured on 3 sources only because Chainlink deprecated those USD feed contracts on Ethereum mainnet; the bench exposes them honestly with the disclaimer in the provider tag and still computes the deviation across the remaining trio. The `ocb_oracle_last_round_age_seconds{source="chainlink"}` companion gauge surfaces Chainlink's on-chain update gap separately, so a high deviation can be attributed to the right cause (a lagging feed vs. a CEX outlier) without confusing the reader.
Frequently asked
What is oracle deviation and why does it matter?
Oracle deviation is the basis-point gap between two oracles' reported price for the same asset at the same moment. It matters because every on-chain product that settles in fiat (perp DEXes, lending markets, derivatives, stablecoin issuers) picks one oracle and inherits its drift. If your lending market uses Chainlink ETH/USD and Chainlink lags the CEX print by 30 bps during a fast move, every liquidation triggered in that window over- or under-pays the keeper by 30 bps, which is the difference between a healthy liquidation engine and a bad-debt spiral. The bench measures this gap live so you can size the risk before you ship the integration.
Why is SOL deviation so much higher than BTC?
Two reasons, both structural and both informative. (1) Chainlink's SOL/USD mainnet feed is configured with a 0.5% deviation trigger and a 1-hour heartbeat, so during fast SOL moves the on-chain price legitimately lags the live CEX print until the trigger fires. (2) SOL is more volatile per unit time than BTC/ETH, so even within the trigger window there's more price travel between updates. The companion gauge `ocb_oracle_last_round_age_seconds{pair="SOL/USD"}` regularly shows ages in the multiple thousands of seconds (vs ~1000-1500 s for ETH/BTC/AVAX/LINK), which is the on-chain confirmation that the deviation is Chainlink lag, not a CEX outlier. The bench surfaces this honestly, it's exactly the signal a protocol team needs to see before settling SOL perps off a single oracle.
What does 'Chainlink lag' mean in practice, does the oracle break?
Chainlink doesn't break, but it intentionally doesn't push every tick. On-chain updates cost gas; Chainlink node operators only post a new round when either (a) the off-chain aggregated price has drifted by a configured threshold from the last on-chain value (the 'deviation trigger', typically 0.25-0.5% for blue chips, 1% for long-tail) or (b) the configured heartbeat interval has elapsed since the last update (typically 1 h for blue chips, longer for stale-tolerant assets). Between updates, the on-chain value is whatever was last pushed, so during a fast move the on-chain price can lag the off-chain market by 1× the deviation threshold for as long as it takes the next round to land. That's by design: it trades latency for gas cost. The bench's `ocb_oracle_last_round_age_seconds` gauge surfaces the gap directly so the lag is never invisible.
Why are XRP, ADA and DOGE measured on only 3 sources?
The Chainlink mainnet AggregatorV3 contracts for XRP/USD, ADA/USD and DOGE/USD are deprecated, node operators no longer post fresh rounds to them, so the on-chain `updatedAt` is days or weeks stale. Rather than averaging in a fossilized value (which would either falsely deflate the deviation by anchoring it to an outdated number or falsely inflate it once the live market moves away), the bench excludes Chainlink from the deviation matrix for these three pairs and computes the disagreement across Pyth + Binance + Coinbase only. The provider tag for each of these pairs flags '3 sources' explicitly. Protocols settling XRP, ADA or DOGE on Ethereum mainnet through Chainlink should confirm the feed's status directly, these are tier-2 assets on Chainlink's Ethereum L1 deployment and the production feeds for them live on other chains (BSC, Solana for ADA/DOGE).
MATIC vs POL, what changed and which one is this bench measuring?
Polygon migrated its native token from MATIC to POL on September 4 2024, 1:1. POL is the new token; MATIC is the legacy ticker. The four oracles handled it differently: Pyth renamed the feed to POL/USD; Coinbase delisted MATIC-USD and lists only POL-USD; Binance kept MATICUSDT as a frozen historical pair (don't use it, it detaches from spot) AND lists POLUSDT; Chainlink left the mainnet feed contract at its old address with the legacy `description()` of 'MATIC / USD' but it tracks the POL token. The bench resolves all four to POL and keeps the label `pair="MATIC/USD"` for query continuity. The reported price is POL/USD and all four sources agree on the same underlying asset.
Why these four oracles and not Redstone, Uniswap TWAP or DIA?
Two filters: 'free no-auth public endpoint' and 'continuous gauge'. Chainlink, Pyth Hermes, Binance and Coinbase all expose continuously-updating price feeds via REST or eth_call with no API key, which lets the bench scrape them at 30 s cadence indefinitely at zero cost. Redstone is push-pull/on-demand, the on-chain price is only refreshed when a transaction includes the signed off-chain quote in calldata, so there is no continuous gauge to scrape without deploying an integration contract that pulls on a timer. Uniswap V3 TWAP is a derivation of the same CEX prints we already poll directly, plus it needs per-pool integration code. DIA has smaller deployment footprint than the four kept. We may add Redstone and a pure-on-chain reference (TWAP) in a follow-up bench that runs on-chain itself; for the off-chain comparison this is the honest set.
How often is each oracle polled and why 30 s?
Every (source, pair) is polled every 30 seconds. The cadence is bounded below by Chainlink's update mechanics, the on-chain feed only refreshes on deviation or heartbeat, so polling faster than the feed updates returns the same value and wastes RPC calls. 30 s is fast enough to catch every Chainlink round within ~half a heartbeat of its on-chain landing while keeping total request volume at 80 req/min across all 4 sources × 10 pairs (well under every free-tier ceiling). Pyth Hermes, Binance and Coinbase tickers refresh continuously so 30 s polling captures them at full freshness.
Source code github.com/OpenChainBench/OpenChainBench/tree/main/harnesses/oracle-deviation
