F
Sample audit 1 · US equities
SPY 50/200-day golden cross
Sharpe 0.77 over 1994–2026, but it trailed buy-and-hold SPY by 1.8% a year and fell further behind in the holdout.
Sample audits
Real reports on strategies everyone knows
These are complete audits of well-known public strategies, produced with the same toolkit and rubric as a paid audit. Read one before you request an audit.
Equities
F
Sample audit 2 · US equities
RSI(2) mean reversion on SPY
The best of 60 settings trailed buy-and-hold SPY by 5.9% a year; its excess Sharpe over buy-and-hold fell from −0.16 in-sample to −0.92 in the holdout.
More equity and bond audits: Batch 2, classic market rules (14 audits on SPY, QQQ, IWM, EFA, sector and Treasury ETFs) and Batch 1 on SPY (ten TradingView built-ins). All are sealed, and none survive. See every audit.
Crypto
F
Sample audit 3 · Crypto
Bitcoin 50-day moving-average momentum
Sharpe 1.47 over 2016–2026. Annualised average excess return −2.6 points a year vs holding Bitcoin; compound growth 80.8% vs 66.0% a year (different measures). Graded F: its excess-return evidence didn’t survive the overfitting tests.
F
Sample audit 4 · Crypto · preregistered and sealed
TradingView Supertrend Strategy (ATR 10, factor 3) on Bitcoin
The built-in default compounded at 2.4% a year against 70.4% for buy-and-hold Bitcoin. None of the 30 preregistered variants showed a statistically robust edge over holding (three long-only variants compounded faster, but all had a negative average excess return).
Objective and protocol sealed with RFC 3161 timestamps (DigiCert, FreeTSA) before the data were fetched. Graded under rubric v1.1.
On-chain Indicator Audit, Batch 1. Twelve famous Bitcoin cycle indicators, each frozen as first published and tested only after its publication date, sealed before the on-chain data were fetched: six price and block-header indicators (Pi Cycle Top, 200-week MA, Mayer Multiple, 2-Year MA Multiplier, Puell Multiple, stock-to-flow) and six UTXO metrics we computed ourselves from the public blockchain (MVRV Z-score, NUPL, SOPR, Reserve Risk, short-term-holder cost basis, realized price). None supported: three claims had enough independent episodes to test, and none survived; nine produced too few episodes for a reliable conclusion. See the results.
More crypto audits: Batch 1 on Bitcoin and the on-chain indicator audit.
How to read these. Samples 1–3: we sealed our grading rubric (v1.0) with two RFC 3161 timestamps, from DigiCert and FreeTSA, before the graded run. Sample 4 goes further: its objective and its whole protocol (rule, 30-variant grid, costs, dates, holdout) were sealed before its data fetch, exactly as we ask of clients (we had already seen Bitcoin prices in sample 3). Every rubric grades a rule against simply holding the asset it trades. None of the four showed a statistically robust edge over it. For the Bitcoin 50-day rule the two return measures point different ways: its average excess return was −2.6 points a year, while it compounded at 80.8% a year against 66.0%. Samples 1–3 did cut the worst drawdown (34% vs 55%, 36% vs 55% and 59% vs 84%); the Supertrend default did not (88% vs 84%). “Reduce drawdown at acceptable cost” is a separate objective that clients can choose, but only by declaring it before an audit. We had already seen the first three results, so we deliberately did not grade them on it after the fact.
Prices from Yahoo Finance’s public chart endpoint (SPY) and the Coinbase Exchange public API (BTC); we publish derived statistics only, never raw prices; no redistribution.
Sample audits analyze public rules on historical data for illustration. They are not recommendations to trade any of these strategies, and a grade is not a prediction.
Want your own backtest audited?
The same sealed rubric and tests, applied to your strategy, with a written report and fix list.
Send a high-level description. We’ll confirm scope, then send payment and secure-upload instructions.
Out of Sample · newsletter
Get new audits by email
Out of Sample, free, no spam. Every result published, including the failures.
Subscribe to Out of Sample
You sign up on our newsletter page (hosted by Beehiiv); unsubscribe any time. Want a specific indicator tested? Request an audit.