Indicator Audit · Batch 1
Ten TradingView built-in strategies, twenty sealed audits
We audited ten of TradingView’s built-in strategies with the settings they ship with, on SPY and on Bitcoin. The whole protocol was sealed before this batch’s data fetch; we had previously seen SPY and BTC prices in sample audits 1–4 (disclosed below). None of the 20 defaults beat simply holding the same asset, on average or compound return, and none showed a statistically robust edge: 0 of 20 survive Holm’s correction, and all 20 grade F.
How it was done
- Preregistered, then sealed. One document fixed, for all 20 audits: each rule and its defaults, a grid of the variants traders commonly try (250 in total), costs, data sources, dates, the holdout split, the objective (“beat buy-and-hold of the same asset”), Holm’s correction across the 20, and a promise to publish every result. It was timestamped by DigiCert and FreeTSA (RFC 3161) on 2026-09-25 at 18:21:44 UTC, before the SPY data were fetched.
- Run once each under our sealed rubric v1.1. Every report cites the seal.
- Batch test. Each audit’s Hansen SPA p-value (the best variant of its family against buy-and-hold) is corrected with Holm’s method across all 20. An audit “survives” only if its adjusted p is 0.05 or less.
What we found
Across all 20 families, not one of the 500 variant runs (250 variants × 2 assets) earned more on average than holding the asset after costs. Every SPA p-value was 1.000.
On SPY, holding compounded at 10.9% a year. Nine of the ten defaults lost money outright. On Bitcoin, holding compounded at 70.4% a year, and the closest default (MACD) managed 28.7%. The built-ins are stop-and-reverse, always long or short, so shorting assets that rose over the period was costly. Long-only variants did better. None had a positive average excess return, although 13 of the Bitcoin ones compounded faster than holding (best 86.4% vs 70.4% a year); none showed a statistically robust edge.
Statistics on past data only. This is not a verdict on indicators in general, and not a recommendation to use or avoid any of them.
All 20 results
SPY (US equities)
| Asset | Strategy (default, long/short) | Variants | Grade | Compound return a year vs holding | Excess Sharpe | DSR | PBO | Holdout excess Sharpe (in → out) | Holm p | Survives |
|---|---|---|---|---|---|---|---|---|---|---|
| SPY | Supertrend | 30 | F | −3.7% vs 10.9% | −0.46 | 0.000 | 0.77 | −0.38 → −0.66 | 1.000 | no |
| SPY | MACD | 48 | F | −4.1% vs 10.9% | −0.48 | 0.000 | 0.32 | −0.45 → −0.55 | 1.000 | no |
| SPY | RSI | 24 | F | −2.8% vs 10.9% | −0.59 | 0.000 | 0.37 | −0.50 → −0.82 | 1.000 | no |
| SPY | Bollinger Bands | 32 | F | 2.5% vs 10.9% | −0.40 | 0.000 | 0.43 | −0.32 → −0.60 | 1.000 | no |
| SPY | MovingAvg2Line Cross | 40 | F | −0.6% vs 10.9% | −0.37 | 0.000 | 0.20 | −0.37 → −0.38 | 1.000 | no |
| SPY | Parabolic SAR | 18 | F | −6.9% vs 10.9% | −0.61 | 0.000 | 0.29 | −0.69 → −0.41 | 1.000 | no |
| SPY | Stochastic Slow | 16 | F | −5.1% vs 10.9% | −0.59 | 0.000 | 0.67 | −0.48 → −0.92 | 1.000 | no |
| SPY | Channel BreakOut | 8 | F | −8.5% vs 10.9% | −0.65 | 0.000 | 0.14 | −0.67 → −0.59 | 1.000 | no |
| SPY | Keltner Channels | 24 | F | −2.6% vs 10.9% | −0.41 | 0.000 | 0.54 | −0.38 → −0.49 | 1.000 | no |
| SPY | Momentum | 10 | F | −1.1% vs 10.9% | −0.38 | 0.000 | 0.66 | −0.42 → −0.27 | 1.000 | no |
Bitcoin
| Asset | Strategy (default, long/short) | Variants | Grade | Compound return a year vs holding | Excess Sharpe | DSR | PBO | Holdout excess Sharpe (in → out) | Holm p | Survives |
|---|---|---|---|---|---|---|---|---|---|---|
| BTC | Supertrend | 30 | F | 2.4% vs 70.4% | −0.52 | 0.006 | 0.32 | −0.47 → −0.76 | 1.000 | no |
| BTC | MACD | 48 | F | 28.7% vs 70.4% | −0.29 | 0.072 | 0.50 | −0.19 → −0.68 | 1.000 | no |
| BTC | RSI | 24 | F | −37.9% vs 70.4% | −0.95 | 0.000 | 0.77 | −0.98 → −0.95 | 1.000 | no |
| BTC | Bollinger Bands | 32 | F | −47.6% vs 70.4% | −1.17 | 0.000 | 0.28 | −1.26 → −0.91 | 1.000 | no |
| BTC | MovingAvg2Line Cross | 40 | F | 9.6% vs 70.4% | −0.47 | 0.000 | 0.19 | −0.42 → −0.71 | 1.000 | no |
| BTC | Parabolic SAR | 18 | F | −11.6% vs 70.4% | −0.67 | 0.000 | 0.08 | −0.67 → −0.75 | 1.000 | no |
| BTC | Stochastic Slow | 16 | F | −57.2% vs 70.4% | −1.28 | 0.000 | 0.75 | −1.33 → −1.22 | 1.000 | no |
| BTC | Channel BreakOut | 8 | F | −0.8% vs 70.4% | −0.56 | 0.010 | 0.71 | −0.54 → −0.67 | 1.000 | no |
| BTC | Keltner Channels | 24 | F | 15.4% vs 70.4% | −0.45 | 0.003 | 0.35 | −0.48 → −0.38 | 1.000 | no |
| BTC | Momentum | 10 | F | 19.5% vs 70.4% | −0.38 | 0.027 | 0.53 | −0.23 → −0.96 | 1.000 | no |
Seal. Preregistration SHA-256 35b4d77e284614c15fafdacaf91e122b600e520c1ea61e33627a64c3497c765d; seal record 9f25abc24d54753071fc54da00bda52705ea2e7c0b6d2ba3011d7c009fa22d5c. The preregistration, receipts and code will be published with the public toolkit, which is being prepared (samples/batch1/).
Disclosures. BTC Supertrend is carried over unchanged from sample audit 4, as preregistered. The BTC price file predates this batch; the preregistration disclosed it and pinned its hash. Fills are approximated at the daily close, and short-side funding is not modelled. We had already seen SPY and BTC results from sample audits 1–4. There were no deviations from the preregistered protocol.
Prices from Yahoo Finance’s public chart endpoint (SPY) and the Coinbase Exchange public API (BTC); we publish derived statistics only, never raw prices; no redistribution.
These audits analyze public rules on historical data for illustration. They are not recommendations to trade any of these strategies, and a grade is not a prediction.
Want your own backtest audited?
The same sealed rubric and tests, applied to your strategy, with a written report and fix list.
Send a high-level description. We’ll confirm scope, then send payment and secure-upload instructions.
Out of Sample · newsletter
Get new audits by email
Out of Sample, free, no spam. Every result published, including the failures.
You sign up on our newsletter page (hosted by Beehiiv); unsubscribe any time. Want a specific indicator tested? Request an audit.