Indicator Audit · Batch 3
The strongest published evidence, judged on its own claims
We audited six strategies with some of the strongest support in the literature: volatility-managed equity, the low-volatility anomaly, time-series momentum, Faber’s timing model and risk parity. Each was judged on the objective its own source claims, declared and sealed before this batch’s data fetch; we had previously seen prices for SPY, EFA, IEF and the sector SPDRs in earlier audits (disclosed below). None of the six meets its claim under our test (0 of 6 survive). One came close, and only half of its claim held.
The closest call: Faber timing on EEM, VNQ and TLT
These are markets we had never examined. The timing rule cut drawdowns, robustly. The maximum drawdown was 14.5% against 43.9% for holding the basket, the bootstrap 5th percentile of the reduction was 3.5%, and the Holm-adjusted p is 0.0006. Every check passed except one.
Faber’s claim is lower drawdowns at similar returns. Before the run we declared that giving up at most 1.0% a year would count. It gave up 3.2% a year. So its grade is capped at C, and it does not count as meeting the claim.
The rest
Volatility-managed SPY (Moreira & Muir) raised the Sharpe ratio from 0.65 to 0.79. But the improvement did not survive the bootstrap and the 12-variant correction (grade D).
Low volatility (a sector rule and USMV), time-series momentum and risk parity did not improve the Sharpe ratio on this history. Time-series momentum compounded faster than holding the same ETFs only through leverage financed at 0%.
We added a new objective, “improve Sharpe vs benchmark”, as rubric v1.2 and sealed it before this batch. No earlier audit was re-graded. Statistics on past data only; not advice, and not a verdict on the papers.
Equities and equity ETFs
| Rule | Market | Objective (its own claim) | Variants | Grade | Primary result | Holm p | Survives |
|---|---|---|---|---|---|---|---|
| Volatility-managed equity (Moreira & Muir) | SPY | (c) improve Sharpe | 12 | D | Sharpe 0.79 vs 0.65 (improvement +0.14, 5th pct −0.01); holdout +0.17 → +0.07 | 1.0000 | no |
| Low-volatility sectors (low-volatility anomaly) | 9 sector SPDRs | (c) improve Sharpe | 9 | F | Sharpe 0.62 vs 0.51 (improvement +0.12, 5th pct −0.06); holdout +0.20 → −0.08 | 1.0000 | no |
| Minimum-volatility ETFs (USMV, SPLV) | USMV / SPLV | (c) improve Sharpe | 2 | F | Sharpe 0.88 vs 0.92 (improvement −0.04, 5th pct −0.25); holdout +0.02 → −0.23 | 1.0000 | no |
Multi-asset and bonds
| Rule | Market | Objective (its own claim) | Variants | Grade | Primary result | Holm p | Survives |
|---|---|---|---|---|---|---|---|
| Time-series momentum (Moskowitz, Ooi & Pedersen) | SPY, EFA, EEM, IEF, TLT | (c) improve Sharpe | 8 | F | Sharpe 0.53 vs 0.65 (improvement −0.13, 5th pct −0.52); holdout −0.02 → −0.28 | 1.0000 | no |
| Faber GTAA timing on unseen markets (EEM, VNQ, TLT) | EEM, VNQ, TLT | (b) reduce drawdown | 8 | C | max drawdown 14.5% vs 43.9% (5th pct of the cut 3.5%); return cost 3.2%/yr vs declared 1.0% | 0.0006 | no |
| Risk parity stocks/bonds (Asness, Frazzini & Pedersen) | SPY / TLT | (c) improve Sharpe | 8 | D | Sharpe 0.79 vs 0.79 (improvement +0.00, 5th pct −0.17); holdout +0.17 → −0.16 | 1.0000 | no |
Sources. Moreira & Muir (2017), J. Finance; Blitz & van Vliet (2007), JPM, and Baker, Bradley & Wurgler (2011), FAJ; Moskowitz, Ooi & Pedersen (2012), JFE; Faber (2007), J. Wealth Management; Asness, Frazzini & Pedersen (2012), FAJ. Full citations are in each report.
Seals. Rubric v1.2 was sealed at 20:33:45 UTC; the preregistration and six declarations at 20:39 UTC on 2026-09-25, before the new data were fetched. Preregistration SHA-256 2142d17c29db1f2b0080e225cfbcd15ba67582b4122adcc6c78b0aad1325d690; seal record 90ff232e75d552219386a15d1bc0990a86b40a925d2a7dc1405e7be67995e009.
Disclosures. Before this batch we had seen prices for SPY, EFA, IEF and the sector SPDRs, but not for EEM, TLT, VNQ, USMV or SPLV. The volatility-managed and sector audits therefore used prices we had seen. Cash earns 0% and borrowing costs 0%. Commodities are excluded from time-series momentum. There were no deviations from the preregistered protocol.
Prices from Yahoo Finance’s public chart endpoint; we publish derived statistics only, never raw prices; no redistribution.
These audits analyze published rules on historical data for illustration. They are not recommendations to trade any of these strategies, and a grade is not a prediction.
Want your own backtest audited?
The same sealed rubric and tests, applied to your strategy, with a written report and fix list.
Send a high-level description. We’ll confirm scope, then send payment and secure-upload instructions.
Out of Sample · newsletter
Get new audits by email
Out of Sample, free, no spam. Every result published, including the failures.
You sign up on our newsletter page (hosted by Beehiiv); unsubscribe any time. Want a specific indicator tested? Request an audit.