# SMARTBETA-1: analysis notes (implementation choices and interpretations, fixed before the run)

**Written:** 2026-09-26 (UTC), after the data download (17:11Z) and **before any return statistic was computed** from the real data. It is sealed with RFC 3161 timestamps (`holdout-audit seal-doc`, `analysis-notes-seal/`) before `run.py` is executed, and `run.py` refuses to run unless the seal verifies.

**Governs:** nothing on its own. `prereg/PREREG.md` (SHA-256 `42bfad1372704447674cde6efdf17064dd0b290e6be351659783c78d958b7c54`, sealed 2026-09-26T16:00:46Z; OSF https://osf.io/2tpn5) governs. This note records how its text is implemented where the text leaves a detail open. None of these items changes a test, an objective, a tolerance, a window, a cost, a seed or a decision rule. If any item were found to conflict with PREREG.md, PREREG.md wins and the conflict is reported as a deviation.

**Contact with the real data so far:** the download itself (`fetch_data.py`), file hashes (`MANIFEST.json`), and a structural look at each file: first and last dates, row counts and the French files' section headers and column names. The code was tested end to end on **synthetic** returns (the real trading dates with random-noise returns) in a scratch directory; nothing from that dry run is kept or published.

## Code and data fixed at sealing

| File | SHA-256 |
|---|---|
| `run.py` | `fe2445c5dfea3efd858d297e9552be22998059a6801b342a79385440b3e03740` |
| `sb_stats.py` | `fbd311c7ab4218e39be66b0441ecdc2cc0a4986743c6399c12f899103ca3774d` |
| `summarize.py` (formatting only) | `70066eceef24ef5b7e0502a4f29676bd4489764829e053713a0b0535c5e096b5` |
| `fetch_data.py` | `f3e6eb9e33a0a71adb603fe3ecbf95052d6f9fa7085b015475da0ddfe6c994fb` |
| `MANIFEST.json` (hashes of all 21 raw files) | `4b02268ab4cecdbcb41d6bc9adf7752e04729050157e52db4161d0aa0e094590` |

`summarize.py` only formats `results.json` into Markdown. Wording edits to it after the run are allowed if they do not change any number or verdict; they will be visible in git history.

## Data

1. **One download session.** All seven Yahoo series (MTUM, QUAL, VLUE, SIZE, USMV, SPLV, SPY) and the seven French files were downloaded on 2026-09-26 between 17:11:01Z and 17:11:20Z, after the seal. SPY, USMV and SPLV were downloaded fresh, as PREREG §5 allows; the new files are the ones used. Raw files are in `raw/` (git-ignored, never published).
2. **First close on or after inception (PREREG §6.1).** Yahoo's first rows are: MTUM, VLUE and SIZE 2013-04-18 (inception 2013-04-16); QUAL 2013-07-18 (2013-07-16); USMV 2011-10-20 (2011-10-18); SPLV 2011-05-05 (2011-05-05). The sealed rule ("first Yahoo daily close on or after the inception date") applies as written, so these are not deviations. They are listed in the reports.
3. **French files** were built from the 202608 CRSP database, so French series end in 2026-08. The files and column names match PREREG §5 and §6.4 (no renamed dataset; no successor needed). Missing-value codes (−99.99, −999) are treated as missing.
4. **Quality check (ii)** needs an NYSE calendar. SPY's trading days are used as the NYSE calendar. A "gap" is a run of more than 5 consecutive SPY trading days with no ETF close.
5. **Issuer cross-check (iii)** is done after the run, as PREREG §5 says, from the issuer's published calendar-year total returns. Only the maximum absolute difference is published. If the issuer's figures cannot be obtained by our fetcher, that is recorded as a dated deviation (check not completed). The check cannot change any result; if it reveals a data problem, the problem is stated and the fund is still reported (PREREG §5).

## Primary tests (PREREG §6.1, §7)

6. **Series.** The ETF and SPY adjusted closes are inner-joined on dates from the ETF's first close to 2026-09-25, and daily returns are computed on the joined dates. The first return is day 2 over day 1. The 5 bps day-1 purchase cost is applied as turnover 1.0 on that first return (as in Batch 3's ETF audit); turnover is 0 afterwards.
7. **Engine inputs.** `n_trials = 1`, no variant matrix, rubric v1.2, 252 periods a year, 1,000 draws, seed 20260925, holdout split = rubric default (last 30%). `data_received_utc` = the later download time of the ETF's and SPY's files. The objective declarations were not sealed separately; they are hashed inside the sealed PREREG.md (§9), and `run.py` checks every §9 hash before running.
8. **Primary p.** Objective (a): the engine's haircut p-value with N = 1 (one-sided, P(Z ≥ excess Sharpe × √years)). Objective (b): the engine's haircut p-value on dCVaR with N = 1.
9. **"Primary objective check is not FAIL".** (a): the engine's `beats_benchmark` flag, which is exactly "SPA check not FAIL and annualised mean excess > 0" (the rubric's objective cap). (b): the `drawdown` and `tolerance` checks both not FAIL.
10. **"Point estimate in the claimed direction"** (for choosing between "not shown to deliver" and "went the other way") is read from the point estimate of the **primary statistic** whose p-value enters Holm: (a) annualised average excess return > 0 (equivalently excess Sharpe > 0); (b) dCVaR > 0 (tail loss smaller than SPY's). For (b) the drawdown and return-cost figures are always printed next to the verdict, so a reader sees both halves of the claim.

## Secondary tests (PREREG §3, §7)

11. **(c) improve Sharpe, all six:** one-sided p = the rubric v1.2 paired-bootstrap z of dSR (`sharpe_bootstrap`, same seed and draws), N = 1. **(a) beat SPY, USMV and SPLV:** the objective (a) haircut p-value from the same audit, N = 1. Holm across these 8. The status of each one's primary check ((c): PASS if the 5th percentile of dSR > 0, CAUTION if dSR > 0, else FAIL; (a): the objective cap) is reported. No separate letter grade is issued for secondary objectives: the sealed declarations name only the primary objective, and the rubric grades an objective only against a declaration.

## Metrics for all six (PREREG §6.2)

12. **Intervals.** "90% interval" = 5th and 95th percentiles of the paired stationary bootstrap, using exactly the rubric's draws (1,000; mean block max(5, round(T^(1/3))); `default_rng(20260925)`). The statistics are computed on the audited series (net of the 5 bps entry cost) against SPY on the same dates.
13. **Compound excess** = CAGR(ETF) − CAGR(SPY), with CAGR = (∏(1 + r))^(252/T) − 1; terminal wealth of $1 = ∏(1 + r). A sign disagreement with the average excess return is flagged.
14. **CAPM.** Daily excess returns over the risk-free rate. The daily rate is the French monthly RF spread geometrically over that month's trading days in the sample. September 2026 is not yet in the French file, so it uses the 2026-08 RF (carried forward; listed in results.json). OLS with Newey-West (Bartlett) standard errors, 5 lags; 90% interval = ±1.645 SE; alpha annualised × 252.
15. **Sharpe net of French RF** (descriptive): monthly month-end-to-month-end total returns, full months after the inception month to 2026-08, minus monthly RF, annualised × √12.

## Descriptive analyses (PREREG §6.3, §6.4)

16. **Common window:** the dates shared by all seven series from 2013-07-17 to 2026-09-25; returns on those dates; no cost (descriptive). QUAL's first Yahoo close is 2013-07-18, so the first common return is on the next shared date.
17. **Index-regime subperiods:** the audited net series split at each index-change date in PREREG §2; the change date belongs to the new index.
18. **Calendar years:** compound ETF and SPY total return within each calendar year, ETF minus SPY; the first and last years are partial and marked.
19. **Broad-market sensitivity:** monthly ETF return minus French market return (Mkt-RF + RF), full months after the inception month to 2026-08; annualised mean and i.i.d. t; the same months against SPY for comparison.
20. **Before vs after:** windows exactly as §6.4, with "first month available" meaning the first non-missing month of the series used (RMW and the variance sorts start 1963-07). A long-only proxy's window starts at the later of the window start and the proxy's first month. "Market" = Mkt-RF + RF from the 3-factor file. The ETF side-by-side is monthly ETF minus SPY total return over the post-launch months. The Welch t (post minus pre, long-short mean) is shown against both pre-launch windows. Two readings are tabulated, without inference: "long-short mean lower after launch than before", and "long-only proxy's post-launch mean excess over market ≥ the ETF's post-launch mean excess over SPY" (not available for SIZE, which has no long-only proxy).