Public audit · LB-1 · FX · preregistered

The London Breakout, tested as written

It went the other way. As coded, the London Breakout lost money on GBP/USD net of costs over 26 years: net Sharpe ratio −0.88 in-sample (2000–2023) and −1.08 in a locked holdout (2024-01-01 to 2026-09-24). All 15 settings lost, and it lost even before trading costs. Grade F.

We took the London Breakout script from je-suis-tm/quant-trading and ran it exactly as written, with its default settings, on 1-minute GBP/USD prices. Every rule, setting, cost and test was sealed with RFC 3161 timestamps before we loaded any price. We promised in public to publish the result whatever it showed (the thread); it was run once, and this is all of it (proof).

Result

“Went the other way”: it lost, before and after costs

−0.88net Sharpe ratio in-sample, 2000-05-30 to 2023-12-29 (95% interval −1.29 to −0.48)

−1.08net Sharpe ratio in the locked holdout, 2024-01-01 to 2026-09-24 (p = 0.9631)

0 of 15settings made money, in-sample or in the holdout

−0.27Sharpe ratio before any costs (in-sample): it lost even trading for free

The verdict wording was fixed in advance (PREREG §7.2). Holdout Audit rubric v1.2, full sample: grade F (full report). The in-sample rubric run is also F (Stage A report).

The result in one table

The default settings (100-pip false-alarm limit, 30-minute entry window), net of a 2-pip round-trip cost. Returns are on the notional traded.

	In-sample 2000-05-30 to 2023-12-29	Locked holdout 2024-01-01 to 2026-09-24	Full sample
Years of weekdays	23.2	2.7	26.0
Average return a year, net of costs	−3.97%	−4.46%	−4.02%
Sharpe ratio (annualised, net)	−0.88	−1.08	−0.90
95% interval for the Sharpe ratio	−1.29 to −0.48	−2.26 to +0.10
Before any costs: Sharpe / return a year	−0.27 / −1.22%	−0.29 / −1.21%
Round-trip trades	4,788	585	5,373
Winning trades	47.9%	48.7%	48.0%
Average trade before costs	−0.65 pips	−0.73 pips	−0.66 pips
Preregistered test (one-sided p)	P1 = 1.0000 1 − DSR at N = 15	P2 = 0.9631
Holm-adjusted p	1.0000	1.0000
Rubric v1.2 grade	F		F

In plain words

The rule was right on slightly fewer than half of its trades. Its winners and losers were about the same size, so it lost a little on average before paying anything to trade. After a 2-pip round-trip cost it lost about 4% of the traded amount a year. That held over 2000–2023, and again on the 2.7 years of data we kept locked away until the 2000–2023 results were committed.

What each test says

P1, in-sample. Did the default beat zero over 2000–2023, allowing for the 15 settings tried (Deflated Sharpe Ratio)? No: DSR ≈ 0, p = 1.0000.
P2, holdout. Did it beat zero over 2024-01-01 to 2026-09-24? No: Sharpe −1.08 over 2.75 years, p = 0.9631.
Verdict rule. Neither test is significant and the full-sample net return is below zero (−4.02% a year), so the preregistered wording is “Went the other way.”
Rubric, grade F. Fails the multiple-testing checks (DSR, haircut, SPA), the holdout check (−0.88 → −1.08), stability (ahead in 19% of calendar years), costs (break-even below zero) and the parameter surface. Passes the overfitting check (PBO 0.10): it was not tuned into looking good; it lost with every setting.

Did any setting work? No: all 15 lost

The script has two settings. We ran a grid of 15 combinations: false-alarm limit 50, 75, 100, 125 or 150 pips, and entry window 15, 30 or 45 minutes. In the code the target and stop are half the false-alarm limit, so they scale with it. Net Sharpe ratios:

In-sample: from −0.99 to −0.42. All 15 negative.
Holdout: from −1.54 to −0.69. All 15 negative.
The in-sample best (150-pip limit, 15-minute window, −0.42) scored −0.86 in the holdout.
Ranking: the in-sample ranking of the settings barely carried over (Spearman ρ = 0.14, p = 0.63).

Net Sharpe ratio: in-sample / holdout

False-alarm limit	15 min	30 min	45 min
50 pips	−0.62 / −0.75	−0.82 / −0.69	−0.99 / −0.79
75 pips	−0.78 / −1.04	−0.91 / −1.02	−0.98 / −1.15
100 pips	−0.67 / −0.78	−0.88 / −1.08 default	−0.93 / −1.38
125 pips	−0.59 / −1.21	−0.77 / −1.32	−0.77 / −1.54
150 pips	−0.42 / −0.86 in-sample best	−0.64 / −1.00	−0.68 / −1.20

From stageA/results.json (variant_sr_annual) and stageB/results.json (holdout_by_variant).

Costs and the clock: nothing rescues it

Declared in advance; descriptive, no test. Net Sharpe ratio of the default settings, with the average net return a year in brackets.

Change	In-sample	Holdout
Base case: 2-pip round-trip cost (1.0 pip per side)	−0.88 (−3.97%)	−1.08 (−4.46%)
Cheaper: 1-pip round trip (0.5 pip per side; raw-spread account)	−0.58 (−2.59%)	−0.69 (−2.84%)
Dearer: 4-pip round trip (2.0 pips per side; retail standard account or 2000s spreads)	−1.49 (−6.73%)	−1.86 (−7.72%)
No costs at all (gross)	−0.27 (−1.22%)	−0.29 (−1.21%)
Break-even cost per side	−0.33 pips: none	−0.37 pips: none
Fill at the next bar instead of the signal bar	−0.94 (−4.24%)	−1.07 (−4.42%)
DST-aware London clock (range 07:00–07:59, entry from 08:00, exit 17:00 London time, all year)	−0.85 (−3.90%)	−1.28 (−5.40%)
April–October (roughly British Summer Time, when the coded clock is an hour late)	−1.00	−0.60
November–March (roughly GMT, when the coded clock matches the London open)	−0.72	−1.78
Before the script was published (2000-05-30 to 2018-04-16)	−0.71 (−3.04%)	–
After it was published (2018-04-17 to 2023)	−1.33 (−6.81%)	–
Low / middle / high GBP/USD volatility (thirds)	−0.34 / −1.39 / −0.87	−0.08 / −1.52 / −1.28

Costs. A break-even cost below zero means no cost level, not even zero, makes the rule profitable. Cheaper execution only makes it lose less.

Clock (DST). The script’s hours are fixed at UTC−5 all year, so for about seven months of each year its “London open” is one hour late. Moving to the real London clock does not rescue it: on that clock it lost about as much in-sample and more in the holdout.

By year. In-sample the rule made money in 5 of 24 calendar years (2004, 2005, 2006, 2009, 2015). In the holdout it lost in all three (Sharpe 2024 −0.40, 2025 −1.00, 2026 to September −2.22). The worst years were 2010 (−3.50), 2016 (−2.55) and 2022 (−2.11).

How trades ended (in-sample / holdout): 50-pip target 1,662 / 109 (average +54.2 / +52.4 pips); 50-pip stop 1,621 / 117 (−54.1 / −53.2); flattened at 17:00 UTC 1,246 / 333 (−0.0 / +1.3); opposite breakout in the entry window, which only flattens, 259 / 26 (−20.9 / −13.3).

The question

The London Breakout script in je-suis-tm/quant-trading, a GitHub repository with about 10,800 stars, is presented as “a fascinating information arbitrage across different markets in different time zones”: the last hour before the London open absorbs the overnight news, and a breakout of that hour’s range at the open is traded. The script itself computes no profit or loss.

Claim tested: as coded, with its default settings, does the London Breakout earn a positive risk-adjusted return on GBP/USD net of realistic trading costs?

We test the code, not the idea. A negative result says nothing about other implementations of a London breakout.

The public commitment

The question was asked in public in this thread on X.
We committed (our reply) to take the strategy from the repo as written, seal the rules and settings before touching any data, and publish the full result whatever it showed.
We posted the sealed plan before loading any price (sealed plan post).
We posted the result in the same thread (result post). This page is the full write-up.

The strategy, as coded

Code by je-suis-tm: London Breakout backtest.py in je-suis-tm/quant-trading, pinned at commit 611b73f (Apache-2.0). Credit for the strategy and code is theirs; a byte-identical copy with its licence is in our public folder.

Instrument and clock. GBP/USD 1-minute prices, timestamps in fixed UTC−5 (HistData’s “EST without DST”).
Range. High and low of every price in hour 02 (02:00–02:59 UTC−5 = 07:00–07:59 UTC), fixed at the 03:00 bar.
Entry. During 03:01–03:29, long on the first price above the high, short on the first below the low, unless the price is more than 100 pips beyond the level (“false alarm”).
Exit. From 03:30, exit when the price moves 50 pips from entry either way (target or stop, same size).
Time exit. Everything is flattened at the first bar of hour 12 (12:00 UTC−5 = 17:00 UTC).

How it was done

Ported, not fixed. Where the code is clear we follow it, even where it looks unintended. Where it is silent or would crash, the reading was declared in advance (PREREG §3.3).
Sealed before data. Rules, settings grid, 2-pip round-trip cost, periods, tests, α and verdict wording were timestamped by DigiCert and FreeTSA on 2026-09-28 at 13:23:36 UTC, before any GBP/USD price was loaded.
Holdout locked. In-sample (2000–2023) results were committed and hashed before any holdout file (2024–2026) was downloaded.
Run once, publish all. One confirmatory run per stage; every statistic, sensitivity and deviation is below or in the public folder.

Reproduction check

Our port matches the author’s own code on the author’s own data. On the repo’s data/gbpusd.csv (1-minute GBP/USD, 2018-06-01 to 2018-06-29, 29,698 bars), the author’s functions and our port gave identical signals and thresholds on every bar: 42 signal bars, 0 mismatches (repro/repro.json).

Disclosure: June 2018 sits inside our in-sample, so the author had seen those days. It is after the script was first published (2018-04-17).

Data quality

Price level vs the Federal Reserve. HistData at noon New York time against FRED DEXUSUK: in-sample 5,804 days, median gap 6.0 pips, 99th percentile 62 pips, 129 days over 50 pips (mostly the 2008–2009 crisis and October 2022); holdout 681 days, median 4.5 pips, 99th percentile 33 pips, 2 days over 50 pips.
Coverage. Weekdays with a full range hour, a 03:00 bar and a 12:00 bar: in-sample 68.5% (thin early years); holdout 98.6%.
Bad ticks. 79 in-sample and 2 holdout bar-to-bar moves exceeded 100 pips, including a vendor spike on 2000-11-24 and the 2016 Brexit vote. Every bar was kept (no exclusions).

The data

Source: HistData.com free 1-minute GBP/USD bars (bid prices, fixed UTC−5), the source the script itself links to. HistData’s history starts on 2000-05-30.

In-sample: 2000-05-30 to 2023-12-29, 6,041 weekdays, 7.86 million bars.
Holdout: 2024-01-01 to 2026-09-24, 714 weekdays, 1.01 million bars.

Raw data are not redistributed. We publish statistics only. MANIFEST.json holds the SHA-256 of all 35 HistData zips, CSVs and status reports, so anyone who downloads the same files can confirm they match. Canonical panel hashes: Stage A fb1865a3…b904d8, full sample edf2cc47…701741.

Free vendor data, with gaps and some bad ticks, and no warranty. The FRED and coverage checks expose problems but do not fix them.

Deviations: both listed; neither changes a test or the verdict

DEVIATION-001: companion code

Sealed 2026-09-28 at 13:31:54 UTC (DigiCert, FreeTSA), before it was used and before Stage A ran.

The sealed run.py leaves out some declared items: the reproduction, FRED and coverage checks, per-year and volatility tables, the rank correlation, the holdout interval and the HTML report. A companion script, lb_extra.py, computes them using the sealed code unchanged. The HTML report is refused unless its grade and headline equal run.py’s own. The download script fetch_histdata.py is hashed there too.

DEVIATION-002: holdout end date

Sealed 2026-09-28 at 13:59:20 UTC (DigiCert, FreeTSA), after Stage B ran and before any result was published.

HistData’s September 2026 file ends at 2026-09-24 19:58 (UTC−5), so the holdout ends one trading day earlier than planned (2026-09-24, not 2026-09-25). The file was used as published; no other source was used, no bars were filled, and no alternative end date was run.

Also noted: PREREG §13 (“owner decisions … deleted at sealing”) was left in the sealed text by mistake. It is harmless; the decisions are recorded in its header.

Read with care

One script, not the idea. This tests one script as coded, not the idea of a London breakout. Other implementations (a DST-aware clock, intrabar stops, other pairs or other exits) are not covered; the DST-aware clock was run only as a sensitivity.

Low holdout power. 2.7 years can only detect large edges: 80% power needs a Sharpe ratio of about 1.5–1.7. The holdout interval (−2.26 to +0.10) does not rule out a small positive edge from that period alone. The 23-year in-sample does: its interval is entirely below zero (−1.29 to −0.48).

Simplified fills. Trades fill at the close of the 1-minute bar that triggers them, stops and targets included; there is no intrabar high/low logic, because the code has none. Next-bar fills barely change the result.

One flat cost model over 26 years. Real spreads were wider in the 2000s and narrower today. The rule loses even at zero cost, so no cost level changes the direction.

Free vendor data. Bid prices only, with gaps, some bad ticks and thin early years (checks above).

The author’s own search is unknown. N = 15 is a floor; this matters little when every setting lost.

Past data only. Results on 2000–2026 need not hold later.

Not advice. These are statistical findings about past data. They are not investment advice, not a recommendation to trade any currency, and not a statement about the author’s skill.

Seal and verification

The proof that the test was fixed before any price was loaded. Back to top

Everything is public: the plan, seals, code, results and reports are in holdoutlabs-io/holdout-audit/samples/london-breakout.

Preregistration. PREREG.md, SHA-256 ccadb2a06596ca698085300525b0833d109eec70e01e0829645f98bafbbce9f5, timestamped by DigiCert (RFC 3161) on 2026-09-28 at 13:23:36 UTC and FreeTSA at 13:23:46 UTC. Tokens in prereg-seal/.

DEVIATION-001, SHA-256 5144d8be9b2076820358fd4505e19f9181f8f951966f1b55bee59ed32f95d094, DigiCert and FreeTSA at 13:31:54 UTC (deviation-001-seal/).

DEVIATION-002, SHA-256 86be89136b0b6122ae8eb40f48868bed6ef1bccd5036f6416aad34f8f9a4f767, DigiCert and FreeTSA at 13:59:20 UTC (deviation-002-seal/).

Holdout lock. HOLDOUT-UNLOCK.json records the SHA-256 of the committed in-sample results (4d25d550…b07318) and was written at 13:47:04 UTC; the downloader refuses holdout files until it verifies. registry.jsonl is the hash-chained log of all three seals.

Verify it yourself

git clone https://github.com/holdoutlabs-io/holdout-audit
cd holdout-audit/samples/london-breakout
sha256sum PREREG.md
# ccadb2a06596ca698085300525b0833d109eec70e01e0829645f98bafbbce9f5

# FreeTSA: CA files from freetsa.org/files/
openssl ts -verify -data PREREG.md \
  -in prereg-seal/ccadb2a06596ca69.freetsa.tsr \
  -CAfile cacert.pem -untrusted tsa.crt

# DigiCert: DigiCert Trusted Root G4 (PEM)
openssl ts -verify -data PREREG.md \
  -in prereg-seal/ccadb2a06596ca69.digicert.tsr \
  -CAfile DigiCertTrustedRootG4.pem
# Verification: OK

# Show the time and the message imprint:
openssl ts -reply -in prereg-seal/ccadb2a06596ca69.digicert.tsr -text

The same commands work for each deviation with its own .md file and seal folder. Change one byte of a file and verification fails.

Order of events, 2026-09-28 (UTC)

13:23:36
Plan sealed (DigiCert, FreeTSA). No GBP/USD price had been loaded.
13:31:54
DEVIATION-001 sealed (companion code), before Stage A ran.
Before 13:47
Stage A run once on 2000–2023; results committed.
13:47:04
Holdout unlocked: in-sample results hashed into HOLDOUT-UNLOCK.json. Only then were the 2024–2026 files downloaded.
After 13:47
Stage B run once on the holdout; verdict and full-sample grade computed.
13:59:20
DEVIATION-002 sealed (holdout end date), before any result was published.

Reports: full-sample report (headline) · Stage A report (in-sample). Engine: holdout-audit, rubric v1.2.

Want a strategy tested in public?

Name a public indicator or strategy and we may audit it the same way: rules sealed before data, run once, published whichever way it falls. Or get your own backtest audited privately, with a written report.

Request a public audit Start a private intake Compare tiers

Send a high-level description. We’ll confirm scope, then send payment and secure-upload instructions.

Out of Sample · newsletter

Get new audits by email

Out of Sample, free, no spam. Every result published, including the failures.

Subscribe to Out of Sample

You sign up on our newsletter page (hosted by Beehiiv); unsubscribe any time. Want a specific indicator tested? Request an audit.