Validation

Does the model reproduce what the grid actually did in 2024?

EVIDENCE · FIRST-CLASS

Validation is a deliverable, not an afterthought. The heart of it is a single run: the actual 2024 GB fleet, against actual 2024 weather, reproducing observed dispatch within stated, evidence-based tolerances.

The headline result

The model’s one free quantity in the 2024 reference run is the dispatched gas residual — the gas it must burn to balance the system once every observed must-take trace is in place. It lands within 0.91% of the observed total.

Modelled gas (CCGT+OCGT) 73.45TWh

Observed gas (FUELHH) 72.79TWh

Deviation +0.91%

Pinned tolerance ±0.01TWh

Modelled value pinned in grid-cli/tests/regression_2024.rs (PINNED_GAS_TWH = 73.45). Observed value from docs/notes/2024-validation-pack-report.md §2.

What is held to observed, and what the model produces

This distinction is the whole validation, so it is stated plainly rather than buried. In the 2024 reference run:

Held to observed (exogenous must-take traces): wind, solar, nuclear, biomass, hydro, coal, pumped-storage round-trip, and net interconnector imports are set from measured 2024 outturn.
Produced by the model (the residual it must dispatch): gas. Gas is the quantity free to be wrong, and it is the quantity the tolerance gates.

So the validation question is precise: given the real non-gas system, does the engine burn the right amount of gas to keep the lights on, half-hour by half-hour? Answer: yes, to within 0.91%.

Diagnostics from the same run

Quantity	Value	Meaning	Source
Net imports	33.30 TWh (−0.003% vs observed)	Exogenous trace-plumbing check — confirms imports enter the balance intact	`regression_2024.rs` `PINNED_NET_IMPORTS_TWH`
Total curtailment	0.137 GWh over 2 periods	The tightest-surplus moment of the year; near-zero curtailment is expected for the 2024 fleet	`regression_2024.rs` `PINNED_CURTAILMENT_TWH`
Firm-share mean	0.515	Mean dispatched-firm fraction — a whole-year fingerprint of the run	`regression_2024.rs` `PINNED_FIRM_SHARE_MEAN`
Pumped-storage net	−0.60 TWh	Round-trip loss carried as an observed trace, kept off modelled gas	`regression_2024.rs` `PINNED_PUMPED_STORAGE_NET_TWH`

The tolerance has teeth

An easy way to fake a good gas number is to let a stray must-take wedge collapse onto modelled gas. The engine pins that failure explicitly as a counterfactual: if the FUELHH “other” wedge (3.35 TWh of real but unmodelled infeed) were mis-assigned to gas, the gas gate would read +5.30% and fail the ±5% validation tolerance. The gate is calibrated to catch exactly that class of error — see the counterfactual pinned in grid-cli/tests/regression_2024.rs.

Why wind and solar are not in the headline table

A naive fuel-by-fuel comparison would be misleading, and we decline to publish one. Under the engine’s total-generation convention (D3), modelled wind and solar include embedded (behind-the-meter) generation, whereas the transmission-metered FUELHH actuals do not — observed transmission solar is literally 0.00 because all GB solar is embedded. Placing modelled 13.95 TWh of solar next to an observed 0.00 would be a category error. The like-for-like comparison is gas, which is not embedded. The convention and its consequences are documented in Limitations and the engine’s docs/06-conventions.md.

Tolerance frame

Tolerances are evidence-based, set just outside the irreducible data discrepancies quantified in docs/notes/2024-validation-pack-report.md, not aspirational. The starting frame from docs/05-validation.md:

Annual gas burn: ±5% (achieved: +0.91%)
Net annual imports (modelled, continental tier): ±15%
Monthly generation-mix correlation: ≥ 0.95
% periods gas-marginal: ±3 points

Provenance

Engine: grid-sim v0.1.0 (github.com/grid-modeller/grid-sim)
Scenario: scenarios/gb-2024-reference.toml
Pinned test: grid-cli/tests/regression_2024.rs
Run digest: 779d7444…2541abd (PINNED_DIGEST)
Observed data: NESO Data Portal · Elexon BMRS FUELHH (2024)