CFSDS hourly-reanalysis case

Purpose

This page records one independent CFSDS event run with hourly reanalysis weather. It is a depth-with-limit case, not a station-observed hourly case.

Case

  • CFSDS 2003_526, event 2003-08-20 13:00 to 2003-09-04 23:00 local.

  • Final observed scar: 520 ha.

  • Grid: 90 x 76 cells at 100 m, EPSG:26911.

  • Fuel/terrain: FireSTARR Fuel 2023 V1.0 tile 11_0.

  • Weather: Open-Meteo hourly reanalysis.

  • Ignition: first-day burned-cell centroid, reconstructed.

  • Starting codes: FFMC 90.83, BUI 74.12.

Uncalibrated result

Metric

Value

Simulated area

5,325 ha

Observed area

520 ha

Area bias

+924%

Perimeter bias

+496%

IoU

0.096

Sorensen

0.175

Centroid distance

175 m

Orientation error

21.6°

The model overpredicts area and perimeter. No calibration was applied.

Limits

  • Reanalysis weather, not an on-site station.

  • Daily satellite perimeters.

  • Reconstructed ignition.

  • One case only.

The full report is notes/reports/p166-reanalysis-case.md. The close-out is notes/reports/p166-closeout.md.

Multi-case distribution

P167 adds five more independent CFSDS reanalysis cases. All six overpredict final area.

Summary

Value

Case count

6

Median IoU

0.096

Best IoU

0.127

Worst IoU

0.079

Median area bias

+9.25x

Area bias range

+6.68x to +11.60x

Overpredicted cases

6 of 6

The failure is systematic across years, sizes, and fuel codes. It is a scale/shape problem, not a single-case outlier. The full distribution is in notes/reports/p167-reanalysis-benchmark.md.

Start-perimeter correction

P168 reran the six cases from the observed first perimeter instead of a point.

The correction did not remove the bias:

  • median IoU 0.095;

  • best IoU 0.097;

  • worst IoU 0.079;

  • area bias +8.77x to +11.60x;

  • all six still overpredicted.

Point ignition was a protocol flaw, but not the dominant cause. The model fills large connected burnable areas while the observed fires stop far short.

Growth-trajectory diagnosis

P169 compared simulated daily area with observed CFSDS cumulative area.

  • median day-1 ratio: 1.07x;

  • every case is >2x by day 2;

  • median final ratio: 12.07x at radius 2 and 11.90x at radius 1;

  • all six remain >2x at both radii.

The initial growth is approximately correct. The failure is late/continued growth. Neighbor radius is not the cause. The next fix should model containment/stopping or verify the weather/FFMC sequence, not apply a global ROS multiplier.

Weather consistency

P170 compared reanalysis-derived daily FWI with CFSDS daily FWI.

  • median absolute FFMC delta: 0.77;

  • median absolute BUI delta: 11.62;

  • median absolute ISI delta: 0.88;

  • reanalysis ISI was higher on 56 of 68 days.

FFMC and ISI are close enough that weather is not the primary cause of the 10-14x area bias. BUI is higher, but it is not the dominant ROS driver in FBP. The next test is containment/stopping or event timing, not weather replacement.

Terrain sensitivity

P171 reran the six cases with flat terrain.

  • median DEM area ratio: 10.35x;

  • median flat area ratio: 10.35x;

  • flat terrain reduced the ratio in 0 of 6 cases;

  • IoU was unchanged.

Terrain is not the cause of the overprediction in this path. The remaining candidates are fuel mapping, ROS scale, or containment/stopping.

Fuel-map sensitivity

P172 compared per-cell fuel with the dominant homogeneous fuel.

  • median per-cell area ratio: 10.35x;

  • median homogeneous-dominant area ratio: 10.25x;

  • homogeneous fuel reduced the ratio in 2 of 6 cases, only slightly;

  • IoU changed little.

Fuel heterogeneity is not the cause. The remaining explanations are suppression/containment or a genuine ROS-scale issue. The six cases are small fires and may be suppression-dominated.

Large-fire benchmark

P173 ran four large fires at 500 m with the same protocol.

  • median large-fire IoU: 0.238;

  • median large-fire area bias: 2.89x;

  • best/worst area bias: 1.67x/4.12x.

Small fires:

  • median IoU: 0.096;

  • median area bias: 10.35x.

The large-fire bias is far smaller, supporting small-fire suppression as a major cause. Residual large-fire overprediction remains. The large cases use a coarse 500 m grid.

Resolution sensitivity

P174 reran two large cases at 100 m.

  • 2003_96: area bias 4.12x -> 3.67x; IoU 0.192 -> 0.209.

  • 2003_663: area bias 1.67x -> 1.39x; IoU 0.350 -> 0.387.

Resolution explains part of the bias, but a 1.4x-3.7x residual remains. The residual is not just a coarse-grid artifact.

ROS-scale sensitivity

P175 ran two 100 m large cases at head-ROS multipliers 1.0, 0.75, 0.5, and 0.25.

The final area and IoU were unchanged. Lowering ROS only delays arrival; the model still fills the connected burnable basin. The residual bias is not a uniform ROS-scale error. It is a containment/stopping problem or a burnable-basin problem.

Burnable-basin diagnosis

P176 compared simulated and observed area with the connected burnable basin across 12 cases.

  • median simulated area / basin: 0.995;

  • median observed area / basin: 0.130;

  • 8 of 12 simulations fill at least 95% of the basin;

  • all 12 observed fires burn less than 50%.

The model is basin-limited. The next fix is containment/stopping or a defensible burnable-area constraint, not a ROS, fuel, weather, or terrain multiplier.

Free-burn weather ensemble

P178 ran four large free-burn cases with five weather members.

All members produced identical final area and IoU per case. Wind +/-20% and RH +/-10 points gave zero spread in final extent but moved the time to observed area by 2,040-6,600 minutes within a case. The model fills the burnable basin, so weather perturbations change timing, not extent.

The ensemble is degenerate until containment/stopping or a burnable-area constraint is added.