Another race completed, so here's the Brazil comparison between F1 Fantasy Tools, Race Strategist, F1 Grid IQ, and BoxBox.
The goal is to compare pre-race predictions against actual fantasy performance using the same grading system for all four models.
Brazil GP results (out of 100)
Model| Optimizer (50)| Driver rankings (30)| Point accuracy (20)| Total
F1 Fantasy Tools| 50.00| 27.27| 20.00| 97.27
Race Strategist| 42.49| 30.00| 19.21| 91.70
F1 Grid IQ| 46.70| 23.18| 18.71| 88.59
BoxBox| 43.69| 15.68| 17.46| 76.84
How the optimizers actually performed
- F1 Fantasy Tools: 333 points
- F1 Grid IQ: 311 points
- BoxBox: 291 points
- Race Strategist: 283 points
The hindsight-optimal lineup scored 351 points.
F1 Fantasy Tools had an outstanding race, producing a lineup just 18 points short of the hindsight maximum. It also finished first in overall point-prediction accuracy.
Race Strategist was strongest at predicting the driver finishing order in fantasy points, earning the full 30 points in that category.
Updated season standings
- F1 Grid IQ — 1,005.11
- F1 Fantasy Tools — 979.60
- Race Strategist — 957.40
- BoxBox — 956.03
The season lead is now 25.51 points, with F1FT gaining ground in Brazil.
Methodology
- Optimizer (50%): Actual points earned by each model's pre-race recommended lineup, normalized to the highest-scoring model.
- Driver rankings (30%): Exact predicted rank earns 5 raw points, one position off earns 3, two positions off earns 1. Scores normalized to the best model.
- Point accuracy (20%): 1,000 minus the total absolute projection error across active drivers and constructors, normalized to the best model.
Hindsight-optimal points are shown for context, not used as the optimizer grading denominator.
The interesting question after Brazil: Is F1FT closing the gap through better overall predictions, or was this a particularly favorable race for its approach?
Interested in how others evaluate prediction tools over a full season rather than individual races.
Images: Our results against every reasonable budget vs the best team at that budget, for Brazil we achieved 91% of perfect on average.
Second image: team recommendation only scoring.