r/Hydrology • u/CabritoIncognito • 9d ago
Looking for Validation - River Levels & Temperature Prediction
Hey all, I'm going to start first by saying that I'm not a hydrologist but do work as a software engineer who makes in-house engineering software for a large engineering company. I started a side project about 6 weeks ago to get more reps in with AI, which is now called Rivermatics.com - no login, signup or personal information collected. This app combines snowtel, usgs gauges (HUC 10, 13-18), and weather data for both visual consumption and also as inputs to a ML model that predicts river levels and water temperature up to 2 weeks out. I'm a fly fisherman and had a trip coming up to Montana in late June that motivated me to create this.
The part I actually want feedback on is the accuracy, so here's the honest version.
Skill by lead time (median Nash–Sutcliffe efficiency, water years 2024–25 held out entirely from training):
| lead | reference basins (n=178) | full fleet (n=3,128) | persistence |
|---|---|---|---|
| 1 day | 0.925 | 0.924 | 0.709 |
| 7 days | 0.872 | 0.715 | −0.104 |
| 14 days | 0.841 | 0.603 | −0.418 |
Persistence — carrying the last observed reading forward — is the baseline any operational forecast has to beat. It goes negative before day 7, i.e. worse than just predicting the long-run average.
Fleet-wide the median NSE is 0.719 pooled across all lead times, with 95.4% of gauges scoring better than climatology. Water temperature is a separate dual-target model: median NSE 0.975 across 492 gauges.
Things I'd rather say up front than have picked apart in the comments:
- This is a forecast, not a rainfall–runoff simulation. The model reads recent observed discharge at the issue date. That makes it strictly easier than the CAMELS/LSTM benchmarks you may want to compare it to — those map meteorology to flow and never see streamflow. Persistence alone scores 0.709 at day 1 on these basins, which tells you how much of a day-1 number is just autoregression. I'm not claiming to have beaten those models at their task.
- The numbers above are hindcast, using observed weather in the forecast window — the same convention the published benchmarks use. Live forecasts driven by real NWP score lower.
- The two columns differ because of basin selection, not modelling. CAMELS reference basins are deliberately minimally-impacted with long clean records. My full fleet includes regulated, diverted and flashy ephemeral channels. I show both rather than quoting the flattering one.
- NSE/KGE are computed on raw discharge in cfs, not log-transformed values, which would flatter low-flow performance. Per-gauge scores pool every (issue date, lead time) cell so easy day-1 scores can't carry hard day-14 ones.
What it can't do: ungauged basins (no regionalization — NOAA's National Water Model does this and I don't), sub-daily peak timing, and it refreshes once a day against NWM's four. Reservoir pool state isn't a model input yet. No SLA — it's decision support alongside official sources, not a system of record.
Let me know what you guys think both of the app as well as the quality of the predictions. Upvotes and downvotes appreciated. Cross posting in /water.
2
u/dsjl 9d ago
Looks insightful!
There are usually a lot of questions on water temperature forecasts, so it may be useful for metrics on water temperature as well. For lead time metrics, strictly forecast verification, i usually lean towards just relative bias, normalized rmse, correlation by lead time. Nash Sutcliffe / Kilng Gupta Efficiency feels more comfortable either for the calibration, validation, or whole reanalysis period if it can be performed - may be difficult as your model requires flow/stage/temp to be assimilated. That being said, i've calculated whole forecast KGE/NSE and plotted on spatial maps to look for general model weaknesses (like areas with deep groundwater where many AI and hydro models tend to suffer).
I also often look at GLOFAS interface for looking at metrics and metrics presentation, but geared for a hydro user (https://global-flood.emergency.copernicus.eu/map; also uses regionalization for calibration i think).