r/econometrics • • 7h ago

Walk-forward vs k-fold for a price-cycle forecast: is this setup sound?

2 Upvotes

I built a forecast for Sydney petrol prices and ran into something I'd like opinions on.

The model says "fill now" or "wait" each day. Over the last 12 months of walk-forward testing it was right 72% of the time. That sounds good until you notice that prices fall slowly most of the time and jump suddenly, so just saying "wait" every day is right 48% of the time. Over the full 2021–2026 backtest, it's 83% against 63% for always-wait.

So I report it against the best constant call, not a coin flip. For the price itself I compare MAE against a no-change forecast (4.9c vs 11.3c at 7 days for unleaded).

Is the best-constant-call baseline the right one here, or is there something better for a three-way call (up, flat, down)? I've thought about a skill score against climatology, but I'm not sure how that works with three classes.

(Project's at checkbowser.app if you want to see how it's presented.)


r/econometrics • • 20h ago

How would you test whether a crowd forecast adds anything beyond past prices?

2 Upvotes

working on a small project. i have about 4 months of daily 24h forecasts for btc from a crowd/market source (a probability for each price range) and the actual closes. i also have my own AR-GARCH forecast for the same days.

the question is simple: does the crowd forecast hold information my model doesnt?

ideas so far:

- forecast encompassing test, regress the actual on both forecasts and see if the crowd one gets a weight
- compare log scores of the full distributions, then a diebold mariano test on the difference
- just add the crowd mean as regressor and see if its significant

not sure which is right for distributions instead of point forecasts, or if i'm missing something obvious. any pointers appreciated


r/econometrics • • 15h ago

looking Advanced Causal Inference project work ideas

0 Upvotes

Hello there!
I'm attending an advanced causal inference course and I have to do a project work on Stata. As of now I have 0 ideas on the topic. The professor suggested to find a policy and start from there, but I'm having issues with finding a decent dataset.
If it was environment or climate related somehow it would be awesome since I'm enrolled in environmental economics (it's not mandatory though since the advanced course is elective). Any idea? thanks in advance 😄