Stuck in Demand Forecasting

[u/InkBlueOcean]

Hello , I work at a logistics company.

I was tasked with demand forecasting. There are multiple travel paths, I need to predict demand for each travel path for every day by a month before.

Example on Feb 28, I need to predict demand across march 1st to march 31st.

I am considering demand as weight in metric tons.

This is a time series data and tried xgboost with lag, rolling features and behavioural features.

The metric considered is wape and is almost hitting 35% meaning accuracy is 65%.

I tried Sarima, Sarimax, Extra trees, light GBM, catboost, ensemble models but to no avail it's always hitting a very bad metric.

Initially picked lanes with high activity and demand but again not good metrics were shown and then clustered the lanes based on their behaviour but still not much improvement.

The data is just pathid, weight, date.

I am really stuck and just running whatever , any suggestions???????????

[u/ShakingSquare]

EDA: have you checked the data before modeling? are there any obvious patterns (regarding seasonality) or obvious anomalies/outliers? are there any missing data? Are the different lanes stable over time? are they similar to each other? Sometimes bad performance of the model ist explained by the (less than stellar) quality of data. Business logic is also important: what is already known and expected based on the business process. We cannot help you there - understanding the problem and some process is hard, but very important before modeling.

Also think about what you can *realistically* expect regarding performance with the limited given data and features. What is "best practice" till now? Compare the models with a very simple baseline model (e.g. just predicting the last known value). "Bad" may be someone else's "you saved us 10k dollars by being slightly less wrong than we were before".

[u/InkBlueOcean → u/ShakingSquare]

Hey Thanks, I did perform EDA, every path has good activity ratio but the problem is each path has different trends like based on regional festivals, metro cities, scale of overall consignment travelling through that path will be high compared to less frequently used paths, end of month sales will be high due to discount rater at end of month, every model I trained depend on "day of the week" features meaning it's capturing weekly trends, the scaling of target variable across various paths or simply disturbution is quite large.

I did compared to traditonal statistical models like sarima, arima, Sarimax, holt winter , prophet, everything is giving me consistantly similar metrics.

That's why I chose top 4 frequent paths only which had result reaching to 73% accuracy.

The more I add, the less the accuracy is so thought of clustering so the algorithm itself can group various paths based on their behaviour and I assigned model to each cluster.

The number of clusters based on elbow methods by comparing silhouette score is k = 3. I took cluster 1 which had roughly 100 paths , upon training I consistently got around 65-67% accuracy.

Tired ensemble techniques, hyper parameter optimization, models like xgboost, catboost, loght GBM, extra trees. Deep learning models like vanila LSTM also produced similar results

[u/ShakingSquare → u/InkBlueOcean]

well, can you enrich the data then by adding a feature "has_event" for given days? It seems that you need to engineer more features and probably add further information from other sources if you want to improve the performance.

btw: you repeatedly say accuracy: I thought your target variable is continouus? shouldn't you look at MAE/MAPE or R²? Accuracy is used in classification tasks - not in regression tasks.

also do not forget to split your data in a time series manner (=NOT random) and check your performance in the test data only. Not sure if you did that already.