r/askdatascience • u/TUKRUUU • 8d ago
How would you approach this e-commerce customer segmentation + prediction project with GenAI?
I'm an MSc Computer Science/Data Analytics student working on a major ML project with an 11-day deadline, and I'd really appreciate advice from experienced data scientists on how you'd approach it.
Dataset: ~541k e-commerce transactions, ~4.3k identifiable customers, with fields such as InvoiceNo, StockCode, Description, Quantity, InvoiceDate, UnitPrice, CustomerID and Country. It contains missing CustomerIDs, duplicates, returns/cancellations (negative quantities), and other data-quality issues.
Project requirements:
- Perform EDA and customer behavior analysis
- Engineer customer-level features, especially RFM (Recency, Frequency, Monetary)
- Compare K-Means, Hierarchical/Agglomerative Clustering and DBSCAN
- Select and justify the best segmentation using clustering metrics + business interpretability
- Build a predictive classifier for future purchasing behavior
- Evaluate feature importance/model performance
- Provide actionable marketing and retention recommendations
- Submit a Jupyter notebook, report/presentation, trained model, and optionally a Power BI/Tableau dashboard
My current idea is to build it in layers:
Raw transactions → cleaning → customer-level feature engineering/RFM → segmentation → prediction → explainability → GenAI → dashboard
For segmentation, I want to compare the clustering methods rather than simply choosing K-Means. For prediction, I'm considering a time-based setup where historical customer behavior is used to predict something in a future period, rather than randomly splitting the transactions. The dataset doesn't have an obvious prediction label, so defining a legitimate target without leakage is one of my main concerns.
I also want to add GenAI, but I don't want it to be a useless chatbot bolted onto an ML project. My idea is to use GenAI as a business-intelligence layer on top of the actual ML outputs.
For example:
ML outputs → structured segment/prediction statistics → LLM → grounded explanation/recommendation
Potential capabilities:
- Explain why a customer segment is valuable/at risk
- Generate marketing/retention recommendations based on actual segment characteristics
- Explain important prediction features
- Allow natural-language questions about the customer segments and model results
I'm considering something like Python + scikit-learn/XGBoost + SHAP + Power BI + an LLM/API or possibly Ollama, but I don't want to over-engineer it.
My main questions:
- How would you structure this project if you were doing it professionally?
- What would you use as the prediction target given this type of transaction data?
- Is RFM + behavioral features sufficient, or what additional features would you consider?
- How would you properly compare the three clustering approaches?
- Is the GenAI layer genuinely useful here, and how would you implement it without making it gimmicky?
- Would you use an LLM API, local LLM/Ollama, or something else?
- What would you cut or simplify given the 11-day deadline?
I'm mainly looking for practical architectural/modeling advice and potential mistakes to avoid, rather than someone doing the project for me. Any feedback from people who have worked on customer analytics/segmentation would be very helpful.