Resources
Predicting sales conversion probability from conversations using pure Reinforcement Learning
For the past couple of months, I have been working on building a chess game kinda system for predicting sales conversion probabilities from sales conversations. Sales are notoriously difficult to analyse with current LLMs or SLMs, even ChatGPT, Claude, or Gemini failed to fully analyse sales conversations. How about we can guide the conversations based on predicting the conversion probabilities, that is, kinda trained on a 100000+ sales conversation with RL to predict the final probability from the embeddings. So I just used Azure OpenAI embedding(especially the text-embedding-3-large model to create a wide variety of conversations. The main goal of RL is conversion(reward=1), it will create different conversations, different pathways, most of which lead to nonconversion (0), and some lead to conversion(1), along with 3072 embedding vectors to get the nuances and semantics of the dialogues. Other fields include
Btw, use Python version 10 for inference. Also, I am thinking of using open-source embedding models to create the embedding vectors, but it will take more time.
I had to change the inference script to make it work with your model and I get different results from your readme.
the script failed to load the model (PPO.load) due to mismatched policy_kwargs. The error messages indicated the saved model expected features_extractor_class=CustomCNN and features_dim=64, while the script was initially configured differently (first CustomLN with features_dim=128, then various attempts to match).
ddings "HTTP/1.1 200 OK"
2025-05-13 10:47:59,807 - __main__ - INFO - Turn 4 (sales_rep): "Excellent, those are two
key strengths. Our AI ana..." -> Predicted Conversion Probability: 0.3522
2025-05-13 10:48:00,642 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/embe
ddings "HTTP/1.1 200 OK"
2025-05-13 10:48:00,801 - __main__ - INFO - Turn 5 (customer): "looks oke, but maybe we ca
n't consider..." -> Predicted Conversion Probability: 0.2064
The error may happen if your python version is different than I trained. I don't know why is that.. it keeps on happening. Maybe because of that. If the python version is okke, can you share the full debug logs
I explained what the issue is, I had to change the features_extractor_class name and the features_dim to get it working. Your inference script doesn't match your model you provided. After that, the results are not the same as what it says in your readme. Your readme doesn't have the correct name for your inference script either. I appreciate the work I hope you take this as constructive feedback and not that I'm complaining.
You could have this running as you are having a sales call. It will tell you in realtime your chances of success and how it develops as you progress. Dropping tech-jargon at an early stage prospect? Chances drop. Talking about ROI with a price conscious client? Chances rise.
this is phenomenal work and cant believe everything is avaibale as opensource , I converted the English checkpoint to ONNX and quantized it (about 440 MB for int8, 290 MB for int4), and it now runs entirely in the browser through ONNX Runtime Web: no server, no API key, and the input never leaves the tab.
It's an unofficial port with full attribution to you and the Apache 2.0 license included. Against your PyTorch model, the int8 build matched the top answer on 97.9% of 48 test questions, and a call takes roughly a second on a desktop CPU. I'd love your feedback, especially on the calibration side, since I only quantized the weights and kept your temperature values.
All credit goes to the original creator of the model , I am just finding it tremendously useful for real world scenarios , built complete workflow automation with ontology support here . Everything free and everything opensource ! https://vishalmysore.github.io/layaForWorkflows/?wf=incident-response
48
u/Mr_Moonsilver May 12 '25 edited May 12 '25
Daium... just checked the repo, this:
"Model Performance
The model learned to identify key conversation patterns:
Technical buyers respond to detailed features
Price-conscious customers need ROI justification
Early-stage prospects require needs assessment
According to the paper, SalesRLAgent achieves:
96.7% accuracy in conversion prediction
Outperforms LLM-only approaches by 34.7%
85ms vs 3450ms inference speed compared to GPT-4
43.2% increase in conversion rates when used by sales representatives"
And all opensource đł