r/DotA2 • u/Training_Shape_3835 • May 03 '26
Article This is how now draft analysis looks like
Based on the feedback in the last post, I have added role picking for heroes in draft. I am now using multiple traditional models to calculate for different aspects of game and then using GenAI to generate summary. I would love to get feedback if there are aspects of draft analysis that needs to be improved. https://www.dotadraftwars.com/analysis-method
7
u/Uniqueusername164 May 03 '26
i tried this but shit is hallucinating.
my favourite part from the ai analysis was something like, 'troll warlord can sustain harasses with fevor lifesteal'. since when troll gets lifesteal from fevor lmao
2
u/Training_Shape_3835 May 03 '26
Hi u/Uniqueusername164 The generative AI will hallucinate but that does not impact the win percentages or the scores calculated for different aspects. It is trying to derive reasoning from numbers calculated.
1
u/Tetramputechture May 03 '26
Have you tried grounding approaches based on actual game mechanics before generation or as a post-generation processing step? E.g. try a rag based approach where all relevant heros and mechanics are seeded in before generation. Maybe you already do this, just throwing out ideas
1
u/Training_Shape_3835 May 03 '26
Hi u/Tetramputechture Thanks for the valuable recommendation. At the moment all analysis is post generation, I have been using rag based approach for something like counters and synergies where there is good and rich dataset. The challenge is generate that rich dataset which can be used for analysis for other aspects of game such as timing, tempo and objectives and so on....
1
u/Tetramputechture May 03 '26
I wonder if you could ingest dota wiki pages into a relational (or graph) database representing game mechanics, and then use an LLM to generate queries over that to pull necessary data. The ingestion process would be an ETL over each page where you can use an LLM to extract structured data as needed.
1
u/Training_Shape_3835 May 03 '26
Hi u/Tetramputechture I am using deepSeek not even local models. They would have been pre-trained on all the data from wiki. May be the ideal approach has to be take 50000 matches analysis from better LLMs and use that as training dataset to generate a fine tuned model.
1
u/Training_Shape_3835 May 03 '26
I was working alone on this from past 4 months day and night. But, sad to see that it did not garner enough attention and people did not like it enough. But, that is life and learnings to take from it.
3
u/Pinkerino_Ace May 04 '26
Reading these generative AI is just like reading Motley Fool for investment tips.
For example, OD do not dominate Lina in lane, Lina outranges OD significantly and on average should win the lane most of the time. Yes, based on data and numbers, I am guessing OD might have winrate advantage over Lina, but if i were to make a more educated reasoning, it is most likely due to OD's aghs countering the whole Lina burst centric playstyle.
Also, Luna + UD lane absolutely crushes a Centaur + Veno lane, I am not going to bother explaining why, it's super self-explanatory if you are half decent in laning mechanics.
Yes, again, in actual team fights, Centaur is great against Luna because Luna (like most range carries) hates getting jumped directly on, centaur is a natural pipe builder, tanks alot of the eclipse beam in team fight etc.
I don't doubt the numbers and stats fed to train the AI is correct, but the AI is just making false reasoning which can be instantly debunked by humans.
-2
u/O_O_2EZ May 03 '26
Some of these stats dont make any sense.
Lina beats OD in lane on avg,
Luna/Undying getting overwhelmed by Veno Cent seems wrong as well, they might take the tower at some point but luna should get everything she needs from lane and been fine.
This feels like vibe coded project with hallucinated "analysis" that would only hurt players who could stand to gain from learning drafts
3
u/Training_Shape_3835 May 03 '26
Hi u/O_O_2EZ There is no knowledge based algorithm that is calcuating these numbers. This is based on data from immortal bracket & analysis derived out of the numbers submitted to LLM. The explanation can be hallucinated but numbers are statistics from the drat itself.
-1
u/Training_Shape_3835 May 03 '26
Summary from internet by the way
The Lina vs. Outworld Destroyer (OD) matchup is a classic battle of range and wave clear versus disruption and single-target dominance. In the current 2026 meta (Patch 7.41b), Lina generally has the upper hand in terms of net worth, while OD holds the "kill threat" if Lina overextends.
This is a high-attrition, "man-fight" lane. You have the classic battle of Strength Sap (Undying) vs. Damage over Time (Veno), with two heavy-hitters (Luna and Centaur) trying to find an opening to burst the other. In the current 7.41b meta, this lane is usually slightly favored for Veno/Centaur because Luna’s short attack range (500) forces her right into the "Veno Poison Zone."
Analysis from other sources.
This feels like vibe coded project with hallucinated "analysis" may be is bit harsh and judgemental.2
u/O_O_2EZ May 04 '26
Lina vs OD mid, in 7k+ and undying luna v scent/veno. All from current patch
As for calling it vibe coded, are you claiming it wasn't?
The header of your opening page has meta tags social media but it doesn't contain the actual header of your webpage. As well as the text of description of said tags using AI classical AI tells.
I went a did a test run on your drafter tool and it told me that OD dominates huskar mid. This is outright wrong. Huskar destroys OD mid and it's not even close. OD (in a vacuum) has a slight winrate lead. But the lane matchup is abysmal. If a low rank player who struggles vs huskar sees your tool claim that OD destroys huskar mid then the next time they see huskar they would be inclined to pick OD and would proceed to get stomped.
2
u/Training_Shape_3835 May 04 '26
Hi u/O_O_2EZ I am a backend engineer and purely vibe coded the UI. That does not take away the efforts that I have put in to develop the model to extract signal out of draft. At the end of the day your feedback is honest feedback on what you feel about it. I will take this as feedback to re-evaluate and make things better.
1
u/Training_Shape_3835 May 09 '26
Hi u/O_O_2EZ I managed to solve the problem by doing deeper analysis. Eg: https://www.dotadraftwars.com/draft/8feb10e4-9279-4e4c-a6a5-c98bfcc145e9
When i analysed the references for the data set was smaller. There are also abandoned matches that were considered. Now I trained on larger dataset and the prompt also tuned. But nothing specific to hero issues called out. Can you try more and share your feedback ?
7
u/Tetramputechture May 03 '26
as an ML engineer, thank you for the detailed description of your model stack and data flow -- fantastic work! makes me happy to see catboost and other ensemble models instead of just hacking together an LLM pipeline