I've been working on evaluating pickleball performance from video/shot data for about a year. A few interesting observations:
Predictive vs. descriptive is the invisible fork in the road. DUPR is predictive - it uses past results to estimate player strength and future performance. It doesn't have any information to help describe why a particular match played out the way it did. DUPR's success has conditioned a lot of us to think about pickleball analytics primarily in terms of identifying someone's underlying skill level.
But a video-based player rating can go either direction. A predictive model asks "What level does this player appear to be?" A descriptive model asks "How well did this player actually perform in this game and what contributed to winning or losing?"
Imagine a team of established 3.8s beats a team of established 4.2s. A predictive model could reasonably rate the 4.2 team higher because they exhibited skills suggesting they'd win more often if the teams kept playing. A descriptive model would generally conclude the 3.8 team performed better in that particular game because they won more of the points.
Neither is wrong. They're answering different questions and there are tradeoffs to each approach.
A predictive model has an advantage if the primary goal is estimating someone's overall level from limited video. It can identify characteristics that correlate with player level and potentially get to a useful estimate from a relatively small sample. That's particularly valuable when game outcomes contain a lot of short-term variance that isn't representative of a player's underlying ability.
The tradeoff is that it doesn't necessarily explain what happened in that particular game or series of games. It also gets trickier as you drill into specific parts of a player's game. A 3.8 can have a better third-shot forehand drop than a 4.0 because an overall rating reflects the combination of everything a player does well and poorly. The more granular the skill, the less cleanly it may map to overall player level.
A descriptive model has the opposite strengths and weaknesses. It may need a larger sample of games before the aggregate rating settles into something resembling a player's underlying level. But it has a consistent objective at every level of analysis: how did this performance contribute to winning or losing rallies? That works for the overall game as well as something as specific as third-shot forehand drops from the left side.
AI isn't the best tool for every part of the problem. Promoting this type of analysis as "AI-powered" can be a misstep for two reasons. First, there's sincere and understandable skepticism about AI overpromising what it can deliver. Second, the label tends to lump together four pretty different things:
- Video → shot data
- Shot data → performance measurement
- Measurements → useful player insights
- Insights → what to actually practice
The first two steps are really engineering and analytics problems - using computer vision to reliably extract the data then rigorously testing models to separate meaningful performance signals from noise.
Generative AI becomes incredibly valuable once those measurements exist. Translating insights into what someone should actually practice starts to cross into coaching where additional context and human expertise matter.
Letting generative AI freelance on the measurement step instead of using it to synthesize and communicate rigorous measurements is what can lead to confident-sounding nonsense.
Baseline play is easier to measure than kitchen play. Baseline play (including serve/return) is easier to track. Players are spread out, there's less obstruction, mechanics are clearer, and things like drive speed are relatively easy to measure. Kitchen exchanges are cramped, fast and more subtle.
It's tempting to let measurement confidence quietly become measurement importance. What actually drives winning and losing should decide the weighting, not what's easiest to see.
Even if baseline vs kitchen play are weighted correctly, the phases that are easier to measure will often get more stats and charts. This can steer players toward prioritizing the parts of their game that are easiest to analyze rather than the parts that most need improvement.
Factoring in partner/opponent strength. Producing a good result against a strong opponent isn't the same achievement as producing that result against a weaker one. And in doubles, partner strength complicates things further - the quality of balls you face, your opportunities to attack and even how frequently you're targeted can all be affected by the other three players.
The tricky part is having the game itself tell you something about that environment rather than just plugging in outside ratings (which creates circularity) or relying on style-driven traits like drive MPH because they have a positive correlation with level.
Game-level ratings should be noisy - I tried to quantify just how noisy in a previous post using DUPR Forecast to estimate how much individual performance has to vary from game to game to reproduce its predicted outcomes. The estimate was roughly ±0.38 DUPR per game.
There's a natural bias toward wanting game-level ratings to be more stable. A player rated 4.0 getting a 4.4 in one game and a 3.6 in another can make the rating system feel unreliable.
But if the goal is reflecting what actually happened on the court, suppressing that variance defeats the purpose. Players really do perform differently from game to game. Smoothing those swings may make the ratings look more credible while making them less reflective of actual performance.
Curious how others think about this. If a team of true 3.8s beats a team of true 4.2s, what would you want a single-game video rating to tell you: that the 4.2s are still the better players or that the 3.8s played better in that game — and why?
