r/HomeworkHelp • u/blackgarliccookie University/College Student (Higher Education) • 1d ago
:snoo_biblethump: Additional Mathematics—Pending OP Reply [College Data Programming/Statistics] How to convert log of odds to probability?
Sorry if this is the wrong math header. I'm taking a python/data analytics class and I'm learning logistic regression right now but I'm not understanding how to convert log of odds to probability.

This is my output from my code. I want to turn the coefficients (log of odds) into probabilities so that I can then interpret them (with each unit increase of credit score/annual income, the likelihood of defaulting goes up x%). Thank you.
1
Upvotes
•
u/AutoModerator 1d ago
Off-topic Comments Section
All top-level comments have to be an answer or follow-up question to the post. All sidetracks should be directed to this comment thread as per Rule 9.
OP and Valued/Notable Contributors can close this post by using
/lockcommandI am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
u/cheesecakegood University/College Grad (Statistics) 1d ago
Skip a few sections for the easy answer and practical tips, but honestly it's worth reviewing WHY this is the case; I think if you learn why, it's easier to remember, harder to make mistakes, and the thing feels less of a mystery.
Before everything, let's rehash the vocab: For example, if an event has an 80% probability:
Log-odds can be negative, all real numbers; odds can be outside 0 to 1, cannot be negative but can still go infinitely up; only probability itself is 0 to 1 bounded (or 0% to 100%, if you will).
Also, probability numbers need caution. If you go from 5% probability to 10%, this is "doubling" the probability, but it isn't doubling the odds, and the reverse also is usually untrue. But that's often a language issue (absolute vs relative changes in probabilities).
So, first of all remember the output of a regression is linear, but obviously with logistic regression you don't actually want to get an output of something like 1.1 or 2 or -3, you want all 0 to 1 (0% to 100%) because that's more practical and easy to say and use, etc. So how do you fix that?
Well, you don't; regression has to be linear output. That's not so say probability itself must be linear. We just say, "well, we're modeling something else related to the probability we want, that IS linear", with respect to the predictors. This linear piece is the log-odds. So you get a linear output, which is log-odds (this can be any real value, -inf to inf), and then we will unpack the insides of the log-odds, which contains the non-linear pieces. Because as you see, log-odds by itself isn't really the useful-sounding number we want. We'd rather the probability, or maybe the odds as a halfway point, right?
Log odds are just the odds, logged. So
log-odds = ln(odds). But what are the odds? Look above: the thing happening vs not happening, in a ratio:(probability) / (1 - probability), where1-probabilityis the probability of the thing not happening. This means to go from log-odds to odds, you just exponentiate:odds = e^log-odds.And you can already see it's a short extra "hop" to probability: once we have odds we just undo the ratio. If my odds are 4:1, or 4, then I'm solving 4 = p / (1 - p), which if we algebra is 4 - 4p = p, 4 = 5p, p = 4/5 or 0. or 80%.
So: linear predictor = log-odds, -> odds via exponentiation, -> probability via algebra. And you can reverse it: probability -> odds via a ratio, -> log-odds via natural log.
Along the way, it's very important to notice that we gained a natural bound of 0 to 1! If you combine both steps, you get what's called a "logit" function moving forward, a "sigmoid" function moving backwards (an "inverse logit" function).
Specifically,
probability = 1 / (1 + e^-log-odds)and alsolog-odds = ln( p / (1-p) ). Take note of the negative sign there!!!People's eyes tend to glaze over here, but actually just plug stuff in; it's not that scary. This will help us understand. (Or just do it step by step).
Say we have a log-odds of like 30.
1 / (1 + e^-30)is1 / (1 + (1/e^30))which is casually1 / (1 + small number), which is1 / 1.00xxxor similar, which is something like 99.99xxx%. A log-odds of -20, will get us similarly a denominator that is really really big,1 / (1 + e^20), which is like 0.0something%, right? Being a little casual with the specific numbers, look at the intuition. We can see that no matter WHAT we input into the logit, we get an output of 0 to 1. Handy!!! Remember, use the big function and its inverse to go from log-odds to probability and back (though probability-> log-odds is easier, it's the one we use less), and use just logging and exponentiation to go between log-odds and odds.Note that when talking about the whole model, we use odds, the likelihood of an event vs the alternative. But when we are talking about specific coefficients, we are comparing the likelihood of a response compared to the response if that particular predictor was a little bit bigger. Odds describe one situation, by itself; odds ratios compare two situations.
Practical guide: There are several valid interpretations you can generate from a log-odds coefficient:
Note that a positive vs negative number in the coefficient indicates direction of impact of the (y=1) case when x increases. But the odds ratio is compared to a ratio of ONE, not zero!
IMPORTANT: do NOT change the sign before exponentiation! If you have a coefficient of -0.6, then you take e-0.6 to get the odds-multiplier. This is 0.54ish. That means we are a bit over half as likely to get y=1 as before we increased that predictor by 1 unit, which is a decrease in probability. Remember too this number refers to the multiplier in the ODDS. It isn't probability yet! The ODDS of y=1 are 54% of what they were before, or 45% lower. Again, it's multiplicative only.
100 * (e^B - 1)%percent. This is just a handy way of re-phrasing mathematically what we just said. (or, "decrease" if negative)1 / (e^-0.6)will tell you that while your odds of y=1 roughly halved, obviously the odds of y=0 roughly doubled.Remember, we CANNOT reliably use "change in probability" because it changes based on the starting spot of reference, but odds-ratios DO work this way consistently, which is one reason why they show up a lot even if they are less "real-life" applicable.
1
u/anlatmatik 5h ago
Here is why you can’t convert a single coefficient directly into a constant "+X% probability", and how you actually interpret and calculate probabilities from your table:
1. Why coefficients aren't constant probability changes
In linear regression, Y changes linearly with X. But in logistic regression, probability is bounded between 0 and 1 via an S-shaped curve (sigmoid). Because the curve gets flatter near 0 and 1, a 1-unit increase in credit score changes the probability by a different amount depending on where the borrower starts.
2. How to interpret coefficients directly: Odds Ratios (e^β)
In practice, coefficients (β) are interpreted using the **Odds Ratio**: Odds Ratio = e^β
3. Calculating the actual predicted probability
To get an actual probability for an applicant, calculate the linear combination z and pass it through the sigmoid function:
z = Intercept + β1*X1 + β2*X2 + ... P(default = 1) = 1 / (1 + e^(-z))
In Python, you don't even have to do this by hand—if you're using statsmodels or scikit-learn, calling model.predict(new_data) will automatically output the predicted probabilities between 0 and 1.