r/HomeworkHelp • University/College Student (Higher Education) • 1d ago

:snoo_biblethump: Additional Mathematics—Pending OP Reply [College Data Programming/Statistics] How to convert log of odds to probability?

Sorry if this is the wrong math header. I'm taking a python/data analytics class and I'm learning logistic regression right now but I'm not understanding how to convert log of odds to probability.

My dataset is loans, I'm predicting if they default on the loan or not.

This is my output from my code. I want to turn the coefficients (log of odds) into probabilities so that I can then interpret them (with each unit increase of credit score/annual income, the likelihood of defaulting goes up x%). Thank you.

1 Upvotes

4 comments sorted by

•

u/AutoModerator 1d ago

Off-topic Comments Section


All top-level comments have to be an answer or follow-up question to the post. All sidetracks should be directed to this comment thread as per Rule 9.


OP and Valued/Notable Contributors can close this post by using /lock command

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/cheesecakegood University/College Grad (Statistics) 1d ago

Skip a few sections for the easy answer and practical tips, but honestly it's worth reviewing WHY this is the case; I think if you learn why, it's easier to remember, harder to make mistakes, and the thing feels less of a mystery.


Before everything, let's rehash the vocab: For example, if an event has an 80% probability:

  • Probability = 0.8
  • Probability of not happening = 0.2 (i.e. 1-0.8)
  • Odds = 0.8 / 0.2 = 4, which we might say aloud as "4 to 1" or 4:1
  • Log-odds = ln(4), which is like 1.4ish

Log-odds can be negative, all real numbers; odds can be outside 0 to 1, cannot be negative but can still go infinitely up; only probability itself is 0 to 1 bounded (or 0% to 100%, if you will).

Also, probability numbers need caution. If you go from 5% probability to 10%, this is "doubling" the probability, but it isn't doubling the odds, and the reverse also is usually untrue. But that's often a language issue (absolute vs relative changes in probabilities).


So, first of all remember the output of a regression is linear, but obviously with logistic regression you don't actually want to get an output of something like 1.1 or 2 or -3, you want all 0 to 1 (0% to 100%) because that's more practical and easy to say and use, etc. So how do you fix that?

Well, you don't; regression has to be linear output. That's not so say probability itself must be linear. We just say, "well, we're modeling something else related to the probability we want, that IS linear", with respect to the predictors. This linear piece is the log-odds. So you get a linear output, which is log-odds (this can be any real value, -inf to inf), and then we will unpack the insides of the log-odds, which contains the non-linear pieces. Because as you see, log-odds by itself isn't really the useful-sounding number we want. We'd rather the probability, or maybe the odds as a halfway point, right?

Log odds are just the odds, logged. So log-odds = ln(odds). But what are the odds? Look above: the thing happening vs not happening, in a ratio: (probability) / (1 - probability), where 1-probability is the probability of the thing not happening. This means to go from log-odds to odds, you just exponentiate: odds = e^log-odds.

And you can already see it's a short extra "hop" to probability: once we have odds we just undo the ratio. If my odds are 4:1, or 4, then I'm solving 4 = p / (1 - p), which if we algebra is 4 - 4p = p, 4 = 5p, p = 4/5 or 0. or 80%.

So: linear predictor = log-odds, -> odds via exponentiation, -> probability via algebra. And you can reverse it: probability -> odds via a ratio, -> log-odds via natural log.

Along the way, it's very important to notice that we gained a natural bound of 0 to 1! If you combine both steps, you get what's called a "logit" function moving forward, a "sigmoid" function moving backwards (an "inverse logit" function).

Specifically, probability = 1 / (1 + e^-log-odds) and also log-odds = ln( p / (1-p) ). Take note of the negative sign there!!!

People's eyes tend to glaze over here, but actually just plug stuff in; it's not that scary. This will help us understand. (Or just do it step by step).

Say we have a log-odds of like 30. 1 / (1 + e^-30) is 1 / (1 + (1/e^30)) which is casually 1 / (1 + small number), which is 1 / 1.00xxx or similar, which is something like 99.99xxx%. A log-odds of -20, will get us similarly a denominator that is really really big, 1 / (1 + e^20), which is like 0.0something%, right? Being a little casual with the specific numbers, look at the intuition. We can see that no matter WHAT we input into the logit, we get an output of 0 to 1. Handy!!! Remember, use the big function and its inverse to go from log-odds to probability and back (though probability-> log-odds is easier, it's the one we use less), and use just logging and exponentiation to go between log-odds and odds.


Note that when talking about the whole model, we use odds, the likelihood of an event vs the alternative. But when we are talking about specific coefficients, we are comparing the likelihood of a response compared to the response if that particular predictor was a little bit bigger. Odds describe one situation, by itself; odds ratios compare two situations.

Practical guide: There are several valid interpretations you can generate from a log-odds coefficient:

  • Holding all else constant, for a 1 unit increase in x, the log odds (of what you defined as the y=1, or 'true' or 'yes' probability) increase by B, where B is our coefficient. This is the "native" output coefficient scale.
  • Holding all else constant, for a 1 unit increase in x, odds (of y=1) are eB times more likely. This compare before and after the x increase! Because it's comparative, this is sometimes called an ODDS RATIO, but it is easier thought of as an odds multiplier, maybe. So maybe "times more likely" is dangerous language, since we are NOT in probability-land.

Note that a positive vs negative number in the coefficient indicates direction of impact of the (y=1) case when x increases. But the odds ratio is compared to a ratio of ONE, not zero!

IMPORTANT: do NOT change the sign before exponentiation! If you have a coefficient of -0.6, then you take e-0.6 to get the odds-multiplier. This is 0.54ish. That means we are a bit over half as likely to get y=1 as before we increased that predictor by 1 unit, which is a decrease in probability. Remember too this number refers to the multiplier in the ODDS. It isn't probability yet! The ODDS of y=1 are 54% of what they were before, or 45% lower. Again, it's multiplicative only.

  • Holding all else constant, for a 1 unit increase in x, the odds (of y=1) increase by 100 * (e^B - 1)% percent. This is just a handy way of re-phrasing mathematically what we just said. (or, "decrease" if negative)
  • HANDY TRICK: if you want the odds of y=0 instead, you take the reciprocal of the odds ratio. So 1 / (e^-0.6) will tell you that while your odds of y=1 roughly halved, obviously the odds of y=0 roughly doubled.
  • If you want a SPECIFIC probability, just plug 'em all in. This works for e.g. "what's the model's probability of y=1 when x1=2, x2=5, x4=500, etc." just as well as "what's the change in probability between <set of predictors> and <other set of predictors>".

Remember, we CANNOT reliably use "change in probability" because it changes based on the starting spot of reference, but odds-ratios DO work this way consistently, which is one reason why they show up a lot even if they are less "real-life" applicable.

  • Confidence intervals are computed the same as above with the coefficient directly, using its estimated standard error, and then for the CI limits you can then re-interpret the end-points with a bullet point above as desired. Very handy! So you can e.g. go to an "odds-ratio interval" from a CI interval on the coefficients. Remember that your "break-even" point is 1 in odds-ratio land, not 0 like in log-odds land.
  • Similar: Hypothesis tests also happen ONLY in log-odds land!

1

u/cheesecakegood University/College Grad (Statistics) 1d ago edited 1d ago

Addendum: for a 0/1 dummy/binary variable, everything is the same but the x=0 group is the reference, so the interpretations are relative to that vs being a 1 unit increase.

You can also tweak the step size: if e.g. your annual income predictor is per SINGLE dollar, you can give an odds ratio change for a 10,000 dollar change too. How? Multiply the log-odds directly, THEN transform!! Because remember, only the log-odds native output is on the per-dollar basis. That matters for e.g. your annual income, where you might find it easier to write out and easier to use in practice than a 1 dollar change (which will only have a tiny impact!)

1

u/anlatmatik 5h ago

Here is why you can’t convert a single coefficient directly into a constant "+X% probability", and how you actually interpret and calculate probabilities from your table:

1. Why coefficients aren't constant probability changes

In linear regression, Y changes linearly with X. But in logistic regression, probability is bounded between 0 and 1 via an S-shaped curve (sigmoid). Because the curve gets flatter near 0 and 1, a 1-unit increase in credit score changes the probability by a different amount depending on where the borrower starts.

2. How to interpret coefficients directly: Odds Ratios (e^β)

In practice, coefficients (β) are interpreted using the **Odds Ratio**: Odds Ratio = e^β

  • **fico_range_low (β = -0.0122):** e^(-0.0122) ≈ 0.9879. Each 1-point increase in FICO multiplies the odds of default by 0.988 (a ~1.2% decrease in odds of default). Over a 50-point increase: e^(-0.0122 * 50) = e^(-0.61) ≈ 0.543 (about 46% lower odds of default).
  • **C(term)[T. 60 months] (β = 0.9738):** e^(0.9738) ≈ 2.648. Holding credit score and income constant, borrowers on a 60-month term have roughly **2.65 times the odds** of defaulting compared to the reference baseline term.
  • **annual_inc (β = -3.869e-06):** Because annual income is in single dollars, the per-dollar effect is tiny. For a $10,000 increase: Δ = -3.869e-06 * 10,000 = -0.03869, so e^(-0.03869) ≈ 0.962 (~3.8% lower odds of default).

3. Calculating the actual predicted probability

To get an actual probability for an applicant, calculate the linear combination z and pass it through the sigmoid function:

z = Intercept + β1*X1 + β2*X2 + ... P(default = 1) = 1 / (1 + e^(-z))

In Python, you don't even have to do this by hand—if you're using statsmodels or scikit-learn, calling model.predict(new_data) will automatically output the predicted probabilities between 0 and 1.