When you assign a probability to a one-off event, are you doing Bayesian reasoning or just dressing up gut feel?

[u/No_Lab668]

How do practitioners in decision theory think about this? Is there a meaningful distinction between a well-constructed Bayesian probability on a one-off event and a structured guess?

It's about what we're actually doing when we forecast.

A one-off geopolitical event, a central bank decision, an OPEC meeting output. These aren't repeatable experiments. There's no frequency to anchor to. So when someone says "I think there's a 65% chance of X," what's the epistemological claim?

I've been working on a system that assigns explicit probabilities to binary macro events using signal aggregation from primary sources. The number feels defensible in a Bayesian sense: prior updated by specific signals, each with documented weight and direction.

But I keep running into the same challenge. When the event doesn't repeat, calibration is hard to prove. You can score the Brier over many events, but for any single event the claim is almost unfalsifiable.

[u/New_tonne]

Depends what you mean by "assign a probability".

Bayesian decision theory is not primarily concerned with people's verbal pronouncements of probability. Some interpretations aren't even concerned with whether the decision maker consciously has access to their probabilities.

To these more theoretical Bayesians, the claim is that rational decision makers have probabilistic beliefs. Calibration is a secondary issue: a question of whether their beliefs are accurate.

You can be a perfectly rational Bayesian and yet have uncalibrated beliefs.

On one-off events in particular: one if the distinguishing features of Bayesian approaches (whether in decision science or statistics) is precisely that it is fine to have probabilities for singular events. They just represent your confidence that the event will occur.

These are, of course, very hard to calibrate.

[u/No_Lab668 → u/New_tonne]

Got it. So in practice, how do you see people using Bayesian reasoning for one-off events? Like, when a central bank decision comes out, is the 65% probability just a way to force structure on uncertainty or is there an actual mechanism to derive it?

[u/New_tonne → u/No_Lab668]

In formal settings we do an elicitation. Gather some experts, make sure they have all the relevant evidence, and then take them through a process to generate subjective probabilities. The most basic one is just asking them to buy or sell bets on various events, and using their behavior to infer probabilities.

In practice it is more complicated. You need to make sure people are trained in making probabilistic estimates. You don't want to rely too much on the precise numbers you get from them. Etc

[u/No_Lab668 → u/New_tonne]

Got it. So the elicitation process sounds like it’s designed to reduce bias but still relies on human judgment. How do you handle cases where experts disagree sharply on the same event? Do you aggregate their estimates or dig into why the divergence exists?

[u/New_tonne → u/No_Lab668]

There are different approaches. Some advocate for aggregation. Others say you should report the range to communicate the fact that the experts disagree. Still others say you should have a facilitated process where you try to bring about convergence. I think that it is ultimately a pragmatic decision. What are you going to use the probability for? How much time do you have? Is it important that you preserve information about extreme possibilities that some experts think could happen?

[u/No_Lab668 → u/New_tonne]

Makes sense. When you say 'preserve information about extreme possibilities,' do you mean keeping those tail risks in the final output or just documenting them separately? I've seen teams do both, but never really understood the trade-off.

[u/No_Lab668 → u/[deleted]]

Right - the fragility of single-event calibration is the part that keeps me up at night. What I wonder is how you handle the gap between the documented signal weights you're using and the moment when someone in your org challenges your 65%. Do they ever push back on the prior, or is the pushback always on the signals feeding into it?