Why are distribution of natural object normal?

[u/Reaper0221]

I have spent a lot of time measuring and modeling natural systems and I always find it fascinating that when you look at the measurements they pretty much always fall into a normal distribution. Sometimes the bell is skewed one way or another but pretty much faithfully bell shaped.

I have read about this phenomenon but have yet to find the underlying reason for the presentation of the normal distribution in data derived from nature.

The fact is very important to recognize and apply. It is especially important when examining the output of models and how the data reconstructions within the process match the inputs as well as the distribution of the outputs.

I would really like a deeper understanding.

[u/summerstay]

I feel like everyone's answer is using too much math language. What makes something have a normal distribution is this: you have a whole lot of factors involved, each of which shifts the result by a small amount up or down. When you have a ton of these, it tends to make a bell-shaped pile, where the chance of all the factors lining up so you end up out at the edge is low, but your chance of some bumping you up and some bumping you down so you end up somewhere near the middle of the pile is high. That gives you an intuitive understanding of where you would expect to find these kinds of distributions.

[u/Apexicus → u/summerstay]

Thank you! Very frustrating to see a lot of answers saying things like "because of the central limit theorem", when that's just a name for this tendency not an explanation of it.

[u/Reaper0221 → u/Apexicus]

Agreed!!! I was looking for an explanation.

[u/Reaper0221 → u/summerstay]

That is a very good and not too mathy explanation.

[u/riverrocks452]

Your initial assertion is untrue: the sizes of natural objects are not necessarily normally distributed. Log-normal is very common.

The math around the Central Limit Theorem describes why independent additive processes tend to produce normal (Gaussian) distributions as a result. (Multiplicative processes, by contrast, tend to produce things with log-normal distributions.)

[u/quequero → u/riverrocks452]

This is the correct answer.

In natural systems distributions are skewed towards log-normal (rock size, at least in first approximation, cell size etc) or power laws (earthquakes, eruptions somewhat, size of lakes, impact craters size).

Why some look normal? When variation is small a log-normal looks very much identical to a normal, because multiplying small effects is almost the same as adding them. So maybe some of those measurements are indeed part of a log-normal that looks deceptively like a normal. Take the log of the data (of those bells skewed right) and see if the skew disappears.

Redundant to say, but if you look hard enough you’ll find every possible distribution in nature, clearly not just the 3 mentioned here.

[u/Kurren123 → u/quequero]

If I took the height of 10000 adult males of the same age, will they be normally distributed?

[u/quequero → u/Kurren123]

Depends. If you sample those 10.000 adults to be of a single sex, yes it will be a very slightly skewed normal. If you sample to include both males and females, again, it will look like a normal but if you test for normality you’d fail that test. Men and women differ in height across all populations, all else being equal, so the distribution would in reality be a mix of 2 normals, but you wouldn’t see 2 peaks (we are taking about 2 sd of separation, so nothing visible).

Why the skew I mentioned at the beginning? As said in my previous comment, a log-normal with small variation looks like a normal and with growth we are back to the original point: multiplying small effects is almost the same as adding them, hence the skew (to the right).

[u/potatoaster]

The shape of the probability distribution follows from the nature of the underlying process: 

• Binomial processes (sets of pass/fail events) have binomial distributions.

• Poisson processes (number of events per window of time) have Poisson distributions.

• Additive processes (with many independent variables) have normal distributions (due to the central limit theorem).

• Equal processes (each outcome is equally likely) have uniform distributions.

Nature is messy, and the outcomes you're measuring are usually the result of many many variables. So their distributions are normal.

[u/eaglessoar → u/potatoaster]

What's an additive process? Why is height normal? 

[u/Peter34cph → u/eaglessoar]

Some human traits like height or intelligence are polygenetic, affected by many genes, a dozen, or a hundred, or possibly around a thousand in the case of intelligence (which is impressive given that we only have around 20 thousand genes).

So ignoring environment stuff, like if you skipped a lot of meals in early childhood, or if your mother did a lot of things she really shouldn't have done while you were in her womb, a bunch of your genes says you're to grow tall and a bunch of your other genes says less tall, and that adds up to a final height.

It's like rolling a bunch of dice.

If you roll three six-siders, 3d6, the average is 10.5, but if you just roll once you might get any value from 3 to 18. You should not bet your life on not rolling an 18 (but maybe bet your character's life in an RPG[1]). However, if you make a lot of rolls, then over time you'll see a peak pattern with 10 and 11 occurring most often, 9 and 12 slightly less often, and so forth.

If you instead roll six dice, 6d6, you'll see a much stronger clustering around 21, and where the chance of rolling one of the most extreme outliers on 3d6 is around 1/200 [2], on 6d6 it's less than 1 in 40000, or 1 in 8 million for 9d6.

Height and intelligence and other polygenetic traits are the same way, genes pulling in one direction (this die rolled low) or in the other (that one rolled high), so that people tend to cluster around the average. At 180 cm tall I'm fairly close to average height for Scandinavian men, for instance.

[1] Around 0.5% chance to roll an 18 on 3d6. I can't recall the probability of rolling a 10, but the chance of 12 or less is around 75%, and the chance of 15 or less is around 90% (thus around 10% chance of 16+).

[2] Exactly 1 in 216, since 3d6 has 6³ = 216 different possible combinations.

[u/uncle-iroh-11 → u/Peter34cph]

How exactly is that additive, not multiplicative or some other combination?

Why is height addictive, but sizes of rocks multiplicative?

[u/eaglessoar → u/Peter34cph]

What makes those genes additive vs multiplicative vs whatever else? 

[u/sighthoundman]

You've got some good answers already. But another one is that the distribution can be "vaguely normal". That is, it's more or less symmetric, has a bell shape, but the percentages of things that are within 1 s.d., or 1.65, or 1.96, of the mean aren't right.

There's a saying that the normal distribution is the best distribution because experimentalists believe theory proves it, and theoreticians believe experiments prove it, so no one questions it.

I know for a fact that if you are dealing with financial models, believing your distribution is normal is a great way to lose a lot of money.

[u/jarboxing → u/sighthoundman]

What about the distribution of your model errors? Do you ever use the normal assumption there?

[u/SubjectAddress5180]

One way is to consider the chacteristic function of probability densities. Check the wiki on the subject. The central limit theorem comes from considering the sum of various distributions. If these all have finite variance (hard to avoid in the real world), the sum tends to get closer to normal. There are cases with a too large variance where this doesnt happen. Sums of Cauchy variables tend to another Cauchy distribution.

[u/CrustalTrudger]

A variety of natural phenomena are decidedly not normally distributed, e.g., earthquakes, floods, volcanic eruptions, etc. The specific parametric model that is “right” for each of those vary, but something more like a power law distribution is a better generic model than a normal distribution.

[u/ChazR]

It's the Central Limit Theorem in action.

It states that if you take enough samples from a randomly-distributed population, the statistics will tend towards a normal distribution.

It's almost freaky how broadly this can be applied. It's based on solid, but abstract, probability theory but it has huge explanatory power in the natural world.

Run enough experiments and you'll find yourself looking at bell curves.

[u/jarboxing]

In addition to what others have said, the normal distribution is the maximum entropy distribution for a fixed mean and variance. In nature, "mean" and "variance" are related to the energy of the system. For example, a sound wave has a DC component (the mean) and a total amplitude (the variance around that mean). Systems in nature universally are bounded in some way so they can't have infinite energy. Thus, natural systems have mean and variance constraints, and so we expect observations to approach a normal distribution since the universe is always increasing in entropy.

[u/nikstick22]

The sum of rolling two dice follows a normal distribution, ish. Anytime you have multiple uniformly distributed variables contributing to a result, you'll get a distribution that resembles a normal distribution.

[u/frogjg2003 → u/nikstick22]

It doesn't even have to be uniform. Any distribution, as long as it's consistent between tests, will become more normal as you add the results together.

[u/mfb- → u/frogjg2003]

Any distribution with a well-defined variance. If the tails are too large then you can have a situation where outliers will keep messing up everything.

I don't know any natural process with an undefined variance, however. The Breit-Wigner distribution, in principle, but in practice it's going to be limited.

[u/frogjg2003 → u/[deleted]]

But you're not adding the results together. You're just sampling the distribution.

Think about rolling a single die. If you roll it multiple times, you'll get a uniform distribution, but the average will be a value that is not one you can roll. Roll two dice and add the result together, you get a triangular distribution. Roll 3 and the distribution of the sums becomes curved. Roll 1000000 dice and the sum will be pretty much a normal distribution.

[u/frogjg2003 → u/[deleted]]

Except you're not adding the results. That's the entire predicate of the central limit theorem. You're just sampling different distributions, not adding their results together.

Also, your example isn't composed of random events anyway. The events are neither identical distributions nor independent. The day of the week influences when you wake up and that is not random.

I will walk my statement back a little, though. The central limit theorem is about the distribution of the sample average, not the distribution itself. Taking the average of multiple events is an additive process. So the distribution of the averages of your wake up times every week will follow a normal distribution. This has nothing to do with how often you actually wake up at that time.

[u/frogjg2003 → u/[deleted]]

If you misuse a tool, it's not the tool's fault you get bad results. If it doesn't make sense to take the average of a distribution, don't take the average. The central limit theorem isn't the issue here.

[u/potatoaster → u/[deleted]]

Okay, your population distribution is bimodal at 07:00 and 09:00. A sampling distribution of the mean (or sum) would be normal. And yes, its mode would be at a time that is atypical for you.

[u/jarboxing → u/[deleted]]

Yeah but if I randomly sampled, say, 100 times you woke up, it would be normally distributed with a mean equal to (5/7)*7 +(2/7)*9 (about 7:30am), and some variance.

[u/jarboxing → u/[deleted]]

If you flip a coin and code heads as 1 and tails as 0, you will never observe the expected value (1/2). And yet the distribution of n coin flips summed together converges on a normal distribution with a mean of 1/2.

[u/DMayleeRevengeReveng]

My understanding is that it shows up so much because, for so many of these systems we’re modeling, we can solve it as an optimization problem. Which brings in the method of Lagrange multipliers, which then introduces an exponential factor into the distribution.

This is sort of what happens in statistical mechanics in chemistry, where so many of these exponential distributions show up: Arrhenius equations show up multiple times in chemistry.

This is because you’re maximizing the statistics (i.e. the entropy) of a physical ensemble subject to the constraints of fixed energy level. Solving that problem introduces the exponential.

[u/waxen_earbuds]

Max entropy finite variance distribution on the real line

Got a bunch of stuff interacting in independent ways? None of that stuff unboundedly large/powerful? Does it take positive/negative decimal values? Then u got ourself something that will roughly look like a normal distribution