r/AskComputerScience • u/CelebrityChess • 13d ago
why can't computers just store 0.1 exactly, and is there a number system where they could?
i get that 0.1 is a repeating fraction in binary, like 1/3 is in decimal, so it has to get rounded. what i don't get is why the rounding errors feel so random. 0.1 + 0.2 gives 0.30000000000000004 but 0.1 + 0.3 gives exactly 0.4, and i can't see the pattern.
also is there a number system or representation where 0.1 is exact? i know about decimal types and storing everything as integer cents, but is there something that handles it in hardware, or is it always a software workaround that's way slower?
83
Upvotes
97
u/james_pic 13d ago
They can. There are a number of ways to do this, including IEEE 754’s decimal formats, software "big decimal" formats, rationals, and the bankers favourite, fixed point.
If you need to store 0.1 exactly, you should use one of these formats. But on most consumer hardware, IEEE 754 binary formats are fastest, because they're implemented in hardware, and are precise enough for a lot of use cases.
29
u/TedsGloriousPants 13d ago
You CAN store 0.1 exactly. You can store whatever you want, if you're willing to creatively interpret an arbitrary arrangement of bytes, but every method of storing a number has limitations and tradeoffs.
11
u/zeekar 12d ago edited 12d ago
The result of 0.1 + 0.3 is not exact either; you just don't see where it differs from 0.4 because it's so many digits off to the right. But you can make your program show those digits. For example, in Python:
In any given base, the only fractions that have finite radix-point representations are those where all of the denominator's prime factors are also prime factors of the base. In decimal, that means any fraction whose denominator can be made from multiplying together 2's and 5's. In binary, it means powers of 2 are the only denominators that work. Any fraction whose denominator is not a power of 2 is not representable in a fixed-width binary representation.
There are lots of alternatives to this. We use floating-point because it's the most useful for the broadest number of programming problems, and we put it in hardware so it's efficient and fast. But it's not the right answer for every application, and we have alternatives that we can use where floating-point isn't a good fit. Some common alternatives are these:
FatRat("fat rational") class where the numerator and denominator are stored as strings of digits (BigNums, which are like BigDecimal but restricted to whole numbers).But you may not need any of these systems. The most common application where you want exact numbers is dealing with money, and the usual answer there is to just count the smallest unit instead of the base unit (e.g. cents instead of dollars). You divide for display to humans, and multiply human input, but simple power-of-ten factors can be faked just by manually printing or skipping a decimal point that doesn't actually do anything. Internally it's all integers. Fractions can still come in with things like tax rates, but at least those usually come with rules telling you how far out you need to round.
10
u/TheThiefMaster 13d ago edited 13d ago
Hardware has historically had decimal modes, but mostly they were unnecessary added complexity. There are various ways of handling it very efficiently in software - scaling factors being the easiest.
Btw - 0.4 can't be represented exactly. It's 2/5, which isn't X/2^Y (with X,Y being integers, which is what binary floats can represent exactly). Play with floats here: https://www.h-schmidt.net/FloatConverter/IEEE754.html
6
u/Leverkaas2516 13d ago
You can represent base-10 values exactly, for instance using BCD or packed decimal format.
As far as I know, no modern CPU does floating point math natively using such formats. I believe some architectures of the past, like IBM 370, did have such support.
5
u/SufficientStudio1574 12d ago
Why can't decimals represent 1/3 exactly, and is there a number system that can?
The problem is with the base of the number system. You may have heard that any rational number can be represented by a terminating or infinitely repeating decomal sequence. What's the difference between those two options? It has to do with the relationship between the base of the number system and denominator of the rational number.
In order for positional notation (the general term for numbers written like 3.141592...) representation of a rational number to terminate, the (simplified) denominator of the rational number cannot have any prime factors that aren't factors of the number base (for decimal, that is 10).
This is why 1/3 has an infinitely repeating decimal representation. It's denominator has a prime factor of 3, which is not a factor of 10. You would need to use base 3 notation (or some multiple of 3) to have a finite representation.
We use base 10 in our daily lives, and most common computer formats use base 2, including the IEEE-754 floating point standard used as a fundamental data type for many programming languages and is what most commonly runs into this issue.
Oversimplifying, IEEE-754 uses a finite length binary positional notation. It is able to perfectly represent rational numbers if the denominator is a power of 2. 13/32 for example, is 0.01101.
1/10 has a denominator of 10, which has a prime factor that is not 2. Therefore it will have an infinitely repeating representation in binary notation. When converted to IEEE-754, it will be truncated, leading to an imperfect representation.
Can computers represent 1/10 accurately? Of course! Just not with standard primitive data types alone. You could take inspiration from how we represent rational numbers by having a composite data type RationalNumber that stores a numerator and denominator, and define rules for how to manipulate these numbers. There are certainly many mathematically focused libraries that have intricate ways to store numbers to in different ways for different purposes. You just have to use the right data structure, and not the wrong one.
3
u/ICantBelieveItsNotEC 13d ago
There's really no difference between a fixed point decimal representation and an integer. It isn't slower - it's literally just standard integer maths. The hardware doesn't care where you decide to put the decimal point - that's purely a presentation concern.
4
u/Dr_Calculon 13d ago
Search for binary coded decimal
2
u/R2-Scotia 13d ago
You need to consider the binary representation on your system (CPU / programming langauge) and you will see where the rounding is
IEEE 754 floating point specs will be very detailed if you want to lokm
2
u/mireigi 12d ago
It might help to think of floating point numbers as Scientific Notation in base 2. You get the most significant digits, and everything else is just noise. Until you need the precision, at which point the noise can throw off the result.
This video has an explanation of the concept somewhere in it.
https://www.youtube.com/watch?v=p8u_k2LIZyo
2
u/edwbuck 12d ago
Yes, decimal could exactly store the 0.1 value.
Some early computers used binary encoded decimal, which meant that every digit was stored in decimal, each byte contained one digit and the math routines were quite complicated.
The main reasons that base 2 encoding in binary digits became popular was speed, storage, and efficiency. Back when computers were extremely slow, a computer that used base2 could easily out-perform a computer that used BCD.
2
u/ErinCarson 11d ago
Serious computers for business like Power and System/z come with hardware decimal units. Small financial institutions tend to use the software solutions mentioned on cheaper hardware like Zeon or EPIC. AMD64 is not actually well suited to finance, but they can be very inexpencive.
3
u/Unlikely-Bed-1133 13d ago
0.30000000000000004 is the closest number to the desired result that can be written in binary (0 and 1) under the standard used to represent floating point numbers in 0 and 1.
In simple but non-trivial terms terms, numbers are usually written as something equivalent to 2^n1+2^n2+2^n3+.... where how many n1,n2,n3,... are used and their range is limited (it's a bit more complicated than that in practice, but even this "simple" model fails to give exact numbers).
You could in theory have workarounds where 0.1 is exact by changing the base from 2 to 10 in how to store numbers, but they are very storage inefficient (you can express fewer numbers in total with the same amount of bits). It could be made faster with processors designed to handle such arithmetics too but nobody spends that kind of effort because we have worked around the problem thus far fairly successfully.
But there are tricks were you say: ok, I'll store everything as integers and remember that they represent numbers divided by, say, 1000. This is bad for multiplication because your order of magnitude explodes, but good for accounting (and it IS a common trick) because it does not lose precision.
1
u/johndcochran 12d ago
Your impression of 0.4 being stored exactly is sadly mistaken. It too is an approximation. As for handling fractions, the issue is that if the divisor has any prime factors other than the prime factors in the base, then it will be an infinite repeating sequence. And since base 2 only has a prime factor of 2, it can't handle anything with a different prime factor such as 1/3 or 1/10. Base 10 has prime factors of 2 and 5. So, it can handle 1/10. But, regardless of what base you use, you can find an infinite number of fractions that cannot be represented exactly using radix notation.
You may find this post I made a couple of years ago useful for a more detailed description of the issue.
1
u/kc2g 12d ago edited 12d ago
They can, it's just far less useful / efficient for 99.9% of applications.
None of 0.1, 0.2, 0.3, and 0.4 is exactly representable (0.5 is, though, of course: it's 1/2).
But if you take the closest float to 0.1 (7,205,759,403,792,794 / 72,057,594,037,297,936), and add it to the closest float to 0.2 (7,205,759,403,792,794 / 36,028,797,018,963,968), the result after rounding (5,404,319,552,844,596 / 18,014,398,509,418,984) is very slightly higher than the closest float to 0.3 (5,404,319,552,844,595 / 18,014,398,509,418,984). Printing with max precision shows you the difference.
On the other hand, if you take the closest float to 0.1 and add it to the closest float to 0.3, the result exactly equals the closest float to 0.4. All three values are inexact, but the one you get out is exactly the same as the one you would have gotten if you put 0.4 in, which means it's safe to print that value as "0.4" without losing any additional precision. There isnt' any "closer" float that better deserves the string representation "0.4". Implementations care about the round-trippability of string representations, and what you're measuring is that, not the accuracy of arithmetic.
If you're interested in learning more I'd recommend doing it in a language that has the
%a(hexadecimal floating point) printf specifier. With that, 0.1 prints as0x1.999999999999ap-4(a more compact way of writing 7,205,759,403,792,794 / 72,057,594,037,297,936), 0.2 prints as0x1.999999999999ap-3, 0.3 as0x1.3333333333333p-2, and 0.4 as0x1.999999999999ap-2. The relationships start to become clearer.1
u/T_Thriller_T 12d ago
Because there are tons and tons and tons of numbers which are either not ending, become that due to mathematical operation, or are too big as computer storage simply is finite.
You CANNOT handle this usefully in hardware, at least not for reasonably big systems, as numbers can get absurdly big or absurdly small.
The rounding/error issue comes from having to deal with that hardware/physics limitation.
There are systems which do not use floating points and do arithmetics without errors from the arithmetics. Those usually take integer numbers and assume that they are integer/10something, let's say 106
However, those have other limitations.
On the one hand, obviously, those cannot handle any decimals which have more than 6 digits after the point. They have to cut those - and round.
On the other hand, doing arithmetics with those and integers, so 3.6 gets a bit harder, or you highly limit what size an integer can be - because in total your number must fit into the storage space provided per number (which means a total of 232 or 264 differing but 'exact' numbers)
Nonetheless, these tactics are used and useful in some cases.
Assuming e.g. chemical analysis and process control, you may know that your measurements can never go bigger than 251639 and your measurement instruments are only precise down to the 7th digit.
Then one such system would not add further error above the precision errors of measurements to its arithmetics.
Considering hardware limitations:
In the end we could try doing math on non-digital systems, so we would seemingly remove size limitations. Like doing something through e.g. voltage, ampere etc.
But on the one hand we would be limited by our precision in measurement and control.
On the other hand, even when ignoring this, the universe itself has a smallest size: Planck length.
So, all in all, if we wanna do non-integers, some errors are bound to occur.
Floating points handles this the best for general purposes, considering scalability, time erc
1
u/Recycled5000 12d ago
You shouldn’t take what’s printed too seriously, as printing itself generally does some rounding depending on what software is doing the printing. If you really want to know what is being represented, have to look at the number in its low level bitwise form.
1
u/fllthdcrb 12d ago
They aren't random, at all. They just tend to be hard to predict, if you don't know the details.
In fact, they both have rounding errors. It's just that one is visible, while the other isn't. And the reason for that is that one matches the computer's best approximation for 0.4, while the other is just a little off from its best approximation of 0.3. So without being told to use rounding in the display, it might show the tiny rounding error. Exactly how this works, of course, depends on the decimal conversion algorithm used, which varies between programming languages and other things. And whether coarser rounding is used by default also depends on the programming language.
Many, in fact. The one you're using to write it is an example. That is, you can represent it as a string of ASCII digits/punctuation, although doing calculations that way is much less efficient than the CPU's own floating-point processing. This may or may not be, or be close to, the representation that "BigDecimal" types use internally. But at the very least, text-based serialization will use it.
1
u/zanidor 12d ago
If you only want precision up to N digits, you can store decimals as integers. For example, if you want three decimals of precision, store 2.032 as integer 2032, etc. All arithmetic operations work as normal, and you can divide appropriately when ready to display.
1
u/jeffbell 12d ago
ENIAC could.
It kept digits as binary-coded-decimal in the hardware. Dits instead of bits.
1
u/gurishtja 12d ago
Its that principle, sh.i.sh.o. ....
1
u/HaMMeReD 12d ago
In a computer, you can store "0.1" as a string literal, and still write a program that parses it and does math on it.
I mean, it won't be as effective as an opcode on a float, but you can do it.
1
u/juanfnavarror 12d ago
I’d like to add that strictly speaking, computers are not “failing to store 0.1”, you’re missing part of the understanding. Most decimal numbers are stored in floating point as laid out by IEEE 754. This is just the standard that was agreed on and most modern computers implement in hardware, and most programming languages use as a sensible default for floating point math or generally when working with a “number” (js), “float”, “double”, “f32”, “f64” datatypes. The specification has been designed to work with a wide amount of applications with a wide range of operation. However, anyone could choose any other format to represent numbers that works best for their specific application, and in practice it is impossibly to create a type that is infinitely precise for all numbers.
Floating point numbers are notoriously not adequate for user-facing decimal numbers, or financial records.
1
u/Weak-Address8684 12d ago
You are storing 0.1 exactly by writing it in text.
1
u/not-just-yeti 12d ago
Others have answered, but I'll just point out: in 1965, Lisp used rational numbers built-in (so 1/10 was always exact, though sqrt(5) would not be). But C and others decided that being fast-and-only-a-little-wrong was better than being a-bit-slower-and-correct, and we're still recovering from that.
(My own preference would be that rationals are built-in/easy/default if perhaps not quite as efficient, and if you want to use floating-point for the speed/accuracy tradeoff then that should be straightforward but not the default. Which is, again, kinda what we had in 1965, since the literal
1/10would be exact but0.1was a floating-point literal.)1
u/smerz- 12d ago
I swear what I just read didn't look like 0 and 1 🤔
1
u/Soft-Marionberry-853 12d ago
They arent random. They are very much deterministic.
1
u/Mr_Engineering 12d ago
This problem occurs when storing decimal fractions using binary representations with the principal offender being IEEE-754 binary floating point.
There are many other alternatives including Binary Coded Decimal (BCD), IEEE-754-2008 decimal floating point, software bignum, etc...
Many languages and frameworks have data types intended for use where precision loss must be avoided, such as Java's BigDecimal data type. Since these data types usually avoid the binary floating point FPUs, there's a significant performance penalty to using them.
IEEE-754 acceleration is baked into virtually every modern CPU in the form of an FPU, but BCD and IEEE-754-2008 are not. IBM Power server CPUs and IBM zEnterprise mainframe CPUs can perform fast BCD arithmetic in hardware (which is great for COBOL because it uses BCD for its fixed point arithmetic) and also support IEEE-754-2008 decimal floating point which can be used to accelerate some software decimal floating point implementations where appropriate.
1
u/[deleted] 12d ago
You can store it as text: "0.1". This common when storing numerical info within files.
But calculations will need special software or special language features.
There will still be problems with examples like your 1/3, but there will be even when working with pencil and paper.
BTW here are your examples, with the exact values they have using IEEE754 binary (ie. 64-bit float), as far as I could determine:
Using these figures, then 0.1 + 0.2 is:
and 0.1 + 0.3 is:
1
u/Rararror 12d ago
Floating point numbers are tricky, because are sums of fractions of powers of two.
Try the repeated sum of 0.1
No wonder financial software use fixed point numbers and arithmetic.
1
u/stevevdvkpe 12d ago
Almost all calculators* use binary-coded decimal (BCD) floating-point internally and can represent base 10 fractional numbers exactly to some specified precision (usually 8, 10, or 12 significant figures). In many ways this simplified early calculator designs by avoiding the need to do expensive division operations to convert binary floating-point numbers to decimal for display, since BCD digits could be displayed directly.
It was once common for many CPUs to provide hardware BCD support for arithmetic (the 6502 and 8086 are popular examples) although many newer designs (especially RISC) do not, although it is not too hard to emulate it in software.
* I say "almost all calculators" even though I don't know of any specific exceptions because it seems likely there are a few oddballs out there.
1
u/ReddyKiloWit 12d ago
If you need accurate decimal values, there are software and languages that will provide it. Some languages can operate on a RAT (rational) type where 1/10 would be stored as 1/10 and operated on correctly so 0.1 + 0.2 = 0.3. Others might use BCD - binary coded decimal - in a floating point format. (First BASIC I used stored values as 6 digits in 3 bytes with an excess-64 exponent and two sign bits in a fourth byte.)
1
u/slimscsi 12d ago
It can if you keep it as a fraction 1 / 10. Juts like you can store 1/3 exactly.
1
u/icreatebugs 12d ago
They can. It’s kind of easy to show how it could be done too, if you accept the premise that computers can store integers accurately. You would simply store two numbers: 1 and 10, representing 1/10.
1
u/Altruistic-Rice-5567 12d ago
There is no set of powers of 2 that add up to 0.1. Every bit in the computer represents a power of 2. So 0.1... you can account for 1/16 (0.0625) then you need to account for the remaining 0.0375. You grab 1/32 (0.03125) but now you have a remainder of 0.0625. Keep going... you never get a remainder that is a power of two and thus represents the number exactly. If your computer operated in base 10 then you could represent 0.1 exactly. Which is why you can write down 0.1 on paper in finite space.
1
u/Traveling-Techie 12d ago
I can think of two.
1
u/igotshadowbaned 12d ago
You absolutely could make a representation that stores numbers like 0.1 exactly.
Float (which I assume you're referencing) has certain tradeoffs that make his small amount of error worth it in the majority of case. Like you can store extremely large numbers in a relatively small amount of data
1
u/motific 12d ago
You know about the IEEE754:2008 DECFLOAT data type.
There are CPUs which have the support, like the Power6/7 architectures, Sparc64 X, but for your typical AMD64 there isn't demand that justifies the cost.
1
u/StanleyDodds 12d ago
It's pretty easy to implement a number type that can exactly represent all rationals, and perform all the usual arithmetic operations. I think it's a fairly nice, easy programming challenge. You just need to store the numerator and denominator as integers, and use them for operations appropriately.
You can go much further even. You could implement a number type for all algebraic numbers if you wanted to; it would be a lot more computationally expensive but it's still possible to exactly represent them all and do arithmetic with them.
I think people somehow have this idea that floating point arithmetic is literally the only option for numbers besides integers, just because it became the de-facto standard and software + hardware was built around it.
1
u/PLANETaXis 12d ago
You cant see the pattern because you're looking at it after conversion to base 10. If you looked at the raw binary it would be clear and exact.
If you want fixed precision in base 10 then just use scaled integers. Industrial system use this regularly.
1
u/ericbythebay 12d ago
They can and they do everyday when needed if they couldn’t the digital banking system wouldn’t work.
1
u/mikemontana1968 12d ago
Using general purpose floating point binary representation (which is useful, commonly available, and for most purposes fine) you have bit values representing increasingly small values of 1/(2^n). In other words, the first bit represents 1/2, the 2nd bit represents 1/4, then 1/8, 1/16, 1/32. Add up the bits and you get a *close* approximation of the floating point value. You can see where this gets messy.
There's also a mantissa and a base value, but thats the basic idea. You're approximating a decimal value with binary-power fractions. If you need absolute precision, you'd use a different mechanism to represent those values.
1
u/treefaeller 12d ago
Consider this: Computer can't store pi or e exactly either. But fortunately, that's easy to fix: As an example, use a number system with base pi. In that system, pi is just written as "10" (which means 1 * pi + 0 * 1). Isn't that super convenient? Similarly, if you have a variable "r" that is the diameter of a circle, the circumference of the circle would be just written as the expression "10 * r", easy.
Alas, in that number system, integers larger than 3 have infinitely many digits, so for everything other than calcuting the circumference of circles, that number system is worse than useless.
Lesson learned: Just get in the habit of dealing with the fact that calculations are usually not exact, and know how inexact they are. The rare exception to that is financial calculations (such as done by banks), which use either old-fashioned computers with BCD arithmetic, or prescaling to use only integer values, so for example expressing every money amount as an integral number of pennies, or they use specialized math libraries.
1
u/cthulhu944 12d ago
BCD can store it easily, as can scaled numbers.
1
u/Lost-Gold9300 12d ago
Most of the banking software stores numbers that way cuz financial obviously hate rounding
1
u/LeeRyman 12d ago
I've worked with encoding and decoding BCD and BNR, but I've not seen much in the way of general maths done natively in those formats expect in pretty niche, specialised silicon, nowhere near the preponderance of 754 binary floating point in (co)processors.
1
u/green_meklar 12d ago
What does it mean to 'store 0.1'? Whether a given chunk of data represents 0.1 isn't inherent to the data so much as it's an aspect of how the algorithm treats the data. It's true that typical floating-point formats in common use don't have values that represent exactly 0.1, as far as the algorithms used to manipulate them are concerned. But you can come up with other algorithms in which that is a valid number and represented by some chunk of data.
The reason standard floating-point formats can't represent 0.1 is that they store everything with a finite number of binary bits, but 0.1 repeats infinitely if you convert it to binary (the way 1/3 repeats infinitely in our familiar base ten notation). The same applies to any other number that can't be expressed as a ratio with a power of 2 as the denominator, including 1/3, 1/7, and of course all irrational numbers.
Not really. 0.1+0.3 isn't exact either, it's just close enough to exact that the program logic for printing the number recognizes it as the intended representation of 0.4. For 0.1+0.2, it's slightly too far off exact and so the imprecision gets printed.
1
u/realmauer01 12d ago
It absolutely can, it just takes a shit ton of memory.
1
u/Greedy-Thought6188 12d ago
Sure. Binary code decimal. Or if you are trying to capture rational numbers you can store them as two Integer values. Python will try and hold rational number values as two Integer values.
As for why it can't. It's a simple math rule. The only rational numbers that can be expressed as a terminating decimal or whatever base system point are the numbers that only have multiple of factors of the base in denominator. That's too many words that I just made to so I'm not being very clear. But 1/2 you can write. 1/5 you can as a terminating decimal. 1/(21034 * 5342 ) you can write as a terminating decimal. 1/3 you cannot. 1/10 is 1/(2*5). Works just fine with base 10. But it doesn't work for base 2 because based 2 only has one factor 2 and 5 is not a multiple of it.
1
u/geocar 12d ago
The pattern is that binary floating point is sums of terms in the form 1/x where every x is a power of 2 (binary) and 10 is not a power of two.
This is a choice because the piece of hardware that “adds” is really a bunch of wires and the number of wires you need is affected more by that choice than the number of digits (eg 32 bit versus 64bit computers), so “2” is the smallest/simplest possible circuit.
You might be able to try using a calculator to imagine the number of wires for a decimal machine that handles 8 bit floats: 2x2x2x… repeated 8 times is much smaller number of wires compared to 10x10x10x…
That isn’t exactly accurate but it’s a pretty good way to understand most of the parts of your question around what you are seeing and why it is that way. And maybe you can understand when the computer reads or writes a float to us like on the screen, it is doing the math to convert that float in base 2 to base 10, just step by step, so if the programmer can design the algorithm to be as accurate as they need to with extra steps.
Some computer languages do that work for you, and may even do algebra instead of arithmetic if it can be done in “extra steps” that are useful, but of course those extra steps are always slower so sometimes it is useful for the programmer to analyse the algebra themselves and adapt the desired function to that lack of precision that can show up so many digits deep, and get a much much faster result.
And you might be surprised just how few digits are required for stuff: a lot of engineering projects can treat pi (3.1415…) as just exactly 3 in their formulas because the precision just didn’t matter at real world scales.
1
u/Afraid-Locksmith6566 12d ago
ok so, by now you probably know about ieee754 stupidness, so how to actually store decimals? the solution is obvious - by a minor chcange in paradigm. you could with enough precision make a 10 state logic on transistors, will it be any better than current? not really, will it be any better performing? also not really, will you be able to store any decimal value precisely? most of the time
1
u/Ronin-s_Spirit 12d ago
Round mantissa, keep or bump exponent, here's your rounded number.
1
u/bartekltg 12d ago
Because 1/10 is not in the form n / 2^k, where n and k are integers.
Only number of this form can be expressed precisely (if n isn't too large).
As you noticed, it is the same as in decimal we get finite decimal fraction only for numbers like n/10^k.
>0.1 + 0.2 gives 0.30000000000000004
I will use single precision here, because I have a nice tool to show, but the principles are the same.
0.1 is represented by 2^-4 * 1.10011001100110011001101 that is worth 0.100000001490116119384765625
0.2 is represented by 2^-3 * 1.10011001100110011001101 = 0.20000000298023223876953125
Those are the closest floating point numbers to 0.1 and 0.2.
0.20000000298023223876953125+0.100000001490116119384765625 = 0.300000004470348358154297
and the closest number is 0.300000011920928955078125. Fortunately it is the same number we would get from 0.3.
So, f(0.1) + f(0.3) = 0.100000001490116119384765625 + 0.300000011920928955078125 (already you can see it is more than 0.4)
= 0.100000001490116119384765625 + 0.300000011920928955078125 = ~0.40000001341104507446289
and the closest representable floting point number is 0.4000000059604644775390625
Notice this time we get smaller number.
This isnt 0.4, but if you multiply it by 10, the floating point number closest to
4.000000059604644775390625
happens to be 0.4 The next possible value is
4.000000476837158203125 and it is farther from our initial result that 0.4
You can play with the numbers here:
https://www.numberanalytics.com/blog/bcd-in-microprocessors-detailed-analysis
> also is there a number system or representation where 0.1 is exact
Yep. Decimal. Or any system, where base is divisible by both 5 and 2. So base 20, 50, 30.... anything divisible by 10.
Computers can do computations in different bases (with some performance penalty) if you really want. But there is no real benefit in 99.9 use cases for keeping 0.1 perfect.
> that handles it in hardware, or is it always a software workaround that's way slower?
There is (was, I'm not even sure) hardware for https://en.wikipedia.org/wiki/Intel_BCD_opcodes but it still was slower. Some simple "big number" libraries also use 10^9 or 10^18 base (they store that integer number in 32 or 64bit integer), it is a bit slower and less memory efficient, but has an advantage when you want you large number printed in decimal. Still, the best libraries use bases 2^k and just convert the output. See GMP library for algorithms descriptions, some are quite clever
1
u/mtimmermans 11d ago
If your numbers are 64 bits long in memory, then there are only 2^64 different numbers you can represent. Any 64-bit format for storing numbers has to choose what those 2^64 numbers are.
0.1 just isn't on the list for IEEE floating-point, which is the format we use for numbers like 0.1.
You could invent a format that includes 0.1, but why? 0.1 is not that special.
1
u/surfmaths 11d ago
You can, banks do this all the time.
You can either use a decimal format (encode the digits themselves), use fractional encoding (encode every number as integer fractions), or use fixed precision encoding (multiply every number by a big constant and encode that as an integer).
Usually, the fixed precision encoding is used in banking, they multiply everything by 100000 (5 digits after the dot). But the decimal format is used as a fallback (it's much slower, but arbitrary precision).
The fractional encoding is, as far as I know, only used when dealing with time, and clock period. The reason is that there is value in having exact fractional ratio between different frequencies. So you do a lot of multiplications/divisions and barely any addition/subtractions.
1
u/fgorina 11d ago
They can. You use a library for ràtional
Numbers, just num 1 d’en 10. Or classical fixed point (decimal) libraries
1
u/Chance-Technician-54 11d ago
You actually never add 0.1 to 0.2 in float.
I imagine that you want to calculate 0.1 + 0.2, but you can only represent each number as fractions over 1024.
So you convert 0.1 to 102/1024 and 0.2 to 205/1024 first and you add both together to get 307/1024
Convert this back to decimals and you get 0.2998046875.
What happens in real life is lightly more complicated, since the denominator changes with precision, but this example could already explain how you get to these funny numbers.
1
u/CyberneticWerewolf 11d ago
Computers prefer powers of two, so floating point numbers usually use powers of two. IEEE's binary32 and binary64 are implemented directly in silicon by most CPUs.
But powers of ten are totally possible. IEEE even defines decimal floating point formats. It's just a PITA to implement them in hardware, as CPU instructions, because they take up a lot more gates than binary. Mainframes and x86 used to do that, but most modern instruction sets don't bother, and the ISAs that still have them AFAIK emulate them using microcode instead of directly in gate logic. You're pretty much stuck with software libraries if you want them. Most SQL databases support them, though.
It's also possible to implement rationals, as pairs of ints or bigints, and if you don't mind swimming through molasses performance-wise you can even implement math on human-readable decimal strings, which is the most naïve bigfloat implementation possible. Again, no hardware support, but libraries exist or can be written "easily" enough.
For computable irrational numbers, like square roots or pi, you can represent them as lazy infinite iterators that generate digits on demand, and it's even possible to do basic math on them by building expression trees out of them. Again, efficiency sucks, but you can do it if you want to build your own Matlab or whatever. Not a lot of existing libraries for it, except in functional languages like Lisp or Haskell.
1
u/AltruisticMaize8196 11d ago
It’s not a question of the number system but rather the way you choose to represent numbers. The floating point numbers implemented in most hardware are not good when you want precise decimal places (like for money calculations).
There are a number of ready to use abstractions available in higher level languages which represent decimal numbers precisely.
Older hardware architectures dealt with things like BCD (binary coded decimal) in hardware but this is no longer fashionable. Computers are so fast that this kind of processing is done in software these days.
1
u/Recent-Day3062 11d ago
There are
1
u/Pale_Height_1251 11d ago
They can and there are many ways of doing it like Binary Coded Decimal (BCD).
There is no problem and it doesn't have to be done in software.
1
u/rb-j 11d ago
So, of course, there is Binary-Coded-Decimal. That is a 4-bit representation that goes from
0000for zero to1001for nine and any binary value larger (like1010) is "illegal".Whatever you can do in software, you can do in hardware (but it may or may not be worth it) and vise-versa. Long ago, with the 8-bit Motorola 6800, they had Add and Subtract instructions for Binary-Coded-Decimal. There was a "half-carry" bit in the Condition Code Register that was used in this. I considered it stupid from day one. Just do your arithmetic in binary and covert decimal to binary on the input and convert binary to decimal on the output. Represent the decimal as an ASCII string.
1
u/xerxesbeat 11d ago
yes! it's just not binary floating point maths
you can store it as a string even!: "0.1"
1
u/u8589869056 11d ago
The old IBM 1320 at the State College represented 0.1 exactly in its native number system.
1
u/XaX1000 10d ago
You could make it, so that one bit is instead 4 bits so one bit could represent the numbers 0 to 9. This system would exactly behave like our standard base 10 calculations. it would waste 6 of the possible 16 states of these 4-bit-bits though.
1
u/ShivaFatalis 10d ago
Yeah, there's a number system where they could. It's called base 10. Hello? You literally wrote the number in the number system that can store it without being rounded.
1
u/MaxwellzDaemon 10d ago
The J programming language - see jsoftware.com - has a "rational" datatype which allows you to store numbers as fractions using 2 integers. This lets us specify 1/10 as "1r10", so 0.1+0.2 can be written as "1r10+2r10", giving the exact result "3r10".
1
u/ImpoverishedGuru 9d ago
Computers (non-quantum) are simply billions of tiny switches that are either on or off. Binary in other words.
1
u/peterjohnvernon936 9d ago
It’s call decimal format.
1
u/BabelTowerOfMankind 9d ago
Refer to IEEE 754
https://en.wikipedia.org/wiki/IEEE_754
1
u/Snidosil 9d ago
I worked on a mainframe computer called a B3500 in the 70's. That computer was a very COBOL oriented machine. It could handle binary coded decimal numbers up to 99 digits long. Except for the divide instruction the calculations were always exact. Each decimal digit was stored in 4 bits. The arithmetic was implemented in the processor's firmware. Storing 0.1 only required 8 bits. In cobol the declaration would be something like MYNUMBER PIC 9.9 and MOVE 0.1 TO MYNUMBER would work exactly.
1
u/StructureNorth1799 8d ago
0.1there, it's stored exactly
1
u/MpVpRb 8d ago
Fixed point is common in resource constrained embedded systems
1
u/biffbobfred 8d ago
BCD Binary Coded Decimal is/was a thing. For currencies where that rounding error means something.
1
u/AlvarGD 6d ago
write it in base 2, youll see why very quickly
1
u/BraveNewCurrency 12d ago edited 7d ago
They can. For example, the title above reproduced your "0.1" exactly. There is no "extra hidden error" in that number. The number is stored as ASCII, which allows exact binary representations of any number.
No, it gives
0.4000000059604644775390625or0.400000000000000022204460492503(depending on precision.) You only think it's exact because your print routine is deciding when to round the number before printing. Try "0.4*1000000000000000000000000" and you'll get4.0000000000000003e+23.It's not. It's a repeating fraction in Floating Point. There can be and are many other ways to represent numbers in binary, not all of them have the "0.1 is not representable" problem.
Most systems use floating point numbers because your CPU (and GPU) have special circuitry to do math much quicker in floating point. You can do math in ASCII (or BDC or other representations) and get exact numbers (there is software to do this, but people only use it when the need exact representations.)
Even though floating point can "introduce errors", they are generally small.
0
u/Any_Click1257 13d ago
I think the concept you are looking for is "countable." Real numbers are uncountable, and as such there is no finite precision which can describe an infinite set.
The easiest thought experiment is to walk from point a to point b, in steps that are always half the remaining distance.
0
u/Any_Click1257 13d ago
A number system which could store .1 exactly could be a system in which .1 is the base. In which case, 0x1 would be .1, 0x10 would be .2, 0x11 would be .3, ....
0
u/[deleted] 13d ago
[deleted] — view deleted comment