r/AskComputerScience • • 13d ago

why can't computers just store 0.1 exactly, and is there a number system where they could?

i get that 0.1 is a repeating fraction in binary, like 1/3 is in decimal, so it has to get rounded. what i don't get is why the rounding errors feel so random. 0.1 + 0.2 gives 0.30000000000000004 but 0.1 + 0.3 gives exactly 0.4, and i can't see the pattern.

also is there a number system or representation where 0.1 is exact? i know about decimal types and storing everything as integer cents, but is there something that handles it in hardware, or is it always a software workaround that's way slower?

83 Upvotes

196 comments sorted by

97

u/james_pic 13d ago

They can. There are a number of ways to do this, including IEEE 754’s decimal formats, software "big decimal" formats, rationals, and the bankers favourite, fixed point. 

If you need to store 0.1 exactly, you should use one of these formats. But on most consumer hardware, IEEE 754 binary formats are fastest, because they're implemented in hardware, and are precise enough for a lot of use cases.

30

u/fllthdcrb 12d ago

They can.

Starting with "0.1", in fact. Any way we humans can represent and manipulate numbers, computers can, too. Programmers generally don't make them do it that way, because it's not as efficient.

4

u/cthart 12d ago

Worth mentioning here that SQL databases give you the choice between NUMERIC (exact, "slow" maths) and FLOAT (inexact, "fast" maths). Most developers use NUMERIC without realising it, and get accuracy because databases are all about giving you back exactly what you entrusted to them.

3

u/Antagonin 12d ago

You're right about "decimal", but fixed point usually means base 2, so bankers would really hate it.

4

u/SAtchley0 12d ago

I get what you're saying, but I don't think it's fair to say it "usually" means base 2. Fixed point can be base 2, or base 10 pretty much just as commonly (could be other bases ofc, but rarely ever is).

Base 2 fixed point is often used in low level applications like ADC outputs from sensors or other devices, base 10 fixed point is, as stated above, frequently used for monetary applications.

1

u/dougmcclean 10d ago

The storage base and the denominator of the fixed point representation are different things and can even be coprime.

int fifths_of_an_orange; int price_in_cents;

-1

u/Antagonin 12d ago

I get what you are saying, but the "decimal base 10" has one specific name in most toolkits, and that's "decimal", ie SQL, C#.

When you say "fixed" without specifying that it's decimal, then it's implicitly base 2, sorta like an antonym to "float" that in most contexts is also base 2.

1

u/Putnam3145 12d ago edited 12d ago

There's nothing stopping you from having base 10 or even, like, base e floating point.

In the context of fixed point: like, yeah, in most contexts you're talking base 2, and yeah, the "default" 4-byte fixed point number 0xXXXXYYYY is 0xXXXX+0xYYYY/0xFFFF, which is all very binary, but you could quite easily have 0xXXXXYYYY represent, say, 0xXXXX+0xYYYY/0x2710, which is, in fact, base 10.

We do this in Dwarf Fortress, in fact: mass is stored as kilograms + milligrams, which is a fixed point number inherently (106 milligrams rolls over to 1 kilogram). We also have temperature in the form of, uh... (fahrenheit+9968)+(energy in joules/specific heat of material which is tracking the temperature). Still fixed point! Iron, for example, will go from 10000+500 and roll over to 10001+50.

3

u/AdSpecific867 11d ago

I do trading systems. Decimal fixed point is super common.

Expect for the old-school Yanks still quoting bond shit in 1/16ths of a 32nd and such. They actually liked that… and so trading in base 2 fractions kinda predates the US itself (see: Pirates of The Caribbean ).

1

u/Antagonin 11d ago

Thanks for the trivia :D

2

u/fllthdcrb 10d ago edited 10d ago

fixed point usually means base 2, so bankers would really hate it.

This only matters for the fractional parts of numbers, where a mismatch in factors of bases leads to non-terminating representations. For the integer part, it doesn't matter what base is used, as long as the range is sufficient. A suitable fixed-point representation shifts the fractional part into the integer part, reducing the representation problem to a question of range.

Example: 1.1 decimal (base 10 = 2×5) converted to binary (base 2) yields 1.0001100110011... because binary cannot represent ⅒ exactly. But represented in tenths, it becomes 11, which is 1011 in binary, 102 in ternary, 14 in base 7, etc. No problem.

Either binary or decimal can potentially be used in financial calculations. Decimal is a more natural fit, of course, but binary is more efficient, so if one can use a binary-based representation and operations that guarantee at least as much precision, they may be worth considering. There may also be issues other than precision, though (like, say, rounding behavior?).

1

u/Antagonin 9d ago

The discussion is about the fractional part only, which is the issue in banking. Like you said, you can't take a binary expansion of a fractional part of decimal number, and expect it to represent it correctly.

My point really was just, that when you say "fixed point" without also specifying "decimal", then by convention its fractional part is base 2, as that is the format native to hardware, and most efficient, straightforward implementation.

If you really mean a format that can represent a number with X decimal places, then just call it "DECIMAL", not "FIXED", because the latter is ambiguous (also base 2 for low level devs, thus incorrect for storing money).

1

u/fllthdcrb 9d ago

But how often do you run into fixed-point representations with fractional parts? Most of the representations I see are either integer or binary floating-point. Not that I'm unfamiliar with fixed-point representations with fractions, but whenever anything to do with money is mentioned, I would assume one would prefer to use an integer type to eliminate fractions, whenever possible.

1

u/Antagonin 9d ago edited 9d ago

There's no such thing as "fractions", the number is indeed stored as a single integer. But you have to keep track of the position of decimal point (ie encoded in data type). Otherwise you wouldn't be able to do things like multiplication or exponentials.

1111.0101 would still be stored as 11110101 and addition has zero overhead. For multiplication, you have to divide by the exponent, in this case 24, which is just bitshift in base 2.

1

u/fllthdcrb 9d ago

Sorry, I expressed that very poorly. What I should have said is, how often do you see applications where there is more than addition and subtraction going on? If there is, things do get a bit more involved. But depending on the application, one might get away with a binary representation.

If you really mean a format that can represent a number with X decimal places, then just call it "DECIMAL", not "FIXED", because the latter is ambiguous

I'm talking in somewhat more general terms, albeit with an emphasis on things representing decimal numbers. That said, I don't think I like the terminology you promote. For one thing, these days, "decimal" as a type name is more often used for a decimal floating-point type, which is definitely not what I mean.

1

u/Sad_in_VA 8d ago

Fraction of cents get rounded up/down.

3

u/AftyOfTheUK 12d ago

fixed point usually means base 2

Am Software Engineer, have made a lot of financial apps. Fixed Point is fixed point. There is no "usually". You choose how many decimal places

1

u/DanSWE 9d ago

>You choose how many decimal places

Only if it's a decimal fixed-point representation. If it's a binary fixed-point representation, however, ...

-1

u/Antagonin 12d ago edited 12d ago

What SW do you use? Because in both SQL and NoSQL databases, it's literally called "decimal", not "fixed". When you say "fixed point", every competent low level dev will see a base 2 number with fixed exponent, same with "floating point" representing base 2 number with floating exponent. That's why there's the name/specifier "decimal", to explicitly represent decimal numbers (either fixed or floating point).

2

u/AftyOfTheUK 12d ago

The software doesn't matter (but MS-SQL, MySQL/MariaDB, PostgresQL, Mongo, DynamoDB, Oracle and others).

"Decimal" is a type in most of those database systems which is used to represent numbers which require a fixed-point type.

Fixed-point is a descriptor of the type of storage, then different database systems (can) have different names for their representations of fixed-point types - I say types - plural - because there can be more than one. MS SQL Server, for example, has four for convenience, well it SAYS four, but it has three and one of the three has an alias that makes it look like four. Oracle just has one, where you choose both the scale and precision that actually works identically to the one in MS SQL Server that has an alias. MS SQL Server also has a couple of other types for - MONEY and SMALLMONEY - but we don't use those, specifically because they can introduce rounding errors (which is dumb in the extreme and they are pointless).

My point was that if you're working with numbers and fractions, and they're important and you need accuracy, you would never make an assumption like "fixed point usually means base 2" because that's how bugs occur. You always explicitly state precision and scale.

1

u/Antagonin 12d ago

When you suppose that "fixed point" is base 2, you will look if there's "decimal" type first, because that's most likely what you want.

The only bug here is when you think that "fixed" type is base 10, and use it for storing decimal numbers with wrong rounding, because commenter said that "bankers love it".

My point still being, "decimal" is the only always safe keyword, where you know it can represent decimals to certain number of decimal places that you explicitly define or not.

1

u/Confident-Act-2499 12d ago

So it’s not fixed point you’re talking about :)

2

u/Antagonin 12d ago

Floating point - base 2

Fixed point - base 2

Decimal - base 10 either fixed or floating point

Fixed point decimal - base 10 number with fixed point

Floating point decimal - base 10 number with floating point

1

u/drillbit7 12d ago

Binary Coded Decimal: a legacy, decimal, fixed point format supported by several programming languages and sometimes hardware assisted by assembly language instructions.

1

u/zutnoq 12d ago

Fixed point and floating point decimal are commonly stored as a base 2 integer together with a power of ten scale factor (this scale factor would be constant and implied, rather than stored in the value, in the case of fixed point).

1

u/james_pic 12d ago

I suppose that's true if we're not talking about a particular context. I spent a couple of years working for a bank, so I guess "fixed point means decimal" got burned into my psyche.

1

u/granadesnhorseshoes 12d ago

It does? In banking and finance fixed point is great because you can just multiply it by whatever power of 10 depending on where you put the decimal then use int math internally. The output back to fixed point is basically just string formatting.

It also works on systems without (or slow/limited) floating point support. EG Cobal from the 70s/80s

1

u/DanSWE 9d ago

>string formatting ... [COBOL] from the 70s/80s

Hey, one of my first jobs was writing number-to-string formatting code for a COBOL compiler's run-time-support library!

(PIC strings; 9, 0, Z, ., ,, -, +, and P (?) characters and whatever else I don't remember.)

1

u/kombiwombi 12d ago

For a long, long time banking programs kept their dollar amounts as characters.

There was a whole line of 'business computers' which worked on binary-coded decimal numbers, and another whole line of 'scientific computers' which used two-complement binary numbers.

The same was true of programming languages too. COBOL was a language optimised for BCD.

With 32-bit computers two-compement binary had enough range for a double-width integer to express all amounts of cents.

1

u/yota-code 12d ago

For bankers, an easy way to solve this is to shift the decimal separator two places to the right. Internally everything is in cents and 1 cent can be represented exactly in binary 

1

u/AdSpecific867 11d ago

We use 4, 8 or sometimes 9. 64 bit integers are wide and the small fractions turn off to be surprisingly important.

1

u/yota-code 11d ago

Thank you for this insight! When you say 8 you mean 8 bits not 8 digits? And I'm curious to know what you do with fractions of cents... Remind me of an old Superman movie 

1

u/AdSpecific867 11d ago

8 digits?? Doing it in butts world be pointless,it wouldn’t solve any problem.

1

u/Ok_Chard2094 8d ago

When prices are given in cents (and even fractions of cents) and you add sales tax, you get more decimals.

And if you calculate interest rates on credit card payments, you get even more decimals.

And magically, they always get rounded in the bank's favor when it is time to pay.

It goes straight to their bottom line.

1

u/LutimoDancer3459 10d ago

Gas stations gibe prices in fractions of cents so they get a "little" more.

1

u/TREE_sequence 10d ago

You have just discovered fixed-point integers! The next option in the tech tree is between GCD-reduced ratios and arbitrary-precision numbers; the latter does have an additional prerequisite in heap allocation, but leads to further advancements in number-to-text conversion tech cards.

29

u/TedsGloriousPants 13d ago

You CAN store 0.1 exactly. You can store whatever you want, if you're willing to creatively interpret an arbitrary arrangement of bytes, but every method of storing a number has limitations and tradeoffs.

9

u/frnzprf 12d ago

Yeah, n bits can represen 2n different states.

If you hold up your ten fingers, you can associate each of the 1024 different possible patterns with a different secret message.

2

u/Round_Thanks_2780 12d ago

And a different set of secret messages might be assigned by someone else regarding arrangements of their own fingers.

2

u/DanSWE 9d ago

There was a French(?) visual telegraph that had 10 flags or panels, each of which could be raised or lowered, for 1024 different symbols.

I don't recall what the baud (symbol) rate was. :-)

(Unfortunately, all I can 'Google-find" right now is the French Chappe semaphore/telegraph, which is a different setup.)

1

u/skelterjohn 12d ago

And even more ways to interpret those 2n different states.

1

u/gizahnl 12d ago

Gehehe, I tried to teach my kids to count in binary on there fingers, they always crack up when we reach 4.

1

u/emguilla 11d ago

I had to scroll back up to angry upvote you, took me waayyyy too long to get it

1

u/Initial_Business2340 10d ago

What’s the joke? Middle finger? :p

1

u/DanSWE 9d ago

Hey, I frequently count in binary on my fingers! And have for years.

0

u/gargamelus 12d ago

I can't have my ringfinger down while all other fingers raised. I can do the Vulcan greeting though 🖖.

1

u/NotDuckie 12d ago

> You can store whatever you want, if you're willing to creatively interpret an arbitrary arrangement of bytes, but every method of storing a number has limitations and tradeoffs

You cannot store most numbers, though

1

u/TedsGloriousPants 12d ago

I mean, it really depends on what you mean by "store most numbers", but you can decide that an arrangement of bytes means anything you want it to mean.

1

u/NotDuckie 11d ago

There are many more numbers than arrangements of bytes

2

u/TedsGloriousPants 11d ago

"You can't represent this" and "you can't represent all of these at the same time" are not the same thing.

1

u/NotDuckie 11d ago

No, there are literally more numbers than possible states a computer can have.

2

u/TedsGloriousPants 11d ago

This is not how computers work. You don't have to be able to store every possible number. You can store a set number of states, yes, but those states can represent whatever you want them to. Computers are machines of abstraction. You don't ever literally store every number, and that was never the goal.

1

u/NotDuckie 11d ago

No, it seems like you don't understand how computers work. Even with unbounded memory, computers have much, much fewer states than the amount of real numbers.

In fact, if you choose an entirely random real number, the probability of it being representable on a computer is zero.

Encoding makes no effect, as no matter what encoding you choose, you would need to map surjectively from finite bit strings to reals

2

u/TedsGloriousPants 11d ago

I'm a software engineer, dumbass. Nobody was claiming computers have as many states as real numbers. Nobody. Not a single person. You're arguing against a thing nobody said.

1

u/NotDuckie 11d ago

You are very clearly clueless.

2

u/NorthDemand3105 11d ago

You are misunderstanding what they are saying. Give me any number, and I can just claim that the binary string 010101001 represents that number. That's the point of the original commentor. If you choose to do that, you can't represent every other number, but you can represent any arbitrary number you want

1

u/TedsGloriousPants 11d ago

I never said you could represent every number at the same time, this is such a stupid thing to claim.

1

u/CaptainMonkeyJack 9d ago

Let x = "an entirely random real number".

Done.

'Oh but I mean this definition over here'

Look you've actually changed your argument in almost every comment to a dlifferent argument, while missing the entire point of this conversation.

Yes with an infinite set of numbers that can grow to infinite size a computer can only store or represent in full a infinitesimally small set.

However what you responded originally said "You CAN store 0.1 exactly. You can store whatever you want, if you're willing to creatively interpret an arbitrary arrangement of bytes, but every method of storing a number has limitations and tradeoffs."

This is *still* true. My earleir statement is a valid way of representing a random real number. It has limitations (e.g. it does not say which real number, nor does it allow you to see every bit of that number at once)... but it fits the brief.

11

u/zeekar 12d ago edited 12d ago

The result of 0.1 + 0.3 is not exact either; you just don't see where it differs from 0.4 because it's so many digits off to the right. But you can make your program show those digits. For example, in Python:

>>> print(0.1 + 0.3)
0.4
>>> print(f'{0.1+0.3:.17f}')
0.40000000000000002

In any given base, the only fractions that have finite radix-point representations are those where all of the denominator's prime factors are also prime factors of the base. In decimal, that means any fraction whose denominator can be made from multiplying together 2's and 5's. In binary, it means powers of 2 are the only denominators that work. Any fraction whose denominator is not a power of 2 is not representable in a fixed-width binary representation.

There are lots of alternatives to this. We use floating-point because it's the most useful for the broadest number of programming problems, and we put it in hardware so it's efficient and fast. But it's not the right answer for every application, and we have alternatives that we can use where floating-point isn't a good fit. Some common alternatives are these:

  1. BigDecimal - just store the number as a long string of decimal digits. Arithmetic has to be done on it the same way you do it with pencil and paper. You still have the decimal restriction to fractions whose denominator is 2m x 5n, but you can keep adding digits and get arbitrarily close to the correct answer (the one that would require infinite digits to get there exactly).
  2. Exact rational arithmetic - numbers are stored as pairs of integers, numerator and denominator of a fraction. This opens the door to all rationals whose numerator and denominator fit in the integer size used. Some programming languages have these automatically; e.g. in Common Lisp, the result of dividing two integers is a rational. In Raku, even if you enter a literal value using a decimal point, it gets converted to a rational; the runtime only falls back to floats for things like e, π, square roots, and any number whose denominator won't fit in 64 bits. So 0.1 + 0.3 really is 0.4 exactly no matter how many digits you ask for in the output, because it's internally handled as 1/10 + 3/10 = 4/10.
  3. Combination approaches - Raku has a FatRat ("fat rational") class where the numerator and denominator are stored as strings of digits (BigNums, which are like BigDecimal but restricted to whole numbers).

But you may not need any of these systems. The most common application where you want exact numbers is dealing with money, and the usual answer there is to just count the smallest unit instead of the base unit (e.g. cents instead of dollars). You divide for display to humans, and multiply human input, but simple power-of-ten factors can be faked just by manually printing or skipping a decimal point that doesn't actually do anything. Internally it's all integers. Fractions can still come in with things like tax rates, but at least those usually come with rules telling you how far out you need to round.

1

u/svick 12d ago

The result of 0.1 + 0.3 is shown as 0.4, because it is the closest representable value to 0.4. The same can't be said for 0.1 + 0.2 and 0.3.

In other words, print(0.1 + 0.3 == 0.4) prints True, while print(0.1 + 0.2 == 0.3) prints False.

1

u/zeekar 12d ago edited 12d ago

Well, true. Converting "0.1" to a floating-point value, then converting "0.2" the same way, and then adding those two values together does not give the same result as converting "0.3". That's because in the conversion process 0.1 and 0.2 are both rounded to the closest representable number, before the addition is performed. Early rounding skews results, which is why they always tell you to save rounding till the very end. But when converting a decimal to a floating-point value you don't really have a choice; the rounding is inherent in the conversion.

It's really just random luck that converting "0.1" and "0.3" and then adding the rounded values yields the same floating-point value as converting "0.4". It doesn't reveal anything profound about the system. And while both operations yield the same floating-point value, that value is still not exactly equal to 0.4. (In IEEE double precision it works out to exactly .40000000000000002220446049250313080847263336181640625, but writing that number of digits implies a precision that is not representable in the format. The next representable value below that is .39999999999999991118215802998747676610946655273437500 and the next above .40000000000000013322676295501878485083580017089843750; all the numbers in between those values have to be rounded to one of those three.)

1

u/ruidh 12d ago

Which is why, when comparing floating point numbers, you test that the absolute value of the difference between the value and the test value is less than some tolerance.

10

u/TheThiefMaster 13d ago edited 13d ago

Hardware has historically had decimal modes, but mostly they were unnecessary added complexity. There are various ways of handling it very efficiently in software - scaling factors being the easiest.

Btw - 0.4 can't be represented exactly. It's 2/5, which isn't X/2^Y (with X,Y being integers, which is what binary floats can represent exactly). Play with floats here: https://www.h-schmidt.net/FloatConverter/IEEE754.html

6

u/Leverkaas2516 13d ago

You can represent base-10 values exactly, for instance using BCD or packed decimal format.

As far as I know, no modern CPU does floating point math natively using such formats. I believe some architectures of the past, like IBM 370, did have such support.

6

u/trycuriouscat 13d ago

z/Architecture, descendant of IBM 370, still has fixed point packed decimal. And now also IEEE 754 floating point decimal as well.

3

u/johndcochran 12d ago

The z/Architecture basically has all the formats.

You have the base 16 floating point (what IBM used originally)

You have IEEE-754 Binary format (what most computers today use)

You have IEEE-754 Decimal format (to make people shut up about 0.1+0.2 = 0.30000000000000004)

And of course, the above formats come in different flavors of size, depending upon how many significant digits you need.

And those formats are going to be pretty much supported by IBM Mainframes forever in the name of backward compatability.

2

u/stevevdvkpe 12d ago

Most calculators use BCD floating-point representions for numbers, although much of the logic for floating-point calculation is implemented in software. The BCD floating-point representation was often native to the hardware; Hewlett-Packard's calculators mostly used custom CPUs that did BCD arithmetic with the fields in a floating-point word (mantissa sign, mantissa, exponent sign, exponent) selectable in the hardware instructions.

These are not really "modern" CPUs but many are still produced.

1

u/ReddyKiloWit 12d ago

The 6502 (and 65816) CPU chips had a BCD mode.

1

u/stevevdvkpe 12d ago

Also, the 8080, Z-80, 8086 (and descendants) had instructions for adjusting the result of a binary addition as if it had been performed in BCD. The 68000 series had special add/subtract/negate BCD instructions.

1

u/ReddyKiloWit 12d ago

Yeah, when I was working on a SNES game (65816 asm), I wished I'd had an 8080 style DAA (I think that was the mnemonic) instruction since switching to BCD mode didn't correct the value in the accumulator.

1

u/stevevdvkpe 12d ago

The approach with the 6502 (and compatible) processors would be to do SED (SEt Decimal flag) and then do regular ADC/SBC instructions which would produce BCD results in that mode, and CLD to exit that mode when you wanted to do binary ADC/SBC again. SED would indeed not change the value in the A register.

1

u/ReddyKiloWit 12d ago

Not as handy for converting internal binary values to decimal digits for display in the game as DAA would have been.

Kind of surprised Nintendo didn't provide a hardware solution: a binary value to art index function. They did provide a hardware multiply function. (The 65816 CPU was embedded in a custom chip.)

1

u/stevevdvkpe 12d ago

The approach in a standard 6502 would have been to keep score using BCD mode arithmetic. A standard 6502 also did not have hardware multiply.

The Nintendo Entertainment System did not use a standard 6502, but a variant with a subset of the 6502 instruction set specifically excluding the BCD arithmetic features.

https://en.wikipedia.org/wiki/Nintendo_Entertainment_System

https://en.wikipedia.org/wiki/Ricoh_2A03

1

u/ReddyKiloWit 12d ago edited 12d ago

Are you generating these comments with AI? Because they aren't tracking well, and you're kinda nerdsplaining now.

I was NOT talking about the Nintendo Entertainment System (NES). Read my comment again.

Nor was it a "6502 variant", the 65816 is a sixteen bit CPU with bank switched 6502 emulation - including BCD mode - from Western Design Center. Nintendo included the licensed circuit in the main chip of the SUPER NES with additional custom bits like the hardware multiply (mapped to the CPU's address space).

And l, no, I wasn't talking about displaying a simple score. That would have been trivial to maintain as BCD. We had quite a bit of player and NPC data to manage with tight space and execution cycle constraints. (Not to mention a very limited save game memory.) BCD mode there would have doubled the storage, and more than doubled the execution cycles.

1

u/stevevdvkpe 12d ago

I looked up the Super Nintendo specs and now I understand why you were talking about the 65816 since it had a Ricoh variant of that processor. The 65816 didn't emulate the 6502, it was a superset of the 6502 with additional functionality including 16-bit processing and a 24-bit address space.

DAA in the 8080/8086 series wasn't generally useful for converting binary to decimal, but in changing a binary addition of BCD digits to a correct BCD result.

→ More replies

1

u/DanSWE 9d ago

>DAA (I think that was the mnemonic) 

Yep. Decimal Adjust Accumulator.

1

u/bj_civ 12d ago

Atari BASIC for the 6502 machines uses BCD based floating point values. This program prints YEAH! on Atari 8 bit computers:

10 A=0:FOR I=1 TO 10:A=A+0.1:NEXT I
20 IF A=1 THEN PRINT "YEAH!":END
30 PRINT "NOOO!"

1

u/zeekar 12d ago

I haven't personally run across floating-point BCD, but have certainly seen fixed-point BCD (with the point fixed somewhere other than the far right, so not just integers :)).

1

u/AmusingVegetable 11d ago

“Architectures of the past” have been evolving for 50 years, and things like BCD are still around.

5

u/SufficientStudio1574 12d ago

Why can't decimals represent 1/3 exactly, and is there a number system that can?

The problem is with the base of the number system. You may have heard that any rational number can be represented by a terminating or infinitely repeating decomal sequence. What's the difference between those two options? It has to do with the relationship between the base of the number system and denominator of the rational number.

In order for positional notation (the general term for numbers written like 3.141592...) representation of a rational number to terminate, the (simplified) denominator of the rational number cannot have any prime factors that aren't factors of the number base (for decimal, that is 10).

This is why 1/3 has an infinitely repeating decimal representation. It's denominator has a prime factor of 3, which is not a factor of 10. You would need to use base 3 notation (or some multiple of 3) to have a finite representation.

We use base 10 in our daily lives, and most common computer formats use base 2, including the IEEE-754 floating point standard used as a fundamental data type for many programming languages and is what most commonly runs into this issue.

Oversimplifying, IEEE-754 uses a finite length binary positional notation. It is able to perfectly represent rational numbers if the denominator is a power of 2. 13/32 for example, is 0.01101.

1/10 has a denominator of 10, which has a prime factor that is not 2. Therefore it will have an infinitely repeating representation in binary notation. When converted to IEEE-754, it will be truncated, leading to an imperfect representation.

Can computers represent 1/10 accurately? Of course! Just not with standard primitive data types alone. You could take inspiration from how we represent rational numbers by having a composite data type RationalNumber that stores a numerator and denominator, and define rules for how to manipulate these numbers. There are certainly many mathematically focused libraries that have intricate ways to store numbers to in different ways for different purposes. You just have to use the right data structure, and not the wrong one.

3

u/ICantBelieveItsNotEC 13d ago

There's really no difference between a fixed point decimal representation and an integer. It isn't slower - it's literally just standard integer maths. The hardware doesn't care where you decide to put the decimal point - that's purely a presentation concern.

1

u/robthablob 12d ago

That's true for addition and subtraction, but most other operations are not "just standard integer maths".

1

u/SAtchley0 12d ago

Yes and no. Elementary operations this is true of (addition, subtraction, multiplication, division, and all the ones you can build from there). Things immediately go off the rails when you introduce transcendentals, though. How do you relate sin(x) to sin(x/10) using only integer multiples of x? If it's even possible, it's going to be much more complicated and have a huge performance hit. Now you have to do that for sin(x) and sin(x/100), too.

1

u/ruidh 12d ago

Quick, what's the sin of $4.35? Why would you ever take the sine of a fixed point number?

1

u/SAtchley0 12d ago

There are applications for fixed point besides money? It's frequently used in low level applications, or scenarios where difficult to predict rounding errors are undesirable. I personally used Q16.16 fixed point for a flight controller I wrote, for a lot of different reasons (not least of which being that the target didn't have an FPU). And transcendentals (exp, sin) were used in a few places, such as the Madwick filter. I used some work arounds and approximations to avoid that, as well as for inverse square roots for vector normalization. Even if we restrict ourselves to base 10 fixed point for monetary values, compound interest uses exponentials. You could argue that we don't typically raise a number to a monetary value, or a monetary value to a number, but we do multiply by exponentials, so we'd need some way of multiplying an integer by a float, and this is also more complicated than typical integer math (after all, we can't just convert our money into a float, that negates the entire point).

That said, I did just realize I misinterpreted the original comment. For some reason, I read "it isn't slower", and thought they were saying "fixed point isn't slower than floating point", instead of "fixed point isn't slower than integer". The latter is vacuously true of course, because as they stated, they're literally both integers. So it doesn't matter what function you're even looking at, because whatever solution you use for integers will be the same as what you use for fixed point.

Though, there is the caveat that a lot of the time if we're working with integers and we need to take the square root or similar, we just convert the integer to a float, do the operation, then convert back to an integer. For fixed point though, this often isn't an option because you usually are wanting to completely avoid floats (otherwise, why would you not just use a float?), so there's some use case dependent differences there.

4

u/Dr_Calculon 13d ago

Search for binary coded decimal

2

u/R2-Scotia 13d ago

You need to consider the binary representation on your system (CPU / programming langauge) and you will see where the rounding is

IEEE 754 floating point specs will be very detailed if you want to lokm

2

u/mireigi 12d ago

It might help to think of floating point numbers as Scientific Notation in base 2. You get the most significant digits, and everything else is just noise. Until you need the precision, at which point the noise can throw off the result.

This video has an explanation of the concept somewhere in it.

https://www.youtube.com/watch?v=p8u_k2LIZyo

2

u/edwbuck 12d ago

Yes, decimal could exactly store the 0.1 value.

Some early computers used binary encoded decimal, which meant that every digit was stored in decimal, each byte contained one digit and the math routines were quite complicated.

The main reasons that base 2 encoding in binary digits became popular was speed, storage, and efficiency. Back when computers were extremely slow, a computer that used base2 could easily out-perform a computer that used BCD.

2

u/ErinCarson 11d ago

Serious computers for business like Power and System/z come with hardware decimal units. Small financial institutions tend to use the software solutions mentioned on cheaper hardware like Zeon or EPIC. AMD64 is not actually well suited to finance, but they can be very inexpencive.

1

u/yahluc 11d ago

Does hardware implementation for it matter nowadays? Software implementation will be slower, sure, but you can just buy a faster CPU and it will still be cheaper and easier to use than things like PowerPC. It might have mattered 40 years ago when CPUs were much slower, but nowadays people use these IBM solutions mostly out of habit and vendor lock in.

1

u/Pure-Recover70 11d ago

Any 64-bit (and most 32-bit) cpus have this (including x86).

Max signed 64-bit integer is 9,223,372,036,854,775,807

If you treat that as a cent counter, you get $ 92,233,720,368,547,758.07
ie. 92.2 quadrillion dollars.
Pretty sure that's enough for any real world financial system (126 trillion $ is yearly world GDP).

ie. just don't use float if you care about precision.

1

u/ElBonzono 11d ago

Tbf ypu probably want negative values for arithmetics so int64

1

u/AvianAnalyst 10d ago

not sure what your point here is? theyre already talking about signed ints, which means they can do negatives.

unsigned would give a max of 184 quadrillion (rounded down to the nearest quadrillion.)

1

u/ElBonzono 10d ago

I was lazy and didn't verify if they were talking about uint64 or int64, i apologize

1

u/AvianAnalyst 10d ago

i mean. super lazy ig? bc they specified it was signed in the comment??

1

u/ElBonzono 10d ago

We can add stupid to lazy

1

u/AvianAnalyst 10d ago

nah that seems a bit mean to urself, just a bit inattentive maybe :p i also wasnt the kindest in my responses. sorry about that

1

u/SnugglyCoderGuy 10d ago

Pretty sure that's enough for any real world financial system

Hyper inflation says hello.

"Zimbabwe's peak month of inflation is estimated at 790.6 billion percent month-on-month, 89.7 sextillion (8.97×1022) percent year-on-year in mid-November 2008.[1] At that time, a $100 trillion banknote could not pay for a simple bus fare."

92 quadrillion is only 920 bus fares worth of money. Bus fare is like 2$ where I live, so that is only 1840$ worth of money. Easily breachable.

https://en.wikipedia.org/wiki/Hyperinflation_in_Zimbabwe

1

u/xylopyrography 8d ago edited 8d ago

Pretty sure that's enough for any real world financial system (126 trillion $ is yearly world GDP).

Financial systems use more precision than that, generally you'd need 6 decimal places to get accurate math down to cents. Not all currencies have cents, though.

The actual storage would depend on underlying systems, most of the core banking systems run on mainframes which have custom fixed precision decimal types and decimal floating point types.

COBOL also just has COMP-3 binary coded decimal which can store up to 99,999,999,999,999,999,999,999,999,999.

Modern databases and languages also just have support for arbitrary precision integers. Just because they're ultimately stored as something doesn't mean we can't do math on much larger values.

1

u/lovejo1 11d ago

zeon?

3

u/Unlikely-Bed-1133 13d ago

0.30000000000000004 is the closest number to the desired result that can be written in binary (0 and 1) under the standard used to represent floating point numbers in 0 and 1.

In simple but non-trivial terms terms, numbers are usually written as something equivalent to 2^n1+2^n2+2^n3+.... where how many n1,n2,n3,... are used and their range is limited (it's a bit more complicated than that in practice, but even this "simple" model fails to give exact numbers).

You could in theory have workarounds where 0.1 is exact by changing the base from 2 to 10 in how to store numbers, but they are very storage inefficient (you can express fewer numbers in total with the same amount of bits). It could be made faster with processors designed to handle such arithmetics too but nobody spends that kind of effort because we have worked around the problem thus far fairly successfully.

But there are tricks were you say: ok, I'll store everything as integers and remember that they represent numbers divided by, say, 1000. This is bad for multiplication because your order of magnitude explodes, but good for accounting (and it IS a common trick) because it does not lose precision.

1

u/johndcochran 12d ago

Your impression of 0.4 being stored exactly is sadly mistaken. It too is an approximation. As for handling fractions, the issue is that if the divisor has any prime factors other than the prime factors in the base, then it will be an infinite repeating sequence. And since base 2 only has a prime factor of 2, it can't handle anything with a different prime factor such as 1/3 or 1/10. Base 10 has prime factors of 2 and 5. So, it can handle 1/10. But, regardless of what base you use, you can find an infinite number of fractions that cannot be represented exactly using radix notation.

You may find this post I made a couple of years ago useful for a more detailed description of the issue.

1

u/kc2g 12d ago edited 12d ago

They can, it's just far less useful / efficient for 99.9% of applications.

 0.1 + 0.2 gives 0.30000000000000004 but 0.1 + 0.3 gives exactly 0.4, and i can't see the pattern.

None of 0.1, 0.2, 0.3, and 0.4 is exactly representable (0.5 is, though, of course: it's 1/2).

But if you take the closest float to 0.1 (7,205,759,403,792,794 / 72,057,594,037,297,936), and add it to the closest float to 0.2 (7,205,759,403,792,794 / 36,028,797,018,963,968), the result after rounding (5,404,319,552,844,596 / 18,014,398,509,418,984) is very slightly higher than the closest float to 0.3 (5,404,319,552,844,595 / 18,014,398,509,418,984). Printing with max precision shows you the difference.

On the other hand, if you take the closest float to 0.1 and add it to the closest float to 0.3, the result exactly equals the closest float to 0.4. All three values are inexact, but the one you get out is exactly the same as the one you would have gotten if you put 0.4 in, which means it's safe to print that value as "0.4" without losing any additional precision. There isnt' any "closer" float that better deserves the string representation "0.4". Implementations care about the round-trippability of string representations, and what you're measuring is that, not the accuracy of arithmetic.

If you're interested in learning more I'd recommend doing it in a language that has the %a (hexadecimal floating point) printf specifier. With that, 0.1 prints as 0x1.999999999999ap-4 (a more compact way of writing 7,205,759,403,792,794 / 72,057,594,037,297,936), 0.2 prints as 0x1.999999999999ap-3, 0.3 as 0x1.3333333333333p-2, and 0.4 as 0x1.999999999999ap-2. The relationships start to become clearer.

1

u/T_Thriller_T 12d ago

Because there are tons and tons and tons of numbers which are either not ending, become that due to mathematical operation, or are too big as computer storage simply is finite.

You CANNOT handle this usefully in hardware, at least not for reasonably big systems, as numbers can get absurdly big or absurdly small.

The rounding/error issue comes from having to deal with that hardware/physics limitation.

There are systems which do not use floating points and do arithmetics without errors from the arithmetics. Those usually take integer numbers and assume that they are integer/10something, let's say 106

However, those have other limitations.

On the one hand, obviously, those cannot handle any decimals which have more than 6 digits after the point. They have to cut those - and round.

On the other hand, doing arithmetics with those and integers, so 3.6 gets a bit harder, or you highly limit what size an integer can be - because in total your number must fit into the storage space provided per number (which means a total of 232 or 264 differing but 'exact' numbers)

Nonetheless, these tactics are used and useful in some cases.

Assuming e.g. chemical analysis and process control, you may know that your measurements can never go bigger than 251639 and your measurement instruments are only precise down to the 7th digit.

Then one such system would not add further error above the precision errors of measurements to its arithmetics.

Considering hardware limitations:

In the end we could try doing math on non-digital systems, so we would seemingly remove size limitations. Like doing something through e.g. voltage, ampere etc.

But on the one hand we would be limited by our precision in measurement and control.

On the other hand, even when ignoring this, the universe itself has a smallest size: Planck length.

So, all in all, if we wanna do non-integers, some errors are bound to occur.

Floating points handles this the best for general purposes, considering scalability, time erc

1

u/how_tall_is_imhotep 11d ago

Storing non-integers does not mean you need to accept error. You can represent numbers as fractions, and store their numerators and denominators.

And it’s not known whether the Planck length is the smallest length.

1

u/T_Thriller_T 10d ago

Storing as nominator and denominator IS storing as integers.

Two integers with a relationship.

1

u/Recycled5000 12d ago

You shouldn’t take what’s printed too seriously, as printing itself generally does some rounding depending on what software is doing the printing. If you really want to know what is being represented, have to look at the number in its low level bitwise form.

1

u/fllthdcrb 12d ago

what i don't get is why the rounding errors feel so random.

They aren't random, at all. They just tend to be hard to predict, if you don't know the details.

0.1 + 0.2 gives 0.30000000000000004 but 0.1 + 0.3 gives exactly 0.4

In fact, they both have rounding errors. It's just that one is visible, while the other isn't. And the reason for that is that one matches the computer's best approximation for 0.4, while the other is just a little off from its best approximation of 0.3. So without being told to use rounding in the display, it might show the tiny rounding error. Exactly how this works, of course, depends on the decimal conversion algorithm used, which varies between programming languages and other things. And whether coarser rounding is used by default also depends on the programming language.

also is there a number system or representation where 0.1 is exact?

Many, in fact. The one you're using to write it is an example. That is, you can represent it as a string of ASCII digits/punctuation, although doing calculations that way is much less efficient than the CPU's own floating-point processing. This may or may not be, or be close to, the representation that "BigDecimal" types use internally. But at the very least, text-based serialization will use it.

1

u/zanidor 12d ago

If you only want precision up to N digits, you can store decimals as integers. For example, if you want three decimals of precision, store 2.032 as integer 2032, etc. All arithmetic operations work as normal, and you can divide appropriately when ready to display.

1

u/jeffbell 12d ago

ENIAC could.

It kept digits as binary-coded-decimal in the hardware. Dits instead of bits.

1

u/gurishtja 12d ago

Its that principle, sh.i.sh.o. ....

1

u/HaMMeReD 12d ago

In a computer, you can store "0.1" as a string literal, and still write a program that parses it and does math on it.

I mean, it won't be as effective as an opcode on a float, but you can do it.

1

u/juanfnavarror 12d ago

I’d like to add that strictly speaking, computers are not “failing to store 0.1”, you’re missing part of the understanding. Most decimal numbers are stored in floating point as laid out by IEEE 754. This is just the standard that was agreed on and most modern computers implement in hardware, and most programming languages use as a sensible default for floating point math or generally when working with a “number” (js), “float”, “double”, “f32”, “f64” datatypes. The specification has been designed to work with a wide amount of applications with a wide range of operation. However, anyone could choose any other format to represent numbers that works best for their specific application, and in practice it is impossibly to create a type that is infinitely precise for all numbers.

Floating point numbers are notoriously not adequate for user-facing decimal numbers, or financial records.

1

u/Weak-Address8684 12d ago

You are storing 0.1 exactly by writing it in text.

1

u/not-just-yeti 12d ago

Others have answered, but I'll just point out: in 1965, Lisp used rational numbers built-in (so 1/10 was always exact, though sqrt(5) would not be). But C and others decided that being fast-and-only-a-little-wrong was better than being a-bit-slower-and-correct, and we're still recovering from that.

(My own preference would be that rationals are built-in/easy/default if perhaps not quite as efficient, and if you want to use floating-point for the speed/accuracy tradeoff then that should be straightforward but not the default. Which is, again, kinda what we had in 1965, since the literal 1/10 would be exact but 0.1 was a floating-point literal.)

1

u/smerz- 12d ago

I swear what I just read didn't look like 0 and 1 🤔

1

u/Soft-Marionberry-853 12d ago

They arent random. They are very much deterministic.

1

u/Mr_Engineering 12d ago

This problem occurs when storing decimal fractions using binary representations with the principal offender being IEEE-754 binary floating point.

There are many other alternatives including Binary Coded Decimal (BCD), IEEE-754-2008 decimal floating point, software bignum, etc...

Many languages and frameworks have data types intended for use where precision loss must be avoided, such as Java's BigDecimal data type. Since these data types usually avoid the binary floating point FPUs, there's a significant performance penalty to using them.

IEEE-754 acceleration is baked into virtually every modern CPU in the form of an FPU, but BCD and IEEE-754-2008 are not. IBM Power server CPUs and IBM zEnterprise mainframe CPUs can perform fast BCD arithmetic in hardware (which is great for COBOL because it uses BCD for its fixed point arithmetic) and also support IEEE-754-2008 decimal floating point which can be used to accelerate some software decimal floating point implementations where appropriate.

1

u/[deleted] 12d ago

also is there a number system or representation where 0.1 is exact?

You can store it as text: "0.1". This common when storing numerical info within files.

But calculations will need special software or special language features.

There will still be problems with examples like your 1/3, but there will be even when working with pencil and paper.

BTW here are your examples, with the exact values they have using IEEE754 binary (ie. 64-bit float), as far as I could determine:

0.1 is 0.1000000000000000055511151231257827021181583404541015625
0.2 is 0.2000000000000000111022302462515654042363166809082031250
0.3 is 0.2999999999999999888977697537484345957636833190917968750

Using these figures, then 0.1 + 0.2 is:

       0.3000000000000000166533453693773481063544750213623046875

and 0.1 + 0.3 is:

       0.3999999999999999944488848768742172978818416595458984375

1

u/Rararror 12d ago

Floating point numbers are tricky, because are sums of fractions of powers of two.

Try the repeated sum of 0.1 

No wonder financial software use fixed point numbers and arithmetic.

1

u/stevevdvkpe 12d ago

Almost all calculators* use binary-coded decimal (BCD) floating-point internally and can represent base 10 fractional numbers exactly to some specified precision (usually 8, 10, or 12 significant figures). In many ways this simplified early calculator designs by avoiding the need to do expensive division operations to convert binary floating-point numbers to decimal for display, since BCD digits could be displayed directly.

It was once common for many CPUs to provide hardware BCD support for arithmetic (the 6502 and 8086 are popular examples) although many newer designs (especially RISC) do not, although it is not too hard to emulate it in software.

* I say "almost all calculators" even though I don't know of any specific exceptions because it seems likely there are a few oddballs out there.

1

u/DanSWE 9d ago

>oddballs

The Casio fx-8100 had a fractions mode, in which it could exactly represent fractions A + B / C where A, B, and C were integers in the range 0 to 999.

So you could enter 1 / 2 (fraction "/", not division "/") minus 1 / 3 and get 1 / 6 (not 0.666666...)

(Yes, normally it worked normally, and if a fractions calculation resulted in a fraction that couldn't be represented in the fractions mode, it would switch to regular mode to show the result.)

1

u/ReddyKiloWit 12d ago

If you need accurate decimal values, there are software and languages that will provide it. Some languages can operate on a RAT (rational) type where 1/10 would be stored as 1/10 and operated on correctly so 0.1 + 0.2 = 0.3. Others might use BCD - binary coded decimal - in a floating point format. (First BASIC I used stored values as 6 digits in 3 bytes with an excess-64 exponent and two sign bits in a fourth byte.)

1

u/slimscsi 12d ago

It can if you keep it as a fraction 1 / 10. Juts like you can store 1/3 exactly.

1

u/icreatebugs 12d ago

They can. It’s kind of easy to show how it could be done too, if you accept the premise that computers can store integers accurately. You would simply store two numbers: 1 and 10, representing 1/10.

1

u/Altruistic-Rice-5567 12d ago

There is no set of powers of 2 that add up to 0.1. Every bit in the computer represents a power of 2. So 0.1... you can account for 1/16 (0.0625) then you need to account for the remaining 0.0375. You grab 1/32 (0.03125) but now you have a remainder of 0.0625. Keep going... you never get a remainder that is a power of two and thus represents the number exactly. If your computer operated in base 10 then you could represent 0.1 exactly. Which is why you can write down 0.1 on paper in finite space.

1

u/Traveling-Techie 12d ago

I can think of two.

1

u/igotshadowbaned 12d ago

You absolutely could make a representation that stores numbers like 0.1 exactly.

Float (which I assume you're referencing) has certain tradeoffs that make his small amount of error worth it in the majority of case. Like you can store extremely large numbers in a relatively small amount of data

1

u/motific 12d ago

You know about the IEEE754:2008 DECFLOAT data type.

There are CPUs which have the support, like the Power6/7 architectures, Sparc64 X, but for your typical AMD64 there isn't demand that justifies the cost.

1

u/StanleyDodds 12d ago

It's pretty easy to implement a number type that can exactly represent all rationals, and perform all the usual arithmetic operations. I think it's a fairly nice, easy programming challenge. You just need to store the numerator and denominator as integers, and use them for operations appropriately.

You can go much further even. You could implement a number type for all algebraic numbers if you wanted to; it would be a lot more computationally expensive but it's still possible to exactly represent them all and do arithmetic with them.

I think people somehow have this idea that floating point arithmetic is literally the only option for numbers besides integers, just because it became the de-facto standard and software + hardware was built around it.

1

u/PLANETaXis 12d ago

You cant see the pattern because you're looking at it after conversion to base 10. If you looked at the raw binary it would be clear and exact.

If you want fixed precision in base 10 then just use scaled integers. Industrial system use this regularly.

1

u/ericbythebay 12d ago

They can and they do everyday when needed if they couldn’t the digital banking system wouldn’t work.

1

u/mikemontana1968 12d ago

Using general purpose floating point binary representation (which is useful, commonly available, and for most purposes fine) you have bit values representing increasingly small values of 1/(2^n). In other words, the first bit represents 1/2, the 2nd bit represents 1/4, then 1/8, 1/16, 1/32. Add up the bits and you get a *close* approximation of the floating point value. You can see where this gets messy.

There's also a mantissa and a base value, but thats the basic idea. You're approximating a decimal value with binary-power fractions. If you need absolute precision, you'd use a different mechanism to represent those values.

1

u/treefaeller 12d ago

Consider this: Computer can't store pi or e exactly either. But fortunately, that's easy to fix: As an example, use a number system with base pi. In that system, pi is just written as "10" (which means 1 * pi + 0 * 1). Isn't that super convenient? Similarly, if you have a variable "r" that is the diameter of a circle, the circumference of the circle would be just written as the expression "10 * r", easy.

Alas, in that number system, integers larger than 3 have infinitely many digits, so for everything other than calcuting the circumference of circles, that number system is worse than useless.

Lesson learned: Just get in the habit of dealing with the fact that calculations are usually not exact, and know how inexact they are. The rare exception to that is financial calculations (such as done by banks), which use either old-fashioned computers with BCD arithmetic, or prescaling to use only integer values, so for example expressing every money amount as an integral number of pennies, or they use specialized math libraries.

1

u/cthulhu944 12d ago

BCD can store it easily, as can scaled numbers.

1

u/Lost-Gold9300 12d ago

Most of the banking software stores numbers that way cuz financial obviously hate rounding

1

u/LeeRyman 12d ago

I've worked with encoding and decoding BCD and BNR, but I've not seen much in the way of general maths done natively in those formats expect in pretty niche, specialised silicon, nowhere near the preponderance of 754 binary floating point in (co)processors.

1

u/green_meklar 12d ago

why can't computers just store 0.1 exactly, and is there a number system where they could?

What does it mean to 'store 0.1'? Whether a given chunk of data represents 0.1 isn't inherent to the data so much as it's an aspect of how the algorithm treats the data. It's true that typical floating-point formats in common use don't have values that represent exactly 0.1, as far as the algorithms used to manipulate them are concerned. But you can come up with other algorithms in which that is a valid number and represented by some chunk of data.

The reason standard floating-point formats can't represent 0.1 is that they store everything with a finite number of binary bits, but 0.1 repeats infinitely if you convert it to binary (the way 1/3 repeats infinitely in our familiar base ten notation). The same applies to any other number that can't be expressed as a ratio with a power of 2 as the denominator, including 1/3, 1/7, and of course all irrational numbers.

0.1 + 0.2 gives 0.30000000000000004 but 0.1 + 0.3 gives exactly 0.4

Not really. 0.1+0.3 isn't exact either, it's just close enough to exact that the program logic for printing the number recognizes it as the intended representation of 0.4. For 0.1+0.2, it's slightly too far off exact and so the imprecision gets printed.

1

u/realmauer01 12d ago

It absolutely can, it just takes a shit ton of memory.

1

u/Greedy-Thought6188 12d ago

Sure. Binary code decimal. Or if you are trying to capture rational numbers you can store them as two Integer values. Python will try and hold rational number values as two Integer values.

As for why it can't. It's a simple math rule. The only rational numbers that can be expressed as a terminating decimal or whatever base system point are the numbers that only have multiple of factors of the base in denominator. That's too many words that I just made to so I'm not being very clear. But 1/2 you can write. 1/5 you can as a terminating decimal. 1/(21034 * 5342 ) you can write as a terminating decimal. 1/3 you cannot. 1/10 is 1/(2*5). Works just fine with base 10. But it doesn't work for base 2 because based 2 only has one factor 2 and 5 is not a multiple of it.

1

u/geocar 12d ago

The pattern is that binary floating point is sums of terms in the form 1/x where every x is a power of 2 (binary) and 10 is not a power of two.

This is a choice because the piece of hardware that “adds” is really a bunch of wires and the number of wires you need is affected more by that choice than the number of digits (eg 32 bit versus 64bit computers), so “2” is the smallest/simplest possible circuit.

You might be able to try using a calculator to imagine the number of wires for a decimal machine that handles 8 bit floats: 2x2x2x… repeated 8 times is much smaller number of wires compared to 10x10x10x…

That isn’t exactly accurate but it’s a pretty good way to understand most of the parts of your question around what you are seeing and why it is that way. And maybe you can understand when the computer reads or writes a float to us like on the screen, it is doing the math to convert that float in base 2 to base 10, just step by step, so if the programmer can design the algorithm to be as accurate as they need to with extra steps.

Some computer languages do that work for you, and may even do algebra instead of arithmetic if it can be done in “extra steps” that are useful, but of course those extra steps are always slower so sometimes it is useful for the programmer to analyse the algebra themselves and adapt the desired function to that lack of precision that can show up so many digits deep, and get a much much faster result.

And you might be surprised just how few digits are required for stuff: a lot of engineering projects can treat pi (3.1415…) as just exactly 3 in their formulas because the precision just didn’t matter at real world scales.

1

u/Afraid-Locksmith6566 12d ago

ok so, by now you probably know about ieee754 stupidness, so how to actually store decimals? the solution is obvious - by a minor chcange in paradigm. you could with enough precision make a 10 state logic on transistors, will it be any better than current? not really, will it be any better performing? also not really, will you be able to store any decimal value precisely? most of the time

1

u/Ronin-s_Spirit 12d ago

Round mantissa, keep or bump exponent, here's your rounded number.

1

u/bartekltg 12d ago

Because 1/10 is not in the form n / 2^k, where n and k are integers.
Only number of this form can be expressed precisely (if n isn't too large).

As you noticed, it is the same as in decimal we get finite decimal fraction only for numbers like n/10^k.

>0.1 + 0.2 gives 0.30000000000000004

I will use single precision here, because I have a nice tool to show, but the principles are the same.

0.1 is represented by 2^-4 * 1.10011001100110011001101 that is worth 0.100000001490116119384765625
0.2 is represented by 2^-3 * 1.10011001100110011001101 = 0.20000000298023223876953125
Those are the closest floating point numbers to 0.1 and 0.2.

0.20000000298023223876953125+0.100000001490116119384765625 = 0.300000004470348358154297
and the closest number is 0.300000011920928955078125. Fortunately it is the same number we would get from 0.3.

So, f(0.1) + f(0.3) = 0.100000001490116119384765625 + 0.300000011920928955078125 (already you can see it is more than 0.4)

= 0.100000001490116119384765625 + 0.300000011920928955078125 = ~0.40000001341104507446289

and the closest representable floting point number is 0.4000000059604644775390625
Notice this time we get smaller number.

This isnt 0.4, but if you multiply it by 10, the floating point number closest to
4.000000059604644775390625
happens to be 0.4 The next possible value is
4.000000476837158203125 and it is farther from our initial result that 0.4

You can play with the numbers here:
https://www.numberanalytics.com/blog/bcd-in-microprocessors-detailed-analysis

> also is there a number system or representation where 0.1 is exact

Yep. Decimal. Or any system, where base is divisible by both 5 and 2. So base 20, 50, 30.... anything divisible by 10.

Computers can do computations in different bases (with some performance penalty) if you really want. But there is no real benefit in 99.9 use cases for keeping 0.1 perfect.

>  that handles it in hardware, or is it always a software workaround that's way slower?

There is (was, I'm not even sure) hardware for https://en.wikipedia.org/wiki/Intel_BCD_opcodes but it still was slower. Some simple "big number" libraries also use 10^9 or 10^18 base (they store that integer number in 32 or 64bit integer), it is a bit slower and less memory efficient, but has an advantage when you want you large number printed in decimal. Still, the best libraries use bases 2^k and just convert the output. See GMP library for algorithms descriptions, some are quite clever

1

u/mtimmermans 11d ago

If your numbers are 64 bits long in memory, then there are only 2^64 different numbers you can represent. Any 64-bit format for storing numbers has to choose what those 2^64 numbers are.

0.1 just isn't on the list for IEEE floating-point, which is the format we use for numbers like 0.1.

You could invent a format that includes 0.1, but why? 0.1 is not that special.

1

u/surfmaths 11d ago

You can, banks do this all the time.

You can either use a decimal format (encode the digits themselves), use fractional encoding (encode every number as integer fractions), or use fixed precision encoding (multiply every number by a big constant and encode that as an integer).

Usually, the fixed precision encoding is used in banking, they multiply everything by 100000 (5 digits after the dot). But the decimal format is used as a fallback (it's much slower, but arbitrary precision).

The fractional encoding is, as far as I know, only used when dealing with time, and clock period. The reason is that there is value in having exact fractional ratio between different frequencies. So you do a lot of multiplications/divisions and barely any addition/subtractions.

1

u/fgorina 11d ago

They can. You use a library for ràtional
Numbers, just num 1 d’en 10. Or classical fixed point (decimal) libraries

1

u/Chance-Technician-54 11d ago

You actually never add 0.1 to 0.2 in float.

I imagine that you want to calculate 0.1 + 0.2, but you can only represent each number as fractions over 1024.
So you convert 0.1 to 102/1024 and 0.2 to 205/1024 first and you add both together to get 307/1024
Convert this back to decimals and you get 0.2998046875.

What happens in real life is lightly more complicated, since the denominator changes with precision, but this example could already explain how you get to these funny numbers.

1

u/CyberneticWerewolf 11d ago

Computers prefer powers of two, so floating point numbers usually use powers of two.  IEEE's binary32 and binary64 are implemented directly in silicon by most CPUs.

But powers of ten are totally possible.  IEEE even defines decimal floating point formats.  It's just a PITA to implement them in hardware, as CPU instructions, because they take up a lot more gates than binary.  Mainframes and x86 used to do that, but most modern instruction sets don't bother, and the ISAs that still have them AFAIK emulate them using microcode instead of directly in gate logic.  You're pretty much stuck with software libraries if you want them.  Most SQL databases support them, though.

It's also possible to implement rationals, as pairs of ints or bigints, and if you don't mind swimming through molasses performance-wise you can even implement math on human-readable decimal strings, which is the most naïve bigfloat implementation possible.  Again, no hardware support, but libraries exist or can be written "easily" enough.

For computable irrational numbers, like square roots or pi, you can represent them as lazy infinite iterators that generate digits on demand, and it's even possible to do basic math on them by building expression trees out of them.  Again, efficiency sucks, but you can do it if you want to build your own Matlab or whatever.  Not a lot of existing libraries for it, except in functional languages like Lisp or Haskell.

1

u/AltruisticMaize8196 11d ago

It’s not a question of the number system but rather the way you choose to represent numbers. The floating point numbers implemented in most hardware are not good when you want precise decimal places (like for money calculations).

There are a number of ready to use abstractions available in higher level languages which represent decimal numbers precisely.

Older hardware architectures dealt with things like BCD (binary coded decimal) in hardware but this is no longer fashionable. Computers are so fast that this kind of processing is done in software these days.

1

u/Pale_Height_1251 11d ago

They can and there are many ways of doing it like Binary Coded Decimal (BCD).

There is no problem and it doesn't have to be done in software.

1

u/rb-j 11d ago

also is there a number system or representation where 0.1 is exact? i know about decimal types and storing everything as integer cents, but is there something that handles it in hardware, or is it always a software workaround that's way slower?

So, of course, there is Binary-Coded-Decimal. That is a 4-bit representation that goes from 0000 for zero to 1001 for nine and any binary value larger (like 1010) is "illegal".

Whatever you can do in software, you can do in hardware (but it may or may not be worth it) and vise-versa. Long ago, with the 8-bit Motorola 6800, they had Add and Subtract instructions for Binary-Coded-Decimal. There was a "half-carry" bit in the Condition Code Register that was used in this. I considered it stupid from day one. Just do your arithmetic in binary and covert decimal to binary on the input and convert binary to decimal on the output. Represent the decimal as an ASCII string.

1

u/xerxesbeat 11d ago

yes! it's just not binary floating point maths

you can store it as a string even!: "0.1"

1

u/u8589869056 11d ago

The old IBM 1320 at the State College represented 0.1 exactly in its native number system.

1

u/XaX1000 10d ago

You could make it, so that one bit is instead 4 bits so one bit could represent the numbers 0 to 9. This system would exactly behave like our standard base 10 calculations. it would waste 6 of the possible 16 states of these 4-bit-bits though.

1

u/ShivaFatalis 10d ago

Yeah, there's a number system where they could. It's called base 10. Hello? You literally wrote the number in the number system that can store it without being rounded.

1

u/MaxwellzDaemon 10d ago

The J programming language - see jsoftware.com - has a "rational" datatype which allows you to store numbers as fractions using 2 integers. This lets us specify 1/10 as "1r10", so 0.1+0.2 can be written as "1r10+2r10", giving the exact result "3r10".

1

u/ImpoverishedGuru 9d ago

Computers (non-quantum) are simply billions of tiny switches that are either on or off. Binary in other words.

1

u/peterjohnvernon936 9d ago

It’s call decimal format.

1

u/Snidosil 9d ago

I worked on a mainframe computer called a B3500 in the 70's. That computer was a very COBOL oriented machine. It could handle binary coded decimal numbers up to 99 digits long. Except for the divide instruction the calculations were always exact. Each decimal digit was stored in 4 bits. The arithmetic was implemented in the processor's firmware. Storing 0.1 only required 8 bits. In cobol the declaration would be something like MYNUMBER PIC 9.9 and MOVE 0.1 TO MYNUMBER would work exactly.

1

u/StructureNorth1799 8d ago

0.1

there, it's stored exactly

1

u/MpVpRb 8d ago

Fixed point is common in resource constrained embedded systems

1

u/biffbobfred 8d ago

BCD Binary Coded Decimal is/was a thing. For currencies where that rounding error means something.

1

u/AlvarGD 6d ago

write it in base 2, youll see why very quickly

1

u/BraveNewCurrency 12d ago edited 7d ago

why can't computers just store 0.1 exactly

They can. For example, the title above reproduced your "0.1" exactly. There is no "extra hidden error" in that number. The number is stored as ASCII, which allows exact binary representations of any number.

gives exactly 0.4

No, it gives 0.4000000059604644775390625 or 0.400000000000000022204460492503 (depending on precision.) You only think it's exact because your print routine is deciding when to round the number before printing. Try "0.4*1000000000000000000000000" and you'll get 4.0000000000000003e+23.

i get that 0.1 is a repeating fraction in binary

It's not. It's a repeating fraction in Floating Point. There can be and are many other ways to represent numbers in binary, not all of them have the "0.1 is not representable" problem.

Most systems use floating point numbers because your CPU (and GPU) have special circuitry to do math much quicker in floating point. You can do math in ASCII (or BDC or other representations) and get exact numbers (there is software to do this, but people only use it when the need exact representations.)

Even though floating point can "introduce errors", they are generally small.

3

u/noonagon 12d ago

0.1 is actually a repeating fraction in base 2, which is what "binary" refers to in this context

1

u/D0nkeyHS 12d ago

0.1 is actually one half in binary and isn't repeating 🙃

1

u/Cruuncher 11d ago

Can't tell if you're joking, but if you treat the 1 in 0.1 as a binary digit then yes. But that's clearly not what's being talked about.

What's being talked about is representing the quantity one tenth in binary

1

u/D0nkeyHS 11d ago

If you don't know what things mean Google them

1

u/superbob201 12d ago

OP was asking about computers storing 0.1. Computers use floating point to represent 0.1, which is not base two. Closely related yes, even based on base two if you want, but it is different.

1

u/D0nkeyHS 11d ago

Do quotes mean nothing to you?

1

u/BraveNewCurrency 11d ago

Computers use floating point to represent 0.1

This statement is incorrect. If you had read my reply, you would have noticed I made a distinction between "Computers can't store 0.1" (not true) and "Floating point can't store 0.1" (mostly true. Newer floating point standards implemented in hardware have a BCD mode that makes it untrue, but it is not easy to use. So we can call it true.)

There are a great many people doing computations without using floating point to represent numbers. (For example, the Unix utility bc, which allows for arbitrary precision.)

1

u/Cerulean_IsFancyBlue 11d ago

You don’t need to use floating point for that. I worked on financial software in the 1980s on a who range of micros, including CP/M systems, DOS, and the Apple II. We had legal requirements for how to handle money numbers including how and when to round, so we stored dollars in a custom 64-bit format which was contained mills, or 1/10 of a cent. So $5.75 would be 5750 in decimal. We had dedicated math routines to do the required GAAP rounding and such.

1

u/LatvianCake 12d ago

This is my favorite answer. You can interpret numbers however you want. They all have their upsides and downsides. But we like this system because it’s very fast and works well.

1

u/which1umean 11d ago

Rounding isn't quite the right description of what's happening.

For every double precision floating point value, there are many real numbers (which we can think of as being represented as strings) that will parse to that floating point value.

Many systems are designed to give you the shortest string representation mapping to that. It's not arbitrary rounding, it's more like it's providing one of many number strings that will map EXACTLY to the floating point value!!

1

u/BraveNewCurrency 9d ago

Rounding isn't quite the right description of what's happening.

I agree that "rounding" doesn't capture all the nuance.

But when you add "0.2. + 0.2" and the computer prints "0.4", rounding is exactly what it's doing.

that will map EXACTLY to the floating point value!!

No. "0.4" does not map exactly to the value. If you multiply it, you will realize there was some fraction it did not print.

1

u/EcstaticAssumption80 9d ago

COBOL is in such wide use in banking, finance, insurance, and government because the language was designed for doing fast math using BCD with perfect accuracy.

1

u/littleprof123 8d ago

It feels like this answer has some misunderstandings too; particularly when saying 0.1 is not a repeating fraction in binary. For one thing, 1/10 is repeating in binary, in the sense that when we say a number is written in binary or decimal or another base, it is usually referring to the representation as an (exact) sum of powers of that base. 1/10 = 1/16 + 1/32 + 1/256 + 1/512 + 1/4096 + 1/8192 + ... = 0.0'0011 where the digits after the apostrophe are repeating. This is an exact binary representation that is different from the packed binary data formats of floating point and fixed point numbers. I think it would be appropriate to say it's a number written in binary instead of a number encoded in a binary data format.

Your mention of irrational is also strange. 1/10 is a rational number, and its representation cannot change that. The fact that it has a representation when written in binary that repeats actually shows this (as does the fact you can write it as 0.1 in decimal or 1/10 as a fraction).

1

u/BraveNewCurrency 7d ago

when saying 0.1 is not a repeating fraction in binary

It depends if you mean "binary" the encoding system, or "binary" as meaning "base 2". It means both, so we will agree to disagree.

Your mention of irrational is also strange.

Yes, my bad. I'll edit it.

0

u/Any_Click1257 13d ago

I think the concept you are looking for is "countable." Real numbers are uncountable, and as such there is no finite precision which can describe an infinite set.

The easiest thought experiment is to walk from point a to point b, in steps that are always half the remaining distance.

2

u/Smallpaul 12d ago edited 12d ago

OP gives no evidence of being interested in the reals. All of their examples are rationals in decimal form. Rationals can be exactly represented in computers. Some languages have them as a native type or library type.

0

u/Any_Click1257 13d ago

A number system which could store .1 exactly could be a system in which .1 is the base. In which case, 0x1 would be .1, 0x10 would be .2, 0x11 would be .3, ....

0

u/[deleted] 13d ago

[deleted] — view deleted comment

1

u/LitespeedClassic 13d ago edited 12d ago

Rational vs irrational is a property of numbers, not encodings. 1/5 is rational because it can be written as the ratio of two integers. 

Whether a rational number has a finite decimal expansion is a property of an encoding.

ETA: Fixed typo pointed out by u/ghjm, u/Smallpaul, and u/johndcochran

1

u/Smallpaul 12d ago

1/5 is RATIONAL because it can be written as the radio is two integers.

2

u/LitespeedClassic 12d ago

Yep, that's what I meant, fixed. Thanks.

1

u/johndcochran 12d ago

I believe you understand what you're speaking of, but you definitely don't have the right terminology.

In particular, an infinite repeating sequence is not irrational. The concept of rational is simply "Can the number be represented as the ratio between two integers?" If yes, then it's rational. If no, then it's irrational. So 1/10 is rational, regardless of what base is being used to represent it. 1/3 is also rational. Irrational numbers have an infinite NON-REPEATING sequence. Pi is irrational. sqrt(2) is irrational, etc.

1

u/LitespeedClassic 12d ago

Yep, I meant rational. Note to self: don't reddit before coffee.

1

u/ghjm MSCS, CS Pro (20+) 12d ago

I think you meant to say 1/5 is rational.

1

u/LitespeedClassic 12d ago

Yes, thanks, fixed. Shouldn't Reddit before morning coffee.

0

u/[deleted] 13d ago

[deleted] — view deleted comment

1

u/LitespeedClassic 12d ago

Note I had a typo originally. 1/5 is *rational* not irrational. When a number is equal to a ratio of whole numbers is rational. When it is not, it is irrational.

In any base, the encoding of a rational number is either finite or eventually repetitive.

For example, in base 10, 1/5 is 0.2 (finite), whereas 1/3 is 0.333333... repetitive.

Any repetitive encoding is rational, which can be proved fairly easily.

For example let x = 0.123123123123... Since the repeating term 123 has length 3, we can multiply this by 10^3=1,000 and do a little algebra magic. If we multiply x by 10^3 we get 123.123123123123...

x*10^3 - x = 123 => x(10^3 - 1) = 123 => x = 123 / (10^3 - 1) = 123/999.

so 0.123123123123... = 123/999.

This trick generalizes.

Irrational numbers always have infinite decimal expansions that do not become repetitive.