Log In  

Hi, i'm new to coding in general, but i can't understand why some of the arithmetic operations return not exact value.

printh(1 * 0.1) -- 0.1
printh(-1 * 0.1) -- -0.09998

How do i fix this without rounding? Thanks

P#29782 2016-10-01 04:14 ( Edited 2016-10-01 16:03)

This is a common quirk with floating and fixed point arithmetic (Pico-8 numbers use fixed point) and stems from the fact that certain fractions can not be exactly represented and will instead be rounded to the nearest possible value.

In Pico-8, a number is stored in 32 bits: 16 bits are used to represent the whole number (from -32768 to 32767) and the remaining 16 are used to represent the fraction as a fraction of 65536 (from 0/65536 to 65535/65536). So if the value 32768 is stored in the fractional part of our number, we get 32768/65536 = 0.5.

This should explain why you get these inaccuracies: numbers that can not be represented as a fraction of 65536 will be rounded up or down. Numbers like 0.5 and 0.25 can be exactly represented, but if we take a number like 0.1, the faction that lies the closest is is 6554/65536 = 0.100006103515625. The inaccuracy is negligible most of the time, but you have to be extra careful when doing exact comparisons between these numbers as, say, 0.100006103515625 * 5 = 0.500030517578125 is not exactly 0.5 (just like how if you were to round 1/3 to 0.33 and do 0.33 + 0.33 + 0.33 = 0.99, which is not exactly 1).

In short, it's just how arithmetic like this works and it's something you will have to live with. It's usually not a problem but you should be a bit careful when doing exact comparisons.

Edit: I forgot to say, when Pico-8 outputs a number to the screen or console, it's usually rounded to the nearest "nice-looking" number, which is why it might tell you that your number is 0.1 when in actuality it is just slightly more or less than 0.1.

P#29786 2016-10-01 06:32 ( Edited 2016-10-01 10:43)

ohh i see, thanks

P#29792 2016-10-01 10:14 ( Edited 2016-10-01 14:14)

I went through this sometime ago with the team at BlitzMAX. Was getting inaccurate results. Let's see how PICO stacks up to the "test."

n=0
cls()
for i=0,3,.01 do
  print(i)
  n+=1
  if n==19 then
    n=0
    repeat
      flip()
    until btnp(4)
    cls()
  end
end

As you can see, REALLY inaccurate all over the map. In fact the only language I came across that showed any decent accuracy with Real numbers was before Windows time. It was called QBasic 4.5. :)

P#29799 2016-10-01 12:03 ( Edited 2016-10-01 16:04)

[Please log in to post a comment]