Log In  

Hi, I have a trouble with floating point arithmetic.

function _draw()
    cls()
    print( 256 / 65536 * 256 )
    print( 256 / (256*256) )
end

This snipet output "-2" and "32768".
Why this snipet dosn't output "1" and "0.00390625" ?

I try same thing on lua demopage.
https://www.lua.org/cgi-bin/demo
It outputs "0.00390625".

Thanks.

P#61679 2019-02-09 06:59 ( Edited 2019-02-09 07:02)

65536 overflows. The largest number PICO-8 can handle is 32767.99999

P#61684 2019-02-09 12:08

PICO-8 doesn't use the same number representation as regular Lua, so the Lua demo page exhibits different behavior.

Lua's numbers are 64-bit floating-point numbers. However, PICO-8's numbers are fixed-point, with 16 bits before the decimal point (-32767 to 32768), and 16 bits after the decimal points.

P#61693 2019-02-09 18:30

@Xii, @tobiasvl Thanks!!

I understood that limitation.

P#61710 2019-02-10 05:45
1

This thread helped me with a problem I was having: I was using rnd(65536) in a program, and it ran fine in the development environment; but when I exported the app to HTML, I kept getting zero as my random value.
Thanks!

P#69184 2019-10-23 02:50
1

Ah ! This actually answers the question on why my earlier code of counting high numbers failed only in the BBS but not in immediate cart.

https://www.lexaloffle.com/bbs/?pid=68949

And later when I recoded it where no actual high numbers were used, it didn't:

https://www.lexaloffle.com/bbs/?pid=69113

Thanks, WaterSplash ! That was very helpful.

It's probably not a good idea to skirt 32767 or -32767, much like sailing ships reaching the ends of the earth.

There be monsters ...

P#69186 2019-10-23 03:15 ( Edited 2019-10-23 04:58)

[Please log in to post a comment]

Follow Lexaloffle:          
Generated 2024-03-28 18:15:10 | 0.025s | Q:21