Log In  

If you multiply a fraction that isn't a power a number greater than 1 the returned result will be incorrect (for example PICO-8 returns 9.9991 from print(0.1*100)). I'm aware that this is a problem inherent to fixed point numbers, however some special cases would be greatly appreciated.

P#68160 2019-09-26 21:05

@Catatafish ah, good point, i was trying to extract digits of a number because tostr() and by extension print() rounds small fractions incorrectly (0.0001 becomes 0 for example) i should probably bring that up in a new report and close this one.

P#68175 2019-09-27 02:31

Oh right, I misunderstood the issue, sorry.
Depending on what you're doing with it you could use the hex version of tostr + regular print to see precise values. For non-debug purposes you'll probably have to roll your own.

P#68176 2019-09-27 02:58

@Catatafish yeah, im just gonna have to roll my own :(

P#68178 2019-09-27 04:07

Can you explain further? It seems part of this conversation got deleted and I'm interested in the details. thanks!

P#68179 2019-09-27 04:47

Yeah, sorry for that too!
It wasn't relevant, so I removed it to avoid derailing the thread (and then wished I hadn't as now it looks weird), but the short version is I went off on a diatribe about floating point.

P#68208 2019-09-28 01:30 ( Edited 2019-09-28 01:31)

[Please log in to post a comment]

Follow Lexaloffle:        
Generated 2022-08-10 03:58:59 | 0.008s | Q:17