Log In  

0

i've discovered an issue with sqrt(), and it's not about overflow.

since pico-8 uses fixed-point, i don't expect sqrt() to be exact, but this behaviour is odd enough that it messes up some simple comparisons.

try the following:

> print(sqrt(3.75))
1.977
> print(sqrt(3.8))
1.9917
> print(sqrt(3.85))
2.0063
> print(sqrt(3.9))
2.0209
> print(sqrt(3.95))
2.0355
> print(sqrt(4))
2
> print(sqrt(4.05))
2.0125
> print(sqrt(4.1))
2.0249

you'd expect the results to keep increasing, but as you can see, right around 3.85, the result is greater than 2. at sqrt(4), it goes back to 2 as one would expect. it's a weird discontinuity and breaks some of my code.

the following program shows the discontinuity visually in a graph:

cls()
for x=1,128 do
  y=sqrt(x/16)
  pset(x-1, 127-y*32, 7)
end

this is in pico-8 0.1.11g

thanks!

sqrt math
P#49194 2018-02-11 23:55 ( Edited 2018-02-13 17:48)

::

Confirmed on my Windows build:

> ?sqrt(3.85)
2.0063
> _

Not sure about other platforms.

Wonder if it's a lua-based sqrt() implementation and thus suffers from doing its work in fixed-point. That would be... a strange choice. I'd have expected sqrt() just to convert the result of the C lib version.

@zep?

P#49205 2018-02-12 04:57 ( Edited 2018-02-12 04:59)

::

Just noticed there's a discontinuity at 0.25 as well:

> ?sqrt(0.24)
0.4899
> ?sqrt(0.25)
0.5
> ?sqrt(0.26)
0.4987
> _

If anything, it's like the entire range of 0.25..2.0 is shifted up on the graph.

P#49206 2018-02-12 05:22 ( Edited 2018-02-12 05:23)

::

Just noticed there's a discontinuity at 0.25 as well:

> ?sqrt(0.24)
0.4899
> ?sqrt(0.25)
0.5
> ?sqrt(0.26)
0.4987
> _

Edit: Another at 9... maybe it's some kind of piecewise solution that's not calculating very well with fixed-point, so the endpoints of each section aren't matching up as they should.

Edit 2: Yeah, another at 16. Looks like it's in sections located between the squares of certain numbers, like 0.5, 1, 2, 3, 4, etc.

P#49207 2018-02-12 05:29 ( Edited 2018-02-12 05:31)

::

Sometimes I get too involved in very mundane subjects and spend way too much time writing code no one really wants.

Cart [#49227#] | Copy | Code | 2018-02-12 | Link
0

The numbers are the left and right side x,y values, in hex.

Noteable (hex) values of X:
00.40 (1/4)
04.00
09.00
10.00 (16)
19.00 (25)
24.00 (36)

P#49224 2018-02-12 15:45 ( Edited 2018-02-12 16:13)

::

ok that is too cool!

P#49257 2018-02-13 17:48

Log in to post a comment

user:
password:

New User | Account Help
:: New User
X
About | Contact | Updates | Terms of Use
Follow Lexaloffle:        
Generated 2018-02-18 01:08 | 0.244s | 1835k | Q:23