Log In  

I'm trying to calculate distance between 2 points and the result is a negative number...?

It only happens for some value combinations and always results in -4

Here's example of it failing. According to my calculator the result should be 233.238076

sqrt((208-408)^2+(320-200)^2))

Anyone else come across this? Known bug somehow?

I thought maybe using abs() would help but gets same result.

P#41870 2017-06-23 00:21 ( Edited 2017-06-23 13:41)

(208-408)^2+(320-200)^2 is equal to 54400. PICO-8 can only represent numbers up to 32767, so the calculation "overflows" and gives a weird result.

One solution is to scale the points down a bit before calculating the distance so the addition doesn't overflow. You can scale everything back up at the end -- something like sqrt(((208-408)/16)^2+((320-200)/16)^2)*16

P#41874 2017-06-23 02:56 ( Edited 2017-06-23 06:56)

You're running into integer limits (32768).

Take the first part of your calculation: (208-408). This equals -200, and that squared is 40,000.

Now, 40k in binary is ‭1001110001000000‬. When you have a 16-bit number like that (and PICO-8 only supports 16-bit numbers), the first bit is used to signify whether the number is negative or not. If it's a 0, it's a positive number, but if it's a 1, it's a negative number.

So when you get a number like ‭1001110001000000‬ (40,000), it looks at that first bit and says, "Oh, negative number. Gotcha." And then it uses the rest of the number to find out just how negative it is.

It actually counts up from the most negative number (-32768), so the rest of your number in binary (1110001000000‬) means it's 7232 up from -32768. Or -25536.

Try it. Have PICO-8 do print(-200*-200) and see what you get. You'll get -25536.

You can also try print(32767), which is ‭0111111111111111‬ and it'll print fine because note the first bit is still 0. But then try print(32768), which is ‭‭1000000000000000‬‬ and it'll print -32768 because the first bit is now a 1. If you try print(32768+1), which is ‭1000000000000001‬ and you'll see it print -32767, because it's counting 1 up from the most negative number of -32768.

Maybe more explanation than you were looking for, but hope that helps.

P#41875 2017-06-23 03:02 ( Edited 2017-06-23 07:02)

You can also early-exit if you just want to find out if something is closer/further than the distance.

local dx,dy = player.x - object.x,player.y - object.y
if abs(dx) < hit_range and abs(dy) < hit_range then
  if dx*dx +dy*dy < hit_range*hit_range then
    -- we have a collision!
  end
end
P#41876 2017-06-23 05:28 ( Edited 2017-06-23 09:28)

Thanks all. Yeah...totally wasn't thinking about the number cap. Foo.

The actual value isn't that important...I'm just trying to find out which object is closest to an AI. Not for collision purposes but as a destination.

Thanks for the tips. The scaling down option sounds like a simple solution, I'll give that a shot.

But also I guess if the return is failing due to the integer limit, that means that object is super far away anyway.

P#41886 2017-06-23 09:41 ( Edited 2017-06-23 13:41)

[Please log in to post a comment]

Follow Lexaloffle:          
Generated 2024-03-28 17:23:25 | 0.006s | Q:14