Log In  

as a lot of us are brutally aware of, pico-8's limitations are very fun, but can also have a serious impact on code readability. one particularly nasty problem is that when saving tokens, descriptive names for constant values are one of the first things to go out the window. Up till now, this was an unavoidable problem. However, with the recent introduction of the INCLUDE directive, pico-8 now has a preprocessor and a model for how preprocessor directives should work. I'd like to propose a DEFINE directive for supporting simple text substitution.

for example, let's say we are doing a simple branch based on a cel value:

 local c=mget(x,y)
 if c==4 then
 elseif c==5 then

it's impossible to even guess what this code does. but it's hard to argue that adding "local health_pickup=4" is worth the tokens.

instead, if we had a DEFINE directive, we could change the above to:

 #DEFINE health_pickup 4
 #DEFINE coin_pickup 5
 local c=mget(x,y)
 if c==health_pickup then
 elseif c==coin_pickup then

this would be flattened to be identical to the first code snippet when exported to p8.png. the result is far more readable, and uses no extra tokens. I think DEFINE directives should be able to be placed anywhere, to enhance readability, but I don't believe they need to be scoped, or nestable for that matter. since it'd be possible for a macro name to shadow a variable name, I'd suggest syntax highlighting, changing the text of a macro usage to orange, which would make accidentally using a macro when you meant to use a variable much rarer.

what do y'all think? is this a feature you would like to see in pico-8? I'm trying to get a discussion about this going, so ideas and feedback would be very welcome!

P#73423 2020-02-24 19:29

If the #define is flattened when exporting to .p8.png then it's not much different from having your personal minifier/obfuscator that could do a lot more than that, is it?

With PICO-8 doing code compression anyway, I don't think there's much usefulness to that feature, to be honest.

P#73425 2020-02-24 19:52

@sol I buy the ‘reverse engineering’ use case a lot.
@samhocevar ‘persona minifier’?? I don’t have that and not sure anyone want to spend time maintaining a lex/yacc/regex parser just for defines!

P#73426 2020-02-24 20:09
:: dredds

You don’t need to write your own with lex & yacc.

You can use m4 on Mac and Linux.

Bad luck on Windows :-(

P#73428 2020-02-24 20:32
:: sparr

@samhocevar a "personal" anything means you can't use the feature in code you share on the BBS. Features like this should be available to everyone.

P#73430 2020-02-24 21:04
:: sol

@samhocevar while it is possible to use your own minifier, by that logic INCLUDE is also redundant. it is of course not necessary, but such a feature would simply make working within the pico 8 environment itself less painful and more fun. there are already p8 projects of mine I don't relish the thought of returning to simply because I used so many magic numbers.

it is true that it's a shame that these DEFINEs wouldn't appear on the BBS code, but the only other way I can see that would solve this kind of problem is introducing a token-free CONST declaration at the lua level, which I think would be a lot more work and lead pico 8 farther away from being pure lua than most people would be comfortable with. and of course, even then you'd hit the compression limit and be back to removing descriptive names, which is basically an intractable limit of the p8.png format.

P#73431 2020-02-24 21:08

@freds72 handling #defines is trivial, there is nothing to maintain: "cpp -P cart.p8 > newcart.p8", that’s all… the point of a custom minifier is that you don’t use it just for defines, and plenty of people have posted their minifiers here.

@sparr I don’t understand; personal doesn’t mean secret or private. I was making the point that collapsing #defines when saving to .p8.png, as suggested by OP, did obfuscate the code.

P#73434 2020-02-24 22:01

Lua's actually getting constants in 5.4.

local minint <const> = math.mininteger
local maxint <const> = math.maxinteger

local intbits <const> = math.floor(math.log(maxint, 2) + 0.5) + 1
assert((1 << intbits) == 0)

assert(minint == 1 << (intbits - 1))
assert(maxint == minint - 1)

... But that's even more text and tokens. And I had to manually escape the angle brackets, so that's kind of an issue.

P#73438 2020-02-24 22:26
:: sol

@Saffith that's actually super interesting, I didn't know lua was getting constants!

as an aside I suppose I wasn't clear but what I meant by "a CONST declaration" was actually just a type of variable declaration that ONLY accepts string/number literals and cannot be mutated after the fact. "const numberFour=2+2" would NOT be valid, you'd need a single literal, but you wouldn't get a token penalty for it.

but yeah that's not a great idea imo, and the fact that lua is gonna be reserving const as a keyword is another reason why this problem is probably best not solved at the language level. hence, why I think preprocessor directives are a perfect fit.

P#73439 2020-02-24 22:37
:: Felice

God, that syntax though. Why the angle brackets? Seems like a modifier like 'local' would have done fine.

const x = 12
local const y = 34

Just like local, it would imply that this is a new instance of a (non-)variable with that name and do inline substitutions thenceforth.

P#73445 2020-02-25 02:22 ( Edited 2020-02-25 02:25)

just realized that #define won’t indeed survive publication to the bbs.
it then remains a ‘local dev’ comfort thing :/

P#73447 2020-02-25 09:55
:: sol

@freds72 yeah it's unfortunate. I don't see a way around it though, if somehow we were to introduce something similar at the language level, people would start hitting the compression limit and return to using magic numbers.

.p8 files enforce the token limit, but not the compression limit, which sort of suggests a distinct role from .p8.png. I think DEFINE fits here like a lego brick and gives .p8 more of its own identity-- INCLUDE already exists as precedent for this, anyway.

I did think of one important change to my proposed syntax, though. it would probably make sense to add an equals sign, i.e.

#define my_pet_clowns_shoe_size=17

that way if it comes time to publish and there are extra tokens left over, a user could simply delete all the "#DEFINE"s and in the common case, they'd end up with valid lua code.

P#73451 2020-02-25 13:54
:: Felice

If you're going to change the typical syntax for #define to something a preprocessor wouldn't recognize, you might as well just switch to a const keyword and handle it with the special code you'd need for #define v=n anyway. It'd look a lot more like lua and be more intuitive.

P#73464 2020-02-25 23:08 ( Edited 2020-02-25 23:09)
:: sol

@Felice I'm confused, are you suggesting that adding the equals sign to the #define syntax is bad because third party preprocessors wouldn't recognize it?

P#73465 2020-02-26 00:52

I would very much like exactly that kind of #DEFINE option - my immediate instinct as a new PICO-8 programmer has been to waste tokens on exactly this kind of thing out of a desire to make things readable, and an explicit built-in way to not do that would be very welcome.

P#73493 2020-02-27 01:06
:: Felice


Yes, that's what I'm saying.

Preprocessors like the one for C/C++ expect "#define my_pet_clowns_shoe_size 17" with no equals sign. If you're going either to rewrite the preprocessor or to write a new preprocessor to look for "#define my_pet_clowns_shoe_size=17", you might as well make it more lua-like and have it look for "const my_pet_clowns_shoe_size=17".

P#73499 2020-02-27 11:07 ( Edited 2020-02-27 11:17)
:: sol

@Felice I’m not sure why we’re worried about what third party preprocessors would do. you’re either gonna use a third party preprocessors or the builtin one, I don’t think compatability is a use case?

@packbat yeah lol I totally get this. part of the reason I want this is bc I’ve been using pico-8 for too long and I’ve gotten too used to using magic numbers. I need to get back in the habit of using descriptive names, but I’m not quite ready to move on from pico-8 yet, either.

P#73513 2020-02-27 18:38 ( Edited 2020-02-27 18:39)

I feel like adding a #DEFINE would be bloody brilliant. Heck, even if you were to throw the readability of the code out the bloody window, this feature is great for learners. Like let's say someone's made a pretty cool rougelike but all the function names are "M" or "F" or literally the most ambiguous letter of the alphabet or in the worst case a symbol. This then discourages said learner who doesn't even know which way is up! Let alone how they define the name of a table that looks like a bar code number. I personally would accept this change with open arms. Even if it wasn't necessarily defined through the writing of code, I think there should be a note system were you select a certain section of your code and through some hotkey (say CTRL+ALT+TAB+N for examples sake) you were able to attach a definition note to your code. Nothing fancy, just ease of access. Man alive people, this is Pico-8 were talking about it's supposed to be a comfortable environment, I shouldn't need a bloody bar code scanner to figure out the difference between a move variable, and a draw function.

P#73515 2020-02-27 19:23 ( Edited 2020-02-27 19:24)
:: sol

@Chaotic Squirrel yes! this would be a very useful change. but in the interest of clarity I want to make sure it's understood that due to the way pico-8 flattens preprocessor directives, these #DEFINE statements would not appear in BBS code...

P#73521 2020-02-27 22:08

After giving it some more thought, I believe this proposal is not useful at all and goes completely against the PICO-8 philosophy.

If I understand correctly, the two major pros are: 1. it helps reduce token count; 2. it implements constness.

I believe the number of tokens gained with such a feature is ridiculously low and if one’s a dozen tokens short it’s probably an indication that something else is wrong with the code.

Also, constness of global variables is trivially implemented using metatables; for instance the following code protects every variable that starts with “g_”:

g_speed = 2
g_gravity = 9.81
other_variable = 2

_ENV = setmetatable({}, {
  __index = _ENV,
  __newindex = function(t,k,v) if (sub(k,1,2)=='g_') assert(false,'error: '..k..' is const') end

other_variable = 3  -- ok
print(g_speed)      -- ok
g_gravity = 3       -- runtime error

Finally, I believe that diverging even more from standard Lua is not desirable, as it will confuse users. For instance, Zep’s decision that the “local” keyword would cost zero tokens indicates that he’s aware that using “local” is important both for performance and code quality, and that PICO-8 users should not be penalised for using it.

P#73524 2020-02-27 23:17

@samhocevar I think it might be a feature one CAN use not HAS TO use.In some cases it might just be good to use this as a clarity tool because the other half of Pico-8 is learning, and if no one knows what the heck your talking about then what are you learning? That arbitrary code is good code? That sounds anti-Pico-8 in some ways. The main one being that it is making your environment confusing not cozy (As Mr. Joseph White puts it).

P#73525 2020-02-28 00:34 ( Edited 2020-02-28 00:35)
:: sol

@samhocevar constness is not the goal here, the reason we are talking about constness is bc I mentioned that a hypothetical lua-level alternative to DEFINE would be a token-free construct that is rigid enough that it doesn't enable the user to skirt the existing token limit.

so yes, it's all about improving readability and using fewer tokens, (though advanced users might find neat ways to use DEFINE I'm sure). I'm glad you haven't bumped into this issue but there are a lot of games that come up a few dozen tokens short, for which this feature would be extremely useful

P#73531 2020-02-28 03:10
:: Felice


I disagree strongly that it goes against the philosophy.

PICO-8 is semi-emulating the era of 8-bit computing and gaming. One of the things it does to make it moreso the case is to allow us to do low-level trickery with hardware registers if we like that kind of thing, and usually once you move to that level on an old 8-bit system, you start meddling with asm, and even then assemblers would provide functionality for constants, so you could name the registers and memory ranges.

We don't have access to the, uh, token-asm, but that just means we do peek/poke instead of LDA/STA. So if nothing else, it'd be very nice to define register names, e.g. p8pencol for 0x5f25/24357: "savecol=peek(p8pencol)" is much easier to write and more intuitive to read than "savecol=peek(0x5f25)".

But nobody wants to spend tokens on defining a nice, easy-to-use, easy-to-read constant, especially when there are dozens of them in the case of register names alone. Thus, a very valid reason for the preprocessor request.

P#73530 2020-02-28 03:28 ( Edited 2020-02-28 04:33)

> there are a lot of games that come up a few dozen tokens short, for which this feature would be extremely useful

@sol how many is a lot? I have analysed the code of more than two thousand carts and found no evidence of that claim.

P#73534 2020-02-28 16:01
:: Felice

I think everyone runs up against the token limit when they try to do something ambitious. Even I've written an app big enough to tickle the limit, and I'm super-conscious of token usage.

Remember that you're only seeing the carts that came in under the limit. There are a lot more out there, stuck in local .p8 mode because they're over the limit and can't be posted to the BBS.

P#73541 2020-02-28 16:11 ( Edited 2020-02-28 16:23)
:: Felice

I will say, by the way, that I think it's unlikely zep will add anything like this. But that doesn't mean I don't think it'd be a good idea. Those two thoughts can exist simultaneously.

P#73542 2020-02-28 16:16 ( Edited 2020-02-28 16:17)

@samhocevar - hitting token limit? all my
games! (Attack on Deathstar, Nuklear Klone, Tiny Sim, Snow...)

P#73546 2020-02-28 17:03
:: sol

@Felice ”I will say, by the way, that I think it's unlikely zep will add anything like this.”

well, don’t jinx it! xD
personally, I think it’s more unlikely that zep wouldn’t ever add another directive, seeing as he already went to the trouble of writing the preprocessor. though I have no idea if DEFINE would be one of those additions! but honestly, it prolly doesn’t make sense to speculate on what zep would or wouldn’t do, as that’s ultimately up to zep

P#73545 2020-02-28 17:10
:: Felice


Sorry to be a downer there. I'd love to see a feature for constants. I was more intending to keep people's expectations realistic, given zep's apparent keenness to lock down the feature set soon. I'm always happy to be proven wrong though. :)

P#73558 2020-02-28 23:24

@freds72 all your carts use a custom minifier with variable renaming, so I guess a #define feature would not be terribly useful to them, would it?

P#73564 2020-02-29 11:42

only NK had full blown minifier - all other carts is just stripping off tabs & comments (to get under compression ratio). Eg I don’t need a pre-processor to run the cart during dev.
That’s why I always provide link to my github repo, and this where #define would be useful.

P#73566 2020-02-29 11:46

[Please log in to post a comment]

Follow Lexaloffle:        
Generated 2020-09-18 16:35 | 0.212s | 2097k | Q:90