Log In  


In the spirit of sharing readable code, it should not cost tokens for things that only serve to make the code more readable. There are exceptions, of course, that can't be helped, where you might choose a short and less-readable algorithm over a more verbose one that does the same thing -- such as what I call the poor-man's ternary (c = a > b and 3 or 2) versus the "if" block (if a > b then c = 3 else c = 2 end).

Anyhow, back to the point, constants are one thing that serve no purpose other than to make reading and debugging easier. Thus, I feel, declaring a constant should cost no tokens in order to encourage the use of them. Referencing a constant, of course, should still cost a token.

Now, as most of you have already shouted out, the primary flaw with this proposal is that Lua does not support constants. Constant declarations would have to be added to the PICO-8 flavor of Lua which, well, is already not pure Lua. Furthermore, Lua was surely not imagined to be limited by tokens... so it seems fair to tweak PICO-8 Lua in this way in response to PICO-8's limitations.

Why?

Well, currently, I have 80 variable declarations (which are really just constants in their function) occupying 240 tokens. I am running out of tokens and want those tokens back from the constants. Thus, I will likely write a preprocessor (or modify @dddaaannn's excellent picotool) to strip out and replace the constants. I am marking my constants like this in preparation for that work:

    --[[const]] c_black = 0
    --[[const]] c_dkblue = 1

... but it would just be much simpler if I didn't have to pull such shenanigans to get my tokens back (which will also have the side effect of making the code I may distribute less readable).

1


I've been putting thought* into a constant inlining feature for picotool, along with build-time literal expression evaluation and dead code elimination. I was expecting to be able to do this, at least in some cases, with a simple version of dataflow analysis, looking for globals that are assigned only once with an immutable value type.

I'm interested to know more about which cases I can't reliably handle with analysis such that comment annotations would be required. AFAIK tools like JSCompiler, which uses comment-based type annotations for JavaScript, mostly use const annotations so that they can reinforce your stated intent by throwing a build error when consts are redefined. That might be nice to have for Lua anyway.

picotool's libraries would be good for implementing just the const inlining part with your comment annotation idea. Crawl the AST, trim and remember const definitions, insert the value nodes where the symbol appears, and report an error when the symbol is an lvalue (the left side of an assignment). A complete solution would have to watch for non-const locals shadowing const globals, and it wouldn't work for mutable value types (tables) that can be modified and aliased. But an incomplete solution could still be useful.

  • fwiw I feel weird mentioning unimplemented future picotool plans. I only intend them as hypotheticals for the sake of discussion. It's just a roadmap right now, waiting for my next chunk of free time. :)

Yes please! This would be be perfect. And it makes sense it terms of Pico-8's limitations.

I'd love to see this in Pico-8.

The only thing I came up with so far was to deserialize a string so to speak. So you can just put an object in a string and it would deserialize that in that very object without costing a ton of tokens, has anyone done that already btw?


Hi dddaaannn, I agree that comment annotation shouldn't be required with the right tools. I'm just going the comment route for now because it is the easiest to script for. I've long considered the idea that the P8 author could give free tokens to simple variable assignments that never see a second assignment. It sounds simpler, however, for the P8 author to add "const" or "constant" to the language than to analyze it out. In the meantime, though, we do what we can to tool around it.

I like your literal expression idea too. Not having to simplify all math would help with self-documentation. Currently, I account for that by expanding the math in a comment next to the simplified value. Not a huge deal to me but an interesting idea to not have to do that.

I can't think of much where auto analysis might get you in trouble. Perhaps global assignments inside of loops? But if you're only considering immutable assignments as constant candidates, then you probably have that covered. That makes me wonder how other languages treat constants. That is, if any allow assignments to mutables by making a copy of the value at time of assignment. I can't say I've paid that much attention to that nuance over the years. Anyhow, the only other thing I can think of is that the programmer may not want a variable inlined because they are not done working with it yet. But, then, I expect you would only inline on certain commands, such as minification, and not formatting, in which case, who cares? Perhaps they want to balance token use with readability upon distribution, at which point, they "might" care if it is ongoing work. Probably best left as an option: only look for constant tags, or squeeze as much as possible.


Just as a side note, I cloned your picotools repo and I've been working with it for a few days now. I ran out of chars long before tokens and now need a minifier to put in a build process to keep working (have yet to figure out that whole process regarding updates to gfx, sfx, etc). I was looking at another Lua minifier before I found yours. Didn't try it, though, because a P8-specific one sounded more hopeful. Anyhow, I've updated to the latest P8 Lua and even have the glyphs working (like I want at least), among other tweaks to get luamin fully functional with Python 3.4. I'm about to start the constants scrub -- although I'm sure I'm going to do it the simple way versus the right way -- I don't know how your engine works well enough to do it the right way. I must say, you write beautiful code. It is a work of art and I thank you for taking the time do it. I have only damaged it (because I'm only focused on getting luamin to work) but I imagine I might send you a pull request for you to keep what you want and discard the rest. You might find some of it valuable (like the edge problems of minifying arithmetic assignments, ugh, that might be a P8 bug; hard to tell).


Great! Thanks juanitogan. Let me know if there are questions I can answer. I'd be happy to look at your pull request and maybe pick and choose some improvements. :)


I've finished adding constants processing to luamin and it is working great for what I need. I've reclaimed badly needed tokens and char space. I'll get to that pull request after I'm done fiddling with the whole build process. More on that later maybe but it's looking like high-capacity dev work will require building across three p8 files: game_lua.p8, game_lua_fmt.p8, and game.p8.

Anyhow, during this, I noticed a flaw in my proposal for token-free constants. That is, that only constants with only one token on the right side of the assignment should be free. Otherwise, you could do something like store a large table for free. (Not sure if that is what Oli414 was getting at but deserializing into complex references can cost more than simple assignments up front -- although, I store all of my story data in a delimited string -- so it just depends on what you are doing.)

I noticed this single-token caveat when I was about to put my --[[const]] tag in front of a "constant" I use to store an array. With my current process of text replacement, it was pretty easy to see that would cost me more tokens if I processed that "constant" like the others. I don't want to expand that table out in all the places it is referenced. Something to keep in mind that should simplify decisions on what to automate in picotool.


Yeah, s'why I mentioned immutable value types. I think it'd also have to make some decision about character length when it comes to expansion, maybe with a balance between char and token savings as a result of both substitution and literal expression evaluation. Maybe the balance could be adjusted with command line arguments or whatever.

I imagine it mostly being useful for constants that represent short numeric values, like sprite IDs, positions, and sizes. Long strings are unlikely to be useful substituted, and as you noticed tables can't be substituted without changing the semantics of the program.



[Please log in to post a comment]