I think table serialization is a pretty well-known token-saving technique but I hadn't seen anyone post one like this that just operates via peek and poke rather than using strings, so I thought I'd share my table serializer and deserializer here (originally written for PIZZA PANDA) in case this is useful to anyone.
The serializer writes binary data directly to memory so you can use the output however you want, e.g. you can use the "storing binary data as strings" technique to store it as a string, but you could also just store it anywhere in the cart's data.
The deserializer (the part you need to include in your final cart) is 170 tokens and it similarly reads bytes directly from memory.
The serialized format is pretty efficient and uses data types which are more specific than Lua itself, in order to save storage space; each one of the following is a separate "type":
- 8-bit integer
- 16-bit integer
- full "number" (32 bits)
- boolean true
- boolean false
- string
- empty table
- array
- table
This way if your table has a bunch of little 8-bit integers in it, you're not storing a bunch of whole 32-bit numbers for no reason.
The serialized format has the following limitations in order to keep the deserializer small:
- max 255 properties in a table
- max 255 characters in a string
- no "function" type support
Serialize Table
function serialize_table(addr, t, isarray)
poke(addr, count_props(t))
addr += 1
if isarray then
for v in all(t) do
addr = serialize_value(addr, v)
end
else
for k, v in pairs(t) do
addr = serialize_value(addr, k)
addr = serialize_value(addr, v)
end
end
return addr
end
function serialize_value(addr, v)
if type(v) == "number" then
if v & 0x00ff == v then
poke(addr, 0)
addr += 1
poke(addr, v)
return addr + 1
elseif v & 0xffff == v then
poke(addr, 1)
addr += 1
poke2(addr, v)
return addr + 2
else
poke(addr, 2)
addr += 1
poke4(addr, v)
return addr + 4
end
elseif type(v) == "boolean" and v == true then
poke(addr, 3)
return addr + 1
elseif type(v) == "boolean" and v == false then
poke(addr, 4)
return addr + 1
elseif type(v) == "string" then
poke(addr, 5)
addr += 1
local len = #v
assert(len <= 255, "string must be <= 255 chrs")
poke(addr, len)
addr += 1
for i = 1, len do
poke(addr, ord(v[i]))
addr += 1
end
return addr
elseif type(v) == "table" then
if is_empty(v) then
poke(addr, 6)
return addr + 1
elseif is_array(v) then
poke(addr, 7)
return serialize_table(addr + 1, v, true)
else
poke(addr, 8)
return serialize_table(addr + 1, v)
end
end
end
function count_props(t)
local propcount = 0
for k, v in pairs(t) do
propcount += 1
end
return propcount
end
function is_array(t)
return #t == count_props(t)
end
function is_empty(t)
for k, v in pairs(t) do
return false
end
return true
end |
Deserialize Table
-- 170 tokens
-- limitations:
-- * max 255 properties in a table
-- * max 255 characters in a string
-- * no "function" type support
function deserialize_table(addr, isarray)
local t, propcount, k, v = {}, @addr
addr += 1
for i = 1, propcount do
if isarray then
k = i
else
k, addr = deserialize_value(addr)
end
v, addr = deserialize_value(addr)
t[k] = v
end
return t, addr
end
function deserialize_value(addr)
local vtype, v = @addr
addr += 1
if vtype == 0 then -- 8-bit integer
return @addr, addr + 1
elseif vtype == 1 then -- 16-bit integer
return %addr, addr + 2
elseif vtype == 2 then -- number
return $addr, addr + 4
elseif vtype == 3 then -- boolean true
return true, addr
elseif vtype == 4 then -- boolean false
return false, addr
elseif vtype == 5 then -- string
local len = @addr
addr += 1
v = ""
for i = 1, len do
v ..= chr(@addr)
addr += 1
end
return v, addr
elseif vtype == 6 then -- empty table
return {}, addr
elseif vtype == 7 then -- array
return deserialize_table(addr, true)
elseif vtype == 8 then -- table
return deserialize_table(addr)
end
end |
btw if you want to see more context for how I used this in an actual project, here is the source code of that project: https://github.com/andmatand/pizza-panda
also here is a quick example:
-- step 1. do something like this in your build cart
t1 = {
-- put lots of stuff in here
}
-- this returns the address right after the end of what
-- was stored so you can store multiple tables in a row
addr = serialize_table(0x2000, t1)
t2 = {
-- another table
}
addr = serialize_table(addr, t2)
-- etc.
-- step 2. do something like this in your final published cart
t1, addr = deserialize_table(0x2000)
t2 = deserialize_table(addr) |
[Please log in to post a comment]
