Log In  


jasondelaat

I know a lot of people, myself included, usually write their pico~8 code a little off the cuff tinkering with it until it works. Which tends to be more fun in my experience. But it can also be incredibly frustrating if I'm working on sometime more complex where every time I change something I break something else. And in those cases planning out some formal tests helps me maintain my sanity and get the thing actually working much faster than I probably would otherwise. And since I'm working on something fairly complex at the moment, I took a bit of a detour and put together a little test framework and thought I'd make it available for anybody else who might find it useful.

The code is on github: https://github.com/jasondelaat/pico8-tools/tree/release/testo-8
Or you can just copy it here:

 
--------------------------------
-- testo-8: testing framework
-- copyright (c) 2022 jason delaat
-- mit license: https://github.com/jasondelaat/pico8-tools/blob/release/license
--------------------------------
do
   local all_tests = {}

   local function smt(t, mt)
      return setmetatable(t, {__index=mt})
   end

   local function shallow_copy(lst)
      local copy = {}
      for l in all(lst) do
         add(copy, l)
      end
      return copy
   end

   local function filter(f, lst)
      local results = {}
      for l in all(lst) do
         if f(l) then
            add(results, l)
         end
      end
      return results
   end

   local execute_meta = {
      execute=function(self)
         local result = self[4](self[3](self[2]()))
         if self._cleanup[1] then
            self._cleanup[1]()
         end
         return {
            result,
            self[1]..self.when_txt..self.result_txt
         }
      end
   }

   local when_result_meta
   local result_meta = {
      result=function(self, txt, fn)
         local t = shallow_copy(self)
         t.when_txt = self.when_txt
         t.result_txt = 'result '..txt..'\n'
         t._cleanup = self._cleanup
         add(t, fn)
         add(all_tests, smt(t, execute_meta))
         return smt(self, when_result_meta)
      end
   }

   local _cleanup
   local when_meta = {
      when=function(self, txt, fn)
         _cleanup = {}
         local t = shallow_copy(self)
         t.when_txt = 'when '..txt..'\n'
         t[3] = fn
         t._cleanup = _cleanup
         return smt(t, result_meta)
      end
   }

   when_result_meta = {
      when=when_meta.when,
      result=result_meta.result,
      cleanup=function(self, f)
         add(_cleanup, f)
         return self
      end
   }

   local given_meta = {
      given=function(self, txt, fn)
         local msg = self[1]..'given '..txt..'\n'
         return smt({msg, fn}, when_meta)
      end
   }
   function test(name)
      _cleanup = {}
      local t = smt({name..':\n', _cleanup=_cleanup}, given_meta)
      return t
   end

   local function run_tests()
      cls()
      cursor(0, 7)
      local results = {}
      for t in all(all_tests) do
         add(results, t:execute())
      end
      local failures =
         results and filter(function(r) return not r[1] end, results) or 0
      if #failures == 0 then
         print('all '..#all_tests..' tests passed!', 0, 0, 11)
      else
         for f in all(failures) do
            print(f[2])
         end
         rectfill(0, 0, 127, 6, 0)
         print(#failures..'/'..#all_tests..' tests failed:\n', 0, 0, 8)
         cursor(0, 127)
      end
   end

   function _init()
      run_tests()
   end
end
-- end testo-8 ------------------------------

And here's a cart with some simple functions containing errors and a few tests, most of which fail just to give you an idea of how it works.

Cart #testo8_demo-0 | 2022-10-23 | Code ▽ | Embed ▽ | No License
1

Testo-8

The framework is pretty simple. It's just a single function test which returns an object exposing methods— given, when, result and cleanup —for defining the test.

Testo-8 defines an _init function which automatically runs the tests. Just #include testo-8.lua, write tests and run the cart. If you've defined your own _init you'll probably need to comment it out to get the tests to run.

A very simple test might look something like this:

 
test('some simple addition')
    :given('the number one', function() return 1 end)
    :when('adding 2', function(n) return n+2 end)
    :result('should equal 3', function(r) return r == 3 end)

The methods given, when and result —which I'll call clauses—all take a string as their first argument and a function as their second, while test takes a string argument only. The strings are used to build the output message if the test fails.

The function arguments taken by the other methods serve different purposes:

  • given should return the object(s) being tested. (1 in the example)
  • when takes the object(s) being tested as input and does something with it returning the result(s) (add 2)
  • result takes the result(s) and should return a boolean, true if the test passes and false if it fails. (does 1+2 == 3?)

Each test has exactly one given clause below which will be one or more when clauses. Each when clause contains one or more result clauses and can optionally be ended with a cleanup clause. More on that later. So an actual test might look something like this:

 
-- the ...'s are just a placeholders for some appropriate function
test('some test')
    :given('an object to test', ...)
    :when('1st when', ...)
    :result('result 1', ...)
    :result('result 2', ...)
    :result('result 3', ...)

    :when('2nd when', ...)
    :result('result 4', ...)
    :result('result 5', ...)
    :result('result 6', ...)
    :cleanup(...)

    :when('3rd when', ...)
    :result('result 7', ...)
    :result('result 8', ...)

The number of result clauses is the actual number of tests that will be run so the above example would be eight tests. Each result clause is executed as follows: The given clause is executed to generate the object(s) to test. The test object(s) are passed to the when clause which appears above the result and finally the results are passed to the result clause which determines whether the test passes or fails.

So in the above example the given clause will run eight times, once for every result. The first when clause will be called three times and so will the second while the third when clause will only be called twice.

cleanup takes a single function as its argument and is used to clean up after a test if, for instance, the test modifies some global state which needs to be reset in-between tests. The cleanup clause is optional but if it exists will be called after each result clause inside the same when. The cleanup in the above example would therefore be called three times, once after each of results 4, 5 and 6.

For anyone interested in seeing actual tests, here are some I wrote for a simple s-expression parser.

 
function test_given(s)
   return s, function() return s end
end

function len(n)
   return function(lst)
      return #lst == n
   end
end

function index_eq(n, s)
   return function(lst)
      return lst[n] == s
   end
end

function is_table(r)
   return type(r) == 'table'
end

function is_nil(r)
   return r == nil
end

test('empty s-exp')
   :given(test_given('()'))
   :when('parsed', parse)
   :result('should be a table', is_table)
   :result('should have 0 elements', len(0))

test('single element s-exp')
   :given(test_given('(atom)'))
   :when('parsed', parse)
   :result('should have 1 element', len(1))
   :result('element should be atom', index_eq(1, 'atom'))

test('multi-element s-exp')
   :given(test_given('(a b c d e)'))
   :when('parsed', parse)
   :result('should have 5 elements', len(5))
   :result('1st should be a', index_eq(1, 'a'))
   :result('2nd should be b', index_eq(2, 'b'))
   :result('3rd should be c', index_eq(3, 'c'))
   :result('4th should be d', index_eq(4, 'd'))
   :result('5th should be e', index_eq(5, 'e'))

test('nested s-exp')
   :given(test_given('((atom))'))
   :when('parsed', parse)
   :result('should have 1 element', len(1))
   :result('element should be a table', function(r) return is_table(r[1]) end)
   :result('element should have length 1', function(r) return #r[1] == 1 end)
   :result('nested element should be atom',
           function(r) return r[1][1] == 'atom' end
          )

test('multi-element nested s-exp')
   :given(test_given('((a b) (c d) (e f))'))
   :when('parsed', parse)
   :result('should have 3 elements', len(3))
   :result('each element should be length 2',
           function(r)
              for i in all(r) do
                 if #i != 2 then
                    return false
                 end
              end
              return true
           end
          )
   :result('1st contains a and b',
           function(r)
              return r[1][1] == 'a' and r[1][2] == 'b'
           end
          )
   :result('2nd contains c and d',
           function(r)
              return r[2][1] == 'c' and r[2][2] == 'd'
           end
          )
   :result('3rd contains e and f',
           function(r)
              return r[3][1] == 'e' and r[3][2] == 'f'
           end
          )

test('multiply nested s-exp')
   :given(test_given('((a (b c)))'))
   :when('parsed', parse)
   :result('should have 1 element', len(1))
   :result('element length 2',
           function(r)
              return #r[1] == 2
           end
          )
   :result('element [1][1] == a',
           function(r)
              return r[1][1] == 'a'
           end
          )
   :result('element [1][2] is table',
           function(r)
              return is_table(r[1][2])
           end
          )
   :result('element #[1][2] == 2',
           function(r)
              return #r[1][2] == 2
           end
          )
   :result('element [1][2][1] == b',
           function(r)
              return r[1][2][1] == 'b'
           end
          )
   :result('element [1][2][2] == c',
           function(r)
              return r[1][2][2] == 'c'
           end
          )

test('empty string fails to parse')
   :given(test_given(''))
   :when('parsed', parse)
   :result('should be nil', is_nil)

test('unclosed parens fails to parse')
   :given(test_given('((a b) (c)'))
   :when('parsed', parse)
   :result('should be nil', is_nil)

test('too many closed parens fails to parse')
   :given(test_given('((a b) (c)))'))
   :when('parsed', parse)
   :result('should be nil', is_nil)

test('parsing with newlines')
   :given(test_given('(a \n    (b c))'))
   :when('parsed', parse)
   :result('should have 2 elements', len(2))
   :result('1st element should be a', index_eq(1, 'a'))
   :result('r[2][1] == b', function(r) return r[2][1] == 'b' end)
   :result('r[2][2] == c', function(r) return r[2][2] == 'c' end)

Modules and Testing

I rely heavily on an external editor and spreading all my code around a bunch of files. If that's not how you work this may not be super practical. But here's a quick run-down of how I (currently) work on a project.

Even though Pico-8 Lua doesn't technically have modules I generally try to write things in a modular way and #include with the help of do...end gives me something module-like.

A vastly oversimplified example would be something like this:

 
-- player.lua
local pos = {x=64, y=64}
local s = 1

local function move(var, dist)
    return function()
        pos[var] += dist
    end
end

move_left = move('x', -2)
move_right = move('x', 2)
move_up = move('y', -2)
move_down = move('y', 2)

function draw_player()
    spr(s, pos.x, pos.y)
end

Which I include inside of a do...end block like so:

Writing modules like this doesn't really cost much extra because:

  1. These are all functions I'd write anyway
  2. The local keyword doesn't use any tokens
  3. The do...end costs just a single token
  4. The added encapsulation given module local variables means I can't accidentally mess of things like the player position from other parts of my code because pos doesn't exist outside of the module.

Importantly, I don't put the surrounding do...end in the module file itself. Because when it come to writing the actual tests, I'll put those in another separate file and then include it inside the same do...end block as before.

This makes the tests part of the same module so they can access and test all the local data and functions. Once I'm sure everything is working properly I can just comment out the #include for the test file and free up all those tokens.

Issues

  1. Since Lua doesn't have exception handling capabilities like try...catch or similar, I'm not able to intercept certain errors and report them as test failures. So things like attempting to index a nil value, etc. will still cause the cart to crash and you'll have to fix those problems before the test will run.
  2. The above can also lead to occasionally cryptic error messages saying that there's an error with testo-8 itself. This is certainly possible but usually it means you've passed nil, or something else, where testo-8 is expecting a function. If you're frequently commenting out parts of your code make sure you haven't commented out a function which you're using in a test.
P#119475 2022-10-23 14:00 ( Edited 2022-10-25 18:45)


[Please log in to post a comment]