Implementation WIP

Wrangler's Test Suite is under heavy development and more features will be added for more powerful tests.

Wrangler's Test Ruleset

Wrangler provides a ruleset named io.picolabs.test that has the ability to test the public facing event-query API of a ruleset and some internal testing of the event API of a ruleset.

Tests are written as a KRL map that is then given as an event attribute to the test ruleset to run the tests.

The essence of a wrangler test are "kickoff" events and "listener" events. The "kickoff" event is the event that starts the internal event chain in the ruleset under test. When the "listener" events occur as a result of the kickoff event, a KRL expression will be run, which can be used to check for the expected state of both the event and the pico.

Each test is run by the test runner pico creating a unique KRL ruleset that runs the test, then creating a child pico with that ruleset and the ruleset under test installed. The parent pico running the tests then signals to the created children to begin the test. When finished, the child pico will send the test results back to the parent pico. If a test succeeds, the parent will delete the child pico upon receiving the test results. If the test fails, the child pico will not be deleted so that the developer can examine its logs to figure out why the test failed (the ruleset created for that test can also be viewed in the KRL editor).

Once all the test results have been sent back to the parent, or the test group times out, the parent will raise tests:tests_finished with the test report.

Creating Tests

This is an example map of created tests:

Example Test Map with Explanations

// This map would be given as the "tests" attribute to the event tests:run_each_test.
wrangler_tests = {
  "Pico Creation": {
    "kickoff_events": {	// The kickoff event is raised at the beginning of the test, starting the internal event chain of the ruleset under test
      "wrangler:new_child_request":{"name":"blue"}	// The domain and type of the kickoff event is mapped to the desired attributes of that event
    },
    "start_state": <<>>, // Not implemented
    "prototype":<<>>,	// Not implemented, but will allow the test writer to create a prototype at the beginning of the test to holistically test pico systems
    "listeners": {
      "wrangler:child_initialized" : {	// Our listener event is wrangler:child_initialized, which we can expect to occur as a result of wrangler:new_child_request. If it doesn't occur the test will timeout and fail
          "expressions": [	// The expressions to run during the selection on wrangler:child_initialized
            ["Child name should exist", <<wrangler:children(event:attrs{"name"})>>]	// The explanation for the expression is given then the expression itself.
            ,["Pico should only have one child", <<wrangler:children().length() == 1>>]	// The pico this test is running in just created a single child. We should expect wrangler:children() to return an array of length 1
            ,["Child name should be blue", <<wrangler:children().any(function(child){child{"name"} == "blue"})>>] // In our attributes to the kickoff event we said the new child should be named "blue"
          ]
      }
    },
    "meta": [
      "use module io.picolabs.wrangler alias wrangler" // The meta portion is an array of strings that will be added to the meta block of the unique test ruleset. In this case we want to be able to call functions from wrangler, so we include it as a module.
    ],
    "global":[]	// The global portion is an array of strings that will be added to the global block of the unique test ruleset running in the child test pico.
  },
  "Pico Creation with child_creation": {	// This test is very similar to the above test, but uses the other event that creates a new pico, and tests slightly different expressions.
    "kickoff_events": {
      "wrangler:child_creation":{"name":"blue"}
    },
    "start_state": <<>>,
    "prototype":<<>>,
    "listeners": {
      "wrangler:child_initialized" : {
          "expressions": [
            ["Child name should exist", <<wrangler:children(event:attrs{"name"})>>]
            ,["Pico should only have one child", <<wrangler:children().length() == 1>>]
            ,["Child name should be blue", <<wrangler:children().any(function(child){child{"name"} == "blue"})>>]
          ]
      }
    },
    "meta": [
      "use module io.picolabs.wrangler alias wrangler"
    ],
    "global":[]
  }
}

If any expressions have a syntax error the ruleset will raise test:unable_to_create_test_ruleset with the relevant parse error.

Running Tests

The tests can be run by passing them as an attribute to tests:run_each_test. Results will be given in the tests:tests_finished event. The ongoing state of the tests can be queried for by calling the tests:getTestsOverview() function of io.picolabs.test.

Start Tests

always {
raise tests event "run_tests" attributes {
"tests": wrangler_tests,
"ruleset_under_test":"io.picolabs.wrangler"
};

Start Tests

// As tests complete the ruleset raises progress update events
rule receive_progress_update {
  select when tests tests_progress_update
  always {
    ent:progress_report := event:attr("testsOverview")
  }
}

//When the tests are finished this rule is raised
rule tests_finished {
  select when tests tests_finished
  always {
    ent:full_report := event:attr("fullReport");
    ent:progress_report := event:attr("testsOverview")
  }
}

// the getTestsOverview() function returns a map like this:
{
  "failedToStart": {},
  "numTestsRun": 76,
  "numTestsToRun": 76,
  "percentComplete": 100,
  "testBatchTimedOut": false,
  "timedOut": false
}
// The getFullReport() function returns a map like this:
{
  "Pico Creation": {
    "wrangler:child_initialized": {
      "wrangler:children(event:attrs{\"name\"})": "passed",
      "wrangler:children().length() == 1": "passed",
      "wrangler:children().any(function(child){child{\"name\"} == \"blue\"})": "passed"
    }
  },
  "Pico Creation with child_creation": {
    "wrangler:child_initialized": {
      "wrangler:children(event:attrs{\"name\"})": "passed",
      "wrangler:children().length() == 1": "passed",
      "wrangler:children().any(function(child){child{\"name\"} == \"blue\"})": "passed"
    }
  }
}
// It informs you what the listener event was and what expression was attempted to be selected on.