A console for KRL code

The challenge accepted was to provide a way to evaluate simple KRL expressions from within the Pico UI. One can always jump into the Rulesets Editor page and create a new ruleset containing the expression, register it, install it in a pico, go to that pico's Testing tab and run the function in the new ruleset that contains the expression. But that is a lot of manual steps!

The goal is to create a ruleset that one can drop into any pico, then go to that pico's Testing tab, enter an expression into a box, press Enter, and see the value produced by the expression.

A similar feature existed in the Classic KRL, a program for evaluating KRL declarations.

As an additional challenge, we wanted to do this using only KRL, and succeeded with a ruleset named console which can be found in this GitHub repo.

Expanding an input box is explained in the page on using the Testing tab.

The remainder of this document describes the problems encountered along with their solutions.

Proving the concept of a console

The first problem is using KRL to compile the code entered in the expression box.

In KRL, the unit of compilation is the ruleset. So we will have to generate an entire ruleset with the expression embedded in it.

The source code of a ruleset is simply a String that follows the syntax requirements, and strings can be put together quite easily in KRL.

Generating ruleset source code

To generate the source code for a ruleset on-the-fly, we can use code like this:

ruleset gen_rs {
  meta {
    shares __testing, rs
  }
  global {
    __testing = { "queries":
      [ { "name": "__testing" }
      , { "name": "rs", "args": [ "rid" ] }
      ] , "events":
      [ { "domain": "gen_rs", "type": "ruleset_needed", "attrs": [ "rid" ] }
      ]
    }
    rs = function(rid){
      <<ruleset #{rid} {}>>
    }
  }
}

Lines 13-15 generate a string which is the source code for a minimal ruleset, given a ruleset identifier (the function argument of line 13). Notice the use of chevron quotes and a bee-sting to insert the ruleset identifier, in line 14.

The remaining code provides access for testing.

Line 3 ensures that this function (named rs) can be used by anyone who has an event channel identifier (ECI) for the pico in which this gen_rs ruleset is installed. It also makes the name __testing available, so that the function is in the pico's Testing tab (as specified in line 8).

Using the Testing tab

Putting all of this together, having installed the ruleset shown above in a pico named "test" (and using a channel of type "application" named "test"), we can use the Testing tab as a UI to call the function rs with the argument string "my_rid":

Using a browser

We could also invoke the shared function from a browser, by using a URL like this one:

Notice the use of a file suffix .txt in the query URL. This is explained in the page Content negotiation with HTTP, and allows us to get a result which is the content of the string rather than a string literal. We will need that later when we ask the pico to compile the code.

Compiling ruleset source code

Manual solutions

We could of course, copy the source code from the browser and paste it into the Engine Rulesets page.

We could also use the URL in the Rulesets tab of another pico, first pasting the URL into the "install ruleset from URL" box:

And then noting that that pico now has the ruleset installed:

Automating compilation and installation

The pico engine provides an action which, given the location of ruleset source code, will fetch it, compile it, and register it. Actions can only be performed in rules (not in functions), so we add this rule to the gen_rs ruleset:

  rule generate_ruleset {
    select when gen_rs ruleset_needed
    pre {
      rid = event:attr("rid")
      eci = "Y98aQpr5GFHdVKNDEdHXVC"
      url = <<#{meta:host}/sky/cloud/#{eci}/gen_rs/rs.txt?rid=#{rid}>>
    }
    engine:registerRuleset(url=url) setting(rid)
  }

Line 8 provides the URL (assembled in line 6) that we have been using manually to the engine:registerRuleset action. That built-in action uses the url to get source code and compiles the ruleset. If the compilation is successful, the new ruleset is registered with the pico engine, and the action returns the ruleset identifier (which in this case is bound to the name rid by the setting clause).

We run into this problem when triggering the new rule:

At first, nothing appears to be happening, and after some time we see the error message. Furthermore, the new ruleset is not registered with the pico engine.

What is going wrong? The action of registering the ruleset cannot complete, because it is making a query to the pico which is currently running. That query (the one specified in the URL) is queued up for the pico, waiting for it to be free to handle it. It on the other hand is waiting for the action to complete so that it can finish. This deadlock situation is broken by the pico engine which allows each event (or query) to take no more than a certain amount of time. When that time is up, the error shown appears as the "result" of the event. The pico engine log shows "Error: socket hang up" and then shows that the queued-up query returns "ruleset another_rid {}".

The solution to this deadlock problem is to have the gen_rs installed in one pico (in this case, test) for the purpose of getting the source code (using the hard-coded eci), and also installed in a different pico (in this case, another pico) for the purpose of running the rule which gets the source code. This works fine:

Now the ruleset another_rid is registered with the pico engine.

Using a child pico as the other pico

The final piece of the puzzle (automating the creation of a ruleset) is to have our pico create a child pico with the gen_rs ruleset also installed, and call upon it to provide the source code. The rules to do this are shown here:

  rule create_child_pico {
    select when gen_rs ruleset_needed
    pre {
      rid = event:attr("rid")
    }
    fired {
      raise wrangler event "new_child_request" attributes {
        "name": random:uuid(), "rids": [meta:rid], "rid_to_gen": rid
      }
    }
  }
  rule evaluate_expression {
    select when wrangler new_child_created
    pre {
      rid = event:attr("rid_to_gen")
      eci = event:attr("eci")
      url = <<#{meta:host}/sky/cloud/#{eci}/gen_rs/rs.txt?rid=#{rid}>>
      picoId = event:attr("id")
    }
    engine:registerRuleset(url=url) setting(rid)
    fired {
      raise wrangler event "child_deletion" attributes event:attrs
    }
  }

In line 4 we capture the rid supplied by the user and in line 8 we include it as an extra attribute in the event raised to wrangler (lines 7-9) to ask for a new child pico.

In line 15 we recover the user-supplied rid from the attributes which wrangler supplies when it lets us know that the new child pico has been created for us. Lines 16 and 18 also provide an ECI for the child pico along with its own ID. We will need both of those later. The ECI in line 17 where we compute the URL for the source code of our generated ruleset. The ID will be used when additional actions are used later. Line 20 actually fetches the source code, compiles it, and registers the generated ruleset with the engine.

Having gone this far, we no longer need the child pico, so we ask wrangler to delete it (line 22).

The use of a child pico for this purpose is explained in more detail in the page Using a child pico to perform an asynchronous task.

Doing something useful with a generated ruleset

So far, the ruleset (whose source code) we generated was minimal. We will need it to do something, and ultimately to accept an expression and provide the result.

We will first hard-code an expression (actually the one we wanted originally, which triggered this challenge in the first place, meta:host (see the meta page)).

    rs = function(){
      rsn = random:uuid();
      <<
ruleset #{rsn} {
  meta {
    shares result
  }
  global {
    result = function() {
      meta:host
    }
  }
}>>
    }

The new function is different. Rather than a user-defined ruleset identifier, line 2 provides a random one, placed in the generated code by the bee sting of line 4.

The ruleset we will generate has a function (defined in lines 9-11) named result which is shared (by line 6) so that we could invoke it from a browser.

To do that, we would have to install it in a pico, obtain an ECI for that pico, and construct a /sky/cloud query URL. We can automate this, using further engine actions.

  rule evaluate_expression {
    select when wrangler new_child_created
    pre {
      eci = event:attr("eci")
      url = <<#{meta:host}/sky/cloud/#{eci}/gen_rs/rs.txt>>
      picoId = event:attr("id")
    }
    every {
      engine:registerRuleset(url=url) setting(rid)
      engine:installRuleset(picoId,rid=rid)
      http:get(<<#{meta:host}/sky/cloud/#{eci}/#{rid}/result>>) setting(res)
      send_directive("_txt",{"content":res{"content"}})
      engine:uninstallRuleset(picoId,rid)
      engine:unregisterRuleset(rid)
    }
    fired {
      raise wrangler event "child_deletion" attributes event:attrs
    }
  }

The only change, beyond removing references to the rid_to_gen, has been to replace the single action that fetches, compiles, and registers the generated ruleset with a sequence of actions.

The action in line 9 does the compiling and registering as before.

In line 10, we use the engine:installRuleset action to install the new ruleset (identified by its rid) into the child pico.

Then, in line 11, we use http:get (as an action) to invoke the result function in the generated ruleset newly installed in the child pico. This action returns the HTTP response, which we bind to the name res (using the action's setting clause).

Line 12 uses the reserved directive name "_txt" in a send_directive action to provide some content, specifically the content portion of the HTTP response. We just assume it was successful.

Finally, we do some cleanup. Line 13 uninstalls the ruleset from the child pico, so that line 14 can unregister the generated ruleset from the engine. And, in the postlude (lines 16-18) we raise the wrangler event "child_deletion" (line 17) so that the child pico will be removed.

Embedding a user-supplied expression in the generated ruleset

We make changes to allow the user to supply a KRL expression (actually, anything that can appear in a function definition, so a (possibly empty) list of declarations followed by an expression). This will be presented in the order the user-supplied expression moves through this and the generated ruleset. First, changes to the __testing specification:

    __testing = { "queries":
      [ { "name": "__testing" }
      ] , "events":
      [ { "domain": "gen_rs", "type": "ruleset_needed", "attrs": [ "expr" ] }
      ]
    }

We no longer directly test the rs function, and we've changed the expected attribute name from rid to expr in line 4.

  rule create_child_pico {
    select when gen_rs ruleset_needed
    pre {
      expr = event:attr("expr").klog("expr")
    }
    fired {
      raise wrangler event "new_child_request" attributes {
        "name": random:uuid(), "rids": [meta:rid], "expr": expr
      }
    }
  }

The attribute expr provided by the user is passed along as an extra attribute when we raise wrangler event: "new_child_request". This extra attribute is passed through the sequence of events we are initiating in wrangler. We pick it up in the next rule:

  rule evaluate_expression {
    select when wrangler new_child_created
    pre {
      expr = event:attr("expr")
      e = expr.math:base64encode().replace(re#[+]#g,"-")
      eci = event:attr("eci")
      url = <<#{meta:host}/sky/cloud/#{eci}/gen_rs/rs.txt?expr=#{e}>>
      picoId = event:attr("id")
    }
    every {
      // actions unchanged
    }
    fired {
      raise wrangler event "child_deletion" attributes event:attrs
    }
  }

Line 4 retrieves the user-supplied expression from the attributes that have been passed along.

Since line 7 is embedding the expression in a URL, we Base 64 encode it in line 5. The function rs will then expect the user-supplied expression as its argument.

    rs = function(expr){
      rsn = random:uuid();
      e = expr.math:base64decode();
      <<
ruleset #{rsn} {
  meta {
    shares result
  }
  global {
    result = function() {
      #{e}
    }
  }
}>>

In line 1 the rs function accepts its argument and Base 64 decodes it in line 3. The user-supplied expression is embedded, as before, as the function body in line 11 using a bee sting.

The actions (elided earlier) are shown again here for convenience, to place the final mention of the user-supplied expression:

      engine:registerRuleset(url=url) setting(rid)
      engine:installRuleset(picoId,rid=rid)
      http:get(<<#{meta:host}/sky/cloud/#{eci}/#{rid}/result>>) setting(res)
      send_directive("_txt",{"content":res{"content"}})
      engine:uninstallRuleset(picoId,rid)
      engine:unregisterRuleset(rid)

As before, the generated ruleset is registered and installed (lines 1-2). The function result (containing the user-supplied expression), of the generated ruleset is invoked finally in line 3, and the value of that expression is then returned to the Testing tab by line 4. As before the ruleset is cleaned up and the child pico deleted.

The result of the user-supplied expression's evaluation is displayed in the Results section of the Testing tab.