HTTP

Engine Compatibility

This library has not yet been ported to the New Pico Engine

The HTTP Library

The HTTP library provides for more general, flexible access to cloud-based data than dataset and datasource declarations. The HTTP library supports the primary HTTP methods--get, post, put, delete, head, and patch as both functions and actions.

NOTE: Using GET to send data to an API is not very RESTful and is not recommended. KRL allows it for cases where the API has been written poorly and using GET is the only way to interact with it.

The Get Function

The http:get() function can be used as part of any KRL expression. The function always returns a map of name-value pairs, so the expression must expect the data in that format. The http:get() function takes four arguments (in order):

URL—the URL to retrieve
Parameters—a hash that will be used to create the URI encoded query parameters
Headers—an optional hash that will be used to create headers for the GET request
Response headers—an optional array that contains the names of headers from the response to pass back in the response

The resulting map of name-value pairs has the following structure:

content—the content of the response
content_type—the MIME type of the content
content_length—the number of bytes in the content
status_code—the three digit HTTP status code
status_line—the complete status string including the three digit HTTP status code

In addition, the response map will contain any keys given in the fourth optional parameter to the request and their value in the resulting response header.
The following example shows using http:get() to call a URL on example.com:

pre {
  r = http:get("http://example.com/widgets/printenv.pl",
               {"a": "5",
                "version": "dev"
               }
              );
}

Often the parameters won't be a literal map as we've shown here, but a computed value. This function application would make the call to the example.com URL after fashioning a query string from the parameters in the map in the second argument. The URL called would be:

http://example.com/widgets/printenv.pl?a=5&version=dev

Suppose we made the same function application, but also included the third and fourth optional arguments:

pre {
 r = http:get("http://example.com/widgets/printenv.pl",
             {"a": "5",
             "version": "dev"},
             {"X-proto": "flipper"},
             ["flopper"]);
}

When the call is made, the HTTP request will also contain the header

X-proto: flipper

The response will contain the map outlined above, plus a name-value pair with the name flopper and a value equal to the value of the header flopper in the HTTP response.

Usage Notes

Because the content (what you're likely after) is in a map, you will almost always need to pull it out. For example:

url_content = http:get(my_url).pick("$.content");

If you're retrieving JSON, you'll need to turn the string in the content field into a data structure using decode():

json_from_url = http:get(my_url).pick("$.content").decode();

HTTP Authentication

You can perform HTTP Basic authentication on an http:get() by specifying credentials using an alternate argument syntax:

URL—the URL to retrieve
Optional Argument Map—a map specifying the following:
- params—a map of key-value pairs that will be used as the parameters for the GET. These parameters will be URL encoded and appended to the GET request.
- headers—a map of the headers to send with the GET request.
- response_headers—an array of the names of headers from the response to set in the response object.
- credentials—a map of key-value pairs that present the credentials for a URL that requires HTTP Basic authentication.

When using the alternate parameter syntax, params is required even if it is empty.

For example, you could specify credentials for the HTTP Basic authentication as follows:

x1 = http:get(carvoyent_url(my_vehicle_id),
              {"credentials":  {"username": API_key,
                                "password": carvoyent_secret,
                                "realm": "Carvoyant API",
                                "netloc": "dash.carvoyant.com:443"
                                },
               "params":{}
              });

You must correctly specify the realm and net location (including the port as shown) for this to work. The easiest way to get this information by attempting to go to the URL in a browser and gathering it from the challenge window that the browser shows.

Other Functions

In addition to http:get(), you can also use http:post(), http:delete(), http:put(), and http:patch() as functions using this same syntax. This is useful when you want to create functions (e.g. in a module) that interact with an API.

HTTP Actions

Post

Interacting with cloud-based services usually involves sending data. To send data we use the http:post() action. The http:post() action takes a single argument, the URL of the resource accepting the POST request.

The http:post() action accepts the following optional parameters (given using the with syntax):

params—a map of key-value pairs that will be used as the parameters for the POST. These parameters will be form-encoded and sent in the body of the POST request.
headers—a map of the headers to send with the POST request.
response_headers—an array of the names of headers from the response to set in the response object.
body—the body of the POST. This is only sent if the params parameter is empty or undefined and the content-type parameter is set in the headers. This allows for sending raw, unencoded content with the POST.
credentials—a map of key-value pairs that present the credentials for a URL that requires HTTP Basic authentication. See the example above for the format and other information.
autoraise—a string that will be the label applied to the event that this action automatically raises.

The action supports an optional setting clause that gives the name of the variable in which the response to the HTTP request should be stored for later use. This variable is available to any expression executed after the action takes place: expressions in the parameters of later actions or in the postlude of the rule.

The response is returned as a map of the following key-value pairs:

content—the content of the response
content_type—the MIME type of the content
content_length—the number of bytes in the content
status_code—the three digit HTTP status code
status_line—the complete status string including the three digit HTTP status code
label—the autoraise label, if autoraise is set (see above)

The following example shows using an http:post() action inside a rule:

rule r1 {
 select when pageview url #/archives/# 
 http:post("http://www.example.com/go")
  with params = {"answer": "x"} 
}

When this rule is selected, it will POST to the specified URL with a form-encoded body containing the name-value pair shown in the params map.

Here's another example showing the use of the http:post() action to POST raw content—XML in this case:

rule r2 {
 select when pageview url #/archives/# 
 http:post("http://example.com/printenv.pl")
  with body = << <?xml encoding='UTF-8'?>
                 <feed version='0.3'>
                 </feed> >> and 
   headers = {"content-type": "application/xml"};
}

Get

The http:get() action works in the same manner as http:post(), except that an HTTP GET is used instead of a POST and the parameters are encoded as an HTTP query string.

Put

The http:put() action works in the same manner as http:post(), except that an HTTP PUT is used instead of a POST. For example, here is a user defined action, update, that uses http:put():

update = defaction(id, value) {
   http:put(base_url+feed_id) with
     body = {"version": vers,
             "datastreams":[
                {"id":id, "current_value":value}
             ]
            } and
     headers= {"X-ApiKey": api_key,
               "content-type": "application/json"
              }
  }

Delete

The http:delete() action works in the same manner as http:post(), except that an HTTP DELETE is used instead of a POST.

Patch

The http:patch() action works in the same manner as http:post(), except that an HTTP PATCH is used instead of a POST.

Head

The http:head() action works in the same manner as http:post(), except that an HTTP HEAD is used instead of a POST.

Processing the Response

If autoraise is set, the rule will automatically raise an event with the event domain http and the event type equal to the HTTP method used (i.e. post, put, get, delete). The event will contain the following attributes:

label - the value that autoraise was set to. This is useful for distinguishing similar events in the event expression.
content - the content of the response. This is decoded, if possible.
content_type - the content type
content_length - the length, in bytes, of the content
status_code - the HTTP status code
status_line - the textual description of the HTTP status

The autoraise parameter is given with a string as its value. When the explicit event is automatically raised, one of the event parameters will be label and it will have the value of the autoraise string.

Each action defines its own event domain and type. For the http:post() action the event that is raised will have event domain http and event type of post. Similarly, each action will include relevant data from the action. The response values given above are sent as event parameters and can thus be checked as part of the event selection. The http:post() action includes response information from the POST.

The following simple example shows a rule that has an http:post() action with an event autoraise:

rule r1 is active {
  select when pageview "/archives/(\d+)/" setting(year) 
  http:post("http://www.example.com/go")
    with params = {"answer": "x"} 
     and autoraise = "example";
}

This is roughly equivalent to the following rule:

rule r1 is active {
  select when pageview "/archives/(\d+)/" setting(year) 
  http:post("http://www.example.com/go") setting (resp)
    with params = {"answer": "x}; 
  always {
    raise explicit event post with resp
  }
}

The biggest difference is that the autoraise would create an event with the event domain http while the raise statement would create an event with the event type explicit.

Assuming we raised the http:post event in the first rule shown above, we could chain additional rules for subsequent processing of the response. The following two rules check the status code of the response and present a notification of the result:

rule r2 is active {
 select when http post 
               label #example#
               status_code #(2\d\d)# setting (status)
 notify("Status", "Success! The status is " + status);
}

rule r3 is active {
 select when http post
               label #example#
               status_code #([45]\d\d)# setting (status)
  fired {
    log <<Error: #{status}: #{event:param("status_line")}>>;
    last;
  }
}

The second rule fires when the status code in the response indicates an error, logs the error, and uses the last control statement in the postlude to stop subsequent processing of rules in the ruleset.

This example rule shows the content of the response if its content type is “text.”

rule r4 is active {
  select when http post label #example#
  if(event:param("content_type") like "^text/") then
    notify("Page says...", event:param("content"));
}

Rule chaining from an autoraise on an http:post() action provides a convenient and simple way of easily dealing with the results of an action.