Asynchronous API

The asynchronous API provides fine-grained control over query execution - Query execution is started upon receipt of the HTTP request and the response with a query identifier is delivered immediately. While the query continues executing in the background, the client can, by referencing the query by id, request the execution status, cancel the query and upon successful completion, retrieve query results.

Starting Execution

A query can be executed asynchronously by making a POST request to /queries. The query can be specified either by referencing an existing script by the scriptPath parameter or by providing the query contents in the request body.

In the latter case, the scriptPath parameter specifies the query working directory.

Note that paths starting with /user/... are resolved based on the current user context (identified by authentication token).

Syntax validation of the query is performed and in case of an error, an error response is returned immediately. Otherwise, the query starts executing asynchronously in the background and an identifier is returned in the response.

The query identifier can be used in further requests to check the status of execution, cancel the query or retrieve its results.

The result and metadata of a query are retained for 5 minutes after its last state change or the last reference to it via asynchronous API. (The retention period is configurable)

In case you plan to do all the result fetching for this query in a single format and know the format beforehand, specify the Accept header or acceptContentType parameter along with all the resultset customization opt.* parameters when submitting the POST request. This will parallelize resultset serialization, resulting in higher throughput and lower latency. In this case, any format & resultset customization parameters specified with result fetching requests will be ignored.

To defer the choice of format and resultset customization to result fetching time, refrain from specifying aforementioned headers and parameters for this POST request.

Example

Note

You must replace the value of the token in Authorization header with the one in User Properties and host/port if you have changed the wgui.host or wgui.port configuration parameters)

curl http://localhost:8388/API/v1.0/queries?scriptPath=/system/ \
    -H "Authorization: Bearer 4d055ad820051448" \
    -d "dual(1).select(i, ip, t);"

HTTP Request:

POST /API/v1.0/queries?scriptPath=/system/ HTTP/1.1
Host: localhost:8388
Authorization: Bearer 4d055ad820051448
Content-Length: 25
Content-Type: application/x-www-form-urlencoded

dual(1).select(i, ip, t);

HTTP Response:

HTTP/1.1 201 Created
Location: /API/v1.0/queries/4929b65c-8e4e-49c6-8007-45b3697cee03
Content-Type: application/json
Content-Length: 50

{"queryId":"4929b65c-8e4e-49c6-8007-45b3697cee03"}

Requesting Query Status

The status of a query can be checked by making a GET request to /queries/<id>.

The response contains the current state of the query, completion progress (decimal between 0 and 1), the timeline of the state changes and a list of warnings produced.

In case the query has finished (states FAILED, CANCELLED, SUCCEEDED), the results endpoint URL for this query is reported in the Location header.

Example

Note

You must replace the value of the token in Authorization header with the one in User Properties and host/port if you have changed the wgui.host or wgui.port configuration parameters)

curl http://localhost:8388/API/v1.0/queries/4929b65c-8e4e-49c6-8007-45b3697cee03 \
    -H "Authorization: Bearer 4d055ad820051448"

HTTP Request:

GET /API/v1.0/queries/4929b65c-8e4e-49c6-8007-45b3697cee03 HTTP/1.1
Host: localhost:8388
Authorization: Bearer 4d055ad820051448

HTTP Response:

HTTP/1.1 200 OK
Location: /API/v1.0/queries/4929b65c-8e4e-49c6-8007-45b3697cee03/result
Content-Type: application/json
Content-Length: 296

{
  "queryId": "4929b65c-8e4e-49c6-8007-45b3697cee03",
  "state": "SUCCEEDED",
  "progress": 1,
  "warnings": [
    {
        "time":"2019-09-18T18:08:09.714",
        "level":"WARN",
        "message":"blob not accessible: ssh://user:pass@nosuchhost/asd: Unknown host nosuchhost: No address associated with hostname"
    }
  ],
  "stateChanges": [
    {
      "millis": 0,
      "from": "NEW",
      "to": "QUEUED"
    },
    {
      "millis": 1,
      "from": "QUEUED",
      "to": "SCHEDULED"
    },
    {
      "millis": 1,
      "from": "SCHEDULED",
      "to": "RUNNING"
    },
    {
      "millis": 7,
      "from": "RUNNING",
      "to": "SUCCEEDED"
    }
  ]
}

Canceling Query

A query can be canceled by making a DELETE request to /queries/<id>.

Fetching Results

Results of a finished query can be fetched by making a GET request to /queries/<id>/result.

Example

Note

You must replace the value of the token in Authorization header with the one in User Properties and host/port if you have changed the wgui.host or wgui.port configuration parameters)

curl http://127.0.0.1:8388/API/v1.0/queries/4929b65c-8e4e-49c6-8007-45b3697cee03/result \
    -H "Authorization: Bearer 4d055ad820051448"

HTTP Request:

GET /API/v1.0/queries/4929b65c-8e4e-49c6-8007-45b3697cee03/result HTTP/1.1
Host: 127.0.0.1:8388
Authorization: Bearer 4d055ad820051448

HTTP Response:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 61

[{"i":0,"ip":"0.0.0.0","t":"2007-09-24 15:34:12.211 +0300","f_9":null}]

Fetching Result Schema

The schema of a finished query can be fetched by making a GET request to /queries/<id>/result/schema.

The response format is always JSON. The schema is presented as an object, with field names as keys and field types as values.

Example

Note

You must replace the value of the token in Authorization header with the one in User Properties and host/port if you have changed the wgui.host or wgui.port configuration parameters)

curl http://127.0.0.1:8388/API/v1.0/queries/4929b65c-8e4e-49c6-8007-45b3697cee03/result/schema \
    -H "Authorization: Bearer 4d055ad820051448"

HTTP Request:

GET /API/v1.0/queries/4929b65c-8e4e-49c6-8007-45b3697cee03/result/schema HTTP/1.1
Host: 127.0.0.1:8388
Authorization: Bearer 4d055ad820051448

HTTP Response:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 171

{
    "i":"INTEGER",
    "ip":"?IPADDR",
    "t":"TIMESTAMP",
    "f_9":{"?TUPLE":{"foo":{"ARRAY":{"TUPLE":{"bar":"INTEGER"}}}}}
}