Synchronous API

The synchronous API takes less effort to implement on the client-side but provides less control over aspects of query execution. Query execution is started upon receipt of the HTTP request and the corresponding HTTP response is delivered upon query completion.

Executing Stored Script

A stored script can be executed by making a POST request to / with the scriptPath query parameter. The scriptPath parameter contains the path to a stored script file in the SpectX resource tree. Note that paths starting with /user/... are resolved based on current user context (identified by authentication token).

A stored script (a view) can have input arguments, which can be specified as query string parameters.

Note

The API supports only literal argument values.

Example. Let’s use the following view:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
init(
  ip::IPADDR,                                      //mandatory input argument (uninitialized)
  from:TIMESTAMP('2016-03-14 00:00:00.000 +0000'), //optional input argument (initialized to a default value)
  to:TIMESTAMP('2016-03-16 00:00:00.000 +0000')    //optional input argument (initialized to a default value)
);

$pattern = $[/shared/patterns/apache.sxp];

LIST('s3s://sample-data/formats/log/apache/apache_access.log.sx.gz')
| PARSE(pattern:$pattern)
| filter(clientIpv4 = $ip AND timestamp >= $from AND timestamp <= $to)
| select(timestamp, ip:clientIpv4, country:CC(clientIpv4), verb, uri, response)
;

Save the script as /user/example_apache.sx. Execute the script using curl utility:

Note

You must replace the value of the token in Authorization header with the one in User Properties and host/port if you have changed the wgui.host or wgui.port configuration parameters)

curl -XPOST -G \
 -H "Accept: application/json-seq" \
 -H "Authorization: Bearer 4d055ad820051448" \
 -d "scriptPath=%2Fuser%2Fexample_apache.sx" \
 -d "opt.timezone=Etc%2FGMT" \
 -d "ip=117.169.75.66" \
 --data-urlencode "from=2016-03-14 21:34:30.000 +0000" \
 --data-urlencode "to=2016-03-14 21:34:45.000 +0000" \
 http://localhost:8388/API/v1.0/

HTTP Request:

POST /API/v1.0/?scriptPath=%2Fuser%2Fexample_apache.sx&opt.timezone=Etc%2FGMT&ip=117.169.75.66&from=2016-03-14%2021%3A34%3A30.000%20%2B0000&to=2016-03-14%2021%3A34%3A45.000%20%2B0000 HTTP/1.1
Host: localhost:8388
Accept: application/json-seq
Authorization: Bearer 4d055ad820051448

HTTP Response:

HTTP/1.1 200 OK
Content-Type: application/json-seq; charset=UTF-8
Accept-Ranges: items
Content-Length: 300

{"timestamp":"2016-03-14 21:34:31.000 +0000","ip":"117.169.75.66","country":"CN","verb":"GET","uri":"//mysql/scripts/setup.php","response":404}
{"timestamp":"2016-03-14 21:34:35.000 +0000","ip":"117.169.75.66","country":"CN","verb":"GET","uri":"//typo3/phpmyadmin/scripts/setup.php","response":404}

You can change the values of the input arguments (or omit the from and to) to see how it affects the result of the query.

Executing Arbitrary Query

Query script can be specified in the body of a POST request to /. The query working directory is specified in the scriptPath parameter.

Note

The path must have sufficient access rights (at least execute and list) for the user executing the query.

Example: executing the following request using curl:

Note

You must replace the value of the token in Authorization header with the one in User Properties and host/port if you have changed the wgui.host or wgui.port configuration parameters)

curl http://localhost:8388/API/v1.0/?scriptPath=/system/ \
    -H "Authorization: Bearer 4d055ad820051448" \
    -H "Accept: application/csv" \
    -d "dual(1).select(i, ip, t);"

HTTP Request:

POST /API/v1.0/?scriptPath=/system/ HTTP/1.1
Host: localhost:8388
Authorization: Bearer 4d055ad820051448
Accept: application/csv
Content-Length: 25
Content-Type: application/x-www-form-urlencoded

dual(1).select(i, ip, t);

HTTP Response:

HTTP/1.1 200 OK
Accept-Ranges: items
Content-Type: application/csv; charset=UTF-8
Content-Length: 43

"i";"ip";"t"
0;0.0.0.0;2019-09-18 14:31:03

Fetching Data From a Table

Contents of a table created by the save command can be fetched by making a GET request to /tables/<table_path>.

This is effectively the same as executing @[/path/to/data.sxt] command in the script.

Example. Let’s create a stored data table by executing the following query:

1
2
3
    dual(10) | select(sequence:i, ip, desc:s) | save('/user/mytable.sxt', true);

Next, execute the following request using ``curl``:

Note

You must replace the value of the token in Authorization header with the one in User Properties and host/port if you have changed the wgui.host or wgui.port configuration parameters)

curl http://127.0.0.1:8388/API/v1.0/tables/user/mytable.sxt \
    -H "Authorization: Bearer 4d055ad820051448"

HTTP Request:

GET /API/v1.0/tables/user/100dual.sxt HTTP/1.1
Host: 127.0.0.1:8388
Authorization: Bearer 4d055ad820051448

HTTP Response:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 626

[
  {"sequence":0,"ip":"0.0.0.0","desc":"0ho0","f_9":null},
  {"sequence":1,"ip":"0.0.0.1","desc":"1ho1","f_9":null},
  {"sequence":2,"ip":"0.0.0.2","desc":"2ho2","f_9":null},
  {"sequence":3,"ip":"0.0.0.3","desc":"3ho3","f_9":null},
  {"sequence":4,"ip":"0.0.0.4","desc":"4ho4","f_9":null},
  {"sequence":5,"ip":"0.0.0.5","desc":"5ho5","f_9":null},
  {"sequence":6,"ip":"0.0.0.6","desc":"6ho6","f_9":null},
  {"sequence":7,"ip":"0.0.0.7","desc":"7ho7","f_9":null},
  {"sequence":8,"ip":"0.0.0.8","desc":"8ho8","f_9":null},
  {"sequence":9,"ip":"0.0.0.9","desc":"9ho9","f_9":null}
]

Fetching Table Schema

The schema of a table created by the save command can be fetched by making a GET request to /tables/<table_path>/schema.

The response format is always JSON. The schema is presented as an object, with field names as keys and field types as values.

Example. Fetch the schema of the table created in the previous example:

Note

You must replace the value of the token in Authorization header with the one in User Properties and host/port if you have changed the wgui.host or wgui.port configuration parameters)

curl http://127.0.0.1:8388/API/v1.0/tables/user/mytable.sxt/schema \
    -H "Authorization: Bearer 4d055ad820051448"

HTTP Request:

GET /API/v1.0/tables/user/mytable.sxt/schema HTTP/1.1
Host: 127.0.0.1:8388
Authorization: Bearer 4d055ad820051448

HTTP Response:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 171

{
    "sequence":"INTEGER",
    "ip":"?IPADDR",
    "desc":"STRING",
    "f_9":{"?TUPLE":{"foo":{"ARRAY":{"TUPLE":{"bar":"INTEGER"}}}}}
}