Using API

SpectX provides REST API for executing queries. This provides simple and straightforward way to integrate external applications with SpectX.

You can execute query scripts stored in SpectX as well as queries specified in the request.

API root endpoint:

https://<host>:<port>/API/v1.0/

You can obtain sample api query using curl command line utility by right clicking on the saved query script (in the resource tree), select Properties from the drop down menu and choosing “API access” tab.

HTTP Methods

Http POST method is used to execute queries.

Authentication

API access authentication is implemented using authentication token (a shared secret). The authentication token must be passed to SpectX with each request using “Bearer” authentication scheme (as defined in RFC6750 Section 2.1).

Example 1.

$ curl -XPOST -G \
    -H "Authorization: Bearer c04069088ed66696" \
    -d "scriptPath=%2Fuser%2Fexamples%2Faggr_functions.sx" \
    http://localhost:8388/API/v1.0/

The authentication token can be obtained from User Properties dialog.

Query Result Formatting

Resultset

Following resultset formats are available which can be selected in the respective request’s Accept header:

Format Accept header value
csv application/csv
tsv application/tsv
json application/json

Default result format is json.

Example 2: use csv resultset format.

$ curl -XPOST -G \
    -H "Accept: application/csv" \
    -H "Authorization: Bearer c04069088ed66696" \
    -d "scriptPath=%2Fuser%2Fexamples%2Faggr_functions.sx" \
    http://localhost:8388/API/v1.0/

Setting Timezone

The timezone of TIMESTAMP type fields in query results can be set using opt.timezone query parameter - string specifying timezone name (as defined in IANA Time Zone Database). Default is operating system set timezone.

Example 3.

$ curl -XPOST -G \
    -H "Accept: application/csv" \
    -H "Authorization: Bearer c04069088ed66696" \
    -d "scriptPath=%2Fuser%2Fexamples%2Faggr_functions.sx" \
    -d "opt.timezone=America%2FLos_Angeles" \
    http://localhost:8388/API/v1.0/

TIMESTAMP format

The TIMESTAMP type fields presentation (i.e conversion) can be customized using opt.timestampFormat query parameter. It specifies the format string using following pattern letters:

Symbol Meaning Presentation Examples
G era text AD; Anno Domini; A
u year year 2004; 04
y year-of-era year 2004; 04
D day-of-year number 189
M/L month-of-year number/text 7; 07; Jul; July; J
d day-of-month number 10
Q/q quarter-of-year number/text 3; 03; Q3; 3rd quarter
Y week-based-year year 1996; 96
w week-of-week-based-year number 27
W week-of-month number 4
E day-of-week text Tue; Tuesday; T
e/c localized day-of-week number/text 2; 02; Tue; Tuesday; T
F week-of-month number 3
a am-pm-of-day text PM
h clock-hour-of-am-pm (1-12) number 12
K hour-of-am-pm (0-11) number 0
k clock-hour-of-am-pm (1-24) number 0
H hour-of-day (0-23) number 0
m minute-of-hour number 30
s second-of-minute number 55
S fraction-of-second fraction 978
A milli-of-day number 1234
n nano-of-second number 987654321
N nano-of-day number 1234000000
V time-zone ID zone-id America/Los_Angeles; Z; -08:30
z time-zone name zone-name Pacific Standard Time; PST
O localized zone-offset offset-O GMT+8; GMT+08:00; UTC-08:00;
X zone-offset ‘Z’ for zero offset-X Z; -08; -0830; -08:30; -083015; -08:30:15;
x zone-offset offset-x +0000; -08; -0830; -08:30; -083015; -08:30:15;
Z zone-offset offset-Z +0000; -0800; -08:00;
p pad next pad modifier 1
escape for text delimiter  
‘’ single quote literal

The default presentation format depends on resultset format:

Resultset format Presentation format string
csv yyyy-MM-dd HH:mm:ss
tsv yyyy-MM-dd HH:mm:ss
json yyyy-MM-dd HH:mm:ss.SSS Z

Example 4.

$ curl -XPOST -G \
    -H "Accept: application/csv" \
    -H "Authorization: Bearer c04069088ed66696" \
    -d "scriptPath=%2Fuser%2Fexamples%2Faggr_functions.sx" \
    -d "opt.timestampFormat=MMM/dd/yyyy%20HH:mm:ss%20VV" \
    http://localhost:8388/API/v1.0/

Error Handling

SpectX uses HTTP status codes to indicate success or failure of an API call. In general, status codes in the 2xx range means success, 4xx range means there was an error in the provided information, and those in the 5xx range indicates server side errors. Commonly used HTTP status codes are listed below.

Status code Description
2xx Success
4xx Bad request sent to server
5xx Server side error

The body of response will contain JSON formatted error message.

Example 5.

$ curl -XPOST -G \
    -H "Authorization: Bearer c04069088ed66696" \
    -d "scriptPath=/user/examples/noSuchScript.sx" \
    http://localhost:8388/API/v1.0/

HTTP/1.1 400 Bad Request
Content-Type: application/json

{
    "code":"ClientException",
    "message":"script does not exist: /user/examples/noSuchScript.sx"
}

Execute a Stored Query

A stored query script can be executed by specifying scriptPath query parameter to root endpoint.

The scriptPath parameter value must specify path to stored query script file in SpectX resource tree. When script file is not found using specified path error is returned (see Example 5 above).

Note that the path starting with /user/... directory is resolved based on current user context (identified by Authorization Bearer token).

The result is returned to the same socket upon completion (i.e synchronously).

You can obtain curl utility command with correct values for headers and parameters from desired script file properties. Right click on script file in resource tree, choose Properties on the drop-down menu and select API example tab.

Example 6.

$ curl -XPOST -G \
    -H "Accept: application/csv" \
    -H "Authorization: Bearer c04069088ed66696" \
    -d "scriptPath=%2Fuser%2Fexamples%2Fdoc%2Fgetting_started%2Fexample3.sx" \
    -d "opt.timezone=%2B02%3A00" \
    http://localhost:8388/API/v1.0/

HTTP/1.1 200 OK
Date: Tue, 09 Jan 2018 15:25:13 GMT
Content-Type: application/csv; charset=UTF-8

"period";"success";"rejects"
2015-12-31 00:00:00;1;1
2015-12-31 00:00:00;1;1
2015-12-31 00:00:00;1;1
2015-12-31 00:00:00;1;1
2015-12-31 00:00:00;1;1
2015-12-31 00:00:00;1;1
2016-01-07 00:00:00;1;1

Execute Parametrized Query

Stored query script (a view) can have input arguments. When executing the view (i.e performing a select from a view) over the API, the input arguments can be supplied as uri query parameters.

The argument values must be specified using literal notation only.

Example 7. Let’s use the following view:

/user/examples/doc/user_manual/views/example4vargs.sx
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
init(
  ip::IPADDR,                                     //mandatory input argument (uninitialized)
  since:TIMESTAMP('2016-03-14 00:00:00 +0000')    //optional input argument (initialized to a default value)
);

$pattern = $[/shared/patterns/apache.sxp];
@list    = LIST('sx:/user/examples/data/apache_access.log.sx.gz');
@stream  = PARSE(pattern:$pattern, src:@list);

@stream
 .filter(clientIpv4 = $ip AND timestamp >= $since)
;

and execute it over the API using curl command line utility. We must add the input arguments to the skeleton query we obtained from “API access” properties tab. Note that we must urlencode the starting timestamp value, as spaces are not allowed. You can copy and paste the command below (just replace the Authorization: Bearer header value with your own API access token). You should observe 74 Json formatted records returned.

$ curl -XPOST -G \
 -H "Accept: application/json" \
 -H "Authorization: Bearer c04069088ed66696" \
 -d "scriptPath=%2Fuser%2Fexamples%2Fdoc%2Fuser_manual%2Fviews%2Fexample4.sx" \
 -d "opt.timezone=Europe%2FTallinn" \
 http://localhost:8388/API/v1.0/ \
 -d "ip=172.16.4.11" \
 --data-urlencode "since=2016-03-14 21:00:00.000 +0200"

Execute Client Specified Query

Query script can be specified in the request body. The scriptPath parameter must specify valid path to a directory in the resource tree. The directory is used by engine as temporary script location. The path must be accessible for user executing the query.

The result is returned to the same socket upon completion (i.e synchronously).

Example 8.

$ curl http://localhost:8388/API/v1.0/?scriptPath=/system/ \
    -H "Authorization: Bearer 51e73367904424f7" \
    -H "Accept: application/csv" \
    -d "dual(1).select(i, ip, t);"

HTTP/1.1 200 OK
Date: Tue, 09 Jan 2018 16:27:42 GMT
Content-Type: application/csv; charset=UTF-8

"i";"ip";"t"
0;0.0.0.0;2018-01-09 20:05:50

The Using SpectX API from Python sample provides Python implementation of executing client specified query.