sa://

Source Agent

The Source Agent is a SpectX component providing secure, optimized, high-speed access to data stored in on-premises servers.

The schemes (sa:// and sas:// ) refer to the Source Agent RESTful protocol. It is a client-server protocol where the Source Agent server component is installed on the data server and the client for it is the SpectX engine. Both counterparts need configuration, we’ll discuss the client-side here (details on server configuration can be found in SourceAgent Admin Manual Configuration section).

The sa:// scheme uses plaintext, whereas sas:// uses TLS on the transport layer.

Using sa:// in SpectX

The host-less forms sa:/// and sas:/// always refer to a root directory on a Source Agent server running at port 8388 on the loopback interface 127.0.0.1 of the machine SpectX is running on.

To access Source Agent servers running elsewhere, you specify their name in the URI (and possibly a port number if it differs from the standard one (8388)), like: sa://log-server:8080/ or sas://secure-log-server/.

To refer to data in a defined data store, its name should be used instead in the URI.

Datastore configuration

UI

Configuration parameters for both sa:// and sas:// datastore definition:

Name Description
Store name unique name among all defined DataStores.A string. Mandatory parameter
Host hostname or ip address (ipV4 or ipV6) of the SA server. A string. Mandatory parameter
Port port the SourceAgent server is listening on. Integer. Optional, default is 8389
Authentication key shared secret between SA client and server. A string. Optional.
Is cacheable enables caching data by Processing Units.
Hot cache period limits time related data caching to the period specified.
Read ACL specifies blob read ACL

Filesystem

sa:// and sas:// datastore definition files are of JSON structure of the following formats correspondingly (optional parameters can be omitted):

{
  "type": "SA",
  "saStore": {
    "host": "<host>",
    "port": <port>,
    "authKey": "<authKey>",
    "lockTtl": "<lockTtl>",
    "isCacheable": <isCacheable>,
    "hotCachePeriod": "<hotCachePeriod>",
    "connectTimeout": <connectTimeout>,
    "readTimeout": <readTimeout>,
    "maxErrorRetries": <maxErrorRetries>,
    "maxRedirects": <maxRedirects>,
    "userAgent": "<userAgent>",
    "acl": {<rACL>}
  }
}
{
  "type": "SAS",
  "sasStore": {
    "host": "<host>",
    "port": <port>,
    "authKey": "<authKey>",
    "lockTtl": "<lockTtl>",
    "isCacheable": <isCacheable>,
    "hotCachePeriod": "<hotCachePeriod>",
    "connectTimeout": <connectTimeout>,
    "readTimeout": <readTimeout>,
    "maxErrorRetries": <maxErrorRetries>,
    "maxRedirects": <maxRedirects>,
    "userAgent": "<userAgent>",
    "acl": {<rACL>}
  }
}

where

  • <host> is the hostname or IP address (ipV4 or ipV6) of the target Source Agent host. A string. Mandatory parameter.
  • <port> port the SourceAgent server is listening on. Integer. Optional, the default is 8389.
  • <authKey> shared secret between SA client and server. A string. Optional.
  • <lockTtl> is a value for desired lock TTL to be asked from Source Agent server by processing units when locking processing blob content during query execution, in milliseconds. Optional. A non-negative long integer.
  • <isCacheable> enables caching data by Processing Units. A boolean (“true” or “false”)
  • <hotCachePeriod> limits time-related data caching to the period specified. A time period.
  • <connectTimeout> is a connection timeout in milliseconds. A timeout of zero is interpreted as an infinite timeout. The default is 10000. A non-negative long integer.
  • <readTimeout> is a read timeout in milliseconds. A timeout of zero is interpreted as an infinite timeout. The default is 60000. A non-negative long integer.
  • <maxErrorRetries> is number of times the SpectX tries to get access to a requested resource in case it is inaccessible due to network problems until giving up. The default is 3. An integer.
  • <maxRedirects> is number of times the SpectX tries to get a requested resource in case it is inaccessible due to network problems until giving up. The default is 3. A non-negative integer.
  • userAgent is a value for User-Agent to be used when contacting the target Source Agent server. The default value is the string “SpectX” and the current software version. A string.
  • <rACL> is a definition of a blob read ACL for the datastore. A map.