gsuite://

Note

gsuite:// is available only with the Enterprise license.

The scheme gsuite:// provides access to Activities audit logs and Reports available through G Suite Admin SDK Reports API.

SpectX retrieves audit events and reports from the G Suite Admin Reports API depending on Google’s data availability lag times and update frequency (see Advanced Configuration for details).

Note

When retrieved with SpectX, the log files are stored in the local file system [3]. It is the system administrator’s task to maintain the availability of disk space (archive older log files, etc).

Note

Google does not provide G Suite Admin Reports in real-time. The lag times may vary between few minutes to several days, depending on the Report.

The log is a list of JSON records separated with a new line. When downloading activity events or usage report, SpectX automatically converts the records for better log readability and easier information access before saving these to files.

Using gsuite:// in SpectX

The gsuite:// protocol does not allow anonymous access. To query the logs with SpectX, configure access by creating a new datastore and use the URI to that datastore in the queries.

SpectX uses Google’s Server-to-Server Oauth 2.0 protocol for authentication and authorization. You need a service account to authenticate SpectX to G Suite Admin API. The service account must be granted the domain wide authority in order to access end-users activity logs.

Hint

See step-by-step instructions for configuring access to G Suite.

Datastore configuration

UI

Configuration parameters for gsuite:// datastore definition:

Name Description
Store name Unique name among all defined DataStores
Private Key File Content
The content of the private key file (JSON) generated at
NB! The entire content of the file must be copied and pasted!
Admin Role Email The email address of an Admin role [1] user with Reports access privilege.
Enabled Applications
Enables/disables retrieving Activity audit events for
Enabled Reports
Enables/disables retrieving G Suite applications
Customers and Users Usage Reports
Read ACL specifies blob read ACL

Advanced Configuration

Activity logs settings

Name Description
Time limits
Specifies availability lag times and update frequencies [2]
for available G Suite application audit logs.
Time Zone
The timezone to use for constructing audit log file URIs.
Default is the User Properties Timezone.
Hour in
Specifies whether locally stored log file timestamp hour value should be included in the
file name, path or both [3].

Reports settings

Name Description
Time limits
Specifies availability lag times and update frequencies [2]
for G Suite application Customers and Users Usage Reports.

Note that it is not possible to change default time zone (GMT) for Usage Report as it is not supported by G Suite Reports API.

Common settings

Name Description
Log Type in
Specifies whether application name should be included in locally stored
file name, path, or both [3].
[1]Only users with access to the Admin APIs can access the Admin SDK Reports API, therefore the service account needs to impersonate one of those users to access the Admin SDK Reports API. See more at https://developers.google.com/admin-sdk/reports/v1/guides/delegation#delegate_domain-wide_authority_to_your_service_account
[2](1, 2) The update frequency reflects how often an Activity Report is updated. The value is empirically determined and should not exceed the availability lag time. It is used to re-fetch an audit log which last fetch time is within the availability lag and the time since last fetch is bigger than update frequency.
[3](1, 2, 3) Locally stored log files are placed in the following directory structure: $SPECTX_HOME/engine_data/google_api/<datastore_name>/activity/<application_name>/yyyy/MM/dd/<HH>/yyyyMMdd<HH>.<application_name>.json. Here the <application_name> (G Suite application) and <HH> (hour value of the timestamp) may appear in the path or filename or both.

Filesystem

gsuite:// datastore definition file is a JSON file with the following key-value pairs (optional parameters can be omitted):

{
  "type": "GS",
  "gsStore": {
    "email": "<email>",
    "privateKey": "<privateKey>",
    "timeZone" : "<timeZone>",
    "activityTimeWindow" : "<activityTimeWindow>",
    "usageRepTimeWindow" : "<usageRepTimeWindow>",
    "fileNameIncludesApp" : <true|false>,
    "filePathIncludesApp" : <true|false>,
    "fileNameIncludesHour" : <true|false>,
    "filePathIncludesHour" : <true|false>,
    "applications" : "<applications>",
    "reports" : "<reports>",
    "connectTimeout": <connectTimeout>,
    "readTimeout": <readTimeout>,
    "maxErrorRetries": <maxErrorRetries>,
    "userAgent": "<userAgent>",
    "acl": {<rACL>}
  }
}

where

  • <email> email address of a user having access to Google Admin APIs. The service account accesses the API on behalf of this user. A string. Mandatory parameter.
  • <privateKey> Google service account private key file content (JSON), encoded in Base64 format. A string. Mandatory parameter
  • <timeZone> time zone to use for constructing audit log file URIs (does not apply to usage reports which always uses GMT). A string. Optional. Defaults to system’s default time zone
  • <activityTimeWindow> a period from now specifying the time range of audit events to retrieve. A time unit. Optional. Defaults to “180 days”.
  • <usageRepTimeWindow> a period from now specifying the time range of usage reports to retrieve. A time unit. Optional. Defaults to “450 days”.
  • <fileNameIncludesApp> indicates if the log file name should include the audit log type when the URI is constructed. Boolean. Optional. Defaults to true.
  • <filePathIncludesApp> indicates if the log file path should include log type when the URI is constructed. Boolean. Optional. Defaults to false.
  • <fileNameIncludesHour> indicates if the log file name should include the timestamp’s hour value when the URI is constructed. Boolean. Optional. Defaults to false.
  • <filePathIncludesHour> indicates if the log file path should include the hour value when the URI is constructed. Boolean. Optional. Defaults to false.
  • <applications> Comma-separated list of colon-separated key-value pairs. Key specifies audit log “uri” name, value is composed of |-separated triple: Google API audit log type, expected lag time (timeunit) and up-to-date-age (timeunit). A string. Optional. Defaults to the list of all 17 audit log types available through the Reports API with lag times as per Google docs and up-to-date-ages set to minimum meaningful values
  • <reports> list of the same format as <applications> above, with keys “customer” and “user” for specifying available usage reports and related time limits
  • <connectTimeout> Connection timeout in milliseconds. A timeout of zero is interpreted as an infinite timeout. Optional. The default is 10000. A non-negative long integer
  • <readTimeout> Read timeout in milliseconds. A timeout of zero is interpreted as an infinite timeout. Optional. The default is 60000. A non-negative long integer
  • <maxErrorRetries> The number of times SpectX tries to get access to a requested resource before giving up in case it is inaccessible due to network problems. Optional. An integer. The default is 3.
  • userAgent is a value for software agent name to be used when communicating with the Google API. Optional. A string. Default value is composed of a string “SpectX” and a current software version designator.
  • <rACL> is a definition of a blob read ACL for the datastore. A map.

Troubleshooting

If you encounter this error message:

Access denied. Make sure:
- the service account is authorized domain-wide with scope "<scope>"
- the user account "<email_address>" is member of your G Suite domain
- the user account is granted Reports Admin console privilege

when accessing gsuite:// datastore, then make sure you have performed:

  • the Step2 of the G Suite API Access Configuration guide to authorize the service account to make API calls to G Suite Reports Service on behalf of end users in your domain
  • the Step3 of the guide to choose an account with Reports Admin console privilege to be used by the service account to access the API on behalf of.

If the error message is like below:

Invalid email address "<email_address>". Make sure it belongs to a user account in your domain

then most probably you’ve made a typo when entering the email address of the account into Admin Role Email. Make sure the email address belongs to a user account in your domain.

If the error message is like below:

Access denied. Make sure user "<email_address>" is not suspended

then check if the user account used by the service account to access the API on behalf of is active or suspended in Google Admin console.

If the error message is like below:

Admin SDK has not been used in project <id> before or it is disabled. Enable it by visiting
https://console.developers.google.com/apis/api/admin.googleapis.com/overview?project=<id>
then retry. If you enabled this API recently, wait a few minutes for the action to propagate
to our systems and retry.

then check if the Admin SDK is enabled in Google Cloud Console as suggested in the error message above.