Release Notes

v1.4.52 (2020-02-29)

Changed:
  • SpectXDesktop uses integrated GPU on OSX
Fixed:
  • Script files can not be opened after setting user timezone to PST

v1.4.51 (2020-02-25)

Added:
  • Option to hide inaccessible folders in resource tree, configurable per user
  • CTRL + double click, CTRL + ENTER on file resource in resource tree opens it in new tab even if one is already open
  • Operators IS TRUE, IS FALSE, IS NOT TRUE, IS NOT FALSE
  • ‘U’ TIMESTAMP parser format letter for unnecessary non-numeric chars
  • Configuration setting engine.da.http.user-agent
  • Added left join query processing command
Fixed:
  • LIMIT() following PARSE() does not accept numbers bigger than 0x7fffffffffffffff
  • PARSE() does not accept NULL as value for pattern, archive_src and rc parameters

v1.4.47 (2020-01-29)

Changed:
  • JVM upgraded to 1.8.0_242 (Windows64, Linux64, OSX64) and 1.8.0_232 (Arch Linux ARM 64)
Added:

v1.4.46 (2020-01-14)

Added:

v1.4.44 (2019-12-10)

Changed:
  • Chart PNG export at double scale for better quality
Added:
  • Support for ElasticSearch 7.x
Fixed:
  • Chart x-axis labels are masked on left and right edges

v1.4.41 (2019-12-03)

Changed:
  • Datastore creation and configuration flow in WGUI:
  • Reordered buttons in dialogs such that primary button is on the left.
  • Removed ‘Log File’, ‘Data File’ and ‘Text File’ from the ‘New’ menu and resource tree context menus.
  • Double click on a datastore in the resource tree opens the data browser, ‘Configure’ in context menu opens configuration
Added:
  • ES and SAVE_ES have query.es.proxy and query.save_es.proxy options with fallback to http(s).proxyHost and http(s).proxyPort system properties
  • IPNET parser
  • new UDF functions ARRAY_FIRST, ARRAY_LAST, ARRAY_UNIQ
Fixed:
  • API fails to serve empty resultset.
  • WGUI unintended session terminations.

v1.4.40 (2019-11-25)

Changed:
  • DataStore Editor: “Browse” functionality retired in favor of simple connectivity test:
    • “Browse” button renamed to “Test Connectivity”, which when pushed performs an attempt to connect to current datastore using realtime values of configuration parameters and list files in its root directory.
    • The “Test Connectivity” button is disabled for dislocated datastores.
  • Improved responsiveness of map visualization with a large number of features
  • LOAD and LOAD_HTTP UDF builtin functions removed in favor of a new FETCH function
  • SpectX configuration file:
    • ${PROP} construction in configuration file variables, refers corresponding environment variable first, and if it is undefined, refers then corresponding system property
    • configuration parameters expecting directory paths do not support empty values
    • log directory gets created if it is specified but does not exist
Added:
  • TIMESTAMP_NANO matcher
  • ‘f’ TIMESTAMP parser format letter for fractional seconds
  • An existing pattern file can be dragged from resource tree to an open pattern editor
  • Resultset column layout is retained across query executions.
  • Datastore Read ACL retains comments
  • Saved patterns in subfolders are also considered for Input Data Browser -> prepare pattern
  • SpectX API: capability to retrieve the schema of resultsets (for tables, queries)
  • File name filter in DataBrowser’s data store view
Fixed:
  • Resultset column selection is not vertically scrollable.
  • Map initial focus fails for features with invalid coordinates. Now Map supports features with invalid coordinates (clamped to max/min values)

v1.4.39 (2019-09-11)

Changed:
  • Min password length changed from 5 to 8 characters
  • Order of searching of datastores in resource tree changed from system, shared, user to user, shared, system.
Fixed:
  • UI: System Status > Running Queries: sorted state gets lost after a second
  • ISO8601 Timestamp Format Matcher does not honor RFC
  • Globbed blob read ACLs with exact match for gzip files in SA/SAS datastores deny processing of these files if these get indexed on SourceAgent side and get cached on SpectX side.

v1.4.38 (2019-08-27)

Added:
  • Support for reading data from 7z and Rar (v4 and below) archives. See more at Working With Archives.
  • DataAccess/S3: bucket listing response version mismatch handling

v1.4.37 (2019-07-05)

Added:
  • SpectX UI: last login time to users table in Users tab under “Admin - Users and Groups” menu
  • Possibility to configure startup at boot in Linux (with Systemd/SysV-style init/Upstart init systems)
  • Filter(ES_QUERY()) support for ElasticSearch 5.6.*
Changed:
  • Linux/OSX startup script spectx-init.d.sh renamed to spectx.sh . Helper scripts spectx.common.sh removed.

v1.4.36 (2019-06-13)

Added:
  • IWA authenticator: configurable LDAP group name transformation and possibility to abandon group membership information retrieval with user information LDAP query.
  • IWA authenticator: fallback to basic authentication which allows users from machines not joined to Active Directory domain to log in to SpectX with their domain username and password
  • SpectX default environment setup script for linux/osx platforms sets umask to 0077.
  • JRE-bundled packages with SpectX for Arch Linux ARM 64bit (aarch64)
  • UI: Chart supports multiple x-axes to visualize two sets of superimposed value ranges
  • Parsing JSON objects now allows exporting selected members directly to resultset columns using JSON{…}(flat=true) config parameter
  • User Defined Functions can now include multiple statements.
  • New functions: GZIP(), GUNZIP()
Changed:
  • Improved listing operations optimisation in cloud (Amazon/Azure/Google) data stores.
  • Blobs disallowed from reading by blob Read ACL are not displayed in data browser and get not included in listing result by default; new settings for both data browser and listing can be used to configure the behavior correspondingly
Fixed:
  • Globbed blob read ACLs with exact match for gzip files in SA/SAS datastores deny processing of these files if these get indexed on SourceAgent side.
  • In Firefox 67+ submenus of menu ‘Save’ > ‘Result As’ do not work
  • Chart legend does not respond to resize when many columns are displayed

v1.4.35 (2019-04-23)

Added:
  • IWA authenticator: support for multiple search bases in user and group LDAP query filter expressions
  • Elasticsearch support: indexes & structure listing, native search using Query String
  • DataAccess/S3: support for path-style access to buckets
Changed:
  • Elasticsearch support: enhanced control over large responses from Elasticsearch, optimized search predicate delegation by object subfields
Fixed:
  • UI: Incorrect group list rendering in user properties window in case of list sizes greater than 40.

v1.4.34 (2019-04-01)

Fixed:
  • Elasticsearch support: connectivity issues to 6.6+ clusters

v1.4.33 (2019-03-29)

Added:
  • UI:
    • Update download progress indication and cancelling possibility in SpectX Update dialog.
    • Data Store editor: glob patterns can be used in data store Read ACL definitions, in addition to prefixed uri paths.
    • Resource tree filter. Defaults to filename filter, c:<string> searches for files containing ‘string’, and t:<hour/day/week> searches for files modified last hour/day/week.
    • Result set _raw_text column contents exportable to file from ‘Save’ and right click context menus
    • Query filter creation from resultset selection supports time ranges for timestamp columns
  • Support for reading data from ZIP archives. See more at Working With Archives.
  • UserAdmin role.
Changed:
  • Engine:
    • Increased processing speed of Bzip2 files on Intel-based 64 bit OSX and Linux platforms by 20%.
  • UI:
    • Improved editor auto-completion in nested function calls
    • Data Store Editor: ‘.’ symbol is allowed in store name
Fixed:
  • UI:
    • Map feature properties popup displays correct values for tuple type columns
    • Export of chart with column names containing non-ASCII characters

v1.4.31 (2019-02-22)

Changed:
  • SpectX configuration keys engine.fs_access and engine.fs_unmanaged_access are deprecated. Instead, new configuration keys engine.da.protocol.<protocol> can be used to define usage permissions for each data access protocol separately.

v1.4.30 (2019-02-11)

Added:
  • DataAccess/HDFS: High availability clusters accessibility support

v1.4.29 (2019-02-08)

Added:
  • Functions: INDEXOF, ROT13, ESCAPE_OCTAL
Changed:
  • OpenJDK 8 HotSpot Java Runtime Environment is now supported
  • DataAccess/HDFS: improved accessing secure HDFS clusters with multiple Kerberos authentication realms configured
Fixed:
  • On Windows, Data Browser started by pressing Browse button in data store configuration editor window for previously stored configuration, fails to connect to the data store.

v1.4.28 (2019-01-10)

This release contains bug fixes, a few new features and some improvements.

Added:
Changed:
  • Reading data from relational databases: refactored input stream commands
  • UI:
    • Data Store editor: the browse-button uses real-time values in the editor form to connect to the data store. Browsing is restricted to the currently editable store only.
    • DataBrowser: keyboard navigation improved. Arrow, Pg Down/Up, Enter and Backspace keys can be used for navigation in all views.
    • “Admin - Users and Groups” menu: added descriptive text to the User/Group/ExtGroupMapping deletion dialogs.
    • “System status” menu: remastered Effective Settings view.
    • Script and Pattern editors: removed ctrl - shift - arrow tab navigation, text of displayed error messages made selectable.
    • The full name of a logged in user (if set) is used instead of a username in the rightmost menu at the top bar
    • Background version checker: connection exception logging with debug log level (previously: error log level).
  • Engine:
    • Improved processing performance of concatenated GZ/ZZ/PiGZ/PiZZ files.
    • Increased processing speed of GZ/ZZ/PiGZ/PiZZ files on Intel-based 64 bit OSX and Linux platforms by 20%.
Fixed:
  • Listing queries for cloud (Amazon/Azure/Google cloud) targets with the ‘includeContainers’:true argument.
  • Removed unnecessary creation of the folder “users” under the path specified by the config parameter sx.user_data.dir.
  • Failure to attempt to create own log directory on the very first run fixed in both SpectX and SourceAgent startup Windows scripts.