The Most Scalable Splunk Alternative for Log Analysis

When talking to infosec experts about their log-related endeavours, Splunk is a household word. We frequently find ourselves patiently and passionately explaining the differences between this well-known giant and SpectX. Here’s a write-up to go more into details than just stressing the limitations of pricing and data import.

Analytics. Both Splunk and SpectX offer analytics based on structured queries. This allows a wide range of manipulations with data (aggregations, joins, unions, rich library of functions that can be extended by end users). However, while Splunk lets you execute a single search query, SpectX gives you a language that combines an unlimited number of different queries into an executable script. This enables performing complex exploratory analysis, repeatable and easy to read.

Data access. Splunk requires data to be ingested (pulled or pushed) into the system using a wide variety of forwarder components. Setting up a forwarder is a time-consuming process. Data must also be indexed before it can be queried.
SpectX can retrieve stored data using a data reference. The data queried this way might have been stored in the cloud, web, on-premise servers, databases or distributed file systems. SpectX does not require any indexing for executing queries. This means instant access to all the data the end user is allowed to access. The time spent on preparation tasks is dramatically shorter compared to Splunk. The analysis scope can be chosen by the end user on the spot and is not limited to ingested-indexed data.
Working with external data sources (e.g. pulling threat intelligence feeds, analyzing public data from the Internet or acquiring data from relational databases) is a "native" part of SpectX not requiring any third party plugins.

Exploratory Analysis for Stored Data (metadata analytics). The unique approach to data access allows using SpectX for exploring and analysing data assets stored in various locations. Get a regular overview of what data is being stored, for how long, what is the amount, is the retention policy enforced, etc. 

Handling Unstructured Data. Both Splunk and SpectX have a so-called schema-less approach, where fields are extracted from data at query time. However, with SpectX you can detect changes in the expected data structure. This is essential for working with cybersecurity/infosec related scenarios where the unexpected is a given.  Avoiding expensive and prohibitive measures of change control within an organisation is one of the greatest benefits of this flexible processing of volatile structured data (e.g. application logs). Read more on processing machine-generated data here.
SpectX'  language for data extraction and transformation with interactive pattern development assistance a simple and intuitive. There are many features built specifically for quick and creative incident investigation.

Hardware Costs. When it comes to analytics functionality, Splunk relies exclusively on indexing ingested data. This has several impacts on the hardware:
i) CPU resources are shared between search and indexing,
ii) the rate of ingested events is limited by computation of indexes and
iii) indexes use considerable storage space (as much as or more than data).
SpectX does not rely on indexing, therefore:
i) the rate of collected events is limited solely by hardware write speed of the storage,
ii) the CPU resources are solely dedicated to analytics, 
iii) there is no overhead in using storage space.
Combined with SpectX superior performance, the costs on capex/opex hardware are significantly lower. This opens up new use-case scenarios because analysing large amounts of unstructured data can be done on devices with low computing power, eg laptops.
High performance is not just nice-to-have. The main impact for an analyst using SpectX is the freedom of experimenting. Heavily reduced preparation efforts and great performance make SpectX a perfect tool for fast and efficient fact-based business decisions

Add-ons. Splunk has different add-on products, such as behavioural analytics module (Splunk UBA), Enterprise Security, Adaptive Response, allowing to use it as an advanced SIEM. SpectX is not a SIEM but you can use it as one if you're up to spending some time on integrating it with existing (open-source) monitoring/alerting solutions. 

Pricing. SpectX' price depends on the number of CPU cores allocated to the installation. The price of Splunk depends on the data volume ingested. In other words - with Splunk, you need to carefully plan in advance, which data you'll need to prepare to answer a question in the future. A SpectX installation allows you to shoot queries on any (amounts of) raw data whenever the need arises. 

Back to articles