SigScalr Inc., a developer of an open-source observability application designed to process large volumes of data, is unveiling its latest project backed by $1.76 million in pre-seed funding. The platform, known as SigLens, is reportedly capable of executing queries at speeds 50 times faster than Apache Clickhouse and over 1,000 times faster than Elasticsearch.
The company’s proprietary technique, referred to by SigScalr’s founder and Chief Executive, Kunal Nawale, as micro-indexing, apparently makes these speeds possible. This method attaches an index to each column, with dynamic compression adapting as immutable data streams in. Traditional databases regularly incur problems with large and unwieldy indexes. However, SigLens’ micro-index effectively breaks columnar files into smaller segments for quicker searches, despite its index only being 1/100th the size of a conventional one.
Nonetheless, its technology only allows for appending, meaning that log lines cannot be retroactively altered. However, Nawale maintains that in most observability contexts, editing the log line is unnecessary.
SigLens is not just about speed, but also scalability, as it can support thousands of concurrent queries on terabytes of data, all while maintaining a sub-second response time. It offers flexibility in how it can be run – either on-premises to avoid cloud data egress fees or as a Software-as-a-Service application. Furthermore, its query language is compatible with every major observability tool. Nawale suggests that the on-premises version could save users from excessive fees that can amount to around $15,000 a month.
The idea for SigLens was conceived while Nawale was working at Salesforce and using a big-data platform for collecting, searching, and analyzing machine-generated data. Discontent with the platform’s inefficiency, Nawale created SigLens to offer a more efficient solution specifically tailored for observability use cases.
Having only been released to open source a few weeks ago, SigScalr expects to generate most of its revenue from the hosted version of SigLens and from advanced security features included in an enterprise edition. The pre-seed funding round was led by Scribble Ventures Management LLC with co-investments from WestWave Capital LLC and Forward Slash Capital.
Source: Observability startup SigScalr plans to revolutionize log search.
