About Siglake¶

Siglake is a horizontally-scalable, OTLP-native log analytics platform built on Parquet v2, Apache Iceberg, and DataFusion, written in Rust. It is developed by Limnion AI and licensed under the Apache License 2.0.

Status¶

This documentation covers Siglake v0.1.0. See the release list for current releases. The core platform is complete and AWS-validated end to end: ingestion, WAL, Iceberg storage, distributed SQL query, the WAL consumer interface, Helm charts, a Kubernetes operator, and Terraform/EKS deployment.

It has been validated at 200 GB (394 M rows) and 1 TB (2.02 B rows) across roughly 100 AWS benchmark rounds. A four-ingester fleet sustained ~415 K rows/s end to end, and exact row counts held through node crashes.

Siglake focuses on telemetry storage and query. Dashboards, alert evaluation, notifications, and on-call workflows integrate through external tools; these are intentional boundaries of the core product. See Grafana and Jaeger for dashboard and trace integrations. Search uses time ordering and SQL filters. The scope and limitations page records configuration defaults and operational constraints.

History¶

The project was called knulps until 2026-06-12. Pre-rename docs are kept in the internal history, and older git history uses the old name. It is the same system.

The build was phased, each phase shipping compiling, clippy-clean, tested code:

The public design record lives in the source repo under docs/ as DESIGN_*.md. The round-by-round AWS validation reports behind the performance numbers are internal, not part of the public source tree. Public supporting material also includes the OTLP ingest performance note. The comparisons and methodology will be published in the benchmarks repository once it goes public shortly after launch. Until then, the charts are in the Siglake README's Performance section.

Design principles¶

Six commitments show up repeatedly in the code.

Storage stays open. The warehouse is plain Iceberg on Parquet, so any Iceberg reader can query it directly. There is no proprietary format and no second copy.

Accelerators degrade to a scan. Every accelerator is optional by construction. A stale side aggregate, an unrecognized footer version, an unreadable bloom: each falls back to a scan. Pruning artifacts fail closed, because a false negative silently drops rows from a result.

Every persisted format carries a version, in the name the artifact is stored under (siglake.group_counts.v1), in the payload bytes, or both. A reader that does not recognize a version ignores the artifact rather than guessing. Files are never rewritten for format reasons alone; tables converge as compaction rewrites them for its own reasons.

Caches are snapshot-keyed, never TTL-expired. A cache entry must be a pure function of (table, snapshot, query) and invalidate on commit. A TTL'd whole-result cache silently serves stale leading-edge answers, and is prohibited.

Measurements carry their context: scale, topology, and warm or cold state. When a benchmark round turned out to be measuring a result cache rather than the engine, that was recorded as a methodology failure and a cache-bypass switch was added. See Performance.

What is not done is documented. Limitations is maintained as a first-class page and updated with the code that changes it.

License¶

Apache License 2.0. See LICENSE in the source repository.

The vendored Iceberg forks under third_party/ retain their upstream Apache-2.0 LICENSE and NOTICE files. See third_party/README.md and NOTICE.

Links¶

• Source: https://github.com/siglake/siglake
• Benchmarks: the comparisons and methodology will be published in the benchmarks repository once it goes public shortly after launch. Until then, the charts are in the Siglake README's Performance section.
• Security policy: SECURITY.md in the source repository.
• Code of conduct: CODE_OF_CONDUCT.md.