Table schemas¶

Use this page to look up every built-in column, field ID, nullability rule and type that Siglake writes to Arrow and Apache Iceberg. Every table uses stable Iceberg field IDs. A timestamp column is a microsecond timestamptz. In the events table, the required timestamp_ns sibling stores the event's exact nanosecond value. This layout remains compatible with Iceberg format version 2 readers. See External engines.

Run siglake migrate-schema to add newly declared columns to an existing table. The command does not remove, rename, reorder or change existing columns.

events¶

The events table stores OpenTelemetry Protocol (OTLP) logs. Siglake partitions it by day(timestamp) and sorts it by (timestamp, timestamp_ns). The second column orders events that share the same microsecond timestamp. Field IDs 1 through 8 are built-in columns. Promoted OTLP attributes start at field ID 9 and remain nullable.

Which columns are always present, and which are derived from attributes¶

The shape of the events table is eight reserved columns, plus promoted columns from field ID 9 up. Seven of the eight are always present and non-null: timestamp, timestamp_ns, raw, host, source, sourcetype and index. Only attributes is nullable.

Three are taken straight from the OpenTelemetry Protocol (OTLP) record: timestamp, timestamp_ns and raw. Four are derived from OTLP resource or record attributes, with the fixed fallbacks below: host, source, sourcetype and index. attributes stores whatever OTLP attributes are left. Promoted columns are derived from attributes too, and are always nullable.

Column sources¶

• timestamp: Event time in microseconds. Siglake floors the event's nanosecond value so ordering remains monotonic on either side of the epoch.
• host: Resource attribute host.name, or "unknown" when absent.
• source: Resource attribute service.name, scope name or "otel", in that order.
• sourcetype: Record attribute sourcetype, or "otel:logs" when absent.
• index: Record attribute index, or "main" when absent.
• raw: Log body.
• timestamp_ns: Original OTLP time_unix_nano value. This required column breaks ties in the sort order.
• attributes: JSON object containing residual OTLP attributes.
• promoted: Promoted columns, numbered in declaration order from the base the table above gives, always nullable.

attributes is nullable because it was added after v0 tables existed; rows written before the migration read back null.

query_audit¶

Each retained query_audit row records the caller, endpoint, SQL, format, priority, duration, estimate and outcome. Use query_audit to inspect query cost and failures. siglake audit-rotate manages its growth: the bare command deletes all historical rows, while --max-age-secs keeps the rows and expires old snapshots and files.

Query audit fields¶

The query server writes these fields for each retained row.

The timestamp column uses the same microsecond timestamptz type as other Siglake tables. This table has no timestamp_ns column and sorts by timestamp alone.

subject contains the authenticated subject or an anonymous marker. email comes from the JSON Web Token (JWT) when present. priority is interactive or batch. complexity is small, medium, large or huge. truncated records whether the query hit its row cap. status and error record the outcome.

Query audit retention¶

Each flush creates a commit, so this table can accumulate snapshots quickly. See Retention and deletes for rotation.

siglake audit-rotate without --max-age-secs drops and recreates query_audit, deleting its historical rows. With --max-age-secs, it retains the rows while expiring old snapshots and reclaiming unreferenced files.

Find your expensive queries:

SELECT query, count(*) AS runs, avg(duration_ms) AS avg_ms,
       max(estimated_bytes_scanned) AS max_bytes
FROM query_audit
WHERE timestamp >= now() - INTERVAL '24 hours'
  AND complexity IN ('large', 'huge')
GROUP BY query
ORDER BY avg_ms DESC
LIMIT 20;

User indexes¶

Index fields and promotion¶

The document mapping defines a user index table. Every index contains:

• the declared fields, in declaration order, with field IDs 1 through n;
• the required datetime field named by timestamp_field, used for partitioning and sorting;
• a nullable attributes column at field ID n+1. dynamic, the default, keeps unmapped attributes. lenient silently drops them. strict currently keeps them and counts affected rows in siglake_compactor_strict_residual_rows_total. See Limitations.

A user index has an exact-nanosecond timestamp_ns column only when its mapping declares one. Siglake uses it as a sort tiebreaker only when timestamp_field is timestamp. The built-in events and traces templates declare both fields.

Field types map as follows:

text accepts an optional tokenizer. User indexes have no unsigned integer type because these schemas must convert to Iceberg signed integers or floats.

See User indexes.