Architecture
How ClueLake is built.
Written for engineers evaluating this seriously. Where something is a design decision with a trade-off, the trade-off is stated rather than omitted.
Architecture
OpenTelemetry in. Your telemetry stays yours.
ClueLake is designed to fit the tooling you already use. Telemetry arrives over OTLP and leaves through query APIs, structured and readable by things that are not ours.
- 01Kubernetes workloadsyour clusters
- 02OpenTelemetry · eBPF · Hubblecollection
- 03ClueLake ingestauth, tenancy
- 04Policy · enrichmentcardinality, redaction, deployment context
- 05ClickHousecolumnar storage
- 06Query · correlationPromQL, logs, traces, SQL
- 07ClueLake portalinvestigation
See the paths your application does not instrument.
Where a cluster runs Cilium and Hubble, ClueLake can use eBPF flow data to understand traffic and service relationships without application changes. It is used for service relationships and network telemetry — not as a substitute for application tracing.
Your telemetry should not become a hostage.
OpenTelemetry in, query APIs out, and a documented storage model. Self-hosted, the data sits in your own ClickHouse.
The pieces
Storage
One columnar store of wide events. A log line is a span with no duration, so logs and spans share a table — storing them apart is what makes correlation a human activity rather than a query. Metrics are separate, because they arrive pre-aggregated and are queried by series over time.
Tenancy
The workspace is the first expression in every sort key and partition key. Tenancy is a physical property of the storage, not a predicate the query builder has to remember: a query that omits it reads nothing rather than reading everyone.
Ingestion
OTLP over HTTP, and Prometheus remote_write. Each batch is authenticated, placed, validated, filtered by attribute policy, checked against the cardinality budget, enriched with deployment context, metered and written. Refusals are recorded with the series or attribute at fault.
Enrichment
Deployment, image digest, build and commit are stamped at ingest from the platform's own records. They are never read from telemetry attributes, which the sending workload controls.
Cardinality
Budgets are per workspace and count-based. An over-budget series is refused by name and surfaced in the portal, rather than dropped quietly or billed silently.
Query
PromQL for the supported query set, log and trace search, and read-only SQL. Every query is costed before it runs and refused with an estimate if it exceeds the workspace ceiling.
Correlation
Deterministic, not statistical. Candidate changes are ranked by temporal proximity, affected service, shared deployment and image change. No claim is made that this is machine learning, because it is not.
eBPF and Kubernetes
Where Cilium and Hubble are present, flow data contributes service relationships and network telemetry. Feature detection decides whether the integration is enabled; its absence degrades the collector rather than breaking it.
Retention
Applied by the storage engine's TTL rather than by a sweep job. Today it is fixed: 30 days for logs and traces, 400 days for metrics. Setting it per environment is planned.
Query APIs
A Prometheus compatible API, log and trace search, and read-only SQL, all over HTTP with a query key. Telemetry stays readable by tools that are not ours.
Sending telemetry
Standard OTLP. No proprietary agent is required.
exporters:
otlphttp/cluelake:
endpoint: ${CLUELAKE_ENDPOINT}
headers:
authorization: Bearer ${CLUELAKE_INGEST_TOKEN}
service:
pipelines:
traces: { exporters: [otlphttp/cluelake] }
metrics: { exporters: [otlphttp/cluelake] }
logs: { exporters: [otlphttp/cluelake] }