Overview
Ingestion tasks create segments continuously. Without a guard, a single misconfigured table or the cluster as a whole can accumulate hundreds of thousands of segments, which slows queries through per-segment fan-out overhead and puts pressure on ZooKeeper and Helix metadata. StarTree Cloud includes a segment limit checker: a controller-side circuit breaker that tracks per-table and cluster-wide segment counts and automatically blocks new ingestion work when configured thresholds are reached. It runs inside the same resource-utilization framework as the disk utilization and primary-key count checkers, and it is enabled by default. This makes it easier to:- Stop a runaway table before it degrades query latency for the whole cluster
- Cap total segment metadata growth across all tables
- Recover automatically once segment counts drop back under the thresholds — there is nothing to reset
Key concepts
How segment counts are collected
On the controller’s periodic resource-utilization cadence (controller.resource.utilization.checker.frequency), the checker counts every table’s segments directly from Helix ideal state and caches per-table counts plus a cluster-wide total. No server calls are involved, so the collection is cheap at any cluster size.
Check verdicts
Whenever gated work is about to happen, the cached counts are evaluated in order:| Condition | Verdict | Effect |
|---|---|---|
| Cached counts are stale (older than one check interval) | UNDETERMINED | Work is allowed — the checker fails open rather than blocking on missing data |
Cluster-wide total ≥ maxSegmentsInCluster | FAIL | Every table is blocked |
Table count ≥ maxSegmentsPerTable | FAIL | That table is blocked |
| Otherwise | PASS | Work proceeds |
What a FAIL gates
AFAIL verdict gates the same operations as the other utilization checkers:
- Minion ingestion task generation
- Real-time ingestion
- Offline segment uploads
- External Table sync — the watcher consults the checker before each table and skips tables that are over the limit
Configuration
| Config key | Default | Scope |
|---|---|---|
controller.segmentLimitChecker.enable | true | Controller config, read at startup. Disabling requires a controller restart — contact StarTree support. |
controller.segmentLimitChecker.maxSegmentsPerTable | 350,000 | Controller config at startup, or cluster config at runtime. |
controller.segmentLimitChecker.maxSegmentsInCluster | 1,000,000 | Controller config at startup, or cluster config at runtime. |
The per-table default of 350,000 matches the documented capacity guardrail for External Tables, where each source Parquet file becomes one segment.
Tune limits at runtime
Both limit keys are watched as cluster configs, so they can be changed without a controller restart. API Reference:POST /cluster/configs · GET /cluster/configs · DELETE /cluster/configs/{configName}
Raise the per-table limit:
POST /cluster/configs
GET /cluster/configs, and revert to the default by deleting the key:
DELETE /cluster/configs/controller.segmentLimitChecker.maxSegmentsPerTable
The controller logs every applied change and revert:
Monitoring and troubleshooting
Count where a table stands withGET /tables/{tableName}/segments (the segment list length), or watch the controller log for the cluster-wide picture after each periodic count:
FAIL, new ingestion work for the affected table (or every table, for a cluster-wide breach) is skipped on each cycle and re-evaluated on the next one.
Recovering
Pick one, depending on whether the segment count is legitimate:- Reduce the segment count. Tighten retention so old segments age out, or merge small segments (see Merging Small Segments). For External Tables, aim for larger source Parquet files — see the operational guidance on file sizing.
- Raise the limit at runtime with the cluster config API above, if the cluster is sized for it.
- Disable the checker (
controller.segmentLimitChecker.enable=false) — a controller-config change requiring a restart; contact StarTree support. Prefer options 1 and 2.