Memory-Throttled FCFS Query Scheduler

The Memory-Throttled FCFS query scheduler is a first-come, first-served server query scheduler that reduces query concurrency when JVM memory usage crosses a configured threshold. Use it when a server can become memory constrained under concurrent query load and you prefer queueing new work over letting the server continue scheduling queries until the JVM is at risk.

Enable this only after validating it in the target workload. It changes server-side query scheduling behavior and can increase query latency while protecting memory headroom.

Class and Scheduler Name

Setting	Value
Scheduler name	`MemoryThrottledFCFS`
Scheduler class	`ai.startree.query.scheduler.MemoryThrottledFCFSQueryScheduler`

How It Works

The scheduler:

Queues incoming server query requests in first-come, first-served order.
Schedules up to the current maximum allowed running queries.
Checks JVM memory usage with Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory().
Enters throttling when used memory is above the configured percentage of Runtime.getRuntime().maxMemory().
Caps the maximum allowed running queries to the number already running, with a minimum of one.
Slowly increases the maximum allowed running queries after the memory condition clears.
Requests JVM garbage collection if throttling is sustained.

The scheduler starts with no throttling and allows all query runner threads to be used. Throttling only starts after the memory threshold is crossed.

Enable the Scheduler

Set the scheduler name in the Pinot server configuration:

pinot.query.scheduler.name=MemoryThrottledFCFS

Configuration

Server Config	Default	Description
`pinot.query.scheduler.name`	Not enabled by default	Set to `MemoryThrottledFCFS` to enable this scheduler.
`pinot.query.scheduler.throttle.memoryUsagePercentageThreshold`	`95`	Memory usage percentage that triggers throttling. Must be greater than `0` and less than `100`.
`pinot.query.scheduler.throttle.maxRunningQueriesUpdateFrequencyMs`	`1000`	Minimum interval for increasing the allowed running-query count after throttling clears.

Example:

pinot.query.scheduler.name=MemoryThrottledFCFS
pinot.query.scheduler.throttle.memoryUsagePercentageThreshold=90
pinot.query.scheduler.throttle.maxRunningQueriesUpdateFrequencyMs=1000

Use a lower threshold to preserve more memory headroom. Use a higher threshold to allow more concurrency before throttling.

Tuning Example

For a server with a 16 GB JVM heap and a threshold of 90, throttling begins when used heap is above roughly 14.4 GB:

usedMemoryThreshold = Runtime.maxMemory * 90 / 100

When throttling starts:

Already-running queries continue.
New queries wait in the scheduler queue.
The scheduler lowers the maximum allowed running queries to the current running query count, with a minimum of one.
After memory usage drops below the threshold, the scheduler increases the allowed running-query count by one per update interval until it reaches the query runner thread count.

Full Server Config Example

# Query runner pool sizing still controls the upper bound for concurrent query execution.
pinot.server.query.executor.num.groups.limit=100
pinot.server.query.executor.num.groups.per.query=10

# Enable memory-aware scheduling.
pinot.query.scheduler.name=MemoryThrottledFCFS
pinot.query.scheduler.throttle.memoryUsagePercentageThreshold=90
pinot.query.scheduler.throttle.maxRunningQueriesUpdateFrequencyMs=1000

The exact query executor thread settings depend on your server profile. Keep the scheduler threshold aligned with JVM heap size, query memory profile, and expected concurrency.

Metrics to Watch

The scheduler updates StarTree server gauges for query scheduling and throttling:

Gauge	What to Watch
`QUERY_NUM_RUNNER_THREADS`	Upper bound available from the query runner pool.
`QUERY_RUNNING_TOTAL`	Current running query count.
`QUERY_MAX_ALLOWED_QUERIES`	Dynamic concurrency cap set by the scheduler.
`QUERY_THROTTLING_SINCE_MS`	Non-zero when throttling is active.
`QUERY_THROTTLING_TOTAL_MS`	Accumulated time spent throttling.
`QUERY_THROTTLING_GC_REQUESTED`	Indicates that sustained throttling caused the scheduler to request GC.

Healthy behavior after a short spike is:

QUERY_THROTTLING_SINCE_MS becomes non-zero.
QUERY_MAX_ALLOWED_QUERIES drops.
Memory usage falls.
QUERY_THROTTLING_SINCE_MS returns to zero.
QUERY_MAX_ALLOWED_QUERIES ramps back toward QUERY_NUM_RUNNER_THREADS.

Operational Guidance

Start with a threshold between 85 and 95, then tune from observed query latency and heap pressure.
Prefer fixing query memory regressions or oversized result sets before relying on scheduler throttling.
Keep enough broker and client timeout budget for queued queries.
Avoid setting the threshold too low on latency-sensitive clusters; it can queue queries even when the JVM has usable headroom.
If throttling is continuous, add capacity, reduce per-query memory, lower concurrency, or investigate memory leaks.

Troubleshooting

Symptom	Cause	Fix
Server fails to start with an illegal threshold error	`pinot.query.scheduler.throttle.memoryUsagePercentageThreshold` is `<= 0` or `>= 100`.	Set a value between `0` and `100`, such as `90`.
Query latency increases during memory spikes	Queries are queueing while the scheduler throttles.	Confirm memory pressure and either raise capacity, reduce query memory, or adjust the threshold.
Throttling never clears	Used memory remains above the threshold.	Inspect heap usage, query mix, result sizes, segment memory, and possible leaks.
`QUERY_MAX_ALLOWED_QUERIES` ramps up too quickly after throttling	Update frequency is too low for your workload.	Increase `pinot.query.scheduler.throttle.maxRunningQueriesUpdateFrequencyMs`.
Queries still consume all runner threads	Scheduler not enabled on the server, or config was not applied.	Confirm `pinot.query.scheduler.name=MemoryThrottledFCFS` on each server and restart/reload the server process as required.

Get Started

Ingestion

External Tables

Query Data

Manage Data

Observability

Visualize Data

Cluster Operations

Manage Security

Release Notes

Reference

Memory-Throttled FCFS Query Scheduler

Class and Scheduler Name

How It Works

Enable the Scheduler

Configuration

Tuning Example

Full Server Config Example

Metrics to Watch

Operational Guidance

Troubleshooting

​Class and Scheduler Name

​How It Works

​Enable the Scheduler

​Configuration

​Tuning Example

​Full Server Config Example

​Metrics to Watch

​Operational Guidance

​Troubleshooting

Class and Scheduler Name

How It Works

Enable the Scheduler

Configuration

Tuning Example

Full Server Config Example

Metrics to Watch

Operational Guidance

Troubleshooting