> ## Documentation Index
> Fetch the complete documentation index at: https://docs.startree.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# StarTree Engine
## Overview
The StarTree Engine is a beta capability. Its opt-in mechanism may change between releases.
The StarTree Engine is an alternative, high-performance execution runtime for the
[Multi-Stage Query Engine (MSQE)](/corecapabilities/query_data/query_languages/msqe). It keeps
everything you already know about MSQE — the SQL dialect, the query planner, and Pinot's
segment/index-powered table scans — but executes most of the query's operators
(joins, aggregations, sorts, window functions, set operations) in native code instead of Java.
Under the hood it is built on [Apache DataFusion](https://datafusion.apache.org/), a vectorized
query engine written in Rust, extended with a native Arrow-based data exchange layer for moving
data between query stages.
### What changes — and what doesn't
A multi-stage query runs in three kinds of stages:
| Stage | Java MSQE | StarTree Engine |
| ------------------------------------------------------------- | -------------------- | -------------------------------- |
| **Leaf stages** (segment scans, filter/index pushdown) | Pinot Java execution | Pinot Java execution (unchanged) |
| **Intermediate stages** (joins, aggregations, sorts, windows) | Java operators | Native, vectorized operators |
| **Root stage** (final merge on the broker) | Java operators | Native operators |
| **Data exchange between stages** | Java mailbox | Native Arrow-based exchange |
Query planning is unchanged: the broker still parses, plans, and optimizes the query the same
way, so `EXPLAIN` plans, query options, and the general query lifecycle look the same. Because
leaf stages stay in Java, all of Pinot's indexing (inverted, sorted, star-tree, JSON, etc.)
keeps working exactly as before.
## Enabling the StarTree Engine
Enablement has two layers: a cluster-level prerequisite and a per-query switch. The StarTree
Engine is **off by default** — queries run on the Java MSQE runtime unless explicitly routed.
### Cluster prerequisite
Every broker and server must start the native runtime. This is controlled by one configuration
key, set in the broker and server configuration:
```text theme={null}
# Required. Enables the native runtime. 0 = pick an ephemeral free port (recommended).
pinot.startree.rust.mailbox.port=0
```
In StarTree Cloud these settings are managed as part of the deployment and are enabled by
default. Contact StarTree support if the StarTree Engine is not available in your environment.
Optional tuning (defaults shown):
| Key | Default | Purpose |
| -------------------------------------- | ------- | ----------------------------------------------------------------------- |
| `pinot.startree.rust.metrics.port` | `8079` | Prometheus `/metrics` endpoint for the native runtime (`<= 0` disables) |
| `pinot.startree.datafusion.batch.size` | `8192` | Rows per vectorized batch |
Memory and spill-to-disk behavior has its own set of settings — see
[Spill to disk for large queries](#spill-to-disk-for-large-queries).
### Per-query switch
Route an individual multi-stage query to the StarTree Engine with the `workerRuntime` query
option. Using the `SET` command:
```sql theme={null}
SET useMultistageEngine = true;
SET workerRuntime = 'startree';
SELECT ...
```
Or through the query options field of the REST APIs:
```bash theme={null}
curl -X POST http://localhost:8000/query/sql -d '
{
"sql": "select * from baseballStats limit 10",
"queryOptions": "useMultistageEngine=true;workerRuntime=startree"
}'
```
The StarTree Engine only applies to multi-stage queries. Single-stage queries are unaffected by
the `workerRuntime` option.
If a query sets `workerRuntime = 'startree'` but the cluster prerequisite is missing, the query
fails with a "runtime is not registered" error — the cluster configuration must be in place
first.
## When to expect benefits
The StarTree Engine accelerates the parts of the query that run *above* the table scan. It
shines when intermediate stages do significant work:
* **Join-heavy queries**, especially large hash joins between fact tables.
* **Large aggregations and group-bys** over data shuffled between servers.
* **Sorts and window functions** over sizeable intermediate results.
* **Queries that move a lot of data between stages** — the native exchange layer transfers
Arrow batches with far less serialization and garbage-collection overhead than the Java
mailbox.
* **Memory-intensive queries**: the native runtime uses a bounded off-heap memory pool with
spill-to-disk for sorts and aggregations, so large queries degrade gracefully instead of
pressuring the JVM heap.
In internal TPC-H benchmarks, most join- and aggregation-heavy queries ran 20–60% faster than
Java MSQE, and performance continues to improve release over release.
### When it doesn't help
* **Leaf-dominated queries.** If most of the time is spent scanning segments (highly selective
filters, simple aggregations pushed down to the leaf), the leaf stage is identical in both
runtimes and the StarTree Engine has little to accelerate.
* **Very fast, small queries.** Crossing between the JVM and the native runtime has a small
fixed cost per query and per batch. Queries that complete in a few tens of milliseconds on
Java MSQE may see no gain, or a slight regression.
* **A few specific query shapes** currently regress and are being actively closed (for example,
some ASOF joins and certain UNION ALL patterns).
Switching runtimes is a one-line query option, so A/B comparison on your own workload is cheap.
If a query matters to you, benchmark it on both runtimes.
### Join strategies
Beyond the default hash join, two join strategies are available natively through the standard
Calcite join hint:
```sql theme={null}
SELECT /*+ joinOptions(join_strategy='sort_merge') */ a.*, b.*
FROM a JOIN b ON a.k = b.k
```
| Strategy | Hint value | Notes |
| :----------------- | :----------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Sort-merge join | `sort_merge` | INNER/LEFT joins with at least one equality key (including `BIG_DECIMAL` keys). Can spill to disk, so prefer it for very large joins. Falls back to hash join (same results) for unsupported shapes. |
| Native lookup join | `lookup` | Dimension-table joins (INNER/LEFT/SEMI/ANTI; non-equality conditions are allowed for INNER/LEFT only) run natively; the dimension-table lookup itself still happens in Java. Unsupported shapes are query errors, not silent fallbacks. |
### Spill to disk for large queries
Aggregations (`GROUP BY`/`DISTINCT`), sorts, and sort-merge joins can spill to local disk under
memory pressure instead of failing with an out-of-memory error — the query gets slower, not
killed. Spill is backed by a shared native memory pool on each broker and server, sized from
the available off-heap memory (container memory minus the JVM max heap).
| Config | Default | Description |
| :------------------------------------------------------- | :------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `pinot.startree.datafusion.spill.enabled` | `true` | Master on/off switch. When off, the pool is unbounded. |
| `pinot.startree.datafusion.spill.memory.size` | `50%` | Percent of off-heap headroom, or an absolute size (`4G`, `512M`). |
| `pinot.startree.datafusion.spill.compression` | `uncompressed` | Spill file compression: `uncompressed`, `lz4_frame`, or `zstd`. |
| `pinot.startree.datafusion.spill.max.disk.size` | `100G` | Max spill data size before the query fails, or `unlimited`. |
| `pinot.startree.datafusion.spill.dir` | OS temp dir | Absolute path override for spill files. |
| `pinot.startree.datafusion.udaf.size.accounting.enabled` | `false` | Charge Java-heap-backed UDAF state (e.g. `DISTINCTCOUNT`, percentiles, sketches) toward the spill trigger. Off by default since the estimate is heap-based, not off-heap. |
| `pinot.startree.datafusion.udaf.groupby.max.capacity` | `1,000,000` | Max simultaneously-live groups when UDAF size accounting is enabled. |
Malformed or out-of-range values for these settings fail broker/server startup rather than
silently falling back to a default.
**What doesn't spill:** hash joins, nested-loop joins, cross joins, and window aggregates fail
with a resources-exhausted error under memory pressure instead of spilling. If a large join
hits this, use the `sort_merge` join hint above instead of the default hash join, or raise the
memory pool size.
## Unsupported cases
Not every query pattern runs natively yet. There are two very different behaviors, depending on
*when* the engine detects the unsupported pattern.
### Transparent fallback to Java MSQE (query still succeeds)
At planning time the broker checks the query plan for patterns the StarTree Engine does not
support. If it finds one, it silently ignores the `workerRuntime` option and runs the **whole
query on the Java MSQE runtime**. The query succeeds and returns the same results as if you had
never set the option; the only trace is an INFO log line on the broker
(`Disabling DataFusion for this query: ...`).
Patterns that currently fall back at planning time:
* `INTERSECT ALL` and `EXCEPT ALL` / `MINUS ALL` set operations.
* Aggregations that return arrays or collections (e.g. `ARRAY_AGG`).
* Certain optimizer-hint-driven aggregation paths that have no native equivalent, such as
`is_leaf_return_final_result` combined with `SUMPRECISION`, funnel functions, or raw sketch
aggregations.
* `UNNEST` when the column-pruning optimization is enabled for it.
* Unsupported column types, when they are visible in the logical plan (see below).
If the planner finds any of these patterns, the whole query falls back to the classic Java
engine — not just the affected stage.
### Query fails with an error (no fallback)
Some patterns can only be detected after planning, while the plan is being prepared for native
execution on the servers. At that point there is **no fallback**: the query fails and the
client receives an error. To run such a query, remove the `workerRuntime` option (or rewrite
the query).
Cases that fail at runtime:
* **Unsupported column types** reaching an intermediate stage when not detectable at planning
time: `MAP`, `OBJECT`, and exotic array shapes. Supported types are INT, LONG, FLOAT, DOUBLE,
BOOLEAN, TIMESTAMP, STRING, JSON, BIG\_DECIMAL, BYTES, and their common array variants.
* **Unsupported join sub-cases**: LOOKUP joins outside INNER/LEFT/SEMI/ANTI, SEMI/ANTI lookup
joins with non-equality conditions, and certain ASOF join match conditions.
* **Functions outside the supported envelope** — see the function support section below for
the specific lists of native, bridged, and unsupported functions.
### Function support
**Aggregation functions.** The common aggregations run fully natively: `COUNT`, `SUM`, `MIN`,
`MAX`, `AVG`, `BOOL_AND`, `BOOL_OR`, and the population statistics `VAR_POP`, `STDDEV_POP`,
and `COVAR_POP`. Every other Pinot aggregation function — percentiles, `DISTINCTCOUNT` and its
sketch variants, `SUMPRECISION`, funnel functions, and so on — executes through a built-in
Java bridge: results are identical to Java MSQE, at some extra cost when aggregation state
crosses the native/Java boundary. Known exceptions:
* `VAR_SAMP` and `STDDEV_SAMP` currently fail at runtime with an
`Invalid arithmetic operation` error.
* Aggregations that return arrays or collections (e.g. `ARRAY_AGG`) fall back to Java MSQE at
planning time, as noted above.
**Scalar functions.** Comparison, arithmetic, and logical operators, `CASE`, `IN`/`NOT IN`,
and array construction run natively. All other registered Pinot scalar functions are
automatically bridged to their Java implementation as long as their argument and return types
are among INT, LONG, FLOAT, DOUBLE, BOOLEAN, STRING, BYTES, and TIMESTAMP (BIG\_DECIMAL and
JSON results are carried as strings). A scalar function outside that envelope — for example
one that takes or returns arrays or Java objects (theta-sketch set operations being a
supported exception) — fails at runtime with an `Unsupported function` error.
**Window functions.** The standard window functions run natively: `ROW_NUMBER`, `RANK`,
`DENSE_RANK`, `PERCENT_RANK`, `CUME_DIST`, `NTILE`, `LAG`, `LEAD`, `FIRST_VALUE`,
`LAST_VALUE`, and `NTH_VALUE`, plus the native aggregations listed above used with an `OVER`
clause. There is no Java bridge for window functions: a Pinot-specific window function outside
this set fails at runtime.
**`BIG_DECIMAL` precision.** `BIG_DECIMAL` columns are fully supported in comparisons,
sorting, joins, `GROUP BY`, `DISTINCT`, and `MIN`/`MAX`/`SUM`/`AVG`/`COUNT`, using an
order-preserving binary encoding. However, arithmetic (`+ - * / %`) and aggregation over
`BIG_DECIMAL` are computed in double-precision floating point — matching Java MSQE behavior,
not arbitrary-precision decimal arithmetic. If your workload depends on exact decimal
arithmetic beyond double precision, verify results before relying on it. Custom Java
UDFs/UDAFs over `BIG_DECIMAL` are not yet supported on the StarTree Engine.
### Behavioral differences to be aware of
A small number of cases run natively but intentionally behave differently from Java MSQE:
* **Integer division follows standard SQL.** `1 / int_col` truncates to an integer, as in
PostgreSQL, whereas Java MSQE historically evaluated it as floating-point division. If a
query relies on the legacy behavior, add an explicit `CAST(... AS DOUBLE)`.
* **Some Java-MSQE-specific optimizer hints are not honored** (e.g. group-trim hints), so
queries using them may return results computed by a different — standard — strategy.
- **Detailed stats require the streaming stats channel.** Per-stage and per-operator response
stats for natively executed stages — including leaf counters such as `numSegmentsQueried` —
are only delivered when MSQE's streaming stats channel is active. Enable it per query with
`SET streamStats = true;` or cluster-wide with the broker configuration
`pinot.broker.mse.stream.stats=true` (all servers must run a version that supports it).
Without it, StarTree Engine responses carry only broker-side stats. In addition, a few plan
shapes (e.g. semi-join and dynamic-broadcast sub-plans) do not yet surface their leaf-stage
stats. All of this affects observability, not results.
## Monitoring
* The broker logs when a query falls back to Java MSQE and why.
* The native runtime exposes its own Prometheus metrics endpoint
(`pinot.startree.rust.metrics.port`, default `8079`). All metrics are prefixed `rust_`:
| Metric | Type | What it tells you |
| ------------------------------------------------------------------ | --------------- | ----------------------------------------------------------------------------------------------- |
| `rust_spill_pool_reserved_bytes` / `rust_spill_pool_size_bytes` | gauge | Native memory pool usage vs. configured size (labelled by runtime) |
| `rust_operator_spills_total` / `rust_operator_spilled_bytes_total` | counter | Spill events and bytes written by sorts/aggregations under memory pressure |
| `rust_operator_hash_build_seconds_total` | counter | Cumulative time building hash-join tables |
| `rust_mailbox_compress_*` / `rust_mailbox_decompress_*` | counter | Inter-stage exchange traffic: batches per codec, bytes before/after compression, and time spent |
| `rust_mailbox_fanout_payload_reuse_*` | counter | Broadcast exchange payload reuse |
| `rust_mailbox_grpc_recv_tasks_active` / `_total` | gauge / counter | Active and total cross-server exchange receive tasks |
| `rust_tokio_workers` | gauge | Network runtime worker threads |
In StarTree Cloud these metrics are also stored in Prometheus and can be read using Grafana
as usual.
* In addition, brokers and servers export two engine-related meters through the standard Pinot
metrics pipeline: `datafusionOpchainAllocatorLeakCloseCount` and
`datafusionOpchainAllocatorLeakedBytes`. These are safety-net counters that track native
memory reclaimed when a query's allocator is closed while still holding buffers; they should
normally stay at zero.
* Standard Pinot query response metadata is returned for all queries, with the stats-coverage
caveat noted above.