StarTree Engine

​

Overview

​

What changes — and what doesn’t

​

Enabling the StarTree Engine

​

Cluster prerequisite

# Required. Enables the native runtime. 0 = pick an ephemeral free port (recommended).
pinot.startree.rust.mailbox.port=0

​

Per-query switch

SET useMultistageEngine = true;
SET workerRuntime = 'startree';
SELECT ...

curl -X POST http://localhost:8000/query/sql -d '
{
  "sql": "select * from baseballStats limit 10",
  "queryOptions": "useMultistageEngine=true;workerRuntime=startree"
}'

​

When to expect benefits

• Join-heavy queries, especially large hash joins between fact tables.
• Large aggregations and group-bys over data shuffled between servers.
• Sorts and window functions over sizeable intermediate results.
• Queries that move a lot of data between stages — the native exchange layer transfers Arrow batches with far less serialization and garbage-collection overhead than the Java mailbox.
• Memory-intensive queries: the native runtime uses a bounded off-heap memory pool with spill-to-disk for sorts and aggregations, so large queries degrade gracefully instead of pressuring the JVM heap.

​

When it doesn’t help

• Leaf-dominated queries. If most of the time is spent scanning segments (highly selective filters, simple aggregations pushed down to the leaf), the leaf stage is identical in both runtimes and the StarTree Engine has little to accelerate.
• Very fast, small queries. Crossing between the JVM and the native runtime has a small fixed cost per query and per batch. Queries that complete in a few tens of milliseconds on Java MSQE may see no gain, or a slight regression.
• A few specific query shapes currently regress and are being actively closed (for example, some ASOF joins and certain UNION ALL patterns).

​

Join strategies

SELECT /*+ joinOptions(join_strategy='sort_merge') */ a.*, b.*
FROM a JOIN b ON a.k = b.k

​

Spill to disk for large queries

​

Unsupported cases

​

Transparent fallback to Java MSQE (query still succeeds)

• INTERSECT ALL and EXCEPT ALL / MINUS ALL set operations.
• Aggregations that return arrays or collections (e.g. ARRAY_AGG).
• Certain optimizer-hint-driven aggregation paths that have no native equivalent, such as is_leaf_return_final_result combined with SUMPRECISION, funnel functions, or raw sketch aggregations.
• UNNEST when the column-pruning optimization is enabled for it.
• Unsupported column types, when they are visible in the logical plan (see below).

​

Query fails with an error (no fallback)

• Unsupported column types reaching an intermediate stage when not detectable at planning time: MAP, OBJECT, and exotic array shapes. Supported types are INT, LONG, FLOAT, DOUBLE, BOOLEAN, TIMESTAMP, STRING, JSON, BIG_DECIMAL, BYTES, and their common array variants.
• Unsupported join sub-cases: LOOKUP joins outside INNER/LEFT/SEMI/ANTI, SEMI/ANTI lookup joins with non-equality conditions, and certain ASOF join match conditions.
• Functions outside the supported envelope — see the function support section below for the specific lists of native, bridged, and unsupported functions.

​

Function support

• VAR_SAMP and STDDEV_SAMP currently fail at runtime with an Invalid arithmetic operation error.
• Aggregations that return arrays or collections (e.g. ARRAY_AGG) fall back to Java MSQE at planning time, as noted above.

​

Behavioral differences to be aware of

• Integer division follows standard SQL. 1 / int_col truncates to an integer, as in PostgreSQL, whereas Java MSQE historically evaluated it as floating-point division. If a query relies on the legacy behavior, add an explicit CAST(... AS DOUBLE).
• Some Java-MSQE-specific optimizer hints are not honored (e.g. group-trim hints), so queries using them may return results computed by a different — standard — strategy.

• Detailed stats require the streaming stats channel. Per-stage and per-operator response stats for natively executed stages — including leaf counters such as numSegmentsQueried — are only delivered when MSQE’s streaming stats channel is active. Enable it per query with SET streamStats = true; or cluster-wide with the broker configuration pinot.broker.mse.stream.stats=true (all servers must run a version that supports it). Without it, StarTree Engine responses carry only broker-side stats. In addition, a few plan shapes (e.g. semi-join and dynamic-broadcast sub-plans) do not yet surface their leaf-stage stats. All of this affects observability, not results.

​

Monitoring

• The broker logs when a query falls back to Java MSQE and why.
• The native runtime exposes its own Prometheus metrics endpoint (pinot.startree.rust.metrics.port, default 8079). All metrics are prefixed rust_:

  Metric	Type	What it tells you
  rust_spill_pool_reserved_bytes / rust_spill_pool_size_bytes	gauge	Native memory pool usage vs. configured size (labelled by runtime)
  rust_operator_spills_total / rust_operator_spilled_bytes_total	counter	Spill events and bytes written by sorts/aggregations under memory pressure
  rust_operator_hash_build_seconds_total	counter	Cumulative time building hash-join tables
  rust_mailbox_compress_* / rust_mailbox_decompress_*	counter	Inter-stage exchange traffic: batches per codec, bytes before/after compression, and time spent
  rust_mailbox_fanout_payload_reuse_*	counter	Broadcast exchange payload reuse
  rust_mailbox_grpc_recv_tasks_active / _total	gauge / counter	Active and total cross-server exchange receive tasks
  rust_tokio_workers	gauge	Network runtime worker threads

  In StarTree Cloud these metrics are also stored in Prometheus and can be read using Grafana as usual.
• In addition, brokers and servers export two engine-related meters through the standard Pinot metrics pipeline: datafusionOpchainAllocatorLeakCloseCount and datafusionOpchainAllocatorLeakedBytes. These are safety-net counters that track native memory reclaimed when a query’s allocator is closed while still holding buffers; they should normally stay at zero.
• Standard Pinot query response metadata is returned for all queries, with the stats-coverage caveat noted above.