Release version 0.17.0: October 2026

​

Summary

Release 0.17.0 is mostly about making External Tables cheaper to run and easier to trust, and about tightening defaults in the minion tasks that keep upsert tables consistent. It also moves the runtime to JDK 25 and picks up a large Apache Pinot baseline update with a new UUID type, OPEN_STRUCT columns, and GROUPING SETS in the multi-stage engine.

• External Tables: Much lower server heap per segment, near-native index builds on Parquet-backed segments, Iceberg partition pruning at the broker, Parquet INT96 support, working PromQL range functions, and fail-fast sync with health metrics.
• Upsert and minion tasks: A new MERGE_JSON partial-upsert merger, stricter table-config validation, and new defaults for SegmentRefreshTask. Upsert compaction tasks are now rejected on new configs in favor of SegmentRefreshTask.
• Query: Opt-in index-based GROUP BY fast paths, a preview of AI functions in SQL, and more native coverage in the StarTree Execution Engine.
• Apache Pinot OSS: See Apache Pinot Highlights for the open-source changes, including several single-stage engine behavior fixes that can change query results.

​

StarTree Cloud highlights

​

New Features

​

External Tables: lower memory, faster index builds, partition pruning

• Lower server heap per segment: The per-column state a server keeps for each loaded External Table segment dropped from about 4.3 KB to about 0.9 KB. Columns that are never queried hold almost nothing until they’re read. Large External Tables with many columns and segments fit in less heap. For sizing guidance, see Best practices and configs.
• Near-native index builds on Parquet-backed segments: Star-tree, inverted, and range index builds now read Parquet through a build-optimized path. In internal benchmarks, a 300,000-row build went from about 4 minutes to about 3 seconds. See External Table indexes.
• Iceberg partition pruning: Identity, bucket, and truncate partition transforms from Iceberg can be imported so the broker skips segments that can’t match a query. Turn it on with enable_partition_pruning = true in SQL DDL, or enablePartitionPruning in the task config. It’s off by default.
• Parquet INT96 columns: Files with INT96 timestamps, including list<INT96>, now load and query correctly. INT96 footer statistics are ignored, so these columns don’t get min/max pruning. If the table’s time column is INT96, set segmentTimeValueCheck to false. See INT96 in the data type mapping.
• PromQL on External Tables: Range functions (rate, irate, increase, delta) returned null for every bucket before this release, and topk/bottomk didn’t work. Both are fixed, and range functions now scale linearly with the number of samples. See PromQL functions.
• Sync fails fast: External Table sync runs stop as soon as a segment goes to ERROR, and detect stalls after 5 minutes without progress instead of waiting out the full timeout. New controller metrics report unhealthy syncs and consecutive failures. See External Table metrics and alerting.
• Lag snapshot size: The External Table status and lag API now returns a raw totalBytes field alongside the human-readable size. See Ingestion lag.

​

MERGE_JSON partial-upsert merger

A new configurable partial-upsert merger merges JSON objects and MAP values per column instead of replacing them. Set partialUpsertMergerClass to ai.startree.pinot.upsert.merger.ConfigurablePartialUpsertMerger and mark each column with startree.merger.columns.<column>=MERGE_JSON in metadataManagerConfigs. The startree.merger.json.mode option picks append (the default, a recursive merge) or override. Don’t combine it with partialUpsertStrategies on the same table. See Merge JSON fields during partial upsert.

​

Additional New Features

• AI functions in SQL (preview): AI_FILTER, AI_CLASSIFY, and AI_GENERATE call a configured model from a single-stage query. They’re off by default and need to be enabled on the cluster by StarTree. Contact StarTree support if you want to try them.
• Conditional aggregation in Alter Table Task: StarTreeAlterTableTask can aggregate a metric differently based on the value of another column (SUM, MIN, or MAX per value), with a fallback aggregation type for everything else. See Alter Table Task output and rollup options.
• Index-based GROUP BY (opt-in): For low-cardinality dictionary columns with an inverted index, GROUP BY with COUNT, SUM, MIN, MAX, or AVG can be answered from the index instead of scanning rows. Internal benchmarks showed 1.8x to 3.2x speedups. Enable per query with SET useIndexBasedGroupByOperator=true; the server-side switch has to be turned on as well.
• Primary key lookup for upsert tables: A controller debug endpoint, GET /upsert/debug/{tableNameWithType}/metadata, looks up a primary key across all replicas of a RocksDB-backed upsert table and reports whether the replicas agree. It needs table read permission. See Off-heap upsert.
• MongoDB CDC lag: MongoDB change-data-capture tables now report time-based ingestion lag through the standard realtime ingestion lag metrics. See MongoDB CDC.
• System query log controls: A table allowlist limits which tables are captured, and SET skipQueryLog=true excludes a single query. Queries against system_query_log itself, and requests from StarTree’s health-check clients, are no longer captured by default. See Query log.
• Dynamic row filters: StarTree now supports query-time parameterized row filters, allowing a single row-level access policy to serve multiple tenants or users without hardcoding individual filter values. Applications supply parameter values with each query, and StarTree automatically substitutes them into the policy and enforces the resulting row-level restrictions. See Dynamic Row Filters.

​

Improvements

• StarTree Execution Engine: IN and NOT IN predicates now run natively instead of falling back to the Java engine. See function support.
• External Tables: Bounded, off-heap S3 range reads, batched bitmap posting reads, mmap’able Parquet page indexes, and fewer redundant S3 probes during segment load. Struct, list-of-struct, and map-of-struct columns now store nested schema metadata by default (see Breaking Changes). Binary, fixed, and UUID Iceberg columns are inferred as BYTES, and required fields carry NOT NULL into the Pinot schema. See Data type mapping.
• Tiered storage: Time-based tiers on External Tables can now use a non-zero segmentAge. The server disk cache keeps cached pages across restarts, so servers come back warm. See Set the segment age boundary and Server restart optimizations.
• Ingestion: Composite JSON indexes index native MAP values without re-parsing them. See Indexes.
• Raw forward indexes default to V6 and ZSTANDARD: Raw forward indexes that don’t set compressionCodec now use ZSTANDARD instead of LZ4. New clusters also write the V6 format by default, which stores per-value sizes instead of cumulative offsets in each chunk header so the header compresses better. In internal tests on realistic data, ZSTANDARD with V6 made raw forward indexes about 56% smaller than LZ4 with V4. ZSTANDARD alone accounted for about 38%, and V6 cut query CPU 4–8% compared with V4. ZSTANDARD costs more CPU to decompress than LZ4, most visibly on tiered storage. Existing clusters keep writing V4 until StarTree switches them; contact StarTree support to move a cluster to V6. Only newly built segments change, and columns that set rawIndexWriterVersion or compressionCodec keep their value (#17966). See Raw forward index format V6.
• Upsert: Invalid partial-upsert column and function combinations are rejected with HTTP 400 at table create or update, including SQL CREATE. Existing tables aren’t rescanned. See Merge JSON fields during partial upsert.
• Platform: The runtime moved from JDK 21 to JDK 25.

​

Bug Fixes

• Query correctness: Null rows are now left out of sparse and min/max index filter results when null handling is on.
• Upsert and RocksDB: Fixed a race between RocksDB preload finalization and store close, and RocksDB state is now recreated cleanly after a failed preload.
• External Tables: Fixed several Parquet reader issues, including struct leaf pages misaligned across page boundaries and a direct-memory leak in the S3 block cache. The broker now reports when it can’t load the latest Iceberg snapshot instead of silently serving the previous one.
• Minion tasks: Fixed duplicate and unwanted Alter Table Task stale-segment tasks, and moved Alter Table Task deep-store cleanup off the task generation path.
• Security: SegmentPurgeTask now rejects selector queries that contain more than one statement, and Iceberg REST catalog endpoints are validated. Dependency CVEs were fixed in BouncyCastle, Jackson, Guava, lz4, and async-http-client.

​

Breaking Changes

• Upsert compaction tasks are rejected on new configs. Table create and update now reject a config that adds UpsertCompactionTask or UpsertCompactMergeTask to taskConfig, or sets upsertConfig.enableCommitTimeCompaction to true. These can leave replicas with inconsistent metadata. Use SegmentRefreshTask instead; see Upsert compaction with SegmentRefreshTask. Tables that already have one of these settings keep working, and removing the setting is always allowed. There’s no override.
• SegmentRefreshTask skips segments without a recognized sequence ID. skipInvalidSequenceId now defaults to true, so segments whose names don’t carry a sequence ID that SegmentRefreshTask recognizes are left out of merges. Set skipInvalidSequenceId to false in the task config to restore the old behavior. See How to configure the SegmentRefreshTask.
• External Table snapshot defaults changed. For tables without an explicit value, iceberg.snapshotProcessing.retention.maxActiveVersions drops from 5 to 2, so older snapshot versions are pruned on the next retention run. iceberg.snapshotProcessing.batchSize goes from 1,000 to 10,000 files per sync. Set either one explicitly on the table to keep the old value. See Snapshot processing (Iceberg).
• External Tables store nested schema metadata by default. storeNestedSchema.enabled now defaults to true for External Table sync. On existing tables with nested columns, the first sync after upgrade may update the schema and reload segments. Set storeNestedSchema.enabled to false to keep the old behavior. See Best practices and configs.
• Minion tasks are rejected on segment-group tables. A table with segmentGroupConfig.enabled can’t have any minion task in taskConfig, except ExternalTableSyncTask running on the controller. See Segment groups.
• Deletion vector table options were removed. The table-level keys iceberg.dv.fetch.mode, iceberg.dv.puffinCache.maxBytes, and iceberg.dv.puffinCache.downloadTimeoutMs are now ignored. The deletion vector cache is managed at the server level. See Deletion vectors.
• JDK 25 runtime. StarTree Pinot now builds and runs on JDK 25 only. This only matters if you deploy custom plugins in a BYOC environment. Rebuild them against JDK 25 and check any dependency on internal (non-SPI) Pinot classes before upgrading.

​

Apache Pinot highlights

This section describes Apache Pinot (open source) changes in the baseline that ships with StarTree Cloud 0.17.0 compared with 0.16.0. It doesn’t describe StarTree-only extensions, and it leaves out changes that already shipped in 0.16 patch releases. Links in parentheses go to the upstream pull request, which has the full details.

​

New Features

• UUID data type: A logical UUID column type stored as 16 bytes, with ingestion, CAST, literals, bloom filters, GROUP BY, DISTINCT, and multi-stage engine support. New functions include IS_UUID, TO_UUID, UUID_TO_STRING, UUID_TO_BYTES, BYTES_TO_UUID, UUID_VERSION, UUID_TIMESTAMP, UUID_V4, and UUID_V7 (#18870, #19091).
• OPEN_STRUCT columns: A schemaless key/value column type. Keys are discovered at ingestion; frequently present keys are stored as their own columns and the rest go into a JSON column. Query a key with col['key'] (#18643).
• GROUPING SETS, ROLLUP, and CUBE in the multi-stage engine, with GROUPING() and GROUPING_ID() (#18817).
• Window functions: The EXCLUDE CURRENT ROW | GROUP | TIES | NO OTHERS frame clause (#18482), and IGNORE NULLS for LEAD and LAG (#18202). See Window functions in MSQE.
• Segment metadata virtual columns: $creationTime, $startTimeMs, $endTimeMs, $totalDocs, and $segmentCrc. They’re NULL on consuming segments (#19179).
• Approximate function rewrite: An opt-in broker setting rewrites DISTINCTCOUNT and COUNT(DISTINCT) to DISTINCTCOUNTSMARTHLL, and PERCENTILE to PERCENTILESMARTTDIGEST. Override per query with the useApproximateFunction query option (#19510).
• Page cache warmup: The pageCacheWarmupConfig table config warms the OS page cache when a server restarts or a segment is refreshed, with a time limit and QPS limit (#16033).
• Kafka config providers: Realtime streamConfigs can use Kafka ConfigProvider references such as ${file:path:key}, for example to read credentials from a file (#19202).
• Smaller additions:
  • A segmentAgeField tier option (endTime, startTime, or creationTime) (#19415). See Set the segment age boundary.
  • JSON and UUID support in ARRAY_AGG (#19333), and multi-value columns in DISTINCTCOUNTULL (#19302).
  • BYTES array literals (#19247).
  • A setOpOptions hint to run UNION, INTERSECT, and EXCEPT without a shuffle when inputs are colocated (#18804). See Set operations in MSQE.

​

Improvements

• Multi-stage planning: Large IN lists are kept out of Calcite optimization, which cuts planning time for queries with long value lists (#19674).
• Colocated joins: Broker pruning now removes pruned partitions from colocated joins (#19174), and colocated joins work when a partition has no segments instead of failing (#19166). See Table joins in MSQE.
• Routing: Upsert tables using strict replica-group routing now get adaptive server selection when an adaptive selector is configured (#18947).
• Metadata-based aggregation: When a query mixes aggregations that can be answered from segment metadata with ones that can’t, the metadata-answerable ones still skip the scan (#18334).
• Null handling: With null handling enabled, every aggregation now skips NULL inputs, GROUP BY on dictionary and multi-value columns produces a proper NULL group, and filters follow three-valued logic. For example, NOT (col = x) no longer counts NULL rows. Results with null handling off are unchanged (#19158).
• Query options: New broker settings can warn about or reject unknown SQL query options and the legacy OPTION(...) syntax. Both are off by default (#19027, #19570).
• Query log redaction: A broker setting can redact literal values or full SQL text from query logs (#18604).
• Java 25: The Apache Pinot baseline now builds and runs on JDK 25 (#19014).

​

Bug Fixes

• Query correctness: Fixed GROUP BY returning wrong groups or failing when the initial result holder capacity was optimized (#19379), expression filters attributing matches to the wrong rows (#19245), and the final GROUP BY schema depending on which server responded first (#19543).
• Ingestion: Transactional Kafka topics no longer raise false stream data loss alerts (#19161).
• Instance assignment: Fixed fault-domain-aware instance assignment when pool tags don’t start at 0 (#19266).
• Arrow responses: Fixed Arrow-encoded responses failing on non-string object values (#19591).

​

Breaking Changes

• Single-stage engine no longer ignores HAVING and QUALIFY. HAVING used to be dropped silently in some single-stage queries, and QUALIFY was ignored. HAVING is now applied, and invalid HAVING clauses (without GROUP BY, or on an ungrouped column) fail with an error like the multi-stage engine (#19554). QUALIFY is rejected in the single-stage engine; run those queries on the multi-stage engine (#19553). A FILTER (WHERE ...) on an aggregation used only in HAVING or ORDER BY is now applied too (#19437). Queries that relied on the old behavior may return fewer rows or fail.
• Admin API access control is enforced when authentication is configured. If a server access-control factory is configured, server admin routes (other than health checks and table data downloads) require admin authorization (#19233). Controller cluster-level endpoints check the caller’s permission instead of allowing any authenticated user (#19234). LOOKUP() in the single-stage engine requires access to the dimension table, and is rejected if that table has a row-level security filter (#19209). Clusters without access control configured are unaffected. See Manage access.
• Deprecated REST endpoints removed. Controller GET /tables/{tableName}/livebrokers is gone; use GET /tables/livebrokers?tables=.... Server GET /segments/{table}/{segment}/validDocIds and GET /tables/{table}/validDocIdMetadata are gone; use validDocIdsBitmap and POST validDocIdsMetadata (#19141, #19142).
• Upsert and dedup partition count is locked. Changing numPartitions in segmentPartitionConfig on an upsert or dedup table is rejected as a backward-incompatible update (#18868).
• Experimental enriched join removed. The JoinToEnrichedJoin planner rule is ignored and those queries run as normal joins (#19033).
• DDL validation errors return HTTP 400 instead of 500 (#19544).