Skip to main content

​
Overview and Purpose

The MinMax index is a lightweight, coarse-grained index that divides a sorted column into fixed-size chunks of documents and records the minimum and maximum value of each chunk. At query time it uses binary search over these boundaries to skip chunks that cannot match a filter, then runs an exact forward-index scan only on the surviving (“candidate”) chunks.

​
When to use it

Use the MinMax index for large sorted raw columns where you frequently filter with equality or range predicates and want to reduce the number of documents scanned. It is especially useful for wide variable-width columns (STRING, BYTES, BIG_DECIMAL) where scanning every value is expensive. The index only helps with EQ, IN, and RANGE predicates; other predicate types are not routed through it. The index is especially useful for:
  • Time or sequence columns that are sorted during ingestion.
  • High-cardinality columns where a full inverted index would be too large.
  • Equality, IN, and range predicates on sorted raw columns.
  • Object-store backed segments where pruning chunks also reduces remote reads.
This page covers the StarTree minmax index configured on a field. It is different from Pinot’s segment-level columnMinMaxValueGeneratorMode, which writes whole-segment min/max metadata for segment pruning.

​
Availability

The Min/Max index is available in StarTree release 0.15 and later. The corresponding STP release branch is release/1.6.0-STP-2.164.x.

​
How the Index Works

For each configured column, StarTree Cloud divides the sorted raw forward index into document chunks. The Min/Max index records the minimum and maximum column value in each chunk. When a query has a predicate on that column, the server checks the predicate against the chunk boundary:
  1. If the predicate cannot match any value in the chunk, the chunk is skipped.
  2. If the predicate might match the chunk, StarTree Cloud reads and evaluates the candidate rows.
  3. Final predicate evaluation still runs on candidate rows, so pruning does not change query results.
For example, if an eventTimeMillis chunk covers values from 1764556800000 to 1764557399999, a query for eventTimeMillis >= 1764560400000 can skip that chunk immediately.

​
Requirements

  • The column must be single-value.
  • The column must be sorted in each segment.
  • The column forward index must be enabled.
  • The column must use RAW encoding.
  • chunkSize, when configured, must be a positive power of two.
  • Supported data types are INT, LONG, FLOAT, DOUBLE, BIG_DECIMAL, TIMESTAMP, STRING, and BYTES. BOOLEAN, JSON, and MAP are not supported.

​
Configuration

To enable the Min/Max index, add a minmax index to the column’s fieldConfigList entry.
  • chunkSize: number of documents per chunk. If omitted, it defaults to 1024. It must be a positive power of 2.
For offline tables, sort input records by the same column before segment generation. For real-time tables, configure tableIndexConfig.sortedColumn so consuming segments are built in sorted order.

​
Configuration Parameters

​
Choosing chunkSize

There is no automatic per-segment derivation of chunk size — the optimal value depends on data type, cardinality, and query shape. Smaller chunks give finer pruning (fewer rows scanned per surviving chunk) but a larger index; larger chunks give a smaller index but coarser pruning. Variable-width columns (STRING/BYTES/BIG_DECIMAL) scan more bytes per chunk, so they may benefit from smaller chunks.

​
Query Examples

The Min/Max index can prune chunks for range predicates:
It can also prune for IN predicates:

​
Performance Considerations

  1. Sort quality matters: The index is only useful when values are sorted or highly clustered. Unsorted data produces wide chunk boundaries and weak pruning.
  2. Chunk size is the main tuning knob: Smaller chunks usually reduce scanned rows, while larger chunks reduce index size.
  3. Use with raw forward indexes: The index targets raw forward-index reads. Use a range or inverted index instead for dictionary-encoded columns when those indexes fit your workload better.
  4. Reload existing segments: After adding the index to an existing table, reload segments so StarTree Cloud builds the new index files.