PromQL support is built as a plugin on top of Apache Pinot’s Time Series Engine, which is specifically designed to handle time series workloads efficiently. This integration allows you to:
  • Use familiar Prometheus-style queries while leveraging Pinot’s scalable analytics capabilities.
  • Query high-cardinality time series data efficiently.
  • Take advantage of StarTree Cloud optimized storage and indexing.
The Time Series Engine in Pinot provides a pluggable framework that enables native support for various time-series query languages like PromQL. This engine is purpose-built for observability use cases and optimized for processing time series data in a more efficient way than traditional SQL queries.
The Time Series Engine and PromQL support in StarTree Cloud are currently in an alpha release.

​
Prerequisites

Before querying metrics with PromQL, ensure you have:

​
Configuration

No additional configuration is required to enable TSE or PromQL. The Time Series Engine and PromQL support are enabled by default in StarTree Cloud. The PromQL endpoint is available at <Broker-endpoint>:8000/timeseries. Supported PromQL queries can be executed on this endpoint - see Supported PromQL Expressions for details.

​
Data Ingestion Using Real-time Tables

The Prometheus Message Decoder is only supported with REALTIME tables in StarTree Cloud. It cannot be used with OFFLINE tables.
To ingest Prometheus metrics data in real-time, you’ll need to configure both the schema and table configuration properly. The Prometheus Message Decoder expects data in the following format: 
<metric_name>{<label1>=<value1>,<label2>=<value2>,...} <metric_value> <timestamp>

​
Required Schema Configuration

Your schema must include these specific fields with the exact names and data types:
FieldTypeDescription
metricSTRINGName of the Prometheus metric
labelsJSONKey-value pairs of Prometheus labels
valueDOUBLEThe numeric value of the metric
tsTIMESTAMPTimestamp in milliseconds since the Unix epoch
Table Schema

​
Table Configuration

For real-time ingestion, configure your table with the Prometheus Message Decoder:
  1. Set the table type to REALTIME
  2. Configure the stream settings to use PrometheusMessageDecoder
  3. Set appropriate flush thresholds for your use case Table Configuration

​
Sample Data Format

Sample Data

​
Important Considerations

  1. The schema field names (metric, labels, value, ts) must match exactly as specified.
  2. If you need different field names, use transform functions to alias them.
  3. The timestamp must be in milliseconds since Unix epoch.
  4. The labels field must be properly formatted JSON.
For more information, see Prometheus Message Decoder.

​
How to Query PromQL Data

There are two main ways to query PromQL data in Pinot:

​
1. Grafana Pinot Plugin

You can query PromQL data using the Grafana Pinot plugin:
  1. Install the Grafana Pinot plugin.
  2. Configure your Pinot data source in Grafana. Configure Pinot in Grafana
  3. Use the PromQL query editor in your Grafana dashboards. Grafana PromQL Query Editor

​
2. REST API Endpoints

You can query PromQL data using timeseries endpoint:

​
PromQL API endpoint

curl "http://localhost:8000/timeseries/api/v1/query_range" \
  -G \
  --data-urlencode "query=http_in_flight_requests" \
  --data-urlencode "start=1741011820" \
  --data-urlencode "end=1741021820" \
  --data-urlencode "step=30" \
  --data-urlencode "language=promql" \
  --data-urlencode "table=prometheusMsg_REALTIME"
Parameters explained:
  • query: The PromQL expression to evaluate.
  • start: Start timestamp in Unix epoch seconds.
  • end: End timestamp in Unix epoch seconds.
  • step: Query resolution in seconds (interval between data points).
  • language: Query language type (promql).
  • table: Pinot table name containing the metrics data.

​
Supported PromQL Expressions

​
Series Selectors

  • Query Example: http_requests_total
  • Description: Selects time series that match the given metric name.
  • Usage: To retrieve multiple time series.

​
Label Matching and Filtering

  • Query Examples: 
    http_requests_total{method="GET", status="200"}
http_requests_total{method="GET", status="200"}[5m]
  • Operators: =, !=, =~ (regex match), !~ (regex not match).
  • Usage: To narrow down queries to specific metrics or labels.

​
Offset

  • Query Example: http_requests_total offset 10m
  • Description: Shifts the time series back or forward in time.
  • Usage: To compare current data with past data.

​
Functions

  • Supported Functions:
    • Rate: rate(http_requests_total[5m])
    • Increase: increase(http_requests_total[5m])
    • Delta: delta(http_requests_total[5m])
    • Instant Rate: irate(http_requests_total[5m])
  • Usage: To compute rates, changes, and other complex calculations.

​
Aggregation Operators

  • Supported operators:
    • Sum: sum(http_requests_total)
    • AVG: avg(http_requests_total)
    • MAX: max(http_requests_total)
    • MIN: min(http_requests_total)
    • Count: count(http_requests_total)
    • Topk: topk(1, http_requests_total)
    • Bottom: bottomk(2, http_requests_total)
  • Usage: To calculate sums, averages, and other metrics across multiple time series.

​
By and Without Operators

  • Examples: 
    sum(http_requests_total) by (status)
sum(http_requests_total) without(job)
  • Description: Apply aggregation operator by or without labels.

​
Binary Operators

  • Supported operators: +, -, *, /
  • Examples: 
    http_requests_total / http_requests_failed
http_requests_total - http_requests_failed
  • Usage: To combine multiple series mathematically.
For detailed information about these operators and functions, please refer to the Prometheus Query Language Documentation.
Only the operators and functions listed above are supported in Pinot, which is a subset of all Prometheus functions and operators.