This feature is available starting in StarTree release 0.17.0. It requires a StarTree data plane on Google Cloud.
Prerequisites
Grant access to the StarTree principal
Pinot authenticates as the IAM principal of its Kubernetes service account. StarTree Support provides the principal for your data plane. It has this form:roles/managedkafka.client) on the project that contains the cluster:
READ the topic. Topics without ACLs are open to every authenticated principal.
Connect the network
Pinot must reach the cluster’s bootstrap address. Connect the subnet of your StarTree data plane to the cluster; StarTree Support provides the subnet. Managed Kafka then creates DNS entries for the cluster in that network.- The subnet must be in the cluster’s region. A cluster accepts up to 10 subnets.
- If the data plane runs in a different project from the cluster, the data plane’s project must grant the Managed Kafka Service Agent role (
roles/managedkafka.serviceAgent) toservice-<CLUSTER_PROJECT_NUMBER>@gcp-sa-managedkafka.iam.gserviceaccount.com. Coordinate this step with StarTree Support.
Get the bootstrap address
bootstrap.<CLUSTER_ID>.<REGION>.managedkafka.<KAFKA_PROJECT_ID>.cloud.goog:9092.
Create the table
1
In the Data Portal, click Tables and then click Create Table.
2
Select Custom as the Data Source.
3
4
Test the Connection and Configure Data Ingestion
Click Test Connection. When the test succeeds, configure ingestion:These are the standard Kafka ingestion properties. For descriptions, see Configure Data Ingestion in Connect to Apache Kafka.
5
Preview the Data
Click Show Sample Data to preview the source data.
Table configuration example
Table configuration example
To create the table without the Data Portal, put the connection and ingestion properties in
tableIndexConfig.streamConfigs:Connection requirements
Pinot sends the data plane’s Google access token only to Managed Kafka brokers over verified TLS. Before it requests a token, Pinot checks these settings and rejects the connection with an error that names the setting:Troubleshooting
Next Step
Proceed with Data Modeling.

