Skip to main content
This feature requires StarTree release 0.15.0 or later, and must be enabled on demand — contact StarTree support to activate it.
An External Table is a Pinot table whose data stays in Parquet files in your object store — an Iceberg catalog (AWS Glue, Amazon S3 Tables, Unity Catalog, Nessie) or an S3/GCS data lake — instead of being copied into Pinot’s own segment format. Pinot reads the remote Parquet at query time and exposes it through standard SQL. There is no ETL pipeline, no data duplication, and onboarding takes minutes instead of hours. A watcher on the controller detects source-side changes on a schedule and builds segment index files (bloom filters, inverted indexes, range indexes, etc.) alongside the Parquet. Queries use server-side caches — a Parquet data cache, an index cache, and a footer cache — so repeat reads avoid paying S3 round-trip latency. With the right indexes configured, query times drop from minutes to milliseconds on large datasets.
For a single-page view of everything External Tables support — connectors, authentication methods, table formats, and features — see the Support Matrix.

​
How it works

At query time, the server checks its local caches first. On a miss, it fetches the required Parquet column pages or index byte ranges from object storage and stores them for subsequent queries. Index files (built by the watcher at sync time) live in tiered storage and are also cached locally, so filters and aggregations avoid full column scans.

​
Supported sources

catalogType=iceberg-rest works with any Iceberg REST–compliant catalog (built-in service adapters exist for AWS Glue, AWS S3 Tables, Unity Catalog, and generic REST — the last of which also covers Nessie, addressed via its branch/ref as the REST prefix). Data files must be Parquet.
There is no dedicated catalogType for Delta Lake’s native log format. A Delta Lake table with UniForm enabled publishes Iceberg-compatible metadata alongside its Delta log — registering it in a Unity Catalog (serviceType=unity) makes it queryable as an External Table through the same Iceberg REST path used for Glue and S3 Tables.

​
Where to start

New here? Start with a task instead of a vendor: Otherwise, choose your catalog type and pick your preferred onboarding path: AWS Glue Amazon S3 Tables Unity Catalog
  • Onboarding via API — 4-step REST flow; supports Databricks-managed and OSS Unity Catalog, including Delta Lake tables with UniForm enabled
  • Onboarding via Data Portal is not yet available — the API flow is the only option today
Nessie
  • Onboarding via API — 4-step REST flow using the generic Iceberg REST adapter with a branch/ref prefix
  • Onboarding via Data Portal is not yet available — the API flow is the only option today
S3 Data Lake GCS Data Lake After onboarding
  • Monitoring sync progress → Observability — sync status, checkpoint watermark, and source file count APIs
  • Queries are slow → Indexes to add the right indexes, then Best Practices & Configs for caching and tuning
  • Something is broken → Troubleshooting for symptom-based fixes, or FAQ for common questions
New in 0.16.0
  • SQL DDL — create an External Table with a single CREATE TABLE ... WITH (...) statement instead of the 4-step REST flow.
  • Segment Groups — group many small member segments into one logical unit to cut cluster metadata overhead for tables with a large number of source files.
  • Deletion Vectors (preview) — apply Iceberg row-level deletes/updates at query time instead of requiring append-only sources.

​
Page map