Skip to main content
A data catalog is a central registry that tracks a table’s schema, partitions, and file locations, so a query engine can look those up instead of scanning a bucket to rediscover them. If your tables are already registered in one of the catalogs below, StarTree reads that registry directly — you get schema inference, partition awareness, and incremental snapshot sync for free, instead of the raw file-scanning approach in Query Parquet Directly.
A catalog is also the right choice when your source compacts. Because a REST catalog tracks the table’s live file set through snapshot metadata, each sync reconciles to the current files even after the source rewrites and deletes Parquet during compaction. The raw S3/GCS Data Lake source pins segments to physical file paths, so compaction breaks them — see Segments fail to load after upstream compaction. If your pipeline compacts, connect a catalog rather than pointing at the bucket directly.

​
Which catalog do you have?

All of them are reached the same way — catalogType=iceberg-rest — over the Iceberg REST protocol; only the service endpoint and authentication differ. Not sure which authentication method to use for your catalog, or whether a given feature (assumed-role auth, vended credentials, OAuth2) is supported? Each catalog’s onboarding guide has an Authentication section with a runnable config sample per method.

​
How onboarding works, regardless of catalog

Every catalog above follows the same four calls: validate & browse the connection, preview to infer a schema, create the schema, then create the table — after which the controller’s watcher takes over and syncs on a schedule with no further manual steps. Each catalog’s onboarding guide walks through all four with runnable examples. Validate & browse also checks that the storage credentials can actually reach the data bucket, not just the catalog endpoint — for S3 Data Lake and S3 Tables this includes confirming the configured region matches the bucket at that step; for the other catalogs, where no bucket is known until a table is picked, the storage region is confirmed later, once a table is selected in preview or CREATE TABLE. See Support Matrix.

​
What’s next

Once the table is created, add indexes for the columns you filter and group by, enable caching, and track sync health through Observability.