serviceType for Polaris — use the generic spec-compliant adapter: catalogType=iceberg-rest, serviceType=rest, with catalog.iceberg-rest.warehouse set to the Polaris catalog name. Data files must be Parquet.
Using Nessie? It rides the same genericrestadapter with a branch/refprefix— see Nessie: Onboarding via API.
Apache Polaris and Snowflake Open Catalog
Snowflake Open Catalog is Snowflake’s managed Apache Polaris service. It speaks the same Iceberg REST protocol and uses the same principal-based OAuth2 (client-id / client-secret, PRINCIPAL_ROLE:ALL scope) as self-managed Polaris — only the host and token-endpoint path differ. Self-managed Polaris serves the REST API under <polaris-host>/api/catalog; Open Catalog serves it under https://<orgname>-<account_name>.snowflakecomputing.com/polaris/api/catalog. Everything else on this page — principal credentials, scope, catalog-name warehouse — applies to both.
Using Snowflake Horizon Catalog instead? Snowflake has stopped provisioning new Open Catalog accounts and now routes customers to Horizon, which exposes the same Polaris REST API on the Snowflake account host but authenticates differently (a service-user PAT passed as the catalog credential, a
session:role:<role> scope, and a required prefix). It has its own page: Snowflake Horizon Catalog: Onboarding via API.How it works
Onboarding is four calls. Each one feeds the next:All paths are relative to your data-plane base URL. Set
export BASE_URL=https://dp.<data-plane-id>.cp.<region>.startree.cloud/api/pinot (the StarTree Cloud data-plane proxy). A browse call then posts to $BASE_URL/connections/browse.If your controller requires authentication, add an Authorization header to every request — e.g. -H "Authorization: Bearer <token>". The examples below omit it for brevity.Prerequisites
- StarTree 0.16.0 or later with the External Table Beta feature enabled, and tiered storage configured on the cluster. Contact StarTree support if unsure.
- Network access to the controller REST endpoint, plus an
Authorizationtoken if your cluster requires auth. - A running Polaris deployment with its Iceberg REST API reachable: self-managed Apache Polaris serves it under
<polaris-base-url>/api/catalog; Snowflake Open Catalog serves it underhttps://<orgname>-<account_name>.snowflakecomputing.com/polaris/api/catalog. - A Polaris principal with client credentials (
client-id/client-secret) whose principal role maps, through a catalog role, to at least read grants on the target catalog (for exampleTABLE_READ_DATA, which implies the list/metadata grants needed for browsing). See the Polaris access-control docs for creating principals, principal roles, catalog roles, and grants. - The Polaris catalog name (used as the
warehouse) and the namespace/table to onboard. - Read access to the underlying object storage. This connector supports S3-compatible storage for the data files.
Limitations
- No credential vending. Polaris can vend short-lived storage credentials to engines that request them, but the generic
restadapter does not consume vended credentials — configure static keys, an assumed role, or the cluster’s node role for storage explicitly.
Authentication
An External Table authenticates in two places, configured independently:- Catalog (REST) — keys under
catalog.iceberg-rest.auth.rest.*. Polaris always requires authentication; choose one of the methods below. - Storage (S3 data files) — keys under
catalog.iceberg-rest.auth.storage.*. Choose one of the three methods below.
Catalog (REST) authentication: choose one
OAuth2 client credentials (recommended)
The standard Polaris method: the principal’sclient-id / client-secret are exchanged at the Polaris token endpoint for a short-lived token, which StarTree caches and refreshes automatically. authType is optional; it’s auto-detected as oauth2 once the three required keys below are all present.
Polaris requires a scope naming the principal role to activate — use PRINCIPAL_ROLE:ALL to activate all of the principal’s roles, or PRINCIPAL_ROLE:<role-name> to restrict the session to one role.
For Snowflake Open Catalog, the token endpoint is the same path on the account host:
https://<orgname>-<account_name>.snowflakecomputing.com/polaris/api/catalog/v1/oauth/tokens.OAuth2 against an external identity provider
Polaris deployments configured to trust an external IdP (Keycloak, Okta, and so on) accept tokens minted by that IdP instead of the built-in token endpoint. Use the same four keys, pointingoauthTokenUri at the IdP’s client-credentials token endpoint and setting whatever scopes the IdP requires.
Bearer token
If you already hold a valid token (for example, minted out-of-band from your IdP), pass it directly.authType doesn’t need to be set explicitly — it’s auto-detected as token auth when .token is present. Note that a static token is not refreshed: when it expires, syncs start failing until the table config is updated, so prefer the OAuth2 client-credentials flow for anything long-running.
Storage authentication: choose one
Assumed IAM role (recommended for production)
No static secrets in the table config — every S3 read assumesroleArn via STS.
Cluster node role
Omit everyauth.storage.* credential key — only region is required. Credential resolution falls back to the AWS SDK default chain (the node’s instance profile / IRSA role), which must already have S3 access.
Static access keys
Quick tests, or when role-based access isn’t available. Accepts eitheraccessKeyId/secretAccessKey or the shorter accessKey/secretKey aliases.
Selecting the catalog
A Polaris server hosts multiple catalogs. Setcatalog.iceberg-rest.warehouse to the catalog name — the adapter passes it to the server’s config endpoint, which resolves the per-catalog prefix automatically.
catalog.iceberg-rest.prefix to the catalog name whenever the prefix can’t be negotiated automatically. This is required for Snowflake Open Catalog and any deployment whose /v1/config endpoint gates on the warehouse query parameter, or that sits behind a proxy which doesn’t forward the config handshake. Without it, browse fails with 404 Unable to find matching target resource method, because the request falls back to the un-prefixed /v1/namespaces path. The examples below set both warehouse and prefix.
Step 1: Validate and browse the connection
POST /connections/browse
There is no separate “validate” endpoint. Browsing the catalog is the validation step: a 200 with an items list (even an empty one) confirms your credentials, scope, and connectivity.
- Set
pathto""to browse the root — this both validates the connection and lists namespaces. - Set
pathto a namespace name to list the tables inside it.
Request
Response
Step 2: Preview the schema
POST /tables/preview
Samples the source, infers a Pinot schema, and returns an enriched table config (S3 tier, raw field configs, time column) plus sample rows. Review it, tweak if needed, then carry the schema and config forward to steps 3 and 4.
The request and response share the same JSON shape. You send a
tableConfig describing the source; the response fills in schema, the enriched tableConfigs.offline, sampled rows, and a summary.Request
Top-level fields:config.inference — how the schema is derived:
Resolution order: explicit
schema → embedded schema → inferred schema.
config.sampling — how rows are sampled:
Setting the namespace and table. Take them from the browse response — set
catalog.iceberg-rest.table.namespace / .tableName to the NAMESPACE and TABLE you selected.Use executor: controller — it’s required for the controller-watcher flow and the observability endpoints. The input tableConfig is intentionally minimal; /tables/preview returns the complete tableConfigs.offline you persist in Step 4.Response
Adjust the schema (time column, column names, null handling) before moving on.
Step 3: Create the schema
POST /schemas
Send the schema object from the preview response. Rename its schemaName to match your table.
Step 4: Create the table
POST /tables
Send the enriched tableConfigs.offline object from the preview response. Its ExternalTableSyncTask block is what marks the table as external.
schedule (cron) interval. There is no separate start call.
Quickstart: onboard a table end-to-end
The whole flow as one script. Fill in the variables at the top, run the script, and the first sync starts automatically.auth.storage.roleArn); swap in static keys or the node-role default (region only) from Authentication if that fits your setup better.
Once step 6 exits 0, verify it’s queryable.
Monitor onboarding
The status endpoint reports each sync run’s progress, and?includeLag=true adds how far the table trails the upstream catalog. Both require executor=controller (set automatically by this flow). See Observability for full request and response details.
Verify it’s queryable
Whenstatus is COMPLETED and segmentsUploaded matches filesDiscovered, run a query against the broker (or the Data Portal query console) to confirm the data is live:
summary.nSourceRows (preview only samples up to ~100 rows) — confirm it’s non-zero and plausible for your dataset. If status is COMPLETED but the count is 0, give segments a moment to load on the servers, then recheck; if it persists, see Troubleshooting.
What’s next
Now that the table is created and data is loading, these are the highest-impact follow-up steps:- Add indexes for your query patterns. Without indexes, every query scans all remote Parquet data. Add a range index on time/numeric columns, an inverted index on low-cardinality filter columns, and a bloom filter on high-cardinality ID columns. → Indexes
-
Enable caching and preload. Set
enable.prefetch.page.cache=trueandpreload.enable=trueon the S3 tier so index data is served from local disk on repeated queries instead of re-fetched from S3. → Data and Index Caching - Protect large-scan queries from OOM. For tables that receive heavy aggregations or wide scans, enable the query OOM killer so a runaway query is killed instead of crashing the server. → Best Practices & Configs — Query OOM protection
-
Monitor ongoing syncs. Use the status endpoint (with
includeLag=truefor ingestion lag) after each scheduled sync. → Observability
For common questions and failures, see the FAQ and Troubleshooting.

