Snowflake Horizon Catalog on StarTree — query Snowflake Iceberg tables

This feature requires StarTree release 0.16.0 or later, and must be enabled on demand — contact StarTree support to activate it.

This page shows how to onboard a Snowflake Horizon Catalog External Table with the StarTree controller REST APIs. There is no Data Portal wizard for Horizon yet — this API flow is the only way to onboard one today. Snowflake Horizon exposes Snowflake-managed Iceberg tables over the same Iceberg REST protocol as Apache Polaris, served on the Snowflake account host under /polaris/api/catalog. StarTree reaches it through the generic spec-compliant adapter — catalogType=iceberg-rest, serviceType=rest. Data files must be Parquet.

Running self-managed Apache Polaris or Snowflake Open Catalog instead? Those use principal client-id/client-secret credentials — see Apache Polaris & Snowflake Open Catalog: Onboarding via API.

How Horizon differs from Apache Polaris

Horizon speaks the Polaris REST protocol, but authentication and identifier mapping follow Snowflake’s model, not Polaris’s:

	Snowflake Horizon Catalog
REST base URI	`https://<orgname>-<account_name>.snowflakecomputing.com/polaris/api/catalog`
Identity	Snowflake service user + a programmatic access token (PAT) (or key-pair JWT)
Catalog credential	the PAT, passed via `catalog.iceberg-rest.credential` (see Authentication)
`oauthScopes`	`session:role:<snowflake-role>` — Polaris-style `PRINCIPAL_ROLE:ALL` returns `401`
`warehouse`	the Snowflake database name
Namespace	the Snowflake schema (single level; the database is the prefix)
`prefix`	required — set it to the database name (same value as `warehouse`)
Access control	native Snowflake RBAC (`GRANT SELECT` / `USAGE`); the PAT also needs a network policy

Why the PAT goes in catalog.iceberg-rest.credential. Horizon’s token endpoint expects a client_credentials exchange that sends the PAT as client_secret with no client_id. Passing the PAT through catalog.iceberg-rest.credential produces exactly that request. (Setting oauthClientSecret with an empty oauthClientId does not currently work — the empty client id causes the credential to be dropped and the catalog call goes out unauthenticated. Use catalog.iceberg-rest.credential until first-class secret-only support lands.)

How it works

Onboarding is four calls. Each one feeds the next:

Validate & browse   POST /connections/browse   →  validate; list namespaces/tables
Preview             POST /tables/preview       →  infer schema + enriched table config
Create schema       POST /schemas              →  register the schema
Create table        POST /tables               →  register the External Table

The preview call (step 2) is the core step: it samples the source, infers a Pinot schema, and returns a ready-to-use table config. Steps 3 and 4 just persist its output. Once the table exists, the controller’s watcher runs the first sync and re-syncs on schedule — no manual trigger. Track progress with the observability endpoints.

All paths are relative to your data-plane base URL. Set export BASE_URL=https://dp.<data-plane-id>.cp.<region>.startree.cloud/api/pinot (the StarTree Cloud data-plane proxy). A browse call then posts to $BASE_URL/connections/browse.If your controller requires authentication, add an Authorization header to every request — e.g. -H "Authorization: Bearer <token>". The examples below omit it for brevity.

Prerequisites

StarTree 0.16.0 or later with the External Table Beta feature enabled, and tiered storage configured on the cluster. Contact StarTree support if unsure.
Network access to the controller REST endpoint, plus an Authorization token if your cluster requires auth.
A Snowflake account with Horizon Catalog and the target Iceberg table backed by an external volume (S3).
A Snowflake service user with a programmatic access token (PAT), a role that can read the table, and a network policy governing the user (see Snowflake setup below).
The Snowflake database (used as both warehouse and prefix), the schema (the namespace), and the table to onboard.
Read access to the underlying object storage. This connector supports S3-compatible storage for the data files.

Snowflake setup

Run these once in Snowflake (or have an ACCOUNTADMIN do so). Replace the placeholders with your own names.

-- 1. A role that can read the Iceberg table's database, schema, and external volume.
GRANT USAGE   ON DATABASE <database>                 TO ROLE <role>;
GRANT USAGE   ON SCHEMA   <database>.<schema>         TO ROLE <role>;
GRANT SELECT  ON ALL TABLES IN SCHEMA <database>.<schema> TO ROLE <role>;
GRANT USAGE   ON EXTERNAL VOLUME <external_volume>    TO ROLE <role>;

-- 2. A dedicated service user, and the role granted to it.
CREATE USER IF NOT EXISTS <service_user> DEFAULT_ROLE = <role>;
GRANT ROLE <role> TO USER <service_user>;

-- 3. A network policy is REQUIRED before a PAT can be created or used.
ALTER USER <service_user> SET NETWORK_POLICY = <network_policy>;

-- 4. The programmatic access token. Copy the secret it prints — it is shown only once.
ALTER USER IF EXISTS <service_user>
  ADD PAT startree_external_table
    DAYS_TO_EXPIRY   = 90
    ROLE_RESTRICTION = '<role>'
    COMMENT          = 'StarTree External Table catalog access';

The ROLE_RESTRICTION role must match the role named in the session:role:<role> scope you configure below, and it must hold the grants from step 1. Without a network policy in effect, ADD PAT fails and the token endpoint later returns 401 unauthorized_client.

Authentication

An External Table authenticates in two places, configured independently:

Catalog (REST) — the Snowflake PAT, supplied via catalog.iceberg-rest.credential plus the token URI and scope.
Storage (S3 data files) — keys under catalog.iceberg-rest.auth.storage.*. Choose one of the methods below.

Catalog (REST) authentication

Pass the PAT as the catalog credential. StarTree exchanges it at the Snowflake token endpoint for a short-lived token, which it caches and refreshes automatically.

{
  "catalog.iceberg-rest.auth.rest.authType": "oauth2",
  "catalog.iceberg-rest.auth.rest.oauthTokenUri": "https://<orgname>-<account_name>.snowflakecomputing.com/polaris/api/catalog/v1/oauth/tokens",
  "catalog.iceberg-rest.auth.rest.oauthScopes": "session:role:<role>",
  "catalog.iceberg-rest.credential": "<pat>"
}

Key	Value	Description
`catalog.iceberg-rest.credential`	`<pat>`	The Snowflake programmatic access token. Sent to the token endpoint as `client_secret` with no client id.
`catalog.iceberg-rest.auth.rest.oauthTokenUri`	`https://<orgname>-<account_name>.snowflakecomputing.com/polaris/api/catalog/v1/oauth/tokens`	Snowflake’s OAuth token endpoint (the account host, `/polaris/api/catalog/v1/oauth/tokens`).
`catalog.iceberg-rest.auth.rest.oauthScopes`	`session:role:<role>`	Activates the Snowflake role for the session. Must match the PAT’s `ROLE_RESTRICTION`.
`catalog.iceberg-rest.auth.rest.authType`	`oauth2`	Optional; auto-detected as `oauth2` when a credential is present.

The PAT is a secret — treat the table config as sensitive and rotate the token on the DAYS_TO_EXPIRY cadence. A static PAT is not auto-rotated; when it expires, syncs fail until you update the table config with a new one. Key-pair JWT auth (a signed JWT in place of the PAT) works the same way — put the JWT in catalog.iceberg-rest.credential.

Storage authentication: choose one

Static access keys

What most Snowflake external-volume setups use. Accepts either accessKeyId/secretAccessKey or the shorter accessKey/secretKey aliases.

{
  "catalog.iceberg-rest.auth.storage.accessKeyId": "<key>",
  "catalog.iceberg-rest.auth.storage.secretAccessKey": "<secret>",
  "catalog.iceberg-rest.auth.storage.region": "<aws-region>"
}

Assumed IAM role (recommended for production)

No static secrets in the table config — every S3 read assumes roleArn via STS.

{
  "catalog.iceberg-rest.auth.storage.roleArn": "arn:aws:iam::123456789012:role/pinot-external-table-reader",
  "catalog.iceberg-rest.auth.storage.externalId": "<external-id-if-your-trust-policy-requires-one>",
  "catalog.iceberg-rest.auth.storage.region": "<aws-region>"
}

Cluster node role

Omit every auth.storage.* credential key — only region is required. Credential resolution falls back to the AWS SDK default chain (the node’s instance profile / IRSA role), which must already have S3 access.

{
  "catalog.iceberg-rest.auth.storage.region": "<aws-region>"
}

Horizon can vend short-lived storage credentials, but the generic rest adapter does not consume vended credentials — configure static keys, an assumed role, or the cluster’s node role for storage explicitly.

Selecting the database

Set both catalog.iceberg-rest.warehouse and catalog.iceberg-rest.prefix to the Snowflake database name. prefix is required for Horizon: the generic adapter doesn’t send the warehouse query parameter that Snowflake’s /v1/config gates on, so without an explicit prefix, browse falls back to the un-prefixed /v1/namespaces path and fails with 404 Unable to find matching target resource method.

{
  "catalog.iceberg-rest.warehouse": "<database>",
  "catalog.iceberg-rest.prefix": "<database>"
}

The Snowflake schema is the Iceberg namespace — set catalog.iceberg-rest.table.namespace to the schema name and catalog.iceberg-rest.table.tableName to the table.

Step 1: Validate and browse the connection

POST /connections/browse There is no separate “validate” endpoint. Browsing the catalog is the validation step: a 200 with an items list (even an empty one) confirms your PAT, scope, and connectivity.

Set path to "" to browse the root — this both validates the connection and lists schemas (namespaces).
Set path to a schema name to list the tables inside it.

Request

Field	Type	Required	Description
`connection.type`	string	Yes	Use `CATALOG` for all External Table sources.
`connection.params`	object	Yes	Catalog-specific connection settings (see example below).
`path`	string	No	Where to browse. `""` or omitted = root. A schema name = that schema’s tables.

curl -X POST "$BASE_URL/connections/browse" \
  -H "Content-Type: application/json" \
  -d '{
    "connection": {
      "type": "CATALOG",
      "params": {
        "catalogType": "iceberg-rest",
        "catalog.iceberg-rest.serviceType": "rest",
        "catalog.iceberg-rest.restUri": "https://<orgname>-<account_name>.snowflakecomputing.com/polaris/api/catalog",
        "catalog.iceberg-rest.warehouse": "<database>",
        "catalog.iceberg-rest.prefix": "<database>",
        "catalog.iceberg-rest.auth.rest.authType": "oauth2",
        "catalog.iceberg-rest.auth.rest.oauthTokenUri": "https://<orgname>-<account_name>.snowflakecomputing.com/polaris/api/catalog/v1/oauth/tokens",
        "catalog.iceberg-rest.auth.rest.oauthScopes": "session:role:<role>",
        "catalog.iceberg-rest.credential": "<pat>",
        "catalog.iceberg-rest.auth.storage.roleArn": "arn:aws:iam::123456789012:role/pinot-external-table-reader",
        "catalog.iceberg-rest.auth.storage.externalId": "<external-id-if-your-trust-policy-requires-one>",
        "catalog.iceberg-rest.auth.storage.region": "<aws-region>"
      }
    },
    "path": ""
  }'

Key	Value	Description
`catalog.iceberg-rest.restUri`	`https://<orgname>-<account_name>.snowflakecomputing.com/polaris/api/catalog`	The Snowflake Iceberg REST base URI.
`catalog.iceberg-rest.serviceType`	`rest`	Selects the generic spec-compliant adapter — there’s no Snowflake-specific `serviceType`.
`catalog.iceberg-rest.warehouse`	`<database>`	The Snowflake database to browse.
`catalog.iceberg-rest.prefix`	`<database>`	Pins the catalog prefix (the database). Required for Horizon.

Response

{
  "items": [
    { "name": "SCHEMA_ONE", "type": "NAMESPACE" },
    { "name": "nyc_taxi_trips",  "type": "TABLE" }
  ]
}

`items[].type`	Meaning
`NAMESPACE`	A schema you can drill into — pass its `name` as `path`.
`TABLE`	A selectable table.

Step 2: Preview the schema

POST /tables/preview Samples the source, infers a Pinot schema, and returns an enriched table config (S3 tier, raw field configs, time column) plus sample rows. Review it, tweak if needed, then carry the schema and config forward to steps 3 and 4.

The request and response share the same JSON shape. You send a tableConfig describing the source; the response fills in schema, the enriched tableConfigs.offline, sampled rows, and a summary.

Request

Top-level fields:

Field	Type	Required	Description
`tableConfig`	object	Yes	A single OFFLINE table config whose `ExternalTableSyncTask` block describes the source.
`config`	object	No	Sampling and inference controls (see below). Defaults are applied if omitted.
`schema`	object	No	An explicit schema to use instead of inferring one.

config.inference — how the schema is derived:

Field	Type	Default	Description
`embeddedSchemaEnabled`	boolean	`false`	Read the schema embedded in the Iceberg metadata. Set this to `true` for External Tables.
`schemaInferenceEnabled`	boolean	`false`	Infer the schema from sampled rows. Fallback when no embedded schema exists.

Resolution order: explicit schema → embedded schema → inferred schema. config.sampling — how rows are sampled:

Field	Type	Default	Description
`min`	integer	`1`	Minimum records to sample.
`max`	integer	`100`	Maximum records to sample.

curl -X POST "$BASE_URL/tables/preview" \
  -H "Content-Type: application/json" \
  -d '{
    "tableConfig": {
      "tableName": "nyc_taxi_trips_OFFLINE",
      "tableType": "OFFLINE",
      "task": {
        "taskTypeConfigsMap": {
          "ExternalTableSyncTask": {
            "catalogType": "iceberg-rest",
            "executor": "controller",
            "inputFormat": "parquet",
            "catalog.iceberg-rest.serviceType": "rest",
            "catalog.iceberg-rest.restUri": "https://<orgname>-<account_name>.snowflakecomputing.com/polaris/api/catalog",
            "catalog.iceberg-rest.warehouse": "<database>",
            "catalog.iceberg-rest.prefix": "<database>",
            "catalog.iceberg-rest.table.namespace": "<schema>",
            "catalog.iceberg-rest.table.tableName": "nyc_taxi_trips",
            "catalog.iceberg-rest.auth.rest.authType": "oauth2",
            "catalog.iceberg-rest.auth.rest.oauthTokenUri": "https://<orgname>-<account_name>.snowflakecomputing.com/polaris/api/catalog/v1/oauth/tokens",
            "catalog.iceberg-rest.auth.rest.oauthScopes": "session:role:<role>",
            "catalog.iceberg-rest.credential": "<pat>",
            "catalog.iceberg-rest.auth.storage.roleArn": "arn:aws:iam::123456789012:role/pinot-external-table-reader",
            "catalog.iceberg-rest.auth.storage.externalId": "<external-id-if-your-trust-policy-requires-one>",
            "catalog.iceberg-rest.auth.storage.region": "<aws-region>"
          }
        }
      }
    },
    "config": { "inference": { "embeddedSchemaEnabled": true } }
  }'

Setting the namespace and table. Take them from the browse response — set catalog.iceberg-rest.table.namespace (the Snowflake schema) and .tableName to the NAMESPACE and TABLE you selected.Use executor: controller — it’s required for the controller-watcher flow and the observability endpoints. The input tableConfig is intentionally minimal; /tables/preview returns the complete tableConfigs.offline you persist in Step 4.

Response

Field	Type	Description
`schema`	object	The resolved Pinot schema. Use this in step 3.
`tableConfigs.offline`	object	The enriched OFFLINE table config, ready to persist. Use this in step 4. Secrets are masked as `*****`.
`rows`	array	Sample records after schema + ingestion transforms.
`sourceRows`	array	Raw records before transformation.
`summary`	object	Run summary: `nSourceRows`, `nRows`, `nColumns`, and `summary.batch.nMatchingFiles` (files found).

Adjust the schema (time column, column names, null handling) before moving on.

Step 3: Create the schema

POST /schemas Send the schema object from the preview response. Rename its schemaName to match your table.

curl -X POST "$BASE_URL/schemas" \
  -H "Content-Type: application/json" \
  -d @schema.json

{ "status": "nyc_taxi_trips successfully added" }

Step 4: Create the table

POST /tables Send the enriched tableConfigs.offline object from the preview response. Its ExternalTableSyncTask block is what marks the table as external.

curl -X POST "$BASE_URL/tables" \
  -H "Content-Type: application/json" \
  -d @tableConfig.json

{ "status": "Table nyc_taxi_trips_OFFLINE successfully added" }

The controller’s External Table watcher then discovers the table, runs the first sync, and re-syncs at the schedule (cron) interval. There is no separate start call.

The first sync runs on the watcher’s next tick. To kick it off immediately instead of waiting, you can manually trigger a run:

curl -X POST "$BASE_URL/tasks/schedule?taskType=ExternalTableSyncTask&tableName=nyc_taxi_trips_OFFLINE"

Quickstart: onboard a table end-to-end

The whole flow as one script. Fill in the variables at the top, run the script, and the first sync starts automatically.

#!/usr/bin/env bash
set -euo pipefail

BASE_URL="https://dp.<data-plane-id>.cp.<region>.startree.cloud/api/pinot"
AUTH="Authorization: Bearer <token>"
ACCOUNT_HOST="<orgname>-<account_name>.snowflakecomputing.com"
REST_URI="https://$ACCOUNT_HOST/polaris/api/catalog"
TOKEN_URI="https://$ACCOUNT_HOST/polaris/api/catalog/v1/oauth/tokens"
DATABASE="<database>"          # used as both warehouse and prefix
SCHEMA="<schema>"              # the Iceberg namespace
TABLE="nyc_taxi_trips"
ROLE="<role>"
SCOPES="session:role:$ROLE"
PAT="<pat>"
STORAGE_REGION="<aws-region>"
STORAGE_ROLE_ARN="arn:aws:iam::123456789012:role/pinot-external-table-reader"
STORAGE_EXTERNAL_ID="<external-id-if-your-trust-policy-requires-one>"

# 1. Validate & browse root — lists schemas in the Snowflake database.
curl -sf -X POST "$BASE_URL/connections/browse" \
  -H "$AUTH" -H 'Content-Type: application/json' \
  --data-raw "{
    \"connection\": {
      \"type\": \"CATALOG\",
      \"params\": {
        \"catalogType\": \"iceberg-rest\",
        \"catalog.iceberg-rest.serviceType\": \"rest\",
        \"catalog.iceberg-rest.restUri\": \"$REST_URI\",
        \"catalog.iceberg-rest.warehouse\": \"$DATABASE\",
        \"catalog.iceberg-rest.prefix\": \"$DATABASE\",
        \"catalog.iceberg-rest.auth.rest.authType\": \"oauth2\",
        \"catalog.iceberg-rest.auth.rest.oauthTokenUri\": \"$TOKEN_URI\",
        \"catalog.iceberg-rest.auth.rest.oauthScopes\": \"$SCOPES\",
        \"catalog.iceberg-rest.credential\": \"$PAT\"
      }
    },
    \"path\": \"\"
  }" | jq .

# 2. Preview — infer schema + enriched table config.
curl -sf -X POST "$BASE_URL/tables/preview" \
  -H "$AUTH" -H 'Content-Type: application/json' \
  --data-raw "{
    \"tableConfig\": {
      \"tableName\": \"${TABLE}_OFFLINE\",
      \"tableType\": \"OFFLINE\",
      \"task\": {
        \"taskTypeConfigsMap\": {
          \"ExternalTableSyncTask\": {
            \"catalogType\": \"iceberg-rest\",
            \"executor\": \"controller\",
            \"inputFormat\": \"parquet\",
            \"catalog.iceberg-rest.serviceType\": \"rest\",
            \"catalog.iceberg-rest.restUri\": \"$REST_URI\",
            \"catalog.iceberg-rest.warehouse\": \"$DATABASE\",
            \"catalog.iceberg-rest.prefix\": \"$DATABASE\",
            \"catalog.iceberg-rest.table.namespace\": \"$SCHEMA\",
            \"catalog.iceberg-rest.table.tableName\": \"$TABLE\",
            \"catalog.iceberg-rest.auth.rest.authType\": \"oauth2\",
            \"catalog.iceberg-rest.auth.rest.oauthTokenUri\": \"$TOKEN_URI\",
            \"catalog.iceberg-rest.auth.rest.oauthScopes\": \"$SCOPES\",
            \"catalog.iceberg-rest.credential\": \"$PAT\",
            \"catalog.iceberg-rest.auth.storage.roleArn\": \"$STORAGE_ROLE_ARN\",
            \"catalog.iceberg-rest.auth.storage.externalId\": \"$STORAGE_EXTERNAL_ID\",
            \"catalog.iceberg-rest.auth.storage.region\": \"$STORAGE_REGION\"
          }
        }
      }
    },
    \"config\": { \"inference\": { \"embeddedSchemaEnabled\": true } }
  }" > preview.json

# 3. Extract schema and table config from preview response.
jq --arg t "$TABLE" '.schema | .schemaName = $t' preview.json > schema.json
jq '.tableConfigs.offline'                        preview.json > tableConfig.json

# 4. Create the schema.
curl -sf -X POST "$BASE_URL/schemas" \
  -H "$AUTH" -H 'Content-Type: application/json' -d @schema.json

# 5. Create the table — watcher starts the first sync automatically.
curl -sf -X POST "$BASE_URL/tables" \
  -H "$AUTH" -H 'Content-Type: application/json' -d @tableConfig.json

# 6. (optional) Poll until sync completes.
curl -sf "$BASE_URL/tables/${TABLE}_OFFLINE/externalTable/status" -H "$AUTH" \
  | jq -e '.status == "COMPLETED" and .segmentsUploaded == .filesDiscovered'

Once step 6 exits 0, verify it’s queryable.

Monitor onboarding

The status endpoint reports each sync run’s progress, and ?includeLag=true adds how far the table trails the upstream catalog. Both require executor=controller (set automatically by this flow). See Observability for full request and response details.

Verify it’s queryable

When status is COMPLETED and segmentsUploaded matches filesDiscovered, run a query against the broker (or the Data Portal query console) to confirm the data is live:

SELECT count(*) FROM nyc_taxi_trips;

The count will be much larger than the preview’s summary.nSourceRows (preview only samples up to ~100 rows) — confirm it’s non-zero and plausible for your dataset. If status is COMPLETED but the count is 0, give segments a moment to load on the servers, then recheck; if it persists, see Troubleshooting.

What’s next

Now that the table is created and data is loading, these are the highest-impact follow-up steps:

Add indexes for your query patterns. Without indexes, every query scans all remote Parquet data. Add a range index on time/numeric columns, an inverted index on low-cardinality filter columns, and a bloom filter on high-cardinality ID columns. → Indexes
Enable caching and preload. Set enable.prefetch.page.cache=true and preload.enable=true on the S3 tier so index data is served from local disk on repeated queries instead of re-fetched from S3. → Data and Index Caching
Protect large-scan queries from OOM. For tables that receive heavy aggregations or wide scans, enable the query OOM killer so a runaway query is killed instead of crashing the server. → Best Practices & Configs — Query OOM protection
Monitor ongoing syncs. Use the status endpoint (with includeLag=true for ingestion lag) after each scheduled sync. → Observability

For common questions and failures, see the FAQ and Troubleshooting.

Get Started

Ingestion

External Tables

Query Data

Manage Data

Observability

Visualize Data

Cluster Operations

Manage Security

Release Notes

Reference

Snowflake Horizon Catalog: Onboarding via API

How Horizon differs from Apache Polaris

How it works

Prerequisites

Snowflake setup

Authentication

Catalog (REST) authentication

Storage authentication: choose one

Static access keys

Assumed IAM role (recommended for production)

Cluster node role

Selecting the database

Step 1: Validate and browse the connection

Request

Response

Step 2: Preview the schema

Request

Response

Step 3: Create the schema

Step 4: Create the table

Quickstart: onboard a table end-to-end

Monitor onboarding

Verify it’s queryable

What’s next

​How Horizon differs from Apache Polaris

​How it works

​Prerequisites

​Snowflake setup

​Authentication

​Catalog (REST) authentication

​Storage authentication: choose one

​Static access keys

​Assumed IAM role (recommended for production)

​Cluster node role

​Selecting the database

​Step 1: Validate and browse the connection

​Request

​Response

​Step 2: Preview the schema

​Request

​Response

​Step 3: Create the schema

​Step 4: Create the table

​Quickstart: onboard a table end-to-end

​Monitor onboarding

​Verify it’s queryable

​What’s next

How Horizon differs from Apache Polaris

How it works

Prerequisites

Snowflake setup

Authentication

Catalog (REST) authentication

Storage authentication: choose one

Static access keys

Assumed IAM role (recommended for production)

Cluster node role

Selecting the database

Step 1: Validate and browse the connection

Request

Response

Step 2: Preview the schema

Request

Response

Step 3: Create the schema

Step 4: Create the table

Quickstart: onboard a table end-to-end

Monitor onboarding

Verify it’s queryable

What’s next