Skip to main content
Do these steps once in Databricks before you onboard a Unity Catalog table, whichever storage your data lives in. StarTree reads Unity Catalog over the Iceberg REST protocol. That covers native Iceberg tables and Delta Lake tables with UniForm turned on. Data files must be Parquet.

Table requirements

The table must be an Iceberg table, or a Delta table with UniForm’s Iceberg support turned on. A Delta table isn’t exposed over the Iceberg REST endpoint until UniForm is on. For a new Delta table, set the properties when you create it. Databricks recommends id column mapping for new tables:
For an existing table, use ALTER TABLE. name is the only column-mapping mode an existing table can be upgraded to; id can’t be applied after creation:
To check an existing table, run SHOW TBLPROPERTIES <catalog>.<schema>.<table>; and look for the properties above. Iceberg Compatibility V3 needs Databricks Runtime 18 LTS or later and applies to Unity Catalog managed tables. If the table already has V2 on, set 'delta.enableIcebergCompatV2' = 'false' in the same ALTER TABLE, as shown. See Use Apache Iceberg v3 features.
On an older runtime? Use Iceberg Compatibility V2 instead: set 'delta.enableIcebergCompatV2' = 'true' in place of the V3 property in either recipe. V2 needs all writers on Databricks Runtime 14.3 LTS or later, and turning it on for an existing table with ALTER TABLE needs 15.4 LTS or later. See the Databricks UniForm requirements. StarTree onboards either level. The difference that matters downstream is how deletes are stored; see Databricks UniForm caveats.
Column mapping. Delta tables with delta.columnMapping.mode set to name or id are supported. That’s mandatory for UniForm’s Iceberg reads, so it covers most Databricks-managed UniForm tables. Field names inside complex columns (struct, array<struct>, map<_,struct>) are resolved by the storeNestedSchema.enabled task setting, which is on by default. See Schema options for column mapping.

Databricks UniForm caveats

These are Databricks-side behaviours of UniForm that affect a UniForm-backed External Table:
  • New data appears only after Iceberg metadata generation. Databricks converts Delta commits to Iceberg metadata asynchronously, batches frequent commits into fewer Iceberg commits, and skips generation while another run is in progress. So the Iceberg view, and therefore StarTree’s sync, can lag the Delta table. End-to-end freshness is the conversion delay plus the sync schedule. Run MSCK REPAIR TABLE <table> SYNC METADATA in Databricks to force a refresh if the Iceberg view is stale.
  • Keep VACUUM retention longer than your sync interval. VACUUM deletes Parquet files that older Iceberg snapshots still reference, and a sync reading a snapshot older than the retention window fails when it fetches those files. Set retention well above the sync interval, plus any time the sync might be paused.
  • Materialized views and streaming tables need the pipeline-managed path. Iceberg Compatibility V2 doesn’t support them at all. With V3, pipeline-managed materialized views and streaming tables can be exposed over the Iceberg REST endpoint on Databricks Runtime 17.3 or later, in default publishing mode, and only with 'delta.enableRowTracking' = 'true' and 'pipelines.externalMetadata.enabled' = 'true' set alongside the UniForm properties. See Enable external data access to streaming tables and materialized views. Tables with VOID columns stay unsupported.
  • Turning on UniForm can’t be undone for the Delta table. It permanently upgrades the table protocol (minReaderVersion ≥ 2, minWriterVersion ≥ 7) and switches new files from Snappy to Zstandard compression. StarTree reads both, but older external Delta clients reading the same table may break, so check the table’s other consumers first.
  • Deletes can produce Iceberg deletion vectors under Compatibility V3. V3 permits deletion vectors and Databricks turns them on by default for new Iceberg v3 tables, so updates and deletes may be written as Puffin delete files instead of rewritten data files. StarTree applies Iceberg v3 Puffin position deletes at query time, but that’s a preview feature you have to turn on: see Deletion Vectors and contact StarTree to enable it. Equality deletes aren’t applied. A table left at V2 disallows deletion vectors, so deletes stay copy-on-write and every snapshot is self-contained.

Catalog authentication

Pick one way for StarTree to authenticate to the catalog: The all-apis scope matters. Browsing, preview and every sync run go through the Unity Catalog Iceberg REST API, so a token with a narrower scope is rejected (usually a 401 or 403 when the connection is validated) even though it’s valid and unexpired. If your workspace lets you pick a scope when generating a token, pick all-apis. Tokens created without an explicit scope get it by default. Store the client secret or PAT as a StarTree secret and reference it as srn2:secret#... in SQL and table configs. Don’t paste raw values.

Catalog endpoint formats

StarTree uses the endpoint exactly as you give it. It doesn’t rewrite the host or path; it only appends the standard Iceberg REST /v1/... routes. The catalog name you onboard from is used as the warehouse.

Set up a service principal and grants

  1. Create a service principal (skip if you use a PAT). Go to Settings → Identity and access → Service principals → Add. Open it, go to Secrets → Generate secret, and copy the client ID and secret. The secret is shown once.
  2. Grant access. Run this in a SQL editor, using the service principal’s client ID or the PAT owner:
    Grant SELECT on the schema instead to cover every table in it. EXTERNAL USE SCHEMA is what lets Unity hand short-lived storage credentials to StarTree.
  3. Turn on external data access if Unity will vend storage credentials. In the workspace, open Catalog, click the gear icon, choose Metastore, open the Details tab and turn on External data access. Only a metastore admin sees this switch. See Enable external data access to Unity Catalog.
Without step 3 or EXTERNAL USE SCHEMA, browsing the catalog works but table loads fail.

Common errors

These come from the catalog connection, whichever storage the table uses: