> ## Documentation Index
> Fetch the complete documentation index at: https://docs.startree.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Unity Catalog: Onboarding via Data Portal

> Connect a Databricks or OSS Unity Catalog — including Delta Lake tables with UniForm — to StarTree as an External Table with the point-and-click Data Portal wizard, no API calls required.

<Warning>
  This feature is available starting in **StarTree release 0.16.0**. It must be enabled on demand — contact your StarTree representative to have it activated for your environment.
</Warning>

This guide walks through connecting a **Unity Catalog** to StarTree using the **Data Portal UI**. No API calls or JSON configuration required — Data Portal guides you through catalog connection, table selection, and onboarding setup through a point-and-click interface.

Unity Catalog tables are accessed over the **Iceberg REST protocol**, and this path also covers **Delta Lake tables with [UniForm](https://docs.delta.io/latest/delta-uniform.html) enabled** registered in the catalog. This works for both Databricks-managed and OSS Unity Catalog. Data files must be **Parquet**.

<Warning>
  **Known limitation: Delta tables with column mapping.** Delta tables whose `delta.columnMapping.mode` is `name` or `id` — which covers most Databricks-managed UniForm tables — sync successfully but currently fail segment load. Before onboarding a Databricks UniForm table, check its `delta.columnMapping.mode` table property and contact StarTree support to confirm current status. See [Onboarding via API — Limitations](./onboarding-api#limitations) for details.
</Warning>

> Looking for the API-based approach? See [Unity Catalog: Onboarding via API](./onboarding-api).
>
> Using **AWS Glue** instead? See [AWS Glue: Onboarding via Data Portal](../glue/onboarding-data-portal).
>
> Using **Amazon S3 Tables** instead? See [Amazon S3 Tables: Onboarding via Data Portal](../s3tables/onboarding-data-portal).

***

## Prerequisites

Before starting, ensure you have:

* **StarTree 0.16.0 or later** with the external table Beta feature enabled and **tiered storage configured** for your environment. Contact StarTree support if unsure.
* A running Unity Catalog:
  * **Databricks-managed** — a workspace with Unity Catalog enabled, plus either a **personal access token (PAT)** or **OAuth2 service-principal credentials** (client ID + secret) to authenticate with.
  * **OSS Unity Catalog** — a reachable server. The OSS server is unauthenticated by default, so no token is needed.
* The Unity Catalog **catalog name** (used as the warehouse identifier) and the namespace/table you want to onboard.
* Read access to the underlying S3 object storage. Unity Catalog **vends short-lived storage credentials automatically** for tables it manages, so static keys are usually unnecessary — but the storage **region is always required**.
* **For Delta Lake tables: UniForm must be enabled** on the source table so it's exposed over the Iceberg REST endpoint. See [Onboarding via API — Prerequisites](./onboarding-api#prerequisites) for the exact table properties and Databricks runtime requirements.

***

## Step 1: Open the External Tables

1. Log in to **Data Portal**.
2. In the left navigation, go to **Tables**.
3. Click **+ Connect External Table**.

The wizard opens with a connection configuration screen.

***

## Step 2: Configure the Unity Catalog Connection

Choose **Unity Catalog** as the catalog type. It now appears as a top-level catalog option, alongside AWS Glue and Amazon S3 Tables, rather than being nested under a generic Iceberg REST grouping.

Fill in the credentials and connection details:

**Details**

| Field                       | Description                                               |
| --------------------------- | --------------------------------------------------------- |
| **Catalog Connection Name** | A unique name to identify this connection in Data Portal. |

**Metastore** — how Data Portal authenticates with the Unity Catalog Iceberg REST API.

| Field         | Description                                                                                                                                                                            |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **REST URI**  | The Unity Catalog Iceberg REST endpoint — e.g. `https://<workspace-host>/api/2.1/unity-catalog/iceberg-rest` (Databricks) or `http://<host>:8080/api/2.1/unity-catalog/iceberg` (OSS). |
| **Warehouse** | The Unity Catalog catalog name. Required.                                                                                                                                              |
| **Auth Type** | The authentication method: **Personal Access Token (PAT)** or **OAuth2** for Databricks, or leave unset for an unauthenticated OSS Unity Catalog server.                               |

For **Personal Access Token (PAT)**:

| Field            | Description                                                                                     |
| ---------------- | ----------------------------------------------------------------------------------------------- |
| **Access Token** | Databricks personal access token, generated from **User Settings → Developer → Access tokens**. |

For **OAuth2** (Databricks service principal — recommended over a long-lived PAT for automated pipelines, new in 0.16.0):

| Field               | Description                                                                              |
| ------------------- | ---------------------------------------------------------------------------------------- |
| **OAuth Token URI** | OAuth2 token endpoint.                                                                   |
| **Client ID**       | Service-principal client ID.                                                             |
| **Client Secret**   | Service-principal client secret.                                                         |
| **Scopes**          | Space-separated OAuth2 scopes, if your Unity Catalog deployment requires them. Optional. |

**Storage** — credentials for reading the underlying Parquet data files. Unity Catalog vends short-lived storage credentials automatically for most tables, so these can typically be left blank; only **Region** is always required.

| Field          | Description                                                                                                                                        |
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Access Key** | AWS access key ID for S3 data access. Leave blank to use Unity Catalog's vended credentials.                                                       |
| **Secret Key** | AWS secret access key for S3 data access. Leave blank to use Unity Catalog's vended credentials.                                                   |
| **Region**     | AWS region where the S3 data files are stored. Required even when using vended credentials — Unity's short-lived credentials don't carry a region. |

Click **Validate Connection**. Data Portal calls the catalog's validate endpoint and confirms credentials and connectivity before proceeding.

***

## Step 3: Browse and Select a Table

Once the connection is validated:

1. Data Portal lists the available **namespaces** in the Unity Catalog.
2. Select a namespace to expand its tables.
3. Click the **table** you want to onboard.

Data Portal reads the Iceberg schema and derives a Pinot schema automatically.

***

## Step 4: Review the Schema

The auto-generated Pinot schema is displayed for review. You can:

* **Set a time column** — select the column to use as the Pinot time dimension (optional; leave blank for no time partitioning).
* **Include or exclude partition columns** — toggle whether Iceberg partition columns are added as Pinot dimension columns.
* **Rename the schema** — provide a custom schema name, or accept the default derived from the table name.

Click **Next** when the schema looks correct.

***

## Step 5: Configure the Table

Review and adjust the table configuration:

| Setting                 | Default         | Notes                                                                      |
| ----------------------- | --------------- | -------------------------------------------------------------------------- |
| **Onboarding schedule** | Every 5 minutes | Cron expression controlling how often new Iceberg snapshots are onboarded. |
| **Null handling**       | Enabled         | Required for Iceberg schemas that include nullable columns.                |
| **Segment push type**   | Append          | Each new Iceberg snapshot is ingested as a new set of Pinot segments.      |

Click **Create Table** to register the schema and table with Pinot. Data Portal automatically triggers the first onboarding run immediately after creation — no manual step required.

***

## Step 6: Monitor Onboarding

Once the table is created, onboarding starts automatically. The table detail view shows the status in real time:

* **Running** — the task is actively reading Iceberg snapshots and building Pinot segments.
* **Completed** — onboarding finished successfully. The last ingested snapshot ID is shown.
* **Failed** — onboarding encountered an error. The error message and the number of files discovered vs. segments generated are surfaced to help diagnose the issue.

For deeper observability — watcher status, checkpoint values, and per-snapshot file counts — see the [Observability](../observability) page.

***

## Pausing Onboarding

To pause scheduled onboarding from Data Portal:

1. Open the table in the **Tables** view.
2. Click **Pause Sync**. This removes the `schedule` (cron trigger) from the table's `ExternalTableSyncTask` config and stashes its value under `schedule.paused`, so the watcher stops firing new runs. Any run currently in progress completes normally, and existing segments and the last checkpoint are untouched. Click **Resume Sync** to restore the schedule from `schedule.paused` — onboarding resumes from where it left off; it does not restart from scratch.

<Note>
  This is a separate mechanism from the `enabled` key documented in [Best Practices & Configs](../best-practices-and-configs#table-config--externaltablesynctask) — that key can also be set to `false` directly via the table config or API to stop new runs, but the Data Portal's **Pause Sync** button acts on `schedule`/`schedule.paused`, not `enabled`.
</Note>

***

## Frequently Asked Questions

**The Validate step fails — what should I check?**

* For Databricks with **PAT**: confirm the token is valid and hasn't expired, and that the workspace has Unity Catalog enabled.
* For Databricks with **OAuth2**: confirm the token URI, client ID, and client secret are correct, and that the service principal has access to the catalog.
* For **OSS Unity Catalog**: confirm the server is reachable from the cluster and the REST URI includes the correct path suffix (`/api/2.1/unity-catalog/iceberg`).
* Verify the warehouse value matches the Unity Catalog catalog name exactly.

***

**Table load or sync fails even though browsing worked — what should I check?**

* Confirm the storage **Region** is set — it's required even when relying on Unity's vended credentials, which don't carry a region.
* On Databricks, credential vending requires **external data access enabled on the metastore** and the **`EXTERNAL USE SCHEMA`** privilege on the schema for the principal behind your PAT or OAuth2 credentials. Without them, catalog browsing works but table loads fail. See [Onboarding via API — Storage authentication](./onboarding-api#storage-authentication).

***

**Can I onboard multiple tables from the same catalog?**

Yes. After creating the first table, start the wizard again and reuse the same connection credentials. Each table is registered as an independent Pinot table with its own onboarding schedule.

***

**The table was created but onboarding hasn't started — what should I check?**

Data Portal triggers the first onboarding run automatically after table creation. If onboarding hasn't started, check the table's detail page for an error status and review the error message. You can also trigger a run manually via the [trigger API](../observability#trigger-a-sync-run-optional).