> ## Documentation Index > Fetch the complete documentation index at: https://docs.startree.ai/llms.txt > Use this file to discover all available pages before exploring further. # Table Storage Usage > Understand a table's storage in the StarTree Data Portal — Local, Deep, and Tiered Storage, what Used and Orphaned mean, and the underlying API. ## Overview Unexpected growth in table storage increases both infrastructure and object-storage costs. Pinot keeps a table's data in several places — the Pinot servers' local disks, the deep store, and the remote object stores used by tiered storage — and the **Storage** tab in the StarTree Data Portal reports the size at each location and highlights the gap between what *should* be there and what is *actually* stored. Use it to: * Track the total storage footprint across local disk, deep store, and tiered storage * Spot untracked segments and stale versions that no longer belong to the table * Find deleted segments that are past retention but still consuming deep store space * Debug discrepancies during rebalances, segment replacement, and tiered-storage cleanup Most of what you need is on the Storage tab. If you want to script checks or drill into the specific items behind the numbers, the same data is available from the [storage usage API](#using-the-storage-usage-api). The Storage tab and its API are available starting StarTree Cloud version 0.12.0. ## Reading the Storage tab Open a table in the Data Portal and select the **Storage** tab. Storage is grouped into up to three cards, one per physical location: | Card | What it holds | | :----------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Local Storage** | Segment data on the Pinot servers' local disks — the copy that serves queries with the lowest latency. | | **Deep Storage** | The durable backup copy of every segment in external object storage (S3, GCS, HDFS), used for recovery and replication. | | **Tiered Storage** | Cost-optimized tiers holding colder data in remote object storage. Shown only when the table uses [tiered storage](/corecapabilities/manage-data/set-up-tiered-storage/motivation). | ### What each number means Every card shows the same set of figures: | Number | What it tells you | API field | | :---------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------- | | **Used** | Total storage physically occupied at this location, across all replicas. | `actualSizeInBytes` | | **Orphaned** | How much of *Used* is not backed by the table's current data, and may be reclaimable. It bundles three things — unknown files, stale versions, and beyond-retention deleted segments (see [What "Orphaned" includes](#what-orphaned-includes)). | `unexpectedSizeInBytes` | | **Segments** | Number of segments at this location. On Local Storage this is the table's expected segment count; on Deep and Tiered Storage it is the number of items found. | `segmentCount` / `actualItemsCount` | | **Unexpected Segments** | How many individual segments or files are unexpected — unknown items plus beyond-retention. Stale versions add to *Orphaned* size but are **not** counted here. | `unexpectedItemsCount` | | **Avg Segment Size** | Average size of the table's segments, shown per server. A large gap between this and your p90/p99 sizes points to segment-size skew. | `avgSegmentSizeInBytes` | ### Deep Storage breakdown The **Deep Storage** card expands into per-area cards so you can see which part of the deep store is using space: | Card | What it is | | :------------------- | :-------------------------------------------------------------------------------------------------------------------------------------- | | **Online Segments** | Live segments that belong to the table. | | **Deleted Segments** | Segments in the deep store's retention queue after deletion. Within retention they are expected; past retention they count as orphaned. | | **Upsert Snapshot** | Snapshot files for upsert-enabled tables. Never counted as orphaned. | | **Dedupe Snapshot** | Snapshot files for dedup-enabled tables. Never counted as orphaned. | The Deleted, Upsert, and Dedupe cards appear only when they hold data. ## Understanding the numbers ### Used, orphaned, and expected Two ideas explain every figure on the tab: * **Used** is what is physically stored right now. * **Orphaned** is the part of *Used* that is not backed by the table's current metadata. What remains is what *should* be there — the **expected** size: ``` expected = Used − Orphaned ``` Storage is *expected* when it is a live segment recorded in the table's metadata, or a deleted segment still inside its retention window. Everything else is orphaned. ### What "Orphaned" includes The **Orphaned** figure is a single number, but it bundles three different situations, each with a different cause and a different fix: | Category | What it is | | :---------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Unknown items** | Files on storage with **no** record in the table at all — also called *untracked* segments. Usually left by interrupted operations, failed uploads, or manual copies. | | **Stale versions** | Older copies of a segment that is still current, kept alongside the latest version when version cleanup lags. | | **Beyond-retention segments** | Deleted segments still in the deep store past their retention deadline. | **"Orphaned" means the combined total, not one category.** The Storage tab shows a single number labeled **Orphaned** for all three of the above added together. This trips people up because "orphan segment" is also used loosely to mean just the first category — files with no metadata. Throughout this page, **Orphaned** always refers to the combined Data Portal total; the no-metadata files are called **unknown items** (the API's `unknownItems`). A large **Orphaned** figure can therefore be entirely stale versions or beyond-retention segments, with zero unknown items. The API calls this same combined total **unexpected**; to see the split, call it at `verbose=2` (see [Using the storage usage API](#using-the-storage-usage-api)). Stale versions count toward the **Orphaned** size but **not** toward the **Unexpected Segments** count. This is why a table can show a large **Orphaned** size with zero **Unexpected Segments** — the excess is entirely stale versions. ### Why these numbers differ from the old Pinot table size If you are used to the OSS Pinot **table size** API (`GET /tables/{tableName}/size`, shown as *Reported Size* and *Estimated Size* in the Pinot console), the numbers here will look different, and usually larger. That is expected: the two answer different questions. | | Pinot `getTableSize` | Table Storage Usage (this page) | | :---------------------- | :---------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------- | | **Question** | How big is the table *as served on Pinot servers*? | How many bytes is the table *physically storing, everywhere*? | | **Where it looks** | Server local disk only — it polls each server for the on-disk size of the segments it hosts | Server local disk **and** deep store **and** every tiered/remote object store, each reported separately | | **What counts** | Only segments in the table's ideal state (currently served) | Every object found at each location, including orphaned, stale, deleted-but-within-retention, and snapshot data | | **Replicas** | `reportedSizeInBytes` sums all replicas; `reportedSizePerReplicaInBytes` is a single copy | Local Storage **Used** sums all replicas' on-disk bytes | | **Unreachable servers** | Estimates the missing size from other replicas (`estimatedSizeInBytes` ≥ `reportedSizeInBytes`) | Lists the actual objects present at each location; no estimation | **How `getTableSize` computes Reported Size.** For each served segment it polls every server that hosts a replica and reads that replica's on-disk size: ``` size(s, r) = on-disk bytes of replica r of segment s, as reported by its server Reported Size = Σ segments s Σ reporting replicas r size(s, r) Reported Size Per Replica = Σ segments s max over replicas r size(s, r) (≈ one full copy) Estimated Size = Reported Size + Σ segments s ( unreachable replicas of s × largest replica of s ) ``` In words: **Reported Size adds up every replica that actually responded** — roughly one copy's size × replication factor. Replicas on an unreachable server are left out of Reported Size and only approximated in *Estimated Size* (so `Estimated ≥ Reported`), a segment whose replicas all fail to report is flagged *missing* and contributes nothing, and nothing outside the Pinot servers is ever counted — no deep store, no tiered storage. Mapping the old numbers: * The old **Reported Size** maps most closely to **Local Storage → Used**: both sum the on-disk size of served segments across all replicas. Local Storage can be slightly higher because it also includes the tiered-storage preload and metadata caches and any orphaned local files. * The old **Reported Size Per Replica** ≈ **Local Storage → Used ÷ replication factor**. * **Deep Storage** and **Tiered Storage** have **no counterpart** in `getTableSize`. This is usually why the totals look much larger than before: the deep store keeps a durable copy of every segment, and tiered storage holds cold data that servers no longer keep fully on local disk. The old API never counted either. The two views are complementary. Use `getTableSize` for "how much data is my query tier serving," and the Storage tab for "what is my total footprint, and how much of it is reclaimable." A higher total does not mean the table grew — it means you can now see storage that was always there but previously invisible to the table size API (deep store, tiered storage, and orphaned data). ## What to do when Orphaned is high The Storage tab and the API **report** the Orphaned total but never delete it. What to do depends on which of the three [categories](#what-orphaned-includes) it is made of, so start by identifying the category rather than acting on the headline number. The **Orphaned** number bundles all three categories — unknown items, stale versions, and beyond-retention segments. The Storage tab does not break it down; call the API with `verbose=2` and inspect the `unexpected` object to see which one dominates before taking any action. ### Recommended workflow 1. **Rule out transient growth first.** During and shortly after a rebalance, segment replacement, or heavy ingestion, storage legitimately exceeds the expected size because of overlapping placement and intermediate artifacts. Re-check after the operation settles, and across a couple of the controller's periodic cycles, before treating anything as a real orphan. 2. **Identify the exact items and their category** with `verbose=2`. The remedy is driven entirely by category. 3. **Apply the matching remedy:** | Category (`unexpected.*`) | Typical cause | Remedy | | :------------------------ | :----------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `beyondRetentionSegments` | Deleted-segment cleanup is not keeping up | The controller's retention job purges these automatically. A persistent non-zero count means retention is not running or is lagging — verify the deleted-segment retention configuration and that the controller retention task is active. | | `staleVersions` | Re-indexed or reconfigured segments leaving older versions behind | Run [Deep Store Sync](/corecapabilities/manage-data/deep-store-sync) (the Alter Table Task in `reloadOnly` mode) and enable tiered-storage version cleanup so superseded versions age out. | | `unknownItems` | Interrupted ingestion or rebalance, failed uploads, `.tmp` / `.tar.compressed` leftovers, or manually copied files | Transient artifacts on server local disk usually clear on the next segment reload or server restart. For confirmed, persistent items in the deep store or remote object store, contact StarTree support to reconcile them. | Do not manually delete objects from the deep store or remote object store based on this list alone — a wrong deletion removes live data. Confirm an item is genuinely unreferenced, and for deep store or object-store orphans prefer StarTree support over manual deletion. ### Preventing recurrence Most orphaned growth traces back to interrupted work. To keep it from returning: * Avoid stopping ingestion, rebalance, or segment-refresh operations mid-run. * Right-size batch ingestion so failed uploads are rare; for External Tables, aim for larger source files. * Monitor the **Orphaned** figure so growth is caught while it is small, rather than after it accumulates. ## Glossary Segments or files physically present on storage with **no** matching entry in Pinot (ZooKeeper) metadata — also called *untracked* or *orphan* segments. In the API these are `unexpected.unknownItems`. In the Data Portal they contribute to both **Orphaned** (size) and **Unexpected Segments** (count). Common causes: interrupted ingestion or rebalance, failed deep store uploads, and manually copied files. Older copies of a segment that is still part of the table, kept alongside the current version in tiered/remote storage or in the server's preload/metadata caches. A version string encodes the segment CRC and a hash of the table config and schema, so changing the schema or config, or re-indexing a segment, produces a new version; any version on storage that does not match the current one is stale. Reported in `totalStaleVersions` and `unexpected.staleVersions`; their bytes count toward **Orphaned** but not toward **Unexpected Segments**. Deleted segments that remain in the deep store `Deleted_Segments` area past their retention deadline. The deadline comes from the `__RETENTION_UNTIL__` suffix on the file name, or, if absent, from the last-modified time plus the configured deleted-segment retention. Reported in `unexpected.beyondRetentionSegments`. A persistent non-zero count usually means the retention/cleanup job is not running. Live segments that belong to the table, stored in the deep store under `/`. These make up the bulk of *expected* deep store size. Segments moved to the deep store retention queue at `/Deleted_Segments/`. Those within retention are expected; those past retention are counted as beyond-retention (orphaned). Snapshot files written by upsert- or dedup-enabled tables, stored in the deep store. Always reported as expected (never counted as orphaned). Server-local caches used by tiered storage. The **preload index** stages remote-tier segment indexes for low-latency serving; the **metadata cache** holds cached metadata for remote-tier segments. Both are keyed by segment version, so both are checked for stale versions. Percentile and average segment sizes computed over the table's expected segments. Returned per server (`segmentSizeStats`) and aggregated across all servers (`serverAggregateSegmentSizeStats`). A large gap between p50 and p90/p99 indicates size skew. *** ## Using the storage usage API The Storage tab is powered by a controller endpoint. Most users never need to call it directly. Reach for the API when you want to script storage checks, feed a dashboard, or drill into the **specific** orphaned or stale items behind the aggregate numbers — the Data Portal shows the totals but not the individual item names. The endpoint provides a consolidated view for a table across server instances, deep store, and the remote object store, aggregating results collected from each server. ``` GET /tables/{tableName}/storageUsage?verbose={0|1|2} ``` * `tableName` — table name **with type**, for example `myTable_OFFLINE` or `myTable_REALTIME`. * `verbose` — verbosity level (integer, default `0`): | Level | Contents | Use for | | :---- | :----------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------- | | `0` | Summary only: per-location totals and segment size stats. No `breakdown`. | Dashboards, alerting, routine monitoring. | | `1` | Adds the per-sub-directory `breakdown`, with counts and sizes, but **without** the lists of individual unexpected items. | Locating which area is growing. **This is what the Data Portal Storage tab uses.** | | `2` | Full breakdown **including** the `unexpected` item lists (specific segment/version names). | Investigating and cleaning up specific orphaned or stale items. | Listing large deep store or remote object-store prefixes can be slow and may incur object-store request charges. Use `verbose=0` for routine monitoring and reserve `verbose=1`/`2` for investigation. The size and count invariants below always hold (and are enforced by integration tests): ``` unexpectedSizeInBytes = unknownItems.sizeInBytes + staleVersions.sizeInBytes + beyondRetentionSegments.sizeInBytes unexpectedItemsCount = unknownItems.count + beyondRetentionSegments.count (staleVersions are NOT counted) unexpectedSizeInBytes ≤ actualSizeInBytes ``` Note that `staleVersions` appears in the **size** total but not in the item **count** — this is exactly why a location can report non-zero Orphaned bytes with zero Unexpected Segments. Stale versions are counted separately in `totalStaleVersions`. ### Response: summary (`verbose=0`) Per-location scalar totals plus segment size statistics. `breakdown` is omitted. ```json theme={null} { "tableName": "myTable_OFFLINE", "serverInstances": { "Server_pinot-server-0": { "actualItemsCount": 1305, "unexpectedItemsCount": 38, "totalStaleVersions": 0, "actualSizeInBytes": 1119418454, "unexpectedSizeInBytes": 253932, "segmentSizeStats": { "segmentCount": 999, "p50SegmentSizeInBytes": 1100000, "p90SegmentSizeInBytes": 1200000, "p99SegmentSizeInBytes": 1300000, "avgSegmentSizeInBytes": 1150000 } } }, "serverAggregateSegmentSizeStats": { "segmentCount": 999, "p50SegmentSizeInBytes": 1100000, "p90SegmentSizeInBytes": 1200000, "p99SegmentSizeInBytes": 1300000, "avgSegmentSizeInBytes": 1150000 }, "deepStore": { "actualItemsCount": 2890, "unexpectedItemsCount": 3, "actualSizeInBytes": 21805648, "unexpectedSizeInBytes": 14193 }, "remoteS3": { "s3Tier": { "actualItemsCount": 5, "unexpectedItemsCount": 0, "totalStaleVersions": 3, "actualSizeInBytes": 1331507185, "unexpectedSizeInBytes": 1006228697 } } } ``` ### Response: full breakdown (`verbose=2`) `verbose=1` adds a `breakdown` for each area but omits the `unexpected` item lists. `verbose=2` additionally lists the individual unexpected contributors — unknown items, stale versions, and beyond-retention segments. ```json theme={null} { "tableName": "fake_table_SRT_Compaction_REALTIME", "serverInstances": { "Server_pinot-server-server-0-1": { "actualItemsCount": 1305, "unexpectedItemsCount": 38, "totalStaleVersions": 0, "actualSizeInBytes": 1119418454, "unexpectedSizeInBytes": 253932, "breakdown": { "localSegmentsDirectory": { "actualItemsCount": 1305, "unexpectedItemsCount": 38, "actualSizeInBytes": 1119418454, "unexpectedSizeInBytes": 253932, "unexpected": { "unknownItems": { "count": 38, "sizeInBytes": 253932, "items": [ "fake_table_SRT_Compaction__0__14199__20251222T2339Z.tar.compressed", "..." ] } } } } } }, "deepStore": { "actualItemsCount": 2890, "unexpectedItemsCount": 3, "actualSizeInBytes": 21805648, "unexpectedSizeInBytes": 14193, "breakdown": { "onlineSegments": { "actualSizeInBytes": 19301473, "unexpectedItemsCount": 2, "unexpectedSizeInBytes": 14193, "unexpected": { "unknownItems": { "count": 2, "sizeInBytes": 14193, "items": [ "fake_table_SRT_Compaction__1__162105__20260107T1819Z.tmp", "fake_table_SRT_Compaction__0__144799__20260107T1819Z.tmp" ] } } }, "deletedSegments": { "actualItemsCount": 1, "unexpectedItemsCount": 1, "actualSizeInBytes": 1201, "unexpectedSizeInBytes": 1201, "unexpected": { "beyondRetentionSegments": { "count": 1, "sizeInBytes": 1201, "items": [ "fake_table_SRT_Compaction__1__10702__20251222T1506Z" ] } } } } }, "remoteS3": { "s3Tier": { "actualItemsCount": 5, "unexpectedItemsCount": 0, "totalStaleVersions": 3, "actualSizeInBytes": 1331507185, "unexpectedSizeInBytes": 1006228697, "breakdown": { "segmentDirectory": { "actualItemsCount": 5, "unexpectedItemsCount": 0, "totalStaleVersions": 3, "actualSizeInBytes": 1331507185, "unexpectedSizeInBytes": 1006228697, "unexpected": { "staleVersions": { "count": 3, "sizeInBytes": 1006228697, "items": [ "3769746207_-1890732847", "3811220169_-1890732847", "3118351592_-1890732847" ] } } } } } } } ``` ### Response schema The response is hierarchical. It is defined below in three parts: 1. [Root response fields](#1-root-response-fields) 2. [The StorageStats object](#2-the-storagestats-object) (reused recursively) 3. [Breakdown definitions](#3-breakdown-definitions) (per-location keys) #### 1. Root response fields | Field | Type | Description | | :-------------------------------- | :------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `tableName` | `string` | Table name with type, for example `myTable_OFFLINE`. | | `serverInstances` | `map` | Map of server instance ID (for example `Server_pinot-server-0`) to a [StorageStats object](#2-the-storagestats-object). Storage on the actual server nodes; powers the **Local Storage** card. | | `serverAggregateSegmentSizeStats` | `object` | [Segment size stats](#segment-size-stats) computed over all expected segments across all servers. Present in every verbosity level, including `verbose=0`. | | `deepStore` | `object` | A [StorageStats object](#2-the-storagestats-object) for the deep store (PinotFS/S3); powers the **Deep Storage** card. | | `remoteS3` | `map` | Map of tier name to a [StorageStats object](#2-the-storagestats-object) for each tiered-storage object store; powers the **Tiered Storage** card. Absent when the table has no S3-backed tier. | #### 2. The `StorageStats` object This structure is used for `serverInstances`, `deepStore`, `remoteS3`, and every nested `breakdown` object. | Field | Type | Description | | :---------------------- | :-------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `actualItemsCount` | `integer` | Total count of unique items (segments/files) physically found at the location. | | `actualSizeInBytes` | `long` | Total physical size, in bytes, of all items found. Shown as **Used**. | | `unexpectedItemsCount` | `integer` | Count of items found that are **not** expected — unknown items plus beyond-retention segments. Excludes stale versions. Shown as **Unexpected Segments**. | | `unexpectedSizeInBytes` | `long` | Total size, in bytes, of all unexpected storage — unknown items **plus** stale versions **plus** beyond-retention segments. Shown as **Orphaned**. | | `totalStaleVersions` | `integer` | *(Tiered/remote and server caches only)* Number of unique segment versions present that differ from the version expected from table config and segment CRC. | | `segmentSizeStats` | `object` | *(Server instances only)* [Segment size stats](#segment-size-stats) for the expected segments on that server. | | `breakdown` | `map` | Per-sub-directory breakdown. See [Breakdown definitions](#3-breakdown-definitions). Present only when `verbose ≥ 1`. | | `unexpected` | `object` | Inside a breakdown entry: itemized unexpected contributors. Contains `unknownItems`, `staleVersions`, and/or `beyondRetentionSegments`. Item lists are present only when `verbose=2`. | ##### Segment size stats Returned as `serverAggregateSegmentSizeStats` at the root and as `segmentSizeStats` per server instance. Computed over the table's expected segments. | Field | Type | Description | | :---------------------- | :-------- | :--------------------------------------------------- | | `segmentCount` | `integer` | Number of segments included in the statistics. | | `p50SegmentSizeInBytes` | `long` | 50th percentile (median) segment size. | | `p90SegmentSizeInBytes` | `long` | 90th percentile segment size. | | `p99SegmentSizeInBytes` | `long` | 99th percentile segment size. | | `avgSegmentSizeInBytes` | `long` | Average segment size. Shown as **Avg Segment Size**. | ##### The `unexpected` object | Key | Description | | :------------------------ | :--------------------------------------------------------------------------------- | | `unknownItems` | Items on storage with no entry in Pinot metadata (untracked segments, temp files). | | `staleVersions` | Older versions of known segments in tiered/remote storage or local caches. | | `beyondRetentionSegments` | Deleted segments past their retention deadline in the deep store. | Each entry is an object with `count` (integer), `sizeInBytes` (long), and `items` (array of segment/version names). The `items` array is populated only at `verbose=2`. #### 3. Breakdown definitions The keys inside a `breakdown` object depend on the location. **Server instance** (`serverInstances`): | Key | Description | | :---------------------------- | :------------------------------------------------------------------------------------------- | | `localSegmentsDirectory` | Segment index directories for locally stored segments, on the server's local data directory. | | `remoteSegmentsPreloadIndex` | Local preload-index cache for a tiered table's remote segments, when preload is configured. | | `remoteSegmentsMetadataCache` | Local metadata cache for a tiered table's remote segments, when configured. | **Deep store** (`deepStore`): | Key | Description | | :---------------- | :------------------------------------------------------------------------------- | | `onlineSegments` | Live segments under `/`. | | `deletedSegments` | Segments in the retention queue at `/Deleted_Segments/`. | | `upsertSnapshots` | Snapshot files for upsert-enabled tables. | | `dedupSnapshots` | Snapshot files for dedup-enabled tables. | **Remote / tiered store** (`remoteS3`): | Key | Description | | :----------------- | :------------------------------------------------------------------------------------- | | `segmentDirectory` | Objects under the configured prefix for the tier, `//`. | ## Replication considerations Replication increases Local Storage usage, because segment directories exist per replica. Deep store and remote object-store sizes are reported independently of replica placement. ## Limitations During cluster transitions such as rebalances or segment replacement, **Used** can temporarily exceed the expected size because of overlapping placement and intermediate artifacts. Re-check after the operation completes before treating the excess as reclaimable.