> ## Documentation Index
> Fetch the complete documentation index at: https://docs.startree.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshooting queries

> Diagnose query timeouts, execution errors, multi-stage engine failures, and sudden latency or error-rate increases.

This guide covers queries that fail, queries that time out, and queries that used to be fast and no longer are.

<Info>
  If you have [Query Logger](/corecapabilities/query_data/advanced_operations/query-logger) enabled, start there — `system_query_log` records every query the cluster served, with timings, scan counts and error text. Most of the diagnosis below is a single SQL query against it.
</Info>

## Start by classifying the failure

The error text tells you which section to read. Match on the most specific string in the message, not the outermost one.

| What you see                                               | Most likely class               | Go to                                                                         |
| ---------------------------------------------------------- | ------------------------------- | ----------------------------------------------------------------------------- |
| `BrokerTimeoutError`, `Timed out while planning query`     | Planning or fan-out cost        | [Timeouts](#timeouts)                                                         |
| Timeout with no planning mention                           | Execution cost or contention    | [Timeouts](#timeouts)                                                         |
| `QueryExecutionError` with a nested cause                  | Data, function, or type problem | [Execution errors](#execution-errors)                                         |
| `Unsupported function`, or an error naming an operator     | Engine capability               | [Multi-stage engine](#multi-stage-engine-issues)                              |
| HTTP 5xx, or a sudden rise in failures across many queries | Cluster-level, not query-level  | [Error-rate spikes](#error-rate-spikes)                                       |
| Results returned, but wrong or inconsistent                | Availability or data problem    | [Inconsistent results](#inconsistent-or-partial-results)                      |
| Quota or rate-limit rejection                              | Quota configuration             | [Query Quotas](/corecapabilities/query_data/advanced_operations/query-quotas) |

## Timeouts

A timeout means the query did not finish inside its budget. The useful question is *where* the time went.

<Steps>
  <Step title="Separate planning from execution">
    An error mentioning **planning** means the query did not get as far as reading data. That points at query shape — very large `IN` lists, many joins, a wide fan-out across segments, or a complex multi-stage plan — rather than at data volume.

    A timeout without a planning mention means execution was too slow, and the scan counts below will tell you why.
  </Step>

  <Step title="Look at what the query scanned">
    In `system_query_log`, the scan fields distinguish an expensive query from a slow cluster:

    ```sql theme={null}
    SELECT logTimestampMs, tableName, timeUsedMs,
           numDocsScanned, numEntriesScannedInFilter, numEntriesScannedPostFilter,
           numSegmentsQueried, numSegmentsProcessed, numSegmentsMatched,
           query
    FROM system_query_log
    WHERE logTimestampMs > now() - 3600000
      AND timeUsedMs > 1000
    ORDER BY timeUsedMs DESC
    LIMIT 20
    ```

    Read the result like this:

    * **High `numEntriesScannedInFilter`** — the filter is not being served by an index. This is the most common cause of a slow query, and the most fixable.
    * **High `numEntriesScannedPostFilter`** — the filter is fine, but the projection is reading a lot of columns or rows. Select fewer columns, or add a `LIMIT`.
    * **`numSegmentsProcessed` close to `numSegmentsQueried`** — segment pruning is not working. Check that the query filters on the time column, and on the partition column if the table is partitioned.
    * **All counts low but `timeUsedMs` high** — the query is not the problem. The cluster is. Go to [Resource pressure](/corecapabilities/cluster-operations/troubleshooting).
  </Step>

  <Step title="Check whether an index is actually being used">
    A high `numEntriesScannedInFilter` on a column you have indexed means the index is not being applied. Common reasons:

    * The index was added but segments have not been reloaded since. Check the `TABLE_SEGMENTS_RELOAD_CHECK` on the Health Dashboard, which will tell you how many segments are pending reload.
    * The filter predicate does not match a form the index can serve — for example a function applied to the indexed column, or a type mismatch that forces a cast.
    * The index exists on some segments and not others, because it was added partway through the table's life.

    [Query Analyzer](/corecapabilities/query_data/query-analyzer) will tell you which of these applies and what to change — see [Reading Query Analyzer results](/corecapabilities/ai/query-analyzer/reading-results) for how to interpret its output.

    Before triggering a reload, [Reload dry run](/corecapabilities/manage-data/reload-dry-run) shows you what a reload would actually change, and its [Troubleshooting](/corecapabilities/manage-data/reload-dry-run#troubleshooting) section covers dry runs that fail or return nothing.
  </Step>

  <Step title="Confirm it is this query and not the load">
    Run the same query when the cluster is quiet. If it is fast then and slow under load, you have a contention problem rather than a query problem — see [Resource pressure](/corecapabilities/cluster-operations/troubleshooting). Scheduler wait time rising while execution time stays flat is the signature.
  </Step>
</Steps>

## Execution errors

An execution error means the query started and then failed. The cause is almost always in the innermost exception, not the outermost message.

<Warning>
  Read the full error, including nested causes. A generic outer message with a specific inner cause is common, and the inner cause usually names the column, value, or type at fault.
</Warning>

Work through these in order:

1. **Does it fail for all inputs, or only some?** Narrow the time filter until the query succeeds. The boundary tells you which data is involved, and that is usually the whole answer — a null in an unexpected column, a value that overflows its type, a malformed JSON document.

2. **Did the schema or a type change?** A column whose type changed will fail on segments written under the old type until those segments are rebuilt. Check your schema history against the age of the segments the query touches.

3. **Is a specific function involved?** Remove functions from the projection one at a time. Errors during result serialization — particularly with sketch and aggregate types — usually point at a function that cannot handle the cardinality or the value range it is being given.

4. **Is it one replica?** If the same query fails intermittently with the same error, one replica may hold a bad segment. The `TABLE_SEGMENT_AVAILABILITY_CHECK` on the Health Dashboard will show unavailable segments. A segment reset or reload is the usual remedy, and both are available from the Health Dashboard where the check supports them.

## Multi-stage engine issues

The multi-stage engine (MSE) supports joins and more complex plans than the single-stage engine, and has a different capability surface. See [Multi-stage query engine](/corecapabilities/query_data/query_languages/msqe) for what it supports.

When an MSE query fails:

<Steps>
  <Step title="Try the same query on the single-stage engine">
    If it works there, you have an MSE-specific issue and you have your workaround while it is investigated. If it fails on both, the problem is not MSE-specific — treat it as an [execution error](#execution-errors).
  </Step>

  <Step title="Check for unsupported constructs">
    An `Unsupported function` error naming a specific function or operator means the plan reached a construct the engine cannot handle in that position. This is often reachable by rewriting — extracting a subquery, materializing an intermediate result, or replacing the construct with an equivalent.
  </Step>

  <Step title="Look at per-stage timings for slow MSE queries">
    `system_query_log` records `stageStats` as JSON, giving per-operator statistics. One stage dominating total time tells you which operator to attack — usually a join whose build side is too large, or a shuffle moving more data than it needs to.
  </Step>

  <Step title="Check partition-aware pruning on partitioned tables">
    If your table is partitioned and the query filters on the partition column, the broker should be able to prune. High `numSegmentsProcessed` despite a partition filter means pruning is not being applied — worth raising, with the query and the table config attached.
  </Step>
</Steps>

## Error-rate spikes

When many queries start failing at once, the cause is rarely any individual query.

<Steps>
  <Step title="Establish the shape of the spike">
    With Query Logger enabled:

    ```sql theme={null}
    SELECT tableName,
           COUNT(*) AS total,
           SUM(CASE WHEN numExceptions > 0 THEN 1 ELSE 0 END) AS failures,
           SUM(CASE WHEN partialResult THEN 1 ELSE 0 END) AS partial
    FROM system_query_log
    WHERE logTimestampMs > now() - 3600000
    GROUP BY tableName
    ORDER BY failures DESC
    ```

    Failures concentrated on one table point at that table. Failures spread evenly across tables point at the cluster.
  </Step>

  <Step title="Check segment availability">
    Server restarts, failed reloads and stuck rebalances all remove segments from service temporarily. `TABLE_SEGMENT_AVAILABILITY_CHECK` and `IDEAL_STATE_EV_MISMATCH_CHECK` on the Health Dashboard both surface this. A mismatch between ideal state and external view means the cluster has not converged — that is usually transient, and worth escalating if it persists.
  </Step>

  <Step title="Check whether you are hitting a quota">
    Rejections due to query quotas look like failures to the application. See [Query Quotas](/corecapabilities/query_data/advanced_operations/query-quotas) for how quotas are configured and what a rejection looks like.
  </Step>

  <Step title="Check resource pressure">
    Servers under memory or CPU pressure fail queries rather than serving them slowly, and garbage-collection pauses show up as timeouts clustered in time. See [Resource pressure](/corecapabilities/cluster-operations/troubleshooting).
  </Step>
</Steps>

## Inconsistent or partial results

A query that returns results but the wrong ones is a different problem from a query that fails.

* **Check `partialResult` and the limit flags.** `system_query_log` records `partialResult`, plus `numGroupsLimitReached`, `groupsTrimmed`, `maxRowsInJoinReached`, `maxRowsInWindowReached` and `maxRowsInDistinctReached`. Any of these being true means the query hit a guardrail and the result was trimmed — the numbers are not wrong so much as incomplete.

* **Check segment availability.** A query that silently reads fewer segments than it should returns a smaller answer. Compare `numSegmentsQueried` between a good and a bad execution.

* **On hybrid tables, check the time boundary.** Real-time and offline halves are separated by a time boundary, and rows can appear missing or doubled if the boundary and the data do not line up. See [Hybrid tables](/corecapabilities/manage-data/hybrid-tables).

* **On upsert tables, check replica consistency.** Upsert tables can serve different answers from different replicas if their state diverges. See the [Upsert operations guide](/corecapabilities/manage-data/upsert-operations-guide).

* **Different results from different tools?** Confirm both are querying the same table type — a query against the base table name behaves differently from one against an explicit `_OFFLINE` or `_REALTIME` suffix on a hybrid table.

## Feature-specific guidance

If the problem is specific to a feature — geospatial queries, star-tree indexes, materialized views, the Elasticsearch gateway, workload isolation — see [Troubleshooting by feature](/corecapabilities/observability/troubleshooting-by-feature#queries-indexes-and-performance), which indexes the troubleshooting sections on those pages.

**Querying an external table?** [External table troubleshooting](/corecapabilities/external-table/troubleshooting#queries) covers timeouts and `servers not responded`, why a first query on a column is slow while later ones are fast, and how to tune a large scan.

## Escalating

If you have worked through this guide, include in your ticket: the full query, the complete nested error, the `requestId`, the engine used, whether it reproduces, and the relevant `system_query_log` row. See [what to collect](/corecapabilities/observability/troubleshooting#before-you-open-a-support-ticket).