Skip to main content
CosmiQ is StarTree’s AI analytics agent for Apache Pinot. Ask it a question in plain English — “What were our top 5 customers by AWS spend last month?” — and CosmiQ writes the Pinot SQL, runs it against your real-time data, and answers in plain language, showing the generated SQL alongside the result. Under the hood, CosmiQ is built on Anthropic’s Claude (via AWS Bedrock) using the Claude Agent SDK, so it reasons step by step — inspecting your table schemas, checking business rules, writing and executing SQL, and iterating if something doesn’t look right — the same way a skilled data analyst works through a request. There is no external LLM dependency: reasoning stays inside your AWS boundary.
Key terms: A Space is a curated semantic layer that scopes CosmiQ to specific tables and encodes your business rules, relationships, and query policy. A Verified Query is a gold-standard question → SQL pair that CosmiQ prefers to reuse or adapt rather than generating SQL from scratch.

​
How it works

​
Architecture

1

Spaces — your semantic layer

Every question is answered within a Space, which scopes CosmiQ to specific Pinot tables and provides context that no database schema can express on its own:
  • Table & column descriptions — human-readable meaning, synonyms, sample values, time formats
  • Business rules — canonical metric definitions, exclusions, domain terminology
  • Relationships — explicit join paths between tables, so the agent does not have to guess
  • Query policy — read-only mode, row limits, query timeouts, disallowed operations, and columns flagged do_not_use (stale, sensitive, or otherwise off-limits) that the agent will never select
  • Verified queries — a growing library of gold-standard question → SQL pairs that CosmiQ prefers to reuse or adapt rather than generating SQL from scratch
At query time, the Space is rendered into a text block and injected directly into the agent’s system prompt — so your business rules are applied before it writes a single line of SQL, not checked afterward. Each Space is its own guardrailed, self-contained context, enabling safe multi-tenant use across teams on the same Pinot cluster.
A Space for cost analytics can say: the canonical cost field is line_item_unblended_cost, the org_tags.Cost column is stale so it’s flagged do_not_use, the fact table joins to org_tags on account ID, and here are three verified queries for the questions people actually ask. With that context loaded, the agent cannot pick the wrong cost column or invent a join — it follows the rules.
2

Bootstrapping a Space (automatic onboarding)

Instead of hand-writing a Space from scratch, point CosmiQ at your Pinot tables and let it bootstrap the Space automatically:
  1. CosmiQ inspects your table schema, config, and a sample of live data (row counts, distinct value counts, min/max ranges, sample values for low-cardinality columns).
  2. An LLM pass generates structured column descriptions, synonyms, and time-format notes.
  3. For multi-table Spaces, it infers cross-table relationships and join hints.
  4. It drafts business rules in plain markdown, tagged by confidence level, for a human to review and edit.
  5. It seeds a handful of starter verified queries so the agent has good examples from day one.
This turns what would be hours of manual schema documentation into a first draft you review and refine.
3

Answering a question

When you ask a question inside a Space, CosmiQ:
  1. Loads and injects the Space’s context (tables, rules, relationships, policy) into the agent’s instructions. The rendered context is cached (5 minutes by default) so repeat questions against the same Space are cheap to serve.
  2. Retrieves any verified queries closely related to your question, matched in layers: a fast lexical pass (token overlap) catches near-exact matches, and a semantic pass (embedding similarity) catches paraphrases the lexical layer misses. When a match is close enough, CosmiQ can optionally auto-execute the verified SQL up front so the agent sees the real data shape before it even starts reasoning.
  3. Lets Claude reason over the request: it can inspect schemas, consult a built-in Pinot query-optimization guide for complex queries (joins, window functions, large aggregations), write SQL, execute it, and inspect the results. The agent is capped at 15 turns per question as a safety ceiling — most answers take 2–4.
  4. If a query fails, the agent reads the actual error and either fixes and retries (for a SQL mistake) or reports exactly what happened — it is designed to never guess about why something did not work.
  5. Streams back a natural-language answer along with the generated SQL, a summary of the steps it took, and the query cost and duration.
Chat answers default to Claude Sonnet for fast time-to-first-byte; Space bootstrapping defaults to Claude Opus, since generating structured Space JSON rewards more careful reasoning.
4

Learning over time

Every answer can be rated 👍/👎. Highly-rated answers with working SQL are surfaced as candidates to promote into the Space’s verified-query library — so the more a Space is used, the better and more consistent its answers become.

​
Under the hood

CosmiQ ships as a single container image that runs in three modes, selected by a SERVICE environment variable. This keeps the chat experience, the MCP tool, and the Spaces management API as one deployable unit while letting each process scale independently. For one question, the request flows: browser → Chainlit UI → MCP client → CosmiQ’s MCP server → Claude agent (Claude Agent SDK, on AWS Bedrock) → Pinot MCP tools → Pinot, with the answer streamed back the same path.
Claude, via the Claude Agent SDK, running on AWS Bedrock — no external LLM dependency, so data and prompts stay inside your AWS boundary. An optional LiteLLM gateway can sit in front of Bedrock for centralized spend metering without any changes to the agent code.
Pinot through MCP tools — schema introspection, query execution, and a built-in query-optimization guide. CosmiQ talks to a remote Pinot MCP server (configured via MCP_SERVER_URL); it never talks to Pinot directly.
A file backend for local development, MariaDB in production, for Spaces, verified queries, and feedback. Rendered Space contexts are cached for 5 minutes by default, so repeat questions on the same Space are cheap.
Auth0 sign-in with a signed session cookie (8-hour session lifetime by default), plus bearer JWT for API and MCP clients. The chat UI and the REST API share the same session.
Python 3.11, dependencies managed with uv, packaged as one container image with three process modes as described above.
For request/response details on the REST endpoints referenced above, see the API Reference.

​
Key capabilities

Natural language to SQL

Ask a question in plain English and get back an executed answer backed by real Pinot SQL — no SQL knowledge required.

Multi-table reasoning

Handles joins, aggregations, window functions, and period-over-period comparisons, with built-in Pinot query-optimization guidance for complex cases.

Domain-aware answers

Business rules and metric definitions from your Space are respected — not just raw column names from the schema.

Verified-query reuse

Trusted, previously-approved SQL patterns are preferred over ad hoc generation when a question closely matches a verified example.

Automatic Space bootstrapping

Turns raw Pinot tables into a documented semantic layer with minimal manual effort — hours of curation compressed into a reviewed first draft.

Transparent reasoning

Shows the generated SQL, the plan the agent followed, and which tools it called — not just a final answer.

Honest failure handling

Distinguishes “this data is not in scope,” “the SQL was wrong and I fixed it,” and “the cluster has a problem” — never bluffs about what went wrong.

Feedback-driven improvement

Thumbs up/down on answers feeds a review queue and a path to grow the verified-query library automatically.

​
Benefits

Faster time-to-insight

Non-technical users get answers without writing SQL or waiting on a data team.

Consistency at scale

Verified queries and business rules mean the same question gets the same trusted answer, from anyone, every time.

Lower onboarding cost

Bootstrapping turns a new Pinot table into a usable, documented Space in minutes rather than days.

Safe multi-tenancy

Spaces let different teams or customers query the same Pinot cluster with separate scopes, rules, and guardrails.

Built for real-time analytics

Purpose-built for Apache Pinot — understands Pinot’s SQL dialect, join strategies, group-by trimming, and segment-aware optimizations rather than generic ANSI SQL.

Auditable

Every question, generated SQL statement, and answer is logged per conversation for review and accuracy tracking.

Enterprise-ready deployment

Runs on AWS Bedrock (data stays within your AWS boundary), containerized, with Kubernetes/Helm deployment options.

Read-only by design

Space policy scopes all queries to read-only analytics, with configurable row limits and timeouts.

​
Next steps

How to Use CosmiQ

Chat UI, MCP tool, and REST API walkthroughs, plus a step-by-step guide to setting up your first Space.

API Reference

Full reference for the REST endpoints — querying, Space management, and feedback — plus the system defaults that govern them.

Security and Compliance

Read-only guarantees, data handling, authentication, and how CosmiQ inherits StarTree Cloud’s platform compliance posture.