Skip to content

Taxonomy

A taxonomy turns a pile of open-text feedback into a browsable tree of topics and subtopics. Hub embeds each record that carries text, hands the vectors to a clustering service, and stores the result as a tree of nodes you can read, edit, and count against — so you can see what people are talking about, and how much, without tagging anything by hand.

Taxonomies are generated on demand as runs. A run reads a defined slice of feedback, produces a tree, and — once it succeeds — becomes the active tree for that slice until a newer run replaces it.

  1. Embeddings first. Taxonomy uses its own embeddings, made with the same provider as semantic search but from the record’s translation where it has one, and stored under their own model key (taxonomy:<EMBEDDING_MODEL>:translated-v1 unless TAXONOMY_EMBEDDING_MODEL sets another). Search vectors do not count. Every record that carries text and has a stored taxonomy embedding participates, whatever its field type. That includes open-text answers and the option labels that choice and matrix questions store on categorical records. So a deployment must have embeddings configured, and the records must already have taxonomy embeddings. When the worker has EMBEDDING_RECONCILE_ENABLED=true, an embedding provider and TAXONOMY_SERVICE_URL, a periodic sweep re-queues any of these records whose embedding is missing, or was computed from text (or a translation) the record no longer has. That includes ones whose attempts ran out on a transient provider error, once EMBEDDING_RECONCILE_RETRY_AFTER_SECONDS has passed. Records the provider rejected outright are not retried until the record is written again. See Embedding repair.
  2. Clustering. When you start a run, Hub sends the in-scope records and their embeddings to the standalone taxonomy service, which groups semantically similar feedback into clusters.
  3. Tree. Hub stores the returned clusters as a tree of nodes — a root, intermediate topics, and leaf subtopics — and marks the run active for its scope.

The compute happens off the ingestion path in a separate service, so creating feedback is never slowed down by taxonomy.

A run is created for a scope with POST /v1/taxonomy/runs and moves through a fixed set of states:

Status Meaning
pending Hub accepted the request and persisted the run.
running Hub handed the run to the taxonomy service.
succeeded The service returned a tree and Hub activated the run.
failed The run ended with an error (with a machine-readable failure code).
canceled Reserved for future user/operator cancellation.

Allowed transitions are pending → running | failed | canceled and running → succeeded | failed | canceled; terminal states are never overwritten. Not having enough input, or the service being unavailable, surface as a failed run with a failure code rather than a separate status.

A run needs a minimum number of embedded records before it will start (see Enabling); below that threshold the request is rejected. Poll GET /v1/taxonomy/runs/{run_id} for status, and list history with GET /v1/taxonomy/runs.

A run covers one scope — the slice of feedback it reads:

  • field — the feedback for a single field_id (for example, one survey question), optionally narrowed by source_type / source_id.
  • directory — all open-text feedback for the tenant, across sources and fields.

Each scope has at most one active taxonomy at a time. Fetch it with GET /v1/taxonomy/runs/active/tree; a fresh successful run for the same scope replaces the previous active tree. Use GET /v1/taxonomy/fields to discover which fields have enough embedded feedback to be worth a run.

A taxonomy tree has three kinds of node:

  • root — one per run; the whole scope.
  • topic (branch) — an intermediate grouping.
  • subtopic (leaf) — a terminal cluster that feedback records actually belong to.

Read the full tree with GET /v1/taxonomy/runs/{run_id}/tree, or drill into a node’s feedback with GET /v1/taxonomy/nodes/{node_id}/records.

Trees are editable. You can rename a node (PATCH /v1/taxonomy/nodes/{node_id}) or soft-remove one (DELETE /v1/taxonomy/nodes/{node_id}). Removed nodes are hidden from the tree and every count — the endpoints only ever return visible nodes — but the underlying feedback is untouched.

To show totals in a taxonomy view, GET /v1/taxonomy/runs/{run_id}/record-counts returns the feedback-record count for every visible node in one call:

Terminal window
curl "http://localhost:8080/v1/taxonomy/runs/RUN_ID/record-counts?tenant_id=org-123" \
-H "Authorization: Bearer your_secure_api_key_here"
{
"counts": [
{ "node_id": "019f6b2a-1e40-7c93-8a11-3d5e7f901a2b", "record_count": 128 },
{ "node_id": "019f6b2a-1e40-7c93-8a11-3d5e7f901b7c", "record_count": 47 },
{ "node_id": "019f6b2a-1e40-7c93-8a11-3d5e7f901c3d", "record_count": 19 }
]
}

Each record_count is a subtree total of distinct records: a topic reports the count across all of its subtopics, and the root reports the run total. Join the node_ids with the tree to render a count next to each topic and subtopic.

Taxonomy is available where the deployment has both embeddings and the taxonomy service configured:

  • Embeddings must be on — set EMBEDDING_PROVIDER + EMBEDDING_MODEL (see Environment Variables); records are embedded before they can be clustered.
  • The clustering service must be reachable — set TAXONOMY_SERVICE_URL, TAXONOMY_SERVICE_TOKEN, and HUB_INTERNAL_API_TOKEN (see Environment Variables).
  • TAXONOMY_MIN_EMBEDDED_RECORDS (default 20) sets how many embedded records a scope needs before a run may start.

With the service unconfigured, or embeddings off, the taxonomy endpoints report that the feature is unavailable.