Pipeline Studio Reference
Pipeline Studio appears after selecting a document in /knowledge-documents. Its top-level steps are:
| Step | Purpose |
|---|---|
| Pipeline Stages | Parse, chunk, summarise tables, and enrich content. |
| Access Policy | Apply document, page, node, or detected-entity restrictions. |
| Embedding | Convert processed chunks into the search index. |
| Testing | Check similarity and inspect nearest matching chunks. |
Pipeline Stages
- Markdown Parsing: choose Direct Extract for digital text or Full Pipeline for OCR, scans, and complex layouts. Full Pipeline exposes parsing tier, pages, batch size, image extraction, languages, and custom prompt.
- Structural Nodes & Chunking: creates semantic nodes and chunks. Optional settings are Max Chunk Tokens and Chunk Overlap.
- Table Summaries: uses a configured summary model and prompt; it is skipped when no tables exist.
- LLM Enrichment: adds contextual metadata, entities, PII categories, and optional extraction-rule results.
Each stage shows its state and can expose its output for review. Selecting or editing an earlier output does not automatically rebuild later results; rerun dependent stages and embedding when needed.
Unlocking
Access Policy and Embedding require parsed and chunked content. Testing requires an embedded document. Optional stages can improve retrieval but do not replace parsing and chunking.
Still stuck? See Get Help with Support or reach your workspace administrator.