Manage Knowledge Documents

What this page is for

The Documents page is the administration workspace for content used by AI Agents. It is separate from the regular Documents library: the library stores working files, while this page prepares selected content for Knowledge search.

Regular users normally do not open this page. They use AI Agents and receive only the content made available through the selected assistant and its access rules.

Understand the page layout

The page has two main areas:

• the left panel contains the content input, document search, document list, preset control, Manage Policies, Refresh list, and More;
• the main area opens Pipeline Studio for the selected document.

Select a document from the left panel before changing its processing, policies, embedding, or tests.

Add content

The current page provides two visible input methods: Documents and Markdown.

Import from Documents

1. Select Documents.
2. Click Browse Documents.
3. Search the compatible files in the Import from Documents dialog.
4. Select one file, then click Import.

The import list supports PDF, DOCX, TXT, MD, and HTML content. A file marked In Knowledge already has a matching Knowledge document; selecting it loads the existing document instead of creating another copy.

Add Markdown

1. Select Markdown.
2. Enter Source filename.
3. Paste the Markdown, or click Load .md to load an .md or .txt file.
4. Click Run Pipeline.

Both the filename and Markdown content are required. Run Pipeline creates the document and starts processing it with the current settings.

Find and select a document

Use Search documents to filter the list by filename. Each document card shows its filename, type or source information, current processing status, and the processing-preset control.

Possible list outcomes include:

• No documents yet — no Knowledge documents have been added;
• No match found — the current search text matches no document; and
• a loading indicator while the list is refreshing.

Click Refresh list after an external upload or Auto-Sync job if the new state is not yet visible.

Use processing presets

Open the processing-preset control on a document card to standardize the settings used by Pipeline Studio. From this menu you can:

• select a saved preset and apply it to the document;
• save the current settings as a new preset;
• update a preset by saving with the same name; or
• delete a preset that is no longer needed.

Deleting a preset does not delete a document. Review Auto-Sync rules before deleting a preset because a rule may depend on it.

Understand Pipeline Studio

Pipeline Studio is organized into four top-level steps:

Stage indicators show whether work has not started, is running, completed, or failed. A later action can remain locked until its required earlier result is ready.

Step 1: Run the pipeline stages

1. Markdown Parsing

Choose how the source is converted to structured Markdown:

• Direct Extract reads digital PDFs and text files quickly. It does not handle scanned or image-based documents.
• Full Pipeline supports OCR, scanned files, and complex layouts.

When Full Pipeline is selected, Parse Options can include:

• Parsing Tier;
• Target Pages;
• Max Pages;
• Batch Size;
• Save Images;
• document Languages; and
• an optional Custom Prompt.

Click Run Markdown Parse. After completion, use View / Edit Markdown to preview, edit, save, copy, download, or view the Markdown full screen. If image extraction was enabled, use View Extracted Images.

2. Structural Nodes & Chunking

This stage turns the Markdown into semantic elements and retrieval-ready chunks. It normally follows parsing with sensible defaults.

Expand Chunking Options only when you need to change:

• Max Chunk Tokens — the maximum size of each chunk; or
• Chunk Overlap — the amount of shared context between neighboring chunks.

After completion, you can use View / Edit Chunk JSON and Browse Nodes to inspect the output.

3. Table Summaries

This stage produces natural-language summaries that make tables searchable. It is skipped automatically when the document contains no tables.

Choose the Summary Model, review or adjust the Summary Prompt, then click Run Table Summaries. Use View Table Summaries to inspect the result.

4. LLM Enrichment

This stage adds context summaries, named entities, sensitive-information detection, and optional custom extraction results to each chunk.

The visible settings are:

• Enrichment Model;
• User Prompt Add-on;
• Base Enrichment Prompt; and
• Extraction Rules (JSON).

Invalid extraction-rules JSON prevents the stage from running. After completion, click View Enrichment Results to inspect chunks, entities, sensitive-information categories, and custom results.

Manage stage versions

Each completed pipeline stage can keep multiple versions. The version picker lets you:

• select which result is passed to the next stage; and
• delete an obsolete stage version.

If you edit an earlier result or select a different version, rerun the dependent later stages so the final index reflects the intended content.

Step 2: Apply access policies

The Access Policy step first shows the current policy summary:

• document policy;
• page overrides; and
• node overrides.

Use Assign Policy and select one of these tabs:

Select the policy, complete the tab-specific fields, and apply it. You can also remove existing document, page, or node overrides from the summary.

Policy changes affect future embeddings. If the document is already embedded, click Sync Access under Sync Access to Search Index so current search results use the latest policy metadata.

Use Manage Policies at the top of the document list to create or edit the available policy definitions. See Manage Knowledge Access Policies.

Step 3: Embed the document

Vector Embedding converts processed chunks into a searchable representation.

1. Select an Embedding Model.
2. Select supported Embedding Dimensions for that model.
3. Review the model-and-dimensions summary.
4. Click Embed Document.

The page warns when a dimension change would rebuild the shared search index. This can remove other embedded documents from the index until they are embedded again. Do not change dimensions without coordinating the full re-embedding work.

Embedding is not a substitute for successful parsing and chunking. Confirm those stages first.

Step 4: Test the result

Similarity Test

Enter:

• a Relevant Query that should match the selected document;
• an Irrelevant Query that should not match it; and
• the required Threshold.

Click Run Similarity Test. The result shows both scores and a PASS or FAIL verdict. A failure can mean the relevant query does not describe the document well, the threshold is too strict, or the document needs to be processed and embedded again.

Nearest Neighbor Test

Enter a query and click Run. The page returns the three closest chunks from the selected document, including their score, source page, section path when available, and content preview.

Use this test to confirm that the expected passage is being retrieved—not only that a numeric score is high.

Open related administration pages

Open More in the Documents header to access:

• Auto-Sync Rules — create and manage automatic processing rules;
• Sync History — open Auto-Sync directly on Job History; and
• Sync Stats — open Auto-Sync directly on Stats.

See Manage Knowledge Auto-Sync.

Delete a Knowledge document

Select the document, then click Delete document in its header. Confirm only when the document should be removed from Knowledge.

This action is separate from deleting the source file in the regular Documents library. If an active Auto-Sync rule still matches the source, the content may be processed again later.

Troubleshooting