> For the complete documentation index, see [llms.txt](https://docs.xenioo.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.xenioo.com/ai/large-language-models/resources.md).
# Resources
Resources are the external knowledge you attach to your AI model. Every document, web page or Confluence space you add here is fetched, split into smaller pieces and indexed, so your model can ground its answers on your own content instead of relying only on what the language model already knows.
You can add as many resources as you need, mixing uploaded files, public web pages and Confluence pages in the same model. Xenioo takes care of downloading, converting and indexing everything for you, and keeps web-based resources up to date automatically.
### Adding a Resource
Click **+ Add Resource** to open the resource dialog. Everything is configured from a single window, organized into tabs: **Main** and **Chunking Strategy** are always available, while **Advanced** and **Exclusion** appear only for Public URL and Confluence resources.
#### Main
**Type**
The **Type** field decides where the content of the resource comes from. The rest of the Main tab changes depending on the type you pick.
* **File** - upload a document from your computer.
* **Public URL** - index a publicly reachable web page, optionally crawling the pages below it.
* **Confluence Page** - index a page (and optionally its sub-pages) from an Atlassian Confluence space.
{% hint style="info" %}
The **Type** cannot be changed once a resource has been created. If you need a different type, delete the resource and add a new one.
{% endhint %}
**File**
Select a document to upload. Xenioo supports text, PDF, Word, Excel, CSV, Markdown, HTML and image files up to **64 MB**.
Supported extensions: `html`, `pdf`, `docx`, `xlsx`, `csv`, `txt`, `log`, `md`, `png`, `jpg`, `jpeg`, `webp`.
{% hint style="info" %}
An uploaded image is indexed by having your model read it and describe it - see Content images.
{% endhint %}
The file is uploaded as soon as you select it.
**Public URL**
Enter the full address of the page you want to index in the **Public URL** field. A small icon next to the field lets you open the address in a new tab to double-check it.
Use **Crawl depth** to decide how much of the site to index:
| Option | What gets indexed |
| -------------------- | ------------------------------------------------------------------------------------------ |
| **Single Page** | Only the page at the address you entered. |
| **1 / 2 / 3 levels** | The page plus every page found below it, following that many path levels on the same site. |
{% hint style="warning" %}
Crawl depth grows quickly. A depth of 3 on a large site can discover hundreds or thousands of pages. Start with a shallow depth and increase it only if needed.
{% endhint %}
**Confluence Page**
Confluence pages are read through the official Confluence Cloud API, so you need to provide credentials:
* **Base Page URL** - the address of the Confluence page or space to index.
* **Username** - the email address of the Atlassian account used to read the content.
* **API Key** - an Atlassian API token for that account.
* **Index all content sub-pages** - when enabled, Xenioo also indexes every page found underneath the one you provided.
{% hint style="info" %}
The API Key is stored securely and never sent back to the browser. When you edit a Confluence resource, the key field shows a placeholder to signal that a token is already set. Leave it untouched to keep the existing key, or click into it to type a new one.
{% endhint %}
**Visibility**
**Visibility** decides who is allowed to see the content of a resource when the model answers.
* **Everyone** (default) - the resource can be used to answer any conversation.
* **An Audience** - the resource is used only when the current contact matches the selected Audience.
{% hint style="info" %}
Visibility is evaluated at reply time against the conversation's variables. If the contact does not match the audience, the resource is simply left out of the answer.
{% endhint %}
**Index content images**
Turn this on to also index the **pictures** inside the content - a diagram, a screenshot, a pasted log line. It is available for every type and defaults **on** for new Public URL and Confluence resources, **off** for files. The behaviour is explained in full under Content images*.*
#### Chunking Strategy
Before it can be indexed, a document is split into smaller **chunks**. The chunking strategy controls how that split happens.
| Strategy | When to use it |
| ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Smart** (recommended) | Xenioo inspects the document size and type and picks the best option automatically. Small documents are kept whole, larger structured files are split by structure, everything else is split recursively. |
| **No Split** | Keeps the document as a single chunk. The document must not exceed the 8k token context limit. |
| **Recursive Text Splitting** | Splits the text into overlapping chunks of a fixed size. A good general-purpose choice for plain text. |
| **Document Structure Based** | Splits along the document's own structure (headings, sheets, pages). Valid for `xlsx`, `docx`, `pdf`, `html` and `md`. |
{% hint style="info" %}
If you are not sure which one to pick, leave it on **Smart**. It resolves the best strategy for each resource at indexing time.
{% endhint %}
#### Advanced
The **Advanced** tab is available only for Public URL and Confluence resources.
**Reindex resource**
Decides whether the resource is automatically kept in sync with its source:
* **Never** - the resource is fetched and indexed once and never refreshed on its own. Choose this for content that does not change. No frequency is shown.
* **Every *****n***** days** - the resource is automatically re-fetched and re-indexed on a schedule. The interval can be set between **1 and 90 days** and defaults to **15 days**.
{% hint style="info" %}
A resource set to **Never** is still re-fetched if you open it and save it manually, or if you use the **Reindex** action from the row menu - "Never" only turns off the automatic schedule.
{% endhint %}
**Additional Headers**
Extra key/value headers sent with every fetch request. Use them when the source requires custom headers (for example an authentication or user-agent header) to be reached.
#### Exclusion
The **Exclusion** tab is available only for Public URL and Confluence resources. It lets you keep unwanted pages out of a scan **before** they are indexed. Both lists take one entry per line and match **case-insensitively**.
| List | What it skips |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Exclude paths** | Discovered pages whose **address** contains one of these fragments - for example `/tag/`, `/login`, `?print`. A matching page is skipped and not crawled into. |
| **Exclude keywords** | Discovered pages whose **content** contains one of these keywords - for example `Page not found`, `Access denied`. |
{% hint style="info" %}
Exclusions apply to the pages **found by the scan**, not to the address you entered. The root page you typed is always indexed; use the rules to trim the pages discovered below it.
{% endhint %}
### The Resources list
Every resource you add appears in the Resources list, with a column for its indexing progress and its key statistics.
| Column | Description |
| --------------------- | ---------------------------------------------------------------------------------------------------------------- |
| **Name** | The display name of the resource: the file name for uploads, the page title or URL for web and Confluence pages. |
| **Size** | The size of the stored content. |
| **Chunks** | How many chunks the content was split into. |
| **Avg. Tokens** | The average number of tokens per chunk. |
| **Chunking Strategy** | The strategy used to split the content. |
| **Created** | When the resource was added. |
| **Embedding Status** | The current indexing state (see below). |
#### Embedding status
| Status | Meaning |
| ------------- | ------------------------------------------------------------------------------ |
| **Preparing** | The content is being fetched or crawled before it can be indexed. |
| **Pending** | The content is waiting in the indexing queue. |
| **Indexing…** | The content is being embedded right now. |
| **Indexed** | The content is fully indexed and available to the model. |
| **Failed** | Something went wrong. Hover the status to read the reason. |
| **Banned** | The content has been excluded from the index on purpose (see Banning content). |
{% hint style="info" %}
While any resource is still settling (Preparing, Pending or Indexing), the list refreshes on its own every few seconds, so you can watch progress without reloading the page.
{% endhint %}
#### Website scans and discovered pages
When you index a Public URL with a crawl depth, or a Confluence page with sub-pages, the page you entered becomes a **scan root** and every page discovered from it is added as a child resource.
By default the list shows only the roots. Turn off **Show only root resources** to reveal the discovered pages, then use the arrow next to a scan root to expand or collapse its pages.
The same grouping is used for images: when Content images is enabled, the pictures found in a page or document appear as child resources under it. A document that carries images is shown with a **"with images"** badge you can expand.
### Managing a Resource
#### Editing a resource
**Double-click** a row (or choose **Edit** from the row menu) to open a resource. Roots and standalone resources can be freely edited except for their type; saving hands them back to the pipeline and re-indexes them.
For a **discovered sub-page**, only the **Chunking Strategy** can be changed. Everything else is owned by the scan root that created it.
{% hint style="info" %}
A resource that is still being crawled or indexed opens read-only. Wait until it reaches **Indexed** or **Failed** before editing it.
{% endhint %}
#### Viewing what was indexed
Open a resource and click **View Index** to see exactly what was stored for indexing. Web pages and Confluence pages are shown as the converted content, files are shown as-is, and an **image** is shown together with a **Detected text** panel holding the description the model read from it (see Content images).
#### Banning content
Sometimes a crawl discovers a page that should not be part of your knowledge base an unrelated section, a login page, an "intrusion" that slipped into the scan. Open the resource and click **Ban** to exclude it.
A banned resource is removed from the index immediately and marked **Banned**. To bring it back, open it again and click **Unban**.
{% hint style="info" %}
A ban is remembered per address. Even after the next automatic reindex which drops and rebuilds every discovered page - the page stays banned until you unban it. The same applies to a per-page chunking strategy you set on a sub-page.
{% endhint %}
#### Reindexing
Web and Confluence resources are re-fetched automatically according to their Reindex resource setting, unless it is set to **Never**. When a resource is reindexed, its scan is rebuilt from scratch: obsolete pages are removed and new ones are discovered.
A resource that ended in **Failed**, or one set to **Never**, can still be pushed back into the queue manually with the **Reindex** action in the row menu.
#### Deleting a resource
Use **Delete** from the row menu to remove a resource.
{% hint style="warning" %}
Deleting a **scan root** also deletes every page and document discovered from it. Xenioo asks you to confirm before removing a website scan.
{% endhint %}
Discovered sub-pages cannot be deleted on their own they are recreated on the next reindex. To keep a specific sub-page out of the index, **ban** it instead.
### Content images
Text is only part of what your content holds a lot of information lives in pictures: diagrams, screenshots, infographics, or a pasted screenshot of a log or table. When **Index content images** is enabled, Xenioo sends each usable image to your model, which reads it and writes a short description. That description is indexed like any other text, so a question about what an image shows can retrieve it.
**Index content images** is a toggle on the resource's **Main** tab, available for every type. It defaults **on** for new Public URL and Confluence resources and **off** for files.
| Resource type | Which images are read |
| ------------------- | ------------------------------------------------------------------------------------------------------------ |
| **Public URL** | Images embedded in the crawled web pages. |
| **Confluence Page** | Images attached to and shown on the crawled pages. |
| **File** | Images embedded in an uploaded PDF or Word document. An image uploaded on its own is described the same way. |
Each kept image becomes a **child resource** of the page or document it came from, grouped underneath it exactly like a discovered sub-page so you can expand a page (or a "with images" document) to see its pictures, and deleting the parent removes them too.
{% hint style="warning" %}
Reading images uses **your own model** and therefore consumes tokens, billed to your model's account. Turn the option on where images carry real information and leave it off where they are purely decorative.
{% endhint %}
#### What gets skipped
To avoid spending tokens on chrome, Xenioo automatically drops images that are clearly not content **before** sending anything to the model:
* very small images - icons, emoji, avatars, favicons;
* hairline dividers, rules and spacers (anything under \~20 px on its short side);
* extreme slivers (very long, very thin bars);
* animated GIFs, which are usually decorations rather than information.
Everything else is kept including wide, short screenshots such as a pasted log line or a snippet of a table.
#### Seeing what was read from an image
Open an image resource and click **View Index**: the viewer shows the picture together with a **Detected text** panel containing exactly the description the model produced and indexed. It is the quickest way to check that a diagram or screenshot was understood the way you expected.