ENTERPRISE AI WORKSHOP v1.0.4

CHANNEL AI ARENA

↓ Knowledge ZIP STANDBY Admin ↗
COMPETITION HISTORY

Passing checks by round

PassedFailedReview

Facilitator sign in

Enter the Arena administrator password to manage the competition.
CHANNEL AI ARENA · PARTICIPANT GUIDE

Build your own RAG assistant.

Start with a custom model, then give it your own documents. Test and improve until it handles all five challenges.

Open WebUI link is not configured yet. Ask your facilitator. ↓ Download knowledge ZIP
STEP 1 OF 6

Create your custom model first

Open Open WebUI using the link above. Choose Workspace on the left, make sure you're on the Models tab, then click Create.

1 Workspace & Models 2 Create a model

You are creating your own team assistant, not modifying the shared base model.

Open WebUI Workspace and Models screen with Workspace and Create highlighted 1 2
Open WebUI → Workspace → Models → Create
LEARN THE CONCEPTS

Understanding retrieval-augmented generation

Optional reference material. Your six workshop steps remain unchanged.

01What is RAG?How documents reach an AI assistant

Retrieval-augmented generation (RAG) lets a language model use information retrieved from a document collection when answering a question. Rather than relying only on what the model learned during training, the application finds relevant passages and provides them as context.

1PrepareImport and index documents
2SearchFind relevant passages
3GenerateAnswer using retrieved context
4CheckReview answer and sources

The documents are generally split into searchable chunks. Search can use keywords, vector embeddings, or both. The retrieved context is sent to the language model with the user's question. RAG is not the same as retraining the model—changing documents changes what the assistant can retrieve, not its underlying weights.

In Open WebUI, a knowledge collection provides document retrieval. Your model's system prompt influences how it responds. You can test the combined behavior in chat and revise either part.

02Common RAG failure modesWhere document-grounded answers can go wrong

Failures can happen at any stage. A plausible-sounding answer does not guarantee that the right information was retrieved or used correctly.

Documents and indexing

Missing coverage

The answer is not present in the uploaded information.

Outdated content

The indexed material no longer reflects the latest source.

Import and parsing errors

Tables, formatting, images or file content are lost during extraction.

Chunking problems

A passage is split apart from the context needed to interpret it.

Search and retrieval

Retrieval miss

A useful passage exists, but the search fails to find it.

Irrelevant matches

Similar-looking passages crowd out the material that matters.

Ranking or recall errors

Too few results are retrieved, or the most useful ones are ranked too low.

Terminology mismatch

Names, codes, abbreviations or paraphrases aren't matched correctly.

Context and generation

Context overload

Too much retrieved text distracts the model or exceeds its context window.

Unresolved ambiguity

Multiple passages are hard to reconcile or lack enough context.

Unsupported answers

The model fills gaps with invented or unverified claims.

Incorrect attribution

An answer cites a source that does not support its claim.

Instruction interference

Text inside retrieved content may be treated as instructions rather than data.

Incomplete answers

The response omits important qualifying details or misreads the question.

Access and operation

Access mistakes

Retrieval uses the wrong collection or reveals information across boundaries.

Tool and API failures

Search tools loop, time out, or produce malformed responses.

Latency and cost

Repeated retrieval or large context makes answers slow and expensive.

Evaluation mismatch

A test result reflects retrieval variation, prompt changes, or an unsuitable grading criterion.

A useful debugging approach is to separate: what was in the documents, what was retrieved, and what the model answered. Each can fail independently.

How to play: Build and test your assistant using your own knowledge files. Green = pass · red = fail · amber = review.

Step 1 of 6

Admin details

Participant answer