In short
- Training your own AI is neither necessary nor sensible for most businesses. The usual route is called retrieval-augmented generation, or RAG for short.
- With RAG, the system searches your documents for suitable passages for every question and lets the language model answer only from them, with sources.
- Quality depends above all on your documents: current, unambiguous, without duplicates and with clear access rights.
- Language models can run in EU data centres or on your own servers. Local AI brings control, but costs hardware and maintenance.
Train your own AI? Usually not necessary
When people talk about “our own AI”, they often mean one of four things. They differ greatly in effort and benefit:
| Approach | What happens | Good for | Limits |
|---|---|---|---|
| Adding documents in the chat | Documents are uploaded for one question | Individual tasks, quick tests | No lasting knowledge, no rights management |
| RAG | The system searches your documents and answers from the passages found | Manuals, policies, product knowledge, customer service | Stands or falls with the quality of the documents |
| Fine-tuning | A model is trained further with examples | Consistent style, fixed formats, special classifications | Poor for facts that change, effort with every change |
| Training your own model | A model is built from scratch | Research, very large organisations | Practically never sensible for small and medium-sized businesses |
For knowledge that changes, such as prices, processes or product data, RAG is almost always the better choice. Fine-tuning can complement it if style or format must be very consistent.
How RAG works
The method was described in 2020 in a paper by researchers led by Patrick Lewis and is now standard for knowledge applications. In simplified form it works like this:
- Collect documents: manuals, policies, product sheets, contracts, earlier replies, from stores such as SharePoint, drives or the wiki.
- Split them up: the documents are divided into sensible sections, for example by headings.
- Make them searchable: each section gets a mathematical description of its meaning and goes into a vector database, often complemented by a classic keyword search.
- Search: for every question the best-matching sections are found, and only from documents the person asking is allowed to see.
- Answer: the language model formulates the answer from these sections and names the sources. If the answer is not in the documents, it says so.
Typical uses
- Internal knowledge: staff ask about processes, policies or responsibilities and get an answer with a link to the document.
- Technical service: service technicians find the right place in manuals and maintenance instructions, even on the road.
- Customer service: an AI chatbot or an email assistant answers questions from your product documentation.
- Contracts and tenders: passages on deadlines, liability or requirements are found quickly in long documents.
Where knowledge bases fail
- Outdated and duplicate documents: if there are three versions of a policy in storage, the AI may find the wrong one.
- Scanned PDFs and tables: without good text recognition and preparation, content gets lost.
- Missing rights: if the AI sees everything, everyone soon sees everything, including salary lists. Access rights must apply during the search.
- No measurement: without test questions with known answers, nobody knows how often the system is right.
- No sources: answers without a reference cannot be checked and squander trust.
Local AI or EU cloud?
There are three ways to run the language model. The large providers offer their models in EU data centres and state that, for business and API offerings, customer data is not used for training by default. Check this in the contract terms. There are also European providers. And there are open models, for example from Mistral AI, Meta or Alibaba, that you can run on your own servers.
Local AI brings full control over the data. In return you need suitable hardware with graphics processors, maintenance and updates, and the best open models do not match the largest cloud models in every task. For many businesses, a model in an EU data centre with a clear contract is the sensible middle way. Local models pay off for particularly sensitive data or very high volumes. Which route suits your data is something AI consulting can settle.
Data protection and security
Personal data in documents remains personal data. You need a legal basis, data processing agreements and deletion rules. In an opinion from December 2024, the European Data Protection Board dealt in detail with AI models and personal data. OWASP explicitly names weaknesses in vector and embedding systems as a risk, for example when rights do not apply during the search. And documents can contain hidden instructions meant to manipulate the system. The rule is therefore: check rights with every search, control sources, log answers.
First steps
- Choose one knowledge area: for example service manuals or internal policies.
- Tidy up documents: delete outdated versions, assign owners.
- Collect test questions: 30 to 50 real questions with the correct answer and reference.
- Pilot with one group: measure results, incorporate feedback, then expand.
Sources
- Lewis et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, NeurIPS 2020
- OWASP: Top 10 for LLM Applications 2025
- European Data Protection Board: Opinion 28/2024 on AI models
- OpenAI: Enterprise privacy
- Regulation (EU) 2016/679 (General Data Protection Regulation): EUR-Lex