Many companies want to give their employees an AI chat without company data ending up with a US service. A proven approach is a self-hosted platform based on Open WebUI. This post shows how such a platform is set up and what to look out for when running it.
What is Open WebUI?
Open WebUI is a self-hosted web interface for large language models. It looks similar to ChatGPT and offers chat history, file uploads, knowledge bases for your own documents, user management with roles, and connections to a range of models. The interface runs on your own servers or in your own cloud environment.
Open WebUI does not generate answers itself. It forwards requests to a language model of your choice.
A typical enterprise setup
An enterprise platform usually has three layers:
- Interface: Open WebUI for chat, documents and user management.
- Gateway: a proxy such as LiteLLM that routes requests to the models, tracks costs per team and manages API keys centrally.
- Models: either cloud models in a European region, such as Azure OpenAI, or local open-source models such as Llama or Mistral, served with Ollama or vLLM.
With a gateway in place, you can swap models without changing anything for your users.
Cloud or on-premise?
Cloud models hosted in the EU deliver the best answer quality and need no GPU hardware of your own. A data processing agreement and a European region are essential.
Local models never leave your network and suit particularly sensitive data, up to fully offline operation. In return, you need suitable hardware, and answer quality depends heavily on the model you choose.
In practice, many companies combine both: local models for confidential content and cloud models for everything else.
Single sign-on and permissions
Open WebUI supports sign-in via OAuth and OpenID Connect, for example with Microsoft Entra ID. Employees log in with their existing company account. Groups and roles let you control who may use which models and knowledge bases.
Open WebUI and GDPR: what matters
- Decide where chat histories and documents are stored, and define retention periods.
- Sign data processing agreements with every cloud provider involved.
- Configure logging deliberately: as much as needed for operations and cost control, as little as possible.
- Train employees on which data they may enter.
A self-hosted platform is not a free pass. But it gives you the control that public services lack.
Common pitfalls
- Licence: Check the current Open WebUI licence terms, especially regarding branding in larger deployments.
- Updates: The project moves fast. Plan regular updates and test them before rolling them out.
- Knowledge bases: The quality of answers based on your own documents depends heavily on how those documents are prepared. Read more in our post on RAG for SMEs.
How DFT IT can help
We build and run platforms like this in Azure, on-premise or fully offline. Learn more on our Generative AI page. To roll it out to your team, we offer matching AI training courses. Get in touch if you are planning your own AI platform.