Run a Local LLM on Your Computer: Ollama and Open WebUI Guide

What it is

Running a local large language model (frequently abbreviated as LLM) means hosting and executing artificial intelligence software entirely on your own computer hardware. Instead of transmitting prompts and sensitive data across the internet to remote corporate data centers, local execution keeps all computations directly on the local machine. This architecture ensures complete privacy, provides offline accessibility, and gives individual computer owners full control over their models and conversation histories.

To achieve a seamless local AI experience, two primary tools are commonly paired together: Ollama and Open WebUI.

Ollama operates as a local model execution engine and manager. Distributed under the open-source MIT license according to its repository metadata, Ollama simplifies the process of obtaining, configuring, and executing open models locally or in cloud environments. It manages the underlying complexities of machine learning inference and utilizes the open-source llama.cpp project (founded by Georgi Gerganov) as a supported backend engine. Ollama handles downloading model weights, loading them into memory, and exposing a local REST API and command-line interface.

Open WebUI acts as the visual, user-facing layer. It is an extensible, self-hosted web interface that provides a clean, browser-based environment reminiscent of modern chat applications. While Ollama handles the heavy lifting of running the model weights in the background, Open WebUI provides the graphical interface where users type questions, view responses, upload reference materials, and configure settings. Together, Ollama and Open WebUI form a complete, self-contained AI workstation on a personal computer.

What it can do

The combination of Ollama and Open WebUI provides a versatile environment that spans interactive conversation, programming workflows, document retrieval, and external platform integrations.

Model Execution and Integrations

Ollama enables computers to run a broad catalog of open models, including Gemma, DeepSeek, Qwen, Kimi, GLM, MiniMax, and gpt-oss. Through Ollama, users can interact with models directly from the command line, run models programmatically, or connect models to external applications and developer tooling:

• Coding Integrations: Ollama connects to coding tools, autonomous agents, and editors such as Claude Code, Codex, Copilot CLI, DeepSeek Harness, Droid, and OpenCode. Its ecosystem includes support for code editors and extensions such as Cline, Continue, Void, Copilot for Obsidian, twinny, gptel, Ollama Copilot, Obsidian Local GPT, Ellama, orbiton, AI ST Completion, VT Code, QodeAssist, AI Toolkit for VS Code, Open Interpreter, and oxi.
• Messaging and AI Assistants: Through an OpenClaw integration, Ollama can operate as an AI assistant spanning communication channels like WhatsApp, Telegram, Slack, and Discord. Community bots also support platforms such as Telegram and Discord.
• Developer Libraries and Frameworks: Ollama features official libraries for Python (ollama-python) and JavaScript (ollama-js), alongside community integrations for languages such as Go, Java, Rust, Ruby, C#, .NET, Swift, Julia, PHP, and Dart. It also integrates with agent frameworks like AutoGPT and crewAI, as well as observability platforms like Opik, OpenLIT, Lunary, and Langfuse.

Web Interface and Workspace Features

Open WebUI provides extensive functionality to manage and enhance conversational interactions:

• Chat Experience: Open WebUI connects directly to an Ollama instance running on the same computer or communicates with an Ollama instance hosted on a remote server.
• Document Analysis and Retrieval-Augmented Generation (RAG): The platform can read plain text formats directly, including CSV, HTML, TXT, MD, Markdown, RST, and XML files. To process complex documents such as PDFs and Office files, Open WebUI can connect to external document extractors like Tika or Docling. It supports vector storage via PostgreSQL with pgvector, and can connect to embedding providers like Ollama, OpenAI, or Azure OpenAI to search knowledge bases and answer questions from documents.
• Speech and Audio: Open WebUI can integrate with external speech-to-text and text-to-speech services, while its standard container image bundles built-in speech and embedding runtimes.
• Code Interpreter: Open WebUI incorporates a code interpreter powered by Pyodide to run Python code directly in the browser environment.
• Authentication and Privacy Modes: Open WebUI supports multi-user environments with individual logins, or it can run in single-user mode without authentication. For sensitive or isolated environments, an offline mode setting prevents outbound connections to external services such as Hugging Face.

How to get started

Getting started involves installing Ollama on the host operating system, running a model, and deploying Open WebUI using Docker.

Step 1: Install and Launch Ollama

Ollama provides installation methods across macOS, Windows, and Linux.

On macOS, install Ollama by running the following command in a terminal:

curl -fsSL https://ollama.com/install.sh | sh

Users can also choose to download the manual installer file (Ollama.dmg) from the project website.

On Windows, install Ollama by running the following command in PowerShell:

irm https://ollama.com/install.ps1 | iex

Alternatively, Windows users can download the OllamaSetup.exe installer manually.

On Linux, run the installation script:

curl -fsSL https://ollama.com/install.sh | sh

After installation completes, start Ollama from the terminal:

ollama

To download and immediately start an interactive chat session with an open model such as Gemma 4, run:

ollama run gemma4

Step 2: Deploy Open WebUI Using Docker

The officially supported and recommended installation method for Open WebUI is Docker. Docker runs the interface in an isolated container on the machine.

Before launching the container, generate a persistent 32-byte hexadecimal secret key by running openssl rand -hex 32 in your terminal. Supplying a fixed secret key ensures that user sessions remain active when the container restarts.

Execute the following command to start Open WebUI using the standard image, replacing your-secret-key with the generated key:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data -e WEBUI_SECRET_KEY=your-secret-key --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Understanding what these flags do:

• -d: Starts the container in the background (detached mode).
• -p 3000:8080: Makes the web interface accessible on port 3000 of your computer.
• --add-host=host.docker.internal:host-gateway: Allows the Open WebUI container to connect over internal networking to Ollama running on the host computer.
• -v open-webui:/app/backend/data: Mounts a persistent Docker storage volume so that user accounts, chats, and configurations survive container recreation.
• -e WEBUI_SECRET_KEY=your-secret-key: Maintains persistent authentication sessions across restarts.
• --restart always: Ensures the container restarts automatically if the host system reboots.

Step 3: All-in-One Bundled Containers (Optional)

Open WebUI also offers container variants that bundle both Ollama and Open WebUI inside a single container.

For a CPU-only bundled setup:

docker run -d -p 3000:8080 -v ollama:/root/.ollama -v open-webui:/app/backend/data -e WEBUI_SECRET_KEY=your-secret-key --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama

For systems with supported GPU hardware:

docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data -e WEBUI_SECRET_KEY=your-secret-key --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama

Once the container is running, open any web browser and navigate to http://localhost:3000 to create your initial administrator account and begin chatting.

Pricing and requirements

Pricing and plans are subject to the official project pages; visit Ollama and the Open WebUI Quick Start Documentation for current details. According to its repository metadata, Ollama is distributed as open-source software under the MIT license.

System and Hardware Requirements

• Operating Systems: Ollama supports macOS, Windows, and Linux. Open WebUI functions on macOS, Windows, and Linux across x86_64 and ARM64 architectures (including hardware like the Raspberry Pi and NVIDIA DGX Spark).
• Runtime Dependencies: Open WebUI’s recommended setup requires Docker. It can also be deployed via Python for manual or low-resource environments, or via Kubernetes for orchestrated enterprise environments.
• GPU Acceleration: While models can run on CPUs, GPU acceleration is supported. Open WebUI provides specialized container tags built against CUDA 12.8 (:cuda, requiring --gpus all) and CUDA 12.6 (:cuda126).
• Storage and Image Sizes: According to the Open WebUI documentation, download sizes vary depending on the container image variant. The standard :main image downloads at approximately 1.62 GB on Linux/amd64 because it includes the machine learning stack, local embeddings, and speech runtimes. The :slim variant downloads at approximately 196 MB (an 88% reduction saving about 1.42 GB on a fresh pull, checked on October 10, 2026), omitting internal model runtimes in favor of external providers.
• Database Support: Open WebUI supports SQLite and PostgreSQL databases, and provides file storage via local storage or Amazon S3 buckets.

Who it is for

The pairing of Ollama and Open WebUI is designed for a broad spectrum of users:

• Beginners and General Users: Anyone looking for an accessible, private AI chat experience on their personal computer without sending personal conversations or sensitive documents to third-party cloud services.
• Software Developers: Programmers seeking to incorporate local language models into code editors, command-line interfaces, or custom software projects via REST APIs and native client libraries.
• System Administrators and Teams: Technical operators who need a self-hosted, multi-user web interface capable of running within local area networks or air-gapped environments with full control over user access, data storage, and knowledge bases.
• Researchers and Explorers: Enthusiasts who want to experiment with different open-source model architectures on consumer hardware.

Alternatives

• LM Studio is an application designed to discover, download, and run local language models on personal computers.
• llama.cpp is an open-source project that serves as a foundational engine for running language models on consumer hardware.

Sources

• Ollama
• Open WebUI Quick Start
• LM Studio
• llama.cpp