Self-Hosted AI Stack

Self-Hosted AI Stack is a chat tool. An MIT-licensed Docker Compose project that deploys local models, chat, document retrieval, transcription and speech on your own Linux server, with optional HTTPS and NVIDIA GPU support.

Self-Hosted AI Stack is an open-source Docker Compose project that bundles the pieces of a private AI setup into one deployable package. Instead of a hosted service, it is a set of configurations for running local language models, browser chat, document processing, speech transcription and speech generation on a Linux server the user controls, whether that is a machine at home or a cloud instance. The website itself only provides project information and resources; the software lives in a GitHub repository. The bundle combines several well-known components. Ollama runs local models, and AnythingLLM supplies a browser-based chat interface. Docling parses documents and embeddings support semantic search and retrieval over them. ScribeCrate handles audio transcription, while Kokoro generates speech. LiteLLM routes model requests through a single gateway, and an MCP Gateway connects supported tools. Fresh deployments generate API keys and set password protection for AnythingLLM, so a new install is not left open by default. In practice, the project suits developers, homelab enthusiasts and small teams who want AI features without sending data to outside providers. Deployment runs through Docker Compose, with optional HTTPS and optional NVIDIA GPU acceleration; CPU configurations also exist. Users can start from a preconfigured lightweight stack or edit the Compose files to run only the services they need. Memory and performance needs depend on which services and models are chosen, so hardware planning matters. Licensing is split. The stack's configuration is MIT licensed, while each included service and model keeps its own license. Costs come from hardware, cloud hosting and any external providers a user adds. A paid book, The Self-Hosted AI Builder's Guide, covers deployment, security, backups and daily operation, and a free readiness guide helps people assess their computer and compare local, cloud, managed and hybrid approaches. Compared with hosted chat platforms, it trades convenience for control, and it sits closer to a DIY infrastructure kit than to a polished product.