KoboldCpp runs AI models on your own computer. This community wiki has the details. New to it? The beginner guide takes you from download to your first chat in four steps. The project also has its own KoboldCpp wiki on GitHub.
Download and install
Section titled “Download and install”Every release file and what it needs: Release files and system requirements. Step by step for Windows, Linux, macOS, Android, and Colab and Docker. Also: Updating, Build from source and Staying safe.
Models and hardware
Section titled “Models and hardware”- Models: GGUF and quantization, Where to get models, How much memory a model needs.
- Hardware: GPU backends, GPU layers, Multiple GPUs, Saving VRAM, Old PCs and CPU-only, Apple Silicon, RPC.
Launcher and settings
Section titled “Launcher and settings”The launcher tour explains every tab. Also: Context size, Config files, Templates, Command line, Admin mode.
Features
Section titled “Features”KoboldAI Lite, Image and video generation, Speech-to-text, Text-to-speech, Music, Vision and audio input, Embeddings, Web search and MCP tools, KoboldCpp Agent, AI Horde worker, Multiplayer and server saves.
Apps and API
Section titled “Apps and API”Connect SillyTavern and other apps, the API overview and endpoints, Passwords and security, Remote access, Streaming and multiple users.
Troubleshooting
Section titled “Troubleshooting”Won't start, Model won't load, Out of memory, GPU not used or slow, Bad or strange output, Connection and API problems, OS-specific problems. Short answers: the FAQ.