gputaskman: see what is really using your GPU memory
Added NewCopyable prompt below
Published Last verified Released
Our verdict
Answers the question every local-model user hits: Task Manager says "15 GB used", but by what? gputaskman shows per-process VRAM, GPU-accessible system RAM (spill) and engine load, then links processes to runtimes (CUDA, HIP, Vulkan, Direct3D, ggml, ONNX Runtime) and model files, so you can see whether to shrink the context, pick a smaller quant or close the browser before loading a bigger model. The README keeps measurements and estimates apart and documents live checks: on an RTX 4050 laptop on Linux every per-process size matched nvidia-smi, and a llama.cpp server on a Radeon RX 9070 showed 3.95 GB, matching 2.49 GB of weights plus 1.21 GB of f16 KV cache plus compute. It launched on 3 Nov 2026, reached about 3,400 stars in under two weeks, ships frequent releases (v0.4.3 on 15 Nov) and was covered by Neowin. Limits: the author labels it early and experimental; model classifications and GGUF-based breakdowns are estimates; what it can measure depends on the platform (WSL relies on Windows host counters, Intel and some AMD paths are less tested, and Apple Silicon has no separate VRAM to split); it does not dump VRAM. The live-check numbers are the author's, and we have not found an independent hands-on yet or run it ourselves.
- Who it's for
- Builders running local models with Ollama, LM Studio or llama.cpp, plus anyone fine-tuning or running vision, speech or diffusion models on their own GPU
- Leverage
- Know exactly what is filling your GPU, down to weights versus KV cache, so you can size context and quantization to fit instead of guessing after out-of-memory errors
- Effort
- Low
- Date added
What it does
gputaskman (shanselman/gputaskman, MIT) is a Python terminal UI with a device view (VRAM, GPU and memory activity, PCIe traffic, temperature, power, history charts), a process view (local GPU memory, GPU-accessible system RAM, estimated commit, engine activity, detected runtimes), a system RAM and CPU panel, and a model inspector that shows which model a process holds, how it was identified, architecture, quantization, layers, weights and estimated KV cost. Live server hints add context size, busy slots and throughput where available. It reads NVML, AMD ADL or SMI, Windows WDDM counters, Linux DRM data and macOS IOAccelerator statistics, with no administrator rights, API key or CUDA Toolkit needed. --once and --json give scriptable output from the same collector, and sessions can be recorded and replayed offline; review a recording before sharing because it can include local ports. It installs from PyPI with uv or pipx, as portable Windows ZIPs and Linux tarballs, or from a clone.
How builders use it
- Install with uv tool install gputaskman (or the portable Windows or Linux build) and run gputaskman while your model server is loaded.
- Open the model inspector on the llama.cpp, Ollama or LM Studio process to see weights, KV cache and compute buffers separately.
- If KV cache dominates, lower the context size; if weights dominate, try a smaller quant; if memory has spilled to system RAM, close other GPU apps first.
- Use gputaskman --json in a script to log VRAM per process while you benchmark different models or settings.
What to expect
Out-of-memory and slow-generation problems become a clear reading (which process, weights or KV cache, spilled or not) instead of trial and error.
Copy this: Install + one-shot checks
# Official source: github.com/shanselman/gputaskman (MIT). Python 3.11+.
uv tool install gputaskman # or: pipx install gputaskman
gputaskman # live terminal UI
gputaskman --once # one text snapshot
gputaskman --json > gpu.json # same data as JSON for scripts
# Portable builds (no Python needed): GitHub Releases
# gputaskman-VERSION-windows-x64.zip
# gputaskman-VERSION-linux-x64.tar.gz (glibc 2.35+, NVIDIA driver or amdgpu/i915)
# macOS (Apple Silicon): run from a clone with: uv run gputaskman
# Ask your agent to tune a local model with it:
Run gputaskman --json while my llama.cpp server is loaded. Report weights, KV cache and compute buffer sizes for the model process and how much memory spilled to system RAM, then suggest the largest context size that fits in VRAM.Related discoveries
See the newest picks →- ToolCoding & Shipping Agents
Aider
Open-source A.I. pair programming in your terminal, with git commits built in.
Effort: Low#Local / BYO model#Open source
- ToolCoding & Shipping Agents
opencode
An open-source coding agent for the terminal that works with many model providers.
Effort: Low#Local / BYO model#Open source
- ToolCoding & Shipping Agents
magpie: choose the model for every coding agent in one app
A menu bar app and local gateway that lets Claude Code, Codex, Gemini CLI and dozens of other agents run on whichever model, key or plan you pick.
Effort: LowPrompt#Local / BYO model#Open source
- ToolMulti-Step Agent Setups
Hermes Agent: a free, self-hosted agent with skills, cron and a Kanban board
Nous Research's MIT-licensed agent that runs on your server, keeps memory and skills, runs scheduled jobs and coordinates work on a built-in Kanban board.
Effort: MediumPrompt#Local / BYO model#Open source
Get the best new A.I. picks in your inbox
The best new agent workflows and tools we’ve vetted, with a clear verdict on each. Short, occasional, useful.