# wimpy-setup: a dual-GPU inference box you can rebuild from scripts

*2026-06-28*

> Reproducible setup scripts for a Ryzen 7700 bare-metal host running AMD R9700 and RTX 5060 Ti inference concurrently — llama.cpp twice, llama-swap once.


This is not a guide. It's the actual, ordered, idempotent script set that builds my inference server — shared because rebuilding from memory is how homelabs die. Wimpy is a Ryzen 7 7700 with two GPUs serving LLM inference *at the same time*: an AMD Radeon AI PRO R9700 (32GB, ROCm) and an NVIDIA RTX 5060 Ti (16GB, CUDA), behind one llama-swap endpoint.

The dual-GPU part is the reason this repo exists. HIP and CUDA backends of llama.cpp share library filenames, so they can't coexist in one prefix — there are two isolated builds (`/usr/local` for ROCm, `/opt/llama-cuda` for CUDA) and one llama-swap config with two non-exclusive groups. A `-cuda` model entry is the same GGUF as its AMD twin with a different binary and device pin, so either card can serve it. GPUs are pinned by UUID (the Ryzen iGPU also enumerates as a ROCm device and will happily steal your workloads if you let it).

Features:

- Nine ordered idempotent host scripts (`run-all.sh --from/--only`) covering base system through KVM
- Dual-GPU llama.cpp: ROCm gfx1201 + CUDA sm_120 builds in isolated prefixes, one llama-swap over both
- `fetch-model.sh` — full GGUF acquisition pipeline: inspect, validate, GPU smoke-test, register, auto-deploy with rollback; handles HF repos, URLs, and local files
- Automatic mmproj projector resolution for vision-language models, renamed to prevent collisions between repos
- Mid-pipeline failures write a resume dossier instead of leaving you stranded
- Parameterized Hermes VM provisioning (`hermesvm-setup.sh --hostname X`), clone-and-go for every future VM
- Bidirectional SSH trust scripts with merge-never-replace sshd dropins — re-running can't lock anyone out
- Stubbed test suite that runs without network or GPU

Source: [github.com/rahlquist/wimpy-setup](https://github.com/rahlquist/wimpy-setup)