slug-tts-stt: giving a mini-PC ears and a mouth, on two different chips

Thu, 06 Aug 2026 00:00:00 +0000

A voice assistant needs two things from its hardware: it must hear you and it must answer you. On a single-box setup those workloads fight for the same chip — unless you deliberately split them. That’s the whole idea here: STT runs on the XDNA2 NPU (Whisper large-v3-turbo via FastFlowLM), TTS runs on the Radeon 840M iGPU (Qwen3-TTS 1.7B via qwentts.cpp/Vulkan). Both exposed as OpenAI-compatible HTTP endpoints on your LAN. The reasoning brain lives elsewhere; this box is only ever the audio edge.

quill-of-hermes: LAN OCR that eats PDFs and multi-page TIFFs

Thu, 30 Jul 2026 00:00:00 +0000

OCR as a service usually means cloud APIs, per-page billing, and your documents on someone else’s disk. This is the local counterargument: a FastAPI wrapper around OvisOCR2 running GGUF via llama.cpp’s llama-mtmd-cli, serving your LAN. POST an image, a PDF, or a multi-page TIFF; get back clean Markdown.

The design decisions worth noting: each page is sent to the model independently and results are joined with a horizontal rule, so a 50-page PDF degrades gracefully instead of failing whole. Jobs are serialized to keep the GPU sane, there’s a five-minute timeout per page so one bad scan can’t wedge the queue, and temp files are cleaned unconditionally. The service deliberately stays separate from llama-swap because multimodal llama-server support for this model wasn’t proven — llama-mtmd-cli was.

Ai on AiSlackin

slug-tts-stt: giving a mini-PC ears and a mouth, on two different chips

quill-of-hermes: LAN OCR that eats PDFs and multi-page TIFFs