https://lowbudgetlab.com/ Real experiments with local AI on low-budget hardware en https://lowbudgetlab.com/posts/p100-own-inference-engine/ https://lowbudgetlab.com/posts/p100-own-inference-engine/ Sun, 25 Oct 2026 00:00:00 GMT Patching llama.cpp got a 27B model to 32 tok/s on two P100s. A from-scratch CUDA engine built for just this card and model reached 49 tok/s plain and about 118 tok/s with the model's own draft head. https://lowbudgetlab.com/posts/p100-power-and-cooling/ https://lowbudgetlab.com/posts/p100-power-and-cooling/ Sun, 25 Oct 2026 00:00:00 GMT The P100 is a passive 250 W card. What a 125, 150 and 250 W cap does to its clocks and temperatures, why nvidia-smi -lgc fails on Pascal, and the fan mistake that let both cards hit 79 °C. https://lowbudgetlab.com/posts/p100-video-and-fine-tuning/ https://lowbudgetlab.com/posts/p100-video-and-fine-tuning/ Sun, 25 Oct 2026 00:00:00 GMT Two P100s can make a short AI video clip and can fine-tune a 27B model. Both work. Neither beats a single RTX 3070 for video, and fine-tuning is measured in days. https://lowbudgetlab.com/posts/securing-a-home-ai-agent/ https://lowbudgetlab.com/posts/securing-a-home-ai-agent/ Sun, 25 Oct 2026 00:00:00 GMT An agent that can read your mail, run commands and browse the web needs more than a good prompt. Ten practices for the tool layer, where the real risk is, plus a checklist. https://lowbudgetlab.com/posts/two-p100s-ubuntu/ https://lowbudgetlab.com/posts/two-p100s-ubuntu/ Sun, 25 Oct 2026 00:00:00 GMT Moving both P100s into one Ubuntu 26.04 machine. The setup was easy except for one thing: GPU-to-GPU copies ran at 0.14 GB/s until I changed one kernel setting. https://lowbudgetlab.com/posts/long-context-small-cards/ https://lowbudgetlab.com/posts/long-context-small-cards/ Sun, 27 Sep 2026 00:00:00 GMT YaRN makes huge context windows easy to turn on. On small cards it's the KV cache, not the setting, that decides whether it works. https://lowbudgetlab.com/posts/p100-pairing-gaming-gpu/ https://lowbudgetlab.com/posts/p100-pairing-gaming-gpu/ Sun, 27 Sep 2026 00:00:00 GMT How to split one model across a 16 GB P100 and an 8 GB RTX 3070, and why the smaller card sets every limit. https://lowbudgetlab.com/posts/p100-software-2026/ https://lowbudgetlab.com/posts/p100-software-2026/ Sun, 27 Sep 2026 00:00:00 GMT 'No kernel image is available for execution on the device.' Here's which llama.cpp and PyTorch builds still run on sm_60. https://lowbudgetlab.com/posts/p100-worth-it-2026/ https://lowbudgetlab.com/posts/p100-worth-it-2026/ Sun, 27 Sep 2026 00:00:00 GMT Around $89 for 16 GB of HBM2. It's the cheapest VRAM you can buy, as long as you can live with the catches. https://lowbudgetlab.com/posts/q4-vs-q5-prompt-speed/ https://lowbudgetlab.com/posts/q4-vs-q5-prompt-speed/ Sun, 27 Sep 2026 00:00:00 GMT I assumed the bigger quant was costing me prompt speed. Fifty production logs said otherwise. https://lowbudgetlab.com/posts/splitting-27b-across-gpus/ https://lowbudgetlab.com/posts/splitting-27b-across-gpus/ Thu, 24 Sep 2026 00:00:00 GMT An RTX 3070 plus a Tesla P100 in one box beats adding more GPUs over the network. Here's why, with numbers. https://lowbudgetlab.com/posts/big-ubatch-prompt-speed/ https://lowbudgetlab.com/posts/big-ubatch-prompt-speed/ Thu, 20 Aug 2026 00:00:00 GMT Raising llama.cpp's ubatch took a 27B model from 171 to 286 tok/s on long prompts. Then it quietly disappeared from my launchers. https://lowbudgetlab.com/posts/framepack-image-to-video-3070/ https://lowbudgetlab.com/posts/framepack-image-to-video-3070/ Thu, 02 Jul 2026 00:00:00 GMT A 4.5-second, on-model animation of a cartoon sprite on a green screen, in about 7 minutes on an 8 GB card. https://lowbudgetlab.com/posts/kv-cache-q4-moe-speedup/ https://lowbudgetlab.com/posts/kv-cache-q4-moe-speedup/ Wed, 13 May 2026 00:00:00 GMT Quantizing the K cache to q4_0 and updating llama.cpp took an 80B MoE coder model from 12 to 20 tok/s.