All posts

Every write-up, newest first. Each one comes from a real run on the hardware in the lab.

  1. Why I wrote my own inference engine for two P100s

    Patching llama.cpp got a 27B model to 32 tok/s on two P100s. A from-scratch CUDA engine built for just this card and model reached 49 tok/s plain and about 118 tok/s with the model's own draft head.

    Local AITesla P100Multi-GPU
  2. P100 power caps, clocks and cooling: what the numbers say

    The P100 is a passive 250 W card. What a 125, 150 and 250 W cap does to its clocks and temperatures, why nvidia-smi -lgc fails on Pascal, and the fan mistake that let both cards hit 79 °C.

    Local AITesla P100Used GPUs
  3. Beyond chat: video generation and fine-tuning on two P100s

    Two P100s can make a short AI video clip and can fine-tune a 27B model. Both work. Neither beats a single RTX 3070 for video, and fine-tuning is measured in days.

    Local AITesla P100Multi-GPU
  4. Securing a local AI agent: best practices

    An agent that can read your mail, run commands and browse the web needs more than a good prompt. Ten practices for the tool layer, where the real risk is, plus a checklist.

    Agents and securitySecurityPrompt injection
  5. Two P100s in one Ubuntu box: driver 580, CUDA 12.9 and the IOMMU trap

    Moving both P100s into one Ubuntu 26.04 machine. The setup was easy except for one thing: GPU-to-GPU copies ran at 0.14 GB/s until I changed one kernel setting.

    Local AITesla P100Multi-GPU
  6. Stretching context to 500K on 8 GB cards, and why I backed it out

    YaRN makes huge context windows easy to turn on. On small cards it's the KV cache, not the setting, that decides whether it works.

    Local AIllama.cppLong context
  7. Pairing a P100 with a gaming GPU: splits, buffers and the small-card trap

    How to split one model across a 16 GB P100 and an 8 GB RTX 3070, and why the smaller card sets every limit.

    Local AITesla P100Multi-GPU
  8. CUDA 13 and PyTorch dropped the P100: what still works

    'No kernel image is available for execution on the device.' Here's which llama.cpp and PyTorch builds still run on sm_60.

    Local AITesla P100CUDA
  9. Is a used Tesla P100 worth it for local LLMs in 2026?

    Around $89 for 16 GB of HBM2. It's the cheapest VRAM you can buy, as long as you can live with the catches.

    Local AITesla P100Buying guide
  10. Q4_0 vs Q5 on a 27B: prompt speed is basically a wash

    I assumed the bigger quant was costing me prompt speed. Fifty production logs said otherwise.

    Local AIllama.cppQuantization
  11. Splitting a 27B model across mismatched GPUs: tensor vs layer vs RPC

    An RTX 3070 plus a Tesla P100 in one box beats adding more GPUs over the network. Here's why, with numbers.

    Local AIllama.cppMulti-GPU
  12. The one flag that made prompt processing 67% faster (and why I lost it)

    Raising llama.cpp's ubatch took a 27B model from 171 to 286 tok/s on long prompts. Then it quietly disappeared from my launchers.

    Local AIllama.cppPrompt speed
  13. Image-to-video on an 8 GB RTX 3070 with FramePack

    A 4.5-second, on-model animation of a cartoon sprite on a green screen, in about 7 minutes on an 8 GB card.

    AI mediaFramePackImage to video
  14. +65% decode on an 80B MoE from two cheap changes

    Quantizing the K cache to q4_0 and updating llama.cpp took an 80B MoE coder model from 12 to 20 tok/s.

    Local AIllama.cppKV cache