Is a used Tesla P100 worth it for local LLMs in 2026?

2026-09-27 · Local AI · Hardware · Tesla P100 16GB, RTX 3070 8GB

If you want to run bigger local models, VRAM is the thing you’re buying. The cheapest VRAM on the used market right now is a nine-year-old datacenter card.

Prices (eBay US, fixed-price listings, checked 2026-09-27)

*Prices include shipping. The P100 median is from 61 listings, but good ones regularly go for about $89, and the cheapest was $75. The 3070 figure is the median of 46 listings. Parts-only and wrong models are excluded. Prices move, so treat these as a snapshot.

For the price of one used 3070 you can buy three P100s: 48 GB versus 8 GB.

What it does well

• Holds a lot of model. In my setup the P100 carries about 73% of a 27B’s weights while an 8 GB gaming card does the rest (Part 1 of the speed series).
• Memory bandwidth. HBM2 is fast, and token generation is mostly a bandwidth problem.
• FP16 compute is solid, unlike many other cheap datacenter cards of its era.

The catches

1. Software support is fading. It’s compute capability 6.0 (sm_60). CUDA 13 builds and current PyTorch wheels skip it; see the next post for what still works.
2. No fast bf16. Use float32 or fp16 for anything that defaults to bf16.
3. Passive cooling. It’s made for server airflow and has no fan of its own. In a desktop you need to add one.
4. No display outputs. It’s a pure compute card, so you need another GPU or integrated graphics for your monitor.
5. It’s slower than modern cards per token. You’re buying capacity, not speed.

Who should buy one

• Yes: you have a gaming card that almost fits the model you want, and 16 more GB would get you there.
• Yes: you’re on a tight budget and want 27B–30B-class models at usable speed.
• No: you need the latest PyTorch features, bf16 training, or plug-and-play.
• Consider instead: a used RTX 3090 (24 GB, much faster, modern software) if the budget stretches that far.

Next: getting software running on it in 2026.