| Qwen3.8-27B, custom Q4-class mix | Tesla P100 16GB + Tesla P100 16GB | short prompt | 49 tok/s plain decode, ~118 tok/s with MTP (3 drafts) | own engine vs patched llama.cpp (32 tok/s) |
| Qwen3.8-27B, custom Q4-class mix | Tesla P100 16GB + Tesla P100 16GB | 2048-token prompt | 558 tok/s prompt processing | own engine vs patched llama.cpp (~405 tok/s) |
| Wan 2.2 TI2V-5B (Q8, 20 steps) | Tesla P100 16GB + Tesla P100 16GB | 704x1280, 73 frames | 12.8 min per clip (RTX 3070 in ComfyUI: 8.5 min) | own CUDA engine, model split across both cards |
| Wan 2.2 TI2V-5B Turbo (Q6_K, 4 steps) | Tesla P100 16GB + Tesla P100 16GB | 704x1280, 73 frames | 4.6 min per clip (RTX 3070: 4.0 min) | — |
| Qwen3.8-27B QLoRA r16 | Tesla P100 16GB + Tesla P100 16GB | 2048-token samples | ~147 s per step of 8 samples (~5 days per epoch of 22,800 samples) | — |
| Qwen3.8-27B Q6_K | Tesla P100 16GB + Tesla P100 16GB | 155K (q8_0 KV) | 55.9 tok/s decode | iommu=pt, own engine, MTP 3 |
| Qwen (local models) | RTX 3070 8GB | 458K | Runs and answers correctly with q8_0 KV cache, 7.8 of 8 GB VRAM used | YaRN ×1.75 (up from 262K native) |
| Qwen3.8-27B Q5 | RTX 3070 8GB + Tesla P100 16GB | 160K | 9.3–9.9 tok/s decode | -sm tensor, split 6.5 / 16 |
| Qwen3.8-27B Q5 | RTX 3070 8GB + Tesla P100 16GB | 111K | 95–143 tok/s prompt processing on 3K+ token prompts | Q5 instead of Q4_0 (110–170 tok/s) |
| Qwen3.8-27B Q5 | RTX 3070 8GB + Tesla P100 16GB | 111K | 9.9 tok/s decode, 83 tok/s prompt (tensor split) | -sm tensor instead of -sm layer (7.8 decode / 89 prompt) |
| Qwen3.8-27B Q5 | RTX 3070 8GB + Tesla P100 16GB + remote P100 over 1GbE | — | 5.91 tok/s decode (layer split over RPC) | Added a second machine's P100 over the network |
| Qwen3.8-27B Q4_0 | RTX 3070 8GB + Tesla P100 16GB | 7.8K-token prompt | 286 tok/s prompt processing | batch 8192 / ubatch 2048 (was 512 / 256: 171 tok/s) |
| HunyuanVideo FramePack I2V | RTX 3070 8GB | — | 4.5 s clip in ~7 min, no OOM | First end-to-end image-to-video run on this card |
| Qwen3-Coder 80B | Consumer GPU + DDR4 offload | 131K | 20.1 tok/s short prompt, 22.5 tok/s long prompt | K cache q8_0 to q4_0, --cache-reuse 256, llama.cpp b9006 to b9133 |