Local AI Hardware Checker

1. 1. Pick hardwareChoose a dedicated GPU, unified memory, or CPU-only and select a matching preset.
2. 2. Set modelChoose model size, quantization, context, and workload.
3. 3. Use the fitCheck status, compare quant levels, and copy the setup.

It depends on quantization and context. Rough starting points: 8B Q4 often fits around 8 GB VRAM, 14B Q4 usually needs 12 to 16 GB, 32B Q4 starts around 24 GB, and 70B Q4 usually needs multi-GPU, large unified memory, or heavy offload.

Q4 greatly reduces memory demand and is the realistic starting point for many local tests.

For smooth use, the GPU with enough VRAM usually matters more. CPU and RAM help with CPU-only and offload, but once a model spills out of VRAM, it often becomes much slower.

Yes, if enough RAM is available. It can be much slower than GPU inference.

Yes, as a planning aid. Ollama, LM Studio, and llama.cpp can differ in memory layout, but VRAM, model size, quantization, and context remain the most important levers.

Long contexts need extra KV cache. That is why 32k or 128k can need much more VRAM/RAM than 4k or 8k.

On Apple silicon and comparable systems, CPU and GPU share the same memory. The calculator therefore subtracts a system reserve and evaluates the remaining shared memory instead of separate VRAM.

The calculator uses a small public JSON cache. The Raspberry Pi can refresh it weekly from public UL benchmark rankings and LinkLoot adds VRAM/AI notes for common GPUs.