AISignal
Subscribe

Best GPU for Running Local LLMs (2026): How to Choose

For running language models at home, the graphics card's memory (VRAM) decides what fits. A model that doesn't fit in VRAM either won't load on the GPU or spills into system memory, which is much slower.

These picks are VRAM tiers rather than single cards, matched to model sizes with the same estimate as our run-locally calculator. Check a specific model there before you buy.

Last updated · How we picked

As an Amazon Associate we earn from qualifying purchases. Links open an Amazon search by spec, where Amazon shows current models, prices and availability. We list no prices, ratings or discounts.

Check a specific model's memory needs in the run-locally calculator →

Quick picks by situation

  1. Best for 7B–14B models

    Graphics card with 12 GB of VRAM

    Who it's for
    First local-LLM setups: chat, coding help and RAG experiments with small and mid-size models.
    Why
    By our calculator's estimate, 7B–8B models need about 6 GB at 4-bit and 12B–14B models about 8–10 GB, so 12 GB leaves room for longer context.
    Skip if
    You already know you want 24B-plus models; you would outgrow it.
    What to check
    Some card names come in more than one memory size. Confirm the VRAM on the exact listing, and check your power supply and case length.
    See options on Amazon

  2. Best for models up to about 24B

    Graphics card with 16 GB of VRAM

    Who it's for
    People who want mid-size models with room for longer context or a second small model.
    Why
    A 24B model at 4-bit needs about 15 GB by our estimate, a tight but workable fit in 16 GB.
    Skip if
    You mainly run 7B–8B models; 12 GB covers them.
    What to check
    Check the card's power connectors against your power supply, and that the slot and case have room for its length and thickness.
    See options on Amazon

  3. Best for 27B–32B models

    Graphics card with 24 GB of VRAM

    Who it's for
    Developers who want the larger open-weight models most people run at home.
    Why
    27B–32B models need about 17–20 GB at 4-bit by our estimate, so 24 GB fits them with context headroom. It is the most common target for a single-card local setup.
    Skip if
    You need 70B-class models on one card; they take about 41 GB at 4-bit.
    What to check
    High-end cards draw a lot of power. Check your power supply's wattage and native connector against the card maker's recommendation, and measure case clearance.
    See options on Amazon

  4. Best for 70B models on one card

    Workstation graphics card with 48 GB of VRAM

    Who it's for
    Teams and serious hobbyists who want 70B-class models on a single card.
    Why
    A 70B model at 4-bit needs about 41 GB by our estimate, which fits in 48 GB with room for context.
    Skip if
    A unified-memory machine with 64 GB or more would suit you, and its software support covers your tools.
    What to check
    Workstation cards often use blower coolers and different power connectors. Check airflow, connector type and driver support for your operating system.
    See options on Amazon

  5. Pair with: a power supply that fits the card

    ATX 3.1 power supply, 1000 W class

    Who it's for
    Anyone moving to a high-end card on an older power supply.
    Why
    Card makers list a recommended power supply wattage and connector for each card. A supply that meets both natively avoids adapter cables.
    Skip if
    Your current supply already meets the card maker's recommended wattage and has the right connector.
    What to check
    Match the wattage to the card maker's recommendation, the connector (12V-2x6 or PCIe 8-pin) to the card, and the form factor (ATX or SFX) to your case.
    See options on Amazon

How to choose

VRAM decides which models fit

Weights take roughly half a gigabyte per billion parameters at 4-bit, about one at 8-bit and two at full 16-bit precision, plus room for context (the KV cache) and runtime overhead. Runtimes such as llama.cpp can split layers between GPU and CPU when a model doesn't fit, but the CPU part runs much slower.

4-bit is the usual starting point

Most people run quantized models at home. 4-bit formats (such as Q4_K_M in GGUF, or AWQ) cut memory to about a quarter of full precision, with some quality loss that varies by model. 8-bit stays closer to the original and needs about twice the memory of 4-bit.

Context length costs memory too

Long context windows grow the KV cache. If you plan to paste in whole codebases or long documents, leave several gigabytes of headroom above the model's weights. The calculator shows what each context option adds.

Check software support for the exact card

NVIDIA cards use CUDA, which most local-LLM tools support first. AMD cards work through ROCm or Vulkan in tools such as llama.cpp; check your runtime's supported list for the exact card. Intel cards have their own backends.

Power, size and airflow

Check the card's length and slot width against your case, its power connectors against your power supply, and that the case moves enough air. A card that throttles or doesn't fit is a return.

One card or two

Some runtimes split a model across two GPUs and add up their VRAM. It works, but needs a motherboard with two suitable slots, a bigger power supply and more setup. One card with enough VRAM is simpler.

FAQ

How much VRAM do I need to run a 7B model?

About 6 GB at 4-bit with 8K context by our calculator's estimate, so an 8 GB card runs it and 12 GB leaves headroom. At full 16-bit precision it needs about 15 GB.

Can I run a 70B model on a 24 GB card?

Not entirely on the card. At 4-bit it needs about 41 GB. You can offload part of it to system RAM, which is much slower, or use 48 GB of VRAM, two cards, or a unified-memory machine with 64 GB or more.

Is a used graphics card worth it?

It can be. Check the exact model and VRAM, the remaining warranty and the return policy, and test it under load as soon as it arrives.

Do I need a graphics card at all?

No. Small models run on a CPU with enough system RAM, and Apple silicon Macs and unified-memory mini PCs use their shared memory instead. A dedicated card is the most common route to larger models on a PC.

How we picked

  • We grouped graphics cards by VRAM, the spec that decides which local models fit, instead of naming single cards whose listings and prices change.
  • Model-size fits come from the same estimate as our run-locally calculator (4-bit weights, 8K context, about 75% of unified memory usable by the GPU). Real use varies by runtime, context length and model.
  • Links open Amazon searches by spec, not a single listing, so you compare current models, prices and reviews yourself.
  • We have not tested these products hands-on for this guide and make no claims about speed or how any listing performs.

Related buying guides

More: All buying guides · Black Friday guide · Deals