Engineering Overview: The 6GB Mobile VRAM Reality
The NVIDIA GeForce RTX 3060 Laptop GPU (GA106 silicon) remains one of the most widely owned mobile GPUs in the developer community. Equipped with 6 GB of GDDR6 memory on a relatively wide 192-bit bus delivering 336 GB/s of memory bandwidth, it punches significantly above budget 128-bit cards in raw token throughput.
However, in local AI, 6 GB is a strict hard boundary. In Windows 11, the Desktop Window Manager (DWM) and display driver reserve approximately 0.6 GB to 0.8 GB of VRAM, leaving you with roughly 5.2 GB to 5.4 GB of usable VRAM for Ollama, llama.cpp, or vLLM.
Tier 1: Models That Run 100% in VRAM (Maximum Tok/s)
To achieve maximum generation speed (no PCIe bottleneck), the combined weight matrix, KV cache, and CUDA runtime context must fit completely within your 5.4 GB available VRAM:
- DeepSeek-R1-Distill-Qwen-1.5B (Q8_0): Consumes ~2.2 GB VRAM total with an 8K context. Delivers blazing-fast chain-of-thought reasoning at 95 – 125 tokens/sec. Perfect for real-time code completion in VS Code (via Continue.dev) or inline chat.
- Llama 3.2 3B Instruct (Q8_0 or Q4_K_M): At Q4_K_M, the model requires ~2.8 GB total. Runs at 70 – 95 tokens/sec. Ideal for fast document summarization, email drafting, and multi-turn conversational agents.
- Qwen 2.5 Coder 7B (Q4_K_M with 4K context): The absolute coding champion that fits on a 6GB card! The weights take ~4.4 GB. With a 4,096 context window and FlashAttention enabled, total VRAM consumption is ~5.2 GB. It runs at 42 – 55 tokens/sec.
- DeepSeek-R1-Distill-Qwen-7B (Q4_K_M): Reasoning model fitting tightly in ~5.3 GB VRAM. Runs at 38 – 50 tokens/sec. Keep context length under 4,096 tokens to prevent driver out-of-memory paging.
Can an RTX 3060 Laptop Run 14B Models (DeepSeek R1 14B or Qwen 14B)?
Yes, but with Partial CPU RAM Offloading. A 14B model quantized at Q4_K_M requires approximately 9.2 GB of memory. Since your GPU has 6 GB, Ollama / llama.cpp will offload approximately 22 out of 48 layers to your GPU, while the remaining 26 layers are processed by your laptop's CPU and system RAM.
| Model | Quantization | VRAM Split | System RAM | Tok/s (Generation) | Experience |
|---|---|---|---|---|---|
| Qwen 2.5 Coder 7B | Q4_K_M | 5.2 GB (100% GPU) | 0 GB | 48 tok/s | Instant, fluid coding |
| DeepSeek R1 7B | Q4_K_M | 5.3 GB (100% GPU) | 0 GB | 44 tok/s | Fast reasoning thoughts |
| Qwen 2.5 Coder 14B | Q4_K_M | 5.5 GB (55% GPU) | 4.5 GB | 12 – 16 tok/s | Usable, moderate pause |
| DeepSeek R1 14B | Q3_K_M | 5.6 GB (60% GPU) | 3.2 GB | 14 – 18 tok/s | High intelligence, slower |
Optimized Ollama Configuration for 6GB Laptops
To prevent Windows from throwing Out of Memory errors on 6GB cards, set these environment variables before starting Ollama:
# Enable FlashAttention to cut KV cache memory by 40%
set OLLAMA_FLASH_ATTENTION=1
# Prevent concurrent model loads from eating VRAM
set OLLAMA_NUM_PARALLEL=1
# Run your chosen 7B coder
ollama run qwen2.5-coder:7b