Weights are the starting point
Approximate weights as parameters × bytes per parameter. Seven billion parameters in FP16 require about 14 decimal GB for weights alone, not the complete runtime.
TECHNICAL GUIDE
How Much GPU Memory Do You Need?
Start with weights, then account for context, concurrency and runtime overhead.
Hardware and application reference imageryUNDERSTAND THE TRADE-OFFS
Work through the technical requirements
Quantisation has overhead
INT8/INT4 reduce weight size but require scales, metadata and runtime resources. Validate quality and operator support.
Context & concurrency
KV cache varies with architecture, context and request batches. The same model can require very different memory under different serving conditions.
Converge with tests
Fix model, precision and request distribution; measure peak memory, first-token latency and throughput, then retain headroom.
What do we need next?
Applications and models, data scale, user or node counts, latency and throughput goals, existing resources, budget and site constraints. Existing measurements help narrow the shortlist.
Turn requirements into a reviewable plan
Discuss selection, bills of materials, samples and procurement on a project basis. Model availability, implementation, installation and validation scope require agreement after requirements review.
Discuss your technical requirements ↗LET’S BUILD YOUR NEXT CONFIGURATION
Discuss your requirements ↗