Workload guidance: when to choose GPU or CPU
Use this practical guide to decide whether to request GPU or CPU capacity and to prepare the details we need for a fast, accurate quote.
At a glance
GPUs excel at highly parallel, matrix-heavy workloads such as deep learning training, large-scale fine-tuning and high-throughput inference. CPUs are typically better for general-purpose compute, batch processing, ETL, compiles and workloads that scale horizontally across many smaller cores.
GPU workloads (choose GPU when…)
- You are training deep learning models (transformers, CNNs) or fine-tuning large pretrained models.
- You need low-latency or high-throughput inference (multiple concurrent requests or real-time responses).
- Your workload uses frameworks that leverage CUDA, cuDNN, TensorRT or similar GPU-accelerated libraries.
- You perform GPU-based video processing, rendering or other massively parallel numerical work.
What to include when requesting a GPU quote:
- Target framework and versions (PyTorch, TensorFlow, JAX)
- Preferred GPU model (if known) or outline of performance target (FP32/FP16/INT8)
- Concurrency (number of simultaneous jobs or clients)
- Typical job duration and expected daily/weekly GPU hours
- Data size, storage and transfer requirements
CPU workloads (choose CPU when…)
- Workloads are single-threaded or benefit from many small cores (web services, compilers).
- Batch jobs, ETL, data preprocessing and orchestration tasks where GPUs add little value.
- Your pipeline scales horizontally across many instances rather than relying on GPU acceleration.
What to include when requesting a CPU quote:
- Estimated vCPU count and memory per instance
- Expected concurrency and typical job duration
- Storage type and I/O patterns (large sequential reads vs many small random reads)
- Any special networking or latency requirements
Decision checklist (quick)
- Is the workload parallel and matrix-heavy? —> GPU
- Does it need many small cores or single-threaded performance? —> CPU
- Do you require GPUs for inference latency or throughput? —> GPU
- Is cost per hour and scaling via many instances more important? —> CPU
Example request templates (copy into email or quick-quote)
Include these fields in your request to speed response times:
Workload type: Training / Inference / Rendering / Batch
Frameworks: PyTorch 2.0, CUDA 12
Preferred hardware: 1x A100 / or equivalent performance
Concurrency: 4 simultaneous training jobs
Duration: 8 hours per job, expected 5 days per week
Estimated GPU hours per week: 160
Data: 5 TB object storage, 200 GB active dataset
Region / availability: Australia east, moderate availability
Special requirements: GPU passthrough, persistent volumes
Developer tips & cost-conscious practices
- Use mixed-precision training where safe (FP16/BFloat16) to reduce GPU time and memory.
- Profile before scaling — a small-sample run often reveals bottlenecks.
- Cache datasets on fast local SSDs when repeat training runs occur to reduce cloud egress and I/O wait.
- For inference, consider batching requests or model quantisation to lower compute needs.
Next steps — request a quote or developer access
When ready, request a quote with the template above. For API access and test environments, request developer access.