Table of Contents
By Khimananda Oli | Last reviewed: August 2026
Data center GPUs like the A100 and H100 are often too large for individual inference or development tasks, leading to massive resource waste when a single user occupies an entire card. Using MIG: Partition NVIDIA GPUs allows you to split a single physical accelerator into up to seven fully isolated instances, each with dedicated compute, memory, and cache. This guide provides the exact configuration steps, profile selection logic, and Kubernetes integration patterns needed to implement GPU partitioning safely in production. Before enabling this feature, review your workload characteristics as outlined in GPUs for AI: what developers need to know to ensure partitioning aligns with your performance requirements.
nvidia-smi -mig 1, reboot the host, create GPU instances using predefined profiles (e.g., 1g.5gb), and expose them to containers via the NVIDIA Container Toolkit or Kubernetes device plugin. Each partition provides hardware-isolated compute, memory, and L2 cache for secure multi-tenant workloads.How does MIG: Partition NVIDIA GPUs actually work?
Multi-Instance GPU (MIG) is a hardware-level virtualization technology available on NVIDIA Ampere (A100, A30) and Hopper (H100, H200) architectures. Unlike software-based GPU sharing such as MPS or time-slicing, MIG enforces isolation at the silicon level. When you use MIG: Partition NVIDIA GPUs, the GPU’s streaming multiprocessors (SMs), L2 cache, and DRAM bandwidth are physically divided into distinct slices that cannot interfere with each other.
Each GPU Instance (GI) contains a fixed allocation of SMs and a dedicated portion of L2 cache. Within each GI, you can further create Compute Instances (CI) that share the GI’s resources but have separate scheduling contexts. Memory is allocated in fixed granules—typically 5 GB, 10 GB, 20 GB, or 40 GB on an 80 GB A100—and is backed by dedicated ECC-protected VRAM regions. This means a noisy neighbor in one partition cannot cause memory corruption, cache thrashing, or latency spikes in another partition.
The critical distinction from virtualization approaches like vGPU is that MIG does not require a hypervisor or license server. It operates entirely within the GPU firmware and driver stack, making it suitable for bare-metal Kubernetes nodes, containerized training clusters, and compliance-sensitive environments where hypervisor overhead is unacceptable. For teams managing database workloads alongside AI inference on shared infrastructure, understanding these isolation boundaries is as important as PostgreSQL administration essentials for preventing cross-workload interference.
Which MIG profiles should you choose for different workloads?
Selecting the right profile is where most MIG deployments fail. Profiles are denoted as Xg.Ygb, where X represents the number of GPU slices (compute units) and Y represents the memory allocation in gigabytes. On an 80 GB A100, common profiles include:
- 1g.5gb: 1/7th compute, 5 GB VRAM. Ideal for lightweight inference, tokenization, or CI/CD validation jobs.
- 1g.10gb: 1/7th compute, 10 GB VRAM. Suitable for medium-sized models (e.g., BERT-base, ResNet-50) or vector embedding generation.
- 2g.20gb: 2/7ths compute, 20 GB VRAM. Good balance for fine-tuning smaller LLMs or running RAG pipelines with moderate context windows.
- 3g.40gb: 3/7ths compute, 40 GB VRAM. Appropriate for larger inference workloads or batch processing with high memory pressure.
- 7g.80gb: Full GPU. Use when a single workload requires all resources; disables MIG benefits but maintains compatibility.
A common mistake is over-provisioning memory while under-utilizing compute. If your workload is compute-bound (e.g., matrix multiplication in transformers), prioritize profiles with more GPU slices even if memory seems tight. Conversely, for retrieval-augmented generation where KV-cache dominates VRAM usage, favor higher memory allocations. Always benchmark with realistic payloads before committing to a profile topology. Teams evaluating cost trade-offs between dedicated and shared GPUs should also consult rent vs buy GPUs for AI workloads to align partitioning strategy with financial models.
| Profile | Compute Slices | Memory (A100 80GB) | Best For | Avoid When |
|---|---|---|---|---|
| 1g.5gb | 1/7 | 5 GB | Inference <2B params, preprocessing | Training or long-context LLMs |
| 1g.10gb | 1/7 | 10 GB | BERT, embeddings, small RAG | Models >7B parameters |
| 2g.20gb | 2/7 | 20 GB | Fine-tuning 3B–7B models | High-throughput serving |
| 3g.40gb | 3/7 | 40 GB | 13B inference, batch ETL | Latency-critical real-time apps |
| 7g.80gb | Full | 80 GB | Training, 70B+ models | Multi-tenant environments |
How do you configure and enable MIG on Linux hosts?
Enabling MIG requires administrative access and a host reboot. The process is identical across Ubuntu, RHEL, and SUSE systems with NVIDIA drivers ≥470. Follow these steps precisely:
- Verify GPU support: Run
nvidia-smi --query-gpu=name,mig.mode.current --format=csv. Only A100/A30/H100/H200 will show MIG capability. - Enable MIG mode: Execute
sudo nvidia-smi -i 0 -mig 1for GPU index 0. Repeat for each GPU or use-i 0,1,2,3for multiple cards. - Reboot the host: MIG mode changes require a full system restart.
sudo reboot. - Create GPU instances: After reboot, list available profiles with
nvidia-smi mig -lgip. Create instances usingnvidia-smi mig -cgi 9,9,9 -C(creates three 1g.10gb instances on GPU 0). - Persist configuration: Add instance creation commands to a systemd service or cloud-init script to survive reboots. Never rely on manual recreation in production.
# Example: Create mixed MIG topology on A100 80GB
sudo nvidia-smi -i 0 -mig 1
sudo reboot
# After reboot: create 2x 1g.10gb + 1x 2g.20gb
sudo nvidia-smi mig -i 0 -cgi 9,9,14 -C
# Verify created instances
nvidia-smi mig -lgiNote that MIG configuration is destructive: creating new instances deletes existing ones. Always export your current topology with nvidia-smi mig -lgii before making changes. In automated environments, store desired state in Git and reconcile via Ansible or Terraform rather than imperative scripts.
How do you integrate MIG with Kubernetes and containers?
Kubernetes does not natively understand MIG partitions. You must use the NVIDIA GPU Operator with MIG support enabled. The operator deploys the device plugin, validates MIG configuration, and exposes partitions as schedulable resources like nvidia.com/mig-1g.10gb.
Install the operator via Helm with MIG strategy set to "mixed" or "single":
helm install gpu-operator nvidia/gpu-operator \
--namespace gpu-operator --create-namespace \
--set mig.strategy=mixed \
--set mig.partitions[0].name=1g.10gb \
--set mig.partitions[0].count=3 \
--set mig.partitions[1].name=2g.20gb \
--set mig.partitions[1].count=1In your pod spec, request specific MIG devices instead of whole GPUs:
resources:
limits:
nvidia.com/mig-1g.10gb: 1
memory: 8Gi
requests:
nvidia.com/mig-1g.10gb: 1
memory: 8GiCritical considerations for Kubernetes integration:
- Node labeling: Label nodes with their MIG topology so pods land on correctly partitioned hardware. The operator auto-labels, but verify with
kubectl get nodes -l nvidia.com/mig.config=all-disabled. - DaemonSet tolerance: Ensure monitoring agents (Prometheus node-exporter, DCGM) tolerate MIG taints and query per-instance metrics via
DCGM_FI_DEV_GPU_UTIL{gpu_instance_id="X"}. - Security context: MIG partitions inherit host-level GPU access controls. Use Pod Security Standards to prevent unauthorized privilege escalation. Never run untrusted code without additional sandboxing beyond MIG.
For teams already running observability stacks, integrating MIG metrics follows the same patterns described in Prometheus metrics monitoring fundamentals—just add DCGM exporter targets and adjust recording rules for per-instance aggregation.
What are the limitations and operational risks of MIG?
MIG is powerful but not universal. Understanding its constraints prevents costly misconfigurations:
- No dynamic resizing: Changing partition topology requires deleting all existing instances and recreating them. This is disruptive and cannot be done live. Plan capacity upfront.
- Limited profile combinations: Not all profile mixes are valid. An A100 80GB supports exactly seven 1g.5gb OR three 2g.20gb + one 1g.10gb, etc. Consult NVIDIA’s official profile matrix before designing topologies.
- No inter-partition communication: MIG slices cannot directly share memory or signal each other. Multi-GPU training across MIG partitions is impossible; use NCCL only within a single GI or across physical GPUs.
- Driver/firmware coupling: MIG behavior changes between driver branches. Pin driver versions and test upgrades in staging. Driver 535+ improved H100 MIG stability significantly over 520.
- Monitoring gaps: Standard
nvidia-smishows aggregate stats. Per-instance telemetry requires DCGM ≥3.3. Without it, you’re flying blind on utilization and thermal throttling per slice.
In my experience helping Nepal-based AI startups optimize cloud spend, the biggest pitfall is treating MIG as a cost-saving panacea without modeling actual workload concurrency. If your inference traffic is bursty and unpredictable, time-slicing with fractional GPUs may outperform static MIG partitions. Always measure before optimizing.
Implementing MIG: Partition NVIDIA GPUs in production
Successfully deploying MIG: Partition NVIDIA GPUs requires treating GPU topology as immutable infrastructure. Define your desired partition layout in version control, automate reconciliation via IaC, and validate with synthetic benchmarks before admitting user workloads. Monitor per-instance utilization through DCGM and set alerts for underutilized partitions—they indicate either over-provisioning or scheduling failures. Remember that MIG solves a specific problem: secure, predictable multi-tenancy on expensive accelerators. It is not a substitute for proper capacity planning, workload profiling, or architectural review. If your team needs help designing GPU infrastructure that balances cost, security, and performance, reach out to discuss your specific requirements.
