Local NVMe Storage¶

Many instance types ship with NVMe disks physically attached to the host. These are far faster than network-attached volumes (EBS, Persistent Disk, Azure Disk) and are the right backing store for scratch space, caches, spill files, and analytical workloads that re-read large working sets.

How you get at them differs by cloud, because the clouds model them differently:

AWS and Azure: Automatic, No Configuration¶

On AWS and Azure, the amount of local NVMe is fixed by the instance type — you cannot ask for more or less of it. So there is nothing to configure, and no plan-spec field. Select an instance type that has local NVMe and Omnistrate prepares it for you.

At node bootstrap, Omnistrate:

1. finds the local NVMe devices, ignoring the OS disk and any attached data disks
2. combines them into a single RAID 0 array when there is more than one
3. formats it and mounts it, relocating the kubelet root onto it

Because the kubelet root is relocated, this happens automatically for anything kubelet places there:

• emptyDir volumes
• hostPath volumes under the kubelet directory
• dynamically provisioned local volumes, such as Local Path Provisioner with the default nodePathMap of /var/lib/kubelet/local-path-provisioner

No storage class, no volume annotation, no extra flag. A pod writing to an emptyDir lands on the RAID array.

Choosing an Instance Type with Local NVMe¶

AWS — the families whose name carries a d, plus the storage-optimized families:

• m6id, m7gd, c6id, c7gd, r6id, r7gd and similar d-suffixed variants
• i3, i3en, i4i, im4gn, is4gen storage-optimized families

To check a specific type:

aws ec2 describe-instance-types --instance-types m6id.8xlarge \
  --query 'InstanceTypes[].InstanceStorageInfo'

A non-empty result with "NvmeSupport": "required" means it has local NVMe.

Azure — the families whose name carries a d from v6 onward, the storage-optimized L-series, and several HPC and GPU families:

• v6/v7 d families: Ddsv6, Edsv6, Ddsv7, Edsv7, Eadsv7, Dldsv6, Dpdsv6, Epdsv6, Fadsv7 and related variants
• L-series: Lsv2, Lsv3, Lasv3, Lsv4, Lasv4, Laosv4
• FXmdsv2, HBv4, HX
• NVMe-backed GPU families such as NCadsH100v5 and NCADSA100v4

To check a specific SKU:

az vm list-skus --location eastus2 --size Standard_E32ds_v6 -o json \
  | jq -r '.[0].capabilities[] | select(.name | startswith("Nvme"))'

A non-zero NvmeDiskSizeInMiB means it has local NVMe. NvmeDiskSizeInMiB divided by NvmeSizePerDiskInMiB gives the number of disks.

Azure Specifics¶

By default AKS places the ephemeral OS disk on the local NVMe, which consumes an entire disk — on single-disk SKUs such as Standard_E2ds_v6 that leaves nothing for your workload. Omnistrate therefore provisions local-NVMe node pools with a managed OS disk, so the full local NVMe capacity is available to you.

GCP: Configure Local SSD Explicitly¶

GCP is different because local SSD is an attached resource — for many machine families you choose how many disks to attach. Since there is a number to specify, it has to be configured, using configurationOverrides on the instance type.

There are two fields, for the two ways GCP exposes local SSD.

ephemeralStorageLocalSsdConfig¶

Backs the node's ephemeral storage with local SSD, so emptyDir volumes and container image layers land on it. This is the closest equivalent to the AWS and Azure behavior above.

At least one of the two must be set, and any value given must be greater than zero.

x-omnistrate-compose-spec:
  services:
    warehouse:
      x-omnistrate-compute:
        instanceTypes:
          - name: n2-standard-8
            cloudProvider: gcp
            configurationOverrides:
              ephemeralStorageLocalSsdConfig:
                localSsdCount: 2

localNvmeSsdBlockConfig¶

Attaches local SSDs as raw block devices, left unformatted for the workload to manage itself. Use this when your application wants to own the device directly.

localSsdCount is required and must be greater than zero.

services:
  - name: warehouse
    compute:
      instanceTypes:
        - name: n2-standard-8
          cloudProvider: gcp
          configurationOverrides:
            localNvmeSsdBlockConfig:
              localSsdCount: 1

The number of local SSDs you can attach depends on the machine type and zone. See Google's local SSD documentation for the limits.

These Fields Are GCP-Only¶

Both fields are rejected at spec import for any other cloud provider:

ephemeral storage local SSD configuration is not supported for AWS
local NVMe SSD block configuration is not supported for Azure

This is intentional, not a gap. On AWS and Azure local NVMe is intrinsic to the instance type and there is no count to choose, so the equivalent behavior is automatic — see AWS and Azure: Automatic, No Configuration above.

Verifying It Works¶

On a node you expect to have local NVMe, check that the kubelet root is on the array rather than the OS disk:

kubectl debug node/<node> -it --image=busybox -- \
  nsenter -t 1 -m -- df -h /var/lib/kubelet

From inside a pod, check where its volume actually lives:

kubectl exec <pod> -- df -h /path/to/volume

If the mount points at the OS disk (/dev/root on Azure, the root device on AWS), the instance type most likely has no local NVMe. Confirm with the aws or az command in Choosing an Instance Type with Local NVMe.

Related Guides¶

• Compute Management — selecting instance types per cloud provider
• Persistent Volumes — durable storage that survives node replacement
• Storage Classes — pre-provisioned classes for network-attached storage