https://easontechtalk.com/aws-eks-cni-plugin-ip-allocation-deep-dive/
Prometheus for AWS EKS CNI
- awscni_assigned_ip_addresses (IPs actually handed to pods)
- awscni_total_ip_addresses (IPs attached to ENIs, including warm/reserved)
- awscni_ip_max (per-node hard ceiling)
- awscni_no_available_ip_addresses (counter of failed pod assignments)
Explaination
- In the AWS VPC CNI model each node’s pods consume secondary private IPv4 addresses on the node’s ENIs (network cards). The IPAM daemon (aws-node) manages that pool:
- whenever the node needs more pod IPs it first “pre-provisions” them by attaching an ENI (if necessary) and assigning extra secondary IPs to that ENI—this is the warm pool. How aggressively it pre-provisions is driven by environment variables like WARM_ENI_TARGET and WARM_IP_TARGET.
- those extra IPs exist on the ENI but are not yet bound to any pod. They are simply reserved, ready for the next pod that schedules.
- when a pod starts, IPAMD picks one of those already-attached addresses and assigns it to the pod. Only then does the CNI plugin wire the pod’s network namespace to that IP.
The metric awscni_total_ip_addresses counts every secondary IP that is currently attached to the instance’s ENIs—both the ones already in use by pods and the unused warm-pool addresses. awscni_assigned_ip_addresses counts only the subset currently bound to pods. Therefore, the difference total – assigned tells you how many warm/reserved IPs are sitting on the ENIs waiting to be used.
