A frequent topic of discussion for OpenStack administrators is how to properly configure CPU allocation to balance performance and density. This blog post, inspired by a recent internal chat, will walk you through the key configuration options in OpenStack Nova for managing CPU overcommitment, with pointers to the official upstream documentation.
CPU overcommitment allows you to present more virtual CPUs (vCPUs) to your instances than the number of physical CPU cores available on the compute host. This is a common practice in virtualized environments, leveraging the fact that most instances don't use 100% of their assigned CPU all the time.
It's important to remember that in Nova, CPU overcommitment only applies to instances with a shared CPU policy (hw:cpu_policy='shared'). Instances that require dedicated CPUs (hw:cpu_policy='dedicated') get exclusive access to physical cores, and those cores cannot be overcommitted.
For most deployments, you only need to focus on two primary configuration options to manage CPU allocation and overcommitment effectively. These settings are configured in your nova.conf file, typically on each compute node.
This is the foundational setting. It tells Nova which physical CPU cores on the host are available for instances that use shared vCPUs. By defining this set, you can easily reserve specific cores for the host operating system or other critical processes, ensuring they don't have to compete with virtual machine workloads.
The recommended approach is to explicitly list the cores you want Nova to manage. For example, on a host with 16 cores (0-15), you might reserve the first two cores (0 and 1) for the host and let Nova use the rest:
[compute]
cpu_shared_set = 2-15This method is clear and avoids complex calculations. You know exactly which cores are for the host and which are for your instances.
- Upstream Documentation: compute.cpu_shared_set
This setting defines the overcommitment ratio. The default value is often 16.0, meaning you can schedule up to 16 vCPUs for every 1 physical CPU core in the cpu_shared_set however the modern default is set via initial_cpu_allocation_ratio to 4.0.
For instance, if cpu_shared_set includes 14 cores and cpu_allocation_ratio is set to 16.0, the Nova scheduler will report that this host has 14 * 16 = 224 available vCPUs for scheduling.
You can adjust this value based on your specific workloads. If your instances are CPU-intensive, you might lower this ratio. If they are mostly idle, you might increase it to maximize density. In general exceeding 4 is not recommended for most workloads but anything in the range of 1.0-16.0 is accpatble. Exceedign 16.0 is stongly discouraged.
[DEFAULT]
cpu_allocation_ratio = 3.0- Upstream Documentation: DEFAULT.cpu_allocation_ratio
While the two options above will cover 99% of use cases, it's helpful to be aware of a couple of others.
If you prefer to manage the allocation ratio dynamically through the Placement API instead of a static configuration file, you can use initial_cpu_allocation_ratio. This sets the starting value, which you can then adjust on the fly using Placement API calls without restarting the compute service. Most users, however, opt for the simpler static configuration approach with cpu_allocation_ratio.
- Upstream Documentation: DEFAULT.initial_cpu_allocation_ratio
Before cpu_shared_set was introduced, reserved_host_cpus was the primary way to reserve cores for the host. This option simply subtracts a number of cores from the total reported vCPU count after the allocation ratio is applied.
For example, with 10 physical cores, a ratio of 4.0, and reserved_host_cpus = 4, the available vCPUs would be (10 * 4) - 4 = 36. This can be confusing. Using cpu_shared_set is the modern, recommended best practice because it separates host reservation from the overcommitment calculation, making your configuration much more predictable.
- Upstream Documentation: DEFAULT.reserved_host_cpus
To configure CPU allocation in OpenStack Nova, focus on these two steps:
- Isolate host cores: Use
compute.cpu_shared_setto define which physical cores Nova is allowed to use for shared instances. - Set the overcommit level: Use
DEFAULT.cpu_allocation_ratioto specify how many vCPUs can be provisioned per physical core.
By using these two settings, you can create a clear, simple, and effective CPU overcommitment strategy for your OpenStack cloud. For more in-depth information, always refer to the official OpenStack Nova Configuration Guide.