Hyper-Threading (HT) is sometimes viewed as the mystical voodoo that enables you to have twice as many logical cores on your physical CPU. Occasionally, the voodoo is abused when sizing the quantity of virtual machine vCPUs for a workload. After all – if the vSphere host says I have 8 logical processors available to me, can’t I use all of them for a single VM and its vCPUs? Hyper-Threading definitely has its advantages, but to properly use the technology in your vSphere host, you must understand a little of the basics.
VMware’s Mark Achtemichuk has also weighed in on this subject in his When to Overcommit vCPU:pCPU for Monster VMs post.
Hyper-Threading in a Nutshell
Without going too deep into the weeds, the main point of HT is to allow two different compute tasks (threads) to schedule execution resources on the same underlying physical core. There are often periods of time where a physical core is underutilized because a compute task is waiting on a device. Other tasks would be blocked from executing on the physical core until the running task completed. HT allows the scheduler to assign two tasks to a physical core, effectively sharing the execution resource.
Key takeaway: Each physical core contains a single execution resource. HT allows for greater scheduling efficiency, but cannot speed up a single task.
Over Sizing the vCPUs in a Virtual Machine
A vSphere host is smart enough to identify an Intel CPU that can perform Hyper-Threading. It will display the quantity of physical sockets, physical cores per physical socket, and logical processors. If your server does not have HT, the quantity of physical cores and quantity of logical processors will be equal. When HT is enabled, the quantity of logical processors will be double that of the physical cores.
You may have a desire to create a virtual machine that can consume all of your logical processors. You will not encounter an error if you try to do this, but it is ill advised. Let’s look at why.
In this example, I’ll use one of my home lab servers. Each server has an Intel Xeon E3-1230 processor with four physical cores and HT enabled. This gives me 8 logical processors. If I create a virtual machine with 8 vCPUs, it will look a bit like this:
Putting aside co-stop and other scheduling concerns with having a single VM consume my entire CPU, there are also issues with running tasks inside of my virtual machine. The VM believes that it has access to 8 different CPU cores. In reality, the vCPUs are sharing sibling Hyper-Threads (two Hyper-Threads on the same underlying core), which is something the CPU Scheduler tries to avoid.
What if the VM guest tries to run 8 tasks that require 100% of the execution resources from the cores? There are only 4 execution resources available. Thus, this would cause a great deal of CPU Ready because the VM would be ready to consume CPU time on 8 cores but only 4 are actually available. If you looked at ESXTOP, you would most likely notice that each HT was at 50% UTIL, and each PCPU (physical core) would be at 100% UTIL.
If I had two physical sockets of the configuration shown above, the CPU Scheduler would likely place 4 of the vCPUs on CPU 0 and the other 4 vCPUs on CPU 1. This would avoid any need to schedule vCPUs on sibling Hyper-Threads, but may cause other headaches elsewhere with NUMA node locality, co-stop, and workload contention.
Key takeaway: Avoid creating a single VM with more vCPUs than physical cores. Having multiple VMs that, when combined, exceed the number of physical cores is typically OK – this is called overcommitment.
Thoughts
Hyper-Threading offers a great deal of scheduling flexibility in almost all virtualization scenarios. It’s perfect for sharing the physical execution resources available with a large quantity of virtual machines. Just be careful that you don’t abuse the logical processors by creating a virtual machine that has more vCPUs than there are physical cores available. Always try to keep the quantity of vCPUs set to a size that is needed by the virtual machine, and if you must grow to a size that is larger than your physical core count, it’s time to either buy a bigger host or split up the roles serviced by the virtual machine.
