33 Responses

  1. AlphenIT
    AlphenIT at |

    Interesting piece Chris!
    Citrix in the past also has done some writing on this and if I compare your key takeaway:
    Avoid creating a VM with more vCPUs than PHYSICAL cores.
    With theirs:
    It is advisable not to allocate more vCPU than there are LOGICAL cores within the given hardware.
    I’m getting even more confused..maybe I’m crossing logical with physical but confusing it remains (as Yoda would say).
    I’m referring to these articles:
    Any thoughts?

  2. @vmMarkA
    @vmMarkA at |

    Consider the advantage of parallelism.

    [corner case]

    I agree with Chris that I often size to the physical core count to be conservative and ensure a solid performance profile. That said, sometimes there is an advantage of offering more vCPUs, and therefore more parallel threads, to an application – IF – you can be reasonably assured the threads don’t saturate the underlying physical execution resource. This type of configuration allows the advantage of many threads to run in a short and/or bursty type pattern which can benefit some workloads. In this case I use the word “evaluate” to see if you can improve application KPI’s by using more logical CPUs than physical CPUs.

  3. Chris Conlan
    Chris Conlan at |

    I have noticed since moving to Intel i7-4770 (Quad Core w/ HT) from an AMD FX-8320 (8 core), my CPU is at like 70% for all 4 cores. Does the % take into affect HT?

    1. Josh Townsend
      Josh Townsend at |

      Check out VMware KB 2055995 (http://kb.vmware.com/kb/2055995): Difference between cpu.usage and cpu.utilization counters for HostSystem object.

      “The counters for the HostSystem object provide this information:

      cpu.utilization – Provides statistics for physical CPUs.
      cpu.usage – Provides statistics for logical CPUs. This is based on CPU Hyperthreading.”

  4. Welcome to vSphere-land! » CPU Links

    […] Hyper-Threading Gotcha with Virtual Machine vCPU Sizing (Chris Wahl) VMware vSphere 4: The CPU Scheduler in VMware ESX 4 (VMware Tech Paper) The CPU Scheduler in VMware ESX 4.1 (VMware Tech Paper) The CPU Scheduler in VMware vSphere  5.1 (VMware Tech Paper) Virtual Machine Monitor Execution Modes: in VMware vSphere 4.0 (VMware) Does corespersocket Affect Performance? (VMware vSphere Blog) Four Things You Should Know About ESX 4’s Scheduler (Virtual Performance) […]

  5. REZ
    REZ at |

    Is this “best practice” per Socket or per core(s)? For instance, I get that in a 1 socket 6 core processor a VM should not have more that 6 vCPUs. That makes sense to me. But what about a host with 2 socket, 6 core processor (12 cores total). Does this mean that my “monster VM” can have 12 vCPUs? Does the scheduler span processes across cores?

    Or will it schedule 6 threads…then wait for the other 6 threads from the other 6 vCPUs on the other socket?

    1. REZ
      REZ at |

      I wrote:

      “Does the scheduler span processes across cores?”

      I meant:

      Does the scheduler span processes across sockets? lol


  6. faf1967
    faf1967 at |

    I have a ESX 5.1 cluster running a windows 2008 SQL cluster and a Windows 2012 SQL cluster. We will have 2 (2 node 2008) and 2 (2 node 2012) clusters. Should I turn on hyper-threading?

    Thanks in advance for any responses.

  7. Cheap and green ESXi homelab – Part I | brunier.nl

    […] deliver that big an advantage, especially in my lab environment. Nice piece on this subject: http://wahlnetwork.com/2013/09/30/hyper-threading-gotcha-virtual-machine-vcpu-sizing/ I decided to buy the Intel i5 4440 processor. A common CPU, on-cpu graphics, well […]

  8. Amritanshu Ranjan
    Amritanshu Ranjan at |

    Hi Chris,
    I am a newbie and your article is a relief
    Central Idea is hyper threading is helpful only if I create 4 vms and assign a core each.

    AMD FX 8350 says that they have 8 cores (some post suggests 4 strong and 4 weak cores)
    i5 4670k says they 4 cores with hyper threading option.

    Which one should I go for if I need a strong processor to run multiple VMs..
    Can I go AMD and run more than 4 VMs simultaneously?

  9. Nested lab on for a few days gives error?

    […] PM #25 Hyper-Threading Gotcha with Virtual Machine vCPU Sizing | Wahl Network Originally Posted by ChooseLife Congratulations, you have passed the necessary […]

  10. TtG
    TtG at |

    This article made me doubt that it is a good idea to allocate 12 cores to a virtual machine on a 6-core processor with hyper-threading. Thankfully I did not trust it blindly and made some tests.
    Use case: I have 3 VMs, 1 is router/firewall (pfSense), 1 is NAS storage (Solaris), 1 is Win7 x64 sometimes doing CPU-intensive tasks, such as video encoding. I allocate 12 cores to the Win7 VM and 1 core to the other VMs. I measured how fast I can encode videos and here are my test results:

    a) bare-metal Win7 x64 installation: 7.93 FPS
    b) VM under ESXi with 12 cores allocated: 7.71 FPS
    c) VM under ESXi with 6 cores allocated: 6.28 FPS
    This is on Xeon 1650v3 CPU (6 cores with hyper-threading). Higher FPS is better, of course.

    Conclusion: best performance is achieved when I allocate all logical cores to the CPU-demanding VM.

  11. Aroop
    Aroop at |

    Wonderful insight, just what i needed.

  12. When CPU metrics with Hyperthreading, Monster VMs and VMware Make No Sense | Blue Shift

    […] In this example, our 2 socket, 8 core system with 16 cores total now presents 32 cores to VMware. Chris Wahl has an excellent post on this topic which I strongly encourage you to read, but for now I’m just going to “borrow” […]

  13. Michel Gokan (Kogan)
    Michel Gokan (Kogan) at |

    In case of Xen, I normally pin VCPUs to DomUs to avoid internal CPU scheduling as much as possbile. In real production environments, isn’t it better to just pin each VCPU to a physical core ? I know this might reduce the number of VMs that can be run on a single machine but I think its more stable.

  14. Lost
    Lost at |

    Hi Chris,

    Would you recommend Intel Xeon E5-2660 v3, which has 20 HT, or would you go with something like Intel Xeon E5-4627 v3 for a virtual environment? Each host would have about 15 Servers running 2008/2012 R2, and about 8GB of ram. I am in the process of upgrading to a Cisco UCS B420 host, and trying to decide on CPU’s.

  15. Somesh
    Somesh at |

    I got a host that has a 12 cores, so according to you i have a total of 24 VCPU. I have 6 VM running on that server which requires a total of 19 cores but after allocating required cores to the VM i ran a prerequisite tool and i got i the following message “Action: The number of CPU cores configured for the virtual machines in the inventory exceeds the number of CPU cores available to the host. Configure virtual machines with appropriate capacity”. This is a Hyper V environment

  16. Cheap and green ESXi homelab – Part I

    […] deliver that big an advantage, especially in my lab environment. Nice piece on this subject: http://wahlnetwork.com/2013/09/30/hyper-threading-gotcha-virtual-machine-vcpu-sizing/ I decided to buy the Intel i5 4440 processor. A common CPU, on-cpu graphics, well […]

Share your point of view!