121 Responses

  1. Sean Duffy
    Sean Duffy at |

    Nice post Chris! I love reading about different Whitebox builds. I will also take this route in the near future as I am tired of running 2-3 x ESXi nested Hosts in Workstation with only 16GB of main PC host RAM to play with! I have been using SSD in my system for VM Datastores and must say it makes such a difference compared to my older SATA drives that were running datastores from various FreeNAS/OpenFiler/HP VSA VMs!

    Reply
  2. @dboftlp
    @dboftlp at |

    Hey, nice write up. I built out my initial vLab on his initial Baby Dragon v1. and just purchased a memory upgrade to 32GB (and I’m impatiently waiting on Provantage to get it here!). At any rate, you mentioned the cost of RAM and I just have to put in my 2 cents. My initial build was for 16GB and when I went back and looked, the 32GB upgrade cost me about $60 less! I jumped on it and I’m thinking the single 32GB host is going to be my Inception box for some time (vCloud design in hand!). I’m also rocking a Synology DS1010+ I picked up open box from Newegg for an absolute steal and have been so impressed with it, it’s seriously a killer NAS. You’ll love the SuperMicro boards, they are awesome!
    Enjoy!

    @dboftlp

    Reply
  3. Keith
    Keith at |

    Nice setup! I’m building out 2 x Shuttle SH67H3 systems w/ Core i7-2600, 16GB, USB Boot but still need to choose a NAS appliance. I also want to go back and upgrade them to 32GB… I’m currently running FREENAS in a VM via workstation. Synology, QNAP, DROBO…choices, choices..

    Reply
  4. rsts11: Building my compact VMware server at home « rsts11 – Robert Novak on system administration

    [...] friend Chris Wahl just put together his home whitebox systems. He went a bit more serverward, and he’s going with a pre-constructed NAS from Synology [...]

  5. rsts11
    rsts11 at |

    I ended up going with the Sh67H3 because of the 32GB RAM (which ended up being fairly cheap, $60/8gb stick)… but it’s not ECC, and the remote management card is going to cost me an extra $100 probably. So yeah, server grade and desktop grade can have their handoffs. :)

    Looks like Komputerbay doesn’t have DDR3 ECC non-registered RAM, or I’d suggest giving them a try through Amazon. And best price I could find was $130ish too, a hardforum link.

    Reply
  6. Review: Supermicro ESXi 5 Whitebox Home Lab Servers « Wahl Network

    [...] Review: Supermicro ESXi 5 Whitebox Home Lab Servers Mar 24, 2012 Now that my new vSphere lab is up and running, I wanted to write up a brief review on the Supermicro servers running ESXi 5. For details on the design and links to parts, review my previous post on the home lab. [...]

  7. benoit
    benoit at |

    hi

    does the config work with FT and vt-d etc.

    thanks

    Reply
  8. Arturo
    Arturo at |

    is there a particular reason why you didn’t buy 1X http://www.newegg.com/Product/Product.aspx?Item=N82E16820139077 and 1X of http://www.newegg.com/Product/Product.aspx?Item=N82E16820139268 to save $6 or is it just not recommended to mix and match.

    Reply
  9. jasemccarty
    jasemccarty at |

    Chris,

    Good setup. It is very similar to my setup: http://www.jasemccarty.com/blog/?p=1516.

    I chose the TYAN S5510GM3NR to overcome the multi-nic driver situation… Other than the nic difference, I would have likely picked the Super Micro board.

    Also have to agree with you that the IPMI/iKVM are essential to have, especially if you have to get into it remotely. I keep my lab at a Co-Lo site.

    Great info,
    Jase

    Reply
  10. Aaron
    Aaron at |

    Hello Chris,

    I am also doing a Baby Dragon II system, everything is exactly what you have used except that I went for this USB stick (http://www.newegg.com/Product/Product.aspx?Item=N82E16820235062) because my OCD just couldn’t handle the cap on the Lexar lying around indefinitely for the rest of its life. My build wont be as much for vmware experimentation, but more for home use. As such, I will only be building one server and it will be running a 2008R2 and WHS 2011 and a few other vms for media services. I included two Intel E1G42ETBLK cards (http://www.newegg.com/Product/Product.aspx?Item=N82E16833106034). One dedicated to iSCSI MPIO and the other for vm traffic. The onboard NIC is the WAN connection for my virtual pfSense machine. I will be booting the server up for the first time this evening.

    I am using a Synology DS1512+ which is amazing so far, very excited to see its iSCSI performance.

    Thanks again for the great write up!

    Reply
    1. jasemccarty
      jasemccarty at |

      I looked at those cards… But couldn’t stomach $150 each for them. I went with 2 of these: http://www.newegg.com/Product/Product.aspx?Item=N82E16833106033 at $29 each. They work just as well.

      Cheers,
      Jase

      Reply
      1. Aaron
        Aaron at |

        I looked at those exact cards but a colleague told me he wasn’t able to get vmware to recognize them! I guess I’d better check more sources next time…

      2. Aaron
        Aaron at |

        Those HP cards were some nice freebies!

  11. Chris de Groot
    Chris de Groot at |

    Thanks for all your posts and all included information. I finnally got the right information to buy a VMware Whitebox Home Lab. The update on “VMware Site Survey” is pretty nice. Can’t wait till all the parts arrive. I haven’t decided yet on the NAS/SAN to use. My main optie is a I3-530/8GB FreeNAS ZFS-based array of three 1,5TB 5400rpm disks. Got some advice?

    Cheers!

    Chris

    The Netherlands

    Reply
  12. ITforMe
    ITforMe at |

    FYI for those using this SuperMicro board. It seems to be pretty common that many add-in PCI NICs are not recognized at all by ESXi. This can be fixed by downgrading the BIOS from 1.1a to 1.0.c. I had this occur with a couple of dual port cards on this motherboard, and after performing the step that worked for others, I can confirm that it worked for me as well.

    Reply
  13. Norbs
    Norbs at |

    Great job, I just set up an Ivy Bridge i7 system using a MSI-Z77MA-G45 board. So far everything is working great and VT-D is supported.

    Reply
  14. Bulding an ESXi Home Server
    Bulding an ESXi Home Server at |

    [...] [...]

  15. voom
    voom at |

    @Norbs. Can you describe the parts you used for your Ivy Bridge I7 system? Please list.

    Great write up Chris.

    Reply
  16. Sate
    Sate at |

    Hi Guys,

    Help please: I’m planning to make a dual boot server (Win2008R2 + ESXi 5.0).

    I would like to have the 2tb internal drive as a data store for VMs.
    QUESTION:Can I open those VMs with the VMWare Workstation 8 when booting with the Windows 2008 server?

    If not, what other solution can I use to have access to those VM from ESXi and VMW Workstation 8 without having to convert them.

    Cheers in advance!!!
    scan I use the biggest disk to store VM and open them from the ESXi and from VMWare workstation?

    Reply
  17. ESX_Student
    ESX_Student at |

    Anyone know if all the PCI slots work properly with add-in cards with the latest 2.0a BIOS update? Looking at building a lab with this board and while I’d like to use Ivy bridge since it’s out also want the most compatibility and stability. Thanks.

    Reply
  18. Mel
    Mel at |

    Chris,
    Have you found a financially viable option for 8GB sticks of ECC UDIMMs yet?

    Reply
  19. Three Labs for three reasons « A glimpse into the life of IT

    [...] My physical hosts (qty. 2) most closely reflect the setup that Chris Wahl has described in his home lab.  The main differences are: 1.) I dumped 32GB of RAM in each host, and 2.) I threw in two dual [...]

  20. Angelo
    Angelo at |

    Interesting enough I purchased everything you listed Chris (x2) and neither system will POST, no beeps nothing. (Looked around and I have seen that if you have an older BIOS version it may cause this issue).

    Reply
  21. Angelo
    Angelo at |

    LOL Didn’t even catch that but the SOB should still post unless this board requires mem to boot.

    Reply
    1. ESX_Student
      ESX_Student at |

      Nice, so looks like once you get the correct memory you should be all set. It will be interesting to see if yours shipped with the newer BIOS already on there and if you have any issues withe ESX seeing add-on NICs in certain PCI slots. Looks like from Newegg the last few people have gotten the newer Ivy supported BIOS shipped.

      Reply
    2. Paul T.
      Paul T. at |

      Angelo, I am having the same problem, maybe for the same reason. I am waiting for my UDIMM’s to replace the RDIMM’s that I ordered. Were you able to get this to work/

      Reply
      1. Paul T.
        Paul T. at |

        That is my experience as well, hopefully I get the UDIMM’s today. However, shouldn’t I get a bunch of beeps, 5 short and 1 long if there is no memory installed on this MBD?

      2. Paul T.
        Paul T. at |

        Yup, it was RAM, and I guess it doesn’t beep with no RAM installed, but that’s ok. Got the RAM today: 32Gb Kingston KST KVR13E9K2/16I R. Booted right up, ESXi installed without a hitch and I have my first VM running now. Great little box so far.

      3. Markus
        Markus at |

        Had the exact same issue, I was baffled since there was no beeps or blinking LEDs. I was about to RMA the motherboard when it struck me that I had ordered Registered DIMMs. It’s been way to long since I built a box and didn’t order a Dell :)

  22. Mattia
    Mattia at |

    Very nice build! Does anyone know if the motherboard raid controller is supported in esxi?

    Reply
  23. Alex
    Alex at |

    Hi, congratulation, your whitebox mech is very interesting, i’m ready to build the same system, but i have a question for you.

    I’ve downloaded the mainboard X9SCM-F user-manual, and it says about networking:

    “One (1) Intel 82574L Gigabit Ethernet controller for
    LAN 1, and one (1) Intel 82579 PHY for LAN 2″ , is then the lan 2 supported from esxi5?

    Thank you!

    Reply
  24. ESX_Student
    ESX_Student at |

    Thought I would share this. Looks like Supermicro has a updated X9SCM-IIF Micro-ATX board that comes with the two out of the box compatible Intel® 82574L NICs and should come with the newer Ivy CPU supported BIOS.

    Reply
  25. Richard Robbins
    Richard Robbins at |

    Do you recommend connecting the Scythe fans to fan connectors on the motherboard or do you prefer to wire them to directly to the power supply? The fans are three pins fans so they won’t be controlled by the motherboard in any event.

    Reply
  26. Richard Robbins
    Richard Robbins at |

    I’ve configured the ESXi host and coupled it with a ReadyNAS device that I have which supports iSCSI for the datastore — so my box has no internal storage and no optical drive. I’ve configured a virtual machine on the datastore but can’t figure out how to install an OS on the virtual machine. I’ve tried associating an iso file to the VM from vSphere client and I’ve tried associating an iso file with the virtual CD device on the ESXi host. I’m sure I’m missing something obvious but I’m stumped. I don’t want to break down an purchase an optical drive for the host just because I’m stuck. Can you point me in the right direction?

    Reply
  27. bw
    bw at |

    hey Chris,

    really useful information here. i’m about to embark on building a “whitebox” . a bit new to this “whitebox” area. been using VMWare server 2.0 on top of win 2003 for years. time to update my homebased technology.

    i’m wondering as a general setup situation, these ESXi boxes contain the disk storage where the VM’s will reside or is storage external like on a qnap or synology box?

    if some one could enlighten me on this aspect that would be great. thx

    Reply
  28. The All Flash Synology DS2411+ Array | Wahl Network

    [...] but also after a lengthy review of various products in the market. Although I’ve enjoyed my DS411 NAS, which can house four drives, I’ve been hitting a capacity vs performance wall. SSDs are [...]

  29. Home labs - a scalable vSphere whitebox | www.vExperienced.co.uk

    [...] The Baby Dragon II, (with a minor change via Chris Wahl) [...]

  30. Building new whitebox servers for VMware home lab - Virtualization Tips

    [...] baby dragon II build, there are a few revisions – rootWyrm or Chris Wahl About Brian Brian is a Technical Architect for a VMware partner and owner of this website. He is [...]

  31. Vote For The Top Virtualization Blogs, Win A Cookie! via @ChrisWahl | Wahl Network

    [...] Building ESXi 5 Whitebox Home Lab Servers [...]

  32. keamas
    keamas at |

    Hi great setup I think I will build kind of the same box.
    But how about a internal Raid Controller?
    Is there a good and cheap one which you can recommend for the setup?

    I don’t want to use a external NAS.

    Reply
  33. WebbosWorld» Blog Archive » ESXi 5.1 Whitebox Build

    [...] on it) I decided to look at a white-box. Some great sites that have detailed white-box builds are WahlNetwork.com and RootWyrm.us.to be sure to check them [...]

  34. Maxing Out SuperMicro Whitebox Servers With 32 GB of RAM via @ChrisWahl | Wahl Network

    [...] when I first posted my SuperMicro Whitebox build for my Wahl Network home lab I was limited to using 4 GB SDRAM sticks. The availability of 8 GB [...]

  35. Building new whitebox servers for VMware home lab - Virtualization Tips | VirtualizeTips

    [...] The baby dragon II build, there are a few revisions – rootWyrm or Chris Wahl [...]

  36. sea pro
    sea pro at |

    Hello.
    I hope this post isn’t dead and others are still participating. I’ve been looking over Brian’s post and followed a link there to yours. Your configured system ‘fits’ into the smaller type of systems that I’d like to build and I’d like to thank everyone for their additional posts and tips. Now that this post is a few months old – can you update it with any additional ‘gotachas or patches’ that you’ve had to make. I have a synology 1812+ @ home that I’ll be using and wanted to build a couple of smaller case boxes to pound it with. I’m mostly wanting to have this as a lab, but also (to get work to help offset some costs) have the ability to test some DR for them. I was thinking of upping or maxing out the memory and need several more nic’s to get everything working and ‘similiar’ to my work enviroment. I’m also going to purchase the vmware essentials plus package and manage the systems that way. Before I use these parts, are there any ‘tweaks’ out there with newer boards or faster procs that I might could squeeze into this build – staying in a small for factor case..maybe for a few more $$ ?

    Any thoughts from anyone – and thanks again for all the great info.

    mark

    Reply
  37. sea pro
    sea pro at |

    Wow. Only took 2 days to get the approval for the balance of $$ to get the equipment. I’ll be ordering the the equipment over the next week or so – and can’t wait to get started. I’m going ahead with the 32Meg ram chips, and a few different fans (scythe SY1225SL12M). the reviews were good on them and even though they are a little more powerful, 1200cfm vs 850cfm – it should only help in keeping things cool, while staying pretty quiet. I like these nics, (Intel PWLA8492MT PRO/1000 MT PCI/PCI-X Dual Port Server) but as they are ‘full’ heigth – but low profile. As you’ve put the boxes together, will I have enough room for them? I’ve decided to get the vmware essentials package to use to play with too. :)

    mark

    Reply
  38. mark
    mark at |

    dolt !!!!
    should have followed the link for the ups – as the pic is there..sorry :)

    Reply
  39. Kuntal Patel
    Kuntal Patel at |

    Hi Chris,
    Are you using the stock Intel CPU coolers? I am about to place the order for the hardware so I just wanted to confirm how the stock cooler was working out for you, assuming you are using it. Thanks

    Reply
  40. Dave
    Dave at |

    Hey Chris does your SuperMicro boxes make siren noises when booting up? I noticed my 2 boxes make a siren noise when booting up one in ESXI its fine. I checked the temperatures and they are 32C and all fans are working. Was just curious if you had the same issue.

    Reply
  41. Alex
    Alex at |

    Hi Chris how it is going?

    Finally i’ve buy one of your system, with 16Gb of ram.
    The Seasonic is awesome! and also no NOISE

    Anyway I’m testing some datastore with NFS on FreeNAS, I’m build a Home-made-NAS based on a cool supermicro board.
    Ceck my blog for more info about the board.
    I’m very busy at the moment, cause i’m setting up my home-lab inside a ex-data center rack (it’s a gift ;-) a tremendous Sun-Rack 900-38.
    When done, picture on my blog ;-)

    PS: I’ve see that you have put some network power with 10Gb FCOE cards! good job (how much does it will cost? )
    see you soon

    Reply
  42. ESXi at Home: an Exercise in …
    ESXi at Home: an Exercise in … at |

    [...] team, to thank for their suggestions. They pointed me to a number of good resources, including two blogs that were also dedicated to this sort of thing. So, I did a little research, and came to a number [...]

  43. David
    David at |

    Thanks for your build info Chris.

    I just made an order based on your recommendations:

    SeaSonic SS-520FL2 520W ATX12V / EPS12V 80 PLUS PLATINUM Certified Full Modular Active PFC Power Supply New 4th …
    LIAN LI PC-V351B Black Aluminum MicroATX Desktop Computer Case
    Kingston 32GB (4 x 8GB) 240-Pin DDR3 SDRAM DDR3 1600 ECC Unbuffered Server Memory w/TS Model KVR16E11K4/32
    2 x Noctua NF-S12A ULN 120mm Case Fan
    Intel Intel Xeon E3-1270V3 Haswell 3.5GHz LGA 1150 80W Quad-Core Server Processor BX80646E31270V3
    SUPERMICRO MBD-X10SLM+LN4F-O Micro ATX Server Motherboard LGA 1150 DDR3 1600
    SAMSUNG 840 Pro Series MZ-7PD256BW 2.5″ 256GB SATA III MLC Internal Solid State Drive (SSD)

    I can’t wait to test this config with ESXi 5.1. I build this home lab server in order to have my own SharePoint environment. Hope it will do the job!

    Thanks again!

    Reply
    1. Pete
      Pete at |

      Hey David, I was wondering if you managed to get this configuration working with ESXi5.1 or later. I am planning to get the X10SLM+LN4F board myself and wanted to make sure this works. The new Intel i210 NICs seem to cause issues with ESXi based on other posts.
      Thanks much!
      Peter

      Reply
  44. Mike Miller
    Mike Miller at |

    Hi,

    I was wondering how good the ESXi health monitoring is for SuperMicro boards. I’m used to HP kit where you have a full complement of health status available – FAN speed and health, RAID / HDD health, etc etc.

    Obviously the RAID health depends on the card. Since SuperMicro seem to use the LSI 2308 chip on their RAID capable motherboards – anyone got any experience of the RAID status monitoring for these controllers?

    Regards,
    Mike

    Reply
  45. vmware motherboard compatibility for ESXi | my repository synology

    […] Whitebox Home Lab Servers here  […]

  46. Andy
    Andy at |

    Hi Chris,
    Thanks for your info. I’m trying to build a similar system. I’d like to know if I select this motherboard, do I need to purchase licence for IPMI features? Furthermore, I’m thinking to start with 2*2TB HDD for raid 1, does integrated raid controller recognised by ESXI 5.x?

    Thanks

    Andy

    Reply
  47. Jay
    Jay at |

    Chris,

    I actually built my box using all your exact parts about a year ago. I am now thinking I would like to put the guts (most of them) into a 1u or 2u case that is rack mountable. An alternative would be a 2u/3u case on a shelf. I just want to cut back a little on the size so I can have this server and around 12 1u network devices all fit into a 46″ portable rack cabinet.

    I still need silent fans since this is at my house. Have you ever looked into this? Any recommendations?

    Thanks,

    Reply
  48. The New Haswell Fueled ESXi 5.5 Home Lab Build

    […] in March of 2012 I posted the build components for my ESXi 5.0 home lab server. Since we’re coming up on 2 years from that point, and there’s all sorts of crazy […]

  49. My lab and the birth of the portable Ikea lack 19” datacenter rack - frankdenneman.nl

    […] and the Intel 1GB Ethernet port. Once this machine is due for renewal, a white box following the baby dragon design will replace […]

  50. Larry
    Larry at |

    Hi Chris, I’m using almost exactly the same specs you have listed in this article and I was hoping you can help answer a question regarding upgrading to ESXi 5.5 from 5.0;

    Will I lose the NIC configuration for the Intel 82579LM that had to be added in as a custom vib if I upgrade to 5.5?

    Reply
  51. TinkerTry IT @ home | Superguide: Home virtualization server enthusiasts’ colorful variety of ESXi whiteboxes

    […] by Chris Wahl on Mar 13 2012 wahlnetwork.com/2012/03/13/building-esxi-5-whitebox-home-lab-servers […]

  52. Jay
    Jay at |

    Chris,

    Can this current build, just as you have it posted here be upgraded to ESX 5.5? I’m hoping I am just search the HCL wrong…:)

    Thanks,

    Reply
    1. benoit
      benoit at |

      Hi i have the same setting and i update to 5 to 5.5 , i did not lost any nic config , everything when well…

      VMware ESXi, 5.5.0, 1474528

      Reply

Share your point of view!