Designing An Entire Rack of Home Lab Gear for 400 Watts
Two major challenges when building a home lab is figuring out the power consumption and heat dissipation that will radiate from the systems. These two are usually coupled together: a system that is hungry for power usually needs a bit more cooling and emits a fair amount of noise. I pondered on these same challenges when building my home lab and came up with the idea that it must be possible to “go green” and still have a viable home lab. The first time I formally wrote on the idea was around building a new, green workstation in my “Building A Green Performance Desktop” challenge. But I never did take the time to really go over those same underlying efforts that punctuated the home lab build.
To borrow one of the most horribly abused business jargon in history, let’s “open the kimono” on my home lab’s green design strategy and go over some key decisions that were made along the way.
I can almost hear the moaning for using that kimono saying- delicious!
Functional Design Comes First
As with any good design, the first step is to figure out exactly what you are trying to accomplish. In my case, it was “to build a playground for learning geeky stuff.” Fair enough, you might comment, but that doesn’t really drive an intelligent discussion around design, does it? I’ll break my original lab thoughts down into these 5 items:
- Physical vSphere hosts in a realistic cluster, able to run about 10 VMs each
- Physical networking infrastructure, 1GbE to all ports, multilayer switch abilities (layer 3)
- Shared storage via NAS, both NFS and iSCSI
- Entire lab should be relatively quiet for a person sitting 4 feet away
- Able to run on a single UPS that can plug into a 15A socket (standard for most homes in the US)
This isn’t a full fledged set of requirements and constraints, but was a good start to getting my thoughts onto paper. Sometimes the starting point is the most difficult, but I find just dumping thoughts to paper can help get things kicked off.
Incorporating Green Ideas
My design strategy was divided into several major sections. If you look at the parts list on my Supermicro ESXi 5 Whitebox build, you’ll get an idea of my thoughts behind each item.
For servers I find this is relatively straight forward. Each item usually has a few variations. The CPU, for example, comes in multiple wattages. I chose the latest for the time with “Sandy Bridge” CPUs running at 80W, which was roughly middle of the road on power consumption. Some CPUs were greater than 100W, while others were near 65W. This seemed like a good niche for keeping away from incredibly high performance, and as such, high heat.
Another way to go green is by using as few moving parts as possible. In my case, I chose a USB stick to be the boot device for each server. This results in negligible amounts of power, heat, and noise. It’s also a bit cheaper than a pair of spinning drives and significantly cheaper than SSDs, but has the downside of being a single point of failure. Fortunately, the hypervisor is relatively unimportant, and rebuilding on top of a replacement USB stick is a small amount of effort. USB 2.0 has the added bonus of being a limitation on power draw, since a device is limited to 500 mA (or about 2 watts) of power.
Power Plant and Fans
Stock fans are usually garbage and should never be trusted. They are typically noisy, small, and consume too much power for the CFM (airflow) that is provided. I always shop for custom fans that are large in size and low in dBA (noise). All of my servers are equipped with 120mm fans that spin at about 800 RPM. They are so quiet (about 10 dBA) that you literally can not hear them.
It is also rare that you would want to use the included power supply with a server. It’s typically a throw away unit that sucks up power and pumps out noise and heat. My advice is to go with something that is rated Platinum (or better) by 80 Plus. This is a measure of efficiency of the unit, with a higher rating meaning more power is actually being produced with less waste. At Platinum, you’ll see about 90% or higher efficiency across a variety of loads. The cost will be a tad higher on your initial purchase, but the savings will return to you in the form of low heat, quiet operation, and a reduced power bill.
Thoughts – And Home Lab End Results
If you’ll notice on the “My Home Lab” page – I have a screen shot of the UPS system showing about 423 watts being consumed during normal operation. The entire list of gear on the page is tied to that single UPS. I’m quite pleased with putting the thought of being green into my lab, as it is both more environmentally friendly and person-sitting-next-to-it friendly.