I’ve found that most folks only deal with logs after a disaster strikes. Production is down, check the logs! Much like backups, a failure to proactively collect and store logs means there is no going back to magically find the logs, and troubleshooting is made significantly more difficult. In fact, many vendor support organizations will ask that you then turn on logging to some safe place and wait for disaster to strike a second time so that they can determine root cause. Bleh!
Let’s collectively get into the habit of treating logs like production data by making sure they are continuously gathered, protected, and reviewed for issues. And what better place to get your feet wet than a lab environment?
In this post, I’m going to transform my Synology DS411 NAS into a Syslog server using DSM 5.0.
In DSM 5.0, the ability to become a syslog server is native – the packaged application that was required in DSM 4.0 is no longer necessary. Begin by opening up the Log Center and configuring the Log Receiving and Storage Settings areas.
Start off with the Storage Settings menu. You’re required to specify a destination for the syslogs. I ended up just making a folder named Syslog and checking the boxes for Compress log archives and Archive logs separately according to device as shown below.
The next step is to enable Log Receiving. I checked the box for both BSD and IETF formats. Note that UDP is the default protocol and is typically what I use.
ESXi Syslog Configuration
Let’s point a few ESXi servers to the new syslog server. From the vSphere Client, navigate to a Host > Configuration > Software Advanced Settings > Syslog > Global. The specific setting is Syslog.global.logHost.
Enter the destination address for your syslog server. The format is
<protocol>://<hostname or ip>:<port>
Because I also use Log Insight, there are two entries in the list. A comma will separate out the various log hosts, making the resulting entry look like this:
Repeat this process for any other hosts in your environment, or just use a PowerCLI script.
Older PowerCLI code (depreciated in 5.5)
[code language=”powershell”]Get-VMHost | Set-VMHostAdvancedConfiguration -Name Syslog.global.logHost -Value "udp://nas1.glacier.local:514,udp://10.0.0.84:514"[/code]
Current PowerCLI code (source)
[code language=”powershell”]Get-AdvancedSetting -Entity (Get-VMHost) -Name syslog.global.loghost | Set-AdvancedSetting -Value "udp://nas1.glacier.local:514,udp://10.0.0.84:514"[/code]
I also found this KB entitled Configuring advanced options for ESXi/ESX to be educational. Also, you might want to peek at Jon Kohler’s blog post on why TCP or SSL are better protocols to use for vSphere (instead of UDP). Keep in mind that the Synology NAS cannot do both UDP and TCP at the same time, so you’ll have to pick one protocol. Since almost all of my systems default to UDP in the lab, I’ve chosen that protocol.
ESXi Firewall Configuration
Although I did not find this to be the case in my lab, Mike Tabor comments that you might need to enable the ESXi firewall rule for syslog activities. Navigate to the Host > Configuration > Software > Security Profile and enable the checkbox next to syslog as shown below:
The PowerCLI code is:
[code language=”powershell”]Get-VMHost | Get-VMHostFirewallException -Name ‘syslog’ | Set-VMHostFirewallException -Enabled:$true[/code]
After pointing a few other devices to the DS411 Syslog server and port, I saw logs coming in. By clicking on the Overview area, a list of incoming logs are shown in a graph. The “localhost” entity is my NSX Manager, which was never renamed to anything fancy. I really should rename it to something more descriptive.
Here’s another view while powering down a bunch of virtual machines in the lab at once.
Not too bad of a feature, and now my DS411 is doing a lot of nice little utility stuff that I had scattered around the lab. For example, I used to use vCenter as my syslog server, but this seems much more efficient. Is it as robust as Log Insight or other commercial products? No way. But if you own a Synology NAS, you already own the ability to collect logs, so why not use it?
Keep in mind that if you want to collect logs from the Synology NAS, it might be best to ship them somewhere else. If something were to happen to your NAS, there would be no way to gather the logs. I’ve run into this situation with production environments, where all logs were going to a storage array that ended up being involved in some trouble. Just be cognizant of where you’re logging to and what would happen if that log destination were to fail. 🙂