Odd Problem – sometimes you do need to get irritated!

I have added a Synology (DS1813+) into my lab.  It has been around for a while to help with a few different things, but now is going to expand the number of VMs I can host.  The management of it has been fine.  But when I added it to the isolated and non-routing storage network things got weird.

Nothing could ping it.  I tried different ports on the switch, and on the NAS.  I had a bad cable and I thought that was it.  Nope.  I had a very old firmware on the switch and nope, that was not it – and yes, it was hard to upgrade.  Restarts and resets made no difference.  I could substitute the switch for another switch and the problem remained.

Then it got weird.  Actually weirder.  I directly connected a laptop to the NAS.  And surprise me to no end it worked.  I then removed all network connections from the switch (so the lab just went down), and only put the NAS and the laptop and it worked. I added two ESXi hosts and another NAS.  Now the laptop could ping the switch, and the NAS. The two ESXi host could not ping the NAS but could ping each other.

Bear in mind through all of this management and VM networks were fine, and the management access to switches, NAS, or ESXi was fine throughout.

While everything worked before adding this Synology I got a little suspicious while troubleshooting when I had a vMotion issue (due to the STORAGE switch being disconnected).  It turns out that vMotion was using in some cases the Storage network and maybe vice versa.  Now I am irritated and need to get very low in the stack.

I stopped troubleshooting and picked one host and physically traced the cables, and confirmed the switch config.  When I could ping the new NAS, and old NAS, I moved to the next host.  Traced the cables and switch config.  Now when it was good, I could ping the new NAS, old NAS and the other host. Now for the rest of the hosts.

When I was done, all was good.  Turns out the network of vMotion and the Storage network were co-mingled.  Enough so that vMotion and storage worked, but one more device did not.  Synology Support thought it was network, and they were sort of right, but I would have appreciated a little more support.  For example, they could have told me no traffic hit the storage network port on the NAS.  Cisco declined to help.  My switch was built in 2011, and I bought it in 2012 so by the time I bought it there was no more warranty.  But the poor or no support made me get desperate and that made the difference.  My last work lab was 10 GB so there was not so many connections to worry about!  In my personal lab I am simulating VLAN’s by using extra switches, and while I thought that made things easy it did allow me to make some config issues and once I figured that out it did not take long to fix everything.

So now that the lab is working and has more storage, I can get onto the fun things – like the new HTML 5 client for vCenter – and my View upgrade.

I suspect most of you won’t have issues like this.  But just in case you might, I share my painful story. Don’t hold it against me.


=== END ===

Leave a Reply