A very odd error – host configuration changes will not be saved

I upgraded to vSphere 6 a while back, and shortly after that I noticed an odd error.

Backing

Everything still worked, and I was busy so …. I did nothing.

I did an update yesterday using VUM.  After it was done I saw that one host, the same one above, that was now compatible with critical and not critical baselines was not compatible with my vSphere 6 baseline.  So a bit odd.  When I investigated I saw another side of this error.

VUMError

So it made me think maybe there was something wrong with the USB stick, or maybe it was out of space.  So I thought to replace the USB stick.  I bought one of the new 32 GB very small SANDisk USB sticks and wondered about how was I going to do this.  I found this article and thought it would work for me.  BTW, it did not work for me until I configured my Fusion USB ports as v3 – they default to 2.0.  At the point where it asks about disks use the local disk to local disk clone option.

Once I cloned my boot USB to the new bigger one I was able to successfully boot my server.  But it did not talk to vC.  When I disconnected it, and went to connect it I was worried when I saw it seemed to think it had three VMs on it.  Which was not true and I was worried about how that might impact the infrastructure.  So restarts did not change that, I thought I would install vSphere 5.5 to the new USB and see what happens.  I would upgrade it the same way as I did previously and see if the error comes back.

So when I had vSphere 5.5 installed, and used VUM to upgrade it – it didn’t work.  Said I needed to restart and try again.

Restart

Also, another interesting message was seen in VUM.

MoreOdd

Why is Incompatible seen for Upgrades?  I think it should be non-compliant so very odd.  It makes me think something is odd with my hardware – like maybe something not on the HCL?  But that makes no sense as I have another physical server that is the same as this problem beast.

I check CloudPhysics as they have an excellent card (PCI I/O devices) that can show you what is not on the HCL or not supported but this host is newly added back to the cluster so not seen in CloudPhysics yet.  But I have restarted and am trying to do the upgrade again.

The update seemed to go well, and I watch the server boot up.  And then it got weird.

IMG_2698

 

IMG_2699

And then it rebooted and now is sitting with a blinking icon.  So no more server.   Maybe I should try installing vSphere 6?  I think the issue is that there is a disk related driver in vSphere 5 that is not in vSphere 6.  My guess is that my other identical hardware server has that driver, maybe from before 5.5 and was upgraded to 6 successfully and for some reason this server was not.  It may have been a clone type issue with the USB, but when I installed new 5.5 U2 and that worked, the upgrade to 6 should have worked.  But I will investigate disk drivers now that are not in 6.0.

This is a USB and not a network or storage component and nothing suggests I am missing a driver.

I see the error I have via Google and someone else has it with a poorly seated USB that they release and then all OK.  For them it seemed to be BIOS and SD related.  They eventually solved things (?) by going to USB.  I am using USB, so it is interesting.  But a clue I think.

So what did I do?  I did a destructive install – meaning overwrote – on the two new USB and everything has been working for a week or so now.  So the morale of the story – is with this error message you may solve it by re-seating the USB if you are HP, but if not replace the USB and do not clone the old one but install new.

It turns out that this issue came back.  But, also, I figure out a theme.  I have two USB ports in front of the server hardware.  One of them has the USB with the ESXi image on it.  The other one I use for the keyboard when I am working at the console.  Turns out that if I have that second USB port plugged in with a keyboard, after some time it will cause the issue that this article is about.  Interesting stuff – right?

This kind of issue after an upgrade normally means you have a missing device driver.  Here is an article that helps nicely with that.

Update:

  • 2/12/16 – update for what I think caused this.
  • 2/11/16 – it’s back.  Crap.

Michael

=== END ===

 

Tagged with: ,
Posted in Home Lab, Issue
2 comments on “A very odd error – host configuration changes will not be saved
  1. albertwt says:

    Hi Mike,

    Last time I got this similar issue of Lost connectivity to storage device mpx.vmhba32:C0:T0:L0. Path vmhba32:C0:T0:L0 is down. Affected datastores: “”. The ESXi OS was installed on SD-Card on HP Blade server, the fix was by updating the iLO firmware to the altest version using HP SPP.

    http://h17007.www1.hp.com/us/en/enterprise/servers/products/service_pack/spp/index.aspx

    Hope that helps as well.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: