My vSphere 6.0 U1b Upgrade Checklist

Hello all,

This is a big deal for me.  This is the production part of my lab so this means my whole lab will now be at 6.0 U1 and I am quite excited with that.  I did upgrade a small part of my lab - documented here - and as part of that I upgraded my infrastructure - so vROps, VIN, LI, PowerCLI and View are all current (or almost as I did not see the need to do View 6.2.1).  So that means this is more of a vSphere Upgrade rather then a lab wide upgrade.  I need to do my VCP6-DCV upgrade test soon so this upgrade and the end result will be helpful with that. So lets get started.

BTW, I am using the VCSA (5.5 U3) and Dell R720 hardware.  I will upgrade to VCSA with an embedded PSC and then re-point.  Or maybe do that as part of the upgrade!

The build of vCenter (not VCSA) before I start is 3252642 and ESXi is 3343343.

Note: I accidentally published this when I was editing it.  Sorry about that.  I will be sure to note when it is live in the updates section.

Dependencies

While I have most or all of these taken care of, I have them listed here to be sure.

• LI 3.0 - release notes, docs, bits - works with 5.5 U3 and 6.x - done
• vR Ops 6.1 - release notes, docs, bits - works with 5.5 U3 and 6.x - done
• VIN 5.4.8 - release notes, docs, bits - works with 5.5 U3 and 6.x - done
• Dell OpenManage for vCenter - release notes, docs, bits, - works with 5.5 U3 and 6.x - I was not able to make this work after my test upgrade so will be testing it again after I am 6.0 U1.
• Veeam v8 - release notes, bits - works with 5.5 U3 and 6.x - done
• vCenter Support Assistant - release notes, docs, bits, - works with 5.5 U3 and 6.x - done
• ESXi Host Client v4 - info and bits, supports 5.5 U3 and 6.x. I was using an old one but will wait until hosts are updated and then use the v6 bundle to upgrade.
• I do not often say this but I should.  Make sure your environment is healthy before you do this upgrade.  This means host and vC need to resolve in DNS properly, backups should work without error, and vROps should be showing mostly green and where it doesn’t you understand why.
  • If your vCenter appliance was installed at 5.0 and you just updated it as necessary and you are going to upgrade now you may get a certificate error.  If this might happen to you, log into the appliance VAMI as the root user, change to the Admin tab, and check the box labeled Certificate Regeneration, and then restart the appliance.  This is also something you need to do if you ever rename your vC.  Which is not a good thing to do normally. After this regen of the cert you will not have an issue with certs upgrading.  If you have done your own certs this will not be an issue you will experience.
  • So remember, healthy means all hosts reporting to vC OK, DNS healthy, time in sync, and happy users too!
  • BTW, if you are not sure if you need to do the cert regen thing you should do it.  Will not take long and won’t hurt.
  • I did have a little issue.  It was a dead 10 GB card in of my hosts, and I did not get notified of it, but I saw my vDS was not synchronized.  So that took a little while to deal with!

Requirements

• vCenter - bits, release notes, docs - 6.0 Update 1b
• ESXi - bits, release notes, docs - 6.0 Update 1b
• Upgrade doc - Order of upgrade article,  upgrade HTML, upgrade PDF
• 6.x license should be handy

I like to have a backup of vCenter, and VUM before I start.  You can also do snapshot if you like.  I like to have no outstanding patches of ESXi too.  Make sure you don’t upgrade to 6 in this way but rather just patch to the current level - remember vCenter must be upgraded before hosts.  BTW there was an issue with upgrade from ESXi 5.5 U3b to 6 but that has been addressed in 6.0 U1b.

BTW, have the VCSA 6.0 U1b ISO uploaded to your ISO location so we can mount it on your desktop.

Process

The order is important (this is a simple upgrade, but often it is much more so there is a good guide here on order of operation).

vCenter

We need to connect the VCSA ISO to our desktop.  You can find the detailed instructions for upgrading the VCSA here.  I suggest you deploy to Standard Switch and after the upgrade is successful then change the network that hosts it to whatever you need too.

Here is an outline:

• DRS in cluster you are hosting the VCSA in that is being upgraded should be set to manual.
• Mount ISO
• Install Client Integration Plug-in (in the vcsa folder)
• Start the installer (vcsa-installer.html in root of ISO)
• Select Upgrade
• Answer the questions - make sure to read carefully!  You can use the same name for the new appliance that the old one uses.
• It will take a while - maybe 30 minutes for me - to get the migration done.
• Log into the vSphere Web Client and make sure all is good.  I got an error when I tried that.

• So I updated the C# client so I could see if that would work.  I also wanted to see if the old appliance was actually off.
• Once I got the C# client installed and tried logging on I got a system error on my account message.  It looks like the Windows session is the issue.  If you manually enter credentials you do not get system error.  But my creds don’t work, but using [email protected] does.
• So I logged in using the Host Embedded ESXi Client.  I confirmed old VCSA was off.  So I restart the new one.
• Log in on the VAMI port - :5480 - and make sure NTP is still set.  It was not set in my case. You may want to change the password expiry too while here.
• After the restart, and while I can log in as [email protected] I still cannot log in using my AD credentials.  I wonder if I lost AD connection?
• Yes, I can confirm I lost the AD configuration. Easy and quick to add it back.  I was worried I might have lost all my permissions for AD users but no, that was still all there.  Once the source is added, I can log in fine.
• Add the vSphere 6 license.  You do have 60 days to do it but no need to wait.
• DRS back to fully automatic.
• Move the new VCSA from the VM Network, or the (ephemeral) port group to where it would normally be.

VUM

We need this working to do the easier upgrade of the ESXi hosts. So expand the ISO, or attach it to your VUM server.

• When on your VUM server start the installer (autorun).
• Select VMware Update Manager.
• It should prompt about the upgrade.
• You will need the appropriate service account(s) passwords.
• You do need to upgrade the database.
• It should not take long to do the upgrade.
• You will need to log out and log back into the vSphere Web Client.  If you do not see the VUM applet on the home page restart the VUM server and once it is up, log out and log in again - in the Web Client.

ESXi

Once VUM is working upgrading ESXi hosts becomes much easier.

• You can use that Home screen Update Manager applet to create a vSphere 6.0 U1b baseline and apply it to your hosts - or cluster.  First upload an ESXi image, and then create a baseline.
• You can visit a host, or cluster, and select their Update Manager button on the Manage tab.  This will allow you to scan, remediate, attach or detach baselines.
• Should not take long to do 3 or 4 hosts - maybe 20 minutes ish.

VMware Tools

You have a variety of options to do this.  You can use VUM (here is how), or you can trigger an automatic update on a VM from their Summary screen in the Web Client.

As of today - 1/10/15 there is an issue with 6.0 U1b where if you update the tools in a template you might have issues.  So I am going to wait on doing that.

External PSC

While we could not change from internal to external during, I considered it a best practice to have an external.  But now we can change it doesn’t seem as important.  I can instead have one if I need one, or not.  Just need to know the requirements!

Testing

• vC shows version - 3339084 (good), VCSA shows version - 3343022 (this is version from logging in on port 5480).
• ESX shows version - 3380124 (good)
• VMs are running.  All key are good - database, email, desktop.
• VMware Tools look ok? If I have done it yet that is - which I haven’t.
• vR Ops looks ok - no, looks like it needs to register again. Logging in looks like I had to start it.  Data node down was the message.  So I started it.  It has spent some time on waiting for analytics.  Wait a day or three and connect again it is still having issues.  Data Retriever is not initialized yet.  I get that without logging in.  Just connect to the UI.  I think I will need to talk to support on this.  And yes, it did work fine before upgrade.  I do think that this is related to the migration nature of this upgrade.
• VIN data looks ok no, looks like it needs to register again.  Quick look finds nothing.  I wonder if I have to install again?  That would suck as I would have to redo some of the apps I have defined. I wonder if I could do it via vROps if it was working? I think I will need to talk to support on this.  And yes, it did work fine before the upgrade. I do think that this is related to the migration nature of this upgrade.
• Dell works? I should note that this did not work before the upgrade.  I thought it would work after the upgrade.  But while it registers with vC with no issue.  It is not seen in the Web Client UI. I guess I will have to call support.
• Veeam can do backup? - Actually some jobs failed and some passed.  Investigation showed it was related to the fact I regenerated the cert on the vCenter.  To fix this just reconnect to the vC in the Backup Infrastructure area.  Once done it works fine.  With one exception - jobs based on tabs don’t work.  I had to change them to backup the template or VM by name.  Then it worked.
• Client Integration supported log in still works? - yes
• Cleanup
  • Delete old vCSA.

Links and Background

• Upgrading VCSA to embedded PSC using Simple method - here
• How to upgrade VCSA 5.5 to 6 - here
• Upgrading to vSphere vCenter 6 best practices - here
• How to upgrade VCSA 5.5 to 6 - here
• How to upgrade from VCSA 5.x and 6.x to VCSA 6.0 U1 - here
• Upgrading from vSphere 5.x to vSphere v6 Best Practices - here
• Here is an article about a painful vSphere 6 upgrade, and one about a VUM upgrade that might be useful if you have issues.
• Here is my first vSphere 6 upgrade experience article
• Upgrade order of operation - here

BTW

• I like to have my admin account in both the SystemConfigurationAdmin group as well as the Administration Group in SSO.  This means if the default [email protected] account breaks (or I misplace the password), then I still have all necessary access.

Things left to do

• Update vDS switches - if you forgot you will see something like below.  Just use the Actions menu to access the Upgrade prompt. - done - worked easy and no negative impact.

• vR Ops working
• VIN working
• Dell Open Manage working
• Support Assistant may or may not be working.  Cannot access the UI any longer.

Updates

• 1/21/16 - 12:58 - added the things left to do section and the update vDS switch info.
• 1/14/16 - 1632 actually published.
• 1/14/16 - accidentally published.  Damn it.

Summary

This has gotten my lab upgraded.  But I had some network issues - dead 10 GB card, dropped AD source after the upgrade, and some management tools issues.  Plus no more tag support in Veeam for me.  This means this is one of the worst upgrades for me.

And I have to call support on vR Ops, VIN and Dell.  The Dell issue though, existed before the upgrade.  I will update this article as I fix these issues.  Sorry about not completing this sooner, and more complete.  Just ran out of time.

As always, thanks for your attention and time!  Questions and comments are always welcome too!  And considering how bad things went, if you have corrections, or suggestions on doing better, they are particularly appreciated.

Michael

=== END ===