I had chatted a while back with some customers that don’t use Virtual Labs. And while I used to have them in my lab I do not right now. So I am bringing them back – so this article is helpful for me, but hopefully for those people that don’t use Virtual Labs – or SureBackup Jobs. I think everyone should. Yes, I know they are a little complex but I will help with that in this article, and I think you will see they are not too complex. And I think we can all guess that Veeam might make things easier as time goes on.
We will verify my mail server which is quite important to me. So being able to verify them without me doing the work each backup is a good idea. One odd thing is I know my mail server needs AD. So somehow I need to manage that.
We will only verify a few machines on a single host. These machines will be backed up but not replicated. In another article I will cover off replicated machines and multiple host testing. I like starting out a little easy. This is all done in a vSphere lab. Much of this will apply to a Hyper-V lab though.
We will do the following – create an application group, create a virtual lab, and create a SureBackup job and schedule it. We will in fact link it to a backup job so that after the backup the SureBackup job will fire automatically.
Things to think about
- You need to be using Enterprise Edition of Veeam B&R.
- VMware Tools should be installed – for more reasons then just this reason!
- Firewall in VMs must support ping operations.
- We are doing a simple example and that means single host so we don’t need a distributed switch.
- We are doing this using non – replicated virtual machines.
- For your SureBackup job, I suggest you pick a logical group of virtual machines that is very small – 2 or 3 max. I picked my mail servers.
- This was done with vSphere 6.0 U2 and Veeam B&R 9.0 U1 (and later with vSphere 6.5d and Veeam B&R 9.0 U2). I also confirmed it works great with vSphere 6.5 U1f and Veeam B&R 9.5 U3.
Create an Application Group
I start in the Admin Console on my desktop and change to the Backup Infrastructure area, followed by expanding SureBackup, and selecting Application Groups.
Lets use the Add Group button to start the wizard.
I like how Veeam puts my credentials into the description. I add to it a little info on what this application group is for.
Now we use the Add VM button to add the virtual machines to our group. You can chose VMs from backups, replicas, or from storage snapshots. Look forward to a very interesting article on day using VMs from storage snapshots. But in the meantime, for this article, use the from backups option.
I select the virtual machines that are my domain controllers.
However, before we Next, notice that the Role column is sort of blank? We have some work to do there.
So select your first DC – which is my DC / Global Catalog VM, and use the Edit button.
Note I have selected DNS, Domain Controller, and Global Catalog? You must only select one of your domain controllers as DC but you can select your other DC as DNS if that is appropriate. I do not normally make any other choices in the other tabs. It is important to note that the Test Scripts tab allows you to have your own scripts. I have an article on that in progress currently as that is so very important. Your own application can be tested with a script that you add in. Very powerful! I should mention that the Credentials tab is only necessary to use if the account the job fires under does not have the access – and currently do not touch it due to this issue.
Both of my virtual machines have a role now and so we are ready to continue, and finish.
We can have more then one application group, and they can be very customized with the canned roles, but also the very powerful scripts. The reason for this application group is to provide AD to my application being tested – my mail server.
Create a Virtual Lab
We continue in the Admin console and select Backup Infrastructure, expand SureBackup, and select Virtual Labs.
We can use the Add Virtual Lab button. This will start a wizard.
I like to use a name that makes sense, and use the description to provide some insight. We now move the Host info screen.
Important to note that you select a host that is managed by vCenter, and is in fact connected to Veeam via vCenter, as well as the cluster in use has DRS enabled, or a standalone host works too. Select your host.
We need to select the datastore now.
We now need to make some proxy decisions. This is an important step that in fact allows you to separate your virtual test lab from your production network.
Generally you will need this proxy appliance and the service of ‘firewall’ it provides. The first Configure button will allow you to change the name of the proxy (and it will be seen in vCenter), but the second one will allow you to configure your proxy production side networking. DHCP is OK for me.
In our next screen, we are going to use the Advanced single-host option as it is the best one to start with. The other article I have mentioned I am working on is with replicated virtual machines and the Advanced multi-host configuration.
On the next screen we see Isolated Networks. Normally we do not need to make changes there.
However, on our next screen we need to do some work. We need to make sure the ‘inside’ network IP of the appliance is in the same network scheme as our VMs so that things like ping test and script tests can work.
So lets use the Edit button.
So I change the IP address to the scheme that is in the VMs – best practice is to set it to the default gateway that the VM(s) are used to. So I change 192.168.1.1 to 192.168.9.100. I also disable DHCP as it is not necessary. Be aware that the Masquerade network address may need to be changed if you have a variety of virtual labs. Once we make the changes and continue we see Static Mappings. Normally that is not used.
Now we see the typical Summary (or Ready to Apply) screen.
Once you hit Next you will see things applied and a very nice Log screen.
We now have an application group, and a virtual lab to run it in. Wow. Again, we can have multiple application groups, and multiple virtual labs too.
Create a SureBackup job
As if it was a theme, I start in the Admin console on my desktop. I change to Backup & Replication, and select the SureBackup Job button to start the wizard.
As always the name is connected with the other components. On the next page we see the Virtual Lab info – and can change to the right Virtual Lab as necessary.
In the next screen we see the choices of Application Groups. We get to choose the AD group so our mail server has what it needs.
We now have the ability to Link Jobs. This screen is where you connect the backup job to test. In our case it is the backup job that does the mail server.
In the next screen – Settings, we can make a few useful decisions.
I always do the email notification – especially when things are first configured! The Backup file integrity check is quite interesting as it does a CRC check of the VMs. It can take a bit of time but not so much when a few small VMs are in the job.
Next we will see the schedule of operation.
As you can see, I enable, and select the job that it is associated with. Next is the Summary screen and we just Next or Finish through it.
Now we have an application group created, a virtual lab created, and now a SureBackup job. Ready for some testing!
The first test showed that things didn’t work. So I created a troubleshooting section below that may help you out.
You can wait for the linked job to run, and the SureBackup job will follow it. The email is often very useful for troubleshooting as well as knowing things worked.
And this means my backups are in fact ready to restore – they work. This test proves that. So every time the backup of AD occurs, after it is done the most recent backup is used to do the test and confirm all good. This is not a test where a VM is restored and considered good. We are getting the VMware heartbeat, a ping test successful, and then scripts used to confirm services are running. Pretty good!
No IP set on inside of test!
During setup I had some issues. I had a failure rather then a success email and here it is (just a sample screenshot):
Here is the info on the failure from the UI.
So it looks like to me that the email is showing the better info. Which is often the case. It is interesting that we can see the heartbeat from VMware Tools was found, but no ping. So now, since I do not see the solution anywhere, we need to check the logs.
I was working on the Admin Console on my desktop and not on the backup server, so I had to exported the logs for the specific SureBackup job (you can find out more on export). I then looked into the Job.SBJ_AD.log file (BTW, SBJ_AD is my SureBackup job for my AD backup). I looked for the approximate time of the job and look what I saw.
It looks like the IP address the ping was coming from is 192.168.255.8. My DCs are assigned static 192.168.9.8 and .10. So no route. A short distance after this error in the log file it shows more on how application test as the IP – as seen above 192.168.9.8 – was not reachable. So confirmation. The solution was to make sure on the inside of the proxy router the gateway was the same as outside of the virtual lab. I have also seen similar errors when there is no DHCP inside the virtual lab but yet it is needed.
Bad credentials or is it?
If you use bad (or good) credentials in the Application Group for individual VMs, but everything else was OK, the notification email will look like below.
When you search the logs for this SureBackup job you will see something like below.
This appears to be a credentials issue but in fact is not. It is a bug due to a change in the Microsoft OS. Find out more here. It can be avoided by not doing anything with the Credentials in the application group.
- All Veeam B&R technical article I do – link
- Exporting Logs – link – very handy for when working on the console and when you don’t have access to the backup server directly to go and look at the logs.
- Leave the Credentials area blank when editing VM roles in Application Groups – link
- External access for your Virtual Lab – here – looks good but have not tried it yet. Most handy info though.
There is no reason – at least for small numbers of protected virtual machines – to not test after every backup. This means you are better protected. If you have a lot of VMs and lots of backups, you can test what is important on your schedule Plus, there is other uses for your virtual labs we will cover off in other articles – like patch testing.
- 3/6/18 – confirmed all good on later VMware and Veeam software. Also added a bit of detail in a few places and corrected a spelling mistake.
- 6/18/17 – some mistakes and some confusion that is now cleared up!
- 1/25/17 – added the info above on best practice to change the inside address to the default g/w the VM is used to – in the VL config.
- 11/6/16 – added the link to the article on external access.
- 8/9/16 – I was confused on the purpose of the Linked jobs. Now I am not. So I changed the screenshot above, and added a little text. No more confusion. And I may do an article in the future on how to exploit Linked Jobs.
- 8/3/16 – First created this article.
Thanks for reading, and as always, let me know if you have questions.
=== END ===