I have seen a lot about how cool ReFS is and how important – like Ricks article. I have not seen so much, certainly not in one spot, on how to make it work, and how to make sure it is in fact working for you. So that is what I will cover in this article.
ReFS has been around for a while, but it wasn’t until Win2K16 that things got exciting. It is a filesystem – which means when you format a disk as ReFS that is the file system – that looks like NTFS, and holds your data. What makes it really different is that it is a very well designed files system that has more features and functionality than previous Windows file systems. It is some of these features that make such good sense for Veeam to use them.
On an NTFS volume when you have a synthetic full operation – like on Saturday in my lab – you have blocks moved to produce a full. That takes more disk space and it takes time. Often it takes a lot of time.
So in ReFS it changes so that when that synthetic full occurs no blocks are moved – instead metadata is updated – through something called a block clone operation – and you end up with that full backup but with only pointers changing it happens very fast and without more disk space in use.
BTW, some hard technical facts are in one test it went from 5 hours of time to 30 minutes of time for that synthetic full. Another example is 30 minutes to 3 minutes.
So I think any and all Veeam customers – who are using a Windows hosted repository – should have it hosted on a ReFS volume – BTW, you don’t to upgrade your domain, all you need is one server running Win2K16. But, depending on how you do ReFS with Veeam it can produce the results we want or not. Results we want is the drastic reduction in job time as I mentioned above, and the results we don’t want are no change. So read on to see how to get the results you want.
How to get an ReFS partition
We need to format with different parameters to get the results we need. See below for what it might look like.
How to confirm we have an ReFS partition
Once a disk is formatted it is hard to tell quick and easy that it is ReFS or not. Use the info below to confirm what you have.
Use can use the the two commands below to see if a partition is ReFS and if it is 64k block size or not.
fsutil fsinfo volumeinfo disk:
fsutil fsinfo refsinfo disk:
You can see the result below – and they are what we want.
Here is an example we don’t want – ReFS but the wrong block size.
Here is what NTFS looks like.
Ready to go
Now you can start doing backups to the new ReFS partition and when a synthetic full occurs, you should have it go much faster.
Using a new ReFS based repository
You need to have Veeam recognize that ReFS based repository, and as a result of that it will write out to it with that knowledge. That is key and that is why the potential block clone occurs. So if you drag and drop VBK files to this new repository you will not see an improvement in your backups. At least not right away as a full backup will need to occur for Veeam to know it is on a Win2K16 server and that there is additional API support.
How to tell a synthetic full using block clone?
So aside from a faster backup, how do we tell if a ReFS operation happened. Which is often on Saturday. Below is the Advanced Settings from a job.
You can look at the history of a job to see what we need. We look to see the value between the square brackets. It should read fast clone. See below for an example.
Some common alternatives you will see are see below. NFS means you are backing up a VM hosted on a NFS datastore that the backup server has access to the NFS datastore. This is a very fast way of doing things, and lessens the impact on an ESXi host. The hot add means that you have a virtual backup server and it is directly accessing the VMDK of the VM you are backing up.
Things to watch out for
If you copy your disk backup files to the ReFS disk you will find out the there is no change – so the backup jobs on the weekend (or whenever your synthetic full occurs) takes the same time (or you see hotadd instead of fast clone). The reason for this is that the Veeam software needs to know it is ReFS and then write to it as ReFS file system. In your case you just copied files. In this case the easiest thing to do is schedule a compact on the files you copied.
Once the compact is done the backup files will have been written out for ReFS and your improvements will occur. Another alternative would be to do a full backup, and then wait for the synthetic full to occur.
So I think you now have enough info to make ReFS work for you. Even understand when it is not and what to do. I would like to add that I think you need to leave enabled Corruption Detection. Some might consider this as extra but I think backups are important and this is quite handy. It also checks the backup from the file point of view, and not the block as ReFS does.
If you forget about ReFS, and you try and create a Repository on a Veeam server you will get a reminder.
So if you see that, you know you need to format again and use the right options!
When things go wrong
If you have a volume you have formatted as ReFS you have some additional data protection. It is called Integrity Streams and with a single volume you will be alerted when data corruption occurs. And eventually it will occur. This alert is in the form of an event in the event logs.
Important to note that this event includes the name of the file that has the problem. You need to connect the file name to your backups and for the VM impacted you do immediately an Active full backup. From that point on no more issue. If you are using StorageSpaces Direct you will have this sort of issue fixed automatically. If you are using Log Insight my Veeam Content Pack in the next update will capture this issue and alert on it.
I have picked up a few things since I wrote this article and thought I would add them here.
- Always keep current on patches and updates. Microsoft has released updates that really help ReFS but also make sure all the hardware is updated as well.
- A rough guideline that Microsoft has started that has seen a lot of sucess is to use a ration of 1 GB of memory per 1 TB of storage.
- If you are using a SOBR make sure to set Data Locality.
- Make sure you do the Veeam exclusions for your anti – malware software. You can find out what they are in this article.
So you have learned about making ReFS work now, and you should start testing it out. You will be impressed I am sure. If you want to learn more you can check out this video.
- 12/5/18 – updated with the miscellaneous tidbits above.
- 10/15/17 – here is an article that is a nice overview of ReFS. Thanks to Anton for sharing it with me. Also, you likely heard that MS doesn’t support ReFS on SAN attached storage. That is not that surprising when you think about it since it cannot test all the controllers our there. But, if you are using tier 1 storage that is Enteprise ready I believe you will be fine. This is not something that will hold me back from using ReFS. I will of course not expect MS to support me if my VM is using SAN-attached storage from Synology for an ReFS partition. But, the fact is I just use ReFS inside my VMs so no issue.
- 6/21/17 – here is an article that can help you figure out your savings. Quite interesting.
- 3/25/17 – There is an important patch for ReFS and it is here. That patch needs a registry and I hear that option 1 for the registry change is the one to use.
- 3/21/17 – I had a question about SOBR – if you want you can in fact use SOBR in this ReFS enabled situation as we talk about above, but make sure to use Data Locality mode. If you use Performance Mode, Veeam will write bits in a variety of places and it may mean that ReFS doesn’t get to do its thing as it cannot connect the bits to the metadata – the bits might be on a different server.
Questions or comments always welcome,
=== END ===