File Migration Series – Planning

Hello all,

I continue today with what I started recently – a series on moving to new storage.  Sorry about the delay!  This article will be about some of the specific planning you need to do.  Before you start you need to answer a number of questions.

Questions to think about

• Do I have a diagram?  Probably not yet but as you answer the questions it will help you build that diagram.
• How much time will I need?  This is related to how much data do I have?
• Does the data have metadata that needs to be preserved?  Normally this is yes, but a no makes things much easier!
• What tools do I need?  There is not many to choose from, but they do have different parameters and support so it is important to choose right.
• Am I preserving directory structure – meaning just moving it to a new location but keeping the same structure?  This is likely yes, or maybe even a heck yes.  Normally we keep the structure to make it easier for people to deal with in the new location.  I always tried to make it transparent.  Meaning that the user(s) did not know that last week they were on storage X and today on storage Y.
• Where is the GPO / Login Script that sets up drive letters for people and any folder redirection as well.
• The most important thing to be aware of, meaning the one if you skip that will haunt you afterwards, is this: does anyone have Excel spreadsheet (or any app for that matter) that has file locations inside of it?  This is most likely in finance first, and then HR.  But these spreadsheets will need to be updated after the move and in my experience half of the users had forgotten how they had done it in the first place so they needed help to fix.  Fixing did not take long if you knew what had changed.  In any communication to users about this migration you need to have a short section about the change for people have files referenced in their spreadsheets.
• I also like to find a bunch – say 10, of the important files / folders.  After testing, and then after the actual migration, I will use these 10 examples to confirm that the file and metadata is safe and where it should be.
• Communications plan – you need potentially to think carefully on this.  Who to notify, what to say, how much detail, and what is the impact is very important to understand and detail.  Plus, also handy to mention it would be good if they logged out from their computer each night to help us do good copies.

Answers to think about

Here are some sample answers that work for me, and I think with some slight tweaks maybe work for you.  But if nothing else, more things to think about for your migration.

I like to have a diagram, so here is a simplified one as an idea.  It only shows the first page, but the second would detail some of the permissions and owners to help with testing.  Another page would list the people who own the ‘special’ files.  Another one would have the AD / GPO related info like login script and drive letter info.  Click on the diagram to see it in more detail.

• The next thing to have an idea of is how long will the copy take?  Will it take more then a night or a weekend? Often in my career it did.  So that means I use one or two weeks of copying at night.  But before that comes the test copy.  I have often used Robocopy (intro and commands) for this stage as it was always easy to find.  So pick a subset of the data and copy it.  Remember we want to avoid the issue of open files, or bothering people but get a very rough idea of how quick we can copy things.  Remember that this is a very rough idea of copy speed.  File size, and number of files will really impact time.  As a result of this I will generally use a two day window minimum for doing the copy.  If you have any budget at all, and if this will be a big migration effort then use Secure Copy.  A lot more options with it – even an Admin override that is handy, and a range of features such as reports and the option for email reports.  It is more full featured and faster as well.  Below is a sample RoboCopy command-line – be aware that we will talk more about the actual migration and tools in the next article.

• I will (and so will most of you) almost always have metadata to deal with.  This includes the obvious like file and folder date / time, file and folder owner, and permissions.  But there are things like extended attributes and audit information as well.  Both of these are a little unusual, but audit information is sometimes necessary to be copied.  Some arrays like NetApp and EMC will have extended attributes off by default so no need to worry.  I would strongly suggest in the previous point where I talk about how long things will take, that you use that opportunity to make sure that the metadata that is important to you is copied.  See below a common error that is connected to metadata.  In this case the owner of a file on the source doesn’t exist on the destination.  This means that the file will end up being owned by the account that is used for the Robocopy.  This is often not good but it is what you get from Robocopy in a number of error situations and what other tools such as Secure Copy give you options to handle – if you are lucky even a mapping capability!

• We also have to decide on tools that we will use.  I have already mentioned Robocopy, (command-line, and examples) but I normally almost always used Secure Copy. The range of features it has, and the performance too, is very handy indeed.  It has a collection of utilities that it did not have in the past when I worked with it.  One is particularly useful – remapping home folders.  When you move files to a new storage platform, often you need to move users too and that means their home folders will need to change so it is nice to have a utility to do that rather than PowerShell or manual.  I also like the reports and dashboard.  In the next article we will use a tool and spend more time on it but this is the stage where you decided which tool.  Another tool I have heard of people using successfully lately is one new to me – SyncBack.
• I am definitely going to preserve the folder structure.  In fact, after the copy – or should I say mirror is complete, I will in fact use the old drive letter on the new storage.  The idea is to make as little impact on my users as possible.
• It is important to know where the login script is and how it works.  There are a variety of ways to do that and so you need to confirm where it is.  Generally there is a login script that defines the S: that points to peoples shared folders and that likely needs to be updated.  If folder redirection is in use, and it certainly should be, then you will need to know how that works so you can update it.  We will pay particular attention to this in our next article.
• In my case there is no Excel spreadsheets that have UNC paths in it that will frustrate me to no end.  Generally in my past there was, in particular in economics / finance where someone was doing the super budget spreadsheet and it had links everywhere.  It is key to know about that in advance.  There has also been PR people doing the same thing but with PowerPoint.  So watch out!
• And yes, I will have a list of files, and folders that I know the correct metadata info to spot check and make sure they have the right ACL and Owner info.  As a matter of fact you can do a PowerShell dump of the file info that may be useful.  I normally kept the source locked down but available in case of issues.

So you have gone through some questions to think about, as well as some answers to think about.  You should do a task plan up now so that you can make sure you have covered everything off!  I suspect you wanted me to do your task plan?  Sorry, but I think that would have been not good for you.  I think you should now have the info you need to do your task plan and it will be better since you did it.

 Useful Links

• Robocopy commands – http://technet.microsoft.com/en-us/library/cc733145(WS.10).aspx
• Robocopy example commands – http://social.technet.microsoft.com/wiki/contents/articles/1073.robocopy-and-a-few-examples.aspx
• Secure Copy – http://www.quest.com/secure-copy/
• Secure Copy tour – http://www.quest.com/secure-copy/walkthrough/default.html
• This series – https://notesfrommwhite.net/tag/file-migration/
• SyncBack – http://www.2brightsparks.com/syncback/syncback-hub.html

I hope that this helps, and we will see you in the next one!

BTW, the first article in this series is here, and you are reading the second.  You can watch the series from here as well.

Michael

=== END ===