RichCopy under the covers – File Migrations exposed!

I’m sure by now you’ve read my introduction to RichCopyIntroducing RichCopy – your Robocopy replacement!  Well, today I spent some time doing speeds and feed tests over numerous protocols in my lab environment.  I want to establish a disclaimer that this is no way a reflection of the ‘speed’ you will get, or not get respectively – More that the sheer differences between single threaded operation (re:xcopy, robocopy, rsync for windows) will be infinitely improved when using this tool! Oh, and I’ll also go into some of the cool depth you can cover!

I ran a number of very small file copies (<4k file size files) in the 15,000 to 90,000 file range and got the following results!

Speeds and Feeds tests!

The specific items I want to call out here are – when doing a single threaded copy (such as the CIFS job which took 57:16 – well, frankly that is as close to a ‘robocopy’ or ‘xcopy’ we’d get and when you compare it to slightly increasing the number of simultaneous threads – OMFG look at the results! Cut down by a lot of the time! Imagine if that were your production data migration – Cutting that seriously by 1/6th or even more depending upon the workload! The same can be said if you have ‘changed’ data that it is looking to update only the changed blocks.  So much potential!  This particular host was a slooooow VM which can somewhat explain why in general it was slow.  However the slowness is reflective across the board, so the 1/6th speed improvement is likely to be the same if not better in any other situation or circumstance! 

I definitely encourage checking it out, as well as the numerous advanced features this tool offers.

These advanced features… I’m going to discuss in a few moments as well ;)

For those of you who are new to RichCopy – Honestly you can get by with the simplest of actions by simply modifying your thread numbers.  But if you want to get into the special details of it, let me reveal to you a number of VERY cool functions and features.  Oh and might I add, ADVANCED features that let you take it up a notch, not to mention the amazing HELP file too!

Starting with the mothership of all functions! The Advanced option! Which enables a whole series of new options enabled when you go to your File Copy options!

Taking RichCopy to the Advanced! File Copy Options Process Control

The Process Control lets you really dive down into some functions such as your cache sizes and more! Really quite cool.  Though frankly as I jump to the “Others” section, one of the coolest features if the fact that when you modify your options in the GUI, you are presented with your CLI parameters!

Command Line Parameters - CLI so cool!

Though don’t get me wrong, File Attributes, Error handling, inclusion and exclusions, FTP and other things themselves –are- quite cool!

File Attributes and Error Handling Inclusion (Exclusion looks the same, so I didn't dup it!) FTP, Other and more!

Though hands down, the most important part of this tool is the fact that it EXISTS, it is FREE and it is fricking COOL!   So, there’s no reason to sit idly by paying for SecureCopy, or watching your serial operations SLOWLY move by as you migrate data! Get out there, and play around with RichCopy today! It’ll make you wish you had this tool for the past 10 years like MS had!

Introducing RichCopy – your Robocopy replacement!

Alright, for those of you in the know, we all know that RichCopy isn’t anything new.  Infact it’s been being used internally at Microsoft for the past 10 years – however after a near decade of ‘Seriously?! We haven’t released this to the public?!’ it is indeed now available!   There’s a whole slew of details at others blogs RichCopy Build 4.0.216 has been posted to the Microsoft Download Center  and the TechNet Magazine article Utility Spotlight RichCopy and especially, be sure to visit the creator of this amazing tool Ken Tamaru’s blog!

So, what is this amazing tool?!  It’s free first of all! And allows you even more granular control over your copy processes, including multi-threaded copy operations!

As you can see here in this image, I’m copying a number of various sized files in different locations simultaneously!    Though one of the major perks is not in the initial copy, but instead in the situation of changed file copies (in our common “what’s changed incremental” model? :) Well, notes about that, taken verbatim from Ken’s blog show:

RichCopy 10 threads 

There must be many users who use RichCopy to copy only updated files. Most of users assign only 1 thread for directory search; however you can dramatically accelerate the performance of source and destination comparison by assigning multiple threads, especially this works well when files are distributed to multiple directories as RichCopy assign 1 thread to each directory search, not a tree.

Here is an example. (local to local, but different storage)

(1 million files in source and destination)
Thread # for Directory Search
1 about 10 minutes
2 about 6 minutes
4 about 2 minutes
8 about 1 minutes

RichCopy Options

Hands down, one of the coolest things is the level of options you can set.

The specific options I’d like to highlight are number of threads you can assign to specific operations.  That way you can increase not only the number of directories you traverse looking for files (or changed files) but also the number of files you can copy simultaneously!  This is a lifesaver when you’re copying many small files which when operating sequentially tends to take a lifetime!

 

I did run a number of speed tests on my machine, however speeds when run from my SSD tend to suffer, compared to if I were copying from say, one SAN to another SAN (something more realistic especially in a migration scenario)

 

In any sense, it’s a great tool, which operates via CLI or GUI and I’ll be sure to use or introduce it into my future migration opportunities as applicable :) Oh and be sure to click on any instance of “RichCopy” in this post in order to get a direct link to its download! :)