Any suggestions for a tool (or set of tools) to migrate filesystems into or out of HNAS clusters?
Would need to be able to handle 500 Million files in about 50k directories.
If you are looking for a costing solution based on HNAS, then you can go with (IDR) Intelligent data replication for file level and (IBR) - Incremental Block-level replication, or Object Replication also called as Jetmirror.
For Host based solution - following are the options
1. rsync - (Mostly used in Unix/Linux environments / NFS and free)
2. robocopy - for windows environments / CIFS and free
3. securecopy - for windows environments / CIFS license required.
I think that IDR, IBR, and Jetmirror only work HNAS to HNAS. (please let me know if I'm wrong on that)
Rsync and Robocopy would possibly work OK, if I had unlimited (or a LOT) of time for cutover. The sync process can definitely occur in the background, but a final scan through 500M files to look for changes takes way too long.
Not sure on Securecopy (sounds somewhat promising: Data Migration & File Server Consolidation Software | Dell Software but would have to check on whether it does things in a similar way and would have the same problem because of the large number of files)
Retrieving data ...