How do I know how big to make the filesystem?
The size of filesystem or “cache” depends on a few things.
First, keep in mind that that the stubbing task runs approximately every hour.
That means, if the filesystem has less than 10% free (default) , it will start stubbing the files in chronological order of the last update, date and time.
Second, the Migration task can run every 15 minutes, but please do not do that. It puts additional burden on the system every 15 minutes.
This is really about Recovery Point Objective. I would recommend 12am and 12 pm.
Sizing needs to be, the amount of ACTIVE files that will be held on the local HDI, AND enough free space to accept NEW files for that HOUR between stubbing jobs.
(note: files can only be stubbed if they were successfully migrated.)
As an example, using the default stubbing setting of 10% free. A 4TB file system would have 400GB of free space.
If a user started copying all of his music to the HDI, every hour the Stubbing process would start trying to maintain the 10% free by stubbing old files.
If the user was to put OVER 400GB within the hour, BEFORE stubbing could take make room, they would get Disk Full message.
The stubbing limit can be changed via command line arcreplimitset (Set the threshold at which stub processing is performed)
If the HDI was used to store database dumps, you may want to set the arcreplimitset to 50% or maybe even 25% depending on the amount it ingests hourly.
As noted previously, only files that have been migrated, can be stubbed. SO if you dumped 4TB of NEW files in an HDI before migration ran, stubbing could not stub any of those files, and you would get disk full.
Additionally If you are replacing an existing system, Windows Server or NAS, a good place to start is the Incremental backups.
This will tell you the change rate of files every day. So your HDI file system should be larger than the largest incremental.
Also using the 80/20 rule can be helpful. 80% of files are inactive and 20% is active.
So it comes down to how much data is accessed daily, AND how much is new data ingested between migrations and stubbing.
Its tempting to make the migration very frequent, but keep in mind it will be competing with the users for resources.
Retrieving data ...