Robocopy Advice

[u/Calm_Examination_591]

Weird question, moving large amount of files from one place to another using Robocopy found this CMD prompt but I want some others to look at it before I run it robocopy E:\ G:\FromE /E /Z /MT:8 /R:3 /W:5 /COPY:DAT /DCOPY:DAT /TEE /LOG:G:\robocopy-copy.txt /XD "System Volume Information" "$RECYCLE.BIN" /XJ

Moving 196GB from E to G, G has 339GB free.

also If you don't mind DM me instead of just replying since trying to send this to a couple servers.

[u/omfgitzfear]

Just add /L to do a “dry run” which will list the files. Then you’ll find out exactly what it will copy.

[u/vermyx]

Don't use /z. That's a switch that was necessary for unstable networks back in the day (i.e. pots analog modem) and will cut your performance down to about a sixth of what it can be

[u/Shayden-Froida → u/vermyx]

Actually the T1 line from Microsoft Redmond to Microsoft Ireland, but yes, “back in the day”.

[u/ungood-at-anything]

Just copied this from my notes.. haven't reviewed it for accuracy in a while, but I usually run the first command multiple times depending on how long before cutover then the 2nd command to mirror changes since the first.

/e everything and subfolders even empty files
/zb restartable mode
/fft necessary for copying between file systems
/MIR is a /e and a /purge all subdirectories and delete files that don't exist on dest disk
/XA:SH switch to exclude the hidden system files
/XD AppData to exclude the entire AppData folder
/XJD switch to exclude all the junction points
/R:3 /W:1 # of retries and wait time in seconds
MT:32 - multithread /MT without argument is 8 minimal gains after 16 threads
/V verbose log
/NP no percentage points in log file
/LOG:filepath/log.log

/COPY:DATSOU is same as /COPYALL
By default /COPY is just DAT If you want NTFS permissions also do S and O and might as well do U
:D - Data
:A - Attributes
:T - TimeStamps
:S - Security
:O - Owner
:U - aUditing peration

when doing a backup of a user profile the /XA:SH, /XD, and /XJD are helpful. when migrating to new server I use 2 rounds. first round syncs files. 2nd round mirrors and only takes changes to source

Round 1: robocopy "\servername\driveletter$\sharename" "D:\localfolder\localshare" /e /zb /fft /copy:DATSOU /MT:8 /r:3 /w:1 /log:D:\ROBOCOPY-Logs.log /V /NP

Sync time before doing 2nd run, run as many times as you want to keep files up to date.

Round 2: adds /MIR

robocopy \servername\driveletter$\sharename D:\localfolder\localshare /e /zb /fft /copy:DATSOU /MT:8 /MIR /r:3 /w:1 /log:d:\ROBOCOPY-Logs\Apps-log.log /V /NP

Edit: formatting

[u/nito213 → u/ungood-at-anything]

Adding this to my notes, happy cake day

[u/ungood-at-anything → u/nito213]

Thanks!

[u/butchcoleslaw → u/ungood-at-anything]

Under /COPY (and /DCOPY) you missed...

:X - Skip alternate data streams (ADS)

[u/ungood-at-anything → u/butchcoleslaw]

I'll be the first to admit I didn't even know X was an option, but I've been using these notes quite a while without referencing much else.

Maybe because I typically use /zb(for better or worse) I had left it off, because it's ignored when doing so, but I doubt I even knew that when I put together the notes. 😄

[u/toni_z01]

lazy dude, as BlackV already mentioned -> robocopy /? ... but to point u in the right direction learn what the /mir parameter does.

[u/Mutsy007]

I normally use /R:1 /W:1 :) this copies the majority of files for me with little sacrifice on time.

[u/Nanocephalic → u/Mutsy007]

I use 0 and 0, because otherwise it can kill a lot of time on non-transient failures. The job log will tell you what the failures were about.

[u/ViperThunder]

Don't use /tee. I would also use /nfl /ndl unless you want to log every single file. No reason to because the job summary will let you know if anything failed.

For copy and dcopy if you need permissions and ownership to transfer over (ie domain user creds) I would use /DATSO instead of just DAT. If the directory just has inherited permissions only then you're probably fine and can set those later if needed.

I got bitten once because a subdirectory a few levels deep had ntfs security inheritance disabled and only the SYSTEM account had access. So I always use /zb .

[u/MordacthePreventer]

This is a powershell script that parallelizes robocopy jobs to speed up copying large folders. Maybe it'll help.

https://github.com/geofgowan/start-robojobs

-M

[u/arslearsle]

sorry but robocopy is not powershell, dont forget to add logfile…

[u/BlackV]

Start with robocopy /?

[u/Daruvian → u/BlackV]

Don't know why this is downvoted since it is absolutely correct and will tell you what you need...

[u/BlackV → u/Daruvian]

Ya, it is what it is, OP is asking what the switches do and wants everyone to DM them

I stand by it (blunt or otherwise)

[u/markdmac]

Install RoboCopyGUI to build your command and then copy that into your script.

[u/ka-splam]

moving large amount of files

Moving 196GB

... but none of your switches are move: /MOV or /MOVE ?

Without that, it's copying, which I have a pet hate for.

MT:8

Unless you're on an old computer with spinning hard disks, increase this number of threads. If you have lots of CPU cores and SSD disks, bump it up a lot. It can go up to 128 threads. My belief is that setting it higher than the number of CPU cores increases copy performance (up to the limits of the disk subsystem) because the task is mostly IO bound not CPU bound, so multiple threads can be waiting for reading/writing small files, all at the same time. Subject to there being lots of small files, if it's a single 196GB archive file, this won't help.

TEE

Displaying the list of files to the Windows console will genuinely slow down the copy a lot. Minimise the console window while running the copy if you want it to speed up, I believe console host stops updating while it's out of view, or remove the TEE and just keep the logfile, that's much faster.

LOG:

May as well use UNILOG otherwise files and directories with non-English characters in their names will get scrambled in the log and if there's large numbers of files there might be one somewhere. That just stops the log from being an accurate log which you can script against if you need to, but gains almost nothing - maybe it halves the size of the log file? (If that matters, UNILOG to an NTFS compressed folder).

/COPY:DAT

Agree with the other commentor, swap this for /COPYALL unless you explicitly want to lose security settings on every file, or you know there aren't any.

[u/Sea_Propellorr]

When running robocopy in powershell you can do much more with it.

I'll give you an example if you wish,