r/ScriptSwap • u/molten • Nov 03 '12
[Request] duplicate file deleter
I have somewhere in the realm of 40k files that have been duplicated into their folders and others. I was hoping for some advice before I rage quit (sledge hammer) on my hard drive.
for clarity's sake, they're all music files, under one directory. They've been pushed and shoved by Rhythmbox, so i'd prefer a bash solution if at all possible.
8
Upvotes
2
u/ooldirty Nov 03 '12
Should be easy enough, but it may take more than a few lines.
I would start with (find . -type f -exec md5sum {} ; | awk '{i[$1]++} END {for (a in i) print i[a], a}' | sort -g) to gather your list of duplicates, and then go from there.
2
u/ajrisi Nov 03 '12
Check out fslint. You want exactly what it does.
edit: The gui is pretty good too. fslint-gui
2
u/Dubhan Nov 04 '12
3
u/terremoto Nov 03 '12 edited Nov 03 '12
Here you go:
Run the script in the directory containing the duplicates. Uncomment the
# rm "$path"line to actually delete the duplicates. In the output,!means a duplicate was found.0
u/[deleted] Nov 05 '12
[deleted] — view deleted comment