How to Find duplicate copies of files Using fdupes in Ubuntu

Source: http://www.ubuntugeek.com/how-to-find-duplicate-copies-of-files-using-fdupes-in-ubuntu.html

FDupes or fslint GUI

FDupes uses md5sums and then a byte by byte comparison to find duplicate files within a set of directories. It has several useful options including recursion.

Install fdupes in ubuntu

sudo aptitude install fdupes

This will install all the required packages for fdupes

Using fdupes

Fdupes syntax

fdupes [ options ] DIRECTORY

Available Options

-r --recurse - include files residing in subdirectories

-s --symlinks - follow symlinked directories

-H --hardlinks - normally, when two or more files point to the same disk area they are treated as nonn-duplicates; this option will change this behavior

-n --noempty - exclude zero-length files from consideration

-f --omitfirst - omit the first file in each set of matches

-1 --sameline - list each set of matches on a single line

-S --size - show size of duplicate files

-q --quiet - hide progress indicator

-d --delete - prompt user for files to preserve, deleting all others

-v --version - display fdupes version

-h --help - displays help

Fdupes Examples

1) fdupes -r ./stuff > dupes.txt

Then, deleting the duplicates was as easy as checking dupes.txt and deleting the offending directories. fdupes also can prompt you to delete the duplicates as you go along.

2) fdupes -r /home/user > /home/user/duplicate.txt

Output of the command goes in duplicate.txt

fdupes will compare the size and MD5 hash of the files to find duplicates

You can also use fslint to do this job this is simple GUI tool to do this.