X-Git-Url: https://www.fleuret.org/cgi-bin/gitweb/gitweb.cgi?p=finddup.git;a=blobdiff_plain;f=finddup.1;h=896cc88a4090a8fc9e85f965bd52424d56d60c8e;hp=f88f42fddb0a9b77b6b8a0997091f52fd05be0be;hb=HEAD;hpb=da07c4f78dce0021ed889ea0db818cb55d96ceed diff --git a/finddup.1 b/finddup.1 index f88f42f..896cc88 100644 --- a/finddup.1 +++ b/finddup.1 @@ -1,4 +1,4 @@ -.TH "FINDDUP" 1 "Mar 2010" "Francois Fleuret" "User Commands" +.TH "FINDDUP" "1.2" "Apr 2011" "Francois Fleuret" "User Commands" \" This man page was written by Francois Fleuret \" and is distributed under a Creative Commons Attribution-Share Alike @@ -10,27 +10,136 @@ finddup \- Find files common to two directories (or not) .SH "SYNOPSIS" -\fBfinddup\fP [OPTION]... DIR1 [[not:]DIR2] +\fBfinddup\fP [OPTION]... [DIR-OR-FILE1 [[and:|not:]DIR-OR-FILE2]] .SH "DESCRIPTION" -With a single directory argument, \fBfinddup\fP prints the duplicated -files found in it. With two directories, it prints either the files -common to both DIR1 and DIR2, or with the `not:' prefix, the ones -present in DIR1 and not in DIR2. +With one directory as argument, \fBfinddup\fP prints the duplicated +files found in it. If no directory is provided, it uses the current +one as default. + +With two directories, it prints either the files common to both DIR1 +and DIR2 or, with the `not:' prefix, the ones present in DIR1 and not +in DIR2. The `and:' prefix is assumed by default and necessary only if +you have a directory name starting with `not:'. Files are handled like +directories containing a single file. + +This command compares files by first comparing their sizes, hence goes +reasonably fast. + +When looking for identical files, \fBfinddup\fP associates a group ID +to every content, and prints it along the file names. Use the \fB-g\fP +to switch it off. + +Note that +.B finddup DIR +is virtually the same as +.B finddup -i DIR DIR .SH "OPTIONS" .TP -\fB-h\fR -display help and exit +\fB-v\fR, \fB--version\fR +print the version number and exit +.TP +\fB-h\fR, \fB--help\fR +print the help and exit +.TP +\fB-d\fR, \fB--ignore-dots\fR +ignore files and directories starting with a dot +.TP +\fB-0\fR, \fB--ignore-empty\fR +ignore empty files +.TP +\fB-c\fR, \fB--hide-matchings\fR +do not show which files from DIR2 correspond to files from DIR1 +(hence, show only the files from DIR1 which have an identical twin in +DIR2) +.TP +\fB-g\fR, \fB--no-group-ids\fR +do not show the file group IDs +.TP +\fB-t\fR, \fB--time-sort\fR +sort files in each group according to the modification times +.TP +\fB-q\fR, \fB--trim-first\fR +do not print the first file in each group +.TP +\fB-p\fR, \fB--show-progress\fR +show progress information in stderr +.TP +\fB-r\fR, \fB--real-paths\fR +show the real path of the files +.TP +\fB-i\fR, \fB--same-inodes-are-different\fR +files with same inode are considered as different +.TP +\fB-e \fI\fR, \fB--exec \fI\fR +execute the provided command for each group of identical files, with +their names as arguments .TP -\fB-r\fR -shows the real path of the files +\fB-f \fI\fR, \fB--result-prefix \fI\fR +for each group of identical files, write one result file whose name is +the given prefix string followed by the group number, and containing +one file name per line .SH "BUGS" -Every pair of different files with same content are listed, which -results in K^2 printed lines when K files are similar. +None known, probably many. Valgrind does not complain though. + +Since files with same inodes are considered as different when looking +for duplicates in a single directory, there are weird behaviors -- not +bugs -- with hard links. + +The current algorithm is dumb, as it does not use any hashing of the +file content. + +Here are the things I tried, which did not help at all: (1) Computing +md5s on the whole files, which is not satisfactory because files are +often not read entirely, hence the md5s cannot be properly computed, +(2) computing XORs of the first 4, 16 and 256 bytes with rejection as +soon as one does not match, (3) reading files in parts of increasing +sizes so that rejection could be done with only a small fraction read +when possible, (4) using mmap instead of open/read. + +.SH "WISH LIST" + +The format of the output should definitely be improved. Not clear how. + +There could be some fancy option to link two instances of the command +running on different machines to reduce network disk accesses. This +may not help much though. + +.SH "EXAMPLES" + +.B finddup -p0d blah + +.fi +List duplicated files in directory ./blah/, show a progress bar, +ignore empty files, and ignore files and directories starting with a +dot. + +.B finddup -qtg + +.fi +List all files which are duplicated in the current directory, do not +show the oldest in each each group of identical ones, and do not show +group numbers. This is what you could use to list what files to +remove. + +.P +.B finddup sources not:/mnt/backup + +.fi +List all files found in \fB./sources/\fR which do not have +content-matching equivalent in \fB/mnt/backup/\fR. + +.P +.B finddup -g tralala cuicui + +.fi +List groups of files with same content which exist both in +\fB./tralala/\fR and \fB./cuicui/\fR. Do not show group IDs, instead +write empty lines between groups of files of same content. .SH "AUTHOR"