[ About | Licence | Contacts ]
Written by Oleksandr Gavenko (AKA gavenkoa), compiled on 2017-10-13 from rev 5ee52f8d25b5.

Duplication.

Search for duplicate lines.

http://en.wikipedia.org/wiki/Duplicate_code
Wiki page.
http://students.cis.uab.edu/tairasr/clones/literature/
Code Clones Literature.
https://moz.com/devblog/near-duplicate-detection/
MinHash vs SimHash algorithm explanation.

Open source or free licence:

Proprietary or restricted licence:

http://stackoverflow.com/questions/191614/how-to-detect-code-duplication-during-development
How to detect code duplication during development?
https://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis
List of tools for static code analysis.
http://askubuntu.com/questions/434545/identify-duplicate-lines-in-a-file-without-deleting-them
Identify duplicate lines in a file without deleting them?
http://stackoverflow.com/questions/13046791/how-to-delete-the-repeat-lines-in-emacs
How to delete the repeat lines in emacs.
http://emacs.stackexchange.com/questions/13092/how-can-i-highlight-duplicate-lines
How can I highlight duplicate lines?
https://www.emacswiki.org/emacs/DuplicateLines
Duplicate Lines.

Search for duplicate files.

This utilities only search for duplicate files:

http://duff.sourceforge.net/
duff home page
http://freedup.org/
freedup home page
http://dupedit.com/
dupedit home page
http://rdfind.pauldreik.se/
Rdfind home page
http://code.google.com/p/softenido/wiki/FindRepe
FindRepe home page

fdupes

$ sudo apt-get install fdupes

See:

http://code.google.com/p/fdupes/
fdupes home page
http://ru.wikipedia.org/wiki/Fdupes
fdupes wiki page
http://packages.debian.org/search?keywords=fdupes
fdupes Debian package

freedups

Freedups searches through the directories you specify. When it finds two identical files, it hard links them together. Now the two or more files still exist in their respective directories, but only one copy of the data is stored on disk; both directory entries point to the same data blocks.

http://www.stearns.org/freedups/
freedups home page

dupmerge

Dupmerge reads a list of files from standard input (eg., as produced by "find . -print") and looks for identical files. When it finds two or more identical files, all but one are unlinked to reclaim the disk space and recreated as hard links to the remaining copy.

https://sourceforge.net/projects/dupmerge/
dupmerge home page

ssdeep

ssdeep is a program for computing context triggered piecewise hashes (CTPH). Also called fuzzy hashes, CTPH can match inputs that have homologies. Such inputs have sequences of identical bytes in the same order, although bytes in between these sequences may be different in both content and length.

http://ssdeep.sourceforge.net/
ssdeep home page

comparator

Available under Cygwin. Find duplication in source files:

$ comparator -s 5 $dir1 $dir2