Remove Duplicate Lines

Clean up a mailing list, a keyword list, log lines or any pasted column by removing lines that appear more than once. You decide whether the first or the last copy survives, or whether every repeated line should go, whether “Apple” and “apple” count as the same, and what happens to empty lines. A table shows which lines were repeated and how often.

How to use this tool

1. Paste your list, one item per line.
2. Choose which copy to keep, or keep only the lines that appear once.
3. Tick “ignore case” and “ignore spaces” if small differences should not matter.
4. Optionally sort the result, then copy it.

Formula

Two lines are duplicates when their comparison keys match: key = line (trimmed if chosen, lower-cased if case is ignored)

The original order is kept unless you choose to sort. “First copy” keeps each line where it first appears, “last copy” where it last appears, and “only unique” removes every line that has a twin. Empty lines are removed by default; they can also stay in place or be treated like any other line. The output keeps the wording of the copy it keeps, so with “ignore case” the first spelling wins.

Example

The sample has 7 lines: apple, banana, Apple, cherry, banana, “ cherry” and banana. With the defaults (first copy, case matters, spaces ignored) the result is apple, banana, Apple, cherry: 3 lines removed, and the table shows banana found 3 times and cherry 2 times. Ticking “ignore case” also removes Apple.