Clean a list without changing its values

To clean a list while preserving how the retained items are written, work on a copy, remove only unnecessary lines and start with exact duplicates. Treat sorting and whitespace removal as separate choices. An uppercase letter, a leading zero or a space may be part of an identifier.

Decide what can be removed

Save the original before making any changes. Check that each line really is a separate item. A repeated entry may represent several actual transactions; an empty line may separate groups. Removing them would then change the information, even if the text on the remaining lines stays intact.

Do not paste structured CSV directly into these line-based tools. A quoted field can contain a line break, and its spaces are part of its value, as described in RFC 4180. For that format, see the CSV-to-JSON walkthrough instead.

Remove only the lines you do not need

The Blank Line Remover removes lines with no characters and lines consisting entirely of whitespace characters recognized by JavaScript.

An ordinary space is one specific character; “whitespace” also includes tabs and some Unicode spaces, including the non-breaking space. The tool preserves spaces on lines that contain text. However, an invisible character is not necessarily recognized as whitespace: a line containing a zero-width space U+200B remains.

Choose the right duplicate-matching rule

The Duplicate Line Remover offers four options:

• “Exact text (default)” preserves differences in case and spacing.
• “Ignore letter case” matches uppercase and lowercase variants according to the browser’s language.
• “Ignore leading/trailing whitespace” compares lines without their outer whitespace.
• “Ignore case and outer whitespace” combines both relaxed matching rules.

In every mode, the first matching line stays exactly as written in the input. Ignoring its whitespace during comparison does not remove that whitespace from the result. Internal spaces remain significant. Visually identical accented characters with different Unicode encodings are not automatically normalized for deduplication.

Test with sample reference codes

Here is a short list. The middle dot · represents an actual space, and ⇥ represents a tab; these symbols are used only to make the example readable.

REF-0010

·REF-0002·
REF-0002
REF-0002
ref-0002
REF-2
·⇥

Removing blank lines deletes the second and last lines. Exact matching then removes only the second identical occurrence of REF-0002. It keeps REF-0010, ·REF-0002·, REF-0002, ref-0002 and REF-2, in that order.

On the list with blank lines removed, the mode that ignores whitespace keeps ·REF-0002· and removes the two following REF-0002 entries. The lowercase variant remains. With the combined option, that variant disappears too. REF-2 remains distinct from REF-0002: leading zeroes are not removed.

If you actually want to remove outer whitespace, use Trim Lines. This step changes the values. It leaves internal spaces and empty lines in place. Apply it only if that whitespace is unwanted, then repeat exact deduplication if needed. The behavior is based on String.trim().

Sort only if the order can change

Sort Lines Alphabetically offers “As text (item10 before item2)” and “By numeric value (item2 before item10)”. With REF-10 and REF-2, these modes give REF-10 followed by REF-2, and the reverse order, respectively.

Numeric mode compares numbers within the text without rewriting the reference codes: leading zeroes remain. Lines that compare as equal keep their original order; empty lines move to the top.

Both modes follow the browser’s language rules, ignore letter case and may treat some accented characters as equivalent. The ordering is therefore not a strict character-by-character comparison. The Intl.Collator documentation explains these differences between languages. Keep the original order if it represents a timeline or a priority.

Checks before replacing the original

• The original is saved separately.
• Every removed line was meant to be removed.
• Letter case, meaningful spaces and leading zeroes are preserved.
• The result has been tested in its destination application.

Frequently asked questions

Why is an apparent duplicate still there?

Check whitespace, letter case and Unicode characters. Exact mode respects those differences; do not change the rule without understanding their purpose.

Should I always remove whitespace before duplicates?

No. Start with exact comparison. Sometimes ignoring whitespace is enough to deduplicate while preserving how the first occurrence is written.

Does local processing mean no network connection?

Processing runs in your browser. Opening the page loads code and resources from the website: it does not guarantee browsing without network requests.