Text Manipulate Tool
Manipulate text with various case conversions and view detailed character statistics
Remove Duplicate Lines
Remove repeated lines from any list. Ignore case, trim spaces and sort the result. Free and private.
0 lines in · 0 duplicates removed · 0 left
About This Tool
NinjaFunction's Remove Duplicate Lines tool cleans a list in your browser. Paste one item per line and get every line once, in its original order. Choose to ignore capital letters, trim spaces, drop empty lines and sort the result from A to Z, Z to A or by length. It shows how many duplicates were removed. Your list is never uploaded.
🔑 How to Use
- Paste your list into the left box, with one item on each line.
- Choose options: ignore case, trim spaces, remove empty lines.
- Pick a sort if you want the result in order, or keep the original order.
- Check the count under the result to see how many duplicates were removed.
- Click Copy and paste the clean list where you need it.
💡 Pro Tips
- ✔ Clean email lists: tick Ignore case so Name@x.com and name@x.com count as the same address.
- ✔ Trim spaces is on by default because trailing spaces are the most common hidden difference.
- ✔ Sort A to Z makes it easy to spot near-duplicates side by side.
- ✔ Need to compare two texts instead? Try the Text Diff Tool or the Text Manipulate Tool.
When Is This Useful?
- Email and phone lists: remove repeats before importing into a mailing tool.
- Keywords: clean a keyword list for SEO or ads.
- Code and logs: list each unique error message or file name once.
- Spreadsheets: copy a column, remove repeats and paste it back.
How the Tool Compares Lines
Each line is compared exactly, character by character. With Ignore case on, capital and small letters are treated the same. With Trim spaces on, spaces at the start and end are removed first, so ' apple' and 'apple' match.
A Worked Example
| Your list | Settings | Result |
|---|---|---|
| apple, Apple, apple, banana | Defaults | apple, Apple, banana |
| apple, Apple, apple, banana | Ignore case on | apple, banana |
| ' apple', 'apple' | Trim spaces off | Both stay (they differ by a space) |
| ' apple', 'apple' | Trim spaces on | apple |
Common Mistakes When Cleaning Lists
- Hidden spaces. Text copied from a web page or spreadsheet often has a space at the end of a line. The lines look the same but are not, so they are kept. Turn on Trim spaces.
- Different capital letters. Email addresses are not case-sensitive, so Sam@mail.com and sam@mail.com are the same person. Turn on Ignore case for emails, tags and keywords.
- Sorting before checking. If order matters, such as a ranked list, do not sort. The tool keeps the first copy of each line in its original place.
- Lines that are nearly the same. 'New York' and 'New York' (two spaces) are different. Sort A to Z to put near-duplicates side by side, then fix them by hand.
Ideas for Using It
- Merge two mailing lists: paste both, then remove duplicates.
- Make a clean list of unique domains, tags or product codes.
- Find the unique values in a column copied from Excel or Google Sheets.
- Tidy a robots.txt, hosts file or ignore file by removing repeated rules.
Frequently Asked Questions
How do I remove duplicate lines from a list?
Paste your list into the box, one item per line. The result on the right shows each line once, in the order it first appeared. Click Copy to use it.
Does it keep the first or last copy of a repeated line?
It keeps the first copy and removes the later ones, so your original order is kept unless you choose to sort.
Is 'Apple' the same as 'apple'?
Not unless you tick Ignore case. By default the tool treats them as two different lines.
Why do lines that look the same stay in the list?
Usually one has a space at the end. Keep Trim spaces ticked to remove spaces at the start and end of each line before comparing.
Is my list uploaded anywhere?
No. The lines are compared in your browser and are never sent or stored.