CSV deduplication guide
How to remove duplicate rows from a CSV safely
A duplicate is only meaningful after you choose the comparison rule. Compare every column for exact duplicate rows, or select keys such as Order ID. For each matching key, LocaCSV keeps the first row in input order.
Example: one row per order
An export contains Order ID O-102 twice. If the newest row should survive, sort by Updated at descending before deduplicating.
- Comparison key: Order ID
- Case-sensitive comparison
- Keep the first occurrence
Input with a repeated order
The duplicate has the same values, but key-based matching would also merge rows whose other fields differ.
Order ID,Customer,Amount,Status
O-101,Northstar Labs,128.00,Paid
O-102,Maple Supply,84.00,Paid
O-102,Maple Supply,84.00,Paid
O-103,Harbor Foods,62.00,PendingCSV after deduplication
The first O-102 row remains. Of the four input rows, three remain and one duplicate is removed.
Order ID,Customer,Amount,Status
O-101,Northstar Labs,128.00,Paid
O-102,Maple Supply,84.00,Paid
O-103,Harbor Foods,62.00,PendingRemove duplicates in five steps
Keep the original file until you have reviewed the result.
- Open Remove duplicates and choose one CSV up to 50 MiB.
- Confirm the encoding, delimiter, headers, and preview.
- Compare all columns or select one or more key columns.
- Choose whether letter case matters, then run the operation.
- Before downloading, confirm every input row is either kept or removed as a duplicate.
Verify the rows that remain
Confirm that three of the four input rows remain and one was removed, then sample the result to make sure the right row remains.
- Every input row is either kept or removed as a duplicate
- The first row for a repeated key remains
- Blank keys compare equal to other blank keys
- Spaces and look-alike characters written in different forms are not changed
Common deduplication mistakes
Deduplication does not perform fuzzy matching or clean inconsistent values.
- Using Customer Name alone and removing different customers
- Treating O-102 and o-102 as equal while case sensitivity is on
- Leaving trailing spaces that create separate keys
- Expecting the latest row to remain without sorting first
When to use this tool
Use deduplication to reduce repeated rows. Use Filter rows for a condition, or Merge CSV files to stack separate exports.
- Good fit: consistent order or customer IDs
- Good fit: fully identical rows
- Sort first: control which version remains
- Not a fit: fuzzy name or address matching