Published
Reading time
4 minutes

Find and Fix Hidden Whitespace in CSV Files

Hidden whitespace in CSV files breaks lookups, joins and deduplication. Learn to spot trailing, non-breaking and zero-width spaces, then fix them safely.

WhitespaceData cleaningCSV

You run a lookup and half the rows come back as not found, even though you can see the matching values right there. A pivot table shows Pro twice. A duplicate check misses obvious repeats. Very often, the cause is whitespace you cannot see.

This guide lists the characters responsible, explains how they break everyday work, and shows how to find and remove them without damaging the rest of your data.

The characters you can’t see

“Whitespace” covers more than the space bar. These are the characters that most often turn up in exported spreadsheets:

CharacterCode pointWhere it comes from
Leading or trailing spaceU+0020Typing, padded exports, copy-paste
Non-breaking spaceU+00A0Web pages, emails, word processors, PDFs
TabU+0009Text copied out of other tables or terminals
Zero-width space, joiner and non-joinerU+200B–U+200DWeb content, chat apps, rich text editors
Byte order markU+FEFFThe start of some UTF-8 files; occasionally pasted mid-text
Line breaks inside a cellU+000A, U+000DMulti-line addresses, notes and comment fields

Line breaks inside a cell are legal in CSV as long as the field is wrapped in double quotes, as RFC 4180 describes. This record has two fields, and the second spans two lines:

name,address
Ana Ruiz,"12 High Street
Springfield"

A proper CSV parser reads that as one row. A tool that splits on every line break reads it as two broken rows.

What hidden whitespace breaks

  • Lookups. An exact-match VLOOKUP or XLOOKUP for ACME-001 will not find ACME-001 .
  • Joins and merges. Matching two exports on customer ID or email fails silently for every key with a stray character, in spreadsheets, databases and scripts alike.
  • Duplicate checks. ana@example.com and ana@example.com are different values, so both survive. See how to remove duplicate rows from a CSV.
  • Filters, pivots and charts. Pro and Pro show up as two categories, splitting your counts.
  • Numbers stored as text. A number with a stray space or a non-breaking space may not be recognized as a number, so it is left out of sums. Some locales also use non-breaking spaces as thousands separators, so 1 200 may contain one.

How to detect it

In a spreadsheet

Add a helper column and compare lengths. If these disagree, the cell has extra spaces:

=LEN(A2)
=LEN(TRIM(A2))

Because Excel’s TRIM ignores non-breaking spaces, also check the first and last characters directly. =UNICODE(LEFT(A2,1)) and =UNICODE(RIGHT(A2,1)) return the code of each character; 32 is an ordinary space, 160 a non-breaking space and 8203 a zero-width space. UNICODE is available in Google Sheets and in Excel 2013 and later.

In a text or code editor

Many code editors can render whitespace as visible dots and arrows, and some highlight invisible Unicode characters by default. Opening a copy of the CSV this way quickly shows trailing spaces and tabs.

With a validator

The free CSV validator reports stray whitespace along with structural problems such as unterminated quotes, which matter here because a broken quote turns line breaks inside cells into broken rows.

Step by step: cleaning whitespace safely

  1. Keep the original. Work on a copy.
  2. Check the structure first. Make sure quotes are balanced and every row has the same number of fields, or later fixes will act on misaligned data.
  3. Trim every cell. Remove leading and trailing whitespace, including non-breaking spaces and tabs.
  4. Decide about inner spaces. Collapsing Ana Ruiz to Ana Ruiz is usually safe for names and addresses. It is not safe for fixed-width codes or any field where spacing carries meaning.
  5. Remove zero-width characters. They almost never belong in IDs, emails or codes. Be careful in free-text columns: the zero-width joiner is part of some emoji sequences, and the zero-width non-joiner is used in writing systems such as Persian.
  6. Decide about line breaks inside cells. Keep them in notes or addresses if the destination handles them; replace them with a space if the file feeds a system that expects one line per record.
  7. Re-test what was failing. Rerun the lookup, join or pivot and confirm the counts now line up.
Tip

Trim before you deduplicate or fill blanks. A cell that contains only spaces looks empty but is not treated as blank by many tools until it has been trimmed.

Fixing it with spreadsheet formulas

If you prefer to stay in a spreadsheet, these formulas cover the common cases. Put them in a helper column, check the result, then paste the values back.

Trim, treating non-breaking spaces as spaces:
=TRIM(SUBSTITUTE(A2,CHAR(160)," "))

Remove zero-width spaces:
=SUBSTITUTE(A2,UNICHAR(8203),"")

Replace line breaks with a space:
=SUBSTITUTE(SUBSTITUTE(A2,CHAR(13),""),CHAR(10)," ")

Two things to know. First, Excel’s TRIM also collapses repeated spaces between words into one, so it is not purely a leading and trailing trim. Second, CLEAN removes non-printing characters such as tabs and line breaks, but it deletes them rather than replacing them with a space, which can glue words together.

In Google Sheets, Data → Data cleanup → Trim whitespace trims the selected cells in place.

Whichever route you take, be careful when the CSV goes through a spreadsheet at all: opening it can change leading zeros, long numbers and dates. See how to open a CSV in Excel without breaking your data.

Trim whitespace in your file

The free Trim Whitespace tool trims leading and trailing whitespace, including non-breaking spaces, in every cell. You can also collapse repeated inner spaces and remove zero-width characters. It runs in your browser, so the file is never uploaded, and because it treats the CSV as text, nothing else about your data changes.

In the CleanYourDataset workspace, trimming is the first recommended fix. It shows how many cells will change before you apply it, and you can undo it afterwards. The guide covers the full cleaning order, and you can create a free account to save datasets privately.

Keep reading

All articles

Clean your next
export in minutes.

Create an account

Already have one? Log in