RecalcfxSpreadsheets, and the wrong answers that look rightAll 42 entries
fxthe bytes are fine and the reader guessed wrong, or the bytes are gone
ABCDEFGH
1

3.4 · part 3

Encodings, and the question marks in an export

The visible damage tells you whether the bytes survived, which decides whether the file can be recovered at all.

Teams applying the checks in this entry can also compare operational tools for productivity monitoring software, keeping time and review ownership outside the calculation file itself.

2

Established onExcel 365, Windows: BOM recognised, CSV UTF-8 save optionExcel 2016: system code page on double clickGoogle Sheets, web: UTF-8 assumed, no BOM written

A delimited file is a sequence of bytes. Which characters those bytes represent depends on an encoding, and the file has nowhere to record which encoding was used.

What goes wrong

A file written in the modern variable-width encoding, read as though it were one of the older single-byte ones, produces a recognisable pattern: a capital A with a tilde before an accented letter, or a short run of symbols where an apostrophe or a dash should be.

The pattern is diagnostic. Seeing it means the bytes are intact and the reader made the wrong assumption, which is good news, because the file is recoverable by reading it again correctly.

The other symptom is a diamond containing a question mark, which means the byte could not be decoded at all and the software substituted a character reserved for the purpose. That one is not always recoverable, because some routes discard the original byte.

The marker that helps, and its cost

A short sequence at the start of a file can declare the encoding. Excel on Windows recognises it and reads the file correctly; without it, a double-clicked file has historically been read using the system's older code page.

The cost is that the same sequence is not expected by a great deal of other software, which reads it as three stray characters at the beginning of the first field. So a file that opens correctly for a colleague breaks a script, and a file that works with the script opens wrongly for the colleague.

Excel offers a save format described as comma-separated with that encoding, which writes the marker. It is the correct choice when the recipient is a person and the wrong one when the recipient is a program.

What the other product assumes

Google Sheets assumes the modern encoding throughout, for import and export, and does not write the marker. Files moving between the two products in either direction therefore hit exactly the mismatch above, in one direction or the other, depending on who opens what.

Why this is not a solved problem

Because the fix requires either declaring the encoding somewhere the format has no room for, or every reader defaulting to the same assumption. The second has very nearly happened over twenty-five years and the remaining exception is the double-click route in the most widely used spreadsheet, which is the exact route most people take.

WrittenMunoz — Caferead assomething elseMuñoz – CaféThe pattern is the diagnosisa capital A with a tilde before an accented letter means one encoding read as anothera diamond with a question mark means the byte could not be decoded at alland the file says nothing about which encoding it used, because there is nowhere to say it
3.4A name written in one encoding and read under another. The pattern of damage is the diagnosis and decides whether recovery is possible.

What to do

Import rather than open, and choose the encoding in the dialogue. Both products let you, and it is the same dialogue that lets you declare column types, so one habit fixes two entries of this part.

Include a canary. When producing files, put a value containing an accented character and a dash in a known row. Checking that one cell after import answers the encoding question in a second, and nothing else does.

Say which encoding you used, in the documentation or the file name, since the file cannot say it.

What not to do

Do not repair mojibake by hand with find and replace. The mapping is not one to one, the same visible sequence can come from different originals, and a manual pass fixes the common cases and leaves the rest in a state where the original is no longer inferable.

Read the file again with the right encoding instead. If the bytes were preserved, that is complete and instant.

Checking whether the bytes survived

If the visible damage is the tilde-A pattern, they did, and rereading works. If it is the replacement character, they did not, and the original byte is gone from that copy; go back to the source file rather than to the one somebody saved after opening it.

That distinction is worth teaching to anybody who handles these files, because the second case is created by the well-meant act of opening the file to check it and saving it.

What we cannot verify

The encodings, the marker and the save options are documented and demonstrable. The historical behaviour of the double-click route differs across versions and regional installations, and the account here reflects current builds tested in August 2026 plus the vendor's documentation for older ones. Google Sheets behaviour was established by testing.

The encoding question for the other direction

Producing a file for a spreadsheet user and producing one for a program are different jobs with incompatible answers, and the choice cannot be deferred.

Where both audiences exist, produce two files. That sounds wasteful and is cheaper than any of the alternatives, all of which consist of one audience discovering the problem later and asking somebody to fix it by hand.

In short

  1. A delimited file has nowhere to record which encoding produced its bytes.
  2. The tilde-A pattern means the bytes are intact and the reader assumed wrongly.
  3. The replacement character means the byte is gone from that copy.
  4. A marker at the start declares the encoding to one product and confuses many others.
  5. The two products assume differently, so files break in one direction or the other.
  6. Put an accented character in a known cell and check it after every import.
src

For further context, see the Unicode Standard.