Text field showing strange text is submissions: ��

Support
1

I'm transferring the data collected in ODK to Salesforce, but I'm getting an "invalid character reference" error because some submissions in a text field have the following string at the end: ��

Why is this happening and can it be prevented?

Many thanks
Michael

2

Can you provide any details about the data collection process...

  • What device(s) are being used for data collection?
  • What language is the data being submitted in?

Also, what are you using for the data transfer... is this over the API or via a CSV export/ import or something else?

3

Hi Dan, thanks for the reply. The devices are Android tablets but I can't be sure of the exact make. All the cases we have encountered using English.

Since my message last week there have been some more unusual characters appearing, but these haven't caused a problem when transferring to Salesforce.

4

When you open the data export in a spreadsheet software, can you get the data to display correctly?

5

When I open the data in Excel, it appears like this (attached), but I'm pretty sure these are not the characters the users are entering. It's only the submissions with the upside down question marks that are creating an error in Salesforce, but the others are strange nonetheless
ODK characters

6

Are you using ODK Central?

Reading through some other threads:

  • Another possibility is that your hosting service is behind a proxy (e.g., Apache HTTPD) and that proxy is adding or stripping headers from the requests. One of these headers specifies the incoming character set, and if absent, will cause this issue. (from ODK in different languages - #17 by Mitch_S)
  • This could be an encoding problem on the spreadsheet tool side. You should try to force the encoding while opening the csv to UTF8. Are you using Microsoft Office? It can be troublesome when dealing with CSV files. I've heard that LibreOffice tends to have fewer issues.You might also try searching the internet for general guidance on opening CSV text files in Excel, for example: https://stackoverflow.com/questions/43307776/how-to-open-a-text-file-with-excel-in-utf-8-encoding
7

Thanks Dan, yes we're using ODK Central. I'll take a look at the other points with our IT team and see what we can find. Cheers

1 Like
8

This is most certainly a Unicode issue and we'll need the specifics of how you are transferring data into Salesforce before we can help. We'll also need to know what version of Central you are using.

9

Thanks Yaw. We're using version 1.2 so we'll look into upgrading as well as the Unicode issue.