General Discussion

Encoded Characters in CSV Files Opened in Excel

In certain scenarios, customers may need to upload CSV files that contain non-English characters in the ProntoForms portal.  

For example, many customers find that the bulk dispatch feature for a form saves a lot of time as forms do not need to be dispatched manually one by one.  

When the Active Dispatch CSV Template is downloaded from the ProntoForms portal, it is UTF-8 encoded and when the file is uploaded to the ProntoForms portal, it is expected to be in a UTF-8 encoding format as well. 

In this sample Active Dispatch CSV Template that is opened in Excel, an answer of "Montréal" is being dispatched in a dropdown question.

If the CSV file is saved in Excel without UTF-8 encoding and uploaded to the ProntoForms portal as the bulk dispatch input, the data will not be able to be dispatched and an error such as the one below will be shown.

This issue may also manifest in other areas such as manual upload or FTP-based data sources.  

Resolving the issue:  

For manual or FTP data sources, if the data is already saved and being uploaded as UTF-8 encoded to the ProntoForms portal, select the "UTF-8 Encoded" checkbox in the data source creation pane.

In some instances, saving the CSV file as UTF-8 encoded in Excel and uploading the file to the ProntoForms portal will resolve the character encoding issues.  

Excel may still force a different encoding, such as WINDOWS-1252, while saving the file so the below steps will need to be followed to load the CSV file as a UTF-8 encoding and then to save the file as UTF-8 encoded:  

1. Open a blank workbook in Excel, navigate to the Data tab on the top ribbon, and select "From Text/CSV."
   
   
   
2. Select the CSV file in the File Explorer.
   
   
3. Once the data loads in Excel, ensure that the File Origin dropdown is set to "Unicode (UTF-8)" and select Load. Note that this task may format and style the columns depending on the data that is already present in the file.
   
   
   
   If the Active Dispatch Template CSV file is being used for bulk dispatching, a best practice would be to save the original file in UTF-8 encoding under a different name, fill in the template with the dispatch data, and then save the filled-in template under a different name. For each new round of dispatching, the original UTF-8 encoded template file can be used.
   
   
   
   
   
4. Fill in and verify that all data is inserted into the file correctly and then save the file as UTF-8 encoded.
   
   
5. Upload the file to the ProntoForms portal and ensure that there are no errors in the bulk dispatch template or in the forms where the new or updated data sources are used.  

#TechTalkSupport
------------------------------
Jesse Collier
Senior Technical Engineer
ProntoForms
------------------------------

Hello @Jesse Collier,

I have created a handlebars document template with file extension .csv

The resulting document is not UTF-8 and I do not see any flag to select this option.

How can this be achieved?

Thanks a lot.

Encoded Characters in CSV Files Opened in Excel

In certain scenarios, customers may need to upload CSV files that contain non-English characters in the ProntoForms portal.  

For example, many customers find that the bulk dispatch feature for a form saves a lot of time as forms do not need to be dispatched manually one by one.  

When the Active Dispatch CSV Template is downloaded from the ProntoForms portal, it is UTF-8 encoded and when the file is uploaded to the ProntoForms portal, it is expected to be in a UTF-8 encoding format as well. 

In this sample Active Dispatch CSV Template that is opened in Excel, an answer of "Montréal" is being dispatched in a dropdown question.

If the CSV file is saved in Excel without UTF-8 encoding and uploaded to the ProntoForms portal as the bulk dispatch input, the data will not be able to be dispatched and an error such as the one below will be shown.

This issue may also manifest in other areas such as manual upload or FTP-based data sources.  

Resolving the issue:  

For manual or FTP data sources, if the data is already saved and being uploaded as UTF-8 encoded to the ProntoForms portal, select the "UTF-8 Encoded" checkbox in the data source creation pane.

In some instances, saving the CSV file as UTF-8 encoded in Excel and uploading the file to the ProntoForms portal will resolve the character encoding issues.  

Excel may still force a different encoding, such as WINDOWS-1252, while saving the file so the below steps will need to be followed to load the CSV file as a UTF-8 encoding and then to save the file as UTF-8 encoded:  

1. Open a blank workbook in Excel, navigate to the Data tab on the top ribbon, and select "From Text/CSV."
   
   
   
2. Select the CSV file in the File Explorer.
   
   
3. Once the data loads in Excel, ensure that the File Origin dropdown is set to "Unicode (UTF-8)" and select Load. Note that this task may format and style the columns depending on the data that is already present in the file.
   
   
   
   If the Active Dispatch Template CSV file is being used for bulk dispatching, a best practice would be to save the original file in UTF-8 encoding under a different name, fill in the template with the dispatch data, and then save the filled-in template under a different name. For each new round of dispatching, the original UTF-8 encoded template file can be used.
   
   
   
   
   
4. Fill in and verify that all data is inserted into the file correctly and then save the file as UTF-8 encoded.
   
   
5. Upload the file to the ProntoForms portal and ensure that there are no errors in the bulk dispatch template or in the forms where the new or updated data sources are used.  

#TechTalkSupport
------------------------------
Jesse Collier
Senior Technical Engineer
ProntoForms
------------------------------

Hello @Jesse Collier,

I have created a handlebars document template with file extension .csv

The resulting document is not UTF-8 and I do not see any flag to select this option.

How can this be achieved?

Thanks a lot.

Encoded Characters in CSV Files Opened in Excel

In certain scenarios, customers may need to upload CSV files that contain non-English characters in the ProntoForms portal.  

For example, many customers find that the bulk dispatch feature for a form saves a lot of time as forms do not need to be dispatched manually one by one.  

When the Active Dispatch CSV Template is downloaded from the ProntoForms portal, it is UTF-8 encoded and when the file is uploaded to the ProntoForms portal, it is expected to be in a UTF-8 encoding format as well. 

In this sample Active Dispatch CSV Template that is opened in Excel, an answer of "Montréal" is being dispatched in a dropdown question.

If the CSV file is saved in Excel without UTF-8 encoding and uploaded to the ProntoForms portal as the bulk dispatch input, the data will not be able to be dispatched and an error such as the one below will be shown.

This issue may also manifest in other areas such as manual upload or FTP-based data sources.  

Resolving the issue:  

For manual or FTP data sources, if the data is already saved and being uploaded as UTF-8 encoded to the ProntoForms portal, select the "UTF-8 Encoded" checkbox in the data source creation pane.

In some instances, saving the CSV file as UTF-8 encoded in Excel and uploading the file to the ProntoForms portal will resolve the character encoding issues.  

Excel may still force a different encoding, such as WINDOWS-1252, while saving the file so the below steps will need to be followed to load the CSV file as a UTF-8 encoding and then to save the file as UTF-8 encoded:  

1. Open a blank workbook in Excel, navigate to the Data tab on the top ribbon, and select "From Text/CSV."
   
   
   
2. Select the CSV file in the File Explorer.
   
   
3. Once the data loads in Excel, ensure that the File Origin dropdown is set to "Unicode (UTF-8)" and select Load. Note that this task may format and style the columns depending on the data that is already present in the file.
   
   
   
   If the Active Dispatch Template CSV file is being used for bulk dispatching, a best practice would be to save the original file in UTF-8 encoding under a different name, fill in the template with the dispatch data, and then save the filled-in template under a different name. For each new round of dispatching, the original UTF-8 encoded template file can be used.
   
   
   
   
   
4. Fill in and verify that all data is inserted into the file correctly and then save the file as UTF-8 encoded.
   
   
5. Upload the file to the ProntoForms portal and ensure that there are no errors in the bulk dispatch template or in the forms where the new or updated data sources are used.  

#TechTalkSupport
------------------------------
Jesse Collier
Senior Technical Engineer
ProntoForms
------------------------------

Hi Alicia, 

That's a great question! If the file is not already UTF-8 encoded when it's downloaded from the TrueContext portal, you should still be able to follow the "load from Text/CSV" process that I outlined in the "Resolving the issue" section of this post and Excel will do the heavy lifting for you. 

It looks like Calvin has already mentioned a potential solution as well, thank you Calvin! 

------------------------------
Jesse Collier
Senior Technical Engineer
TrueContext
Atlanta

Original Message:
Sent: 10-04-2024 04:19
From: Alicia Rico
Subject: This Week in Support: Solving a common ticket about encoded characters in CSV files

Hello @Jesse Collier,

I have created a handlebars document template with file extension .csv

The resulting document is not UTF-8 and I do not see any flag to select this option.

How can this be achieved?

Thanks a lot.

------------------------------
Alicia Rico
CI
Johnson Controls

Original Message:
Sent: 11-17-2022 08:00
From: Jesse Collier
Subject: This Week in Support: Solving a common ticket about encoded characters in CSV files

Encoded Characters in CSV Files Opened in Excel

In certain scenarios, customers may need to upload CSV files that contain non-English characters in the ProntoForms portal.  

For example, many customers find that the bulk dispatch feature for a form saves a lot of time as forms do not need to be dispatched manually one by one.  

When the Active Dispatch CSV Template is downloaded from the ProntoForms portal, it is UTF-8 encoded and when the file is uploaded to the ProntoForms portal, it is expected to be in a UTF-8 encoding format as well. 

In this sample Active Dispatch CSV Template that is opened in Excel, an answer of "Montréal" is being dispatched in a dropdown question.

If the CSV file is saved in Excel without UTF-8 encoding and uploaded to the ProntoForms portal as the bulk dispatch input, the data will not be able to be dispatched and an error such as the one below will be shown.

This issue may also manifest in other areas such as manual upload or FTP-based data sources.  

Resolving the issue:  

For manual or FTP data sources, if the data is already saved and being uploaded as UTF-8 encoded to the ProntoForms portal, select the "UTF-8 Encoded" checkbox in the data source creation pane.

In some instances, saving the CSV file as UTF-8 encoded in Excel and uploading the file to the ProntoForms portal will resolve the character encoding issues.  

Excel may still force a different encoding, such as WINDOWS-1252, while saving the file so the below steps will need to be followed to load the CSV file as a UTF-8 encoding and then to save the file as UTF-8 encoded:  

1. Open a blank workbook in Excel, navigate to the Data tab on the top ribbon, and select "From Text/CSV."
   
   
   
2. Select the CSV file in the File Explorer.
   
   
3. Once the data loads in Excel, ensure that the File Origin dropdown is set to "Unicode (UTF-8)" and select Load. Note that this task may format and style the columns depending on the data that is already present in the file.
   
   
   
   If the Active Dispatch Template CSV file is being used for bulk dispatching, a best practice would be to save the original file in UTF-8 encoding under a different name, fill in the template with the dispatch data, and then save the filled-in template under a different name. For each new round of dispatching, the original UTF-8 encoded template file can be used.
   
   
   
   
   
4. Fill in and verify that all data is inserted into the file correctly and then save the file as UTF-8 encoded.
   
   
5. Upload the file to the ProntoForms portal and ensure that there are no errors in the bulk dispatch template or in the forms where the new or updated data sources are used.  

#TechTalkSupport
------------------------------
Jesse Collier
Senior Technical Engineer
ProntoForms
------------------------------