CSV export & import function for Japanese with Excel
Hi I am writting this post for Japanese customer but I belive any countries who use double-byte are facing same issue. So, I would like to have more opinition for my request.
Saving file in Excel with Japanese character will generate a CSV file with Shift-JIS, so we are not able to import file by using "pxUploadCSVResults" for import and " ConvertResultsToCSV " for export.
* This is a default behaviour of Excel if there is Japanese characters but other language such as Korea, Chines may be facing a same problem.
1. to have parameters or something else to specify import character code for "pxUploadCSVResults"
2. to have parameters or something else to specify with BOM or without BOM for "ConvertResultsToCSV "
Here are details for these requests:
[1. Import]
Step2 (java) in "pxUploadCSVResults"
we comment the line below
//cofileContent = new com.pega.pegarules.pub.util.Base64Util().decode(strFileData);
And then add 3 lines below instead
try {
fileContent = new String(org.apache.commons.codec.binary.Base64.decodeBase64(strFileData),"MS932");
} catch (java.io.UnsupportedEncodingException e) {
}
However, "pxUploadCSVResults" is Final, so we do not want to modify code in Step2.
Rather we want to have a parameter, which we can specify "character code" to use for import action
[2. Export]
Step 5 (java) in "ConvertResultsToCSV", we had to add 3 line for steam to add BOM to properly open UTF-8 file with Excel
Because, Excel do not recognize CSV file without BOM. Therefore, we want to have an option to "set BOM" somewhere in "ConvertResultsToCSV".
try{
// Create file
PROutputStream stream = new PROutputStream(f);
// adding BOM to file stream
stream.write(0xef);
stream.write(0xbb);
stream.write(0xbf);
PRWriter out = new PRWriter(stream, null, false);
[Important notice]
Both rules are final, so we can not modify code
Please see attached screen shots and CSV file with Shift-JIS.
**Moderation Team has archived post**
This post has been archived for educational purposes. Contents and links will no longer be updated. If you have the same/similar question, please write a new post.
Updated: 27 Apr 2022 16:38 EDT
Pegasystems Inc.
GB
Hi Okar,
Sounds like you have diagnosed the issue in some detail; but are not able to implement the solution due to the fact that the Rules involved are 'Final' : if my understanding is correct -please may I ask you to log a Service Request with GCS and provide these details; we can engage with our Subject Matter Experts and see whether this is viable for a HFIX - or perhaps if there is no urgent requirement at this point - a ENHANCEMENT or 'fix-in-future' request ?
Thanks,
John
Updated: 27 Apr 2022 16:38 EDT
Pegasystems Inc.
AU
Thank you, John.
In fact, I had submited SR (SR-A101230) already but I will submit new SR for an Enhancement.
Regards,
Ryo
Updated: 27 Apr 2022 16:38 EDT
Pegasystems Inc.
IN
Hi Ryo,
An enhancement request has been created for this query on the internal portal. I have tagged the request ID along with the issue description in the Related Support Case Number section.
Thanks!
Vidyaranjan A V | Community Moderator | Pegasystems Inc.
Updated: 27 Apr 2022 16:38 EDT
Pegasystems Inc.
In Pega 7.2.1 the pxUploadCSVResults doesn't seem to be used any more.
We were having issues importing Thai characters, and were using a local fix as a work-around.
Updated: 27 Apr 2022 16:38 EDT
Barclays UK PLC
GB
Hi okar1,
Instead of adding BOM in the stream, why didn't you the try the following:-
PRWriter out = new PRWriter(stream, "UTF-8", true);
check the PRWriter class in the engine API.
PRWriter(java.io.OutputStream aOutStream, java.lang.String aEncoding, boolean aIncludeBOM)
Updated: 27 Apr 2022 16:38 EDT
Pegasystems Inc.
JP
PRWriter out = new PRWriter(stream, "UTF-8", true);
may be a good idea, but it doesn't fix the problem. Only writing the BOM seems to do the trick.
Updated: 27 Apr 2022 16:38 EDT
MBAFC
CN
I have faced the same issue when delegate the data type to end user.
When export or import the CSV with Chinese, there will have some display issue.
Does there are any enhance for this?
Instellars
SG
@Shipeng_Tian did you get the solution for this? facing the same issue for traditional Chinese and simplified Chinese
Updated: 24 Nov 2022 11:50 EST
Pegasystems Inc.
GB
@BHANUKIRANT I will try to find an SME to check the status of the logged Enhancement request.
Note that Pega Platform and the PegaRULES database support the UTF-8 or UTF-16 Unicode character sets as documented here: Internationalization and localization
Please see a similar question raised in questions
Issue in Opening downloaded CSV file with Japanese characters in Excel
and
Do the steps in the latter question help resolve your issue?