Question
Scotiabank
CA
Last activity: 2 Feb 2016 12:31 EST
SOAP service input with french characters is cutting off the XML following the special character. Is it an encoding issue in the request being sent to PEGA ?
Request :
<EventData>
<LawyerAmendments>
<Notes>
<Note>
<ID>0</ID>
<NotesType>STANDARD</NotesType>
<Status>NEW</Status>
<Subject>Funds Not Yet Received/Fonds non reçus à ce jour</Subject>
<Details>We look forward to receiving the new mortgage advance amount
</Details>
</Note>
</Notes>
</LawyerAmendments>
</EventData>
What we get in PEGA:
<EventData>
<LawyerAmendments>
<Notes>
<Note>
<ID>0</ID>
<NotesType>STANDARD</NotesType>
<Status>NEW</Status>
<Subject>Funds Not Yet Received/Fonds non re</Subject>
</Note>
</Notes>
</LawyerAmendments>
</EventData>
Pegasystems Inc.
US
Where did you get the request in Pega? Was it from the debug log of the class com.pega.pegarules.integration.engine.internal.services.soap.SOAPService? If not, can you enable the debug log for this class, send the debug log?
Scotiabank
CA
Given below is data from logs:
2016-01-26 12:50:57,893 [ WebContainer : 12] [ ] [ ] [ ] ( services.soap.SOAPService) DEBUG cs1bcmapd01.bns| Proprietary information hidden - Received SOAP request message, trying to obtain SOAP Action value from SOAPAction HTTP Header
2016-01-26 12:50:57,893 [ WebContainer : 12] [ ] [ ] [ ] ( services.soap.SOAPService) DEBUG cs1bcmapd01.bns| Proprietary information hidden - Obtained SOAP Action value from SOAPAction HTTP Header: urn:PegaRULES:SOAP:RPFFWSOAPServices:LawyerDealEvents#ProcessLawyerDealEvents
2016-01-26 12:50:57,893 [ WebContainer : 12] [ ] [ ] [ ] ( services.soap.SOAPService) INFO cs1bcmapd01.bns| Proprietary information hidden - SOAP Request Envelope:
<soapenv:Envelope xmlns:ns="http://schemas.bns.com/eDocs/PegaIntegration/ExternalInterface/20151201" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
Given below is data from logs:
2016-01-26 12:50:57,893 [ WebContainer : 12] [ ] [ ] [ ] ( services.soap.SOAPService) DEBUG cs1bcmapd01.bns| Proprietary information hidden - Received SOAP request message, trying to obtain SOAP Action value from SOAPAction HTTP Header
2016-01-26 12:50:57,893 [ WebContainer : 12] [ ] [ ] [ ] ( services.soap.SOAPService) DEBUG cs1bcmapd01.bns| Proprietary information hidden - Obtained SOAP Action value from SOAPAction HTTP Header: urn:PegaRULES:SOAP:RPFFWSOAPServices:LawyerDealEvents#ProcessLawyerDealEvents
2016-01-26 12:50:57,893 [ WebContainer : 12] [ ] [ ] [ ] ( services.soap.SOAPService) INFO cs1bcmapd01.bns| Proprietary information hidden - SOAP Request Envelope:
<soapenv:Envelope xmlns:ns="http://schemas.bns.com/eDocs/PegaIntegration/ExternalInterface/20151201" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"><wsse:UsernameToken wsu:Id="UsernameToken-4CBD6DBEA5F5A58111145383065745229"><wsse:Username>mrl_ist_user</wsse:Username><wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">Password12</wsse:Password></wsse:UsernameToken></wsse:Security></soapenv:Header>
<soapenv:Body>
<ns:LawyerDealEventsRequest>
<MessageID>R111111</MessageID>
<TimeStamp>?</TimeStamp>
<NumberOfMessages>?</NumberOfMessages>
<!--Optional:-->
<Messages>
<Message>
<EventType>AMENDMENTS</EventType>
<EventTimeStamp>2014-10-14T08:45:15.107417-04:00</EventTimeStamp>
<FCTURN>14254001983</FCTURN>
<LenderReferenceNumber>3569654</LenderReferenceNumber>
<UserName>
<FirstName>Robert</FirstName>
<MiddleName>Gerhard</MiddleName>
<LastName>Janzen</LastName>
</UserName>
<DealStatus>ACTIVE</DealStatus>
<EventData>
<LawyerAmendments>
<Notes>
<Note>
<ID>0</ID>
<NotesType>STANDARD</NotesType>
<Status>NEW</Status>
<Subject>Funds Not Yet Received/Fonds non re足 ࡣe jour</Subject>
<Details>We look forward to receiving the new mortgage advance amount</Details>
</Note>
</Notes>
</LawyerAmendments>
</EventData>
</Message>
</Messages>
</ns:LawyerDealEventsRequest>
</soapenv:Body>
</soapenv:Envelope>
2016-01-26 12:50:57,958 [ WebContainer : 12] [ STANDARD] [ ] [ MTG:01.01.01] ( services.soap.SOAPService) INFO cs1bcmapd01.bns| Proprietary information hidden - SOAP Response Envelope:
<?xml version="1.0"?>
<soap:Envelope
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<ns1:LawyerDealEventsResponse xmlns:ns1="http://schemas.bns.com/eDocs/PegaIntegration/20151201"> <StatusCode>0</StatusCode> <StatusMessage>Success</StatusMessage> </ns1:LawyerDealEventsResponse>
</soap:Body>
</soap:Envelope>
Pegasystems Inc.
US
Can u attach the complete log? I want to review the response as well. What version of PRPC is this and what functional impacts are you aware of?
Scotiabank
CA
Here is the log :
2016-01-26 12:50:29,076 [PegaWorkManager : 10] [ STANDARD] [ ] [ ] ( internal.async.Agent) INFO - System date: Tue Jan 26 12:50:29 EST 2016 Total memory: 8,594,128,896 Free memory: 6,782,625,216 Requestor Count: 53 Shared Pages memory usage: 0%
2016-01-26 12:50:41,717 [ WebContainer : 4] [ ] [ ] [ ] (pega.TRACE.requestor_lifecycle) INFO cs1bcmapd01.bns| Proprietary information hidden - initial codeset map: {omdm-connect=01-01-01@1969-12-31 19:00:00.0, scotiapro-connect=01-01-01@1969-12-31 19:00:00.0, pega-enginecode=07-10-15@2016-01-22 10:16:00.0, customer=06-01-01@2016-01-20 16:32:00.0} Current codeset map: {omdm-connect=01-01-01@1969-12-31 19:00:00.0, pega-enginecode=07-10-15@2016-01-26 09:19:00.0, scotiapro-connect=01-01-01@1969-12-31 19:00:00.0, customer=06-01-01@2016-01-20 16:32:00.0} activation rejected for requestor H33355C244BCE90049B2FB69ACC30C611. Acting as if no passivated state was found
2016-01-26 12:50:41,723 [ WebContainer : 4] [ STANDARD] [ ] [ BNSNUApp:01.01.01] ( internal.mgmt.Executable) ERROR cs1bcmapd01.bns| Proprietary information hidden - Execution denied for class: @baseclass (activity: pzHandleUnknownSession)
2016-01-26 12:50:41,724 [ WebContainer : 4] [ STANDARD] [ ] [ BNSNUApp:01.01.01] ( internal.mgmt.Executable) ERROR cs1bcmapd01.bns| Proprietary information hidden - Execution denied for class: @baseclass (activity: ActivityStatusExceptionHandler)
Here is the log :
2016-01-26 12:50:29,076 [PegaWorkManager : 10] [ STANDARD] [ ] [ ] ( internal.async.Agent) INFO - System date: Tue Jan 26 12:50:29 EST 2016 Total memory: 8,594,128,896 Free memory: 6,782,625,216 Requestor Count: 53 Shared Pages memory usage: 0%
2016-01-26 12:50:41,717 [ WebContainer : 4] [ ] [ ] [ ] (pega.TRACE.requestor_lifecycle) INFO cs1bcmapd01.bns| Proprietary information hidden - initial codeset map: {omdm-connect=01-01-01@1969-12-31 19:00:00.0, scotiapro-connect=01-01-01@1969-12-31 19:00:00.0, pega-enginecode=07-10-15@2016-01-22 10:16:00.0, customer=06-01-01@2016-01-20 16:32:00.0} Current codeset map: {omdm-connect=01-01-01@1969-12-31 19:00:00.0, pega-enginecode=07-10-15@2016-01-26 09:19:00.0, scotiapro-connect=01-01-01@1969-12-31 19:00:00.0, customer=06-01-01@2016-01-20 16:32:00.0} activation rejected for requestor H33355C244BCE90049B2FB69ACC30C611. Acting as if no passivated state was found
2016-01-26 12:50:41,723 [ WebContainer : 4] [ STANDARD] [ ] [ BNSNUApp:01.01.01] ( internal.mgmt.Executable) ERROR cs1bcmapd01.bns| Proprietary information hidden - Execution denied for class: @baseclass (activity: pzHandleUnknownSession)
2016-01-26 12:50:41,724 [ WebContainer : 4] [ STANDARD] [ ] [ BNSNUApp:01.01.01] ( internal.mgmt.Executable) ERROR cs1bcmapd01.bns| Proprietary information hidden - Execution denied for class: @baseclass (activity: ActivityStatusExceptionHandler)
2016-01-26 12:50:43,188 [ WebContainer : 11] [ STANDARD] [ ] [ EDB:01.01.02] (ardingFW_Int_Onboarding.Action) INFO cs1bcmapd01.bns| Proprietary information hidden|Rest|EDBRESTServices|Services|CreateAppJs|A7F7053A4AD451165EEB450CF268445DC -
CreateEDBOCase: START.
2016-01-26 12:50:52,463 [ WebContainer : 6] [ STANDARD] [ ] [ BNSNUApp:01.01.01] ( internal.mgmt.Executable) ERROR cs1bcmapd01.bns| Proprietary information hidden - Execution denied for class: @baseclass (activity: pzHandleUnknownSession)
2016-01-26 12:50:52,463 [ WebContainer : 6] [ STANDARD] [ ] [ BNSNUApp:01.01.01] ( internal.mgmt.Executable) ERROR cs1bcmapd01.bns| Proprietary information hidden - Execution denied for class: @baseclass (activity: ActivityStatusExceptionHandler)
2016-01-26 12:50:54,451 [ WebContainer : 4] [ ] [ ] [ ] (pega.TRACE.requestor_lifecycle) INFO cs1bcmapd01.bns| Proprietary information hidden - initial codeset map: {omdm-connect=01-01-01@1969-12-31 19:00:00.0, scotiapro-connect=01-01-01@1969-12-31 19:00:00.0, pega-enginecode=07-10-15@2016-01-22 10:16:00.0, customer=06-01-01@2016-01-20 16:32:00.0} Current codeset map: {omdm-connect=01-01-01@1969-12-31 19:00:00.0, pega-enginecode=07-10-15@2016-01-26 09:19:00.0, scotiapro-connect=01-01-01@1969-12-31 19:00:00.0, customer=06-01-01@2016-01-20 16:32:00.0} activation rejected for requestor H3BB0BBC2C403609B808A4BF8B0490BDF. Acting as if no passivated state was found
2016-01-26 12:50:57,893 [ WebContainer : 12] [ ] [ ] [ ] ( services.soap.SOAPService) DEBUG - Received SOAP request message
2016-01-26 12:50:57,893 [ WebContainer : 12] [ ] [ ] [ ] ( services.soap.SOAPService) DEBUG cs1bcmapd01.bns| Proprietary information hidden - Received SOAP request message, trying to obtain SOAP Action value from SOAPAction HTTP Header
2016-01-26 12:50:57,893 [ WebContainer : 12] [ ] [ ] [ ] ( services.soap.SOAPService) DEBUG cs1bcmapd01.bns| Proprietary information hidden - Obtained SOAP Action value from SOAPAction HTTP Header: urn:PegaRULES:SOAP:RPFFWSOAPServices:LawyerDealEvents#ProcessLawyerDealEvents
2016-01-26 12:50:57,893 [ WebContainer : 12] [ ] [ ] [ ] ( services.soap.SOAPService) INFO cs1bcmapd01.bns| Proprietary information hidden - SOAP Request Envelope:
<soapenv:Envelope xmlns:ns="http://schemas.bns.com/eDocs/PegaIntegration/ExternalInterface/20151201" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"><wsse:UsernameToken wsu:Id="UsernameToken-4CBD6DBEA5F5A58111145383065745229"><wsse:Username>mrl_ist_user</wsse:Username><wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">Password12</wsse:Password></wsse:UsernameToken></wsse:Security></soapenv:Header>
<soapenv:Body>
<ns:LawyerDealEventsRequest>
<MessageID>R111111</MessageID>
<TimeStamp>?</TimeStamp>
<NumberOfMessages>?</NumberOfMessages>
<!--Optional:-->
<Messages>
<Message>
<EventType>AMENDMENTS</EventType>
<EventTimeStamp>2014-10-14T08:45:15.107417-04:00</EventTimeStamp>
<FCTURN>14254001983</FCTURN>
<LenderReferenceNumber>3569654</LenderReferenceNumber>
<UserName>
<FirstName>Robert</FirstName>
<MiddleName>Gerhard</MiddleName>
<LastName>Janzen</LastName>
</UserName>
<DealStatus>ACTIVE</DealStatus>
<EventData>
<LawyerAmendments>
<Notes>
<Note>
<ID>0</ID>
<NotesType>STANDARD</NotesType>
<Status>NEW</Status>
<Subject>Funds Not Yet Received/Fonds non re足 ࡣe jour</Subject>
<Details>We look forward to receiving the new mortgage advance amount</Details>
</Note>
</Notes>
</LawyerAmendments>
</EventData>
</Message>
</Messages>
</ns:LawyerDealEventsRequest>
</soapenv:Body>
</soapenv:Envelope>
2016-01-26 12:50:57,958 [ WebContainer : 12] [ STANDARD] [ ] [ MTG:01.01.01] ( services.soap.SOAPService) INFO cs1bcmapd01.bns| Proprietary information hidden - SOAP Response Envelope:
<?xml version="1.0"?>
<soap:Envelope
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<ns1:LawyerDealEventsResponse xmlns:ns1="http://schemas.bns.com/eDocs/PegaIntegration/20151201"> <StatusCode>0</StatusCode> <StatusMessage>Success</StatusMessage> </ns1:LawyerDealEventsResponse>
</soap:Body>
</soap:Envelope>
2016-01-26 12:51:00,030 [ WebContainer : 5] [ STANDARD] [ ] [ BNSNUApp:01.01.01] ( mgmt.util.URLAccessContext) WARN cs1bcmapd01.bns| Proprietary information hidden - URLAccessModeWarn:URLAccessPermitted URLAccessDetail CSRFAttack Invalid harness ID HIDEFE09197265E3E5AE2B49C8203F0EEE2 :From SetProcessWindowName :CVG 1 1 0 0 0 :MQD pzHarnessID=HIDEFE09197265E3E5AE2B49C8203F0EEE2&pyActivity=SetProcessWindowName&processWindowName=Composite_H33355C244BCE90049B2FB69ACC30C611&returnWindowName=true
Pegasystems Inc.
GB
Also (I think this is unlikely, but worth eliminating) - does your incoming XML packet have an XML header or not ?
If not: that should rule out an encoding issue (it will default to UTF-8), but if it was like this for instance:
<?xml version="1.0" encoding="ISO-8859-1"?>
Then any XML Parser would choke when it came across any character outside of ASCII codepage (so the accented character would do it here).
Are you able to save your XML and validate it separately outside of PRPC ? For instance just loading it into a browser will check the XML is well-formed or not.
Pegasystems Inc.
GB
Sorry - I posted before I saw your reply. (I see there is no encoding string in your XML by the look of it: so this means the content should be assumed to be 'UTF-8"....whether it is or not, has be checked)
The logs (or the post on the Mesh) seem to have interpretted your characters as something other than French as well ?
<Subject>Funds Not Yet Received/Fonds non re足 ࡣe jour</Subject>
This does hint at an encoding issue : although perhaps the logs themselves are just showing this....
Have you tried 'tracing on the wire' here: capturing that actual TCP traffic - using 'tcpmon' or something similar ?
You need to be sure what exactly is being transmitted here I think.....
Scotiabank
CA
Hi John, I am not able to download tcpmon in Bank's nextwork. But the sender confirmed that they now changed their code it to use UFT-8. Now the message is being converted from
<Subject>Funds Not Yet Received/Fonds non reçus à ce jour</Subject> to <Subject>Funds Not Yet Received/Fonds non re?us ? ce jour</Subject>.
It has put "?" for the french characters.
Thanks
Sweekar M
Pegasystems Inc.
DE
Hi Sweekar,
what operating system is used on the other side?
That the French characters are now replaced by question marks ("?") indicates that the XML now pretends to be UTF-8, but it is not!
Have you tried SoapUI instead of the external system to send messages to PRPC? What have been the results?
Have you tried that the other way round (using SoapUI to receive the requests, instead of PRPC)? The results here?
If you are not able to download and to install SoapUI (as it happen to you with tcpmon, as John suggested), try wget (if you have a UNIX/Linux system at hand), or in worst case, you need to craft your own little testing tool.
What you need to know is what is really move back and forth on the wire - meaning you need the raw bits, and not what is being logged by PRPC after it has attempted to process what it got. This is what tcpmon had shown you, this is what SoapUI could give you.
Scotiabank
CA
We are using AIX.
Got permission to install TCPMON.
Tried using SOAP UI too, getting same behavior.
Request using UTF-8:
POST /prweb/PRSOAPServlet/SOAP/RPFFWSOAPServices/LawyerDealEvents HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "urn:PegaRULES:SOAP:RPFFWSOAPServices:LawyerDealEvents#ProcessLawyerDealEvents"
Content-Length: 2056
Host: cs1bcmapd01.bns:80
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
<EventData>
<LawyerAmendments>
<Notes>
<Note>
<ID>0</ID>
<NotesType>STANDARD</NotesType>
<Status>NEW</Status>
<Subject>Funds Not Yet Received/Fonds vous êtes goo</Subject>
<Details>We look forward to receiving the new mortgage advance amount $303,000.05 today for closing. Thank you.</Details>
</Note>
</Notes>
</LawyerAmendments>
</EventData>
Request using iso-8859-1:
We are using AIX.
Got permission to install TCPMON.
Tried using SOAP UI too, getting same behavior.
Request using UTF-8:
POST /prweb/PRSOAPServlet/SOAP/RPFFWSOAPServices/LawyerDealEvents HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "urn:PegaRULES:SOAP:RPFFWSOAPServices:LawyerDealEvents#ProcessLawyerDealEvents"
Content-Length: 2056
Host: cs1bcmapd01.bns:80
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
<EventData>
<LawyerAmendments>
<Notes>
<Note>
<ID>0</ID>
<NotesType>STANDARD</NotesType>
<Status>NEW</Status>
<Subject>Funds Not Yet Received/Fonds vous êtes goo</Subject>
<Details>We look forward to receiving the new mortgage advance amount $303,000.05 today for closing. Thank you.</Details>
</Note>
</Notes>
</LawyerAmendments>
</EventData>
Request using iso-8859-1:
POST /prweb/PRSOAPServlet/SOAP/RPFFWSOAPServices/LawyerDealEvents HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=iso-8859-1
SOAPAction: "urn:PegaRULES:SOAP:RPFFWSOAPServices:LawyerDealEvents#ProcessLawyerDealEvents"
Content-Length: 2055
Host: cs1bcmapd01.bns:80
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
<EventData>
<LawyerAmendments>
<Notes>
<Note>
<ID>0</ID>
<NotesType>STANDARD</NotesType>
<Status>NEW</Status>
<Subject>Funds Not Yet Received/Fonds vous êtes goo</Subject>
<Details>We look forward to receiving the new mortgage advance amount $303,000.05 today for closing. Thank you.</Details>
</Note>
</Notes>
</LawyerAmendments>
</EventData>
