Hadoop (HBase) Connectivity Issues
Hi
I am working on Hadoop integration. As part of this exercise, I have created a Data-Admin-Hadoop record and configured the parameters for HDFS and HBase connections. I have imported all the required Jars mentioned in the PDN Article (https://docs-previous.pega.com/decision-management-reference-materials/hdfs-and-hbase-client-and-server-versions-supported-pega-platform).
commons-configuration-1.6.jar
hadoop-auth-2.4.1.jar
hadoop-common-2.4.1.jar
hbase-client-0.98.3-hadoop2.jar
hbase-common-0.98.3-hadoop2.jar
hbase-protocol-0.98.3-hadoop2.jar
htrace-core-2.04.jar
protobuf-java-2.5.0.jar
zookeeper-3.4.5.jar
commons-io-2.4.jar
hadoop-hdfs-2.4.1.jar
protobuf-java-2.5.0.jar
parquet-hadoop-1.8.1.jar
parquet-common-1.8.1.jar
parquet-column-1.8.1.jar
parquet-format-2.3.1.jar
parquet-encoding-1.8.1.jar
htrace-core4.jar
hadoop-auth-2.6.0-cdh5.8.0.jar
hadoop-common-2.6.0-cdh5.8.0.jar
hadoop-hdfs-2.6.0-cdh5.8.0.jar
Hi
I am working on Hadoop integration. As part of this exercise, I have created a Data-Admin-Hadoop record and configured the parameters for HDFS and HBase connections. I have imported all the required Jars mentioned in the PDN Article (https://docs-previous.pega.com/decision-management-reference-materials/hdfs-and-hbase-client-and-server-versions-supported-pega-platform).
commons-configuration-1.6.jar
hadoop-auth-2.4.1.jar
hadoop-common-2.4.1.jar
hbase-client-0.98.3-hadoop2.jar
hbase-common-0.98.3-hadoop2.jar
hbase-protocol-0.98.3-hadoop2.jar
htrace-core-2.04.jar
protobuf-java-2.5.0.jar
zookeeper-3.4.5.jar
commons-io-2.4.jar
hadoop-hdfs-2.4.1.jar
protobuf-java-2.5.0.jar
parquet-hadoop-1.8.1.jar
parquet-common-1.8.1.jar
parquet-column-1.8.1.jar
parquet-format-2.3.1.jar
parquet-encoding-1.8.1.jar
htrace-core4.jar
hadoop-auth-2.6.0-cdh5.8.0.jar
hadoop-common-2.6.0-cdh5.8.0.jar
hadoop-hdfs-2.6.0-cdh5.8.0.jar
I am able to successfully do the test connectivity for HDFS. But HBase configuration had issues with connectivity - when I do test connectivity, got the following message even though I had imported all the required Jars.
"Connection cannot be established. Please check your configuration. Missing HBase Java client classes on the classpath (are jars uploaded into container?)"
Are there any additional Jars that I need to deploy for Pega 7.3?
I am using Pega Personal Edition - 7.3 and Hadoop Cloudera VM distribution.
Need your help to resolve the issues.
2. Deploy the required Jars in Pega Personal Edition v7.3
3. Create Data-Admin-Hadoop record and enter required configuration in it for Both HDFS & HBase.
4. Do the test connectivity.
Error Message
Attempted Solutions
Pegasystems Inc.
IN
Hi Sree Rama ,
Could you please set the log level of com.pega.bigdata.clientjars to DEBUG and do a test connectivity so that we can see which classes are missing in the log .
Thanks,
Vaspi
Updated: 24 Oct 2017 6:11 EDT
Citigroup Inc.
US
Hi Vaspi,
I am getting the following errors in the logs -
Hi Vaspi,
I am getting the following errors in the logs -
2017-10-23 22:44:57,725 [ttp-nio-9080-exec-10] [TABTHREAD0] [ ] [ EDHPega:01.01.01] (il.classloader.PRClassLoaderDB) WARN laptop-sctksc5p| Proprietary information hidden Admin.EDHPega - Invalid reference to non-public interface from class "com.pegarules.generated.activity.ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87" to "com.pega.pegarules.integration.engine.internal.connect.hbase.HBaseConfig"
2017-10-23 22:44:57,741 [ttp-nio-9080-exec-10] [TABTHREAD0] [ ] [ EDHPega:01.01.01] (il.classloader.PRClassLoaderDB) WARN laptop-sctksc5p| Proprietary information hidden Admin.EDHPega - Invalid reference to non-public interface from class "com.pegarules.generated.activity.ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87" to "com.pega.pegarules.integration.engine.internal.connect.hbase.HBasePingResult"
2017-10-23 22:44:57,794 [ttp-nio-9080-exec-10] [TABTHREAD0] [ ] [ EDHPega:01.01.01] (il.classloader.PRClassLoaderDB) WARN laptop-sctksc5p| Proprietary information hidden Admin.EDHPega - Invalid reference to non-public interface from class "com.pegarules.generated.activity.ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87" to "com.pega.pegarules.integration.engine.internal.connect.hbase.HBaseClient"
2017-10-23 22:44:58,384 [ttp-nio-9080-exec-10] [TABTHREAD0] [ ] [ EDHPega:01.01.01] (ta.clientjars.JarsAvailability) WARN laptop-sctksc5p| Proprietary information hidden Admin.EDHPega - Failed to find class during 3rd party jars check
java.lang.ClassNotFoundException: org.codehaus.jackson.Versioned
at com.pega.pegarules.internal.bootstrap.phase2.PRBaseLoader.findClass(PRBaseLoader.java:308) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_121]
at com.pega.pegarules.internal.bootstrap.phase2.PRBaseLoader.loadClass(PRBaseLoader.java:233) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_121]
at com.pega.pegarules.bootstrap.loader.PRAppLoader.loadClass(PRAppLoader.java:787) ~[prbasejava6.jar:?]
at com.pega.pegarules.bootstrap.loader.PRAppLoader.loadClass(PRAppLoader.java:640) ~[prbasejava6.jar:?]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_121]
at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_121]
at com.pega.bigdata.clientjars.JarsAvailability.obtainNewState(JarsAvailability.java:168) ~[bigdata-connector-7.3.0.jar:?]
at com.pega.bigdata.clientjars.JarsAvailability.jarsAvailable(JarsAvailability.java:153) ~[bigdata-connector-7.3.0.jar:?]
at com.pega.bigdata.hbase.client.HBaseProvider.jarsAvailable(HBaseProvider.java:67) ~[bigdata-connector-7.3.0.jar:?]
at com.pega.pegarules.integration.engine.internal.connect.hbase.JavaHBaseClient.ping(JavaHBaseClient.java:94) ~[bigdata-connector-7.3.0.jar:?]
at com.pega.pegarules.integration.engine.internal.connect.hbase.HBaseClient.ping(HBaseClient.java:295) ~[bigdata-connector-7.3.0.jar:?]
at com.pegarules.generated.activity.ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87.step5_circum0(ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87.java:407) ~[?:?]
at com.pegarules.generated.activity.ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87.perform(ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87.java:139) ~[?:?]
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3586) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.mgmt.Executable.invokeActivity(Executable.java:10770) ~[prprivate.jar:?]
at com.pegarules.generated.activity.ra_action_pzcheckconnectionconfiguration_b7047f4fec50b6d9ec81b40c82606163.step9_circum0(ra_action_pzcheckconnectionconfiguration_b7047f4fec50b6d9ec81b40c82606163.java:754) ~[?:?]
at com.pegarules.generated.activity.ra_action_pzcheckconnectionconfiguration_b7047f4fec50b6d9ec81b40c82606163.perform(ra_action_pzcheckconnectionconfiguration_b7047f4fec50b6d9ec81b40c82606163.java:211) ~[?:?]
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3586) ~[prprivate.jar:?]
at com.pegarules.generated.activity.ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.step1_circum0(ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.java:319) ~[?:?]
at com.pegarules.generated.activity.ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.perform(ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.java:70) ~[?:?]
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3586) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivitiesAlt(ThreadRunner.java:646) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.mgmt.PRThreadImpl.runActivitiesAlt(PRThreadImpl.java:481) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.engineinterface.service.HttpAPI.runActivities(HttpAPI.java:3358) ~[prprivate.jar:?]
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequestInner(EngineAPI.java:394) ~[prenginext.jar:?]
at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1315) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1052) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:907) ~[prprivate.jar:?]
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequest(EngineAPI.java:334) ~[prenginext.jar:?]
at com.pega.pegarules.session.internal.engineinterface.service.HttpAPI.invoke(HttpAPI.java:843) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl._invokeEngine_privact(EngineImpl.java:331) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java:274) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java:251) ~[prprivate.jar:?]
at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngineInner(JNDIEnvironment.java:278) ~[prpublic.jar:?]
at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngine(JNDIEnvironment.java:223) ~[prpublic.jar:?]
at com.pega.pegarules.web.impl.WebStandardImpl.makeEtierRequest(WebStandardImpl.java:596) ~[prwebj2ee.jar:?]
at com.pega.pegarules.web.impl.WebStandardImpl.doPost(WebStandardImpl.java:385) ~[prwebj2ee.jar:?]
at sun.reflect.GeneratedMethodAccessor89.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:370) ~[prbootstrap-7.3.0-77.jar:7.3.0-77]
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:411) ~[prbootstrap-7.3.0-77.jar:7.3.0-77]
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowable(AppServerBridgeToPega.java:224) ~[prbootstrap-api-7.3.0-77.jar:7.3.0-77]
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethod(AppServerBridgeToPega.java:273) ~[prbootstrap-api-7.3.0-77.jar:7.3.0-77]
at com.pega.pegarules.internal.web.servlet.WebStandardBoot.doPost(WebStandardBoot.java:121) ~[prbootstrap-api-7.3.0-77.jar:7.3.0-77]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:661) ~[servlet-api.jar:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[servlet-api.jar:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[catalina.jar:8.5.14]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-websocket.jar:8.5.14]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) ~[catalina.jar:8.5.14]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) ~[catalina.jar:8.5.14]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80) ~[catalina.jar:8.5.14]
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) ~[catalina.jar:8.5.14]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) ~[catalina.jar:8.5.14]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799) ~[tomcat-coyote.jar:8.5.14]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) ~[tomcat-coyote.jar:8.5.14]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861) ~[tomcat-coyote.jar:8.5.14]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455) ~[tomcat-coyote.jar:8.5.14]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat-coyote.jar:8.5.14]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-util.jar:8.5.14]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
2017-10-23 22:45:07,904 [egaRULES-MasterAgent] [ STANDARD] [ ] [ ] ( internal.async.Agent) INFO - System date: Mon Oct 23 22:45:07 IST 2017 Total memory: 1,999,110,144 Free memory: 1,239,909,336 Requestor Count: 14 Shared Pages memory usage: 0% Current number of threads: 120
Can you please help me to identify the jar file that contains these classes?
Thanks,
Sreeram
***Updated by moderator: Lochan to remove proprietary information***
Pegasystems Inc.
IN
Hi Sreeram ,
Could you please download the jars from below location and deploy and restart the server and check the log again
Hi Sreeram ,
Could you please download the jars from below location and deploy and restart the server and check the log again
- https://mvnrepository.com/artifact/org.codehaus.jackson/jackson-mapper-asl/1.9.13
- https://mvnrepository.com/artifact/org.codehaus.jackson/jackson-core-asl/1.9.13
Thanks,
Vaspi
Updated: 24 Oct 2017 6:14 EDT
Citigroup Inc.
US
Thanks Vaspi,
I am able to import the jars as suggested by you and restarted the server. This time I got a different error message "
Thanks Vaspi,
I am able to import the jars as suggested by you and restarted the server. This time I got a different error message "
Below is the log file snippet -
2017-10-24 13:57:29,001 [http-nio-9080-exec-3] [TABTHREAD0] [ ] [ EDHPega:01.01.01] (il.classloader.PRClassLoaderDB) WARN laptop-sctksc5p| Proprietary information hidden Admin.EDHPega - Invalid reference to non-public interface from class "com.pegarules.generated.activity.ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87" to "com.pega.pegarules.integration.engine.internal.connect.hbase.HBaseConfig"
2017-10-24 13:57:29,005 [http-nio-9080-exec-3] [TABTHREAD0] [ ] [ EDHPega:01.01.01] (il.classloader.PRClassLoaderDB) WARN laptop-sctksc5p| Proprietary information hidden Admin.EDHPega - Invalid reference to non-public interface from class "com.pegarules.generated.activity.ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87" to "com.pega.pegarules.integration.engine.internal.connect.hbase.HBasePingResult"
2017-10-24 13:57:29,007 [http-nio-9080-exec-3] [TABTHREAD0] [ ] [ EDHPega:01.01.01] (il.classloader.PRClassLoaderDB) WARN laptop-sctksc5p| Proprietary information hidden Admin.EDHPega - Invalid reference to non-public interface from class "com.pegarules.generated.activity.ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87" to "com.pega.pegarules.integration.engine.internal.connect.hbase.HBaseClient"
2017-10-24 13:57:30,860 [http-nio-9080-exec-3] [TABTHREAD0] [ ] [ EDHPega:01.01.01] ( hadoop.util.Shell) ERROR laptop-sctksc5p| Proprietary information hidden Admin.EDHPega - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318) ~[hadoop-common-2.4.1.jar:?]
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333) ~[hadoop-common-2.4.1.jar:?]
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326) ~[hadoop-common-2.4.1.jar:?]
at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76) ~[hadoop-common-2.4.1.jar:?]
at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:93) ~[hadoop-common-2.4.1.jar:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:77) ~[hadoop-common-2.4.1.jar:?]
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:240) ~[hadoop-common-2.4.1.jar:?]
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:255) ~[hadoop-common-2.4.1.jar:?]
at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:283) ~[hadoop-common-2.4.1.jar:?]
at com.pega.bigdata.authentication.HadoopConfiguration.setupAuthentication(HadoopConfiguration.java:35) ~[bigdata-connector-7.3.0.jar:?]
at com.pega.bigdata.authentication.HadoopAuthenticationManager.handleModifiedConfiguration(HadoopAuthenticationManager.java:326) ~[bigdata-connector-7.3.0.jar:?]
at com.pega.bigdata.authentication.HadoopAuthenticationManager.setupAuthenticationAndLock(HadoopAuthenticationManager.java:235) ~[bigdata-connector-7.3.0.jar:?]
at com.pega.pegarules.integration.engine.internal.connect.hbase.JavaHBaseClient.acquireLock(JavaHBaseClient.java:144) ~[bigdata-connector-7.3.0.jar:?]
at com.pega.pegarules.integration.engine.internal.connect.hbase.JavaHBaseClient.doHBaseOperation(JavaHBaseClient.java:212) ~[bigdata-connector-7.3.0.jar:?]
at com.pega.pegarules.integration.engine.internal.connect.hbase.JavaHBaseClient.doHBaseOperation(JavaHBaseClient.java:200) ~[bigdata-connector-7.3.0.jar:?]
at com.pega.pegarules.integration.engine.internal.connect.hbase.JavaHBaseClient.ping(JavaHBaseClient.java:101) ~[bigdata-connector-7.3.0.jar:?]
at com.pega.pegarules.integration.engine.internal.connect.hbase.HBaseClient.ping(HBaseClient.java:295) ~[bigdata-connector-7.3.0.jar:?]
at com.pegarules.generated.activity.ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87.step5_circum0(ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87.java:407) ~[?:?]
at com.pegarules.generated.activity.ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87.perform(ra_action_pztestconnection_2c5c51b190a8d04621d382ef3f718e87.java:139) ~[?:?]
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3586) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.mgmt.Executable.invokeActivity(Executable.java:10770) ~[prprivate.jar:?]
at com.pegarules.generated.activity.ra_action_pzcheckconnectionconfiguration_b7047f4fec50b6d9ec81b40c82606163.step9_circum0(ra_action_pzcheckconnectionconfiguration_b7047f4fec50b6d9ec81b40c82606163.java:754) ~[?:?]
at com.pegarules.generated.activity.ra_action_pzcheckconnectionconfiguration_b7047f4fec50b6d9ec81b40c82606163.perform(ra_action_pzcheckconnectionconfiguration_b7047f4fec50b6d9ec81b40c82606163.java:211) ~[?:?]
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3586) ~[prprivate.jar:?]
at com.pegarules.generated.activity.ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.step1_circum0(ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.java:319) ~[?:?]
at com.pegarules.generated.activity.ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.perform(ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.java:70) ~[?:?]
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3586) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivitiesAlt(ThreadRunner.java:646) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.mgmt.PRThreadImpl.runActivitiesAlt(PRThreadImpl.java:481) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.engineinterface.service.HttpAPI.runActivities(HttpAPI.java:3358) ~[prprivate.jar:?]
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequestInner(EngineAPI.java:394) ~[prenginext.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1315) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1052) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:907) ~[prprivate.jar:?]
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequest(EngineAPI.java:334) ~[prenginext.jar:?]
at com.pega.pegarules.session.internal.engineinterface.service.HttpAPI.invoke(HttpAPI.java:843) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl._invokeEngine_privact(EngineImpl.java:331) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java:274) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java:251) ~[prprivate.jar:?]
at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngineInner(JNDIEnvironment.java:278) ~[prpublic.jar:?]
at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngine(JNDIEnvironment.java:223) ~[prpublic.jar:?]
at com.pega.pegarules.web.impl.WebStandardImpl.makeEtierRequest(WebStandardImpl.java:596) ~[prwebj2ee.jar:?]
at com.pega.pegarules.web.impl.WebStandardImpl.doPost(WebStandardImpl.java:385) ~[prwebj2ee.jar:?]
at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:370) ~[prbootstrap-7.3.0-77.jar:7.3.0-77]
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:411) ~[prbootstrap-7.3.0-77.jar:7.3.0-77]
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowable(AppServerBridgeToPega.java:224) ~[prbootstrap-api-7.3.0-77.jar:7.3.0-77]
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethod(AppServerBridgeToPega.java:273) ~[prbootstrap-api-7.3.0-77.jar:7.3.0-77]
at com.pega.pegarules.internal.web.servlet.WebStandardBoot.doPost(WebStandardBoot.java:121) ~[prbootstrap-api-7.3.0-77.jar:7.3.0-77]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:661) ~[servlet-api.jar:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[servlet-api.jar:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[catalina.jar:8.5.14]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-websocket.jar:8.5.14]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) ~[catalina.jar:8.5.14]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) ~[catalina.jar:8.5.14]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80) ~[catalina.jar:8.5.14]
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624) ~[catalina.jar:8.5.14]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) ~[catalina.jar:8.5.14]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) ~[catalina.jar:8.5.14]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799) ~[tomcat-coyote.jar:8.5.14]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) ~[tomcat-coyote.jar:8.5.14]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861) ~[tomcat-coyote.jar:8.5.14]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455) ~[tomcat-coyote.jar:8.5.14]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat-coyote.jar:8.5.14]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-util.jar:8.5.14]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
2017-10-24 13:57:35,776 [http-nio-9080-exec-3] [TABTHREAD0] [ ] [ EDHPega:01.01.01] ( hbase.zookeeper.ZKConfig) ERROR laptop-sctksc5p| Proprietary information hidden Admin.EDHPega - no valid quorum servers found in zoo.cfg
2017-10-24 13:57:36,957 [http-nio-9080-exec-3] [TABTHREAD0] [ ] [ EDHPega:01.01.01] ( hbase.zookeeper.ZKConfig) ERROR laptop-sctksc5p| Proprietary information hidden Admin.EDHPega - no valid quorum servers found in zoo.cfg
I am using Cloudera (quickstart) VM distribution in Oracle Virtual Box on Windows machine. Is there anything that I additionally need to do to resolve this issue with Zookeeper?
Thanks,
Sreeram
***Updated by moderator: Lochan to remove proprietary information***
Pegasystems Inc.
NL
Hi Sreeram,
it looks like you have already uploaded all required 3rd party jar but now problem looks like an accessibility problem.
On VMware images of cloudera, it is very common to bind hadoop services on 'localhost' rather than an accessible IP. this also includes zookeeper.
it is easy to check this, pls try to connect to hadoop/zookeeper port 2181 from host machine not within vmware. For this zookeeper should be bind to hostname and it shoudl be exposed by vmware settings.
-Mert
Citigroup Inc.
US
Thanks Mert!
I am able to resolve the connectivity issue and was able to proceed further. I had to add an entry in the Windows Hosts file (in Host Machine) to map the localhost IP to VM's hostname (quickstart.cloudera) to resolve this issue.
I'm now stuck with YARN connectivity - when I configure and do the Test Connectivity for YARN, I am getting the following message -
"Connection cannot be established. Please check your configuration. There are no Hadoop jars available on classpath. See PDN ". Do we need to import any additional Jars for YARN connectivity?
Thanks again for all the help.
Best Regards,
Sreeram
Citigroup Inc.
US
Hi Vaspi,
My problem with HBase connectivity is not resolved yet. I moved on to work on YARN connectivity. However, I am getting a message "Connection cannot be established. Please check your configuration. There are no Hadoop jars available on classpath. See PDN ".
I enabled the logger "com.pega.bigdata.clientjars.jarsavailability" to "DEBUG" mode. But I didnt find any clue in the logs when I do test connectivity.
Do we need any additional logs? Please help.
Thanks
Sreeram
Express Scripts.com
US
In my scenario, I am experiencing below error. Can some one shed light on how to fix this? My Hadoop admin told me to see if we can change Pega to look for node /hbase-secure instead of node /hbase to resolve this but not sure how to fix it!
Connection cannot be established. Please check your configuration.
org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.MasterNotRunningException: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
Express Scripts.com
US
Another issue if anyone can help here please..
HDFS connection is successful after I followed exact PDN guidelines however it is failing next day. Up on researching, we have learnt that kerberos authentication related file etc/krb5.conf is getting overwritten by tomcat every 30 minutes if it is modified by user. In my scenario, it is modified by me(user) because that's an updated file with modified information from our Hadoop admin. My connection stays successful until the mit kerberos token is not expired on hadoop side. Once expired, I had to go and replace etc/ Krb5.conf with the conf file received from hadoop admin to re-establish connection. Can we change the behavior such a way that pega can use another location instead of etc/krb5.conf to avoid conflicts.. Or should we really change the Linux behavior to avoid overwriting?
Pegasystems Inc.
IN
Hi,
Thank you for posting your query in the PSC. This looks like an inactive post and hence, we suggest you create a new post for your query. Click on the Write a Post button that’s available on the top right pane of this page. Once created, please reply back here with the URL of the new post.
You may also refer this discussion link as a reference in the new thread.