Question
M1 Ltd
SG
Last activity: 23 Oct 2019 5:42 EDT
Add a node to existing system
Hi,
We are running on a single node for our Pega Platform 7.3. This node is running on a single VM.
Can I add some more nodes to this VM, so that the Pega System can be running more efficiently?
If yes, please share on the detail steps to implement the solution. Any pointer is very much appreciated.
Regards.
***Moderator Edit-Vidyaranjan: Updated Platform Capability***
Cognizant Technology Limited
IN
Hi Tawsern,
We cannot run more than one PRPC instance(Tomcat server) in the same JVM also it's not correct approach for the multi node environments.
Decision Consultant Solutions
GB
You can set up the another Tomcat server as node in your VM but you should be aware of whether the your VM has enough memory and processing power to support more than one node.
Pegasystems Inc.
IN
Hi,
You can make it multinode by changing port of the server for e.g 8080 & 8081.Please make sure that both server is pointing to same database instance.Start both the nodes one by one.After restart you can verify pr_sys_statusnode table.There will be 2 entries with different ip:port.
Please let me know if you need more detail.
Thanks,
Abhinav
M1 Ltd
SG
Thanks for all the advises.
Can I conclude the following:
1) I need to install another Tomcat instance to support additional node
2) For the new Tomcat instance with the new port, I do not need to expose this new port to the users if it is meant for some Pega Processing.
Is there any readup on what is the CPU and RAM required for the VM to support the various node?
Pegasystems Inc.
US
1) I need to install another Tomcat instance to support additional node? Yes
2) For the new Tomcat instance with the new port, I do not need to expose this new port to the users if it is meant for some Pega Processing.? Yes.
I am not sure on the CPU & the RAM part.
Pegasystems Inc.
IN
Hi,
Yes that's correct.One extra point I will add here is please make sure that both the instance are pointing to same database server.Restart node one by one.You will find two entries in pr_sys_statusnode table.
Thanks,
Abhinav
M1 Ltd
SG
Thanks for the input, will explore further on how to do the installation.
By the way, am I correct to say Pega and tomcat should not have any issue running on openJDK Java 8 and 11, and even version 12?
Pegasystems Inc.
IN
Hi,
It depends on the version of pega whether it is supported with that java & tomcat or not .You can check the compatibility of pega versions in below support guide.
https://community.pega.com/knowledgebase/products/platform/deploy?product_version=7101
Thanks,
Abhinav
M1 Ltd
SG
I tried to start up another tomcat instance on the same server, with this setting in the server.xml (1st instance is using port 8443).
<Connector port="8444" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8080" />
However, I encountered this error:
SEVERE [main] org.apache.coyote.AbstractProtocol.init Failed to initialize end point associated with ProtocolHandler ["ajp-nio-8009"]
java.net.BindException: Address already in use
I searched through the server.xml and discovered this setup:
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
Does Pega use this functionality? Can I give another port?
Regards.
Pegasystems Inc.
IN
The address binding will fail for the AJP port, but the server should be started and you should be able to access the environment.
Did you get the server startup message in the startup log?
Pegasystems Inc.
IN
Hi,
Hope you have pointed to same database in context.xml.Please add below line in server.xml & restart node one by one.
<Connector port="8081" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
Don't change any other place.
Please let me know the result.
Thanks,
Abhinav
M1 Ltd
SG
I have uploaded my server.xml. The change done is as highlighted above; the rest remain same as the 1st instance of web server.
<Connector port="8444" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8080" />
The context.xml is just a duplication from the 1st instance, so the connection information is intact.
With regards to Vikash question, No, I do not see the startup message.
This is what I have observed:
When I just run an instant of the web server, there are 2 processes of java:
I have uploaded my server.xml. The change done is as highlighted above; the rest remain same as the 1st instance of web server.
<Connector port="8444" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8080" />
The context.xml is just a duplication from the 1st instance, so the connection information is intact.
With regards to Vikash question, No, I do not see the startup message.
This is what I have observed:
When I just run an instant of the web server, there are 2 processes of java:
[cms_app@cms02d ~]$ ps -ef | grep java
cms_app 23047 22990 0 09:36 pts/0 00:00:00 grep --color=auto java
cms_app 31163 1 3 Oct22 ? 00:38:02 /d01/cms/java-1.8.0-openjdk- Proprietary information hidden.b10-0.el7_6.x86_64/jre/bin/java -Djava.util.logging.config.file=/d01/cms/tomcat/apache-tomcat-8.5.23/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx8192m -Xms2048m -XX:MaxMetaspaceSize=1024m -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses=true - java.awt.headless=true -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -Xloggc:/d01/cms/tomcat/apache-tomcat-8.5.23/logs/2019-10-22_16-09-49-gc.log -Djava.security.egd=file:/dev/urandom -Djavax.net.ssl.trustStore=/d01/cms/java-1.8.0-openjdk- Proprietary information hidden.b10-0.el7_6.x86_64/jre/lib/security/cacerts -Djavax.net.ssl.trustStorePassword=changeit -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -classpath /d01/cms/tomcat/apache-tomcat-8.5.23/bin/bootstrap.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/bin/tomcat-juli.jar -Dcatalina.base=/d01/cms/tomcat/apache-tomcat-8.5.23 -Dcatalina.home=/d01/cms/tomcat/apache-tomcat-8.5.23 -Djava.io.tmpdir=/d01/cms/tomcat/apache-tomcat-8.5.23/temp org.apache.catalina.startup.Bootstrap start
cms_app 31606 31163 0 Oct22 ? 00:08:38 /d01/cms/java-1.8.0-openjdk- Proprietary information hidden.b10-0.el7_6.x86_64/jre/bin/java -ea -javaagent:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/jamm-0.3.0.jar -XX:+CMSClassUnloadingEnabled -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms3971M -Xmx3971M -Xmn200M -XX:+HeapDumpOnOutOfMemoryError -Xss256k -XX:StringTableSize=1000003 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseTLAB -XX:CompileCommandFile=/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/conf/hotspot_compiler -XX:CMSWaitDuration=10000 -XX:+CMSParallelInitialMarkEnabled -XX:+CMSEdenChunksRecordAlways -XX:CMSWaitDuration=10000 -Djava.net.preferIPv4Stack=true -Dcassandra.jmx.local.port=7199 -XX:+DisableExplicitGC -Dlogback.configurationFile=logback.xml -Dcassandra.logdir=/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/logs -Dcassandra.storagedir=/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/data -Dcassandra-foreground=yes -cp /d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/conf:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/build/classes/main:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/build/classes/thrift:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/airline-0.6.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/antlr-runtime-3.5.2.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/apache-cassandra-2.1.14.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/apache-cassandra-clientutil-2.1.14.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/apache-cassandra-thrift-2.1.14.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/commons-cli-1.1.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/commons-codec-1.2.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/commons-lang3-3.1.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/commons-math3-3.2.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/compress-lzf-0.8.4.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/disruptor-3.0.1.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/guava-16.0.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/high-scale-lib-1.0.6.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/jackson-core-asl-1.9.2.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/jamm-0.3.0.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/javax.inject.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/jbcrypt-0.3m.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/jline-1.0.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/jna-4.0.0.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/json-simple-1.1.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/libthrift-0.9.2.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/logback-classic-1.1.2.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/logback-core-1.1.2.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/lz4-1.2.0.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/metrics-core-2.2.0.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/netty-all-4.0.23.Final.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/reporter-config-2.1.0.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/slf4j-api-1.7.2.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/snakeyaml-1.11.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/snappy-java- Proprietary information hidden.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/ST4-4.0.8.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/stream-2.5.2.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/super-csv-2.1.0.jar:/d01/cms/tomcat/apache-tomcat-8.5.23/cassandra/lib/thrift-server-0.3.7.jar org.apache.cassandra.service.CassandraDaemon
After starting up the 2nd instance, I can see 3 java processes, 2 of which are as above, and the 3rd is the new web instance.
However, after a while, the processes will be dropped down to 2; just left with the 2 instances of web server.
I do not see any new node added into the pr_sys_statusnode.
Please advise where have I gone wrong.
Pegasystems Inc.
IN
Hi,
I have mentioned connector port as 8081 but in your code it is 8443, Please add below setting and restart server.
<Connector port="8081" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
Thanks,
Abhinav
M1 Ltd
SG
Thanks for your Patient, Abhinav.
I tried again with what you have suggested. I did see a momentarily display of 2 nodes in pr_sys_statusnodes and then it went back to 1 node.
After a further while, only 2 main tomcat server processes are running.
Regards.
Pegasystems Inc.
IN
Hi,
It means the cluster is getting formed. If 2 tomcat server processes are running it means you have established a multinode environment. Please access both the url e.g
Both should be accessible.
Please let me know If you have any other query.
Thanks,
Abhinav
M1 Ltd
SG
But, that's not I want.
I want separate node so that I can run campaign concurrently.
Besides, when I see only the 2 tomcat processes, I don't think the campaign can be executed.
Regards.
Pegasystems Inc.
IN
Hi,
It means there are two nodes one with port number 8080 & other with 8081. If you want to run a campaign directly then I think you need to define dnodetype as DDS or universal for pega 8.x version.
Thanks,
Abhinav
M1 Ltd
SG
I started the 2 web server instance, and tried to execute a campaign. I got this error:
I started the 2 web server instance, and tried to execute a campaign. I got this error:
com.pega.dsm.dnode.api.dataflow.StageException: Exception in stage: TS Test Insert for DNC Check at com.pega.dsm.dnode.api.dataflow.StageException.create(StageException.java:34) at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageOutputSubscriber.onError(DataFlowStage.java:474) at com.pega.dsm.dnode.api.dataflow.DataFlowStageProcessor.onError(DataFlowStageProcessor.java:86) at com.pega.dsm.dnode.impl.dataflow.strategy.StrategyStageProcessor.onNext(StrategyStageProcessor.java:280) at com.pega.dsm.dnode.api.dataflow.DataFlowStageBatchProcessor.commitBatchInternal(DataFlowStageBatchProcessor.java:130) at com.pega.dsm.dnode.api.dataflow.DataFlowStageBatchProcessor.commitBatch(DataFlowStageBatchProcessor.java:110) at com.pega.dsm.dnode.api.dataflow.DataFlowStageBatchProcessor.onCompleted(DataFlowStageBatchProcessor.java:96) at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onCompleted(DataFlowStage.java:360) at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor$5.process(DataFlowExecutor.java:334) at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor.runEventLoop(DataFlowExecutor.java:203) at com.pega.dsm.dnode.api.dataflow.DataFlow$1.emit(DataFlow.java:287) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:57) at com.pega.dsm.dnode.impl.dataflow.task.strategy.ExecutionStrategy.executeDataFlow(ExecutionStrategy.java:95) at com.pega.dsm.dnode.impl.dataflow.task.strategy.SinglePartitionExecution.executePartitions(SinglePartitionExecution.java:57) at com.pega.dsm.dnode.impl.dataflow.task.ExecutePartitionsTask.execute(ExecutePartitionsTask.java:11) at com.pega.dsm.dnode.impl.dataflow.task.PickupRunTask$ExecutePartitionsRunnable$1.run(PickupRunTask.java:152) at com.pega.dsm.dnode.impl.dataflow.task.PickupRunTask$ExecutePartitionsRunnable$1.run(PickupRunTask.java:149) at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:52) at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:215) at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:90) at com.pega.dsm.dnode.impl.dataflow.task.PickupRunTask$ExecutePartitionsRunnable.run(PickupRunTask.java:149) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at com.pega.dsm.dnode.util.PrpcRunnable$1.run(PrpcRunnable.java:44) at com.pega.dsm.dnode.util.PrpcRunnable$1.run(PrpcRunnable.java:41) at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:52) at com.pega.dsm.dnode.impl.prpc.PrpcThreadFactory$PrpcThread.run(PrpcThreadFactory.java:109) Caused by: com.pega.dsm.dnode.api.ExceptionWithInputRecord: com.pega.dsm.dnode.api.dataflow.StageException: Exception in stage: Event store ... 28 more Caused by: com.pega.dsm.dnode.api.dataflow.StageException: Exception in stage: Event store at com.pega.dsm.dnode.api.dataflow.StageException.create(StageException.java:34) at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onCompleted(DataFlowStage.java:364) at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$SynchronousQueueDataFlowExecutor$4.process(DataFlowExecutor.java:605) at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$SynchronousQueueDataFlowExecutor.runEventLoop(DataFlowExecutor.java:554) at com.pega.dsm.dnode.api.dataflow.DataFlow$5.emit(DataFlow.java:522) at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:335) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:53) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.await(DataObservableImpl.java:99) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.await(DataObservableImpl.java:88) at com.pega.dsm.dnode.impl.dataflow.strategy.DelayedLearning.saveResultsForDelayedLearning(DelayedLearning.java:78) at com.pega.dsm.dnode.impl.dataflow.strategy.StrategyStageProcessor.onNext(StrategyStageProcessor.java:270) ... 27 more Caused by: com.pega.dsm.dnode.api.ExceptionWithInputRecord: All host(s) tried for query failed (no host was tried) at com.pega.decision.eventstore.impl.dataset.eventstore.EventStoreSaveOperation$1.emit(EventStoreSaveOperation.java:124) at com.pega.decision.eventstore.impl.dataset.eventstore.EventStoreDataEmitter.emit(EventStoreDataEmitter.java:49) at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:335) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:53) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.await(DataObservableImpl.java:99) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.await(DataObservableImpl.java:88) at com.pega.dsm.dnode.impl.dataflow.SaveStageProcessor.onNext(SaveStageProcessor.java:107) at com.pega.dsm.dnode.api.dataflow.DataFlowStageBatchProcessor.commitBatchInternal(DataFlowStageBatchProcessor.java:130) at com.pega.dsm.dnode.api.dataflow.DataFlowStageBatchProcessor.commitBatch(DataFlowStageBatchProcessor.java:110) at com.pega.dsm.dnode.api.dataflow.DataFlowStageBatchProcessor.onCompleted(DataFlowStageBatchProcessor.java:96) at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onCompleted(DataFlowStage.java:360) ... 36 more
Please advise.
In addition, do I need to specify all the agents, even those created by Pega, e.g. sending SMS, emails, etc to run on only 1 node? If I do not specify, will Pega pick up the same record by the 2 nodes and send out SMS/emails?
Updated: 23 Oct 2019 5:42 EDT
M1 Ltd
SG
Just to add on, These are the errors that I retrieved from PegaRULES.log
Just to add on, These are the errors that I retrieved from PegaRULES.log
2019-10-23 15:41:43,557 [http-nio-8443-exec-9] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR cms02d.m1.local| Proprietary information hidden tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,599 [ AsyncServices-8] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,605 [ AsyncServices-8] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,607 [ AsyncServices-9] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,615 [ AsyncServices-8] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,618 [ AsyncServices-9] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,620 [ AsyncServices-10] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,621 [ AsyncServices-8] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,625 [ AsyncServices-9] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,627 [ AsyncServices-8] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,627 [ AsyncServices-10] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,636 [ AsyncServices-9] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,638 [ AsyncServices-10] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,639 [ AsyncServices-8] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,643 [ AsyncServices-9] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,648 [ AsyncServices-9] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,651 [ AsyncServices-10] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,656 [ AsyncServices-10] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,661 [ AsyncServices-10] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,729 [http-nio-8443-exec-9] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] ( mgmt.autostreams.HarnessRuntime) ERROR cms02d.m1.local| Proprietary information hidden tawsern.tan - Exception while updating Clipboard Models.
2019-10-23 15:41:43,810 [ AsyncServices-1] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,813 [ AsyncServices-2] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,818 [ AsyncServices-1] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,820 [ AsyncServices-2] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,825 [ AsyncServices-1] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,827 [ AsyncServices-2] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,832 [ AsyncServices-1] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,834 [ AsyncServices-2] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,838 [ AsyncServices-2] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,840 [ AsyncServices-1] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,856 [ AsyncServices-2] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:43,861 [ AsyncServices-1] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,563 [http-nio-8443-exec-2] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR cms02d.m1.local| Proprietary information hidden - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,568 [http-nio-8443-exec-2] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR cms02d.m1.local| Proprietary information hidden - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,573 [http-nio-8443-exec-2] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR cms02d.m1.local| Proprietary information hidden - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,577 [http-nio-8443-exec-2] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR cms02d.m1.local| Proprietary information hidden - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,582 [http-nio-8443-exec-2] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR cms02d.m1.local| Proprietary information hidden - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,586 [http-nio-8443-exec-2] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR cms02d.m1.local| Proprietary information hidden - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,615 [ttp-nio-8443-exec-10] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR cms02d.m1.local| Proprietary information hidden tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,622 [ttp-nio-8443-exec-10] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR cms02d.m1.local| Proprietary information hidden tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,625 [ AsyncServices-4] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,628 [ttp-nio-8443-exec-10] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR cms02d.m1.local| Proprietary information hidden tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,632 [ AsyncServices-4] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,637 [ttp-nio-8443-exec-10] [TABTHREAD3] [ ] [ M1CMS1:01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR cms02d.m1.local| Proprietary information hidden tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,643 [ AsyncServices-4] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,655 [ AsyncServices-4] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,662 [ AsyncServices-4] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)
2019-10-23 15:41:45,674 [ AsyncServices-4] [ STANDARD] [ shared] [ M1CMS1 01.01.01] (Data.Embed_PegaMKT_Measurement_Definition_Interaction.Action) ERROR tawsern.tan - Unable to query VBD: All host(s) tried for query failed (no host was tried)