Question
Philips Corp
US
Last activity: 30 Dec 2015 15:35 EST
how to use prpcUtils.sh importPegaArchive with split schema
We are using Pega 7.1.7 with split schema
I am trying to set up command line deployment using prpcUtils importPegaArchive
Our split schema is divided into pegadat and pegasys
<pegarules>
<!-- This is a minimum format prconfig.xml file. Only the settings which are required to access settings in the database are included.
All other settings which were formerly located only in this file are now Data-Admin-System-Settings. -->
<env name="initialization/settingsource" value="merged" />
<env name="database/databases/PegaRULES/dataSource" value="java:comp/env/jdbc/PegaRULES"/>
<env name="database/databases/PegaDATA/dataSource" value="java:comp/env/jdbc/PegaRULES"/>
we have a connection string for Pega:
jdbc:oracle:thin:@(DESCRIPTION= (LOAD_BALANCE=ON) (FAILOVER=ON) (ADDRESS_LIST= (ADDRESS=(PROTOCOL = TCP)(HOST = qasolodbl5-v5)(PORT = 1531)) (ADDRESS=(PROTOCOL = TCP)(HOST = qasolodbl6-v6)(PORT = 1531))) (CONNECT_DATA=(SERVICE_NAME=monusql_svc)(SERVER=DEDICATED) (FAILOVER_MODE=(TYPE=SELECT)(METHOD=basic))))
that uses a pegaapp user to login.
I am using pegaapp in my connection to import a Pega Archive but i'm getiting table or view does not exist.
Executing prpcUtils.sh ant script...
We are using Pega 7.1.7 with split schema
I am trying to set up command line deployment using prpcUtils importPegaArchive
Our split schema is divided into pegadat and pegasys
<pegarules>
<!-- This is a minimum format prconfig.xml file. Only the settings which are required to access settings in the database are included.
All other settings which were formerly located only in this file are now Data-Admin-System-Settings. -->
<env name="initialization/settingsource" value="merged" />
<env name="database/databases/PegaRULES/dataSource" value="java:comp/env/jdbc/PegaRULES"/>
<env name="database/databases/PegaDATA/dataSource" value="java:comp/env/jdbc/PegaRULES"/>
we have a connection string for Pega:
jdbc:oracle:thin:@(DESCRIPTION= (LOAD_BALANCE=ON) (FAILOVER=ON) (ADDRESS_LIST= (ADDRESS=(PROTOCOL = TCP)(HOST = qasolodbl5-v5)(PORT = 1531)) (ADDRESS=(PROTOCOL = TCP)(HOST = qasolodbl6-v6)(PORT = 1531))) (CONNECT_DATA=(SERVICE_NAME=monusql_svc)(SERVER=DEDICATED) (FAILOVER_MODE=(TYPE=SELECT)(METHOD=basic))))
that uses a pegaapp user to login.
I am using pegaapp in my connection to import a Pega Archive but i'm getiting table or view does not exist.
Executing prpcUtils.sh ant script...
/fs01/apps1/products/prpc_7.1.7/scripts/utils/prpcUtils.sh importPegaArchive --driverClass oracle.jdbc.OracleDriver --driverJAR /fs01/apps1/pega/jdbc/ojdbc7.jar --dbType oracledate --dbURL jdbc:oracle:thin:@//devsolracl5:1531/monusdl_svc --dbUser pegaapp --dbPassword XXXXXX --archivePath /fs01/apps1/pega/temp/StaticContent/global/ServiceExport/ECareFW_03_01_78.jar
_findSystemName:
[echo] select max(s.pySystemName) from PEGAAPP.pr_data_admin s inner join PEGAAPP.pr_data_admin r on s.pySystemName = r.pySystemName where s.pxobjclass = 'Data-Admin-System' and r.pxObjClass = 'Data-Admin-Requestor'
Unable to execute sql query:
select max(s.pySystemName) from PEGAAPP.pr_data_admin s inner join PEGAAPP.pr_data_admin r on s.pySystemName = r.pySystemName where s.pxobjclass = 'Data-Admin-System' and r.pxObjClass = 'Data-Admin-Requestor': ORA-00942: table or view does not exist
Which user should i be using with a split schema???
Thanks in advance
Accepted Solution
Philips Corp
US
Reran once again
BUILD SUCCESSFUL
Total time: 2 minutes 29 seconds
So, i am going to assume that there is something about the first run (maybe creating all that stuff under user.temp.dir) that takes a very long time to complete.
In Summary:
We are doing a pega Import with prpcUtils.sh in a split schema configuration.
passing most parameters on the command line:
. ./prpcUtils.sh ${PEGA_ACTION} --driverClass ${PEGA_DRIVER_CLASS} --driverJAR ${PEGA_JDBC_DRIVER} --dbType ${PEGA_DB_TYPE} --dbURL ${PEGA_DB_URL} --dbUser ${PEGA_JDBC_USERNM} --dbPassword ${PEGA_JDBC_PW} --archivePath ${PEGA_ARCH_PATH}
PEGA_JDBC_USERNM is a DB alias with access to both the rules and data schema's. It is the user defined for JNDI name java:/jdbc/PegaRULES
.
remaining come from prpcUtils.properties where we set:
#rules schema name : the user name is used for default schema name.
rules.schema.name=pegasys
data.schema.name=pegadat
#User Temp Directory. will use default if not set to valid directory
user.temp.dir=/fs01/home/adm1/admin/scripts/jb-deploy/logs (these are not logs so the directory is misnamed)
logs related to the import go by default to: ./scripts/utils/logs (under PRPC product distribution) Make sure you do not have a prlogging.xml file in your PATH that overrides the default unless that's what you want.
Reran once again
BUILD SUCCESSFUL
Total time: 2 minutes 29 seconds
So, i am going to assume that there is something about the first run (maybe creating all that stuff under user.temp.dir) that takes a very long time to complete.
In Summary:
We are doing a pega Import with prpcUtils.sh in a split schema configuration.
passing most parameters on the command line:
. ./prpcUtils.sh ${PEGA_ACTION} --driverClass ${PEGA_DRIVER_CLASS} --driverJAR ${PEGA_JDBC_DRIVER} --dbType ${PEGA_DB_TYPE} --dbURL ${PEGA_DB_URL} --dbUser ${PEGA_JDBC_USERNM} --dbPassword ${PEGA_JDBC_PW} --archivePath ${PEGA_ARCH_PATH}
PEGA_JDBC_USERNM is a DB alias with access to both the rules and data schema's. It is the user defined for JNDI name java:/jdbc/PegaRULES
.
remaining come from prpcUtils.properties where we set:
#rules schema name : the user name is used for default schema name.
rules.schema.name=pegasys
data.schema.name=pegadat
#User Temp Directory. will use default if not set to valid directory
user.temp.dir=/fs01/home/adm1/admin/scripts/jb-deploy/logs (these are not logs so the directory is misnamed)
logs related to the import go by default to: ./scripts/utils/logs (under PRPC product distribution) Make sure you do not have a prlogging.xml file in your PATH that overrides the default unless that's what you want.
Thank you Celeste for all your help on this. Very much appreciated!!
Pegasystems Inc.
US
Philips Corp
US
Thanks Kevin. I thought i was using the user that PRPC was installed with, but maybe not. I'll review the information in the link you provided with the DB group. Maybe they can help me sort through it.
Thanks again!
Pegasystems Inc.
US
In your prpcUtils.properties file set the rules and data schema names to the appropriate values.
#rules schema name : the user name is used for default schema name.
rules.schema.name=
#data.schema.name=
If your RAP import does not involve schema changes you can use the data schema user as your dbUser to start the engine and do the import. If you cannot run as the schema user then you will need to give your pegaapp user the appropriate privileges to do CRUD operations on everything in the rules and data schemas but your first issue is that the default schema names are not specified.
Philips Home Monitoring
US
Celeste,
I am little confused here with this thread. It sounds like if the prpcUtils.properities file is setup correctly to point to the right rules schema and right data schema then we can login as the data schema to execute the import. If we are doing schema changes then we should be using pegaapp (base schema). However, if you read the link (https://community.pega.com/support/support-articles/prpcutils-script-not-working) that Kevin provided it specifically says. "When you specify which user is used in prpcUtils.properties, make sure you specify the "Admin" user which is configured during the installation of PRPC (not the Base, Data or Rules user).". It specifically saying do not use your base account (pegaapp). So I am misinterpreting some wrong here?
Also, another question I have is that link says specify the "Admin" user which is configured. Is there another entry in the prpcUtils.properties file for an Admin user?
Pegasystems Inc.
US
If the split schema upgrade was done correctly the second pass of the migrate script will actually grant all of the privileges between the data and rules schema that are needed to start the pega engine and run nearly anything within pega. The only things the data schema user will not have privileges to do involve schema changes. So you should be able to run the regular web application or prpcUtils as the data schema user as long as what you are doing does not involve schema changes.
The SA article that Kevin referenced is really telling you the privileges that are needed to run the tool including cases when there are schema changes involved. If you can run as an admin user that is the best option to use.
Regardless of whether you are using the data schema or an admin user it is the pega.jdbc.username property that needs to be set. Assuming that PEGAADM7 is actually the admin user this is what the properties need to be set as, replacing the password, rules and data schema names as appropriate.
pega.jdbc.username=PEGAADM7
pega.jdbc.password=password
rules.schema.name=RulesSchemaName
data.schema.name=DataSchemaName
Philips Corp
US
thanks for you input to this discussion. I've been given access to the authorized user. with that user im currently getting:
[java] SEVERE:
[java] Dec 17, 2015 11:01:30 AM com.pega.pegarules.internal.bootstrap.PRMiniLoader
[java] SEVERE: ============================= Failed to bootstrap PegaRULES =============================
[java] Dec 17, 2015 11:01:30 AM com.pega.pegarules.internal.bootstrap.PRMiniLoader
[java] SEVERE:
[java] Dec 17, 2015 11:01:30 AM com.pega.pegarules.internal.bootstrap.PRMiniLoader
[java] SEVERE: Could not load the java class 'com/pega/pegarules/internal/bootstrap/phase2/PRBootstrapImpl.class' from the database table 'PEGAADM7.pr_engineclasses'
[java] Dec 17, 2015 11:01:30 AM com.pega.pegarules.internal.bootstrap.PRMiniLoader
[java] SEVERE: This is usually due to an empty 'PEGAADM7.pr_engineclasses' table, or possibly it does not contain the configured codeset version.
[java] Dec 17, 2015 11:01:30 AM com.pega.pegarules.internal.bootstrap.PRMiniLoader
Does this mean that i still do not have the correct credentials?
Pegasystems Inc.
US
Is PEGAADM7 the name of your rules schema?
Please log into the database as the user who is running the tool and run this database query:
select distinct pzcodesetversion, pzcodeset from PEGAADM7.pr_engineclasses;
Philips Corp
US
Sorry for the lapse in responses. Out on holiday last week.
Attempted using the base user again today (pegaadm) with rules and data schema filled in with the correct pegasys and pegadat schema names. I got much further although still not a satisfying conclusion. Script got to the following point and then just stopped. I killed the update process after 30 minutes.
[java] 2015-12-28 17:12:31,771 [ main] [ ] [ ] [ ] ( internal.mgmt.PRNodeImpl) INFO - Using codeset version and timestamp map: {pega-enginecode=07-10-15@2015-04-24 14:24:00.0, customer=06-01-01@2015-05-21 13:33:00.0, pegafw-channelservices=07-13-01@2015-09-21 11:13:00.0}
[java] 2015-12-28 17:12:33,025 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Generated source : file://runtime:/PRGenJava/
[java] 2015-12-28 17:12:33,025 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Generated classes: file://runtime:/PRGenClasses/
[java] 2015-12-28 17:12:33,026 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Web-tier real root path: null
Sorry for the lapse in responses. Out on holiday last week.
Attempted using the base user again today (pegaadm) with rules and data schema filled in with the correct pegasys and pegadat schema names. I got much further although still not a satisfying conclusion. Script got to the following point and then just stopped. I killed the update process after 30 minutes.
[java] 2015-12-28 17:12:31,771 [ main] [ ] [ ] [ ] ( internal.mgmt.PRNodeImpl) INFO - Using codeset version and timestamp map: {pega-enginecode=07-10-15@2015-04-24 14:24:00.0, customer=06-01-01@2015-05-21 13:33:00.0, pegafw-channelservices=07-13-01@2015-09-21 11:13:00.0}
[java] 2015-12-28 17:12:33,025 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Generated source : file://runtime:/PRGenJava/
[java] 2015-12-28 17:12:33,025 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Generated classes: file://runtime:/PRGenClasses/
[java] 2015-12-28 17:12:33,026 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Web-tier real root path: null
[java] 2015-12-28 17:12:33,026 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Web-tier temporary path: null
[java] 2015-12-28 17:12:33,026 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - PegaRULES real root path: /tmp/pr_bogus_root_directory
[java] 2015-12-28 17:12:33,026 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - PegaRULES effective root path: file://default:/
[java] 2015-12-28 17:12:33,026 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - PegaRULES effective temp path: /tmp/PegaInstallTemp-28-December-2015-17.07.52/pr_temp
[java] 2015-12-28 17:12:33,026 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Using genSourcePath: file://runtime:/PRGenJava/
[java] 2015-12-28 17:12:33,026 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Using genClassPath (database): pegajdbc://30763712:1/
[java] 2015-12-28 17:12:33,026 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Using genClassPath (files): file://runtime:/PRGenClasses/
[java] 2015-12-28 17:12:33,026 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Using external classes in: <not using external classes>
[java] 2015-12-28 17:12:33,026 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Using external jars in: <not using external jars>
[java] 2015-12-28 17:12:33,866 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - Servlet classes: (none)
[java] 2015-12-28 17:12:33,866 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - JSP classes: (none)
[java] 2015-12-28 17:12:33,866 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - EJB classes: (none)
[java] 2015-12-28 17:12:33,866 [ devlinapsl1] [ ] [ ] [ ] (rnal.mgmt.InitialConfiguration) INFO - JMS classes: (none)
[java] 2015-12-28 17:12:34,194 [ devlinapsl1] [ ] [ ] [ ] (rnal.access.CacheContainerImpl) INFO - Using the Dynamic Cache Probation Policy - cache after 2 accesses AND with a blob increment of 102400.
[java] 2015-12-28 17:12:34,733 [ devlinapsl1] [ ] [ ] [ ] (pcentric.RACacheAppCentricImpl) INFO - Prune threshold is 10000
[java] 2015-12-28 17:12:34,733 [ devlinapsl1] [ ] [ ] [ ] (pcentric.RACacheAppCentricImpl) INFO - mFUAReusableEnabled = true, mConstructorReuseEnabled = false, mFUAReusableEnabledStream = true
[java] 2015-12-28 17:12:34,794 [ devlinapsl1] [ ] [ ] [ ] (pcentric.RACacheAppCentricImpl) INFO - Using RACacheAppCentricImpl for assembly cache implementations
[java] 2015-12-28 17:12:34,794 [ devlinapsl1] [ ] [ ] [ ] (pcentric.RACacheAppCentricImpl) INFO - Assembly avoidance will use branch ruleset optimization
[java] 2015-12-28 17:12:34,794 [ devlinapsl1] [ ] [ ] [ ] (nal.cache.AssemblyCacheWrapper) INFO - Assembly Cache Wrapper: configuredCacheType = AC
[java] 2015-12-28 17:12:34,966 [ devlinapsl1] [ ] [ ] [ ] ( internal.mgmt.PRNodeImpl) INFO - MBean registered via PRNodeImpl: 6030b5f2d1ef1bacc377fe017996ae9c
[java] Located App Server independent MBean security configuration: pegajdbc://30763712:0/prresources.jar!//prmbeans.properties
[java] 2015-12-28 17:12:34,977 [ devlinapsl1] [ ] [ ] [ ] ( priv.management.MBeanSecurity) INFO - The following MBean security configuration has been applied:
[java] 2015-12-28 17:12:34,977 [ devlinapsl1] [ ] [ ] [ ] ( priv.management.MBeanSecurity) INFO - deny.RequestorManagement.Clipboard=true
[java] 2015-12-28 17:12:34,977 [ devlinapsl1] [ ] [ ] [ ] ( priv.management.MBeanSecurity) INFO - deny.RequestorManagement.RequestorDetails=true
[java] 2015-12-28 17:12:34,977 [ devlinapsl1] [ ] [ ] [ ] ( priv.management.MBeanSecurity) INFO - deny.DatabaseManagement.DatabaseConnectionDetails=true
[java] 2015-12-28 17:12:35,663 [ devlinapsl1] [ ] [ ] [ ] (riv.management.MBeanManagement) INFO - MBean Management (Web) initialized for DefaultDomain
[java] 2015-12-28 17:12:35,696 [ devlinapsl1] [ ] [ ] [ ] ( internal.mgmt.PRRequestorImpl) INFO - RecordWorkInProgress is not enabled
[java] 2015-12-28 17:12:35,916 [ devlinapsl1] [ ] [ ] [ ] (orization.SessionAuthorization) INFO - RSL Update Behavior: Eager
[java] 2015-12-28 17:12:36,124 [ devlinapsl1] [ ] [ ] [ ] ( internal.access.DatabaseImpl) INFO - Bytes input per interaction warning threshold: 50M
[java] 2015-12-28 17:12:36,127 [ devlinapsl1] [ ] [ ] [ ] (ternal.dictionary.ClassMapImpl) INFO - ClassMap using multi-tenant environment: false
[java] 2015-12-28 17:12:36,127 [ devlinapsl1] [ ] [ ] [ ] (ternal.dictionary.ClassMapImpl) INFO - Reading the class group definitions
[java] 2015-12-28 17:12:37,537 [ devlinapsl1] [ ] [ ] [ ] (ternal.dictionary.ClassMapImpl) INFO - Read 31 class group definitions
[java] 2015-12-28 17:12:37,537 [ devlinapsl1] [ ] [ ] [ ] (ternal.dictionary.ClassMapImpl) INFO - Reading the class definitions
[java] 2015-12-28 17:18:57,578 [ devlinapsl1] [ ] [ ] [ ] (ternal.dictionary.ClassMapImpl) INFO - Read 5682 class definitions
[java] 2015-12-28 17:19:05,489 [ devlinapsl1] [ ] [ ] [ ] ( internal.access.DatabaseImpl) INFO - Database product: Oracle
[java] 2015-12-28 17:19:05,489 [ devlinapsl1] [ ] [ ] [ ] ( internal.access.DatabaseImpl) INFO - Database version: Oracle Database 11g Enterprise Edition Release Proprietary information hidden.0 - 64bit Production
[java] With the Automatic Storage Management option
[java] 2015-12-28 17:19:05,489 [ devlinapsl1] [ ] [ ] [ ] ( internal.access.DatabaseImpl) INFO - Driver: Oracle JDBC driver
[java] 2015-12-28 17:19:05,489 [ devlinapsl1] [ ] [ ] [ ] ( internal.access.DatabaseImpl) INFO - Driver version: Proprietary information hidden.0
[java] 2015-12-28 17:19:05,489 [ devlinapsl1] [ ] [ ] [ ] ( internal.access.DatabaseImpl) INFO - Version of JDBC supported by driver: 4.1
[java] PRPC default rounding mode set to HALF_EVEN
[java] 2015-12-28 17:21:27,569 [ LOAD_INSTANCEFILTER] [ STANDARD] [ ] [ ] ( external.async.IStartupTask) INFO - load instance filter ..starting
[java] 2015-12-28 17:21:27,570 [ LOAD_INSTANCEFILTER] [ STANDARD] [ ] [ ] (al.access.InstanceExistsFilter) INFO - Scanning instance ids
[java] 2015-12-28 17:21:27,651 [AD_DECLARATIVECOMMON] [ STANDARD] [ ] [ ] ( external.async.IStartupTask) INFO - load getDeclarativeCommon ..starting
[java] 2015-12-28 17:21:27,654 [AD_DECLARATIVECOMMON] [ STANDARD] [ ] [ ] ( external.async.IStartupTask) INFO - load getDeclarativeCommon ..done
[java] 2015-12-28 17:23:05,510 [EPAGEDEFINITIONCACHE] [ STANDARD] [ ] [ ] ( external.async.IStartupTask) INFO - load Declarative Page Definition Cache ..starting
[java] 2015-12-28 17:23:43,844 [EPAGEDEFINITIONCACHE] [ STANDARD] [ ] [ ] ( external.async.IStartupTask) INFO - load Declarative Page Definition Cache ..done
Pegasystems Inc.
US
Please attach the prpcUtils.properties file that you are using as well as the complete log from the utils/logs directory.
Philips Corp
US
Hi Celeste,
Please find prpcUtils.properties and log file attached.
Regards
Dave Ronne
Sr. Production Support Analyst
Philips - Home Monitoring IT
Direct: 508-988-1705
Mobile: 508-361-1836
[email protected]<mailto:[email protected]>
Philips Corp
US
Hi Celeste, i responded back to the email ([email protected]) i received this morning with the files attached.
If there is a way to attach them to this discussion i will but i dont know what that is.
Pegasystems Inc.
US
Emailing them worked. If you want to attach them to the discussion instead of emailing you would click on "Use advanced editor" in the top right of the your discussion post and then at the bottom right of the box you will see a paperclip icon labeled Attach.
Anyway back to your issue. You are setting many of the properties that you need for the import via command line parameters but the user temp directory is not one of them so it is default to the windows location in the prpcUtils.properties file which is not valid. Start by updating this property to a valid directory location which the user running the tool can write to.
user.temp.dir=C:/pega_temp
You also seem to have /fs01/home/adm1/prlogging.xml on your path so this version is being used instead of the one that is auto-generated by the tool. If you want to use this one then you need to update it as the location that it is trying to write the log files to is not valid in this context. That is why your log files are not being written to as they should. Or instead of updating this file you could just remove it from the path so that default one is used. Hopefully with these changes we will actually get an error message or more of the log output.
Philips Corp
US
Thanks Celeste.
I'm running with new configuration but i dont see any log files in the user.temp.dir location. instead i see:
extractedFiles load oracledate.conf prbootstrap.properties prconfig.xml pr_temp
Pegasystems Inc.
US
The log files shouldn't be going to the user.temp.dir location. It should be putting the directies/files that you have listed to that location. The OOB prlogging.xml file that the tool generates should be in that directory location. If you made the second change to remove your /fs01/home/adm1/prlogging.xml file from the path then this OOB prlogging.xml will be used and the log files will go into the utils/logs directory. If you just updated your version of prlogging.xml then they will go to wherever your version of prlogging.xml says to put them.
Philips Corp
US
Hi Celeste,
i removed the prlogging.xml file from the path so we are using the default location which appears to be placing the logs in the ./scripts/utils/logs folder under the PRPC distribution.
PRPC-RuleInstaller*.log file stopped recording information 30 minutes ago bu the PRPC-RulesInstaller-ALERT*.log file keeps spitting out the following:
2015-12-29 18:51:52,877 GMT*7*PEGA0005*10851*500*f5e8b28b1b0f3b4047409ea4ad679b10*NA*NA*B9823D5FA1245C88717CFA2488A871085*none*PegaSample*null*4ff0c05daaa08916c2fe59eb7b39accc*N*0*130*StartupTaskUtil LOAD_INSTANCEFILTER*STANDARD*com.pega.pegarules.data.internal.access.DatabasePreparedStatementImpl*NA*NA****NA*NA*NA*NA*NA*NA*NA*Database operation took more than the threshold of 500 ms: 10,851 ms SQL: SELECT COUNT(DISTINCT pxInsId) FROM PEGASYS.pr4_rule WHERE pxObjClass = ?*
2015-12-29 18:51:57,388 GMT*7*PEGA0005*4506*500*f5e8b28b1b0f3b4047409ea4ad679b10*NA*NA*B9823D5FA1245C88717CFA2488A871085*none*PegaSample*null*4ff0c05daaa08916c2fe59eb7b39accc*N*0*131*StartupTaskUtil LOAD_INSTANCEFILTER*STANDARD*com.pega.pegarules.data.internal.access.DatabasePreparedStatementImpl*NA*NA****NA*NA*NA*NA*NA*NA*NA*Database operation took more than the threshold of 500 ms: 4,506 ms SQL: SELECT COUNT(DISTINCT pxInsId) FROM PEGASYS.pr4_rule WHERE pxObjClass = ?*
Hi Celeste,
i removed the prlogging.xml file from the path so we are using the default location which appears to be placing the logs in the ./scripts/utils/logs folder under the PRPC distribution.
PRPC-RuleInstaller*.log file stopped recording information 30 minutes ago bu the PRPC-RulesInstaller-ALERT*.log file keeps spitting out the following:
2015-12-29 18:51:52,877 GMT*7*PEGA0005*10851*500*f5e8b28b1b0f3b4047409ea4ad679b10*NA*NA*B9823D5FA1245C88717CFA2488A871085*none*PegaSample*null*4ff0c05daaa08916c2fe59eb7b39accc*N*0*130*StartupTaskUtil LOAD_INSTANCEFILTER*STANDARD*com.pega.pegarules.data.internal.access.DatabasePreparedStatementImpl*NA*NA****NA*NA*NA*NA*NA*NA*NA*Database operation took more than the threshold of 500 ms: 10,851 ms SQL: SELECT COUNT(DISTINCT pxInsId) FROM PEGASYS.pr4_rule WHERE pxObjClass = ?*
2015-12-29 18:51:57,388 GMT*7*PEGA0005*4506*500*f5e8b28b1b0f3b4047409ea4ad679b10*NA*NA*B9823D5FA1245C88717CFA2488A871085*none*PegaSample*null*4ff0c05daaa08916c2fe59eb7b39accc*N*0*131*StartupTaskUtil LOAD_INSTANCEFILTER*STANDARD*com.pega.pegarules.data.internal.access.DatabasePreparedStatementImpl*NA*NA****NA*NA*NA*NA*NA*NA*NA*Database operation took more than the threshold of 500 ms: 4,506 ms SQL: SELECT COUNT(DISTINCT pxInsId) FROM PEGASYS.pr4_rule WHERE pxObjClass = ?*
2015-12-29 18:51:58,154 GMT*7*PEGA0005*736*500*f5e8b28b1b0f3b4047409ea4ad679b10*NA*NA*B9823D5FA1245C88717CFA2488A871085*none*PegaSample*null*4ff0c05daaa08916c2fe59eb7b39accc*N*0*132*StartupTaskUtil LOAD_INSTANCEFILTER*STANDARD*com.pega.pegarules.data.internal.access.DatabasePreparedStatementImpl*NA*NA****NA*NA*NA*NA*NA*NA*NA*Database operation took more than the threshold of 500 ms: 736 ms SQL: SELECT COUNT(DISTINCT pxInsId) FROM PEGASYS.pr4_rule WHERE pxObjClass = ?*
Pegasystems Inc.
US
Is the PRPC-RuleInstaller*.log at the same point as the out.txt file that you posted earlier? If you search the Alert log do you see a PEGA0008?
How far away is the database server from the machine you are running prpcUtils from? Is it in the same data center? If you are getting a lot of PEGA0005 alerts still it shows that it is still running but the database performance is poor so that is why it is taking so long.
Philips Corp
US
finally completed.
BUILD SUCCESSFUL
Total time: 51 minutes 54 seconds
But why on earth would it take 51 mintues?
From the command line it would take less than 5 minutes.
Philips Corp
US
reran test with Golden Gate replication turned off.
BUILD SUCCESSFUL
Total time: 2 minutes 53 seconds
That's better but we do NOT turn off OGG when deploying from the development console so why do we need to do it when deploying with pegaImport? What is the difference?
Pegasystems Inc.
US
Post both the PRPC-RuleInstaller and Alert log from both runs so we can see where all of the time was spent.
Philips Corp
US
is it possible to get a list of the tables being touched by the import process? We could compare this to the list of tables we are replicating. DBA thnks it likely that the pegaimport was waiting for access to tables which were being replicated.
Pegasystems Inc.
US
The alert logs will tell us what queries took a long time. We can also see from the logs if the time was spent trying to start the engine or on the import itself.
Philips Corp
US
Should have slow running instance (w/OGG) followed by fast running instance (w/o OGG).
Regards
Dave Ronne
Sr. Production Support Analyst
Philips - Home Monitoring IT
Direct: 508-988-1705
Mobile: 508-361-1836
[email protected]<mailto:[email protected]>
Philips Corp
US
between the two runs, the pertinent part seems to be:
slow run with OGG on:
2015-12-29 13:14:41,319 [EPAGEDEFINITIONCACHE] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load Declarative Page Definition Cache ..starting
2015-12-29 13:14:41,319 [AD_DECLARATIVECOMMON] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load getDeclarativeCommon ..starting
2015-12-29 13:14:41,321 [AD_DECLARATIVECOMMON] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load getDeclarativeCommon ..done
2015-12-29 13:15:06,644 [EPAGEDEFINITIONCACHE] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load Declarative Page Definition Cache ..done
2015-12-29 13:53:23,426 [ LOAD_INSTANCEFILTER] [ STANDARD] [ ] (al.access.InstanceExistsFilter) INFO - Added 121297 entries to the instance exists filter in 2,322.087000 seconds with a false positive rate of 0.001
2015-12-29 13:53:23,429 [ LOAD_INSTANCEFILTER] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load instance filter ..done
fast run with OGG off:
2015-12-29 14:23:55,957 [EPAGEDEFINITIONCACHE] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load Declarative Page Definition Cache ..starting
between the two runs, the pertinent part seems to be:
slow run with OGG on:
2015-12-29 13:14:41,319 [EPAGEDEFINITIONCACHE] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load Declarative Page Definition Cache ..starting
2015-12-29 13:14:41,319 [AD_DECLARATIVECOMMON] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load getDeclarativeCommon ..starting
2015-12-29 13:14:41,321 [AD_DECLARATIVECOMMON] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load getDeclarativeCommon ..done
2015-12-29 13:15:06,644 [EPAGEDEFINITIONCACHE] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load Declarative Page Definition Cache ..done
2015-12-29 13:53:23,426 [ LOAD_INSTANCEFILTER] [ STANDARD] [ ] (al.access.InstanceExistsFilter) INFO - Added 121297 entries to the instance exists filter in 2,322.087000 seconds with a false positive rate of 0.001
2015-12-29 13:53:23,429 [ LOAD_INSTANCEFILTER] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load instance filter ..done
fast run with OGG off:
2015-12-29 14:23:55,957 [EPAGEDEFINITIONCACHE] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load Declarative Page Definition Cache ..starting
2015-12-29 14:23:55,958 [AD_DECLARATIVECOMMON] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load getDeclarativeCommon ..starting
2015-12-29 14:23:55,960 [AD_DECLARATIVECOMMON] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load getDeclarativeCommon ..done
2015-12-29 14:24:03,177 [ LOAD_INSTANCEFILTER] [ STANDARD] [ ] (al.access.InstanceExistsFilter) INFO - Added 121297 entries to the instance exists filter in 7.227000 seconds with a false positive rate of 0.001
2015-12-29 14:24:03,177 [ LOAD_INSTANCEFILTER] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load instance filter ..done
2015-12-29 14:24:05,571 [EPAGEDEFINITIONCACHE] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load Declarative Page Definition Cache ..done
fast run with OGG back on:
2015-12-29 14:50:25,713 [EPAGEDEFINITIONCACHE] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load Declarative Page Definition Cache ..starting
2015-12-29 14:50:33,227 [ LOAD_INSTANCEFILTER] [ STANDARD] [ ] (al.access.InstanceExistsFilter) INFO - Added 121297 entries to the instance exists filter in 7.513000 seconds with a false positive rate of 0.001
2015-12-29 14:50:33,228 [ LOAD_INSTANCEFILTER] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load instance filter ..done
2015-12-29 14:50:36,469 [EPAGEDEFINITIONCACHE] [ STANDARD] [ ] ( external.async.IStartupTask) INFO - load Declarative Page Definition Cache ..done
Is there some kind of caching being done the first time we run an import that is not done in subsequent runs?
Pegasystems Inc.
US
If you look at the alert logs there are a lot of select and an insert statement from the first pass which took a long time. The time seems to have been in InstanceExistsFilter building the filter cache so at least some of this would probably only be done during the first pass. Even if those were only done on the first pass the time that these database operations were taking seems to indicate to me that there was something else in play here as well. There were 50+ alerts related to queries on the PEGASYS.pr4_rule table.
This query (with various values for pxObjClass) took 600 ms at times and up to 5 minutes at others.
SELECT COUNT(DISTINCT pxInsId) FROM PEGASYS.pr4_rule WHERE pxObjClass = ?*
There was an insert (not sure how many rows) into PEGASYS.pr_sys_ruleset_index which took 5+ minutes.
Accepted Solution
Philips Corp
US
Reran once again
BUILD SUCCESSFUL
Total time: 2 minutes 29 seconds
So, i am going to assume that there is something about the first run (maybe creating all that stuff under user.temp.dir) that takes a very long time to complete.
In Summary:
We are doing a pega Import with prpcUtils.sh in a split schema configuration.
passing most parameters on the command line:
. ./prpcUtils.sh ${PEGA_ACTION} --driverClass ${PEGA_DRIVER_CLASS} --driverJAR ${PEGA_JDBC_DRIVER} --dbType ${PEGA_DB_TYPE} --dbURL ${PEGA_DB_URL} --dbUser ${PEGA_JDBC_USERNM} --dbPassword ${PEGA_JDBC_PW} --archivePath ${PEGA_ARCH_PATH}
PEGA_JDBC_USERNM is a DB alias with access to both the rules and data schema's. It is the user defined for JNDI name java:/jdbc/PegaRULES
.
remaining come from prpcUtils.properties where we set:
#rules schema name : the user name is used for default schema name.
rules.schema.name=pegasys
data.schema.name=pegadat
#User Temp Directory. will use default if not set to valid directory
user.temp.dir=/fs01/home/adm1/admin/scripts/jb-deploy/logs (these are not logs so the directory is misnamed)
logs related to the import go by default to: ./scripts/utils/logs (under PRPC product distribution) Make sure you do not have a prlogging.xml file in your PATH that overrides the default unless that's what you want.
Reran once again
BUILD SUCCESSFUL
Total time: 2 minutes 29 seconds
So, i am going to assume that there is something about the first run (maybe creating all that stuff under user.temp.dir) that takes a very long time to complete.
In Summary:
We are doing a pega Import with prpcUtils.sh in a split schema configuration.
passing most parameters on the command line:
. ./prpcUtils.sh ${PEGA_ACTION} --driverClass ${PEGA_DRIVER_CLASS} --driverJAR ${PEGA_JDBC_DRIVER} --dbType ${PEGA_DB_TYPE} --dbURL ${PEGA_DB_URL} --dbUser ${PEGA_JDBC_USERNM} --dbPassword ${PEGA_JDBC_PW} --archivePath ${PEGA_ARCH_PATH}
PEGA_JDBC_USERNM is a DB alias with access to both the rules and data schema's. It is the user defined for JNDI name java:/jdbc/PegaRULES
.
remaining come from prpcUtils.properties where we set:
#rules schema name : the user name is used for default schema name.
rules.schema.name=pegasys
data.schema.name=pegadat
#User Temp Directory. will use default if not set to valid directory
user.temp.dir=/fs01/home/adm1/admin/scripts/jb-deploy/logs (these are not logs so the directory is misnamed)
logs related to the import go by default to: ./scripts/utils/logs (under PRPC product distribution) Make sure you do not have a prlogging.xml file in your PATH that overrides the default unless that's what you want.
Thank you Celeste for all your help on this. Very much appreciated!!