We have multiple applications running in our production environment. Each application has work and history extract rules created and a nightly agent is configured for every application to perform extract using "pxExtractDataWithArgs" API. It was working fine till last week. We started seeing BIX Work extract failures for one of our applications in both Pre Prod and production environments (History extracts are working fine. Both, Work and History, are working fine for all other applications ). We ran manually for the missed days and did 'Revalidate & Save' of extract rules for both the environments. There after, its working fine for Pre Prod environment but still seeing the same issue on Production environment. Below is the additional info.

1. Single agent activity calls pxExtractDataWithArgs for both, Work and History, in two different steps in the same order. Does this cause any problem if it takes more time to extract work than history? does the second batch process (history) causes the first batch process to lose connection in any way ?

2. Average number of work records to extracted each night is ~20k -30k. But failed even when there were ~7k records on non peak days. Size of the generated file is ~ (1.5 MB- 6MB). History is also of similar size but working fine.

3. Exception stack trace:

An error occured on executing the query for the report definition - There was a problem getting a list: code: 0 SQLState: 57014 Message: ERROR: canceling statement due to user request

DatabaseException caused by prior exception: org.postgresql.util.PSQLException: ERROR: canceling statement due to user request | SQL Code: 0 | SQL State: 57014 com.pega.pegarules.pub.database.DatabaseException: There was a problem getting a list: code: 0 SQLState: 57014 Message: ERROR: canceling statement due to user request DatabaseException caused by prior exception: org.postgresql.util.PSQLException: ERROR: canceling statement due to user request | SQL Code: 0 | SQL State: 57014 at com.pega.pegarules.data.internal.access.ExceptionInformation.createAppropriateExceptionDueToDBFailure(ExceptionInformation.java:354) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.access.ExceptionInformation.createExceptionDueToDBFailure(ExceptionInformation.java:333) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.access.Lister.convertSqlExceptionToDatabaseException(Lister.java:577) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.access.Lister.listWithResultPresenter(Lister.java:260) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.rd.queryexec.SqlQueryExecutor.executeQuery(SqlQueryExecutor.java:99) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.rd.queryexec.SqlQueryExecutor.executeQuery(SqlQueryExecutor.java:50) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.rd.SQLExecutorResultsCP.executeQuery(SQLExecutorResultsCP.java:110) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.rd.SqlReportExecutor.executeMainReport(SqlReportExecutor.java:156) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.rd.SqlReportExecutor.execute(SqlReportExecutor.java:83) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.rd.SqlReportExecutor.execute(SqlReportExecutor.java:54) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.access.ExtractImpl.getInstanceCount(ExtractImpl.java:2770) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.access.ExtractImpl.doExtract(ExtractImpl.java:1555) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.access.ExtractImpl.runExtract(ExtractImpl.java:1161) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.access.ExtractImpl.init(ExtractImpl.java:890) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.access.ExtractImpl.parseAndRunExtractCommand(ExtractImpl.java:840) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.access.ExtractImpl.executeCommandLineExtract(ExtractImpl.java:769) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.PRDataProviderImpl.executeCommandLineExtract(PRDataProviderImpl.java:1577) ~[prprivate.jar:?] at com.pega.pegarules.session.internal.mgmt.Executable.executeCommandLineExtract(Executable.java:8511) ~[prprivate.jar:?] at com.pegarules.generated.activity.ra_action_pxextractdatawithargs_70c7b1a10890f90316470d704198da0c.step2_circum0(ra_action_pxextractdatawithargs_70c7b1a10890f90316470d704198da0c.java:314) ~[?:?] at com.pegarules.generated.activity.ra_action_pxextractdatawithargs_70c7b1a10890f90316470d704198da0c.perform(ra_action_pxextractdatawithargs_70c7b1a10890f90316470d704198da0c.java:87) ~[?:?] at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3597) ~[prprivate.jar:?] at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivitiesAlt(ThreadRunner.java:646) ~[prprivate.jar:?] at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivities(ThreadRunner.java:572) ~[prprivate.jar:?] at com.pega.pegarules.session.internal.mgmt.PRThreadImpl.runActivities(PRThreadImpl.java:473) ~[prprivate.jar:?] at com.pega.pegarules.session.internal.async.agent.QueueProcessor.executeBatchTask(QueueProcessor.java:297) ~[prprivate.jar:?] at com.pega.pegarules.session.internal.async.agent.QueueProcessor.execute(QueueProcessor.java:362) ~[prprivate.jar:?] at com.pega.pegarules.session.internal.async.BatchRequestorTask.run(BatchRequestorTask.java:1055) ~[prprivate.jar:?] at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1327) ~[prprivate.jar:?] at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1075) ~[prprivate.jar:?] at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:957) ~[prprivate.jar:?] at com.pega.pegarules.session.internal.async.BatchRequestorTask.run(BatchRequestorTask.java:761) ~[prprivate.jar:?] at com.pega.jsr166backport.java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:991) ~[prjsr166java6-3.0.jar:?] at com.pega.jsr166backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:532) ~[prjsr166java6-3.0.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131] Caused by: org.postgresql.util.PSQLException: ERROR: canceling statement due to user request at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2476) ~[postgresql-42.1.1.jar:42.1.1] at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2189) ~[postgresql-42.1.1.jar:42.1.1] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:300) ~[postgresql-42.1.1.jar:42.1.1] at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:428) ~[postgresql-42.1.1.jar:42.1.1] at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354) ~[postgresql-42.1.1.jar:42.1.1] at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:169) ~[postgresql-42.1.1.jar:42.1.1] at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:158) ~[postgresql-42.1.1.jar:42.1.1] at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) ~[?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131] at org.apache.tomcat.jdbc.pool.StatementFacade$StatementProxy.invoke(StatementFacade.java:114) ~[tomcat-jdbc.jar:?] at com.sun.proxy.$Proxy6.execute(Unknown Source) ~[?:?] at com.pega.pegarules.data.internal.store.DatabasePreparedStatementImpl.execute(DatabasePreparedStatementImpl.java:504) ~[prprivate.jar:?] at com.pega.pegarules.data.internal.access.Lister.listWithResultPresenter(Lister.java:238) ~[prprivate.jar:?] ... 30 more

Any pointers would be helpful. Thanks in advance.