All sorts of questions here:

1) If you test the report when connected to Pega directly and not through the gateway, how long does it take?  If it takes a long time, then a "real" timeout error is suspected.  If it doesn't take long, then an error is suspected instead of a "real" timeout error.

2) If the report takes a long time, look at designing it to be more specific so it either searches through less records, or returns fewer records, dependin on whether the long time taken is due to a large data set or a large return set.

My guess is that your gateway error has nothing to do with pega prgateway at all. It is just something from the web server. In general, if the time it takes to run a report is longer than certain timeout value set on the web server. You can see those errors. The actual setting depends on your webserver/appserver setup, please indicate your app server type so that I can send you specific info. This is after you have verified that your report definition runs as expected in terms of performance and amount of data as pointed by Eric above.

nginx is load balancer or webserver & application server is Tomcat 7 . We did the timeout settings in webserver level but still extraction of applicatio document is in loading mode.