Question
Ford Motors Private Limited
IN
Last activity: 28 May 2019 4:23 EDT
Reading HTML table into DOM table which has Huge records
I have a HTML table which has bulk records and it loads in a single page. i am not able to interrogate using DOM TABLE since the table is having huge records.. i need to retrieve entire table and move it to excel.
Could anyone suggest the possible ways?
Pegasystems Inc.
US
Can you clone one row at a time and parse it out that way?
Ford Motors Private Limited
IN
Hi Grona,
Thanks for your response. I tried to clone, but couldn't find the element id. Could you please provide steps if any.
Ford Motors Private Limited
IN
More details about the issue:
Issue Description:
We are trying to interrogate a HTML table which holds huge records(around 5000 dynamically). This dynamic table is loaded in a single webpage based on certain filter conditions. While Interrogating via HTML table designer, we are not able to retrieve the controls of the table. Challenge here is, the Table data is getting refreshed frequently.
Options Explored:
- Tried interrogating via HTML Table Designer –Tried reducing the records while interrogating and captured the control. But the table is not static and gets refreshed.
- Tried via SelectElement – This identifies the table, but data retrieval via loop takes longer time. The dataset is not giving any output datatable and hence we are retrieving data in loop.
- Tried via GetClones – Not able to identify the element id of the particular row/cell match and hence we are not able to proceed.
Pegasystems Inc.
US
Tried via SelectElement – This identifies the table, but data retrieval via loop takes longer time. The dataset is not giving any output datatable and hence we are retrieving data in loop.
This is to be expected for large data sets... Matching to each clone requires the product to iterate through every item. There really isn't a product change that can be made to address this limitation. So you really only have one or two other options. You might have more success by using JavaScript to extract the data or a C# script to extract and parse the data. I find that it is quicker to do it like this. I'd probably try doing it like below:
Tried via SelectElement – This identifies the table, but data retrieval via loop takes longer time. The dataset is not giving any output datatable and hence we are retrieving data in loop.
This is to be expected for large data sets... Matching to each clone requires the product to iterate through every item. There really isn't a product change that can be made to address this limitation. So you really only have one or two other options. You might have more success by using JavaScript to extract the data or a C# script to extract and parse the data. I find that it is quicker to do it like this. I'd probably try doing it like below:
- Interrogate the table tag (but not the rows or cells)
- Try extracting the <table> to a C# script and then parse the data you need into a DataTable
- Import the DataTable to Excel
Ford Motors Private Limited
IN
we have tried the above option and there was no luck.
This issue is still under investigation under SR-C93456
Ford Motors Private Limited
IN
we have closed this for now with an mitigation plan.we are applying the filter conditions of the table and retrieving the records. It was suggested by support that we need to wait until the table is loaded on which the page match rule will match the page.
Pegasystems Inc.
US
Thanks for letting us know! Keep us updated with how it goes!