Question

Page navigate method fails, inconsistently.
I have a web application project which require to navigate 100's of webpage and scrape data. Looks like at some point the htmldocument become null and unable to set url for navigate, even though the match rule satisfy.
Example:
www.abcdefg.com/data/form=1, www.abcdefg.com/data/form=2,www.abcdefg.com/data/form=3.
So i have three webpages to navigate and get data. I have interrogated the first page and supplied further url to webpage with Navigate method. The match rule I provide as webpage host, so I was expecting the webpage should match and proceed. Looks like after 10 or 15th iteration I am getting error that HTMLDocument is null. I do follow the wait for create true path. But still on navigate, it fails.
Appreciate any tips and tricks.