As you are developing a crawler, you should never send POST requests, that means that you will not be able to access pages that have navigation only using _doPostBack.
Typically webmasters add xml sitemaps that contain links to pages of the site:
http://www.xml-sitemaps.com/about-sitemaps.html[
^]
you can make use of them.