I would like to:
1. Load a website "home" page.
2. Enter a search term.
3. Parse the resulting page for content. (I'm okay from this point).
I have a method for doing stages (1) and (2), but they are rather clunky and I'm sure that there must be a better way.
Here's what I do currently:
// ~~> Create a browser, a flag and a "home" page.
// Create browser.
WebBrowser browser = new WebBrowser();
// Create flag for home page loaded.
bool homepageLoaded = false;
// Dummy page (would come from a textbox etc in reality).
string homePage = "http://CompanyName.com/web/";
Then I add a handler and navigate to the page.
// ~~> Add handler and navigate to page.
// Add handler.
browser.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(browser_DocumentCompleted);
// Navigate to web site home page.
browser.Navigate(homePage);
And in the "DocumentCompleted" event, I wait for the home page to be fully loaded. Once this is done, I load the search term.
//private void browser_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
// Wait until the webpage has loaded completely
if (e.Url.AbsoluteUri == homePage)
{
// Set flag.
homepageLoaded = true;
// Create document.
HtmlDocument document = browser.Document;
// Enter search term value.
HtmlElement tb = document.GetElementById("searchTerm");
tb.SetAttribute("value", "12345678");
// Click the button.
HtmlElement btn = document.GetElementById("btnSearch");
btn.InvokeMember("Click");
}
else
{
string url = null;
// Capture new webpage details.
if (homepageLoaded && e.Url.AbsoluteUri.Contains(homePage))
{
// Search result webpage.
url = e.Url.AbsoluteUri.ToString();
// -------------------------------------------------------
// Use this "url" for scraping...(I am okay from here)
// -------------------------------------------------------
}
}
}
As I say, this works but it strikes me as very inefficient. I'm new to this area of coding and would welcome alternative suggestions to this approach.
Thank you.