Introduction
This code sample is intended to be used as a reference for handling the WebRequest
and WebResponse
objects in the System.Net
namespace as well as how to implement the Event-Based Asynchronous Pattern as defined on the MSDN website.
Working the WebRequest and WebResponse
Using the WebRequest
object couldn't be simpler. The following code snippet will create a WebRequest
for www.google.com and get a response.
Dim request As WebRequest = WebRequest.Create("http://www.google.com")
Dim response As WebResponse = request.GetResponse()
Processing the WebResponse
is a little trickier. First, you have to read the response from a stream, and then, based on the WebResponse.ContentType
, you need to figure out how to turn that into the desired output. The following function will take a WebResponse
and read the result into a String
, Image
, or Byte
array.
Public Shared Function ProcessResponseStream( _
ByVal response As WebResponse, _
Optional ByVal defaultCharset As String = "utf-8") _
As Object
Dim st As IO.Stream
st = response.GetResponseStream()
Dim mem As New IO.MemoryStream()
Dim buffer(1024) As Byte
Dim read As Integer = 0
Try
Do
read = st.Read(buffer, 0, buffer.Length)
mem.Write(buffer, 0, read)
Loop While read > 0
st.Close()
response.Close()
mem.Position = 0
Dim contentType As ContentType
contentType = _
New ContentType(response.ContentType)
Select Case contentType.Type
Case "text"
Dim result(CInt(mem.Length)) As Byte
mem.Read(result, 0, CInt(mem.Length))
Dim charset As String
charset = contentType.GetValue( _
"charset", _
defaultCharset)
Dim enc As Encoding
enc = Encoding.GetEncoding(charset)
Return enc.GetString(result)
Case "image"
Return Image.FromStream(mem)
Case Else
Dim result(CInt(mem.Length)) As Byte
mem.Read(result, 0, CInt(mem.Length))
Return result
End Select
Finally
mem.Close()
End Try
End Function
If a web resource requires authentication, you can provide it using the WebRequest.Credentials
property. Simply set it to a System.Net.NetworkCredential
object. This allows you to send the user's name, password, and domain to the server to provide the user access to the web content.
If you want to store the user's credentials between sessions, you need to make sure the data is secure. If you want to learn more about this, look into the DataProtection API in .NET. You can read more about it in my blog post on Protecting Data in .NET.
Event-Based Asynchronous Pattern
This is the same pattern that the BackgroundWorker
uses to provide asynchronous processing with events being raised on the calling thread. This is a very useful pattern, and the .NET threading libraries make it fairly easy to implement. The MSDN documentation is better than anything I can come up with, so I encourage you to read it (I won't be discussing any of the fine print here).
The key to this pattern is the System.ComponentModel.AsyncOperation
class. This is the class that allows you to make cross thread calls. Basically, you instantiate the class on the main thread, pass it to a new thread, and then call AsyncOperation.Post(AddressOf MyMethod, MyArg)
. AsyncOperation
will then use Windows messaging to call your method on the calling thread (you will want to review the documentation for the fine print on this :). When you are done with the thread, call AsyncOperation.PostOperationCompleted(AddressOf MyMethod, MyArg)
.
If you're interested in how I discovered this pattern, you can read my blog post on The Holy Grail of .NET Threading.
BackgroundWebRequest Component
The NetLib project (included for download in this article), uses the Event-Based Asynchronous Pattern to provide a multi-threaded solution to downloading web content. It can be dropped onto a WinForm to be used like the BackgroundWorker
component. Simply set a couple of properties, handle a couple of events, and call the BackgroundWebRequest.GetResponseAsync
method. This method will return an object that acts as the key to the request. All of the events and methods specific to a particular request use this object to identify the request (it's just a System.Object
).
The following code sample shows how the sample project uses the request key to process the BackgroundWebRequest.GetRequestCompleted
event (note, the SetStatus
method simply sets a label to the status of the request).
Private mRequestKey As Object
Private Sub txtAddress_Validated( _
ByVal sender As System.Object, _
ByVal e As System.EventArgs) _
Handles txtAddress.Validated
wrcBrowser.CancelRequest(mRequestKey)
If txtAddress.Text = "" Then
mRequestKey = Nothing
browser.DocumentText = ""
Else
mRequestKey = wrcBrowser.GetResponseAsync( _
txtAddress.Text)
End If
End Sub
Private Sub wrcBrowser_GetRequestCompleted( _
ByVal sender As System.Object, _
ByVal e As GetRequestCompletedEventArgs) _
Handles wrcBrowser.GetRequestCompleted
If e.RequestKey IsNot mRequestKey Then Return
SetStatus(e.Status, e.Error)
If e.Status = WebRequestStatus.Complete Then
browser.DocumentText = CStr(e.ResponseResults)
lblAddress.Text = e.Uri
End If
End Sub
A couple of notes about the sample code. The code has not gone through any sort of rigorous testing. At this point, it is simply a sample application. There are many different error conditions that I am not handling at this time that should be for production code (some of the omissions are intentional, others are not, you'll have to guess which ones are which :). Use the code at your own risk and if you find any major issues, please let me know.