Introduction
Communication over HTTP is a very common task. Most of the .NET programmers are familiar with classes like WebRequest
, WebResponse
or WebClient
which are very useful, but sometimes one needs features they don't offer. In these cases, we could not have another option but to write our own HTTP client implementation based on TCP sockets. This article is an example of how to do so. The reader should know the basics of HTTP protocol.
Background
My first reason for writing my own HTTP client library was the need to upload large files to the server and show the current upload speed. It didn't seem to be very difficult, but I found out standard classes load all output data into memory before send despite the face that the WebRequest
class offers an output stream for POST
data. HTTP protocol is not really complex, so why not just open a socket on port 80 and start communication?
I didn't want to rewrite all my application but just to do some experiments first to ensure my idea is sustainable. I needed to create an abstract
interface, which enables me to return to standard solution at any time. These are my classes:
HttpConnection
- represents an abstract
connection to the web server which can handle only one request at a time HttpSocketConnection
- new client implementation based on TCP sockets HttpWebRequestConnection
- implementation using standard WebRequest
and WebResponse
classes HttpMessage
- container for HTTP request or response including headers and data stream
These classes are all you need to run simple HTTP requests with data uploading/downloading in both ways. I also prepared some code which enables control bandwidth used by network communication. This can be accomplished simply by slowing down read streams - first for the POST
data and second for server response, so I created a stream proxy class slowing down read/write operations for a given stream held inside the proxy.
WatchedStream
- Serves as a proxy for given stream calling events before every read/write operation WatchedStreamEventArgs
- Event arguments for stream transfer events HttpAdvancedStream
- Uses the events from base class to insert some wait time BandWidthManager
- Responsible for measuring time and computing transferred data to keep the desired speed
If you would like to use the library to send POST
requests, you could use some framework to create POST
body. Of course I have created one, and it can prepare request body in two common formats: simple URL encoded and multi-part format.
HttpPostBodyBuilder
- abstract
class for POST
body builders HttpPostBodyBuilder.Multipart
- Prepares a request body in standard multipart format which allows you upload files HttpPostBodyBuilder.UrlEncoded
- Prepares a request body in standard URL format MergingStream
- This stream reads data from multiple given streams, which allows to prepare an entire request body without the need to load all necessary data into memory HttpUtility
- URL encoding logic extracted from System.Web.dll. This allows you to avoid referencing this library and keep application under .NET Framework Client Profile.
Using the Code
I have prepared and attached a client/server sample solution for you to test and better understand the library. There is an ASP.NET web site which allows you to upload image, then it does some graphics operations and sends the result back to the browser. To invoke the service, you can either use your standard browser or the WinForms client emulating browser behavior.
This block of the code is crucial:
this.NotifyState("Converting file...", 0);
HttpConnection http;
switch (httpMethod)
{
case 0:
http = new HttpSocketConnection();
break;
case 1:
http = new HttpWebRequestConnection();
break;
default:
throw new NotSupportedException();
}
var url = "http://localhost:12345/Page.aspx";
var postBody = new HttpPostBodyBuilder.Multipart();
var fileStream = new FileStream
(this.openFileDialog.FileName, FileMode.Open, FileAccess.Read);
var advStream = new BandwidthControlledStream(fileStream);
lock (this.bandWidthSync)
{
this.uploadSpeed = advStream.ReadSpeed;
this.uploadSpeed.BytesPerSecond = 1024 * (int)this.upSpeedBox.Value;
}
postBody.AddData(
"imageFile",
advStream,
Path.GetFileName(this.openFileDialog.FileName),
GetMimeType(this.openFileDialog.FileName)
);
var bodyStream = postBody.PrepareData();
bodyStream.Position = 0;
var req = new HttpMessage.Request(bodyStream, "POST");
req.ContentLength = bodyStream.Length;
req.ContentType = postBody.GetContentType();
req.Headers["Referer"] = url;
advStream.BeforeRead +=
(s, e) => this.NotifyState("Uploading...", e.Position, bodyStream.Length);
var response = http.Send(url, req);
var readStream = new BandwidthControlledStream(response.Stream);
lock (this.bandWidthSync)
{
this.downloadSpeed = readStream.ReadSpeed;
this.downloadSpeed.BytesPerSecond = 1024*(int) this.downSpeedBox.Value;
}
readStream.BeforeRead +=
(s, e) => this.NotifyState("Downloading...", e.Position, response.ContentLength);
this.convertedFile = ReadAll(readStream, (int) response.ContentLength);
this.NotifyState("Done", 100, true);
As you can see, the code snippet is not quite short, but it does a lot of work:
- Chooses method to upload data (WebRequests/sockets)
- Prepares request body with file to upload (file is loaded continuously during sending)
- Prepares speed-limited streams (you can also choose not to use them, or change speed during transfer)
- Uses
BeforeRead
events to inform user about progress - Shows how to set request headers
Limitations
Please don't consider this library as full replacement of HttpWebRequest
and other classes. This is only a simple solution and cannot be used in any scenario because of its limits:
- No support for HTTPS
- No support for proxy
- No caching
- Does not keep opened socket
- Only few HTTP statuses are treated correctly (100, 206, 302 and of course 200)
- Only seek-capable streams can be used as source for upload
History