.NET POP3 MIME Client

• Download source - 76.28 KB

Introduction

The .NET Framework does not offer POP3 or MIME support. For those of you who need it, this article provides support for both POP3 and MIME. This article's intent is not to get into the details of POP3 or MIME. Instead the article focuses on how to use the classes I've provided and finally the internals of the classes in case you'd like to do some tweaking. I've successfully tested the classes on both Yahoo and Gmail throughout development.

Usage

Below is an example snippet of code demonstrating the use of the Pop3Client I created. With the exception of the USER and PASS commands, all implemented Pop3Commands are made available directly from the Pop3Client class. See below for a demonstration of each implemented method's usage and a brief description regarding the purpose of the method.

using (Pop3Client client = new Pop3Client(PopServer, PopPort, true, User, Pass))
{

    /*Peter Huber implemented a Trace event in his Pop3Client as well,
    I used his idea as a model for my Trace event.
    Pretty much the same functionality with a little different implementation
    as the events ultimately raised from the Pop3Client class are initiated
    from the internal command objects and not the Pop3Client class.*/
    client.Trace += new Action<string>(Console.WriteLine);

     /* The Authenticate method establishes connection and executes
     both the USER and PASS commands using
     the username and password provided to the Pop3Client constructor.*/
    client.Authenticate();

    /*The Stat method executes a STAT command against the pop3 server and
    returns a Stat object as a result.*/
    Stat stat = client.Stat();

    /*As seen below, the list of items in a POP3 inbox can be retrieved
    using the List method of the Pop3Client.
    The List method also has an overload to get a single Pop3ListItem*/
    foreach (Pop3ListItem item in client.List())
    {
        /*The MimeEntity returned from the RetrMimeEntity method is converted into a
        MailMessageEx within the RetrMailMessageEx method.  The MailMessageEx class
        inherits System.Net.Mail.MailMessage and adds a few properties related to
        MIME and the Internet Mail Protocol.  One important property added to the
        MailMessageEx class is the Children  property containing a
        List<MailMessageEx> parsed MIME entity attachments whose Media Type
        is message/rfc822.*/
        MailMessageEx message = client.RetrMailMessageEx(item);
        Console.WriteLine("Children.Count: {0}",
            message.Children.Count);
        Console.WriteLine("message-id: {0}",
            message.MessageId);
        Console.WriteLine("subject: {0}",
            message.Subject);
        Console.WriteLine("Attachments.Count: {0}",
            message.Attachments.Count);

        /*Consumers of the Pop3Client class can get a tree of MimeEntities as they
        were originally parsed from the POP3 message they can simply call
        the RetrMimeEntity method of the Pop3Client class and
        work with the MimeEntity object directly.*/
        MimeEntity entity = client.RetrMimeEntity(item);

        /*The Dele method executes the DELE command on the POP3 server the Pop3Client is
        connected to.*/
        client.Dele(item);
    }

    /*The Noop method executes the NOOP command on the POP3 server the Pop3Client is
    connected to.*/
    client.Noop();

    /*The Rset method executes the RSET command on the POP3 server the Pop3Client is
    connected to.*/
    client.Rset();

    /*The Quit method executes the QUIT command on the POP# server the Pop3Client is
    connected to.*/
    client.Quit();
}

The output of the above code is displayed for a Gmail POP3 account that has only one message in its inbox. In the below screenshot, the top line is the response from the server indicating a successful connection. The following lines starting with USER until the RETR request line are the execution of the commands against the POP3 server and the first line of the server response. The lines following the RETR request are the Console.WriteLine used for some of the properties of the MailMessageEx object returned from the RetrMailMessageEx method. Finally, the request and response results of the DELE, NOOP, RSET, and QUIT commands are displayed.

Internals

This library supports executing POP3 requests, parsing the POP3 responses as well as parsing the mail messages returned from the RETR requests into their MIME parts. Internally, the POP3 implementation is made up of various commands, one for each POP3 command and an additional command used to establish the connection with the server. MIME comes into play whenever the RETR method is executed and the lines which are returned in the RetrResponse need to be parsed into MIME parts. The MIME part of this library really is only made up of a couple classes, one to read the POP3 lines and parse them into MimeEntity objects. And, finally a MimeEntity class which really is a structure containing a collection of headers, some decoded content and the ToMailMessageEx method used to convert a MimeEntity into a MailMessageEx.

POP3

Each POP3 command is represented as a command class inheriting from Pop3Command. The POP3 commands are all marked internal and are intended only to be executed from within the Pop3Client class. Internally the Pop3Command is responsible for ensuring the command is in an executable state, sending the command request, and returning the server response from the command request. Each Pop3Command subclass is responsible for creating the request message that will ultimately be sent to the server. The Pop3Command class does encapsulate the creation of a Pop3Response object representing a simple response from the server. For those commands like RETR or LIST which have more complex processing requirements for the parsing of the response message, the CreateResponse method of the Pop3Command class is overrideable allowing inheritors to create their own response type and return it instead of a standard Pop3Response.

Each POP3 command can only be executed in one or more of the following various states based on the POP3 specification, AUTHENTICATION, TRANSACTION and UPDATE. When each Pop3Command is defined, the POP3 state(s) the command can be executed in are hardcoded into the class via the Pop3Command classes constructor as seen below in the QuitCommand class definition.

/// <summary>
/// This class represents the Pop3 QUIT command.
/// </summary>
internal sealed class QuitCommand : Pop3Command<Pop3Response>
{
    /// <summary>
    /// Initializes a new instance of the <see cref="QuitCommand"> class.
    /// </summary>
    /// <param name="stream">The stream.</param>
    public QuitCommand(Stream stream)
        : base(stream, false, Pop3State.Transaction | Pop3State.Authorization) { }

    /// <summary>
    /// Creates the Quit request message.
    /// </summary>
    /// <returns>
    /// The byte[] containing the QUIT request message.
    /// </returns>
    protected override byte[] CreateRequestMessage()
    {
        return GetRequestMessage(Pop3Commands.Quit);
    }
}

Each Pop3Commands state is validated using the classes EnsureState method. The Pop3State enumeration is defined using the flags attribute allowing the enumeration to be treated as a bit field that can be used in bitwise operations. See the Pop3Command.EnsureState method below for how the Pop3State enumeration is used:

/// <summary>
/// Ensures the state of the POP3.
/// </summary>
/// <param name="currentState">State of the current.</param>
protected void EnsurePop3State(Pop3State currentState)
{
    if (!((currentState & ValidExecuteState) == currentState))
    {
        throw new Pop3Exception(string.Format("This command is being executed" +
                    " in an invalid execution state.  Current:{0}, Valid:{1}",
                    currentState, ValidExecuteState));
    }
}

External to the Pop3Command, the Pop3Client class provides the current POP3 state to the command objects via the ExecuteCommand method. The ExecuteCommand method below is used to execute all commands to enforce consistency in how the commands are handled during execution.

/// <summary>
/// Provides a common way to execute all commands.  This method
/// validates the connection, traces the command and finally
/// validates the response message for a -ERR response.
/// </summary>
/// <param name="command">The command.</param>
/// <returns>The Pop3Response for the provided command</returns>
/// <exception cref="Pop3Exception">If the HostMessage does not start with '+OK'.
/// </exception>
/// <exception cref="Pop3Exception">If the client is no longer connected.</exception>
private TResponse ExecuteCommand<TResponse, TCommand>(TCommand command)
    where TResponse : Pop3Response where TCommand : Pop3Command<TResponse>
{
    //Ensures the TcpClient is still connected prior to executing the command.

    EnsureConnection();
    /*Adds an anonymous delegate to handle the commands Trace event in order to
    provided tracing.*/
    TraceCommand<TCommand, TResponse>(command);
    //Executes the command providing the current POP3 state.

    TResponse response = (TResponse)command.Execute(CurrentState);
    //Ensures the Pop3Response started with '+OK'.

    EnsureResponse(response);
    return response;
}

Finally the Pop3Response is created using the CreateResponse method of the Pop3Command class. Below is an example from the StatCommand.CreateResponse method illustrating this scenario returning a custom StatResponse object to the caller.

protected override StatResponse CreateResponse(byte[] buffer)
{
    Pop3Response response = Pop3Response.CreateResponse(buffer);
    string[] values = response.HostMessage.Split(' ');

    //should consist of '+OK', 'messagecount', 'octets'

    if (values.Length < 3)
    {
        throw new Pop3Exception(string.Concat("Invalid response message: ",
            response.HostMessage));
    }

    int messageCount = Convert.ToInt32(values[1]);
    long octets = Convert.ToInt64(values[2]);

    return new StatResponse(response, messageCount, octets);
}

If you'd like more information about POP3 please view Post Office Protocol - Version 3 containing a full explanation of the purpose of each command listed above as well as additional commands which were not implemented.

MIME

The MimeReader class receives a string array of lines into its public constructor which make up the POP3 message. The lines are then stored within a Queue<string> instance and processed one at a time. The MimeReader class is responsible for parsing both multipart and singlepart MIME messages into MIME entities consisting of headers and decoded MIME content. The MimeReader class supports parsing nested MIME entities including those of type message/rfc822. Once the MimeReader has completed processing of the internet mail message, a MimeEntity will be returned containing a tree structure containing the contents of the message.

The RetrCommand overrides the CreateResponse method and returns a RetrResponse object containing the lines of the mail message. The Pop3Client classes RetrMimeEntity method provides the lines returned as part of the RetrResponse object to a new instance of the MimeReader class to parse the messages lines. Finally the MimeReader.CreateMimeEntity method returns a MimeEntity instance representing the MimeEntities contained within the POP3 message. See below for the Pop3Client.RetrMimeEntity method definition:

/// <summary>
/// Retrs the specified message.
/// </summary>
/// <param name="item">The item.</param>
/// <returns>A MimeEntity for the requested Pop3 Mail Item.</returns>
public MimeEntity RetrMimeEntity(Pop3ListItem item)
{
    if (item == null)
    {
        throw new ArgumentNullException("item");
    }

    if (item.MessageId < 1)
    {
        throw new ArgumentOutOfRangeException("item.MessageId");
    }

    RetrResponse response;
    using (RetrCommand command = new RetrCommand(_clientStream, item.MessageId))
    {
        response = ExecuteCommand<RetrResponse, RetrCommand>(command);
    }

    if (response != null)
    {
        MimeReader reader = new MimeReader(response.MessageLines);
        return reader.CreateMimeEntity();
    }

    throw new Pop3Exception("Unable to get RetrResponse.  Response object null");
}

The MimeReader creates a new MimeEntity object and builds a tree of MIME entities using those objects by recursively calling the CreateMimeEntity method. This process continues until all of the lines for the entire internet mail message have been processed. Below is a snippet of code containing the CreateMimeEntity method to show what processing takes place in order to create a new MimeEntity.

/// <summary>
/// Creates the MIME entity.
/// </summary>
/// <returns>A mime entity containing 0 or more children
/// representing the mime message.</returns>
public MimeEntity CreateMimeEntity()
{
    //Removes the headers from the mime entity for later processing.

    ParseHeaders();
    /*Processes the headers previously removed and sets any MIME
    specific properties like ContentType or ContentDisposition.*/
    ProcessHeaders();
    /*Parses the MimeEntities body.  This method causes new
    MimeReader objects to be created for each new Mime Entity found within the POP3
    messages lines until all of the lines have been removed from the queue.*/.
    ParseBody();
    /*Decodes the content stream based on the entities
    ContentTransferEncoding and sets the Content stream of the MimeEntity.*/
    SetDecodedContentStream();
    /*Returns the MimeEntity that was created in the constructor
    of the class and that now has all of its child mime parts parsed
    into mime entities.*/
    return _entity;
}

Parsing the headers really consists of getting name value pairs until a blank line is read. The method somewhat follows the pattern defined by Peter Huber. Based on the MIME spec, we keep reading header lines until we hit the first blank line. When the blank line is encountered, the body of the MIME entity starts and is ready for processing.

/// <summary>
/// Parse headers into _entity.Headers NameValueCollection.
/// </summary></span>
private int ParseHeaders()
{
    string lastHeader = string.Empty;
    string line = string.Empty;
    // the first empty line is the end of the headers.

    while(_lines.Count > 0 && !string.IsNullOrEmpty(_lines.Peek()))
    {
        line = _lines.Dequeue();

        /*if a header line starts with a space or tab then it
        is a continuation of the previous line.*/
        if (line.StartsWith(" ")
            || line.StartsWith(Convert.ToString('\t')))
        {
            _entity.Headers[lastHeader]
                = string.Concat(_entity.Headers[lastHeader], line);
            continue;
        }

        int separatorIndex = line.IndexOf(':');

        if (separatorIndex < 0)
        {
            System.Diagnostics.Debug.WriteLine("Invalid header:{0}", line);
            continue;
        }  //This is an invalid header field.  Ignore this line.

        string headerName = line.Substring(0, separatorIndex);
        string headerValue
            = line.Substring(separatorIndex + 1).Trim(HeaderWhitespaceChars);

        _entity.Headers.Add(headerName.ToLower(), headerValue);
        lastHeader = headerName;
    }

    if (_lines.Count > 0)
    {
        _lines.Dequeue();
    } //remove closing header CRLF.


    return _entity.Headers.Count;
}

Once the headers have been parsed from the MIME entity they need to be processed. If a header is specific to MIME processing, then the header will be assigned to a property on the MIME object. Otherwise the header will be ignored and returned in the headers NameValueCollection on the MimeEntity object for later processing. Some of the useful helper methods such as GetTransferEncoding and GetContentType the MimeReader has are displayed below:

/// <summary>
/// Processes mime specific headers.
/// </summary>
/// <returns>A mime entity with mime specific headers parsed.</returns>
private void ProcessHeaders()
{
    foreach (string key in _entity.Headers.AllKeys)
    {
        switch (key)
        {
            case "content-description":
                _entity.ContentDescription = _entity.Headers[key];
                break;
            case "content-disposition":
                _entity.ContentDisposition
                    = new ContentDisposition(_entity.Headers[key]);
                break;
            case "content-id":
                _entity.ContentId = _entity.Headers[key];
                break;
            case "content-transfer-encoding":
                _entity.TransferEncoding = _entity.Headers[key];
                _entity.ContentTransferEncoding
                    = MimeReader.GetTransferEncoding(_entity.Headers[key]);
                break;
            case "content-type":
                _entity.SetContentType(MimeReader.GetContentType(_entity.Headers[key]));
                break;
            case "mime-version":
                _entity.MimeVersion = _entity.Headers[key];
                break;
        }
    }
}

Now that the headers are parsed, the body is ready to be parsed for a given MimeEntity. Recursion takes place when new MimeReader objects are created by the MimeReader object while the body parsing is taking place and result in adding the MimeEntity objects created to the Children collection of the current MimeEntity.

/// <summary>
/// Parses the body.
/// </summary>
private void ParseBody()
{
    if (_entity.HasBoundary)
    {
        while (_lines.Count > 0
            && !string.Equals(_lines.Peek(), _entity.EndBoundary))
        {
            /*Check to verify the current line is not the same as the
            parent starting boundary. If it is the same as the parent
            starting boundary this indicates existence of a new child
            entity. Return and process the next child.*/
            if (_entity.Parent != null
                && string.Equals(_entity.Parent.StartBoundary, _lines.Peek()))
            {
                return;
            }

            if (string.Equals(_lines.Peek(), _entity.StartBoundary))
            {
                AddChildEntity(_entity, _lines);
            } //Parse a new child mime part.

            else if (string.Equals(_entity.ContentType.MediaType,
                    MediaTypes.MessageRfc822, StringComparison.InvariantCultureIgnoreCase)
                && string.Equals(_entity.ContentDisposition.DispositionType,
                    DispositionTypeNames.Attachment,
                            StringComparison.InvariantCultureIgnoreCase))
            {
                /*If the content type is message/rfc822 the
                stop condition to parse headers has already been encountered.
                But, a content type of message/rfc822 would
                have the message headers immediately following the mime
                headers so we need to parse the headers for the attached message.*/
                AddChildEntity(_entity, _lines);

                break;
            }
            else
            {
                _entity.EncodedMessage.Append
                     (string.Concat(_lines.Dequeue(), Pop3Commands.Crlf));
            } //Append the message content.

        }
    } //Parse a multipart message.

    else
    {
        while (_lines.Count > 0)
        {
            _entity.EncodedMessage.Append(string.Concat
                   (_lines.Dequeue(), Pop3Commands.Crlf));
        }
    } //Parse a single part message.
}

Once the body has been processed, the only remaining thing to do is write the decoded content to the Content stream of the MimeEntity object just prior to returning it. This is done using the SetDecodedContentStream method.

private void SetDecodedContentStream()
{
    switch (_entity.ContentTransferEncoding)
    {
        case System.Net.Mime.TransferEncoding.Base64:
            _entity.Content
                = new MemoryStream(Convert.FromBase64String
                        (_entity.EncodedMessage.ToString()), false);
            break;

        case System.Net.Mime.TransferEncoding.QuotedPrintable:
            _entity.Content
                = new MemoryStream(GetBytes(QuotedPrintableEncoding.Decode
                    (_entity.EncodedMessage.ToString())), false);
            break;

        case System.Net.Mime.TransferEncoding.SevenBit:
        default:
            _entity.Content
                = new MemoryStream(GetBytes(_entity.EncodedMessage.ToString()),
                    false);
            break;
    }
}

Now that the content stream is set, there isn't much more to do besides return the MimeEntity that has been created. The object returned from this method is ready for processing. But, the MimeEntity object returned from the CreateMimeEntity method does not map directly to a MailMessage. For convenience I added a method allowing a MimeEntity to be converted into a MailMessage. Because of the Media Type message/rfc822, I wanted those entities to be pre-parsed and made directly available and ready for use on any MimeEntity containing a message attachment. To facilitate this, I created a class inheriting from System.Net.Mail.MailMessage that has a List<MailMessageEx> property containing a collection of MailMessageEx objects which are mail message attachments to any message. The MailMessageEx object is created by calling the MimeEntity.ToMailMessageEx method.

Conclusion

With this overview of how to use the code and how most of the internals work, you should be well equipped to make use of these classes and make changes or additions where necessary which should allow you to easily incorporate them into your codebase. This library handles the POP3 and MIME protocols and wraps them both up in an easy to use class. MIME messages are ultimately parsed into MailMessageEx objects so that attachments and email body text can be easily accessed.

References

There are many articles already written dealing with both POP3 and MIME. Below are a couple great implementations I reviewed prior to starting my article. I was able to make use of ideas presented in a both articles and did my best to document whenever I used any of those ideas directly without significant modification.

• A POP3 Client in C# .NET by Desmond McCarter
• POP3 Email Client with full MIME Support (.NET 2.0) by Peter Huber
• QuotedPrintable Class by Bill Gearhart
• Post Office Protocol - Version 3
• Multipurpose Internet Mail Extensions (MIME) Part 1: Format of Internet Message Bodies
• Multipurpose Internet Mail Extensions (MIME) Part 2: Media Types
• Multipurpose Internet Mail Extensions (MIME) Part 3: Message Header Extensions for Non-ASCII Text
• Multipurpose Internet Mail Extensions (MIME) Part 4: Registration Procedures
• Multipurpose Internet Mail Extensions (MIME) Part 5: Conformance Criteria and Examples

History

• 2008.02.08
  • Fixed issue found by Shawn Cook where the Body property was empty.
• 2008.02.07 Minor bug fix
  • RetrResponse issue whenever host GMail returns 3 part host message as first line of response.
• 2008.02.05 Bug Fixes and additional command
  • Various minor bug fixes
  • Fixed issue found when parsing GMail headers by D I Petersen. This was caused by an empty / null header.
  • Made changes to the way handling takes place when no response is received from the POP3 server. This fix addresses the issue found by zlezj whenever the size of the buffer lands in the middle of a line terminator.
  • Changed the disconnect to leave the TcpClient alone so it can be reused for subsequent POP3 requests.
  • Added Top command to add support for the non-standard POP3 TOP command providing the ability to download message headers instead of the entire message.
• 2007.11.20 Initial post