ftp2? Evolving File Transfer with msgfiles

• Download source code - 71.2 KB

Introduction

Back in the early 2000s when clouds were just those fluffy white things in the sky, I worked for a startup aimed at changing the way files were stored. We simply called it online file storage. You would upload your files, then you could access them anywhere... and send them to your friends. It wasn't file sharing like Napster; you weren't sharing your CD collection with the whole world. But you could send your CD collection to your sister with ease.

Tech has evolved and improved since then, and you can make the argument that there's no justification for file sharing or sending because there is a bustling marketplace for buying and renting access to files online. Pay your monthly fee for access to a file castle and send whatever you want to somebody else paying a monthly fee to the same castle, problem solved. All legit, all above board. Something like Napster will never be allowed to exist again.

So what's this msgfiles business all about then? msgfiles is all about locality and simplicity. Run the server software somewhere all the people you want to transfer files with can access. Then have them run the client and you can easily send files to each other. Think of this as an improved message-based FTP system with emphasis on the Transfer part of the acronym. It hits the sweet spot between email and FTP. Email is not good with large or lots of files. FTP is not message-based. I'm curious to hear how this strikes you.

ftp2 is an incendiary, grandiose article title, but dig deeper, you may see why it's not too crazy a proposition. Okay, maybe a little crazy...

As An Aside...

I haven't written much C# in a while.

There have been a couple JavaScript web games: tapglasses.io and tiletaps.com.

I made a few iterations on mscript.

And I did a C++ port of a C# NoSQL DB, 4db.

Not much .NET.

My first blush coming back is that C# was already highly productive with the .NET library and LINQ. I think the new AI stuff is way overboard, impossible to use... it just gets in the way. I code because I want to code, not click on code suggestions all the time. Auto-complete is a no-brainer, sliced bread, but the AI has to go, I could not disable it fast enough.

On a sunnier note, the nullability business was annoying at first, but it's done well, and it serves a purpose. It strikes me a bit like Rust, where the compiler is on your side to help you make correct programs. I like that.

So there's good and bad.

I still don't think C# is a good language for large projects. The world is moving away from those, so maybe that's okay.

The msgfiles Client Application

You can install the client and see a screenshot walkthrough on msgfiles.io. I won't repeat all that here, suffice to say...

You launch the client...

1. Enter a display name and your email address.
2. Enter the server address, like an FTP server.
3. The server sends a login code to your email address.
4. You punch that in, then you can send and receive messages.

To send files...

1. Push the Send Files button.
2. Choose who to send the files to.
3. Type in your little message.
4. Pick the files to send.
5. Then the client ZIPs the files...
6. ...and sends the ZIP and the message to the server.
7. The server stores the ZIP and the message...
8. ...and sends emails to your recipients with access tokens.

When you get an email saying you have files...

1. Launch the client.
2. Connect to the server.
3. Push the Receive Files button.
4. Copy the access token from the email and paste it into the client.
5. The client shows you who the message is from and the little message.
6. You run your eyes over this and either punt it or continue.
7. The client downloads the ZIP and shows you a manifest of the contents.
8. You run your eyes over this and either punt it or continue.
9. Then you pick where to put the files, and the ZIP is extracted there.
10. Mission complete!

That's the whole app. Two big buttons, and some simple dialog boxes. Easy peasy!

It could be a lot prettier. Anybody going to Maui soon?

The msgfiles Server Application

You can install the server and get installation steps and maintenance tips on msgfiles.io.

The Code

msgfiles is open source on GitHub with an Apache 2.0 license. It's all .NET 6 C# in one solution including unit tests.

Here is a rundown of the projects in the solution.

There are two application projects, client and server. These projects have very little code in them, just top-level orchestration.

securenet

This low-level library wraps 3^rd party dependencies, including ZIP, AES, and JSON. It also includes the core TLS code including self-signed certificate generation and SslStream wrapper functions. Many of the core building blocks, like the SMTP wrapper class EmailClient and the session management class SessionStore are also here.

msglib

This library implements the message processing in client-side MsgClient and server-side MsgRequestHandler.cs classes. The core MessageStore class is also here.

client

A basic proof-of-concept WinForms application that responds to MsgClient events to displays progress and prompt the user for tokens and confirmation. I imagine sexier applications to take the place of this program; hopefully they will be as simple and easy to use as this humble beginning.

server

Command-line application for running the show on the server side. The server relies on an settings INI file and allow and block list files. The command line prompt gives easy access to these files, and the server picks up some changes and puts them into effect immediately.

In theory, you can take just the securenet project and develop your own client-server applications. It's kind of like http2, it could take you far.

So This is CodeProject...

Enough folklore and projects, let's see some code!

Secure Networking

At the core of msgfiles is basic self-signed secure networking via SslStream:

/// Create a self-signed cert...in six lines of code
public static X509Certificate GenCert()
{
    using (RSA rsa = RSA.Create(4096))
    {
        var distinguishedName = new X500DistinguishedName($"CN=msgfiles.io");
        var request = new CertificateRequest(distinguishedName, rsa, 
                      HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);
        var certificate = request.CreateSelfSigned(DateTimeOffset.UtcNow.AddDays(-1), 
                          DateTimeOffset.UtcNow.AddDays(3650));
        return new X509Certificate2
               (certificate.Export(X509ContentType.Pfx, "password"), 
               "password", X509KeyStorageFlags.MachineKeySet);
    }
}

/// Given a client TCP connection, secure communications with the server
public static Stream SecureConnectionToServer(TcpClient client)
{
    var client_stream = client.GetStream();
    var ssl_stream = 
        new SslStream
        (
            client_stream,
            false,
            (object obj, X509Certificate? cert, X509Chain? chain, 
                         SslPolicyErrors errors) => true
        );
    ssl_stream.AuthenticateAsClient("msgfiles.io");
    if (!ssl_stream.IsAuthenticated)
        throw new NetworkException("Connection to server not authenticated");
    return ssl_stream;
}

/// Given a server TCP connection, secure communications with the client
public async static Task<Stream> SecureConnectionFromClient
             (TcpClient client, X509Certificate cert)
{
    var client_stream = client.GetStream();
    var ssl_stream = new SslStream(client_stream, false, 
    (object obj, X509Certificate? cert2, X509Chain? chain, 
                 SslPolicyErrors errors) => true);
    await ssl_stream.AuthenticateAsServerAsync(cert).ConfigureAwait(false);
    if (!ssl_stream.IsAuthenticated)
        throw new NetworkException("Connection from client not authenticated");
    return ssl_stream;
}

Network Payload Serialization

Once you have secure networking, you want a mechanism for sending payloads back and forth over the network. I chose to have the payloads be like HTTP, and to compress the headers so that significant things like message text and recipients could be put in there. If your little message and list of recipients add up, compressed, to over 64 KB... you might prefer email!

public static int MaxObjectByteCount = 64 * 1024;

/// Given pretty much anything, 
/// turn it into JSON,
/// get the UTF-8 bytes,
/// compress the bytes,
/// make sure it isn't too big,
/// then send it over the stream, length-prefixed
public static void SendObject<T>(Stream stream, T headers)
{
    string json = JsonConvert.SerializeObject(headers);

    byte[] json_bytes = Utils.Compress(Encoding.UTF8.GetBytes(json));
    if (json_bytes.Length > MaxObjectByteCount)
        throw new InputException("Too much to send");

    byte[] num_bytes = BitConverter.GetBytes
           (IPAddress.HostToNetworkOrder(json_bytes.Length));

    using (var buffer = Utils.CombineArrays(num_bytes, json_bytes))
        stream.Write(buffer.GetBuffer(), 0, (int)buffer.Length);
}
public static async Task SendObjectAsync<T>(Stream stream, T headers)
...

/// Receive pretty much anything from a stream
/// Read the length, ensure it's not too much
/// Read the bytes, decompress, turn into a string, JSON parse,
/// and out comes an object
public static T ReadObject<T>(Stream stream)
{
    byte[] num_bytes = new byte[4];
    if (stream.Read(num_bytes, 0, num_bytes.Length) != num_bytes.Length)
        throw new SocketException();

    int bytes_length = 
        IPAddress.NetworkToHostOrder(BitConverter.ToInt32(num_bytes, 0));
    if (bytes_length > MaxObjectByteCount)
        throw new InputException("Too much to read");

    byte[] header_bytes = new byte[bytes_length];
    int read_yet = 0;
    while (read_yet < bytes_length)
    {
        int to_read = bytes_length - read_yet;
        int new_read = stream.Read(header_bytes, read_yet, to_read);
        if (new_read <= 0)
            throw new NetworkException("Connection closed");
        else
            read_yet += new_read;
    }

    string json = Encoding.UTF8.GetString
                  (Utils.Decompress(header_bytes, bytes_length));
    var obj = JsonConvert.DeserializeObject<T>(json);
    if (obj == null)
        throw new InputException("Input did not parse");
    else
        return obj;
}
public static async Task<T> ReadObjectAsync<T>(Stream stream)
...

Access Control

One big topic for running any kind of server is access control. I don't see putting this server on the internet; this is an intranet play. Maybe you want a server in one department and don't want other departments mucking about with it.

So the server has two files for access control, allow.txt and block.txt. You can put full email addresses or domain names with their @ prefixes. If your email address is not on the allow list, or you're blocked, you can't get connect, and you cannot have anything sent to you. The server is locked down.

Here's the code for enforcing access control:

/// Manage allow and block lists of email addresses
/// to validate that a given email address is allowed access
public class AllowBlock
{
    /// Swap in new lists
    public void SetLists(HashSet<string> allow, HashSet<string> block)
    {
        try
        {
            m_rwLock.EnterWriteLock();

            m_allowList = allow;
            m_blockList = block;
        }
        finally
        {
            m_rwLock.ExitWriteLock();
        }
    }

    /// Ensure that an email address or its domain is allowed,
    /// or at least not blocked
    public void EnsureEmailAllowed(string email)
    {
        try
        {
            m_rwLock.EnterReadLock();

            // Normalize the email address
            email = Utils.GetValidEmail(email).ToLower();
            if (email.Length == 0)
                throw new InputException($"Invalid email: {email}");

            // Include the leading @, list files use this to allow/block entire domains
            string domain = email.Substring(email.IndexOf('@')).ToLower();

            // Look for specific email address blocks first, that trumps all
            if (m_blockList.Contains(email))
                throw new InputException($"Blocked email: {email}");

            // Check for specific email address being allowed, this trumps domains
            if (m_allowList.Contains(email))
                return;

            // Check for a whole blocked domain
            if (m_blockList.Contains(domain))
                throw new InputException($"Blocked domain: {email}");

            // Allow a whole domain
            if (m_allowList.Contains(domain))
                return;

            // Failing all of that, if there is an allow list,
            // the email is not on any of them, so they're blocked by default
            if (m_allowList.Count > 0)
                throw new InputException($"Not allowed: {email}");

            // no allow list, not blocked -> allowed
        }
        finally
        {
            m_rwLock.ExitReadLock();
        }
    }

    private HashSet<string> m_allowList = new HashSet<string>();
    private HashSet<string> m_blockList = new HashSet<string>();

    private ReaderWriterLockSlim m_rwLock = new ReaderWriterLockSlim();
}

Sending Email

Another topic for msgfiles is sending email. System.Net.SmtpClient is interesting. It's not thread-safe. It has a non-async/await SendAsync function. The docs suggest that there is network connection pooling under the hood, so let's create an SmtpClient with each message, and use the fire-and-hope SendAsync function:

/// SmtpClient wrapper class
public class EmailClient
{
    public EmailClient(string server, int port, string username, string password)
    {
        m_server = server;
        m_port = port;
        m_credential = new NetworkCredential(username, password);
    }

    public void SendEmail
    (
        string from, // display <email> or just email
        Dictionary<string, string> toAddrs, // email -> display
        string subject,
        string body
    )
    {
        var fromKvp = Utils.ParseEmail(from);

        var mail_message = new MailMessage();
        mail_message.From = new MailAddress(fromKvp.Key, fromKvp.Value);

        foreach (var toKvp in toAddrs)
            mail_message.To.Add(new MailAddress(toKvp.Key, toKvp.Value));

        mail_message.Subject = subject;
        mail_message.Body = body;

        SmtpClient client = new SmtpClient(m_server, m_port);
        client.Credentials = m_credential;
        client.DeliveryMethod = SmtpDeliveryMethod.Network;
        client.EnableSsl = true;
        client.SendAsync(mail_message, null);
    }

    private string m_server;
    private int m_port;
    private NetworkCredential m_credential;
}

ZIP File Processing

ZIP files are central to this application. I created a few wrapper functions around the DotNetZip NuGet package:

/// Create a ZIP file from files and folders to include
public static void CreateZip(IClientApp app, string zipFilePath, IEnumerable<string> paths)
{
    using (var zip = new Ionic.Zip.ZipFile(zipFilePath))
    {
        zip.CompressionLevel = Ionic.Zlib.CompressionLevel.BestSpeed;
        string lastZipCurrentFilename = "";
        zip.SaveProgress +=
            (object? sender, Ionic.Zip.SaveProgressEventArgs e) =>
            {
                if (e.CurrentEntry != null && 
                    e.CurrentEntry.FileName != lastZipCurrentFilename)
                {
                    lastZipCurrentFilename = e.CurrentEntry.FileName;
                    app.Log(lastZipCurrentFilename);
                }

                if (e.TotalBytesToTransfer > 0)
                    app.Progress((double)e.BytesTransferred / e.TotalBytesToTransfer);
            };
        foreach (var path in paths)
        {
            if (File.Exists(path))
                zip.AddFile(path, "");
            else if (Directory.Exists(path))
                zip.AddDirectory(path, Path.GetFileName(path));
            else
                throw new InputException($"Item to send not found: {path}");
        }

        zip.Save();
    }
}

/// Summarize the contents of a ZIP file for the benefit of having an idea
/// whether they are what is expected, and safe
public static string ManifestZip(string zipFilePath)
{
    int file_count = 0;
    long total_byte_count = 0;

    StringBuilder entry_lines = new StringBuilder();
            
    Dictionary<string, int> ext_counts = new Dictionary<string, int>();
            
    using (var zip_file = new Ionic.Zip.ZipFile(zipFilePath))
    {
        foreach (var zip_entry in zip_file.Entries)
        {
            if (zip_entry.IsDirectory)
                continue;

            string size_str = 
                Utils.ByteCountToStr(zip_entry.UncompressedSize);
            entry_lines.AppendLine($"{zip_entry.FileName} ({size_str})");

            string ext = Path.GetExtension(zip_entry.FileName).ToUpper();
            if (ext_counts.ContainsKey(ext))
                ++ext_counts[ext];
            else
                ext_counts[ext] = 1;

            ++file_count;
            total_byte_count += zip_entry.UncompressedSize;
        }
    }

    string ext_summary =
        "File Types:\r\n" +
        string.Join
        (
            "\r\n",
            ext_counts
                .Select(kvp => $"{kvp.Key.Trim('.')}: {kvp.Value}")
                .OrderBy(str => str)
        );

    return
        $"Files: {file_count}" +
        $" - " +
        $"Total: {Utils.ByteCountToStr(total_byte_count)}" +
        $"\r\n\r\n" +
        $"{ext_summary}" +
        $"\r\n\r\n" +
        $"{entry_lines}";
}

/// Extract a ZIP file's contents into an output directory
public static void ExtractZip
       (IClientApp app, string zipFilePath, string extractionDirPath)
{
    using (var zip = new Ionic.Zip.ZipFile(zipFilePath))
    {
        string lastZipCurrentFilename = "";
        zip.ExtractProgress +=
            (object? sender, Ionic.Zip.ExtractProgressEventArgs e) =>
            {
                if (e.CurrentEntry != null && 
                    e.CurrentEntry.FileName != lastZipCurrentFilename)
                {
                    lastZipCurrentFilename = e.CurrentEntry.FileName;
                    app.Log(lastZipCurrentFilename);
                }

                if (e.TotalBytesToTransfer > 0)
                    app.Progress((double)e.BytesTransferred / e.TotalBytesToTransfer);
            };
        zip.ExtractAll(extractionDirPath);
    }
}

Client Message Sending

Further up the food chain is the client-side code for sending a message:

/// Send a message with files to recipients
public bool SendMsg
(
    IEnumerable<string> to, 
    string message, 
    IEnumerable<string> paths
)
{
    using (var temp_file_use = new TempFileUse(".zip"))
    {
        string zip_file_path = temp_file_use.FilePath;

        App.Log("Adding files to package...");
        Utils.CreateZip(App, zip_file_path, paths);

        App.Log("Scanning package...");
        string zip_hash;
        using (var fs = File.OpenRead(zip_file_path))
            zip_hash = Utils.HashStream(fs);

        App.Log("Sending message...");
        long zip_file_size_bytes = new FileInfo(zip_file_path).Length;
        var send_request =
            new ClientRequest()
            {
                version = 1,
                verb = "POST",
                contentLength = zip_file_size_bytes,
                headers = new Dictionary<string, string>()
                {
                    { "to", string.Join("; ", to) },
                    { "message", message },
                    { "hash", zip_hash }
                }
            };
        if (ServerStream == null)
            return false;
        SecureNet.SendObject(ServerStream, send_request);

        App.Log("Sending package...");
        using (var zip_file_stream = File.OpenRead(zip_file_path))
        {
            long sent_yet = 0;
            byte[] buffer = new byte[64 * 1024];
            while (sent_yet < zip_file_size_bytes)
            {
                int to_read = 
                (int)Math.Min(zip_file_size_bytes - sent_yet, buffer.Length);
                int read = zip_file_stream.Read(buffer, 0, to_read);
                if (App.Cancelled)
                    return false;

                if (ServerStream == null)
                    return false;
                ServerStream.Write(buffer, 0, read);

                sent_yet += read;

                App.Progress((double)sent_yet / zip_file_size_bytes);
                if (App.Cancelled)
                    return false;
            }
        }
        if (App.Cancelled)
            return false;

        App.Log("Receiving response...");
        using (var send_response = SecureNet.ReadObject<ServerResponse>(ServerStream))
        {
            App.Log($"Server Response: {send_response.ResponseSummary}");
            if (send_response.statusCode / 100 != 2)
                throw send_response.CreateException();
        }

        return true;
    }
}

Client Message Receiving

And here's the client code for receiving a message:

/// Given a message token, get a message for the current user
/// Returns true if getting the message succeeded
/// Sets shouldDelete to true if the user canceled the operation
public bool GetMessage(string msgToken, out bool shouldDelete)
{
    shouldDelete = false;

    {
        App.Log("Sending GET msg request...");
        var request =
            new ClientRequest()
            {
                version = 1,
                verb = "GET",
                headers =
                    new Dictionary<string, string>()
                    { { "token", msgToken }, { "part", "msg"} }
            };
        if (ServerStream == null)
            return false;
        SecureNet.SendObject(ServerStream, request);
        if (App.Cancelled)
            return false;

        App.Log("Receiving GET msg response...");
        if (ServerStream == null)
            return false;
        using (var response = SecureNet.ReadObject<ServerResponse>(ServerStream))
        {
            App.Log($"Server Response: {response.ResponseSummary}");
            if (response.statusCode / 100 != 2)
                throw response.CreateException();

            msg? m = JsonConvert.DeserializeObject<msg>(response.headers["msg"]);
            string status = m == null ? "(null)" : m.from;
            App.Log($"Message: {status}");
            if (m == null)
                return false;
            else
                msgToken = m.token;

            if (!App.ConfirmDownload(m.from, m.message, out shouldDelete))
                return false;
        }
    }

    {
        App.Log("Sending GET file request...");
        var request =
            new ClientRequest()
            {
                version = 1,
                verb = "GET",
                headers =
                    new Dictionary<string, string>()
                    { { "token", msgToken }, { "part", "file"} }
            };
        if (ServerStream == null)
            return false;
        SecureNet.SendObject(ServerStream, request);
        if (App.Cancelled)
            return false;

        App.Log("Receiving GET file response...");
        if (ServerStream == null)
            return false;
        using (var response = SecureNet.ReadObject<ServerResponse>(ServerStream))
        {
            App.Log($"Server Response: {response.ResponseSummary}");
            if (response.statusCode / 100 != 2)
                throw response.CreateException();

            using (var temp_file_use = new TempFileUse(".zip"))
            {
                string temp_file_path = temp_file_use.FilePath;

                App.Log($"Downloading files...");
                if (App.Cancelled)
                    return false;
                using (var fs = File.OpenWrite(temp_file_path))
                {
                    long total_to_read = response.contentLength;
                    long read_yet = 0;
                    byte[] buffer = new byte[64 * 1024];
                    while (read_yet < total_to_read)
                    {
                        int to_read = (int)Math.Min(total_to_read - read_yet, 
                                       buffer.Length);
                        if (ServerStream == null)
                            return false;
                        int read = ServerStream.Read(buffer, 0, to_read);
                        if (App.Cancelled)
                            return false;

                        if (read == 0)
                            throw new NetworkException("Connection lost");
                        fs.Write(buffer, 0, read);
                        if (App.Cancelled)
                            return false;

                        read_yet += read;

                        App.Progress((double)read_yet / total_to_read);
                    }
                }

                App.Log($"Scanning downloaded files...");
                if (App.Cancelled)
                    return false;
                string local_hash;
                using (var fs = File.OpenRead(temp_file_path))
                    local_hash = Utils.HashStream(fs);
                if (App.Cancelled)
                    return false;
                if (local_hash != response.headers["hash"])
                    throw new NetworkException("File transmission error");

                App.Log($"Examining downloaded files...");
                string manifest = Utils.ManifestZip(temp_file_path);
                if (App.Cancelled)
                    return false;

                string extraction_dir_path = "";
                if (!App.ConfirmExtraction(manifest, out shouldDelete, 
                                           out extraction_dir_path))
                    return false;

                App.Log($"Saving downloaded files...");
                Utils.ExtractZip(App, temp_file_path, extraction_dir_path);

                App.Log($"All done.");
                return true;
            }
        }
    }
}

Server Message Sending

Over on the server side, here's the code for handling a user sending a messaging:

private async Task<ServerResponse> HandleSendRequestAsync
        (ClientRequest request, HandlerContext ctxt)
{
    // Unpack the message
    Utils.NormalizeDict
    (
        request.headers,
        new[]
        { "to", "message", "packageHash" }
    );

    string to = request.headers["to"];
    if (to == "")
        throw new InputException("Header missing: to");

    string message = request.headers["message"];
    if (message == "")
        throw new InputException("Header missing: message");

    long package_size_bytes = request.contentLength;
    if
    (
        MaxSendPayloadMB > 0
        &&
        package_size_bytes / 1024 / 1024 > MaxSendPayloadMB
    )
    {
        throw new InputException("Header invalid: package too big");
    }

    string sent_zip_hash = request.headers["hash"];
    if (sent_zip_hash == "")
        throw new InputException("Header missing: hash");

    Log(ctxt, $"Sending: To: {to}");

    using (var temp_file_use = new TempFileUse(".zip"))
    {
        string stored_file_path = "";
        string temp_zip_file_path = temp_file_use.FilePath;
        try
        {
            Log(ctxt, $"Saving ZIP: {temp_zip_file_path}");
            using (var zip_file_stream = File.OpenWrite(temp_zip_file_path))
            {
                long written_yet = 0;
                byte[] buffer = new byte[64 * 1024];
                while (written_yet < package_size_bytes)
                {
                    int to_read = (int)Math.Min
                                  (package_size_bytes - written_yet, buffer.Length);
                    int read = await ctxt.ConnectionStream.ReadAsync
                               (buffer, 0, to_read).ConfigureAwait(false);
                    if (read == 0)
                        throw new NetworkException("Connection lost");
                    await zip_file_stream.WriteAsync
                          (buffer, 0, read).ConfigureAwait(false);
                    written_yet += read;
                }
            }

            Log(ctxt, $"Hashing ZIP");
            string local_zip_hash;
            using (var zip_file_stream = File.OpenRead(temp_zip_file_path))
                local_zip_hash = await Utils.HashStreamAsync
                                 (zip_file_stream).ConfigureAwait(false);
            if (local_zip_hash != sent_zip_hash)
                throw new InputException("Received file contents do not match 
                                          what was sent");

            Log(ctxt, $"Storing ZIP");
            stored_file_path = m_fileStore.StoreFile(temp_zip_file_path);
            File.Delete(temp_zip_file_path);
            temp_zip_file_path = "";
            temp_file_use.Clear();

            Log(ctxt, $"Storing messages");
            string email_from = $"{ctxt.Auth["display"]} <{ctxt.Auth["email"]}>";
            var toos = to.Split(';').Select(t => t.Trim()).Where(t => t.Length > 0);
            foreach (var too in toos)
            {
                string token = 
                    m_msgStore.StoreMessage
                    (
                        new msg()
                        {
                            from = email_from,
                            to = too,
                            message = message
                        },
                        stored_file_path,
                        local_zip_hash
                    );

                    Log(ctxt, $"Sending email");
                    ctxt.App.SendDeliveryMessage
                    (
                        email_from,
                        too,
                        message,
                        token
                    );
            }
            stored_file_path = "";

            return
                new ServerResponse()
                {
                    version = 1,
                    statusCode = 200,
                    statusMessage = "OK"
                };
        }
        finally
        {
            if (stored_file_path != "" && File.Exists(stored_file_path))
                File.Delete(stored_file_path);
        }
    }
}

Server Message Receiving

And here's the server code for handling a user receiving a message:

private async Task<ServerResponse> HandleGetRequestAsync
              (ClientRequest request, HandlerContext ctxt)
{
    string to = ctxt.Auth["email"];
    m_allowBlock.EnsureEmailAllowed(to);

    Utils.NormalizeDict(request.headers, new[] { "token", "part" });

    string token = request.headers["token"];
    if (token.Length == 0)
        throw new InputException("Header missing: token");

    string part_to_get = request.headers["part"];
    if (part_to_get.Length == 0)
        throw new InputException("Header missing: part");

    bool get_msg = false, get_file = false;
    if (part_to_get == "msg")
        get_msg = true;
    else if (part_to_get == "file")
        get_file = true;
    else
        throw new InputException("Invalid header: part");

    Log(ctxt, $"Get Message: {to} - {token} - {part_to_get}");

    string package_file_path, package_file_hash;
    var msg = 
        m_msgStore.GetMessage
        (to, token, out package_file_path, out package_file_hash);

    if (get_msg)
    {
        if (msg == null)
        {
            var response_404 =
                new ServerResponse()
                {
                    version = 1,
                    statusCode = 404,
                    statusMessage = "Message Not Found"
                };
            return response_404;
        }

        var response =
            new ServerResponse()
            {
                version = 1,
                statusCode = 200,
                statusMessage = "OK",
                headers =
                    new Dictionary<string, string>()
                    { { "msg", JsonConvert.SerializeObject(msg) } },
            };
        await Task.FromResult(0);
        return response;
    }
    else if (get_file)
    {
        if (!File.Exists(package_file_path))
        {
            var response_404 =
                new ServerResponse()
                {
                    version = 1,
                    statusCode = 404,
                    statusMessage = "File Not Found"
                };
            return response_404;
        }

        var response =
            new ServerResponse()
            {
                version = 1,
                statusCode = 200,
                statusMessage = "OK",
                contentLength = new FileInfo(package_file_path).Length,
                headers =
                    new Dictionary<string, string>()
                    { { "hash", package_file_hash } },
                streamToSend = File.OpenRead(package_file_path)
            };
        await Task.FromResult(0);
        return response;
    }
    else
        throw new InputException("Invalid header: part");
}

Server Application Startup

Still on the server side, here is the server's startup code, you can see how it all comes together:

public ServerApp()
{
    string settings_file_path = Path.Combine(AppDocsDirPath, "settings.ini");
    if (!File.Exists(settings_file_path))
        throw new InputException
              ($"settings.ini file does not exist in {AppDocsDirPath}");
            
    m_settings = new Settings(settings_file_path);

    if (!int.TryParse
    (
        m_settings.Get("application", "MaxSendPayloadMB"),
        out MsgRequestHandler.MaxSendPayloadMB
    ))
    {
        throw new InputException("Invalid setting: MaxSendPayloadMB");
    }

    if (!int.TryParse
    (
        m_settings.Get("application", "ReceiveTimeoutSeconds"),
        out Server.ReceiveTimeoutSeconds
    ))
    {
        throw new InputException("Invalid setting: ReceiveTimeoutSeconds");
    }

    if (!int.TryParse
    (
        m_settings.Get("application", "ServerPort"),
        out ServerPort
    ))
    {
        throw new InputException("Invalid setting: ServerPort");
    }

    m_settingsWatcher = new FileSystemWatcher(AppDocsDirPath, "*.ini");
    m_settingsWatcher.Changed += SettingsWatcher_Changed;
    m_settingsWatcher.Created += SettingsWatcher_Changed;
    m_settingsWatcher.Deleted += SettingsWatcher_Changed;
    SettingsWatcher_Changed(new object(), 
            new FileSystemEventArgs(WatcherChangeTypes.All, "", null));

    m_txtFilesWatcher = new FileSystemWatcher(AppDocsDirPath, "*.txt");
    m_txtFilesWatcher.Changed += TextWatcher_Changed;
    m_txtFilesWatcher.Created += TextWatcher_Changed;
    m_txtFilesWatcher.Deleted += TextWatcher_Changed;
    TextWatcher_Changed(new object(), 
                new FileSystemEventArgs(WatcherChangeTypes.All, "", null));

    m_sessions = new SessionStore(Path.Combine(AppDocsDirPath, "sessions.db"));

    m_messageStore = new MessageStore(Path.Combine(AppDocsDirPath, "messages.db"));

    m_fileStore = new FileStore(m_settings.Get("application", "FileStoreDir"));

    m_logStore = new LogStore(Path.Combine(AppDocsDirPath, "logs"), "raw");
    m_accessStore = new LogStore(Path.Combine(AppDocsDirPath, "logs"), "access");

    string mail_server = m_settings.Get("application", "MailServer");
    if (string.IsNullOrWhiteSpace(mail_server))
        throw new InputException("Invalid setting: MailServer");

    int mail_port;
    if (!int.TryParse
    (
        m_settings.Get("application", "MailPort"),
        out mail_port
    ))
    {
        throw new InputException("Invalid setting: MailPort");
    }

    m_emailClient =
        new EmailClient
        (
            mail_server,
            mail_port,
            m_settings.Get("application", "MailUsername"),
            m_settings.Get("application", "MailPassword")
        );

    m_maintenanceTimer = new Timer(MaintenanceTimer, null, 0, 60 * 1000);

    var to_kvp = Utils.ParseEmail(m_settings.Get("application", "MailAdminAddress"));
    m_emailClient.SendEmail
    (
        m_settings.Get("application", "MailFromAddress"),
        new Dictionary<string, string>() { { to_kvp.Key, to_kvp.Value } },
        "Server Started Up",
        "So far so good..."
    );
}

Allow Block List File Loader

Finally, here is the function used to load the allow and block list text files. LINQ may be slow, but it sure is pretty, so where speed isn't the top concern, go for it!

private HashSet<string> LoadFileList(string fileName)
{
    string file_path = Path.Combine(AppDocsDirPath, fileName);
    if (File.Exists(file_path))
    {
        return
            new HashSet<string>
            (
                File.ReadAllLines(file_path)
                .Select(e => e.Trim().ToLower())
                .Where(e => e.Length > 0 && e[0] != '#')
            );
    }
    else
        return new HashSet<string>();
}

Conclusion

Well, that was a whirlwind tour of a .NET client-server intranet application, I hope you enjoyed that.

I haven't written much C# / .NET lately, so please edify me and my fellow dinosaurs with newfangled ways of doing things.

If after reading all this, you think this is something you'd like to try out, send me an email address or few that you'd like to access a demo server with a short "I'm not a robot" message to contact@msgfiles.io and I'll set you up. It's okay to send just one address, You can send files to yourself just like email. The addresses you send should come from an address with the same domain as the addresses.

History

• 11^th September, 2022: Initial version