Verifying .MD5 file verification databases

• Download source - 94.6 Kb
• Download binary - 83.3 Kb

Introduction

Many of you will know what an .sfv file is. For those who don't, it's some kind of file verification database which contains the CRC32 of one or more files. It is used to check file integrity (after download, ...). To process these files (.SFV, .CRC, .CSV and .CKZ.), I recommend using QuickSFV. This article covers a newer standard, a newer hash algorithm: MD5. Sometimes you get .MD5 files. Up to now, I haven't seen any Windows utility which can process these files yet. Some console-based programs can calculate the checksum, and then it's up to you to verify it with the .MD5 file... So I took a look at what the .NET Framework had to offer, and it seemed very easy to calculate the MD5 checksum of a file. Parsing the .MD5 files was no problem: in fact it's plain text, every line begins with a checksum, followed by a few tab characters and ending with a filename.

If you just want to use the utility, then only download the binary. In the .ZIP file is only one executable, and setup is as easy as copying it to the directory you like, make a shortcut and associate it with .MD5 files. If you want to understand the code and hera the whole story, read on.

Using the code

First of all, I built an MD5Verifier class. If you want to use .MD5 file processing in your programs, all code is in this class so you would just have to copy and paste it into your project. Then I started working on the GUI.

The user can optionally chose an encoding for files that can't be processed with the default system encoding. The he/she selects a file to process. The menus and controlbox are disabled, an MD5Verifier is created and MD5Verifier.doVerify() is started in a new thread, so the interface remains responsive. MD5Verifier raises two events: one to inform the GUI the progress and one to inform the GUI that processing has finished.

Building and using the verifier class

The goal is to keep the interface responsive while verifying files. So we know that the method which starts the verify process can have no parameters. Therefore, we create a constructor which sets the filename of the .MD5 file and the encoding. We also need two variables two hold these objects.

using System;
using System.Text;

...

namespace QuickMD5
{
    public class MD5Verifier
    {
        private string file;
        private Encoding enc;

        ...

        public MD5Verifier(string Filename, Encoding UseEncoding)
        {
            // set initial values

            file = Filename;
            enc = UseEncoding;
        }
        
        ...

    }
}

We do not want to create a loop in the GUI to wait for the thread to exit. And, we want to keep the GUI up to date. To do this, we can implement two events: onVerifyProgress and onVerifyDone. The class needs to inform the GUI which file it is processing, the status of the file (still verifying, OK, bad, ...), how many files it has verified, how many of them where OK, bad, missing and the total amount of files to verify. To pass the status to the GUI we create an MD5VerifyStatus enumeration. The status MD5VerifyStatus.None is used at the beginning of the process to reset the counters in the GUI.

During the verification process, all entries in the database are stored in an array, therefore we need a structure which contains the file name and the corresponding checksum string: fileEntry

Now, only one thing is missing, the doVerify() method. Currently the code of the class looks like this:

using System;
using System.Collections;
using System.IO;
using System.Security.Cryptography;
using System.Text;
using System.Windows.Forms;

namespace QuickMD5
{
    public class MD5Verifier
    {
        private string file;
        private Encoding enc;

        private struct fileEntry
        {
            public string checksum;
            public string file;
        }
        
        public delegate void onVerifyProgressDelegate(string file, 
          MD5VerifyStatus status, int verified, int success, 
          int corrupt, int missing, int total);
        public delegate void onVerifyDoneDelegate(Exception ex);
        public event onVerifyProgressDelegate onVerifyProgress;
        public event onVerifyDoneDelegate onVerifyDone;
        
        public enum MD5VerifyStatus 
        {
            None,
            Verifying,
            OK,
            Bad,
            Error,
            FileNotFound,
        }
        
        public MD5Verifier(string Filename, Encoding UseEncoding)
        {
            // set initial values

            file = Filename;
            enc = UseEncoding;
        }
        
        public void doVerify()
        {

            ...

        }
    }
}

In the doVerify() method we use a TextReader object to read the file. The data from the file is converted to text using the selected encoding (accessed through enc).

Each valid line must have a minimum length of 35 characters: a checksum string of 32 characters, two tab characters (haven't seen any other format) and a filename. Therefore, all lines of less than 35 characters are left out. All other lines are assumed to contain a checksum in the first 32 characters and a filename from character 35 and on. All valid entries are stored in an arraylist as fileEntry structures.

        public void doVerify()
        {

            // try to open the file

            
            TextReader tr;
            try
            {
                tr = new StreamReader(file, enc);
            }
            catch (Exception ex)
            {
                onVerifyDone(ex);
                return;
            }
            
            // read the file

            string line;
            ArrayList files = new ArrayList();
            while((line = tr.ReadLine()) != null) {
                if (line.Length >= 35) 
                {
                    fileEntry entry;
                    entry.checksum = line.Substring(0, 32);
                    entry.file = line.Substring(34);
                    files.Add(entry);
                }
            }
            
            // close it

            tr.Close();
            
            ...
            

Before the actual processing of files starts, we change the working directory to the .MD5 file path, because file paths inside this .MD5 file could be relative. We also need to define the counters and reset the GUI counters by calling onVerifyProgress. And of course, if no valid entries were found, we can stop this process right here.

            ...
            
            // try to access directory

            try 
            {
                Environment.CurrentDirectory = 
                     new FileInfo(file).Directory.FullName;
            }
            catch (Exception ex)
            {
                onVerifyDone(ex);
                return;
            }
            
            // display progress

            int ver = 0;
            int success = 0;
            int corrupt = 0;
            int missing = 0;
            onVerifyProgress("", MD5VerifyStatus.None, ver, success, 
              corrupt, missing, files.Count);
            
            // check for empty (maybe invalid) files

            if (files.Count < 1) 
            {
                onVerifyDone(null);
                return;
            }
                        
            ...

To calculate the checksums, we need an instance of the MD5CryptoServiceProvider class. Next, we open each file, calculate its checksum and compare it. The checksum is returned as a byte array by the MD5CryptoServiceProvider class, so we need to convert it using the BitConverter. This outputs a string like "hh-hh-hh..." where hh are hex values. To compare the checksum with the one in the .MD5 file, we need to remove the '-' characters. And, the comparison should be case-insensitive, some we make both checksum strings lowercase.

            ...
            
            MD5CryptoServiceProvider csp = new MD5CryptoServiceProvider();
            
            for (int idx = 0; (idx < files.Count); ++idx) 
            {
                // get file

                fileEntry entry = (fileEntry) files[idx];
                
                // display file name

                onVerifyProgress(entry.file, MD5VerifyStatus.Verifying, ver, 
                  success, corrupt, missing, files.Count);
                
                if (File.Exists(entry.file)) 
                {
                    try
                    {
                        // compute hash

                        FileStream stmcheck = File.OpenRead(entry.file);
                        byte[] hash = csp.ComputeHash(stmcheck);
                        stmcheck.Close();
                        
                        // convert to string

                        string computed = BitConverter.ToString(
                          hash).Replace("-", "").ToLower();
                        // compare

                        if (computed == entry.checksum) 
                        {
                            ++ver;
                            ++success;
                            onVerifyProgress(entry.file, MD5VerifyStatus.OK,
                              ver,
                              success, corrupt, missing, files.Count);
                        }
                        else
                        {
                            ++corrupt;
                            ++success;
                            onVerifyProgress(entry.file, 
                              MD5VerifyStatus.Bad, ver, 
                              success, corrupt, missing, files.Count);
                        }
                    }
                    catch
                    {
                        // error

                        ++ver;
                        ++corrupt;
                        onVerifyProgress(entry.file, 
                          MD5VerifyStatus.Error, ver, 
                          success, corrupt, missing, files.Count);
                    }
                }
                else
                {
                    // file does not exist

                    ++ver;
                    ++missing;
                    onVerifyProgress(entry.file, 
                        MD5VerifyStatus.FileNotFound, ver, 
                        success, corrupt, missing, files.Count);
                }
            }
            
            onVerifyDone(null);
        }

That's it, that's how I made the class. If you want to see how to use it in your application or how the GUI works, you'll have to download the source, because that would lead me too far away from the main problem. To see how to parse the command line, read on.

Application : Parsing the command line

To get the command line, I used Environment.GetCommandLineArgs(), which returns the application path as first argument. All other arguments are searched for valid and existing files, and the first valid file is used to start the application, all other files are opened in new instances of the program.

        [STAThread] static void Main()
        {
            // enable themes support

            Application.EnableVisualStyles();
            Application.DoEvents();
            
            // parse command line, first argument is app executable

            string[] args = Environment.GetCommandLineArgs();
            string file = null;
            if (args.Length > 1) 
            {
                for (int i = 1; (i < args.Length); ++i)
                {
                    // check if argument contains valid filename

                    if (File.Exists(args[i])) 
                    {
                        // if no file found, set it, if valid file(s) 

                        // found, launch new instance

                        if (file == null) 
                        {
                            file = args[i];
                        }
                        else
                        {
                            Process.Start(
             Process.GetCurrentProcess().MainModule.FileName, args[i]);
                        }
                    }
                }
                
                // start interface only on valid commandline

                if (file != null) Application.Run(new MainWindow(file));
            }
            else Application.Run(new MainWindow(null));
        }

Note: to do this you have to change the constructor of your MainWindow to accept a string argument.

History

• Original code (01/07/2004)
• Code optimized and article updated accordingly based on a comment by Nathan Blomquist (01/08/2004)