(untagged)

Multithreading Backup Utility

Taner Riffat

0.00/5 (No votes)

9 Oct 2019

Multithreading is something we will all have to deal with sooner or later. This relatively simple application shows you how to use two threads to copy files. It is also a very handy Windows backup utility with a mirror feature, a batch mode feature, a log file feature and a help button!

Introduction

Backup is a utility program that will copy files from one path to another using one thread to display the names of the files being copied, and another thread to count the number of files and folders at the same time the files are being copied. This means no time is wasted waiting to get the file count before the copy can start; it copies and counts at the same time using two threads.

The Backup application is also a handy utility to copy large volumes of files from one path to another quickly and efficiently. It is fast simply because it will not copy a file that is already in the destination path and has not changed (why copy a file that is already there and hasn't changed?). So it's great for repetitive backups as it only copies new or updated files.

Another added feature is that Backup will not stop copying just because it got a security error on a particular file, or any type of error for that matter (excluding a hardware failure, of course). It will simply log the error and keep going until finished. You can then review the errors in the log file afterwards, which will show the error and the file that failed to copy. Most failures are due to security configuration problems.

Background

If you just want the Backup utility, then there is no need to read on as the rest of this article is a technical discussion on multithreading or threading. The download links are at the top left corner of this page.

So why use threading? One reason may be because you want to be able to press a button on the form while the form is busy but it won't let you because it's busy. The old VB6 way of doing this is to use a Timer perhaps, or use Do Events at regular points in the code while processing. I doubt that Do Events will work for this purpose in VB.NET and although a Timer may solve the problem, there are other reasons to get familiar with threading.

Another reason is that with dual/multi core processing now available and true multitasking at the hardware and operating system level, threading is going to be hard to avoid, especially when performance is a concern and your application is processor intensive.

Okay, so you have decided you want to do threading in .NET. You could use the BackgroundWorker (Windows Form control), but I suspect you should start with the System.Threading namespace and use the Thread class directly. My personal view is that the Thread class is easier to use and gives you more flexibility although the BackgroundWorker will remove some of the grunt programming work.

So what is a thread anyway? Simply put, it is the equivalent to another completely separate program that is initiated from your main program. Once a thread is started, the main program does not know anything about the thread (or threads); they are effectively separate programs in their own right. So some of the first things you may want to know are how to create them, how to pass data between them, how to call the main application from a thread and finally, how does the main application know which thread has called it?

Viewing the code snippets below should help answer these questions.

One final point before we start, using threads to copy files is not necessarily the best application of multithreading as copying files is disk I/O intensive rather than processor intensive. But it's still a fair use as counting files and copying files can occur asynchronously rather than synchronously, so why wait? And besides that, I think Backup is a simple and handy utility at the same time.

Using the Code

This description of the code will not discuss the finer points of how to copy files but simply show the relevant source code to create the threads and use them from the main form application; showing how the threads are created and how parameters can be passed from the threads to the main Windows forms application.

First, take a quick look at the imported namespaces in the Backup form:

Imports System.Threading
Imports System.IO
Imports System.Diagnostics.Process

The System.Threading namespace is imported so you can create a Thread class for each thread. The System.IO namespace for file operations and the System.Diagnostics.Process is used to view the log file with Notepad.

The next code snippet from the Backup utility shows the declaration of a FileCopy class called CopyFiles in the main Windows form:

Imports System.Windows

Public Class Backup

    ' Declare the FileCopy class.
    ' This class will create 3 threads to copy, count and mirror files
    ' and raise events for each, so the events must be handled
    ' to update the form with status data.
    Dim WithEvents CopyFiles As FileCopy

    Private Sub StartCopy_Click(ByVal sender As System.Object, e As System.EventArgs) _
        Handles StartCopy.Click

        ' Create the FileCopy class which will initiate the threads
        CopyFiles = New FileCopy

        ' Initiate the copy, count and mirror threads from the FileCopy class
        CopyFiles.StartCopy()

    End Sub
    .
    . remaining forms code
    .
End Class

Pressing the Go button on the Backup form will initiate the StartCopy_Click subroutine. This subroutine will instantiate the FileCopy class as CopyFiles and then use the CopyFiles.StartCopy() method to begin the copy.

So far, all we have done in the main Windows form is declare a class, instantiate it, and then execute the StartCopy() method. So why have we created a class? Is it just good OOD practice?

And where is the code that declares the threads and starts them? Well, the very important point I am trying to make here, is that if you want to use threads, then it's best to create a class and use that class to start the threads. This is because the class can raise events whenever the threads it initiates need to communicate with the parent thread.

Okay, now let's look at the FileCopy class, and see how it starts the threads:

Imports System.IO

Public Class FileCopy

    ' Declares the variables you will use to hold your thread objects.
    Public CopyThread As System.Threading.Thread
    Public CountThread As System.Threading.Thread
    Public MirrorThread As System.Threading.Thread

    Public Sub StartCopy()

        ' Sets the copy and count threads using the AddressOf the subroutine where
        ' the thread will start.
        CopyThread = New System.Threading.Thread(AddressOf Copy)
        CopyThread.IsBackground = True
        CopyThread.Name = "Copy"
        CopyThread.Start()

        CountThread = New System.Threading.Thread(AddressOf Count)
        CountThread.IsBackground = True
        CountThread.Name = "Count"
        CountThread.Start()

    End Sub
    .
    . code for the rest of the class
    .
End Class

Pretty simple huh? Declare the threads and then have a method start them. So far, so good. But now that we have started the CopyThread and the CountThread, how do we get them to "talk to" the Backup form by passing parameters? How will the Backup form know which thread is doing what? These are reasonable questions when you consider threads run in their own address space and don't know about each other.

If you note the use of the AddressOf operator when instantiating the threads, you will see that each thread is assigned to a private subroutine in the FileCopy class called Copy and Count, respectively. All you really need to do is have your code raise events in these subroutines whenever they want to communicate with the main parent thread (the parent thread is the Backup windows form control/class).

Let's look quickly by expanding the FileCopy class and see how this is done:

Imports System.IO

Public Class FileCopy

    ' Declare the events that will be raised by each thread
    Public Event CopyStatus(ByVal sender As Object, ByVal e As BackupEventArgs)
    Public Event CountStatus(ByVal sender As Object, ByVal e As BackupEventArgs)
    Public Event MirrorStatus(ByVal sender As Object, ByVal e As BackupEventArgs)

    ' Declares the variables you will use to hold your thread objects.
    Public CopyThread As System.Threading.Thread
    Public CountThread As System.Threading.Thread
    Public MirrorThread As System.Threading.Thread

    ' Class variables' Class variables
    Private _filePath As String
    Private _fileSize As String
    Private _copiedFolders As Long
    Private _copiedFiles As Long
    Private _countedFolders As Long
    Private _countedFiles As Long
    Private _mirroredFolders As Long
    Private _mirroredFiles As Long
    .
    . even more class variables but we will just show the relevant ones.
    .

    Public Sub StartCopy()

        ' Sets the copy and count threads using the AddressOf the subroutine where
        ' the thread will start.
        CopyThread = New System.Threading.Thread(AddressOf Copy)
        CopyThread.IsBackground = True
        CopyThread.Name = "Copy"
        CopyThread.Start()

        CountThread = New System.Threading.Thread(AddressOf Count)
        CountThread.IsBackground = True
        CountThread.Name = "Count"
        CountThread.Start()

    End Sub

     Private Sub Copy()
        .
        . this is a program loop with logic to copy files
        .
        Loop to Copy Files

            Copy a file here

            ' Raise the copy status event at the end of the program loop
            RaiseEvent CopyStatus(Me, New BackupEventArgs_
                    ("", 0, _copiedFiles, _copiedFolders))
            Threading.Thread.Sleep(1)
        End Loop

      End Sub

      Private Sub Count()
        .
        . this is a program loop with logic to count files
        .
        Loop to Count Files

            Count a file here

            ' Raise the count status event at the end of the program loop
            RaiseEvent CountStatus(Me, New BackupEventArgs_
                    ("", 0, _countedFiles, _countedFolders))
            Threading.Thread.Sleep(1)
        End Loop

    End Sub
    .
    . code for the rest of the class
    .
End Class

So, there you have it. A basic class declaration that starts threads and raises events whenever it needs to communicate with the main parent thread, (which we know is the Backup Windows form control).

Note the use of Threading.Thread.Sleep. This is needed to allow the parent thread time to do its thing, like refresh the form display and respond to button clicks. Pretty cool so far, but we are not quite there yet.

We now need to look at the main parent thread (the Backup Windows form control) and see the code needed to pass those parameters.

First, take note of the e parameter in the declaration for the events in the code snippet above. The e parameter is a variable of a custom class called BackupEventArgs that has four parameters called FilePath, FileSize, FileCount and FolderCount. This custom class inherits from EventArgs and is just a standard way to declare parameters for events. You could have just as easily declared a separate variable for each of the parameters instead of making a custom class to contain the parameters. To see what the BackupEventArgs class is all about, please refer to the source code, as how you declare parameters for events is a little outside the scope of this article.

For now, just consider that the e variable contains the parameters (FilePath, FileSize, FileCount and FolderCount) that we want to pass to and process in the main parent thread.

So, let's focus back on the parent thread, and let's see the code that deals with the events:

Imports System.Windows

Public Class Backup

    ' Declare the FileCopy class. This class will create 3 threads to copy, 
    ' count and mirror files
    ' and raise events for each, so the events must be handled to update
    ' the form with status data.
    Dim WithEvents CopyFiles As FileCopy

    ' Declare delegate handlers for the copy and count events
    Public Delegate Sub CopyHandler(ByVal FilePath As String, _
            ByVal FileSize As Long, ByVal FileCount As Long)
    Public Delegate Sub CountHandler(ByVal FileCount As Long, ByVal FolderCount As Long)
    .
    . more declarations
    .
    Private Sub StartCopy_Click(ByVal sender As System.Object, _
            ByVal e As System.EventArgs) Handles StartCopy.Click

        ' Create the FileCopy class which will initiate the threads
        CopyFiles = New FileCopy

        ' Initiate the copy, count and mirror threads from the FileCopy class
        CopyFiles.StartCopy()

    End Sub

     Private Sub CopyStatus(ByVal FilePath As String, ByVal FileSize As Long, _
            ByVal FileCount As Long)
        .
        . some code to update the forms display
        .
    End Sub

    Private Sub CopyFiles_CopyStatus(ByVal sender As Object, _
            ByVal e As BackupEventArgs) Handles CopyFiles.CopyStatus
        ' BeginInvoke causes asynchronous execution to begin at the address
        ' specified by the delegate. Simply put, it transfers execution of
        ' this method back to the main thread. Any parameters required by
        ' the method contained at the delegate are wrapped in an object and
        ' passed.

        Me.BeginInvoke(New CopyHandler(AddressOf CopyStatus), _
                New Object() {e.FilePath, e.FileSize, e.FileCount})

    End Sub
    .
    . remaining forms code
    .
End Class

The code snippet above has a CopyFiles_CopyStatus() event that is added to the main form (parent thread) when the CopyFiles variable is declared using WithEvents. It uses the Me.BeginInvoke method of the form control to pass the parameters from the CopyThread to the parent form thread. All Windows controls, including the forms control, have a BeginInvoke method for asynchronous execution, (and an Invoke method for synchronous execution), that allow you execute the specified delegate.

In the code above, the specified delegate is CopyHandler which is assigned to the CopyStatus subroutine using the AddressOf operator. When the CopyFiles_CopyStatus() event occurs, it passes the parameters back to the main parent thread (the Backup Windows form control) by executing the CopyStatus() subroutine via the delegate, asynchronously.

If I have left you confused, just consider that although it looks like the CopyFiles_CopyStatus() event is occurring in the Backup windows form control/class, it actually isn't. It's really occurring in the CopyThread thread, which is a separate bit of code in its own right that doesn't know about the parent thread. A difficult concept to grasp when the event in question is declared in the Windows form control class, but that's how it works.

Moving on, (for those still with me), just a couple of pointers that will help if you are new to threading. If we revise the FileCopy class code snippet below to include the Join method that is available on a thread, we can make one thread wait for another thread at a point in the code.

One problem (challenge actually) I had was that I needed to make the copy thread wait for the count thread to complete before it (the copy thread) terminates naturally (i.e., finishes copying files). You would expect the count thread to finish first, but when there are no files to copy, sometimes the copy thread finished first!

Using the join method allows me to deal with this situation. So remember the Join method if one thread depends on, and must wait for, the completion of another:

Imports System.IO

Public Class FileCopy

    ' Declare the events that will be raised by each thread
    Public Event CopyStatus(ByVal sender As Object, ByVal e As BackupEventArgs)
    Public Event CountStatus(ByVal sender As Object, ByVal e As BackupEventArgs)
    Public Event MirrorStatus(ByVal sender As Object, ByVal e As BackupEventArgs)

    ' Declares the variables you will use to hold your thread objects.
    Public CopyThread As System.Threading.Thread
    Public CountThread As System.Threading.Thread
    Public MirrorThread As System.Threading.Thread

    ' Class variables' Class variables
    Private _filePath As String
    Private _fileSize As String
    Private _copiedFolders As Long
    Private _copiedFiles As Long
    Private _countedFolders As Long
    Private _countedFiles As Long
    Private _mirroredFolders As Long
    Private _mirroredFiles As Long
    .
    . even more class variables but we will just show the relevant ones.
    .

    Public Sub StartCopy()

        ' Sets the copy and count threads using the AddressOf the subroutine where
        ' the thread will start.
        CopyThread = New System.Threading.Thread(AddressOf Copy)
        CopyThread.IsBackground = True
        CopyThread.Name = "Copy"
        CopyThread.Start()

        CountThread = New System.Threading.Thread(AddressOf Count)
        CountThread.IsBackground = True
        CountThread.Name = "Count"

        CountThread.Start()

    End Sub

    Private Sub Copy()
       .
       . this is a program loop with logic to copy files
       .
       Loop to Copy Files

           Copy a file here

           ' Raise the copy status event at the end of the program loop
           RaiseEvent CopyStatus(Me, New BackupEventArgs_
               ("", 0, _copiedFiles, _copiedFolders))
           Threading.Thread.Sleep(1)
       End Loop

       ' After all files have been copied, cause this CopyThread to wait 
       ' until CountThread has finished
       If CountThread.IsAlive Then CountThread.Join()

     End Sub

     Private Sub Count()
       .
       . this is a program loop with logic to count files
       .
       Loop to Count Files

           Count a file here

           ' Raise the count status event at the end of the program loop
           RaiseEvent CountStatus(Me, New BackupEventArgs_
                   ("", 0, _countedFiles, _countedFolders))
           Threading.Thread.Sleep(1)
       End Loop

   End Sub
   .
   . code for the rest of the class
   .
End Class

And one final pointer. How do you terminate a thread? Using the Abort method is frowned upon by some, although there is some consensus that it's okay to use if aborting the whole application, some purists believe it should be deprecated for technical reasons relating to how resources are allocated and unallocated during execution. In other words, you should signal the thread to terminate itself rather than aborting it when it's in the middle of something.

The code snippet below shows a method called StopThreads() in the FileCopy class that simply sets a boolean class variable called _stopNow to True. When the StopThreads() method is executed from the parent thread (Backup), all any threads have to do is check _stopNow and exit the processing loop to terminate the thread nicely, as shown in the Copy subroutine code snippet below (which has been assigned to the copy thread as we saw earlier):

Public Class FileCopy
    .
    . class code
    .
    Public Sub StopThreads()

        _stopNow = True

    End Sub
    .
    . more class code
    .
    .
    Private Sub Copy()

        While Loop to copy files

            If _stopNow Then
                Exit While
            End If

            copy a file
            raise an event to notify parent thread

        End While

    End Sub
    .
    .
    .
End Class

Points of Interest

Much of the code above was developed by copying and adapting code from the MSDN. So if you really want to get into the finer points of threading, there are some very fine examples that have been well explained at msdn.microsoft.com.

I must say that the MSDN is finally improving to the point where it's hard not to be impressed. Some of the articles there even discuss best practice at length with some very fine walkthroughs. So a very special thanks to the coding gurus at MSDN.

And finally, in saying goodbye and thank you to those that have managed to read this far, please vote for my article, be generous and you will be rewarded in kind.

History

October 2007

Uploaded version 1.0.7 which contained bug fixes as well as added a batch mode for running without a form and processing command line parameters.

June 2008

Uploaded version 2.0.0 which is a complete revision of the source code and this article, also added a help file and a mirror copy feature.

July 2008

Uploaded version 2.1.0 which has a new mini status bar as well as some fixes to some faults that may surface in version 2.0.0. So please upgrade.

Also note that so far Backup has not been tested on Windows Vista. You may try it on Vista and, if there are problems, check back here periodically as there should be another release over the next few months that will be tested on Vista.

Upgrading to the Latest Version

If upgrading from a previous version, please uninstall the previous version first. Do this from the Control Panel (Start/Control Panel) by double clicking the Add/Remove Programs icon and selecting Backup from the list.

Using the Latest Source Code

The source code for Backup version 2 was created using Visual Studio 2008 and the .NET Framework 3.5, so older versions of Visual Studio are likely to have compatibility issues.

How to Install

Look for the "Download BackUp" link on the top left of this page. Click the link to download it and then unzip it.
You will see a "setup.exe" file and a "setup.msi" file. If you are installing on Windows 8 or higher, right click the "setup.exe" file, select "properties" then press the "compatibility" tab. Select "Windows Vista (Service Pack 2)" and press the "OK" button. If you are on Windows 7 or less, simply skip this step.
Double click "setup.exe" to install but select your own folder to install to.
It is best to select your own folder to install to e.g., "c:\backup", because if you install to the c:\program files (x86) folder, you will get this error "Access to the path ‘C\Program files (x86)\moneysoft\backup\191006-050101-Log.txt’ is denied". You can "run as administrator" to overcome this error or you can add the write permission to this folder, but it would be easier to install to a new location e.g., "c:\backup"
You should see the Backup icon on your desktop.
If running on Windows 8 or higher, hold down the "shift" key and right click the Backup icon at the same time, then select "run as administrator". If you do not "run as administrator", you will get a permissions error because access to the application directory will be denied by Windows 8 or higher for security reasons.

Troubleshooting

Error Message

Access to the path ‘C\Program files (x86)\moneysoft\backup\191006-050101-Log.txt’ is denied"

Cause

By default, when you install the backup utility, it will install to "C\Program files (x86)" unless you specify a different folder.

Solution

When installing, chose a different installation folder, e.g., c:\backup.

If you have already installed it, simply uninstall first, then reinstall.

Alternative Solution

Run as administrator.

Alternative Solution

Set the write permission on the installed folder.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here