Introduction
Backup
is a utility program that will copy files from one path to another using one thread to display the names of the files being copied, and another thread to count the number of files and folders at the same time the files are being copied. This means no time is wasted waiting to get the file count before the copy can start; it copies and counts at the same time using two threads.
The Backup
application is also a handy utility to copy large volumes of files from one path to another quickly and efficiently. It is fast simply because it will not copy a file that is already in the destination path and has not changed (why copy a file that is already there and hasn't changed?). So it's great for repetitive backups as it only copies new or updated files.
Another added feature is that Backup
will not stop copying just because it got a security error on a particular file, or any type of error for that matter (excluding a hardware failure, of course). It will simply log the error and keep going until finished. You can then review the errors in the log file afterwards, which will show the error and the file that failed to copy. Most failures are due to security configuration problems.
Background
If you just want the Backup
utility, then there is no need to read on as the rest of this article is a technical discussion on multithreading or threading. The download links are at the top left corner of this page.
So why use threading? One reason may be because you want to be able to press a button on the form while the form is busy but it won't let you because it's busy. The old VB6 way of doing this is to use a Timer
perhaps, or use Do Events
at regular points in the code while processing. I doubt that Do Events
will work for this purpose in VB.NET and although a Timer
may solve the problem, there are other reasons to get familiar with threading.
Another reason is that with dual/multi core processing now available and true multitasking at the hardware and operating system level, threading is going to be hard to avoid, especially when performance is a concern and your application is processor intensive.
Okay, so you have decided you want to do threading in .NET. You could use the BackgroundWorker
(Windows Form control), but I suspect you should start with the System.Threading
namespace and use the Thread
class directly. My personal view is that the Thread
class is easier to use and gives you more flexibility although the BackgroundWorker
will remove some of the grunt programming work.
So what is a thread anyway? Simply put, it is the equivalent to another completely separate program that is initiated from your main program. Once a thread is started, the main program does not know anything about the thread (or threads); they are effectively separate programs in their own right. So some of the first things you may want to know are how to create them, how to pass data between them, how to call the main application from a thread and finally, how does the main application know which thread has called it?
Viewing the code snippets below should help answer these questions.
One final point before we start, using threads to copy files is not necessarily the best application of multithreading as copying files is disk I/O intensive rather than processor intensive. But it's still a fair use as counting files and copying files can occur asynchronously rather than synchronously, so why wait? And besides that, I think Backup
is a simple and handy utility at the same time.
Using the Code
This description of the code will not discuss the finer points of how to copy files but simply show the relevant source code to create the threads and use them from the main form application; showing how the threads are created and how parameters can be passed from the threads to the main Windows forms application.
First, take a quick look at the imported namespaces in the Backup
form:
Imports System.Threading
Imports System.IO
Imports System.Diagnostics.Process
The System.Threading
namespace is imported so you can create a Thread
class for each thread. The System.IO
namespace for file operations and the System.Diagnostics.Process
is used to view the log file with Notepad.
The next code snippet from the Backup
utility shows the declaration of a FileCopy
class called CopyFiles
in the main Windows form:
Imports System.Windows
Public Class Backup
Dim WithEvents CopyFiles As FileCopy
Private Sub StartCopy_Click(ByVal sender As System.Object, e As System.EventArgs) _
Handles StartCopy.Click
CopyFiles = New FileCopy
CopyFiles.StartCopy()
End Sub
.
. remaining forms code
.
End Class
Pressing the Go
button on the Backup
form will initiate the StartCopy_Click
subroutine. This subroutine will instantiate the FileCopy
class as CopyFiles
and then use the CopyFiles.StartCopy()
method to begin the copy.
So far, all we have done in the main Windows form is declare a class, instantiate it, and then execute the StartCopy()
method. So why have we created a class? Is it just good OOD practice?
And where is the code that declares the threads and starts them? Well, the very important point I am trying to make here, is that if you want to use threads, then it's best to create a class and use that class to start the threads. This is because the class can raise events whenever the threads it initiates need to communicate with the parent thread.
Okay, now let's look at the FileCopy
class, and see how it starts the threads:
Imports System.IO
Public Class FileCopy
Public CopyThread As System.Threading.Thread
Public CountThread As System.Threading.Thread
Public MirrorThread As System.Threading.Thread
Public Sub StartCopy()
CopyThread = New System.Threading.Thread(AddressOf Copy)
CopyThread.IsBackground = True
CopyThread.Name = "Copy"
CopyThread.Start()
CountThread = New System.Threading.Thread(AddressOf Count)
CountThread.IsBackground = True
CountThread.Name = "Count"
CountThread.Start()
End Sub
.
. code for the rest of the class
.
End Class
Pretty simple huh? Declare the threads and then have a method start them. So far, so good. But now that we have started the CopyThread
and the CountThread
, how do we get them to "talk to" the Backup
form by passing parameters? How will the Backup
form know which thread is doing what? These are reasonable questions when you consider threads run in their own address space and don't know about each other.
If you note the use of the AddressOf
operator when instantiating the threads, you will see that each thread is assigned to a private
subroutine in the FileCopy
class called Copy
and Count
, respectively. All you really need to do is have your code raise events in these subroutines whenever they want to communicate with the main parent thread (the parent thread is the Backup
windows form control/class).
Let's look quickly by expanding the FileCopy
class and see how this is done:
Imports System.IO
Public Class FileCopy
Public Event CopyStatus(ByVal sender As Object, ByVal e As BackupEventArgs)
Public Event CountStatus(ByVal sender As Object, ByVal e As BackupEventArgs)
Public Event MirrorStatus(ByVal sender As Object, ByVal e As BackupEventArgs)
Public CopyThread As System.Threading.Thread
Public CountThread As System.Threading.Thread
Public MirrorThread As System.Threading.Thread
Private _filePath As String
Private _fileSize As String
Private _copiedFolders As Long
Private _copiedFiles As Long
Private _countedFolders As Long
Private _countedFiles As Long
Private _mirroredFolders As Long
Private _mirroredFiles As Long
.
. even more class variables but we will just show the relevant ones.
.
Public Sub StartCopy()
CopyThread = New System.Threading.Thread(AddressOf Copy)
CopyThread.IsBackground = True
CopyThread.Name = "Copy"
CopyThread.Start()
CountThread = New System.Threading.Thread(AddressOf Count)
CountThread.IsBackground = True
CountThread.Name = "Count"
CountThread.Start()
End Sub
Private Sub Copy()
.
. this is a program loop with logic to copy files
.
Loop to Copy Files
Copy a file here
RaiseEvent CopyStatus(Me, New BackupEventArgs_
("", 0, _copiedFiles, _copiedFolders))
Threading.Thread.Sleep(1)
End Loop
End Sub
Private Sub Count()
.
. this is a program loop with logic to count files
.
Loop to Count Files
Count a file here
RaiseEvent CountStatus(Me, New BackupEventArgs_
("", 0, _countedFiles, _countedFolders))
Threading.Thread.Sleep(1)
End Loop
End Sub
.
. code for the rest of the class
.
End Class
So, there you have it. A basic class declaration that starts threads and raises events whenever it needs to communicate with the main parent thread, (which we know is the Backup
Windows form control).
Note the use of Threading.Thread.Sleep
. This is needed to allow the parent thread time to do its thing, like refresh the form display and respond to button clicks. Pretty cool so far, but we are not quite there yet.
We now need to look at the main parent thread (the Backup
Windows form control) and see the code needed to pass those parameters.
First, take note of the e
parameter in the declaration for the events in the code snippet above. The e
parameter is a variable of a custom class called BackupEventArgs
that has four parameters called FilePath
, FileSize
, FileCount
and FolderCount
. This custom class inherits from EventArgs
and is just a standard way to declare parameters for events. You could have just as easily declared a separate variable for each of the parameters instead of making a custom class to contain the parameters. To see what the BackupEventArgs
class is all about, please refer to the source code, as how you declare parameters for events is a little outside the scope of this article.
For now, just consider that the e
variable contains the parameters (FilePath
, FileSize
, FileCount
and FolderCount
) that we want to pass to and process in the main parent thread.
So, let's focus back on the parent thread, and let's see the code that deals with the events:
Imports System.Windows
Public Class Backup
Dim WithEvents CopyFiles As FileCopy
Public Delegate Sub CopyHandler(ByVal FilePath As String, _
ByVal FileSize As Long, ByVal FileCount As Long)
Public Delegate Sub CountHandler(ByVal FileCount As Long, ByVal FolderCount As Long)
.
. more declarations
.
Private Sub StartCopy_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) Handles StartCopy.Click
CopyFiles = New FileCopy
CopyFiles.StartCopy()
End Sub
Private Sub CopyStatus(ByVal FilePath As String, ByVal FileSize As Long, _
ByVal FileCount As Long)
.
. some code to update the forms display
.
End Sub
Private Sub CopyFiles_CopyStatus(ByVal sender As Object, _
ByVal e As BackupEventArgs) Handles CopyFiles.CopyStatus
Me.BeginInvoke(New CopyHandler(AddressOf CopyStatus), _
New Object() {e.FilePath, e.FileSize, e.FileCount})
End Sub
.
. remaining forms code
.
End Class
The code snippet above has a CopyFiles_CopyStatus()
event that is added to the main form (parent thread) when the CopyFiles
variable is declared using WithEvents
. It uses the Me.BeginInvoke
method of the form control to pass the parameters from the CopyThread
to the parent form thread. All Windows controls, including the forms control, have a BeginInvoke
method for asynchronous execution, (and an Invoke
method for synchronous execution), that allow you execute the specified delegate
.
In the code above, the specified delegate is CopyHandler
which is assigned to the CopyStatus
subroutine using the AddressOf
operator. When the CopyFiles_CopyStatus()
event occurs, it passes the parameters back to the main parent thread (the Backup
Windows form control) by executing the CopyStatus()
subroutine via the delegate, asynchronously.
If I have left you confused, just consider that although it looks like the CopyFiles_CopyStatus()
event is occurring in the Backup
windows form control/class, it actually isn't. It's really occurring in the CopyThread
thread, which is a separate bit of code in its own right that doesn't know about the parent thread. A difficult concept to grasp when the event in question is declared in the Windows form control class, but that's how it works.
Moving on, (for those still with me), just a couple of pointers that will help if you are new to threading. If we revise the FileCopy
class code snippet below to include the Join
method that is available on a thread, we can make one thread wait for another thread at a point in the code.
One problem (challenge actually) I had was that I needed to make the copy thread wait for the count thread to complete before it (the copy thread) terminates naturally (i.e., finishes copying files). You would expect the count thread to finish first, but when there are no files to copy, sometimes the copy thread finished first!
Using the join
method allows me to deal with this situation. So remember the Join
method if one thread depends on, and must wait for, the completion of another:
Imports System.IO
Public Class FileCopy
Public Event CopyStatus(ByVal sender As Object, ByVal e As BackupEventArgs)
Public Event CountStatus(ByVal sender As Object, ByVal e As BackupEventArgs)
Public Event MirrorStatus(ByVal sender As Object, ByVal e As BackupEventArgs)
Public CopyThread As System.Threading.Thread
Public CountThread As System.Threading.Thread
Public MirrorThread As System.Threading.Thread
Private _filePath As String
Private _fileSize As String
Private _copiedFolders As Long
Private _copiedFiles As Long
Private _countedFolders As Long
Private _countedFiles As Long
Private _mirroredFolders As Long
Private _mirroredFiles As Long
.
. even more class variables but we will just show the relevant ones.
.
Public Sub StartCopy()
CopyThread = New System.Threading.Thread(AddressOf Copy)
CopyThread.IsBackground = True
CopyThread.Name = "Copy"
CopyThread.Start()
CountThread = New System.Threading.Thread(AddressOf Count)
CountThread.IsBackground = True
CountThread.Name = "Count"
CountThread.Start()
End Sub
Private Sub Copy()
.
. this is a program loop with logic to copy files
.
Loop to Copy Files
Copy a file here
RaiseEvent CopyStatus(Me, New BackupEventArgs_
("", 0, _copiedFiles, _copiedFolders))
Threading.Thread.Sleep(1)
End Loop
If CountThread.IsAlive Then CountThread.Join()
End Sub
Private Sub Count()
.
. this is a program loop with logic to count files
.
Loop to Count Files
Count a file here
RaiseEvent CountStatus(Me, New BackupEventArgs_
("", 0, _countedFiles, _countedFolders))
Threading.Thread.Sleep(1)
End Loop
End Sub
.
. code for the rest of the class
.
End Class
And one final pointer. How do you terminate a thread? Using the Abort
method is frowned upon by some, although there is some consensus that it's okay to use if aborting the whole application, some purists believe it should be deprecated for technical reasons relating to how resources are allocated and unallocated during execution. In other words, you should signal the thread to terminate itself rather than aborting it when it's in the middle of something.
The code snippet below shows a method called StopThreads()
in the FileCopy
class that simply sets a boolean
class variable called _stopNow
to True
. When the StopThreads()
method is executed from the parent thread (Backup
), all any threads have to do is check _stopNow
and exit the processing loop to terminate the thread nicely, as shown in the Copy
subroutine code snippet below (which has been assigned to the copy thread as we saw earlier):
Public Class FileCopy
.
. class code
.
Public Sub StopThreads()
_stopNow = True
End Sub
.
. more class code
.
.
Private Sub Copy()
While Loop to copy files
If _stopNow Then
Exit While
End If
copy a file
raise an event to notify parent thread
End While
End Sub
.
.
.
End Class
Points of Interest
Much of the code above was developed by copying and adapting code from the MSDN. So if you really want to get into the finer points of threading, there are some very fine examples that have been well explained at msdn.microsoft.com.
I must say that the MSDN is finally improving to the point where it's hard not to be impressed. Some of the articles there even discuss best practice at length with some very fine walkthroughs. So a very special thanks to the coding gurus at MSDN.
And finally, in saying goodbye and thank you to those that have managed to read this far, please vote for my article, be generous and you will be rewarded in kind.
History
October 2007
Uploaded version 1.0.7 which contained bug fixes as well as added a batch mode for running without a form and processing command line parameters.
June 2008
Uploaded version 2.0.0 which is a complete revision of the source code and this article, also added a help file and a mirror copy feature.
July 2008
Uploaded version 2.1.0 which has a new mini status bar as well as some fixes to some faults that may surface in version 2.0.0. So please upgrade.
Also note that so far Backup has not been tested on Windows Vista. You may try it on Vista and, if there are problems, check back here periodically as there should be another release over the next few months that will be tested on Vista.
Upgrading to the Latest Version
If upgrading from a previous version, please uninstall the previous version first. Do this from the Control Panel (Start/Control Panel) by double clicking the Add/Remove Programs icon and selecting Backup
from the list.
Using the Latest Source Code
The source code for Backup
version 2 was created using Visual Studio 2008 and the .NET Framework 3.5, so older versions of Visual Studio are likely to have compatibility issues.
How to Install
- Look for the "Download BackUp" link on the top left of this page. Click the link to download it and then unzip it.
- You will see a "setup.exe" file and a "setup.msi" file. If you are installing on Windows 8 or higher, right click the "setup.exe" file, select "properties" then press the "compatibility" tab. Select "Windows Vista (Service Pack 2)" and press the "OK" button. If you are on Windows 7 or less, simply skip this step.
- Double click "setup.exe" to install but select your own folder to install to.
- It is best to select your own folder to install to e.g., "c:\backup", because if you install to the c:\program files (x86) folder, you will get this error "Access to the path ‘C\Program files (x86)\moneysoft\backup\191006-050101-Log.txt’ is denied". You can "run as administrator" to overcome this error or you can add the write permission to this folder, but it would be easier to install to a new location e.g., "c:\backup"
- You should see the Backup icon on your desktop.
- If running on Windows 8 or higher, hold down the "shift" key and right click the Backup icon at the same time, then select "run as administrator". If you do not "run as administrator", you will get a permissions error because access to the application directory will be denied by Windows 8 or higher for security reasons.
Troubleshooting
Error Message
Access to the path ‘C\Program files (x86)\moneysoft\backup\191006-050101-Log.txt’ is denied"
Cause
By default, when you install the backup utility, it will install to "C\Program files (x86)" unless you specify a different folder.
Solution
When installing, chose a different installation folder, e.g., c:\backup.
If you have already installed it, simply uninstall first, then reinstall.
Alternative Solution
Run as administrator.
Alternative Solution
Set the write permission on the installed folder.