Bitsup - Server Upload Utility Using BITS

• Download source - 85.5 Kb

Introduction

Bitsup (Bits Up) is a program designed to automate data transfer from a production machine to a backup server. The program uses the BITS Microsoft Windows service to perform the transfer of files or directories. The program is command line driven, and designed to be run from a task scheduler, batch file, or script.

The Bitsup program creates an upload job in the BITS Microsoft Windows Service. It then compresses the data to be transferred, adds the compressed file or directory to the upload job, and starts the job. At this point, the BITS service takes over and handles the transfer of the file to the server.

The Bitsup program will monitor the transfer and present a progress bar, or can be setup to "Fire And Forget" the job; submitting the job to BITS, then exiting. There is an /Admin switch that will start Bitsup in the Administration mode, which displays all the jobs in the transfer queue, and allows the jobs to be cancelled, paused, resumed or completed.

Background

BITS

Background Intelligent Transfer Service (BITS) is a Windows component that asynchronously transfers files in the foreground or background, throttles the transfers to preserve the responsiveness of other network applications, and will automatically resume file transfers when a computer is restarted and a network connection is re-established.

I think that one of the beautiful things about BITS is that it extends the HTTP protocol to transfer files, and because it uses port 80, it is often immune to firewall restrictions. Also important is the fact that the default setting in BITS is keep trying to send a file for 14 days!

BITS Start Page on MSDN.

BITSAdmin

BITSAdmin is a command-line tool that you can use to create download or upload jobs and monitor their progress. Bitsup is primarily a "BITSAdmin Wrapper" utility. Because the BITS API is old school COM, and I have had enough of that, thanks for asking, I decided that the quickest way to get an interface into BITS was to wrap the BITSAdmin tool.

BITSAdmin Download Page on MSDN.

Zip file compression

Bitsup uses the Sharp Zip Library, and Open Source C# Compression Library. The compression engine supports the ZIP, GZIP, BZIP2 and TAR formats. For the Bitsup program, the compression used is the ZIP format. The compression level can be set from 0 to 9. A compression value of 0 will simply store the file/directory uncompressed inside the zip file. A compression value of 9 will produce maximum compression but will take more time to create the zip file archive. The default value is 6, which can be changed by editing the Bitsup.exe.config file using a text editor, such as Notepad.

I found a nice C# class for wrapping the SharpZipLib on the internet, and converted it to VB.NET for this project. I added some events to the class so that I could display a progress bar in a Windows Form dialog. I have also exposed a property to set the compression level for the zip file.

    Public Event FileProcessed(ByVal sFileName As String, _
                                      ByVal iPercent As Integer)
    Public Event DirProcessed(ByVal sDirName As String, _
                                      ByVal iPercent As Integer)
    Public Event ZipComplete()
    Public Event ZipStarted()

The Compression Progress dialog

Sharp Zip Library home page

Using the code

The heart of Bitsup is the cShell class, which presents a shared method (GetProcessText()) for calling a process, and getting the output from the called process. Bitsup uses GetProcessText() to wrap the BITSAdmin command line tool, and present a graphical interface.

Public Class cShell
    Public Shared Function GetProcessText(ByVal process As String, _
                                          ByVal param As String, _
                                          ByVal workingDir As String) _
                                          As String
        Dim p As Process = New Process
        ' this is the name of the process we want to execute 

        p.StartInfo.FileName = process
        If Not (workingDir = "") Then
            p.StartInfo.WorkingDirectory = workingDir
        End If
        p.StartInfo.Arguments = param
        ' need to set this to false to redirect output

        p.StartInfo.UseShellExecute = False
        p.StartInfo.RedirectStandardOutput = True
        p.StartInfo.CreateNoWindow = True
        ' start the process 

        p.Start()
        ' read all the output

        ' here we could just read line by line and display it

        ' in an output window 

        Dim output As String = p.StandardOutput.ReadToEnd
        ' wait for the process to terminate 

        p.WaitForExit()
        Return output
    End Function
End Class

The Command Line interface

Bitsup uses a command line interface. If Bitsup is started without any command line arguments, a help screen will be displayed, showing the available options.

Errors encountered during the command line parse will be displayed at the top of the screen.

The Help screen

The Command Line format

bitsup /MODE [/TYPE] [filename|directory]

MODE = RESET | ADMIN | TRANSFER

TYPE = FILE | DIRECTORY

Mode of operation

There are three basic "modes of operation" that Bitsup understands. The MODE parameter tells Bitsup which of these three modes to start up in. The MODE parameter must be prefixed by a forward slash. The valid values for the MODE parameter are RESET, ADMIN, and TRANSFER.

The RESET mode parameter causes Bitsup to clear all the transfer jobs from the transfer queue. This will stop any uploads already in progress.

The ADMIN mode parameter will display a listing of all the jobs currently in the transfer queue. Job ID, Job Name, Job State, number of files transferred, Number of files total, Number of bytes transferred, Number of bytes total are displayed for each job.

The TRANSFER mode parameter causes Bitsup to transfer data to the server. If the TRANSFER mode is used, the next parameter must be the TYPE parameter, which tells Bitsup which type of transfer to perform.

Type of transfer

If the MODE parameter is set to TRANSFER, then the TYPE parameter must be set to the type of transfer. Bitsup was designed to be extendable to other types of data besides FILES and DIRECTORIES. In the place where I work, there are several systems that need to be backed up regularly to an offsite location. I designed Bitsup to handle this task. In this code base, I have stripped out the other system specific transfer types.

To add your own custom transfer types is a simple exercise. Thinking about it, I should have created a cTranferType object that was inheritable, and descended the FILE and DIRECTORY transfers. Maybe in version 2.0 that will happen. I did write this in one night, after all.

A FILE transfer type specifies the transfer of a single file. The next parameter would be the full path to the file, including a drive letter. The file will be compressed and transferred to the server.

A DIRECTORY transfer type specifies that Bitsup should compress the contents of a directory and all of its sub-directories, and transfer the compressed archive to the server.

Points of interest

The CodeProject template file, from which I created this page, asks the following question:

Did you learn anything interesting/funny/annoying while writing the code? Did you do anything particularly clever or wild or zany?

Thinking about this question I did come up with one item. I found that the Process object has a problem getting a handle to a program's standard output stream when Bitsup was started via a shortcut. The solution to this was to create a batch file calling Bitsup, and start the batch file from the shortcut.

Dim p As Process = New Process

Also of interest, at least to me, is that I wrote Bitsup in one night. I have since thought of several improvements that I could make, but I am undecided as to whether the time improving Bitsup might be better spent creating a true wrapper around the BITS API. For now, Bitsup does what it is supposed to do, which is to transfer files using the BITS service.

History

• 6/25/2005: Posted to CodeProject.