Download multiple files from an FTP server in .NET

Introduction

This article is a brief description on how to code to download multiple files from an FTP server using FTPWebRequest. It is assumed that the reader has knowledge about the basic concepts of FTP (File Transfer Protocol) and C++.

Development Platform

• Operating System: Windows Server 2003
• Language: Managed C++ (new syntax)
• Environment: Visual Studio 2005 (.NET 2.0)

Background

.NET 2.0 provides the FtpWebRequest and FtpWebResponse classes to facilitate communication with FTP servers. The two classes are similar to HttpWebRequest and HttpWebResponse, respectively.

To initiate, first a request to the FTP server will be initiated, requesting the file needed.

FtpWebRequest ^objRequest = (FtpWebRequest^)WebRequest::Create(
              "ftp://servername:portnumber/filename");

Then, we have to get the response for this request:

FtpWebResponse ^objResponse = 
   (FtpWebResponse^) objRequest->GetResponse();

This response is a raw byte stream consisting of the file that was requested from the server. This stream is then read to get the file requested from the server. This concept is explained in detail below.

Until now, what we have learned is that we could only download one file at a time. What if we need to download multiple files or if we need to download the entire directory structure (files and sub-directories etc.) from an FTP server. As of now, there is no such API in .NET that could help us. So, we have to do things the hard way.

The only help .NET provides is that we can fetch the detailed listing of a directory residing on an FTP server. (That listing contains all the files and sub-directories contained in the directory.)

The response is received as a stream of raw bytes, and:

1. we have to collect that string,
2. and parse it to get the names of all the files and sub-directories,
3. and then loop through the list and request all the files one by one.
4. And, if the item in list happens to be a sub-directory, then again, fetch the detailed listing for it, and fetch all the files/directories contained there in one by one...

Recursion will definitely help us ;)

Using the code

Let's make things clear by using the code:

Make a new project in Visual Studio 2005, Managed C++. Create a class FTPClient. The header file (.h) will look like as:

namespace FTPClientInfo
{
    //The Directory detailed list recieved as ftp response is parsed and 
    //file and directory information is stored as this structure.

    public ref struct FileStruct
    {
        public:
            bool IsDirectory; //if the item is directory
            String ^Name;     //name of file/directory
        };

    public ref class FTPClient
    {
    private:
        String ^pszError;
        array<FileStruct^>^ GetListFromStringData(String ^strFileListData);
    public:
        //this function downloads a file from a FTP server
        //pszSourcePath - Path where file resides on FTP server.
        //pszDestinationPath - Path on local computer where the file
        //                     has to be downloaded

        bool DownloadFile(String ^pszServerIP, int PortNumber,                      
                          String ^pszUserName, String ^pszPassword, 
                          String ^pszFileName, String ^pszSourcePath, 
                          String ^pszDestinationPath);
        //this function downloads contents of directory recursively 
        //from FTP server

        bool DownloadDirectoryContents(String ^pszServerIP, int iPortNumber, 
                                   String ^pszUserName, String ^pszPassword,
                                   String ^pszSourcePath, 
                                   String ^pszDestinationPath);
        String^ GetError();
    };
}

Now, we provide the body for these function in a CPP file:

bool FTPClient::DownloadDirectoryContents(String ^pszServerIP, 
     int iPortNumber, String ^pszUserName, String ^pszPassword, 
     String ^pszSourcePath, String ^pszDestinationPath)
{
    try
    {
        //first we will create the URL to download the file


        String ^pszServerAddress = String::Concat("ftp://", 
                pszServerIP, ":", Convert::ToString(iPortNumber));
        

        
        String ^pszRequestURL = 
          String::Concat(pszServerAddress, pszSourcePath);

        //now request the file to be downloaded

        FtpWebRequest ^objRequest =    
          (FtpWebRequest^)WebRequest::Create(pszRequestURL);
        //set request creddentials (username & password)


        NetworkCredential^ myCred = 
          gcnew NetworkCredential(pszUserName, pszPassword);
        objRequest->Credentials::set(myCred);
        
        objRequest->Method = 
          WebRequestMethods::Ftp::ListDirectoryDetails;

        FtpWebResponse ^objResponse = 
          (FtpWebResponse^) objRequest->GetResponse();
        
        //get the response stream
        Stream ^objResponseStream = objResponse->GetResponseStream();
        StreamReader ^reader = gcnew StreamReader(objResponseStream);
        String ^strFileListData = reader->ReadToEnd();

        array<FileStruct^> ^arrFiles = 
           GetListFromStringData(strFileListData);

        for(int i = 0; i < arrFiles->Length; i++)
        {
            if(arrFiles[i]->IsDirectory == true)
            //if the directory is encountered
            {
                //in case we encounter a directory, first we will
                // create that directory on local computer
                //and then recursively call theis function
                // to download the contents of that sub-directory from
                //FTP server

                String ^pszNewDestinationPath = 
                  Path::Combine(pszDestinationPath, arrFiles[i]->Name);
                String ^pszNewSourcePath = 
                  Path::Combine(pszSourcePath, arrFiles[i]->Name);
                Directory::CreateDirectory(pszNewDestinationPath);
                if(false == DownloadDirectoryContents(pszServerIP, iPortNumber, 
                            pszUserName, pszPassword, pszNewSourcePath, 
                            pszNewDestinationPath))
                    return false;
            }
            else
            //if its a file to be downloaded
            {
                if(false == DownloadFile(pszServerIP, iPortNumber, 
                      pszUserName, pszPassword, arrFiles[i]->Name, 
                      pszSourcePath, pszDestinationPath))
                    return false;
            }
        }
        objResponseStream->Close();
            
        return true;
    }
    catch(Exception ^ex)
    {
        pszError = String::Copy(ex->Message);
        return false;
    }
}

//This is internal function. Its parses the string
//and fills an array of FileStruct structures.
//The records returned in Windows are of the form. 
//For eg. write "dir" command on ftp> dos prompt and you get:

//11-07-07  01:46PM                58880 diwali.doc
//12-10-07  03:06PM       <DIR>          nested
//11-20-07  03:37PM                 4285 problem.txt

//This is a FIXED format and from here we will
//distinguisg betweebn files and directories, 
//by parsing this string.
//As of now there is no in-built API in .NET
//for the same, so we have to do the things hard way :(

//TO DO : There is no other solution as of now except parsing the string.
//        String parsing is inevitable,
//        but we can change and imporove the parsing logic later.

array<FileStruct^>^ FTPClient::GetListFromStringData(String ^strFileListData)
{
    try
    {
        //first split the string by "\r\n" 
        //(Enter) and make an array of individual records

        array<Char> ^separator = {'\r','\n'};
        array<String^> ^arrList = strFileListData->Split(separator, 
           StringSplitOptions::RemoveEmptyEntries);
        
        //create an array of FileStruct type
        array<FileStruct^> ^arrFiles = 
          gcnew array<FileStruct^>(arrList->Length);

        //now parse every record in the list
        for(int i = 0; i < arrList->Length; i++)
        {
            FileStruct ^objFile = gcnew FileStruct();
            String ^strRecord = arrList[i]->Trim();
            String ^strName = strRecord->Substring(39);
            objFile->Name = strName;
            if(strRecord->Substring(24, 5) == "<DIR>")
                objFile->IsDirectory = true;
            else
                objFile->IsDirectory = false;

            arrFiles[i] = objFile;
        }
        
        return arrFiles;
    }
    catch(Exception ^ex)
    {    
        //if an exception occurs, throw it so 
        //that the calling function catches it
        throw gcnew Exception(ex->Message);
    }
}

bool FTPClient::DownloadFile(String ^pszServerIP, int iPortNumber, 
     String ^pszUserName, String ^pszPassword, 
     String ^pszFileName, String ^pszSourcePath, 
     String ^pszDestinationPath)
{
    try
    {
        //first we will create the URL to download the file
        String ^pszServerAddress = String::Concat("ftp://", 
                pszServerIP, ":", 
                Convert::ToString(iPortNumber));
        String ^pszFile = Path::Combine(pszSourcePath, pszFileName);

        //TO DO: Just see if we need to pass the "/"
        //in the path creation or will it be sent by the user herself?

        String ^pszRequestURL = String::Concat(pszServerAddress, pszFile);

        //now request the file to be downloaded

        FtpWebRequest ^objRequest =    
          (FtpWebRequest^)WebRequest::Create(pszRequestURL);
        //set request creddentials (username & password)


        NetworkCredential^ myCred = 
          gcnew NetworkCredential(pszUserName, pszPassword);
        objRequest->Credentials::set(myCred);
        //specify the methoid that we want to download 
        //the file. By the way, "Download File" is 

        //also the default value for Method. 
        //But written here just for the sake of clarity.
        objRequest->Method = WebRequestMethods::Ftp::DownloadFile;

        FtpWebResponse ^objResponse = 
           (FtpWebResponse^) objRequest->GetResponse();

        //get the response stream
        Stream ^objResponseStream = 
           objResponse->GetResponseStream();
        
        //now create a file from the response stream recieved.
        //File will be created at destination path
        //as specified in the parameter and with 
        //the same name as was of source file

        FileStream ^objFileStream = 
           File::Create(Path::Combine(pszDestinationPath, pszFileName));

        array [Byte] ^ buffer = gcnew array[Byte](1024);

        int nBytesRead;
        while(true)
        {
            nBytesRead = objResponseStream->Read(buffer, 0, 1024);
            if(0 == nBytesRead)
                break;
            objFileStream->Write(buffer, 0, nBytesRead);
            objFileStream->Flush();
        }

        objResponseStream->Close();
        objFileStream->Close();

        return true;
    }
    catch(Exception ^ex)
    {
        pszError = String::Copy(ex->Message);
        return false;
    }
}

String^ FTPClient::GetError()
{
    return pszError;
}

Now we need to test this FTP client. Test it using the following code:

FTPClient ^oFTPClient = gcnew FTPClient();
String ^pszServerIP = "127.0.0.1";
int iPortNumber = 21;
String ^pszUserName = "Administrator";
String ^pszPassword = "password";
//location on FTP server from where files are to be downloaded

//NOTE : We must specify a backslash "/" after the path, for this code
//to work. For empty path, specify "/" only.

String ^pszSourcePath = "/sample/";
//location on local system where files must be downloaded

String ^pszDestinationPath = "c:/FTP";

bool bDownload = oFTPClient->DownloadDirectoryContents(pszServerIP, 
     iPortNumber, pszUserName, pszPassword, 
     pszSourcePath, pszDestinationPath);

if(bDownload)
{
    Console::WriteLine("File Download OK");
}
else
    Console::WriteLine(String::Concat("Error Occured :: ", 
                       oFTPClient->GetError()));

Conclusion

This explains how to download multiple files using FTPWebRequest.

The FtpWebRequest and FtpWebResponse classes are not available in .NET 1.0 and 1.1. This code therefore will not work on those frameworks.