Introduction
(This article assumes that you are familiar with Perl and ADO.)
When uploading files from a local PC to a remote FTP-server, I ran into the problem of having to upload a lot of files, where only a few actually changed.
So I decided to compare the timestamps of the local files with those of the remote files. Theoretically, a file needs to be uploaded only if:
- the local modification date is newer than the remote modification date, or
- the remote file doesn't exist at all
In practice, the problem is that the FTP-Server has another timebase (e.g., its clock is some seconds ahead or behind the local clock), so the comparison could fail.
Upload Algorithm
The solution presented in this article handles this by storing the modification dates of the files to upload in a local database and does the comparison with these timestamps.
In a nutshell, the "algorithm" works, for each file to upload, like this:
- Does an entry for the file exist in the database?
- If no, upload it, create new
recordset for this file, storing its path and modification date. - If yes, compare the actual modification date of the file with the stored modification date in the database.
- If actual modification date is newer than the stored one, upload the file, store its modification date in the database.
- If actual modification date is older than the stored one, do nothing.
As a drawback, the very first time the script is started, the database is empty, and therefore it once uploads all files, even if they would already exist on the remote FTP-server.
Starting from the second time, the script is executed, it works as described above.
Script Installation
You must have installed the Windows version of Perl from ActiveState. I always use the latest version, which is ActivePerl 5.6 as of writing this article.
In addition, I'm using some modules that are not in the standard distribution of Perl. These modules can be obtained from CPAN at www.cpan.org.
The following extra modules need to be installed:
Each module contains a readme file which explains how to install.
Script Configuration
When Perl and all modules are installed, you need to configure the script.
The following values must be configured:
| Value | Description | Example |
$ftp_server | The address (IP or name) of the remote FTP-server | "ftp.codeproject.com" |
$ftp_dir | The subdirectory on the remote FTP-server. Use no slash at the end | "/download/files" |
$ftp_uid | The username (user-ID) of the account to log on to the FTP-server | "chris" |
$ftp_pw | The password of the account to log on to the FTP-server | "maunder" |
@src_dir | An array of directories to upload. Each given directory is recursively processed (i.e., subdirectories are processed, too). | ("c:\mywebsite") |
@wc_exclude | An array of substring to match to exclude files or directories. For each found file (i.e., its complete path), a substring-match is executed, and then only if no item is found, this file or directory is processed. | ("_vti",".mdb","\\bak","\\data","server.inc") |
$db_connstr | The ADO connection string to connect to the database. You can use here any valid expression, e.g., an OLE-DB string or an ODBC string. | "Provider=Microsoft.Jet.OLEDB.4.0;Persist Security Info=False;Data Source=\\\\thisismyserver\\dir\\dir\\dir\\upload.mdb" |
$db_uid | The username of the account to connect to the database. Usually this field is empty. | "" |
$db_pw | The password of the account to connect to the database. Usually this field is empty. | "" |
Script Execution
When you configured everything, simple execute the script by double-clicking it in the explorer or by typing "perl upload.pl" at the command prompt.
The script writes output to stdout, as well as to a logfile called "upload.log".
Epilog
If you have any suggestions, found errors or want to comment on that article or script, please write a comment to at the end of this article and I will answer it.
Please keep in mind that I'm doing Perl for only a few months right now. So there are probably a lot of "no really beautiful" code-fragments in my script. So if you have any constructive comments regarding my coding style or the way I use Perl, please tell me!
