Export and Archive your Projects on Schedule via Powershell => made easy!

Introduction

During several years of working as a web developer, I have created quite a number of MVC and WebForms projects. And of course I wanted to have them backed up all the time. Since they all are under some source control, they are essentialy backed up (to Subversion server and Git bare repository in shared folder on the network). Also, a backup image of my computer is being created every day.

But that wasn't enough for me! And being as paranoid about hard drive failures == greedy for N-number of backups of the stuff, as I am, I was also creating manual archives of particular projects... Not to mention, I wasn't removing version control and temporary data (like 'bin' and 'obj' folders) from those archives, as much as I wanted to.

Recently, I got enough of that nasty set of repetitive mouse clicks and filthy drag-n-drops. I thought: "That is stupid, we live in the age of technology and information, there is ISS floating above our heads, LHC constantly failing in Europe, and I'm still creating archives manually, like a cave man... My PC must do this tedious task for me!".

So, in this article, I'll explain what I've done in order to relieve myself from my pain.

Contents

• Goals
• Using Windows batch file
• Using Poweshell (basics)
• Declare a variable in PowerShell
• Declare an array of data in PowerShell
• Run code in 'For' loop
• Run code in 'Foreach' loop 
• Find particular files and archive them
• Writing main PowerShell script
• How to run script on schedule
• References
• History

Goals

This article accomplishes the following goals:

• Copies particular project folders, while preserving subfolder structure
• Removes all version control information (from Subversion and Git)
• Removes temporary, binary, and unneeded files and folders (e.g., 'obj' and 'bin' folders, ReSharper data, and others).
• Archives cleaned up copies of projects
• Finally, does all actions above on schedule

Using Windows batch file

I used to play a lot with Linux Mint, where I enjoyed writing simple shell scripts to accomplish daily tasks, like e.g. shut down WiFi, start a backup, change keyboard layout, mount system image, and so on.

For example, this shell script creates a Gzip archive from all files on the system drive, while excluding itself and certain other system folders:

cd /
sudo tar cvpzf backup.tgz --exclude=/backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/sys  /    

So, I wanted to do something similar in Windows, and hopefully make it smart.

The first thing that came to my mind was writing a simple batch file (text file with .bat extension), so when you execute it, it'll run some commands in sequence.

To get going with a batch file, I firstly needed a Windows command line command which would copy only particular project folders with files from my projects directory and preserve subfolder structure too. After some research I found command called "XCopy", which was perfectly suited for my needs. Here how it looked:

XCopy SourcePath DestinationPath /EXCLUDE:C:\export_exclude.txt /C /F /I /E /R /H /Y  

And now I'll go in little more detail about that command.

/EXCLUDE:C:\export_exclude.txt - path to text file containing all names to be excluded from copying. In my case I added those names:

.svn
.git
.suo
\obj\
Thumbs.db
ReSharper    

Here I got a basic set to be cleaned out - Subversion and Git data (.svn and .git), project temporary data (.suo and \obj\), Windows thumbnails for images (Thumbs.db), and ReSharper project data (ReSharper).

To be more specific, here is how to set a list of exclusions (citation from here):

"List each string in a separate line in each file. If any of the listed strings match any part of the absolute path of the file to be copied, that file is then excluded from the copying process. For example, if you specify the string '\Obj\', you exclude all files underneath the Obj directory. If you specify the string '.obj', you exclude all files with the .obj extension."

Now let me explain the meaning of the keys at the end of that command.

/C - Ignores errors.

/F - Displays source and destination file names while copying.

/I - Displays a list of files that are to be copied.

/E - Copies all subdirectories, even if they are empty.

/R - Copies read-only files.

/H - Copies files with hidden and system file attributes. By default, XCopy does not copy hidden or system files.

/Y - Suppresses prompting to confirm that you want to overwrite an existing destination file.

I've set those keys in a that manner so everything would be copied silently on the background but also I can see some output if I run this command manually.

After that I created a batch file (copy.bat) - which is just a text file with extension .bat. Since I needed to cleanly copy only specific projects my batch file looked like this:

XCopy Project1Path Project1CopyPath /EXCLUDE:C:\export_exclude.txt /C /F /I /E /R /H /Y
XCopy Project2Path Project2CopyPath /EXCLUDE:C:\export_exclude.txt /C /F /I /E /R /H /Y
XCopy Project3Path Project3CopyPath /EXCLUDE:C:\export_exclude.txt /C /F /I /E /R /H /Y
...
XCopy ProjectNPath ProjectNCopyPath /EXCLUDE:C:\export_exclude.txt /C /F /I /E /R /H /Y  

It was working ok. But It just didn't feel right, you know, from programmer's standpoint. Too much repetition (and we don't need no stinkin' badge... oops, repetitions :)).

You can already see how unmaintainable it'll get with a large list of projects. And what if you'd want to change XCopy keys? Doing 'Ctrl+H' in notepad was not too convenient either...

I needed better solution, and It came in a form of PowerShell.

Using PowerShell (basics)

If you already have some experience with PowerShell, you may want to skip to the next section, as this one covers only basics of PowerShell. Otherwise, read on!

PowerShell is an advanced console application for Window. It's purpose is to add advanced functionality to command line operations (similar to bash shell in Linux) and provide powerful environment for writing shell scripts.

PowerShell runs on top of the .NET Framework, so all its power is available to PowerShell. For example, unlike regular batch files, PowerShell script files (text files with .ps1 extension) can have C# like code - declare variables and arrays, 'For' loops and 'If' statements, etc; all of which opens a whole lot of new possibilities to write advanced and effective scripts.

If you have Windows 7 SP1, you, most probably, already have PowerShell in your system. Just search for 'PowerShell' from start menu to check if you have it. If not, download and install it from here: http://www.microsoft.com/PowerShell.

Also remember! After you install PowerShell you won't be able to run PowerShell scripts at first! Sounds ridiculous, ha? Well that is a genuine Microsoft-style security measure. To enable running scripts open PowerShell command prompt and enter (or copy via Shift+Insert) this command:

Set-ExecutionPolicy RemoteSigned

This will enable your system to run scripts created locally, but not downloaded from the internet (you probably wouldn't want to do that anyway)... For more Information about this, read this.

Let's look into basic knowledge I needed to accomplish my goals

Declare a variable in PowerShell

The first thing we need to know, is how to set variables. Open PowerShell prompt and declare a variable like that:

$test = "test"  

This will create a string variable test with a value "test". Here how it looks in PowerShell, first I declared the variable, then I called it by its name:

Pretty straightforward isn't it?

Declare an array of data in PowerShell

After that, I wanted to know how to declare an array of data, in order to keep names of project folders I want to backup:

$array = @(0,9,0,9,1,9,8,2)    

This basically creates a list of integers, and outputting it works the same was as with regular variable:

Looks good, and what do I need next?

Run code in 'For' loop

This works very similar to your regular C# application. Let's use an array $array I declared previously to output it's contents:

$array = @(0,9,0,9,1,9,8,2)

for ($i = 0; $i -le $array.Length - 1; $i++) {
    Write-Host $array[$i]
} 

Now comes the bad news - you cannot simply copy & paste that code into console. You'll rather have to enter it in one line, separated by ';', and it'll look like this:

Not too convenient, right? But let me introduce you to the better way of running and testing PowerShell scripts - PowerShell Script Editor or 'Windows PowerShell ISE'. This program is installed along with PowerShell. From Windows 7 start menu search you can find it by entering 'ise' in search area. Or you can launch it manually from it's folder - C:\Windows\System32\WindowsPowerShell\v1.0\PowerShell_ISE.exe.

Here is how our little script looks now:

All prettily colored, no need to enter commands in one line (and use ';' to separate commands), you can save your script now, and see it's output right away!

Run code in 'Foreach' loop

For loop is pretty good, but I wanted something simpler for my little project, I wanted a Foreach loop:

$array = @(0,9,0,9,1,9,8,2)

foreach ($element in $array) {
    $element
}   

After we run this code, the output is exactly the same as before. And you also can omit 'Write-Host' command:

Next, I'll describe some useful hints for PowerShell, but not necessary related to this article's main goal.

Find particular files and archive them

You can use similar command to find all files on drive 'C:\' with extension '.ps1': 

$FilesList = Get-ChildItem "C:\" -recurse -include "*.ps1" 

More information about Get-ChildItem command here. 

Then you can list all discovered files and their data like this:

$FilesList | Format-Table Name, Directory 

All available columns for the command above are: Name, Directory, Length, Mode, LastWriteTime.

And now - a simple example how to do something with discovered files. For instance, let's archive them with 7-Zip and save resulting archive on the drive 'C:\': 

foreach ($file in $FilesList) {
  $dp = $file.Directory; $fn = $file.Name; $fp = "$dp\$fn";
  cd "C:\Program Files\7-Zip"
  ./7z.exe a C:\PowerShell_Scripts.7z $fp
}  

And that's almost all I needed to know about PowerShell at this point. Now was the time to start writing main script

Writing main PowerShell script

Once again, my main goal was to write a script that should do this:

Create a clean copy of selected projects > Archive cleaned projects > Remove temporary directories.

I created a new empty file export_projects.ps1 (just a text file with .ps1 extension) which will contain all commands below.

Before I started, I realized that it would be better to append current date and/or time to the name of my backup archive, so I added this variable first:

$date = Get-Date -uformat "%Y-%m-%d_%H-%M"  

You can easily format that date/time to look like you need by using keys for -

Date:

%D // full date with slashes '/' (e.g. 03/27/12)
%Y // full year (e.g. 2012)
%y // short year (e.g. 12)
%m // month number (e.g. 03)
%h // month short name (e.g. Mar)
%B // month long name (e.g. March)
%d // day number (e.g. 27) 

Time:

%H // hour in 24-hour format
%I // hour in 12-hour format
%M // minute
%S // second  

To learn even more Get-Date command keys, read this.

Secondly, I declared all necessary variables and arrays:

$exclude = "C:\export_exclude.txt"                   # full path to file with exclusions
$projects_directory = "C:\projects_directory\"       # full path to source projects directory
$projects_destination = "C:\projects_backup\"        # where to copy cleaned projects
$projects_archive = "C:\projects_backup_$date.7z"    # where to save projects archive

# list of project folder names (inside of the 'projects_directory')
$projects = @(
    "Project1",
    "Project2",
    "Project3"
)   

Then I added foreach loop to cleanly copy each project in $projects array using XCopy command I described earlier, by substituting fixed paths with our variables:

foreach ($project in $projects) {
  XCopy $projects_directory$project $projects_destination$project /EXCLUDE:$exclude /C /F /I /E /R /H /Y 
} 

After that, I added commands to archive clean copy of projects using popular 7-Zip archiver. I chose 7-Zip because it's fast, free, and open source. And since all modern archivers have an option to use them via command line, I happily took advantage of that -

First, I told PowerShell to go into 7-Zip program directory:

cd "C:\Program Files\7-Zip" 

Then, I told PowerShell to create an archive by running 7-Zip command line executable file '7z.exe' with 'a' parameter, which is 'add to archive', along with archive destination($projects_archive) and source folder($projects_destination):

./7z.exe a $projects_archive $projects_destination   

And of course you can use any major archiver in a similar manner.

Did you notice './' prefix in front of the '7z.exe'? That is because in PowerShell in order to run any program or execute script by PowerShell, you must prefix program/script name with './', which is completely similar to same action in Linux bash shell (and it is a very good thing :)).

Lastly (after archive was created), I deleted cleaned copy of my projects, using pretty self-explanatory command 'Remove-Item':

Remove-Item -Recurse -Force $projects_destination 

And that successfully accomplished almost all my goals! Now it was the time to make Windows 7 run this script on schedule, so I could kick back and move on with my life.

You can download final script and exclusion file using this link:

• Download export_projects_script.zip

Now was the time for next and final step - schedule this script to run automatically!

How to run script on schedule

To add task in Windows 7, lets open Windows Task Scheduler program. Easy way to find it - just type 'task' in start menu search, and in results click on 'Task Scheduler'.

1) After Task Scheduler is open, click on 'Task Scheduler Library' on the left to expand the list of all current tasks.

2) In the top menu, click 'Action' > 'Create Task...' or right click anywhere on the white space in all tasks list and select 'Create New Task...'.

3) Enter 'Name' of the task and 'Description' (optional), also I suggest selecting 'Run whether user is logged on or not' option, so this task will run even when system was restarted, but not logged on the user's account:

4) Next, select 'Triggers' tab and click 'New...' at the bottom (make sure 'Enabled' checkbox is checked), and select some convenient time to run your script:

5) Now is the time to add an Action that you want Task Scheduler to perform.

It's a little trickier than you might think, as Task Scheduler cannot run PowerShell scripts directly. It can only run a particular program. And it took me quite a while to figure out how to run PowerShell script through Task Scheduler...

The trick is that you can run any PowerShell script by running PowerShell application from standard Windows Command Prompt (or 'Run...' command), by specifying path to script as a parameter:

PowerShell c:\export_projects.ps1 

And here how it looks in Command Prompt:

So, you have to configure Task Scheduler Action in a similar manner. Select 'Action' tab in 'Create Task' window, and click 'New...'.

There, make sure that Action is set to 'Start a program', 'Program/script' is set to only one word 'PowerShell' and path to the PowerShell script is specified as an argument (in 'Add arguments (optional)' field):

To test your new task, right click on it, and select 'Run'. If you refresh tasks list, you can see task's Status will change to 'Running'. And to check if it was successful, scroll right to find a column 'Last Run Result'.

So, that successfully concluded all the goals and made me very happy that I won't have to do the boring backup of my projects ever again (manually), and I can finally concentrate on planetary domination coding. :)

And this is the end of this article. I sincerely hope it was helpful and thank you for reading!

References

• http://www.svnforum.org/threads/31535-TortoiseSVN-export-by-command-line
• http://technet.microsoft.com/en-us/library/bb491035.aspx
• http://technet.microsoft.com/en-us/library/ee176949.aspx
• http://technet.microsoft.com/en-us/library/dd347647.aspx
• http://www.PowerShellpro.com/PowerShell-tutorial-introduction/logic-using-loops/
• http://ss64.com/ps/syntax-stampme.html
• http://technet.microsoft.com/en-us/library/dd347686.aspx

History

• 4/7/2012 - Additional PowerShell basic commands added 
• 3/30/2012 - Initial article published