Introduction
In Visual Studio .NET, I missed some useful commands and tools so I started developing macros and add-ins for this development environment. This project started with some special web reports and grew up to a collection of useful tools that help developing and controlling my work and files. The result is this add-in for Visual Studio 2003 that adds commands for comparing files and folders, an explore command and some useful reports for web projects.
After installing, there are new commands in the command window as well as in the context menu of the Solution Explorer. Most of them use a new Tool window to output the results of the commands and to offer new functionalities in the context of the reported items.
New Things in Version 2.1
A bug fixed.
New Things in Version 2.0
Version 2.0 of the WebReports Visual Studio add-in is a minor step in functionality, but a huge step in productivity and usability, and some bugs were fixed.
WebReports now also supports SubVersion based projects.
Search for the (new) annotations in this document.
After explaining the tools themselves I also give some hints on the problems that I had to solve and where I had to find solutions for some tricky problems. You will see that it is easy to use HTML for formatting the output inside a tool window without a lot of GDI programming.
A setup file (MSI package) and the full source codes are available separately.
Diff Tools
There are two different comparison methods implemented. The first is for comparing two files against each other line by line using a well known comparison algorithm, and the other is for comparing the files in two folders so you can detect new, changed and deleted files.
Diff Files
This command reads two text files and compares them line by line. There are multiple possibilities to launch this tool as well as all the other tools, so you can decide how to use it effectively.
- Select two files in the Solution Explorer by selecting the "newer" file first and then by simultaneous pressing the Ctrl-key selecting the second "older" file. With a right-click, you can then open the context menu of the second file and start the Diff tool.
- You can use context menu on a single file in the Solution Explorer. Because the second parameter is missing, you will be prompted to select the second "older" file by using a regular file-open dialog. This helps for comparing against a file, which is not part of your current project.
- You can launch the command by using the Tools menu. In this case you will be prompted for the two files that will be compared.
- You can also start this command using the command window by typing:
WebReports.Diff newFile oldFile
If the filenames contain blanks they must be surrounded by a single or double quote character. If you do not type in the filenames, you also will be prompted by using the regular file-open dialog.
- When using the command to show the differences between two folders, and when two files with the same name are found, there will be a Diff command beside the files that launches this tool.
- Visual Studio .NET also can be started with parameters at the command line level. This sample starts Visual Studio and a Diff command is executed right after the startup:
devenv.exe /command "WebReports.Diff newFile oldFile"
- All the commands that you start with one of those methods are also "recorded" inside the Command window. Here you can use the functionality of the Command window to rerun an already submitted command.
- The Diff Files command also has a "Refresh" symbol to make it easy to re-compare the files, for example, after changing some lines of the source code files.
The result of the compare is reported into a tool-window that is used for all the output of this add-in tool.
- New lines are shown with a light green background using the line numbers of the first selected file.
- Identical lines are shown without a background color using the line numbers of the first selected file.
- Deleted lines are shown with a light red background using the line numbers of the second file in parenthesis.
New Diff options and switches (new)
There is an option when displaying the differences of two files, that spaces between a word and a non-word character are ignored. This is useful for comparing the source code in a lot of programming languages because the two expressions (a = 0) and (a=0) are considered being equal.
To enable or disable options, it is possible to use the new Option command:
WebReports.Option Diff [/t][/s][/c][/w]
- /t - All text lines are trimmed so leading and trailing blanks make no difference.
- /s - All multiple space characters including the tab character in a row are reduced to one space character before comparing.
- /c - All characters are converted to their lowercase equivalence before the comparison is done.
- /w - All characters between words and non-word characters are deleted before the text lines are compared. This makes some programming styles equal: "a = 0" is the same as "a=0".
It is also possible to add these options as a third parameter to the Diff and the MakePatch command.
Extended Diff command (new)
When you start the "File Differences" command on one file only, there are two new implemented ways to compute, what file it should be compared. When two files are selected, those two files will be compared against each other.
When the SVNMode (new) (see below) is set to true and if there is a base version file using the svn folder structure conventions, this file will be used.
When using the BaseVersionFolder option, the WebReport implementation will search for a file with the same name and relative path.
Working with multiple versions
If you do not have multiple versions of a projects source code, you need no tool for comparing files and directory structures. But a soon as you have to maintain a productive version of a project, while working for the next release at a different location, you need them.
When using the WebReport add-in, I suggest using one of the two following scenarios, because I had them in mind when building the add-in.
Working with a base-version in a different folder
When reaching a milestone or having finished a big part of the project, it is a common approach to copy all files of the project to a different location on the local computer to have a backup-copy and to have the chance to go back to this version of the sources and have a look at it. It is also a good idea to keep a copy of the source code when you start extending an existing project. These code-folders are called the "base-version" in the WebReports add-in and in this document.
In these situations, I suggest to set the Baseversionfolder option to the root of this folder structure by entering the following command in the command-window:.
>WebReports.Option BaseVersionFolder "C:\baseSources\WebReports15\"
baseversionfolder = C:\baseSources\WebReports15\
If you have no typo error, the current value of the option is written to the command window.
When using the "File Differences" command on a single selected file, the corresponding file inside the BaseVersionFolder will automatically be chosen to be compared with the "old" file.
When using the "Folder Differences" command on a single selected folder, the corresponding folder inside the BaseVersionFolder will automatically be chosen to be compared with the "old" folder.
This way of using the WebReports add-in is a real time saver.
Working with a project from Subversion
When you get a project out of a Subversion repository, you get a lot of files in addition to the real source code files inside the ".svn" folders that exist in every folder of the project. Inside these folders, there are special files with the extension .svn-base that contain the base version you�ve got from the repository.
These files will be updated by Subversion when using the update command and should never be changed by yourself.
In these projects, you should check that the svnmode option is set to true:
>WebReports.Option svnmode true
svnmode = True
When using the "File Differences" command on a single selected file, the corresponding file inside the .svn\text-base folder will automatically be chosen to be compared with the "old" file.
When using the "Folder Differences" command on a single selected folder, the files inside this folder are compared against the corresponding files inside the .svn\text-base folder. The "old->new" copy command gets the caption "revered" because this is the common name to this action and the "old->new" command is not shown because this may confuse Subversion a lot.
Functionality of the tool window
You can double-click on every line in the output window to open up the corresponding file and jump directly to the clicked line number. Only double-clicks on red lines jump into the "old" file, all other lines will lead you into the "new" file. Using the shortcut, it is easy to copy text out of one file and paste it into another file.
You can select all the text in the tool window using the Ctrl-A key and then copy it into the clipboard. Because the format used in the tool-window is plain HTML, you can the paste this into Word, Excel or other HTML-ready applications. Even pasting into a plain text-editor is possible, but then you will lose all the colors.
The refresh button at the top can be used to re-compare the files.
The back button can be used to return to the previous command, for example, the Diff Folders command.
Diff folders
This command can be invoked the same way as the Diff Files command, except that it works on two folders.
The compare is done by enlisting all files in the two folders in alphabetic order and reporting the differences by calculating MD5 checksums on every file found in both directories.
Within this tool a lot of new functionality was added:
- Deleted files are reported with a light-red background.
- New files are reported with a light-green background.
- Unchanged files are reported in black color and white background.
- Changed files are reported in red color.
- All filenames are reported by using hyperlinks, so it is easy to open the corresponding file in an editor window. Files that exist in both folders are linked using the new folder.
- It is still possible to start a file-diff command to display the details of the difference.
- All the subfolders are also listed using the same color scheme.
- It is possible to start a new folder-diff command for subfolders that exist in both compared folders.
- The new file can be copied over to the old file.
- The old file can be copied over to the new file.
- The new file can be backed-up.
Before executing a copy command, you must confirm the copy because it is too easy to hit a command link by accident. The overwritten file can be found in the Recycle Bin to enable an emergency restore.
After a copy of a file has finished, the current command is repeated to show the effect of the changes.
You can also start this command using the tool window using:
WebReports.DiffFolder newFolder oldFolder
WebReports.MakePatch oldFile newFile [options] [outfile] (new)
This is the new feature I am working on currently. It is not perfect yet, but I want to show it up to get your feedback � if you like.
Patch files are widely known in the open source and the UNIX communities and are internally used in almost all source control systems. With this new command, you can produce a text file by comparing two files that contain only the differences in a syntax called "unified format". I found this format being the most readable but there are other formats out there as well.
With this command and some more work on top of it, I want to produce a text file that summarizes the differences in the source code (for example, aspx files) of a web project since a specific version that resides somewhere outside of the project directory, named the "old" web.
In one file, you can see all you have done to the project. The next important step then is to use this file to replay the changes in another location. Because I use a common known format, this is possible using the patch.exe that can be found on the internet. As with the Diff algorithm these tools are under GNU license or are part of bigger packages. I think a new command for the WebReports add-in should not be so difficult�
The plan behind this is to support more of the offline and distributed work scenario and the specific needs of managing "rolled out" web sites that are not directly accessible.
SubVersion support (new)
Starting with version 2.0, projects under the Subversion Control System are supported in a very special way. The new SVNMode option is set to true by default to support this scenario, but can be set to false if you do not have any subversion folder structures inside your projects.
Every time a command is used that behaves different in the subversion mode, the implementation checks if a ".svn" folder exists in the current folder, and if this is not the case, ignores any subversion specifics. So if you do not have any subversion folder structures, nothing should be different, even if the SVNMode option is set to true.
Hint:
There is an add-in called Ankh available that supports Subversion based Visual Studio projects but fails with web-projects and is really slow on huge projects.
This feature was implemented to support subversion scenarios without using the Ankh add-in. If you use Ankh, I recommend you to evaluate working with the Webreports add-in and Tortoise. Because there is the explore command available, you can easily pop up the Explorer and manage all the Subversion related tasks.
The Explore command
While using Visual Studio for web development you will fid sooner or later that the Solution Explorer has some commands for copying, pasting and moving files around inside a solution, but that they do not work like their counterparts in the regular Explorer. You cannot copy a file stored in a compressed folder (a.k.a. Zip-file) directly into a folder of a web project nor can you copy a file from the solution onto, for example, your desktop.
Instead of re-working all the needed shell functionality (which sounds a hard work to be done), I added a new command in the context-menu of the project node and folders of a project titled "Explore" that opens a regular Explorer window with the underlying folder in the file system.
After working with this add-in for a while, I found that this little extension is the most used feature of it all.
Testing XSLT transformations
If you are developing XML based solutions in .NET (like I do) that use the XSLT technology, you get no out-of-the-box support for testing your XSLT files in the Visual Studio .NET environment.
This add-in can help a lot in this case by bringing a new menu to the context menu of the Solution Explorer as well as a command for the command window.
Testing transformations by using a macro for Visual Studio is also an option and you can find a solution to this approach by Doug Doedens.
When using an add-in, it is possible to bring the functionality better into the user interface by:
- providing a output panel that can display the generated code
- adding menu entries in context menus inside the Solution Explorer so you can have the commands there where you need them (and only on the corresponding files)
- support commands in the command window that you can use again and again while writing XSLT code and debugging further into the solution of your problem.
This new tool is working in the context menu of files with a *.xsl or *.xslt extension.
When starting this command (with only one file selected), you will see a standard file-open dialog where you can select the file containing the data for the transformation.
Tip 1: If you select two files in the Solution Explorer and one of them is a file with a *.xsl or *.xslt extension, the file-open dialog will not appear and a transformation is done with the other file as the data source.
Tip 2: If you develop XSLT functionality that will act on XML from a database server or another programmable data source (WebService�), save some typical samples of this data to local files and start developing on this data before things get compiled. These files can also be used for quality testing after modifications to the XSLT file.
In the command windows, there also is an option available to control how the output of the transformation is displayed. Using:
webreports.xslttransform
'c:\inetpub\wwwroot\small\doctypes.xslt'
'c:\inetpub\wwwroot\small\DocTypes.xml'
The output will display all the generated tags in a readable form and it will appear as-is in a text-editor. When using the HTML option as the third parameter, like:
webreports.xslttransform
'c:\inetpub\wwwroot\small\doctypes.xslt'
'c:\inetpub\wwwroot\small\doctypes.xml' 'html'
the generated text is interpreted as HTML and directly shown in the tool window.
Backup command
A requested feature was to support a controlled shadow copy of files of the current project that has been changed. I talked to some people and they find this command useful too, so I did it.
The Backup command, now available in the newest-file, the folder Diff output, and the context menu of files in the Solution Explorer makes it easy to copy files to another folder.
The first time a backup command is launched, you get prompted to specify where the file and all further selected files should be copied to.
When copying files, the folder structure from the project will be respected and new sub-folders in the backup folder will be created when needed.
If you have to change the backup folder, it is possible to launch a command via the command window and specify a new backup folder as the second parameter.
Web Analysis Tools (Reports)
I had FrontPage on my system some months (now years) ago for managing some web sites. What I really miss in Visual Studio since then (but no more now) are the reports that where built into that tool. Especially the "Report newest files" gave me a useful list of the files I touched during a bug fixing session or a development day.
Report Newest Files
It is not so hard to get all file modification information by walking recursively through the folders of a project. This implementation reports the top 50 newest files anywhere beyond the project-folder.
Beside the filenames, the date of the last modification is shown and a backup command is available.
Folders with a name starting with an underline character are not analyzed. This is because of a naming convention that exists in FrontPage Webs and the Indexing Service that I have adapted in my work. I only store private and internal information into those folders and avoid links to files inside of them.
With version 1.5, it is possible to launch this tool on any folder when using the command window:
webreports.newest 'c:\inetpub\wwwroot'
Report Oldest Files
This report is just the other end of the list. It was very easy to implement, because the previous port was already done. I hope anyone finds this useful too, for example, while doing a clean-up of an older installation.
With version 1.5, it is possible to launch this tool on any folder when using the command window:
webreports.newest 'c:\inetpub\wwwroot'
Report File Types
This report also walks through all the files and folders of the current project and reports the number of files and their size grouped by the extension.
In addition to the simple list of all found file types, their counter, and sum of the sizes, this tool now also displays two pie charts to illustrate those numbers.
Because the list of the file types may be very long, only the nine most used types are displayed and all the other types are summed up to build one last category.
If you have installed the SVG Viewer 3.0 from Adobe, you can also see the two pie charts that show the relative amount of the most used extensions by occurrence and by file size.
WebReports.Option [name] [value] (new)
Starting with version 2.0, there is now the possibility to define, store and load Options.
Every time a solution is loaded, the options for the WebReport commands are loaded from the solution file too. Storing and loading variables in the solution file is possible by using the Solution.Globals object. If you define a backup folder or a directory containing the latest published version (base-version), you do not have to redefine this folder when you load the solution the next time. I think that it is a good advisory not to port the solution files between computers.
With the Option command, it is possible to view and enter options that change the behavior of some implemented functions. You must enter this command in the Command window, because there is no dialog implemented.
- When giving no parameters at all, the known options are reported to the command window.
- When giving the name parameter only, the parameter with this name is reported to the command window.
- When giving the name and a value parameter, the parameter is set to the new value.
There is actually no Property Page for the Visual Studio Options dialog to enter all the options available in the WebReports add-in.
Using the Commands
Using the Commands in the Command Window
All the commands of this add-in can also be invoked using the command window of Visual Studio .NET.
This is very useful, if you have to compare the same file over and over again and cannot use the refresh command.
Using the command line syntax of Visual Studio
It is possible to launch Visual Studio .NET on the command line and add some commands that are started automatically. Before using this feature, you must allow the command line usage using the Add-in Manager in the Tools menu. Because whitespace characters are used in parameters like folder names, the whole command, including all the parameters, must be surrounded with double-quotes. The parameters of the command must be quoted using single-quote characters.
Sample:
devenv /command "webreports.difffolder
'C:\src\WebReports' 'C:\backup files\WebReports040503'"
Annotations, developing and more
Building an add-in
Writing an add-in is really complicated when you want to realize a rich output experience and some click-functionality. The solution I found was to use the IE browser control inside a tool window and render the results in HTML.
Before thinking about implementing a special tool window for your own tools, you may consider using plain text output windows. This kind of an output window can be used without a lot of implementing but it can be used for writing plain text only. To jump across this limitation, you have to develop an ActiveX control that can be registered inside the development environment.
I knew that IE could also be used as an ActiveX control, so instead of developing my own GDI painting routines, I reused the HTML rendering functionality of IE. Using that trick, it was possible to produce the output I wanted by using HTML tags and CSS attributes.
If you want to implement your own add-in, you may find some hints in the available source code files. Some helpful links are listed below.
Building the Projects
In the source-files, you can find three projects:
VSIcons
is a C++ project that compiles a DLL with the icons that are used in the command-bars.
WebReports
is the main project for the add-in.
WebReportsSetup
is the setup-project for the add-in.
If you like to extend and compile this project yourself, I recommend, that you setup install the built MSI file once. This will create a folder under program files, the three files are copied there in a usable structure and the registry keys are added.
Then save (export) you registry keys into the file SetupEntries.reg so you can reinstall the add-in very quickly.
When you build the WebReports project again, the COM registration of the add-in will point to the bin folder of the WebReports project so the copied DLL will not be used any more.
When you build the VSIcons project again (with C++), you have to copy the DLL into the 1033 folder, so the VS environment can detect the icons inside.
Using Hyperlinks
The core topic I had to discover myself was a way to invoke Visual Studio commands by clicking into the tool window with the reports, and I found the following trick:
The browser control I embed inside the tool window has an event named OnBeforeNavigate that is fired every time a hyperlink is clicked or the content of the displayed page is refreshed. With this event, there is the possibility to cancel the default action (navigating away) and doing some Visual Studio related commands instead. I implemented the following "protocols":
dte:file?file=filename[&line=linenumber]
Open the filename in the editor window and an optional jump to line.
dte:folder?folder=foldername
Open the Explorer window with the folder.
dte:diff?old=oldfile&new=newfile
Start a compare of the two files and display it in the tool window.
dte:difffolder?old=oldfolder&new=newfolder
Start a compare of the two folders and display it in the tool window.
dte:back
Pop the current command and repeat the previous command.
dte:reload
Repeat the current command.
dte:transform&xml=xmlfile&xslt=xsltfile[&html=true]
Transform the XML file using the XSLT file, optionally the output is shown in HTML view.
dte:copy?src=srcfile&dest=destfile
Copy the source file to the destination file.
dte:backup?file=filename
Backup a file.
Since version 1.5, there are more dte:... links used internally. You can see them in the status bar when hovering over hyperlinks.
Useful Macro
When developing your own add-ins, it is possible to register commands and attach them to the context menus in the Solution Explorer.
There are a lot of command-bars registered so it is hard to find the right ones:
Item
This command-bar is used as the context menu in the Solution Explorer when you right-click on files.
Folder
This command-bar is used as the context menu in the Solution Explorer when you right-click on folders.
Project and Project Node
These command-bars are used as the context menu in different forms of projects.
When you play around with adding commands, you will from time-to-time use a macro like this to delete all the commands:
Sub WebReport_RemoveContextCommands()
On Error Resume Next
DTE.CommandBars("Item").Controls("File Differences").Delete()
DTE.CommandBars("Item").Controls("Xslt Transformation").Delete()
DTE.CommandBars("Item").Controls("Backup").Delete()
DTE.CommandBars("Folder").Controls("Folder Differences").Delete()
DTE.CommandBars("Folder").Controls("Explore").Delete()
DTE.CommandBars("Project").Controls("Explore").Delete()
DTE.CommandBars("Project Node").Controls("Explore").Delete()
End Sub
You might have to uncheck all checkboxes for WebReports in the Tools >> Add-In Manager dialog, restart VS.NET, check all checkboxes, and restart the VS Environment again to get all the functions registered properly.
Also very useful is the ReCreateCommands.reg file, that can be used to set a hint into the registry so that all commands will be recreated next time VS starts. Because only the actual commands will be recreated, all the changed and deleted commands will stay registered, so you will use the macro above.
I also saved all the registry entries into the SetupEntries.reg file, so you do not have to install the control, after an unsuccessful start of the VS Environment. Installing these registry settings is the main purpose of the setup project (besides copying three files).
Some words on a good Diff algorithm
It was hard to find a good and reliable Diff algorithm implementation. There are some pretty experimental and also some heuristic versions around in the Open Software community, but none of them were based on .NET.
There exists one publication of a perfect algorithm that can find a list of minimal changes. This algorithm is used by a lot of (almost all) Diff and patch tools but is written under the GNU license and therefore cannot be used in commercial environments. In this, I followed the original academic publication and did a reimplementation of the algorithm in C#. The algorithm was first published on www.gotdotnet.com.
Some work to be done
This may be a useful tool right now, but some work can still be done:
- Some options are hard-coded inside the source files right now. An Option dialog can help here.
- More reports for HTML development (unlinked files, unresolved links, ...).
- Adding a hyperlink-syntax for a single difference in DiffFiles so they can be used to copy over files from one file into the other.
- Better icons :-).
If you have some more ideas for this add-in, found bugs, anything else, just post it.
Still open quests (anyone may help ! ? !)
- In the command window, it is possible to use an intellisense popup to get command completion. The built-in commands also have this feature on the arguments they provide, for example, the File.Open command needs a filename too and you can select it out of a list where all filenames of the current project are enlisted. How can this be implemented?
- I use a special syntax inside the hyperlinks. But instead of implementing a moniker and registering a new special protocol, I capture the event only inside the tool menu. Right now it is not possible to use these hyperlinks outside of the special tool window. How is it possible to implement a moniker in C#? I found no samples or hints.
- Do I really need the C++ project for including some icons?
Resolved Quests
- Right now, I write out a text-file to the temp-folder and then direct IE to load this file. There is a hint in the MSDN documentation about a stream-interface for IE that can be used for loading documents. How can this be used from within managed code?
This quest was solved using the IHTMLDocument2 interface instead of IHTMLDocument. Here the document has a "write" method where the HTML content can be passed as a string. Because the characters are passed directly (as 16-bit values) there is no encoding problem any more.
Links
History
Changes from 1.5 -> 2.0 (new)
See the (new) annotations to find the big new things.
The MenuBar in the tool window is always displayed. In all Reports (except the fileByType report), the menu bar on top of the output window is now always displayed even if you scroll down to the end. It is very easy to refresh the current view by repeating the command using the refresh symbol. Also the scroll position of the window is restored after displaying the new content.
When the current command is repeated using the reload command that is available in the menu bar, the current scroll position is restored after the new output is shown.
The Diff commands are also included in the context menu when you select files of two different projects or the "Miscellaneous Files" project.
In the report of the differences of two folders, the links to analyse the subfolders were broken.
When a command is used, it is reported only to the command window when it differs from the last command in this window. In version 1.5, commands were reported twice in some situations.
The source code has more comments and the XML documentations are improved.
The internal classes where marked public and are now marked internal so no COM GUIDs are registered for these classes.
Changes from 1.2 -> 1.5
See the (new) annotations to find the big new things.
All commands are written into the command window with all their parameters. Now it is simple to restart comparing two files or two folders.
The output of the file type report is now sorted by the size of all files in descending order. This makes the list a bit more useful.
Bug found: An Array out of bounds error in the Diff class was eliminated.
I find it better to use an abstract base class than an interface definition in this case. Because I had to add another method to the IReport interface definition, I had to go through all my classes and add a dummy method in there too. It is much easier to add this function to an abstract base class. It really makes no big difference in the implementation of the reports.
Changes from 1.0 -> 1.2
The documentation was re-written and pictures were added so you have a better idea of what I am writing about.
All the hyperlinks changed their syntax so they better adapt to the common syntax used in the internet.
DiffFiles command
The order of the two files has changed. Normally, you work on the newer file, and because this file is already selected in this case, you only have to select the second, older file then.
In the hyperlink- and command syntax for this command, the order has changed too to be symmetrical in all cases.
The Refresh command was added in the output window.
The Explore command also works now in the context-menu of the project node and opens the root-folder of the project.
