Custom Tools Explained

• Download source code - 90.4 KB

Introduction

Custom tools aren’t a particularly well-known technology – in fact, they are the ‘barely visible’ players of the Visual Studio infrastructure. This article describes what they are, how they are used, and gives an example for programming your own. Please note that this is a tutorial for beginners, so I won’t be showing any advanced stuff here.

In order to compile/run the examples, you need Visual Studio 2008 with Service Pack 1. Please note that the Service Pack is important, since prior to it coming out, VS2008 developers wishing to write custom tools got stuck in limbo due to some conflicting UUIDs. By the way, VS2005 is fine for custom tool development, though there are tiny differences in the API.

You will also need the Visual Studio 2008 SDK installed.

What is a Custom Tool?

Here are some statements which describe a custom tool:

• It’s a file generator.
• It makes code-behind files.
• It extends Visual Studio.
• It’s stored in a ComVisible DLL file.
• It uses the Registry.

A custom tool is a file generator because its purpose is to generate files from an existing file. The original intent of the tool was to generate just one file, but by writing some custom code, you can generate several. What’s the point? Well, how about generating a data set from an XSD? A custom tool is precisely the mechanism for it. One can think of many more uses for custom tools, such as:

• Transforming an XML source file using an external XSLT transformation
• Getting all the images referenced by an HTML file
• Previewing what an XML-serialized form of a class would look like
• … and many more

How do we tell a file to use a custom tool? It’s simple: select the file in the solution tree, and open the Properties window (press F4). Then, type in the name of the custom tool you want to use. Here’s how it looks:

As soon as you specify the custom tool, it will run using the selected file(s) as input. If you misspelled the name, or if the custom tool is broken, Visual Studio will let you know. Provided everything went well, you’ll end up with some freshly generated files! Where do these files go? Well, they go into the code-behind. In other words, they are one level below the selected item in the solution tree. Here’s an illustration:

As you can see on the above screenshot, code-behind files appear just under the item for which a custom tool was specified (e.g., Neurovisual.xml). They all have the same icon with the blue arrow – I have no idea why and, to my knowledge, there is no way to change this.

Now that we know where the generated files go, a good question is when. Well, the files are (re)generated every time, you save the source file (the file that uses the custom tool). You can also force a re-generation by right-clicking the file and choosing Run Custom Tool:

Voilà! You’ve got your generated code. The way this magic is possible is due to the extensibility API that Visual Studio provides. Specifically, it provides us with an interface – IVsSingleFileGenerator – that a custom tool must implement. However, unfortunately, implementing this interface on a public type and compiling a DLL does not make a custom tool available in Visual Studio – some extra steps need to be taken.

Visual Studio needs to be told about your tool. Since VS uses the Component Object Model (COM) for extensibility, it wants to know the Globally Unique Identifier (GUID) of each of your custom tools. This means four things:

• You must give each custom tool a Guid
• You must register your DLL as a COM component (using regasm.exe)
• You must add the Registry entries so Visual Studio can find your tool
• You must place your custom tool in the GAC

In the next section, we shall go through the process of creating a custom tool.

Basic Example

Let’s make a basic custom tool – one that counts the number of lines in a file and generates a text file with that number. Here are the steps to get the tool working:

1. Create an ordinary Class Library project. Nothing special needs to be set here. I’d stick to .NET 2.0, but feel free to use .NET 3.5 if you wish. So long as you don’t throw the UI out of the custom tools, 3.5 should be fine.

   Note: Some people recommend creating integration packages instead, because they can be debugged. I haven't tested this.

2. Add a reference to the Microsoft.VisualStudio.Shell.Interop assembly. This is the assembly that will help us interact with the Visual Studio shell.
3. Create a new class – let’s call it LineCountGenerator. Get the class to implement the IVsSingleFileGenerator interface. Generate the method stubs. The interface is very simple – you get just two methods: DefaultExtension() and Generate().
4. The DefaultExtension() method wants to know what extension the generated file should have. In our case, we want to make a text file, so we return ".txt". Yes, the dot is necessary.
   C#

   public int DefaultExtension(out string pbstrDefaultExtension)
   {
     pbstrDefaultExtension = ".txt";
     return pbstrDefaultExtension.Length;
   }

5. The Generate() function is a bit trickier. Here is its outline:
   C#

   public int Generate(string wszInputFilePath, string bstrInputFileContents, 
     string wszDefaultNamespace, IntPtr[] rgbOutputFileContents, 
     out uint pcbOutput, IVsGeneratorProgress pGenerateProgress)

   Let’s discuss each of the arguments in turn:

   • wszInputFilePath contains the path of the file for which content is being generated.
   • bstrInputFileContents contains the contents of the input file as a single string; this might seem redundant since we already know the path, but what can I say – it’s convenient, saves one line of code.
   • wszDefaultNamespace contains the name of the default namespace for the current solution or folder; it’s a useful piece of information to have for generating code.
   • rgbOutputFileContenst is a bit tricky. Basically, you need to write the bytes of the generated file into this variable. However, you cannot do it directly (hence the IntPtr[] type) – instead, you must use the System.Runtime.InteropServices.AllocCoTaskMem allocator to create the memory and write type bytes in there. This may sound hard, but it’s not – we’ll see how this is done in a moment.
   • pcbOutout must be set to the number of bytes that we wrote to rgbOutputFileContents.
   • pGenerateProgress is an interface that we can use to tell Visual Studio how long the operation will take. This is only useful if your custom tool does something that takes a long time. We’ll ignore this parameter for our trivial example.

   If everything went well, you need to return VSConstants.S_OK from the function – to get this enumeration value, you’ll need to add a reference to Microsoft.VisualStudio.Shell in your project. Or, you can just return 0 (zero).

6. We are ready to fill the Generate() function. Let’s start by getting the line count:
   C#

   int lineCount = bstrInputFileContents.Split('\n').Length;

   Now, we use the Encoding class to get the bytes to write, as well as how many there are:

   C#

   byte[] bytes = Encoding.UTF8.GetBytes(lineCount.ToString());
   int length = bytes.Length;

   Having acquired the bytes, we need to write them using the COM task allocator:

   C#

   rgbOutputFileContents[0] = Marshal.AllocCoTaskMem(length);
   Marshal.Copy(bytes, 0, rgbOutputFileContents[0], length);

   There is no de-allocation of the memory – Visual Studio will do it for us. All that remains now is to set the number of bytes written, and return S_OK.

   C#

   pcbOutput = (uint)length;
   return VSConstants.S_OK;

We’re not done yet! All we’ve got so far is the tool functionality, we haven’t added COM support yet. Let’s do it now.

1. Our custom tool needs to be given a GUID – a unique identifier. The identifier is basically a string of characters. The easiest way to make it is to run guidgen.exe from the Visual Studio command prompt. If you have ReSharper, you can use the nguid Live Template to make one. Add the GUID to your class file so it looks like this:
   C#

   [Guid("A4F30983-CAD7-454C-BB27-00BCEECF2A67")]
   public class LineCountGenerator : IVsSingleFileGenerator
   {
     ⋮
   }

2. You need to mark the assembly as ComVisible for it to work with COM. Open AssemblyInfo.cs, and set the appropriate attribute to true.
3. To be usable in COM, our types need to be registered. .NET types are registered for COM interop by using the regasm.exe tool from the command line. It’s simple – you just type regasm followed by the name of your assembly to register; use the /u flag to unregister.

   Visual Studio can also handle the registration for you. Just open project properties, and select the Build tab. On the bottom, you’ll see the check box to Register for COM Interop. Check it, and you won’t need to regasm your tools while developing them (you’ll still need regasm if you plan to deploy your custom tool).

   

4. We’re almost done – the penultimate step is adding information about the custom tool to the Registry. You need to add a subkey with the tool’s name to the following Registry path:

   SOFTWARE\Microsoft\VisualStudio\visual_studio_version\Generators\{language_guid}

   There are two variables here:

   • visual_studio_version is the version of VS you want the add-in to work with. Version 8.0 corresponds to VS2005, and 9.0 to VS2008.
   • language_guid determines which language the custom tool is available for. The GUID constants are not present in the Interop assemblies, so I just record the constants in the file. For example, the GUID for C# is fae04ec1-301f-11d3-bf4b-00c04f79efbc. Don’t forget the curly braces!

   Now that we know where to place the subkey, let’s discuss what the subkey should contain. Overall, the subkey should contain the following values:

   • The default value should contain a user-friendly description of the custom tool.
   • CLSID should refer to the GUID we made.
   • GeneratesDesignTimeSource is supposed to indicate whether the source file is available to visual designers. The exact meaning of this is uncertain, however. I’d just set this value to 1 (one).

   The simplest was to associate the data above with the custom tool is to associate it with an attribute, so that our Line Counter custom tool would now look as follows:

   C#

   [Guid("A4F30983-CAD7-454C-BB27-00BCEECF2A67")]
   [CustomTool("LineCountGenerator", "Counts the number of lines in a file.")]
   public class LineCountGenerator : IVsSingleFileGenerator

   The CustomTool class (courtesy of Chris Stephano, see [1]) is a simple attribute class – I will not present it here (it’s in the sample code). The only thing to note is that, unfortunately, it cannot inherit from GuidAttribute, which would have made everything look even more elegant. But now, we have a problem: how to integrate all this wonderful metadata and create Registry entries from it.

5. When registering a .NET assembly for COM interop, we have the option to specify static functions that perform custom steps as types are being registered. What better place to add the custom tools to the Registry? All we have to do is get the metadata from the types and register them. To cut the story short, here are the two functions in all their glory:
   C#

   [ComRegisterFunction]
   public static void RegisterClass(Type t)
   {
     GuidAttribute guidAttribute = getGuidAttribute(t);
     CustomToolAttribute customToolAttribute = getCustomToolAttribute(t);
     using (RegistryKey key = Registry.LocalMachine.CreateSubKey(
       GetKeyName(CSharpCategoryGuid, customToolAttribute.Name)))
     {
       key.SetValue("", customToolAttribute.Description);
       key.SetValue("CLSID", "{" + guidAttribute.Value + "}");
       key.SetValue("GeneratesDesignTimeSource", 1);
     }
   }
   
   [ComUnregisterFunction]
   public static void UnregisterClass(Type t)
   {
     CustomToolAttribute customToolAttribute = getCustomToolAttribute(t);
     Registry.LocalMachine.DeleteSubKey(GetKeyName(
       CSharpCategoryGuid, customToolAttribute.Name), false);
   }

   I won’t go into all the plumbing here – these functions use a couple of extra methods that are just utilities for getting the Registry keys built. The important thing here is that by creating these two functions, we make the custom tool self-register with Visual Studio.

After compilation, there are just two steps remaining. You must register the assembly for COM interop, and place it in the Global Assembly Cache (GAC). The order of these two operations is unimportant.

1. As I mentioned before, COM registration of the tool happens either when you compile (provided you specified the corresponding project option), or when you run regasm.exe. To register the tool, you would call:

   regasm YourCustomTool.dll

   and to unregister, you would call it with the /u switch:

   regasm /u YourCustomTool.dll

2. For VS to see your tool, it needs to be in the GAC when VS starts. Thus, before starting up VS, place your custom tool in the GAC with:

   gacutil /i YourCustomTool.dll

   Assembly removal is done with the /u switch, and you must remember to remove the .dll ending, as the tool wants the assembly display name, not the file name.

   gacutil /u YourCustomTool

   If you have set up your project to COM-register your custom tool automatically, feel free to add the call to gacutil to the post-build step. Please note, however, that you might have to specify the full path to the gacutil.exe program.

3. Important note: VS loads your custom tool into memory when it runs. This means that even if you unregister it, recompile and re-register the new version, VS will not see it immediately. You will need to restart VS to see the changes.

Well, that pretty much covers the steps necessary to get your own custom tool working. Let’s recap the steps needed to produce a custom tool.

• Create an ordinary Library project.
• Add a class that inherits from IVsSingleFileGenerator.
• Implement the functions as described earlier.
• Give the class the Guid and CustomTool attributes.
• Copy over the ComRegister/Unregister functions.
• Mark assembly as ComVisible.
• Register the assembly for COM interop.
• Put the assembly into the GAC.

Multiple Files

One of the problems of the IVsSingleFileGenerator is that it only makes a single code-behind file. Sometimes, we might want to have several. Luckily, a fellow by the name of Adam Langley created a solution [2] to this problem that allows our custom tool to create several files. His example is particularly interesting – he shows a generator that takes an HTML file and adds all the images it refers to as code-behind files. For the sake of completeness, I will describe that solution briefly – feel free to read the original article if you are interested.

Here’s a brief reminder about the class you need to derive your custom tools from to get multi-file generation capability. The class is called VsMultiFileGenerator<T>. This class is an enumerator, and the T generic parameter is defined by your subclass. The type can be anything: this generic parameter is mainly for you to process how you see fit. The most sensible choice is to define it as a string.

The abstract methods you need to override are as follows:

• IEnumerator<T> GetEnumerator() is where you return a sequence of elements that would later turn into files.
• string GetFileName(string element) is the method that is responsible for determining the target file name depending on the elements you yielded from the previous method.
• byte[] GenerateContent(string element) is the method that generates a byte stream based on the element we provided earlier.
• byte[] GenerateSummaryContent() generates content for the default element – the one that IVsSingleFileGenerator expects. I’ll describe it in more detail in just a moment.
• string GetDefaultExtension() returns the default extension for the summary content. This method call and the previous one are, basically, propagations of the Generate() and GetDefaultExtension() methods from IVsSingleFileGenerator.

I promised to explain about the ‘summary content’, so here goes. Basically, the multiple-file generator is a single-file generator that also does extra things (such as, you know, generate additional files). After it does that, however, it is forced to create at least one file the old-fashioned way to satisfy the IVsSingleFileGenerator interface contract. This isn’t always so great – for example, if you are writing an adaptive generator that does not know the types of files it will create until it’s actually executed, you’re in trouble – you’ll end up with an extra file being added to the code-behind (because you must have one with a defined extension). This is a cosmetic problem, though, and does not break functionality in any way. And, if you decide to be clever and supply VS with null data and a length of 0 (zero), you will get an error dialog box. Don’t say I didn’t warn you!

To see an example use of the multi-file generator, you can take a look at the multi-file XSL transformer I wrote [3].

Odds and Ends

There are a few things that need to be mentioned with respect to custom tools.

First, integration with source control doesn’t always work the way you want it to. The generated files do seem to be added to source control normally, but sometimes, you might run into situations where some people will see them and others won’t. I have no clue how the mechanics of this work – I have only seen it in SourceSafe, so I kind of hope that TFS is better at handling them. Anyways, a custom tool for XML->XSL transforms did get used on a commercial project – successfully. Just so you know.

If you are wondering what the difference between a custom tool and just a plain boring VS add-in is, well, there isn’t much! In fact, add-ins are better because they do not generate the spurious ‘summary’ file. The Custom Tool mechanism is primarily designed for 1->1 file transforms that happen almost automatically (when you save, mainly). You can also program identical Save-triggered functionality into an add-in. My advice is to use custom tools for basic transforms (e.g., getting a preview of a class as it is serialized). For anything serious, it’s better to write an add-in.

A custom tool doesn’t have to be specified explicitly for a file. Instead, you can associate it with a file extension. For example, Visual Studio does it for the .tt file extension. This allows any file saved with a .tt extension to be executed by the Text Templating processor (also known as T4). Making your own association is easy – when writing the information to the Registry, instead of making a subkey with the name of the custom tool (e.g., MyGenerator), specify the extension of the files that the custom tool will always be applied to (e.g., .myfile). Don’t forget the dot before the extension itself!

Conclusion

I hope that this article has demonstrated that custom tools aren’t that difficult to program. Sure, there are a few steps that need to be taken, but I’ve described them all, so hopefully there’s nothing preventing you from writing your own custom tool if you need one.

If you liked this article, please vote for it. If you did not, please vote anyways, and let me know what I could have done better. Thanks!

References

1. XSL Transform Code Generator for Visual Studio .NET, Chris Stephano
2. Creating a Custom Tool to Generate Multiple Files in Visual Studio 2005, Adam Langley
3. Multi-file XSL Transformation Custom Tool for Visual Studio, Dmitri Nesteruk

History

• 26^th November, 2008: Initial version