If you like this tool, support it by voting, if you don't like it, make your vote verbose...
Pre-Introduction
The XsdTidy tool has been entirely rebuild from scratch using CodeDom which is much easier to handle than Emit. The new version is called Refly and available at the Refly article.
Introduction
XsdTidy is a refactoring tool to overcome some silly limitations of the exceptional Xsd.exe (see [1]) tool provided with the .NET framework. More specifically, XsdTidy addresses the following problems:
- Name normalization: if your XSD schema is using lower case names or more generally non ".NET" normalized names, you will end up with types that will make the FxCop (see [2]) spit out hundreds of infractions.
- Fixed Array Sizes: xsd.exe handles multiple elements by creating an array. There is no problem when you are loading the data, but unfortunately this is not convenient if you want to populate a document since arrays do not support
Add
or Remove
. XsdTidy uses ArrayList
for more flexibility.
- Default Constructor: Xsd.exe does not care about providing a default constructor that initializes the fields with the proper values. This work can become very silly when the object structure is getting big.
XsdTidy achieves refactoring by recreating new classes for each type exported by the Xsd.exe tool using the System.Reflection.Emit
namespace. It also takes care of "transferring" the Xml.Serialization
attributes to the factored classes. Hence, the factored classes are more .NET-ish and still outputs the same XML. Moreover, there is no dependency between the refactored code and the original code.
As a nice application of the tool, a full .NET wrapper of the DocBook schema (see [3]) is provided with the project. This .NET wrapper lets write or generate DocBook XML easily with the help of Intellisense.
Fixing problems
Name conversion
The .NET standards define specific naming convention for all types of data: arguments should be camel case, function names capitalized, etc... This is really helpful to keep the framework consistent. Tools like FxCop help us stay on the "normalized" side.
This problem is tackled the dumb way: given a dictionary of "common" words, the class NameConformer
tries to split a name in separate words, after that it renders it to the needed convention.
There is much room for improvement on the list of words and the algorithm to split the name, any contribution welcome.
FixedArraySize
Arrays are replaced by System.Collection.ArrayList
which are much more flexible. Moreover, array fields are created by default using their default constructor. This is to economize you the hassle of creating a collection before using it.
Properties
Fields are hidden in properties, which is more convenient to use. Moreover, collection fields do not have set property according to FxCop rule.
public class testclass
{
[XmlElement("values",typeof(int)]
public int[] values;
}
becomes:
public class TestClass
{
private ArrayList values;
[XmlElement("values",typeof(int)]
public ArrayList Values
{
get
{
return this.values;
}
}
}
System.Reflection.Emit
The System.Reflection.Emit
namespace is truly and amazingly powerful, it enables you to create new types at runtime and execute them or store them to assemblies for further use. Unfortunately, there are not much tutorials and examples on this advanced topic. In this chapter, I will try to explain my limited understanding of this tool.
What is Emit?
The Emit
namespace gives you the tools to write IL (Interpreted Language) instructions and compile them to types. Hence, you can basically do anything with Emit. A typical emit code will look like this:
ILGenerator il = ...;
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Stfld, fb);
il.Emit(OpCodes.Ret);
this.fb = value;
If you are a newcomer, it can look cryptic, but we'll try to explain the above a bit.
Where to start ?
The problem with Emit
is that debugging is complicated: if you generate wrong IL code, the framework will not execute it, throwing an error without giving any clue.. Moreover, you usually don't have the time to learn the dozens of the code that are part of the OpCodes
class. Therefore, it would be nice to always have some "model" IL and then try to implement it with Emit
.
Hopefully, creating this model is easy! It is possible, using decompilers such as Reflector (see [4]), to read the IL code of any .NET assembly. The idea is simple: open a dummy project where you create the model class that needs to be factored, compile and use a decompiler to read the IL of your model and there you go...you have IL code!
Emit, the basics
I will cover some very basic facts about using Emit
. As mentioned above, the most efficient way to learn is to work with a dummy project and Reflector on the side. We will see here how to make a basic C# statement in some instance method where value
is the first argument and field
is a member of the class.
if (value==null)
throw new ArgumentNullException("value");
this.field=value;
Getting a ILGenerator
Usually, you start by creating an AssemblyBuilder
, then a ModuleBuilder
, then a TypeBuilder
and finally you can add methods to the TypeBuilder
using TypeBuilder.DefineMethod
which returns a MethodBuilder
. This instance is then used to retrieve an ILGenerator
object which we use to output IL code:
MethodBuilder mb = ...;
ILGenerator il = mb.GetGenerator();
OpCodes
The OpCodes
class contains all the IL operations. It has to be used in conjunction with ILGenerator.Emit
as we will see in the following.
Arguments
Each time you call a method (static or non-static), the method arguments are accessible through OpCodes.Ldarg_0
, OpCodes_1
, ... In an instance method, OpCodes.Ldarg_0
is the "this
" address.
Labels
Label
s are used to make jumps in the IL code. You need to set up Label
s if you want to build instructions such as if
...else
.... A Label
is defined as follows:
Label isTrue = il.DefineLabel();
Once the Label
is defined, it can be used in an instruction that makes a jump. When you reach the instruction that the Label
should mark, call MarkLabel
:
il.MarkLabel(isTrue);
Comparing values to null
Comparing a value to null is done using the OpCodes.Brtrue_S
. This instruction makes a jump to a Label
if the value is not null.
Label isTrue = il.DefineLabel();
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Brtrue_S,isTrue);
...
il.MarkLabel(isTrue);
...
Creating objects
To create object, you must first retrieve the ContructorInfo
of the type, push the constructor arguments on the stack and call the constructor using OpCodes.NewObj
. If we use the default constructor of ArgumentNullException
, we have:
ConstructorInfo ci =
typeof(ArgumentNullException).GetConstructor(Type.EmptyTypes);
Label isTrue = il.DefineLabel();
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Brtrue_S,isTrue);
il.Emit(OpCodes.NewObj,ci);
il.Emit(OpCodes.Throw);
il.MarkLabel(isTrue);
...
You can clearly see the "jump across the exception" with the label isTrue
.
Assigning fields
The last step is to assign the field with the value (stored in the first argument). To do so, we need to push the "this
" address on the stack (OpCodes.Ldarg_0
), push the first argument (OpCodes.Ldarg_1
) and use OpCodes.Stdfld
:
FieldInfo fi = t.GetField("field");
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Stdfld,fi);
Finish the work
To close a method, use OpCodes.Ret
:
il.Emit(OpCodes.Ret);
Refactoring
Main steps
The refactoring is handled by the XsdWrappedGenerator
class. The main factoring steps are:
- Create an
AssemblyBuilder
and define a new ModuleBuilder
.
- For each user-provided type that needs to be refactored, define a new
TypeBuilder
in the ModuleBuilder
.
- For public fields in the source type, generate a field in the refactored type.
- Define default constructors for each factored class.
- Add properties for each field in the factored types and copy the XML serialization attributes to the properties.
During the process of factoring, special care is taken about nullable/non nullable types and collection handling:
- collections are preallocated for easier use,
- non-nullable fields are allocated, nullable fields are left to null.
- non-nullable fields are always checked against zero, while nullable fields are not checked.
Once the factoring is finished, the types are created and saved to an assembly.
Using XsdWrappedGenerator
The XsdWrappedGenerator encapsulates all the "wrapping" functionalities: create a new instance, add the types you need to be refactored and save the result to a file:
XsdWrapperGenerator gen = new XsdWrapperGenerator(
"CodeWrapper",
new Version(1.0),
);
gen.AddClass( typeof(myclass) );
...
gen.WrapClasses();
gen.Save();
The name passed to the constructor is used as default namespace and output assembly name.
Using the Command Line application
XsdWrapperGenerator comes with a minimal console application that loads an assembly, searches to types, refactors them and output the results. Calling convention is as follows:
XsdTidy.Cons.exe AssemblyName WrappedClassNamespace OutputNamespace Version
where
AssemblyName
is the name of the assembly to scan (without .dll)
WrappedClassNamespace
is the namespace from which the types are extracted
OutputNamespace
is the factored namespace
Version
is the version number: major.minor.build.revision
NDocBook
DocBook is an XML standard to describe a document. It is a very powerful tool since the same XML source can be rendered in almost all possible output formats: HTML, CHM, PDF, etc... This richness comes to a price: DocBook is complicated for the beginner and it tends to be XML-ish.
This was the starting of the article for me: I needed to generate DocBook XML to automatically generate code in GUnit (see [5]) but I wanted to take advantage of VS intellisense.
The first step was to generate the .NET classes mapping the DocBook schema using the Xsd.exe tool. The generated code had some problems that would make it unusable: non-nullable fields where not initialized automatically and this would lead to a lot of manual work.
Hence, the second was to write XsdTidy and apply it to DocBook. So here's an example of use:
Book b = new Book();
b.Title = new Title();
b.Title.Text.Add("My first book");
b.Subtitle = new Subtitle();
b.Subtitle.Text.Add("A subtitle");
Toc toc = new Toc();
b.Items.Add(toc);
toc.Title = new Title();
toc.Title.Text.Add("What a Toc!");
Part part = new Part();
b.Items.Add(part);
part.Title = new Title();
part.Title.Text.Add("My first page");
using (StreamWriter sw = new StreamWriter("mybook.xml"))
{
XmlTextWriter writer = new XmlTextWriter(sw);
writer.Formatting = Formatting.Indented;
XmlSerializer ser = new XmlSerializer(typeof(Book));
ser.Serialize(writer,b);
}
The output of this snippet looks like this:
="1.0" ="utf-8"
<book xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<title>My first book</title>
<subtitle>A subtitle</subtitle>
<toc>
<title>What a Toc!</title>
</toc>
<part>
<title>My first page</title>
</part>
</book>
Now, with Intellisense on our side, I am much more comfortable with DocBook...
Conclusion
System.Reflection.Emit
is a powerful tool that deservers more attention than it currently has. It can be used to generate optimized parsers (like Regex is doing), runtime typed DataSet
s, etc...
History
- 20/2/2004, initial try-out.
References
- XSD Schema Definition Tool
- FxCop
- DocBook
- Reflector
- GUnit