Introduction
What is a POC? It is a Proof of Concept (POC) or, simply put, the attempt to turn an idea into something tangible.
For software developers, this usually means there is a new library or tool available that looks like it will make your life easier. Or perhaps your boss/lead architect tells you that some new feature must be created that no one has built before. With the understanding that you probably don’t have a specification document for the feature, haven’t seen the library, or don’t even know how to spell it, you set off on your next adventure.
Generally, this is the point at which the new widget being developed is doomed, unless you head the person driving this requirement off at the pass. Tell your supervisor that you need a specified number of days to determine the feasibility of what you are being asked to do. Calmly explain that you will create a new project using best practices only where necessary. This will determine pitfalls and pain points and will, ultimately, provide an insight on whether the project will be a success or failure.
Peers and Peer Review
With any luck, nobody will expect you to be a miracle worker and your peers will be available to help you in a crunch. Just be sure not to rely on them too heavily or it will appear that you are expecting them to do your work. That will only serve to reflect poorly on you.
Peer reviews are important in good software development but in this particular instance, critiquing of best practices should generally be ignored. For the duration of this prototype, best practices should be kept at a minimum as they generally are time consuming and provide no benefit to the issue at hand. The focus needs to be on feasibility, reliability, reproducibility, scalability and any other -ibility that I have left out as well as helping to pinpoint potential issues. Remember your peers may be helping you develop this widget or at least helping to maintain it in the future and if you try to pull the wool over their eyes, you may regret it later.
Documentation
At the end of the allotted time, you need to have a document that is either painstakingly beautiful or written in Notepad containing an explanation of what you have been doing. This is a must. If you just give a verbal presentation of your POC to your boss, he or she might question what you have really been doing and will likely never let you do one again or worse – proceed blindly ahead. A verbal presentation may also result in the loss of critical information.
To make this process easier, keep Notepad open and jot down questions/concerns as you think of them. Note all problems and solutions. This is to aid you later, so don’t write notes so cryptically that you can’t understand them and not in so much detail that you lose your focus. Remember, you’re supposed to be creating a prototype.
The document you produce will provide a platform to stand on and will determine whether or not to proceed or, in some instances, request additional time. It can also be used as a C.Y.A. if the powers that be decide to go forward and things go terribly wrong.
Background
So why am I telling you things that you probably already know? Perhaps it will become clearer if I relay my story of what went terribly wrong on a project I worked on.
A couple months before I started on the project, the powers that be decided that a new .NET web service was needed in order to provide future expansion. The task was given to a senior developer with more than enough experience to accomplish the task. He decided that WCF would be used for the web service and Entity Framework would be used to provide access to the database. Somewhere along the way, a decision was made to create new DTO objects that would be exposed through the web service.
To aid in the creation of the new objects, a separate project was developed to auto generate the classes. It was designed to consume the XML-based EDMX file from EF and produce the classes with additional CRUD methods. A set of classes was created for the server side as well as for the client side. The client side classes were meant to be used in house and provide additional functionality that would be equivalent to the EF with regard to caching. As with most code generation, there comes a point where things don't go according to plan. In this case, it entailed modifying the generated code by hand in order to get it to compile. The hand modifications had to be done every time that a change was made to the generator and the files needed to be regenerated. I don’t believe I need to say anything more about that.
After the project was well underway, I was given my first assignment, to produce some SDK samples to show how to use the new web service. This seemed like an easy enough task to cut my teeth on. To my horror, Visual Studio was unable to generate the proxy objects. There were some customizations done to the WCF service that caused it not to work anymore. After endless hours and lots of trial and error, the problem was found in the custom security implementation and a fix was provided. Back to the task at hand, I was able to create the proxy objects and started writing some basic CRUD examples. I quickly found that the methods I would most likely expect to search with were not available. Further, I was told that I would need to write some custom methods to go in the web service as a partial class and other methods could be added to the generated code. After the new additions were made to the generator, the files needed to be hand edited yet again.
In the end, there were about 50 classes with approximately 10 generated methods per class and there are still plenty of methods missing. Of those 10 methods per class, most required that I continuously request the data from the web service as it would not allow me to retrieve the grandchildren at the same time as the children or parent.
After a period of time, I was given my next assignment – to integrate our new web service with another company’s product, using their framework. Great, it looked like a nice slow pitch over home plate; boy was I wrong. The product and framework were written in Java. Being the second time I had to do such a horrific task, I had an idea what needed to be done and started by slowly banging my head on the wall to sufficiently numb it for what lay ahead. I had the usual sorts of issues with data types, especially date/time fields as well as proxy creation issues. These paled by comparison with an issue I found and was told that I was required to fix.
The issue is that Java requires that the WSDL of the web service be available when instantiating a client proxy. Why? I’m not sure but it is required none-the-less. The framework from the other company was not able to retain a reference to the client proxy and re-use it each time. This meant that the client proxy would have to download the WSDL from IIS again. IIS, by default, doesn’t retain the WSDL any longer than it needs to and will trash it after a period of time. This generally is not a problem but in this case it was taking IIS on average 5 minutes to create the WSDL. Yes, that is correct, 5 minutes. Like most people, if something is taking more than 10 seconds, I consider it broken.
Lessons Learned
This section will give a brief overview, albeit incomplete, of some of the lessons that I learned.
POCs
If we examine the story above, the creation of several POCs would have identified several areas of concern.
Hand editing: The better solution may have been the creation of the classes and then dropping any future regeneration. This is commonly referred to as the 90% rule. If I can do 90% of the mundane task, then I only need to do the 10% custom by hand, thus saving a considerable amount of time. The hand modifications were also only in the head of the developer.
Examining the generated code, 50% of it wasn’t warranted because it involved lookup values and CRUD wasn’t needed. The elimination of these methods or even classes would significantly decrease the WSDL re-generation time from IIS.
WCF and Java don’t always play nicely together. An older more proven technology like ASMX might have been a better choice.
Trying to re-create the functionality of another library, such as caching, will lead to endless hours of pain and frustration. If you really need it, just build what will meet your immediate needs and don’t try to reproduce something that took someone else months to develop.
Peers
At the beginning of the article, I mentioned that your peers need to be involved. Looking at some of the issues on this specific project, peers might have spoken up about what was being developed. This, of course, would only work if you had a coherent team as opposed to lone wolves or developers who were silo’d and didn’t understand what was being developed. Involve them at the very least before presenting your findings to the person requesting the feature.
Specifications
Understanding the business needs and the dream of where the product is going can give insight into how to develop a project. Understanding that a web service is needed to grow the business is important but knowing that our only foreseeable client of this is based on Java is more important. As the web service was only being used as a way to get data into the system quickly without any real business logic, other Java-based technologies should have been considered. The possibility of having 2 web services could also have been discussed.
My POC
When I started working on the project and was seeing all the issues at hand, I decided to create a POC on my off time with the intent of sharing it and generating some discussion on how we might want to proceed. Within the next few days, events happened that made me re-evaluate sharing my ideas. I knew without a doubt that they were not wanted and would generate backlash. This should be a consideration for you as well as whether or not to request a POC; it isn’t worth your job so use your best judgment.
Armed with the knowledge I learned over those couple of days, and knowing that I some of the high level concepts could be applied to other real world scenarios; I decided to continue on with the POC for my own personal knowledge. My POC continued far past what I would normally have produced. I also incorporated some ideas that I had been hearing, reading about or had an interest in learning. Lastly, I cleaned up the code to make it worthy of publication.
I set my goals with the following in mind in order to provide the same environment as what was being created on the original project.
- Auto-generation of new business objects but without resorting to parsing the XML of the EDMX file
- Availability of descendants, if requested
- Flexible data transformation between the DTO and EDMX objects
- Improved functionality for searching
- Decreased time for IIS to create the WSDL
The following were not going to be considered:
- Client side caching
- Interoperability
- Performance by way of metrics testing
Points of Interest
EntityServiceTests contains LambdaHelper – This class is used in PurchaseOrderHeaderTests.cs and provides a way of getting rid of strings often used for class property names. Lambda expressions are also known as a fluent technology. - Generated classes provide references to other classes, the ability to create a key representing the object, and to find a method that doesn’t restrict what to search on.
AdventureWorksEntityModel – is a separate DLL used to communicate with the database by means of the Entity Framework. It contains EntityObject.cs which provides the basic CRUD operations for any type of object as well as the ability to search on any field and retrieve associated data. - The web service contains only the 4 methods for CRUD. This will aid in the regeneration of the WSDL.
ServerRepository is based on generics and will allow for future classes to work. AdventureWorksModel contains a folder for mapping. These classes are responsible for converting the data between the EF model and the custom model by use of an open source project, AutoMapper. I should note that I am using this project in its most basic form. Refer to the site for additional information. - "Parsing of the Entity Framework EDMX file" is performed using a library that I created on top of the Microsoft framework.
- Code generation is done using the Microsoft T4 technology that is baked into Visual Studio 2008/2010 with an additional library “T4 Toolbox”. Additional information can be found at these sites:
Using the Code
After you install the T4 Toolbox, open the solution, build the EdmxLibrary and then modify the path to the EdmxLibrary.dll found within the Driver.tt file in the “Code Generation” project. Whenever you save the Driver.tt file or right click on it and select “Run Custom Tool”, the generator will take over and produce all the model and mapping classes. These files can then be copied over to the AdventureWorksModel for use. In the event that you don’t want to install the T4 Toolbox, I left the generated code in the directory for your use.
It’s possible to have the files output directly to the project but I have opted not to do this. The T4 Toolkit will delete anything that it creates before creating the new files. This will create a problem in the event that you place custom code into the shell of the class. If you want this capability, then I would suggest creating the shell manually as required. Also, after generating the files Visual Studio retains a lock on the EdmxLibrary.dll thus preventing it from being compiled again. For this reason, I have opted to remove it from the build process by default in the “Configuration Manager”.
At this point, you can build the solution, run the unit test cases provided and step through the code to see the points of interest in action.
I should point out that the project started a couple of years back and I have since forgotten some of what I produced. I did update it to Visual Studio 2010, moved tests to Microsoft Test instead of NUnit and verified things just generally worked. I thought that the code was an integral part of the article that I had been meaning to write for quite some time.
Final Thoughts
Hopefully, I’ve provided some insight into the advantages of doing a Proof of Concept as well as to show some practical examples in a variety of different technologies and opened your mind to exploring new alternatives to building software? By exploring and building, I believe that you can become a better developer.
Hope your next adventure is a pleasant one.
