Non-Stopping Upgradable Service Framework

• Download demo - 2 MB

Introduction

As back-end service, we might need to upgrade it many times to meet different requirements, maybe requirements changed, maybe issue found and resolved, maybe business logic changed. All these kinds of things are required to upgrade a service. Then if we can upgrade a service without stop running, it should be a better idea. This framework is designed for upgrading a service without stop running, which supports upgrade on the fly.

Benefits

1. Support upgrade services without stopping it. Usually, an upgrade process may contain: backup program data, stop/uninstall old version, install/start or upgrade new version. It's a long and large process compared to upgrade components directly without stop services.
2. Flexible to extend components. In this framework, support from 0 to numbers of components in this framework, it's easy to define relationship between components.
3. Minimize upgrade process, easy to maintain, which also can avoid many mistakes by manually.
4. Reduce outage time if services running in 24*7 mode. In some areas, services can't stop at all, even you may have backup services, stopped some services may cause others services under high pressure.

Background

Actually, this idea came from my real project, we implemented service running back-end, which designed to collect news and distribute news to downstream. We faced many times regarding to requirements change, business logic change or dependencies changed. And every time we upgrade service, we have to follow a long and hard process to do it, e.g., active some services in some location, inactive some services in another location... upgrade some services... etc., finally, we need to restore all services status back to before upgrade. It's a boring job and you have to following the process carefully to avoid mistakes.

Demo

Steps to demo:

1. Download from the above link, and unzip it to some location, e.g., X:\CodeProject.com, the following steps assume location to X:\CodeProject.com\Non-Stopping-Upgadable_Service_Framework_Demo
2. Open a CMD window and locate to X:\CodeProject.com\Non-Stopping-Upgadable_Service_Framework_Demo\bin, which contains demo program: DemoServiceProgram.exe and component: ClassLibrary1.dll
3. Start DemoServiceProgram.exe, and you will see the content like left windows in the above image. In demo program, we have three components: reader1, processor1, dispatcher1 the reader1 will create a object Created MyObject in ClassLibrary1. No.XX, this object mocked as read object from somewhere, processor1 mocks some business, and finally the object goes to dispatcher1, which may dispatch object to downstream or save it to local file or database.

   

4. Open another CMD window and locate to <span style="FONT-FAMILY: 'Courier New', Courier, mono; COLOR: rgb(153,0,0); FONT-SIZE: 15px">X:\CodeProject.com\Non-Stopping-Upgadable_Service_Framework_Demo\bin</span> , you will see two batch files, _UpdateToClassLibrary1.bat and _UpdateToClassLibrary2.bat. Please run _UpdateToClassLibrary2.bat, and you will see the DemoServiceProgram detected components changed, and re-created components. Then, the DemoServiceProgram starts doing business by ClassLibrary2.dll.

   

   

   

5. These (2 bat files) are shortcuts for simulating an upgrade behavior, Run _UpdateToClassLibrary2.bat will use ClassLibrary2.dll file as new file to replace current using file; Run _UpdateToClassLibrary1.bat will use ClassLibrary1.dll file as new file to replace current using file; The default class library is ClassLibrary1.dll, we have a copy in v1 folder, and another version of it is ClassLibrary2.dll, also have a copy in v2 folder.

If you want replace back using ClassLibrary1, please run _UpdateToClassLibrary1.bat, you will find the program will be back to using components in ClassLibrary1.dll, and started to do business.

Using the Framework

1. Implement the class which requires inherited from IComponent. In demo program, we combined all components (Reader, Processor, Dispatcher) in one assembly, but we recommended one component one assembly in real project.
2. Configurate components in app.config file, in demo project, we have the following configurations:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="componentSection" type="ClassContract.ComponentSectionHandler,ClassContract"/>
  </configSections>
  <componentSection>
    <components>
      <component name="reader1" assemblyName="ClassLibrary1.dll" 
       className="ClassLibrary1.Reader" isUpgradable="true" downstreams="processor1" />
      <component name="processor1" assemblyName="ClassLibrary1.dll" 
       className="ClassLibrary1.Processor" isUpgradable="true" downstreams="dispatcher1" />
      <component name="dispatcher1" assemblyName="ClassLibrary1.dll" 
       className="ClassLibrary1.Dispatcher" isUpgradable="true" downstreams="" />
    </components>
  </componentSection>
</configuration>

At first, we added custom section, set its type to ClassContract.ComponentSectionHandler,ClassContract, then added <components> node, and added all components under <components> node, <component> has the following fields:

You can start framework now when you finished the above things.

Design of Framework

The class diagram of framework is shown below:

Init() Method in Mediator Class

This diagram shows classes from top to bottom, but I will describe classes from bottom to top, the Mediator class is mainly business class, in main()method, we use Mediator like this:

static void Main(string[] args)
{
  Mediator e = Mediator.GetInstance();
  e.Init();
  
  // keep the console alive...
  Console.ReadKey();
}

Firstly, we wrote Mediator class in singleton pattern, in one program, only one mediator is enough, and to protect Mediator only has one instance. We applied singleton pattern on Mediator class. Mediator is responsible for manage components:

• Read component configurations from app.config:
  C#

  ComponentSectionHandler config = ComponentSectionHandler.GetConfigurationSection();

• Create each component, we implemented Observer pattern between Mediator and ComponentManager, ComponentManager is responsible for creating instance of component, and detects assembly change of component. When ComponentManager detected assembly file change, it will inform Mediator class to determine whether to re-create component instance (we can add a property in IComponent interface, to indicate whether component is busy or not, if it's not busy, we can replace it, otherwise we can wait until it's not busy, this doesn't included in demo, just an idea). ComponentManager class calculates assembly file MD5 value to determine whether the file changed or not, file changed event raised by FileSystemWatcher class.
  C#

  //------------------------------
  // Create components
  //------------------------------
  Console.WriteLine("------ Create components ------");
  foreach (ComponentConfigurationElement componentConfig in config.Components)
  {
    ComponentManager componentMgr = new ComponentManager();
    componentMgr.ComponentConfiguration = componentConfig;
    componentMgr.Updated += new EventHandler<ComponentUpdatedEventArgs>(ComponentManager_Updated);
    IComponent component = componentMgr.CreateInstance();
    if (component != null)
    {
      component.Name = componentConfig.Name;
      component.Downstreams = componentConfig.Downstreams;
      componentMgr.EnableMonitor = componentConfig.IsUpgradable;
      this.ComponentList.Add(component);
      Console.WriteLine("\tCreated component: {0}", component.Name);
    }
    else
    {
      Console.WriteLine("\tError to create component: {0}", componentConfig.Name);
    }
  }

• Calculate downstreams and register event for each component, we have two assistant methods to get downstream and upstream components, they are GetUpstreamComponents, GetDownsteamComponents:
  C#

  //------------------------------
  // Register components events
  //------------------------------
  Console.WriteLine("------ Register components events ------");
  foreach (IComponent component in this.ComponentList)
  {
    List<IComponent> downstreamComps = GetDownsteamComponents(component);
    if (downstreamComps != null && downstreamComps.Count > 0)
    {
      component.DownstreamComponents = downstreamComps;
      component.AfterBusiness += Component_AfterBusiness;
      Console.WriteLine("\tRegistered component: {0} downstreams:{1}", 
                         component.Name, component.Downstreams);
    }
  }

  Registered AfterBusiness event for each component, we use event mechanism to pass data between components. In Component_AfterBusiness method, it will inform all downstream component to do its business by call DoBusinesss method, the data contained in ComponentDataEventArgs.Data property, its type is object, if we need filter data in different component, we can filter data in DoBusinesss method in that component.

• Start/stop components:
  C#

  //------------------------------
  // Start components
  //------------------------------
  Console.WriteLine("------ Start components ------");
  this.ComponentList.ForEach(p =>
  {
    p.Start();
    Console.WriteLine("\tStarted: {0}", p.Name);
  });

Component Assembly Changed Methods

When a ComponentManager raised an Updated event, we will know one of components assembly changed, get specific component from ComponentUpdatedEventArgs.ComponentConfiguration.Name, the name is unique, so we can easy get component by enumerate component list.

• Re-create component instance from new version of assembly:
  C#

  // create new component, and check its status
  IComponent newComponent = componentMgr.CreateInstance();
  if (newComponent == null)
  {
    Console.WriteLine("\tError when re-create component {0}", e.ComponentConfiguration.Name);
    return;
  }
  newComponent.Name = e.ComponentConfiguration.Name;
  newComponent.Downstreams = e.ComponentConfiguration.Downstreams;
  newComponent.DownstreamComponents = GetDownsteamComponents(newComponent);
  sb.AppendLine("\tre-created component");

• Replace old instance of component with new instance:
  C#

  // get component which need updated.
  IComponent oldComponent = GetComponentByName(e.ComponentConfiguration.Name);
  if (oldComponent == null) return;
  // stop component
  oldComponent.Stop();
  // un-register event from component
  oldComponent.AfterBusiness -= Component_AfterBusiness;
  // remove component from upstream components
  List<IComponent> upstreamComponents = GetUpstreamComponents(oldComponent);
  upstreamComponents.ForEach(p => p.DownstreamComponents.Remove(oldComponent));
  // remove component from component list
  this.ComponentList.Remove(oldComponent);

• Applied new instance of component:
  C#

  this.ComponentList.Add(newComponent);
  upstreamComponents.ForEach(p => p.DownstreamComponents.Add(newComponent));
  newComponent.AfterBusiness += Component_AfterBusiness;
  sb.AppendLine("\treplaced with new component");
  newComponent.Start();
  sb.AppendLine("\tre-start component");

Notes

1. To sync business between components and Mediator update process, we use lock statement to protect content in multi-threads.
2. In demo program, we combined three components in one assembly, when the assembly changed, which means three components are all updated which will raise three times of event Updated, for sync output in Updated method, we use StringBuilder to collect output in method and output at onetime.

Further Thoughts

In demo framework, we only implemented the mechanism of upgrade component when services running, for further development, I have two suggestions to improve this framework:

1. To support multi-component in one assembly file, but raise only once Updated event.
2. For component configuration, it may have complex properties than this framework, we also can support upgrade component by changed configuration, but not assembly file change.