.NET Libraries and the Art of Backward Compatibility – Part 1

So, you wrote a .NET library, you released it to the public and now you are about to make version 2.0 or 1.1 or even just 1.0.0.0b.

Any change that you are going to make has the risk of introducing one or more of these types of backward incompatibilities:

• Behavioral (your library’s behavior is changing)
• Source (your users’ code may fail compiling)
• Binary (your users’ application may break at runtime)

This blog post will focus on behavioral incompatibilities. I will cover the other two types in upcoming posts:

• Part 2 – Source incompatibilities
• Part 3 – Binary incompatibilities

Behavioral Incompatibilities

Behavioral incompatibilities are introduced when your new library behaves differently than the previous version.

Overall, it is impossible to avoid behavioral incompatibilities. These are few changes that are likely to benefit most of your users, but are nonetheless behavioral changes:

• fixing a bug
• improving performance of a method call
• fixing a typo in an error message
• throwing a new type of exception from a method to better qualify the error

You never know whether any of your users is reliant on the existing (undesirable for most) behavior.

Inheritance Drama

An area where it is very easy to overlook a behavioral change is inheritance.

For example, you may have included in your library a Stream class and, following the notes provided by Microsoft, you have implemented the Read method only:

The asynchronous methods ReadAsync(Byte[], Int32, Int32), WriteAsync(Byte[], Int32, Int32), 
and CopyToAsync(Stream) use the synchronous methods Read(Byte[], Int32, 
Int32) and Write(Byte[], Int32, Int32) in their implementations. 
Therefore, your implementations of Read(Byte[], Int32, Int32) and Write(Byte[], 
Int32, Int32) will work correctly with the asynchronous methods.

I will use the following code as an example:: a stream that reads from another stream and converts ASCII characters to upper case.

public class UppercasingStream : Stream
{
    private readonly Stream BaseStream;
    private const byte UppercaseOffset = (byte)('a' - 'A');

    public UppercasingStream(Stream baseStream) => BaseStream = baseStream;

    public override int Read(byte[] buffer, int offset, int count)
    {
        int len = BaseStream.Read(buffer, offset, count);

        for (int i = 0; i < len; i++)
        {
            byte b = buffer[i + offset];
            if (b >= 'a' && b <= 'z')
            {
                buffer[i + offset] -= UppercaseOffset;
            }
        }

        return len;
    }

    ...

A user of this library may have extended this class to also remove non-alphabetic characters.

public class NormalizingStream : UppercasingStream
{
    public NormalizingStream(Stream baseStream) : base(baseStream) { }

    public override int Read(byte[] buffer, int offset, int count)
    {
        int len = base.Read(buffer, offset, count);

        for (int i = 0; i < len; i++)
        {
            byte b = buffer[i + offset];
            if (b < 'A' || b > 'Z')
            {
                buffer[i + offset] = (byte)'_';
            }
        }

        return len;
    }
}

The NormalizingStream class behaves as expected even when used asynchronously:

using (var reader = new StreamReader(
    new NormalizingStream(
        new MemoryStream(
            Encoding.ASCII.GetBytes("matteo.tech.blog"))),
    Encoding.ASCII))
{
    var result = await reader.ReadToEndAsync();
    //Writes "MATTEO_TECH_BLOG"
    Console.WriteLine(result);
}

A reasonable improvement of our UppercasingStream class would be implementing the ReadAsync method:

public override async Task<int> ReadAsync
       (byte[] buffer, int offset, int count, CancellationToken ct)
{
    int len = await BaseStream.ReadAsync(buffer, offset, count, ct);

    for (int i = 0; i < len; i++)
    {
        byte b = buffer[i + offset];
        if (b >= 'a' && b <= 'z')
        {
            buffer[i + offset] -= UppercaseOffset;
        }
    }

    return len;
}

Unfortunately, this apparently innocuous improvement, breaks our customer’s application preventing the overridden NormalizingStream.Read from being invoked.

using (var reader = new StreamReader(
    new NormalizingStream(
        new MemoryStream(
            Encoding.ASCII.GetBytes("matteo.tech.blog"))),
    Encoding.ASCII))
{
    var result = await reader.ReadToEndAsync();
    //Writes "MATTEO.TECH.BLOG" instead of "MATTEO_TECH_BLOG"
    Console.WriteLine(result);
}

A good way to proof our libraries against this type of backward incompatibilities is to seal all our public classes unless they are explicitly designed to be extended by the user. We can always unseal a class in a later version of our library if a reasonable use case arises.

If we had sealed UppercasingStream, the user would have likely used composition instead of inheritance and our change would be backward compatible.

Obsolescence Is Your Friend

When significant changes in behavior cannot be avoided, we have the option of creating completely new methods and classes and mark the existing ones as obsolete.

This is particularly nice because the Obsolete attribute allows us to provide a message that will show up in the IDE of our users or as compilation messages when they try to use the obsolete components. So we can guide users to the replacement methods and classes.

[Obsolete("This class is obsolete, use SeverelyBuggedClassV2 instead.")]
public class SeverelyBuggedClass
{
}

We can even force a compilation error when a customer tries to use the obsolete component:

[Obsolete("This class is obsolete, use SeverelyBuggedClassV2 instead.", error: true)]
public class SeverelyBuggedClass
{
}

This is useful in extreme circumstances when we really want our customers to move away from the deprecated component. This is better than removing the component because it allows us to provide guidance to the users in the obsolete message. It is also better than just fixing the component because it forces the user to acknowledge the breaking change and take an action instead of possibly being surprised by the unexpected behavioral change.

The drawback of using Obsolete is that our library will become more and more cumbersome to use as time goes by: SeverelyBuggedClass is inherently a nicer name than SeverelyBuggedClassV2.

Communication Is Key

Overall, there is very little we can do to avoid behaviorally incompatible changes, especially when it comes to addressing bugs and poor design choices. But we can make a plan in advance about how we are going to communicate them to the customers.

Imagine figuring out that you have to break backward compatibility just after having promised 10 years of support for your library’s latest LTS version.

A few suggestions.

• Don’t promise a longer support period than necessary: a broken promise is worse than no promise at all
• Make sure you know how to reach your customers when you need to tell them something important: what about a monthly newsletter?
• Make sure you reach the right people: you want your newsletter to go to the engineers, not to the Junk mail folder of the person in accounting who paid for the license
• Prepare your users about the idea of breaking changes: what about a yearly cadence of major version bumps with lots of new features and a few breaking changes

If your customers have the expectations that a new major version will come up every year and there will be good stuff in it, they are likely to be preparing for it and they will look forward to read the announcement of the changes. That is why gamers actually read release notes when a new version of their favorite game is updated!

Documentation and Undefined Behavior

The issue of behavioral compatibility is heavily constrained by what you wrote in your documentation. You can break your customers' expectations either by promising more compatibility than you can afford or by letting your customer think that a behavior they are currently experiencing is part of your guaranteed contract.

Make sure your documentation is as explicit as possible about how your library is supposed to be used and what are the conditions under which it is actually tested. For example, your library may work in a wildly different way under a less privileged account or on a case-sensitive file system.

Also make sure you let your user know about which area of growth you are keeping for your library. If you are documenting that your application will honor the FOO_THREADS and FOO_PROXY environment variables, you may also want to include that all other variables named FOO_something are considered reserved for future use and shouldn’t be used.

Finally, it is a good idea to call out explicitly which behavior is undefined. For example, if your documentation says:

Method Foo will throw
- ArgumentOutOfRangeException if offset is negative
- ObjectDisposedException if the current stream is closed
- System.Exception in case of unexpected error.

The statement about System.Exception makes changing the method to throw new types of exceptions perfectly backward compatible (because any new exception type extends System.Exception).

What Next?

The next blog post will cover source incompatibilities and then we will dive into the much more exotic topic of how to make sure that code compiled against a version of your library will continue to work when used with a newer version.