Strongly Typed, Dependency Managed Azure in C#: Introducing Cake.AzureCLI

The story, nay legend, of providing strongly typed, cross platform, dependency managed access to all 2,935 Azure CLI commands in C#.

You can now have strongly typed, cross platform, dependency managed access to all 2,935 Azure CLI commands in C#, with full intellisense including examples. That's because I just published a Cake plugin for AzureCLI called Cake.AzureCli.

This blog post is a little about what it is and how to use it, but it's more about how I built it. That's because I had a blast solving this problem and my solution might even entertain you: parsing thousands of help files through the CLI, storing results in 16 meg intermediate JSON files, and code generating 276K lines of code with T4 templates.

In the process, I apparently also broke Cake's static site generator.

We also have an issue adding the addin to the https://t.co/CiFD8bupHK site, since the static site generator used to create the site runs into an out of memory exception. We have planned to migrate to @statiqdev once available which should help in such scenarios.

— Pascal Berger (@hereispascal) February 17, 2020

Oops!

But first, I suppose the most relevant information is the what and the how.

Have & Eat Your Cake.AzureCLI

This plugin runs in Cake. If you aren't familiar with cake, please check out Code Hour Episode 16, Intro to Cake, where I go over what it is and why you should care.

If you don't have a spare hour right now: it's a dependency management system (like make, ant, maven, or rake) except in C#. It also has a huge plugin ecosystem, one that's now "slightly" larger with access to all of Azure CLI.

Right, you didn't watch the video, and you're still skeptical, right? You're wondering what was wrong with the official azure-sdk nuget plugins. The answer is: they aren't Cake enabled and so they don't support dependency management. If that statement isn't meaningful to you, please, watch just the "Scripts" section of my talk starting at 9:08.

Now that you're 100% convinced, let's dig in. Using Cake.AzureCLI is as simple as adding a preprocessor directive to pull it from NuGet:

#addin "nuget:?package=Cake.AzureCli&version=1.2.0"

And then accessing commands like Az().. So a simple program to log in and list all your resource groups might look like this:

var target = Argument("target", "Default"); 
var username = Argument<string>("username", null); 
var password = Argument<string>("password", null);  
Task("Login")    .Does(() =>
{    // 'az login' is accessed via Az().Login()    
     Az().Login(new AzLoginSettings 
    {       
        Username = username,                     // all commands can be customized 
                                                 // if necessary with a ProcessArgumentBuilder
        Arguments = new ProcessArgumentBuilder() // anything appended with 
                                                 // .AppendSecret() will be 
                                                 // rendered as [REDACTED]
                                                 // if cake is run with `-verbosity=diagnostic` 
        .Append("--password").AppendSecret(password)    }); });  
        Task("ListResourceGroups")    .IsDependentOn("Login") // yayy dependency management!
        .Does(() => {    // listing names of all resource groups
        Information("Resource Groups:");         // all results are strongly typed as 
                                                 // dynamic if results are json
        dynamic allResourceGroups = Az().Group.List(new AzGroupListSettings());
        foreach (var resourceGroup in allResourceGroups)
        {
            Information(resourceGroup.name); 
        }
    }
);  
RunTarget(target);

And that should hopefully provide enough background to go create SQL instances, scale up or down kubernetes clusters, and provision VMs with dependency management, from the comfort of a language you know and love.

The Making Of

"But Lee, I'm dying to know, how did you build this work of art?"

Oh, I'm so very glad you asked. Writing something this large by hand was obviously not going to work. Plus it needs to be easy to update when Azure team releases new versions. Code generation it was. And I always wanted to learn T4 templates.

I first came up with a data structure, always a solid place to start. I wanted something that would support Azure CLI, but that could also be used to generically represent any CLI tool, because ideally this solution could work for other CLI programs as well. I came up with this:

A Program contains a single root Group (az). Groups can contain other Groups recursively (e.g., az contains az aks which contains az aks nodepool). Groups can contain Commands (e.g., az aks contains az aks create). And for documentation, Commands can have Examples and Arguments.

It's basically a tree, with Commands as leafs, and so will work nicely in json. But how to populate it?

Fill Er Up

"Well, that's not how I would have done it"

said a skeptical co-worker when I told him I was executing thousands of az [thing] --help commands and parsing the results. See, AzureCLI was written in Python and is open source, so theoretically I could have downloaded their source and generated what I needed from there in Python.

But I really wanted a more generic approach that I or someone else could apply to any CLI program. So I parsed each "xyz --help" into an intermediary object: a Page. That's basically just a collection of headers, name-value pairs, and paragraphs. Then I converted pages to groups or commands and recursed to produce a 350,385 line, 15 megabyte behemoth.

Incidentally a fun side-effect of this approach is you can see all the changes across Azure CLI version changes, e.g., this commit shows changes from 2.0.77 to 2.1.0 (although GitHub doesn't like showing diffs across 16 Meg files in the browser for some reason, can't imagine why).

T4 Templates

I'd never used T4 templates. Turns out they're super awesome. Well, super powerful, and pretty awesome anyway. They are a little annoying when every time you hit save or tab off a .tt file, it takes 13 seconds to generate your 178 thousand lines of code -- even on an 8 core i9 with 64 gigs of ram and an SSD. Oh, and then at that scale, Visual Studio seems to crash periodically, although I'm sure Resharper doesn't help.

But whatever, they work. And this part is cool: If you set hostspecific=true, then you can access this.Host to get the current directory, read a json file, then deserializing it to model objects that live in a .Core project that you can reference inside of the tt file yet not reference inside of your main project (Cake.AzureCli). If you're interested, check out Az.tt.

What to generate was interesting too. The easy part is exposing a method to cake. You just write an extension method like this:

public static class AzAliases 
{     
    [CakeMethodAlias]
    public static AzCliGroup Az(this ICakeContext context)
    {
    return new AzCliGroup(context);
    } 
}

And Cake is good to go. But what about generating 2,935 extension methods? Turns out, not such a great idea. The intellisense engine in Visual Studio code is powered by OmniSharp. As awesome as OmniSharp is, it just isn't quite powerful enough to generate intellisense quickly or accurately with that architecture. However, if you group commands into "namespaces" like Az().Aks.Create() instead of AzAksCreate(), then you get nice intellisense at every level:

Conclusion

While this project may not solve world hunger just yet, I do hope it'll make someone's life a little easier. More importantly, I hope this technique will entertain or better yet inspire someone (you) to create something cool. If it does, please let me know about it in the comments below.