MSBuild for Programming and Non‑Programming Chores

• Download source — 2.9 KB

Contents

Why MSBuild?
Cross-Platform
Why .NET?
Code Sample
Why FFmpeg?
Implementation
Project
Properties
Targets and Parallel Execution
Custom Properties
Using Visual Studio Code
Compatibility and Testing
Conclusions

Why MSBuild?

Doesn’t the use of MSBuild for non-programming work look way too esoteric? Well, let’s look closer.

I have used different scripting systems for development using some exotic languages or software development toolchains: Microsoft “CMD” batch, Python, JavaScript with node.js or even badly outdated but still operational WSF, PowerShell, bash, and I’ve never been convinced to prefer one system over another one.

I also use scripting for many non-programming chores. One of my permanent activities requiring considerable scripting effort is handling different media files on different platforms: video, audio, video captions, photographs, and graphics. Recently, I decided to try out MSBuild. One of the reasons for that is that recently I took a chance to delve into BSBuild more than I usually do. Notably, it is related to my recent development found in a GitHub repository dotnet-solution-structure.

I quickly realized that MSBuild provides serious benefits in several ways. Ultimately, they are mostly reduced to the fact that MSBuild offers declarative programming, nothing like imperative programming offered by the usual scripting systems.

It is very beneficial when you have to handle sets of files located in different directories without any guaranteed directory structure. Its extended syntax for file masks is very helpful. Another powerful feature is incremental build. For software development toolchains, incremental build is usual and even mandatory. But usual scripting does not offer anything like that. At the same time, a lot of non-programming development is incremental. And some processing takes a lot more time than typical software build. One notorious example is transcoding using modern video codecs. These days, the better the latency of a technology, the slower the encoding process is.

But probably the most important benefit of using MSBuild is the possibility to use it in a cross-platform way.

Cross-Platform

The sample project provided here is cross-platform. Please see the explanation of why it is so below.

One note on that: every time the OS “Linux” is mentioned, it actually means “non-Windows”. All the elements of the solution designed to work for Linux should also work for any other platform except Windows. For this reason, all the platform-dependent conditions are calculated as $(OS.StartsWith('Windows') — I hope it is self-explanatory.

Why .NET?

MSBuild can be invoked in several ways. First of all, the command is “MSBuild”. MSBuild is also used for .NET projects and solutions, and then it is “dotnet build”.

But I recommend using “dotnet msbuild”. It is clear why “dotnet build” is not suitable for non-.NET solutions, this command does not even have the “target” option. But why “dotnet msbuild” and not “MSBuild”?

The first reason is the deployment: I don’t know where your MSBuild instance is located. Developers often have several MSBuild instances installed at the same time, different versions of them. As to .NET, it is usually installed to have the application path “dotnet” accessible everywhere, and it knows the version of MSBuild supplied with it, and will launch the most appropriate version.

There is another, more subtle reason: with “dotnet msbuild”, you can use very convenient .NET API functions in MSBuild property definitions. For a detailed description of this possibility, please see Microsoft documentation on property functions.

And of course, one major benefit of executing MSBuild via .NET is that this method is cross-platform. It works on any platform where .NET is installed.

“run.cmd”:

dotnet msbuild a.project -t:Transcode

This simple command line is designed to be usable for Windows and other systems. For Linux, it should also be made executable before using it:

chmod -x run.cmd

Code Sample

To evaluate the value of MSBuild, I create the code sample used to transcode any media supported by FFmpeg. The goal is to have an arbitrary set of media files located randomly under some directory, uniformly transcode them all, and place them in a single directory. It is done in a customizable manner: the inputs, file types, and transcoding options are placed in one “Custom” file and can be modified by the final user.

It covers most of the basic features of FFmpeg, where we need one-to-one transcoding. It does not support very tricky situations with sophisticated mapping between input and output elements, multi-pass processes where each pass should use intermediate files obtained on a previous pass, and so on. However, my code sample can be upgraded with additional project files solving those advanced problems, using the same codebase I provided with the present article.

Why FFmpeg?

FFmpeg is probably the most universal and powerful suite used for the processing of a very wide range of media files. For video, it is probably the most fundamental tool. Many people who think that they don’t use FFmpeg actually use it, because it lies in the base of most media editors, generators, and other media software.

It is much less true for photography and other visual arts, and a lot more for everything else, especially video.

Implementation

Project

In this particular project, there is only one project file. Moreover, it is never changed:

“a.project”:

<Project Sdk="Microsoft.NET.Sdk">

    <ItemGroup>
        <InputFiles Include="$(InputMask)"/>
    </ItemGroup>

</Project>

This is the most generic project. It does not have to be changed, because it simply transparently passes input mask defined elsewhere and generates a set of items.

MSBuild concept of items is very powerful. It replaces all those silly iteration loops required by other scripting systems, as well as traversing any directories, recursive or not.

The real set of definitions comes from three common files. One of them, “Custom.props”, is specific to the project and type of processing. The other two are applied to all projects in all downstream subdirectories: “Directory.Build.props” and “Directory.Build.targets”.

This use of the shared properties and targets is analogous to the definitions for Microsoft toolchains: the common predefined sets of properties and targets come with MSBuild, Visual Studio, and other products. This way, more project files using common definitions, can be added.

Properties

“Directory.Build.props”:

<Project>
    <Import Project="Custom.props"/>

    <PropertyGroup>

        <OutputPath>$(ProjectDir)output</OutputPath>

        <Tool>ffmpeg</Tool> <!-- Linux -->
        <!-- For Windows: modify the path to access ffmpeg: -->
        <Tool
            Condition="$(OS.StartsWith('Windows'))"
            >C:/app/Media/ffmpeg/bin/ffmpeg.exe</Tool>

        <Multitasking
            Condition="$(OS.StartsWith('Windows'))"
            >start</Multitasking>

        <Continue>%5C</Continue> <!-- backslash for Linux -->
        <Continue Condition="$(OS.StartsWith('Windows'))">^</Continue>

    </PropertyGroup>

</Project>

Targets and Parallel Execution

“Directory.Build.targets”:

<Project>

    <Target Name="Transcode">

        <PropertyGroup>
            <!-- MBSBuild item transform: -->
            <Commands>@(InputFiles ->
                    '$(Multitasking) $(Tool) -y $(Continue)
                    $(InputOptions) $(Continue)
                    -i %(Identity) $(Scale) $(Continue)
                    $(OutputOptions) $(Continue)
                    $(OutputPath)%(Filename).$(OutputFileType)',
                ' %26 ')
            </Commands>
        </PropertyGroup>

        <MakeDir Directories="$(OutputPath)" />
        <Exec Command="$(Commands)"/>

    </Target>

</Project>

The clause using “@” and “%” is a very interesting and powerful thing, MSBUilt items transform. Please see the Microsoft documentation on MSBuild Transforms for more information. In this clause, we transform the list of items into a list of commands executing FFmpeg.

The list of commands is concatenated using different ways of executing FFmpeg in parallel. It works differently in different systems. What is “%26”? This is the character “&”. MSBuild does not allow direct use of this character, so it should be escaped with a hexadecimal escape notation. For Linux, “&” between commands means the execution of them in parallel. For Windows, this character is also used, but it means consecutive execution of the commands. For execution in parallel on Windows, each command should be prepended with the command “start”, which is defined as the property Multitasking.

If parallel execution should be turned into a consecutive, for whatever reason, how to do it? For Windows, the value of $(Multitasking), “start”, should become an empty string. For Linux, the command delimiter “&” (“%26”) should become double, “&”. In this case, it can be double for Windows, too.

There is another weird property Continue. It is defined to show the command in several lines, mostly for clarity and especially for the publication of the present article. It is a different character for Linux and Windows, “\” and “^” correspondently, and “\” is also the character to be escaped. Please see “Directory.Build.props”.

The other input information is custom and is defined in a separate file “Custom.props”, specific to a particular transcoding task. It supply the items and properties @(InputFiles), $(Options), $(OutputPath) and, $(OutputFileType). So, finally, let’s take a look at this file.

Custom Properties

First of all, note the use of “**” wildcard. It extends the set of input files to the paths on all levels of nesting.

“Custom.props”:

<Project>

    <PropertyGroup>
        <InputMask>../**/*.png</InputMask>
        <OutputFileType>webp</OutputFileType>
    </PropertyGroup>

    <PropertyGroup>
        <Scale>-vf scale=256:-1</Scale>
        <Scale>-vf scale=-1:128</Scale>
        <Scale></Scale>
    </PropertyGroup>

    <PropertyGroup>
        <InputOptions></InputOptions>
    </PropertyGroup>

    <PropertyGroup>
        <OutputOptions>-lossless 0</OutputOptions>
        <OutputOptions>-lossless 1</OutputOptions>
        <OutputOptions>-quality 0</OutputOptions>
        <OutputOptions>-quality 100</OutputOptions>
        <OutputOptions>-preset none</OutputOptions>
        <OutputOptions>-preset default</OutputOptions>
        <OutputOptions>-preset picture</OutputOptions>
        <OutputOptions>-preset photo</OutputOptions>
        <OutputOptions>-preset drawing</OutputOptions>
        <OutputOptions>-preset icon</OutputOptions>
        <OutputOptions>-preset text</OutputOptions>
        <OutputOptions></OutputOptions>
    </PropertyGroup>

</Project>

In these definitions, many different options are shown. As always, the property defined lower overwrites the properties defined above, so many definitions are redundant. They are placed here for convenience and self-documenting purposes. In this example, the properties Scale and Options are empty, but they can be changed by placing another property definition below the last line.

In the definition of Scale, the format -vf scale=<width>:<height> is used, and -1 means that width or height is calculated automatically, to make sure the aspect ratio remains the same.

Using Visual Studio Code

This section is more important for software development use, especially with the use of marginal and not very popular programming systems. Some of them don’t have any IDE, debugger, or build toolchain, and simply offer a few command-line tools like compiler, linker, and the like. How to work with them, even without a Visual Studio Code extension?

This is pretty simple. First, add a member preLaunchTask to some configuration in “launch.json”. This is an example:

“launch.json”:

{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "pwa-chrome",
            "request": "launch",
            "name": "Transcode",
            "preLaunchTask": "Build",
            "runtimeArgs": ["--incognito"],
            "file": "${workspaceFolder}/.vscode/index.html",
            "webRoot": "${workspaceFolder}"
        }
    ]
}

Here, “Build” is a task in the file “tasks.json”:

“tasks.json”:

{
    "version": "2.0.0",
    "tasks": [{
        "label": "Build",
        "command": "${workspaceFolder}/run.cmd",
        "type": "shell"
    }]
}

This way, the launch configuration “Transcode” will execute our “run.cmd” and then report “MSBuild work is complete”. Please see sample “.vscode/*” files for further information.

Compatibility and Testing

The solution is supposed to be compatible with all platforms where .NET is installed.

Tested thoroughly on Linux and Windows.

Conclusions

The solution works smoothly on different platforms. It is very easy to upgrade to a more complicated solution. It shows the most ins and outs important for the development of a wide range of systems working with different input and output files without a rigid directory structure.