The Next Logical Step Using Azure Functions - How to Integrate a Native DLL via P/Invoke ([DllImport])

Introduction

I (and, of course, my company) wanted to continue to use the knowledge that had been cast in C++ code for decades when we switched to Azure Functions.

To all those who have similar plans, I would like to say: Yes, it works!

For those in a hurry - what conclusion would I like to draw?

• I just couldn't get a mixed mode DLL to work - which would have been my favourite.
• But instead I got a native DLL to work- which also serves the intended purpose.
• Key experiences from the nights I've been through are:
  • Start with a simple native DLL to test the technology and procedure.
  • Nothing that works locally is guaranteed to work in Azure as well.
  • Nothing that works in emulation (local debugging) is guaranteed to work in Azure as well.
  • Investigation of the deployment / publishing to Azure is essential.
  • Radical reduction (and static linking) of the dependencies of the native DLL finally fixed my problem.

Background

After reading countless discussions in the different internet forums (I found it annoying that the internet forum of the Azure manufacturer has very little to offer) I found the following discussion: Referencing external assembly with native dependencies in Azure Function

At this point I was already very frustrated (I had started with a mixed mode DLL and failed and I had derived a native DLL from it that I also just couldn't get to work). But this discussion was really pushing me:  Don't give up! Keep trying!

Nothing that works lokally/in emulation is guaranteed to work in Azure as well

My first attempts to integrate a native DLL

• worked well in a unit test with xUnit,
• worked well in local debugging mode but
• failed in Azure and led to the exception "Unable to load DLL '...' or one of its dependencies: The specified module could not be found. (Exception from HRESULT: 0x8007007E)".

Of course I then tried to eliminate the usual suspects step by step - after all, the web is full of well-intentioned advice for the 0x8007007E error:

• The path to the DLL is not part of the search path of the Library Loader -> specify path explicitly.
• Implicit loading does not work -> call the Library Loader explicitly.

In the end it turned out that none of this was the cause. Because the Library Loader automatically includes the folder from which the executing application (in the case of Azure Functions, the managed DLL) was started - and it is the same folder in which the native DLL is located (because of the constraints of Azure publishing procedure).

Notwithstanding - here is the managed code that brings all this to daylight:

/// <summary>
/// This class implements the responder (POST with JSON) to call for any calculation.
/// </summary>
public static class CalculationResponder
{
    /// <summary>
    /// Determine whether the requested library is situated within the search folder.
    /// </summary>
    /// <param name="searchFolder">The folder to search for the requested library.</param>
    /// <param name="requestedLibrary">The library to search for within the search folder.</param>
    /// <returns></returns>
    private static bool IsInFolder(string searchFolder, string requestedLibrary)
    {
        foreach (string s in Directory.GetFiles(searchFolder))
        {
            if (s.EndsWith(requestedLibrary))
                return true;
        }

        return false;
    }

    /// <summary>
    /// The public visible ARURE function 'CalculateLayout' implementation.
    /// </summary>
    /// <param name="req">The request, that can only be of type POST request.</param>
    /// <param name="log">The logger, provided by the runtime environment.</param>
    /// <returns>The <see cref="ObjectResult"/> result of the execution.
    /// Typically an <see cref="JsonResult"/> on POST request success,
    /// or an <see cref="BadRequestObjectResult"/> otherwise.</returns>
    [FunctionName("CalculateLayout")]
    public static async Task<IActionResult> Run(
        [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req,
        ILogger log)
    {
        bool   loadNativeLibSuccess = false;
        var    loadNativeLibProtocol = new List<Tuple<string, string>>();
        string managedLibInstallPath = Path.GetDirectoryName(
            Assembly.GetExecutingAssembly().Location).TrimEnd('\\');
        string nativeLibFileName     = "Symbio.Service.ContentLayouter.Native.dll";

        try
        {
            NativeLibrary.InitErrorHandling(log);
            loadNativeLibProtocol.Add(new Tuple<string, string>("ATTEMPT_01",
                "SUCCEEDED: Loading library without any preparation successfull."));
            loadNativeLibSuccess = true;
        }
        catch (Exception e)
        {

            loadNativeLibProtocol.Add(new Tuple<string, string>("ATTEMPT_01",
                "FAILED: Loading library without any preparation not successfull."));
            loadNativeLibProtocol.Add(new Tuple<string, string>("exception", e.Message));
            loadNativeLibProtocol.Add(new Tuple<string, string>("installPath",
                managedLibInstallPath));
            loadNativeLibProtocol.Add(new Tuple<string, string>("environmentPath",
                Environment.GetEnvironmentVariable("PATH")));
            loadNativeLibProtocol.Add(new Tuple<string, string>("libraryLocated",
                (IsInFolder(managedLibInstallPath, nativeLibFileName) ?
                    "YES-" + nativeLibFileName : "NO")));
        }

        if (loadNativeLibSuccess == false)
        {
            // NativeLibrary.SetDllDirectory("D:\\home\\site\\wwwroot\\bin");
            string oldCWD = Directory.GetCurrentDirectory();
            Directory.SetCurrentDirectory(managedLibInstallPath);

            try
            {
                NativeLibrary.InitErrorHandling(log);
                loadNativeLibProtocol.Add(new Tuple<string, string>("ATTEMPT_02",
                    "SUCCEEDED: Loading library with adopted CWD successfull."));
                loadNativeLibSuccess = true;
            }
            catch (Exception e)
            {
                loadNativeLibProtocol.Add(new Tuple<string, string>("ATTEMPT_02",
                    "FAILED: Loading library with adopted CWD not successfull."));
                loadNativeLibProtocol.Add(new Tuple<string, string>("exception", e.Message));
                loadNativeLibProtocol.Add(new Tuple<string, string>("oldCWD", oldCWD));
                loadNativeLibProtocol.Add(new Tuple<string, string>("newCWD",
                    Directory.GetCurrentDirectory()));
            }

            Directory.SetCurrentDirectory(oldCWD);
        }

        if (loadNativeLibSuccess == false)
        {
            IntPtr hModule = IntPtr.Zero;
            try
            {
                hModule = NativeLibrary.LoadLibrary(managedLibInstallPath + "\\" +
                    nativeLibFileName);

                NativeLibrary.InitErrorHandling(log);
                loadNativeLibProtocol.Add(new Tuple<string, string>("ATTEMPT_03",
                    "SUCCEEDED: Loading library with previous call to 'kernel32.dll' " +
                    "successfull."));
                loadNativeLibSuccess = true;

                if (hModule != IntPtr.Zero)
                    NativeLibrary.FreeLibrary(hModule);
            }
            catch (Exception e)
            {
                loadNativeLibProtocol.Add(new Tuple<string, string>("ATTEMPT_03",
                    "FAILED: Loading library with previous call to 'kernel32.dll' " +
                    "not successfull."));
                loadNativeLibProtocol.Add(new Tuple<string, string>("exception", e.Message));
                loadNativeLibProtocol.Add(new Tuple<string, string>("hModule",
                    (hModule).ToString()));
            }
        }

        if (loadNativeLibSuccess == false)
        {
            StringBuilder sb = new StringBuilder();
            bool          firstEntry = true;
            foreach (var tuple in loadNativeLibProtocol)
            {
                sb.Append((firstEntry ? "": ",") + "\"" +
                   tuple.Item1 + "\":\"" + tuple.Item2 + "\"");
                if (firstEntry)
                    firstEntry = false;
            }
            return (ActionResult)new JsonResult("{" + sb.ToString() + "}");
        }

        // Here goes the code that does the actual processing of the request.
    }
}

Investigation of the deployment / publishing to Azure

I use Visual Studio 2019, .NET Core 3.1 and .NET SDK Functions 3.0.3 for the managed DLL that contains the Azure Functions. Who would have thought that Azure would like to run such modern tools in a 32 bit runtime environment - not me! (It is the year 2020 - I think serious applications can expect a 64 bit runtime environment.)

My step 1: Radically change everything to x64.

On my Visual Studio 2019 menu bar:

• Build -> Configuration Manager ... -> Active solution configuration: "Debug" (and afterwards for "Release" the same) and Active Solution platform: "x64"
  • For my C# Azue Function project: Switch Platform to "x64" (I had to prepare the x64 platform first. This worked after I selected "<New ...>" in the dropdown and set within the upcoming Dialog Copy settings from: "Any CPU"; Create new solution platforms: "off").
  • For my native DLL project: Switch Platform to "x64"
  • I also have an XUnitTest project - I apply the same as for my C# Azue Function project

On my Visual Studio 2019 tool bar:

• Solution Configuration: "Debug"
• Solution Platform: "x64"

On my C# Azue Function project:

• Context menu -> Publish ... -> Edit (a very small link button below the Azure Resource drop down)
  • Configuration: "Release | x64"
  • Target Runtime: "win - x64"

On my Azure Portal:

• Resource group <to publish to> -> the app service <to publish to> -> configuration -> general settings
  • Platform: "64 Bit"

My step 2: Ensure the native DLL will be published.

There are several recommendations in different forums on how to correctly integrate a native DLL into an Azure Function project deployment. I have chosen to use a very simple way, where no configuration files have to be edited manually:

• I found out that my native DLL is built to either "Project Root\x64\Debug" or "Project Root\x64\Release".
• I have added a Pre-build event command line to the C# Azure Function project that copies the native DLL into the souce code folder of the C# Azure Function project.
• I have added the copy of the native DLL to the list of source files of the C# Azure Function project and configured it to "Copy always".

Radical reduction (and static linking) of the dependencies of the native DLL

Typically a new "Dynamic-Linked-Library (DLL)" project with the filter "C++", "Windows" and "Library" is created with dependencies to these libraries:

• kernel32.lib; user32.lib; gdi32.lib; winspool.lib; comdlg32.lib; advapi32.lib; shell32.lib; ole32.lib; oleaut32.lib; uuid.lib; odbc32.lib; odbccp32.lib; %(AdditionalDependencies)

I have reduced the dependencies to these libraries:

• kernel32.lib; user32.lib; gdi32.lib; %(AdditionalDependencies)

Don't forget to keep all changes for release and debug in sync!

And i have switched to link all dependencies statically.

• For Debug:Multi-threaded Debug (/MT)
• For Release: Multi-threaded (/MT)

I'm sure I overstepped the mark with some of the above. But removing unnecessary restrictions from a working environment step by step is certainly easier than initially setting up a working environment.

In the end the publishing procedure produces the following output:

[01] 1>------ Build started: Project: <managed-lib-name>, Configuration: Release x64 ------
[02] 1>        1 Datei(en) kopiert.
[03] 1>....Managed -> C:\<project root>\bin\x64\Release\netcoreapp3.1\bin\<managed-lib-name>
[04] 2>------ Publish started: Project: <project name>, Configuration: Release x64 ------
[05] 2>Restore completed in 81 ms for C:\<project root>\<project name>.csproj.
[06] 2>copy C:\<solution root>\x64\Debug\<native-lib-name> C:\<project root>
[07] 2>        1 Datei(en) kopiert.
[08] 2>C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\...
[09] 2><managed-project-name> -> C:\<project root>\bin\x64\Release\netcoreapp3.1\win-
       x64\bin\<managed-lib-name>
[10] 2><managed-project-name> -> C:\<project root>\obj\Release\netcoreapp3.1\win-x64\PubTmp\Out\
[11] 2>    Could not evaluate '<native-lib-name>' for extension metadata. Exception message:
       Bad IL format.
[12] 2>Publishing C:\<project root>\obj\Release\netcoreapp3.1\win-
       x64\PubTmp\<managed-project-Name> ...
[13] ========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
[14] ========== Publish: 1 succeeded, 0 failed, 0 skipped ==========

I would like to give you a few more explanations:

• Line [02] indicates the execution of my Pre-build event command line for compilation.
• Line [07] indicates the execution of my Copy always property for publishing preparation.
• Line [11] is just a warning and can be ignored. Currently, the publishing process does not distinguish between managed and native libraries and attempts to apply checks for managed libraries to native libraries.

Points of Interest

I believe this is the first illustrated guide on the web how to integrate a native DLL into Azure Functions (or I just didn't find the right search words) and hope it spares others the odyssey I went through.

I did not say any word about P/Invoke ([DllImport]) in the whole tip - use this technique as you already know it from .NET Framework! And dont't forget to hand over always ::CoTaskMemAlloc() dynamically allocated strings (never hand over const string literals).

History

07^th February, 2020: Initial version
11^th February, 2020: added C# code to investigate the Azure environment and explanation of the publishing log