Automate D365 CI/CD Programmatically

Congratulations, the domain name shown above is your trial organization.

Go immediately to Azure Portal to sign in with the Administration email shown above.

Start a D365 Free Trial

Sign in D365 Free Trial page with the same account create earlier.

Click on any of the "Try for free" like the one under "Dynamic 365 Customer Service", you will be redirected to the CRM instance (like https://orgxxxxxx.crm5.dynamics.com) for 30 days trial:

Register Your D365 App

It is good to follow tutorials like this or this to register the CRM trial app.

1. Search and open "App Registrations" from the top bar.
2. New Registration to choose "Accounts in this organizational directory only (... single tenant)" will make the following authentication and authorization much easier.
3. I prefer to add the URL of the above CRM instance as the Redirect URI.
4. Keep the Application (client) ID in the Overview page for later use.
5. API permissions to "Add a permission", click "Dynamic CRM", choose "Delegated Permissions" and check "user-impersonation" before "Add permissions".
6. Certificates and Secrets to "New client Secret" and ensure keeping the value of the secret safely for later use.

Create Application User

This is the changed part compared with the previous tutorial: I wasted quite some time to realize that "Application Users" view is hidden in CRM Settings > Security > Users.

The Manage application users in the Power Platform admin center page, updated on 02/16/2022, shows the right steps. Since the CRM has been registered in Azure, "Create a new app user" > "Add an app" will show the newly registered app:

Ensure to add the right Security roles before "Create". In my case, I just added "System Administrator" role.

Use ClientSecret Connection String to Access CRM

As the sample Client Secret based authentication suggested, it shall be composed like:

"AuthType=ClientSecret;url=https://yourorg.crm4.dynamics.com;ClientId={AppId};
 ClientSecret={ClientSecret}"

Save the composed string as $con, running Connect-CrmOnline cmdlet proves it is working:

Demystify Microsoft.Xrm.Sdk Entities

Entity is the base class for all types of entities in Microsoft Dynamics 365. Subclasses of Entity are of fixed entityName or logicalName referring to the actual name of corresponding tables. Entity.Attributes, like a Dictionary<string, object>, is used to get or set the collection of attributes for the entity, and different implementations of Entity accept different set of keys and values because their corresponding SQL tables have different schemas.

The relationships between relevant Entities are enforced as the primary keys/foreign keys of the entries between tables, represented as EntityReference usually composed with the logicalName and Guid that are the table name and primary key of the associated entity respectively.

Talking about the plugin registration process, as commonly agreed pluginassembly, plugintype, sdkmessageprocessingstep, sdkmessageprocessingstepimage, sdkmessageprocessingstepsecureconfig entity types, plus solution are all Entity types to be concerned, as proved with a simple query:

Plugin Registration or Update

To debug PluginRegistration by following Debug Xrm.Toolbox, I append the command line argument "/overridepath:absolutionPathOfDllFile" instead of ""/overridepath:." to load the right assembly file Xrm.Sdk.PluginRegistration.dll after fixing two minor issues:

1. The logic of GenerateFriendlyName(string newName, out bool ignoreFriendlyName) in CrmPlugin.cs to prevent update or show the actual friendlyname attribute.
2. The Id property of PluginAssemby.cs failed to show Id.

It shows the registration happened in quite a straight-forward manner:

1. Load assembly metadata and content as BASE64 string from the DLL file.
2. Compose PluginAssemby instance with corresponding values.
3. Use the IOrganizationService instance to create above PluginAssembly instance.
4. Reflection to get all IPlugin and WorkCode implementations to create corresponding PluginType entities.
5. Use the IOrganizationService instance to create them.

So the pseudo procedure of automate Build&Deploy to update an existing plug in DLL can be:

1. Load assembly metadata and content as BASE64 string from the DLL file.
2. Initiate PluginAssemby instance with corresponding values.
3. Query to get existing with the same name.
4. Refresh the newly initiated instance with values from existing instance if they are not of modified, created or versions.
5. Use the IOrganizationService instance to update the refreshed PluginAssemby instance.
6. Reflection to get all IPlugin and WorkCode implementations to create corresponding
7. Retrieve all existing PluginType entities associated with the existing PluginAssemby instance.
8. Compare the differences between collections of 6) and 7) to perform Create, Delete or Update accordingly.

Code Snippets

To access IOrganizationService, I am using CrmServiceClient that only needs connection string configured in the previous post which allow basic operation like Publish All Customization:

PublishAllXmlRequest publishAllXmlRequest = new PublishAllXmlRequest();
var response = service.Execute(publishAllXmlRequest);

Get Children Entities

Thanks to the structured naming conventions, it is easy to define generic methods to get entities associated:

public static IEnumerable<Entity> GetChildren(this IOrganizationService service, 
string entityName, Entity parentEntity, string parentKeyName = null, params string[] keys)
{
    //Get the attribute name referring parent id by the concerned 
    //children entity type if it is not given
    string parentEntityTypeId = parentKeyName ?? $"{parentEntity.LogicalName}id";
    QueryExpression queryExpression = new QueryExpression(entityName)
    {
        ColumnSet = keys.Length == 0 ? new ColumnSet(true) : new ColumnSet(keys),
        Criteria = new FilterExpression()
        {
            Conditions = { new ConditionExpression(parentEntityTypeId, 
            ConditionOperator.Equal, parentEntity.Attributes[parentEntityTypeId]) }
        }
    };
    
    var entities = service.RetrieveMultiple(queryExpression).Entities;
    return entities;
}

public static IEnumerable<TEntity> GetChildren<TEntity>(this IOrganizationService service, 
Entity parentEntity, string parentKeyName = null, params string[] keys)
    where TEntity : Entity
{
    string entityName = typeof(TEntity).Name.ToLower();
    var entities = service.GetChildren(entityName, parentEntity, parentKeyName, keys)
        .Select(entity => entity.CastTo<TEntity>());
    return entities;
}

Then all entities associated with a PluginAssembly of known name can be retrieved conveniently:

public static IDictionary<Type, Entity[]> 
       GetPluginAssemblyEntities(this IOrganizationService service,
    string assemblyName)
{
    IDictionary<Type, Entity[]> result = new Dictionary<Type, Entity[]>();
    
    // Retrieve the first PluginAssembly only
    var plugin = service.EntityByName<PluginAssembly>(assemblyName);
    if (plugin is null)
    {
        // Nothing found for PluginAssembly with given name, return empty dictionary instead?
        return null;
    }
    
    // Save the first PluginAssembly
    result.Add(typeof(PluginAssembly), new []{plugin});
    
    // Retrieve all associated PluginType entities
    var pluginTypes = service.GetChildren<PluginType>(plugin);
    result.Add(typeof(PluginType), pluginTypes.Cast<Entity>().ToArray());
    
    // Get all associated Steps of all PluginType entities
    var steps = pluginTypes.SelectMany(
        plugin => service.GetChildren<SdkMessageProcessingStep>(plugin));
    result.Add(typeof(SdkMessageProcessingStep), steps.Cast<Entity>().ToArray());
    
    var stepImages = steps.SelectMany(step =>
        service.GetChildren<SdkMessageProcessingStepImage>(step));
    result.Add(typeof(SdkMessageProcessingStepImage), stepImages.Cast<Entity>().ToArray());
    
    return result;
}

Wrap Guid as EntityReference

Though it is possible to parse the entity type to get properties of EntityReference type, a static dictionary is much cheaper:

private static Dictionary<Type, string[]> 
    TypeEntityReferenceAttributes = new Dictionary<Type, string[]>()
{
    {typeof(PluginAssembly), new[] {"organizationid"}},
    {typeof(PluginType), new[] {"pluginassemblyid", "solutionid", "organizationid"}},
    {typeof(SdkMessage), new[] {"organizationid"}},
    {typeof(SdkMessageFilter), new[] {"organizationid", "sdkmessageid"}},
    {typeof(SdkMessageProcessingStepImage), 
    new[] {"organizationid", "sdkmessageprocessingstepid"}},
    {typeof(SdkMessageProcessingStep), new[] 
    { "eventhandler", "organizationid", "impersonatinguserid",
        "plugintypeid", "sdkmessagefilterid", 
        "sdkmessageid", "sdkmessageprocessingstepsecureconfigid"}},
};

Then one or multiple entities can be associated by creating EntityReferences:

public static TEntity Associate<TEntity>(this TEntity entity, params Entity[] associations)
    where TEntity : Entity
{
    var entityReferenceAttributeNames = TypeEntityReferenceAttributes[typeof(TEntity)];
    foreach (var association in associations)
    {
        if (entity == null) continue;
        string associationIdName = $"{association.LogicalName}id";
        
        //Some Guid shall be wrapped as EntityReference
        entity[associationIdName] = entityReferenceAttributeNames.Contains(associationIdName)
            ? new EntityReference(association.LogicalName, 
              (Guid)association[associationIdName])
            : association[associationIdName];
    }
    
    return entity;
}

Update Plugin

Update plugin DLL can basically follow this pattern to get PluginAssembly by name, update it and its PluginTypes:

        public static PluginAssembly UpdatePluginAssembly
        (this IOrganizationService service, string assemblyPath, string solutionName = null)
        {
            var pluginAssembly = AssemblyHelper.LoadPluginAssembly(assemblyPath);
            string assemblyName = pluginAssembly.Name;
            
            // Retrieve solution entity by name, it can be null if solutionName 
            // is null or no such solution
            var solution = solutionName is null ? null :
                service.EntityByName("solution", solutionName);
            pluginAssembly.Associate(solution);
            
            // Ensure the pluginAssembly has been registered, otherwise throw Exception
            var existingEntity = service.EntityByName<PluginAssembly>(assemblyName);
    if (existingEntity == null)
    {
        throw new InvalidOperationException
              ($"Cannot found PluginAssembly with name {assemblyName}");
    }
    
    // Update pluginAssembly by inherit missing attributes from the existingEntity
    pluginAssembly = pluginAssembly.Inherit(existingEntity);
    
    //Update pluginAssembly and print out changes
    service.Update(pluginAssembly);
    pluginAssembly = service.EntityByName<PluginAssembly>(pluginAssembly.Name);
    EntityHelper.PrintAttributeDifferences(existingEntity, pluginAssembly);
    
    // Update PluginTypes one by one for simplicity
    var existingPluginTypes = service.EntitiesByAssemblyName
        <PluginType>(assemblyName).ToDictionary(p => p.Name, p => p);
    var freshPluginTypes = AssemblyHelper.LoadPluginTypes(assemblyPath).ToDictionary
                           (p => p.Name, p => p);
    
    string[] allPluginNames = existingPluginTypes.Keys.Concat
                              (freshPluginTypes.Keys).Distinct().ToArray();
    foreach (var pluginName in allPluginNames)
    {
        // Get the current plugin entity, associate it to PluginAssembly and Solution
        var plugin = freshPluginTypes[pluginName].Associate(pluginAssembly, solution);
        
        if (!freshPluginTypes.ContainsKey(pluginName))
        {
            //Not included in the fresh PluginAssembly, delete it
            service.Delete(plugin.TypeName, (Guid)plugin.PluginTypeId);
            Debug.WriteLine($"PluginType deleted: {plugin.Name}{{{plugin.PluginTypeId}}}");
        }
        else if (!existingPluginTypes.ContainsKey(pluginName))
        {
            service.RegisterPluginType(plugin);
        }
        else
        {
            var updatedPlugin = service.UpdatePluginType
                                (plugin, existingPluginTypes[pluginName]);
            Debug.WriteLine($"PluginType updated: 
                           {updatedPlugin.Name}{{{updatedPlugin.PluginTypeId}}}");
        }
    }
    
    return pluginAssembly;
}

Validation

To validate that the above Plugin registration works, I followed the tutorial: Write and register a plug-in:

1. Write a plug-in.
2. Register the DLL by running a unit test.
3. Register step manually.
4. Update and rebuild the plug-in code.
5. Update the DLL by same unit test.
6. Validate the changes applied in D365.

Further Thoughts

It is still strange for me when Microsoft has not include all assets of a solution into source control.

For example, SdkMessageProcessingStep entities could be defined as:

• Either as JSON or XML since they can be saved in SQL table.
• Or defined as strong-typed models with Attribute or Properties of the PluginType entity.

If that happened, then contents within the solution.zip are not needed in the source repository, the versions of projects can always be easy to update, DLL uploaded with dependencies Updated/Created to the target environment with instruction of metadata or environment variables will save a lot of time and problems.

History

• 27^th February, 2022: Initial version