JsonPathToModel project

Introduction

Initially, the JsonPathToModel was created as a part of the BlazorForms project https://github.com/ProCodersPtyLtd/BlazorForms and was used to implement two-way bindings between Blazor controls and the memory model.

A few projects allow using JSONPath to extract values from JSON, but working with .NET memory models was impossible, so we developed it.

The latest version of JsonPathToModel is implemented using Reflection and covered by dozens of unit tests. It supports only public properties and straightforward dot notation. It also supports Array, List, and Dictionary collections.

One of the features we can add is to support the "fast reflection" technique to use Sigil.Emit to generate IL byte code for each JSONPath, which should increase the performance to 100 times compared to the Reflection. 

Background

JSONPath is a query language for JSON, similar to XPath for XML. It allows you to select and extract data from a JSON document. You use a JSONPath expression to traverse the path to an element in the JSON structure. You start at the root node or element, represented by $, and reach the required element in the JSON structure to extract data from it.

Using the code

There are some examples of how to extract values from complex in-memory graphs.

I used the AutoFixture package to generate the model and populate all its properties including collections.

Now I create a JsonPathModelNavigator instance and can execute GetValue or SelectValues methods to extract model values using JSONPath expressions like "$.Person.PrimaryContact.Email.Value".

using AutoFixture;
using JsonPathToModel.Tests.ModelData;

namespace JsonPathToModel.Tests;

public class SampleClientModelTests
{
    [Fact]
    public void GetValue_ShouldReturn_ForLongPath()
    {
        var navi = new JsonPathModelNavigator();
        var model = GenerateSampleClient();
        var expected = model.Person.PrimaryContact.Email.Value;

        var result = navi.GetValue(model, "$.Person.PrimaryContact.Email.Value");
        Assert.True(result.IsSuccess);
        Assert.Equal(expected, result.Value);
    }

    [Fact]
    public void GetValue_ShouldReturn_Array()
    {
        var navi = new JsonPathModelNavigator();
        var model = GenerateSampleClient();

        // property notation
        var result = navi.GetValue(model, "$.Roles");

        Assert.True(result.IsSuccess);
        Assert.NotNull(result.Value as Role[]);

        // explicit array notation
        result = navi.GetValue(model, "$.Roles[]");

        Assert.True(result.IsSuccess);
        Assert.NotNull(result.Value as Role[]);

        // explicit all items notation
        result = navi.GetValue(model, "$.Roles[*]");

        Assert.True(result.IsSuccess);
        Assert.NotNull(result.Value as Role[]);
    }

    [Fact]
    public void GetValue_ShouldReturn_ArrayItem()
    {
        var navi = new JsonPathModelNavigator();
        var model = GenerateSampleClient();
        var result = navi.GetValue(model, "$.Roles[1]");

        Assert.True(result.IsSuccess);
        Assert.Equal(model.Roles[1], result.Value as Role);
    }

    [Fact]
    public void GetValue_ShouldReturn_ArrayValue()
    {
        var navi = new JsonPathModelNavigator();
        var model = GenerateSampleClient();
        var result = navi.GetValue(model, "$.Roles[1].Name");

        Assert.True(result.IsSuccess);
        Assert.Equal(model.Roles[1].Name, result.Value as string);
    }

    [Fact]
    public void SelectValues_ShouldReturn_ArrayValues()
    {
        var navi = new JsonPathModelNavigator();
        var model = GenerateSampleClient();
        var result = navi.SelectValues(model, "$.Roles[*].Name");

        Assert.True(result.IsSuccess);

        Assert.Collection(result.Value,
            e => Assert.Equal(model.Roles[0].Name, e as string),
            e => Assert.Equal(model.Roles[1].Name, e as string),
            e => Assert.Equal(model.Roles[2].Name, e as string));
    }

    private SampleClientModel GenerateSampleClient(string id = "1")
    {
        var client = new Fixture()
            .Build<SampleClientModel>()
            .With(p => p.Id, id)
            .Create();

        return client;
    }
}

and the model code is here:

namespace JsonPathToModel.Tests.ModelData;

public class SampleClientModel
{
    public string Id { get; set; }
    public string? BusinessId { get; set; }
    public Person Person { get; set; }
    public Role[] Roles { get; set; }
    public bool IsDeleted { get; set; }
}

public class Person
{
    public string Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public List<Email> Emails { get; set; } = [];
    public List<Address> Addresses { get; set; } = [];
    public Contact PrimaryContact { get; set; }
}

public class Contact
{
    public Email Email { get; set; }
    public Phone Phone { get; set; }
}

public class Email
{
    public string Value { get; set; }
}
public class Phone
{
    public string Value { get; set; }
}

public class Address
{
    public string AddressType { get; set; }
    public string StreetNumber { get; set; }
    public string StreetType { get; set; }
    public string[] AddressLine { get; set; }
    public string CityName { get; set; }
    public string PostalCode { get; set; }
    public string Country { get; set; }
    public string State { get; set; }
}

public class Holding
{
    public string Code { get; set; }
    public string MarketCode { get; set; }
    public string Name { get; set; }
    public decimal Amount { get; set; }
}

public class Role
{
    public string RoleId { get; set; }
    public string Name { get; set; }
}

You can download the project from Github https://github.com/Binoculars-X/JsonPathToModel and run all tests locally.

Please read the README.md file for the JSONPath expression limitations.

The NuGet package you can find here https://www.nuget.org/packages/JsonPathToModel/

Performance

In the latest version 1.2.6, we have improved performance, changing String.Split parsing to a well-known Tokenizer pattern. We also implemented the "fast reflection" technique suggested by Nick Chapsas YouTube blogger (my respect bro!) and improved performance of a typical straight dot operation "$.Person.FirstName" from 800 ns to 25 ns, and collection operation "$.Person.Emails[1].Value" from 800 ns to 120 ns.

You can execute GetValue/SetValue with the straight dot JSONPath at the speed of 40 million per second!

You can play with performance measurement on your local using the BenchmarkConsoleApp app JsonPathToModelBenchmarks class. Remember you need to run this app in the "Release" build.

Use cases and how to use it

Binding UI Control and Model

When you build a Blazor dynamic page that can show controls defined by the User, stored in a database, or retrieved from a third party and you need to bind this dynamic content to an in-memory Model, a flexible and elegant solution will be to use JSONPath to extract values from the Model to render in the controls, and when a control value is changed, set the changed value back to the Model.

We use this approach in the BlazorForms project, which allows us to generate dynamic pages on the fly.

protected override async Task OnInitializedAsync()
{
    Value = _jsonPathModelNavigator.GetValue(Model, FieldBinding).ToString();
}

async Task ValueChanged(string value)
{
    _jsonPathModelNavigator.SetValue(Model, FieldBinding, value);
}

In this example when a Blazor control is rendered it uses a Value that is extracted from the Model using JSONPath FieldBinding. When the control is changed it value is propagated back to the Model. The FieldBinding can be configured and supplied as a parameter, so the control code can be generic.

Importing external data and mapping it to your Model 

Another area where JSONPathToModel can be useful is if you have a system with a Model that is read and stored in your database, and you are implementing a tool to import data from different clients' schemas. In this case, you can build a generic data importer that receives data mappings as input and uses it to map the client's CSV files to your Model classes.

The mappings can look like a map between a CSV file position and a Model JSONPath

#person-map.CSV

0, $.Id

1, $.FirstName

3, $.LastName

7, $.DateOfBirth

8, $.Email

Having such mappings your generic code can easily map imported data to your Model objects

private Person MapPerson(string[] csv, List<Mapping> mappings)
{
    var person = new Person();

    foreach (var mapping in mappings)
    {
        var result = _jsonPathModelNavigator.SetValue(person, mapping.JsonPath, csv[mapping.Position]);

        if (result.IsFailed)
        {
            throw new MappingException(result.Errors);
        }
    }

    return person;
}

This code accepts CSV data and a list of mapping objects, retrieved from the person-map.CSV file. Iterating through mappings, it sets values from csv array to the person object properties.

Attaching Validation Rules to the Model without hardcoding for each property

For large models that have hundreds of fields, it is not an easy task to implement validation logic.

If you try to solve it using a straightforward approach you will need to hardcode every rule for each field, for example, for the required/mandatory fields you will need to copy-paste the same code that checks if a field is empty or null and generate error message providing the field name and other context details; for emails, you will need to hardcode regex validation and copy-paste it again for each email fields in your model.

One of the approaches is to use Attributes to specify rules attached to the Model fields. However, this will complicate the Model definitions and will couple the Model and the Rules Engine.

The elegant way to implement this use case is to define rules metadata separately as a list of objects that keep a reference to each field and the rules it should validate. Then the Rule Engine can extract each field value and validate it by the defined rules.

Initially, the rules metadata can be defined in code and later stored in a database

var p = new RulesMetadata()
{
    Person =
    [
        new()
        {
            FieldName = "First Name",
            Presentation = PresentationType.Mandatory,
            DataType = FieldDataType.FreeText,
            JsonPath = "$.Customer.Person.FirstName",
            DataAccess = new()
            {
                View = DataAccessRole.User,
                Modify = DataAccessRole.Admin
            }
        },
        new()
        {
            FieldName = "Last Name",
            Presentation = PresentationType.Mandatory,
            DataType = FieldDataType.FreeText,
            JsonPath = "$.Customer.Person.LastName",
            DataAccess = new()
            {
                View = DataAccessRole.User,
                Modify = DataAccessRole.Admin
            }
        },
        new()
        {
            FieldName = "Email",
            Presentation = PresentationType.Optional,
            DataType = FieldDataType.FreeText,
            RegExPattern = "^[\\w-\\.]+@([\\w-]+\\.)+[\\w-]{2,4}$",
            JsonPath = "$.Customer.Email[*].Value",
            DataAccess = new()
            {
                View = DataAccessRole.User,
                Modify = DataAccessRole.Admin
            }
        },
    ]
};

As you can see JsonPath stores string path to the Model field.

Now you can apply rules one by one in a generic way

public ValidationResult ValidateModel(DataModel model, RulesMetadata config)
{
    var result = new ValidationResult();
    var fields = GetConfigValidatableFields(config);

    foreach (var field in fields)
    {
        var contexts = CreateRuleContexts(model, field);

        foreach (var context in contexts)
        {
            // run rules on each field for each context
            foreach (var rule in _rules)
            {
                var validationResult = rule.Validate(context);
                result.AddValidationRuleResult(validationResult);
            }
        }
    }

    return result;
}

private List<ValidationRuleContext> CreateRuleContexts(DataModel model, ValidatableField field)
{
    var values = _jsonPathNavigator.SelectValues(model, field.JsonPath);
    var result = values.Select(value => new ValidationRuleContext(model, field, value)).ToList();
    return result;
}

Rules code can be

public class MandatoryFieldNotEmpty : ValidationRuleBase
{
    public override ValidationRuleResult Validate(ValidationRuleContext context)
    {
        if (context.Field.Presentation == PresentationType.Mandatory &&
            (context.Value == null || string.IsNullOrWhiteSpace(context.Value.ToString())))
        {
            return Error($"Mandatory field is empty", context);
        }

        return Ok();
    }
}

and

public class RegExPatternNotMatched : ValidationRuleBase
{
    public override ValidationRuleResult Validate(ValidationRuleContext context)
    {
        if (context.Field.RegExPattern != null && context.Value != null)
        {
            var r = new Regex(context.Field.RegExPattern, RegexOptions.IgnoreCase);
            Match m = r.Match(context.Value.ToString());

            if (!m.Success)
            {
                return Error($"The field value does not match the validation pattern", context);
            }
        }

        return Ok();
    }
}

Thank you for reading.