ChoETL is an open source ETL (extract, transform and load) framework for .NET. It is a code based library for extracting data from multiple sources, transforming, and loading into your very own data warehouse in .NET environment. You can have data in your data warehouse in no time.
Table of Contents
ChoETL is an open source ETL (extract, transform and load) framework for .NET. It is a code based library for extracting data from multiple sources, transforming, and loading into your very own data warehouse in .NET environment. You can have data in your data warehouse in no time.
This article talks about using ChoJSONReader component offered by ChoETL framework. It is a simple utility class to extract JSON data from file / source to objects.
Features
- Uses NewtonSoft.JSON parser under the hood, parses JSON file in seconds and also handle large file without any memory issues.
- Stream based parsers allow for ultimate performance, low resource usage, and nearly unlimited versatility scalable to any size data file, even tens or hundreds of gigabytes.
- Event based data manipulation and validation allows total control over the flow of data during the bulk insert process.
- Exposes
IEnumarable list of objects - which is often used with LINQ query for projection, aggregation and filtration etc. - Supports deferred reading.
- Supports processing files with culture specific date, currency and number formats.
- Recognizes a wide variety of date, currency, enum, boolean and number formats when reading files.
- Provides fine control of date, currency, enum, boolean, number formats when writing files.
- Detailed and robust error handling, allowing you to quickly find and fix the problems.
This framework library is written in C# using .NET 4.5 Framework / .NET core 2.x.
- Open VS.NET 2013 or higher.
- Create a sample VS.NET (.NET Framework 4.5) Console Application project.
- Install
ChoETL via Package Manager Console using Nuget Command based on the .NET environment:
Install-Package ChoETL.JSONInstall-Package ChoETL.JSON.NETStandard
- Use the
ChoETL namespace
Let's begin by looking into a simple example of reading JSON file having three fields:
Listing 3.1 Sample JSON data file (emp.json)
[
{
"Id": 1,
"Name": "Jeanette"
},
{
"Id": 2,
"Name": "Giavani"
}
]
There are number of ways you can get the JSON file parsing started with minimal setup.
It is the zero config, quick way to load a JSON file in no time. No POCO object is required. The sample code below shows how to load the file.
Listing 3.1.1 Load JSON file using iterator
foreach (dynamic e in new ChoJSONReader("emp.json"))
{
Console.WriteLine(e.Id);
Console.WriteLine(e.Name);
}
Sample fiddle: https://dotnetfiddle.net/aGzea1
Listing 3.1.2 Load JSON file using loop
var reader = new ChoJSONReader("emp.json");
dynamic rec;
while ((rec = reader.Read()) != null)
{
Console.WriteLine(rec.Id);
Console.WriteLine(rec.Name);
}
Sample fiddle: https://dotnetfiddle.net/wy3g2L
This is another zero config way to parse and load JSON file using POCO class. First, define a simple data class to match the underlying JSON file layout.
Listing 3.2.1 Simple POCO entity class
public partial class EmployeeRec
{
public int Id { get; set; }
public string Name { get; set; }
}
In the above, the class defines two properties matching the sample JSON file template.
Listing 3.2.2 Load JSON file
foreach (var e in new ChoJSONReader<EmployeeRec>("emp.json"))
{
Console.WriteLine(e.Id);
Console.WriteLine(e.Name);
}
Sample fiddle: https://dotnetfiddle.net/wA6aws
In this model, we define the JSON configuration with all the necessary parsing parameters along with JSON fields matching with the underlying JSON file.
Listing 3.3.1 Define JSON configuration
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Id"));
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));
In the above, the class defines two properties matching the sample JSON file template.
Listing 3.3.2 Load JSON file without POCO object
foreach (dynamic e in new ChoJSONReader("emp.json", config))
{
Console.WriteLine(e.Id);
Console.WriteLine(e.Name);
}
Sample fiddle: https://dotnetfiddle.net/DCTYbS
Listing 3.3.3 Load JSON file with POCO object
foreach (var e in new ChoJSONReader<EmployeeRec>("emp.json", config))
{
Console.WriteLine(e.Id);
Console.WriteLine(e.Name);
}
Sample fiddle: https://dotnetfiddle.net/WpzPvH
This is the combined approach to define POCO entity class along with JSON configuration parameters decorated declaratively. Id is required field and Name is optional value field with default value "XXXX". If Name is not present, it will take the default value.
Listing 3.4.1 Define POCO Object
public class EmployeeRec
{
[ChoJSONRecordField]
[Required]
public int Id
{
get;
set;
}
[ChoJSONRecordField]
[DefaultValue("XXXX")]
public string Name
{
get;
set;
}
public override string ToString()
{
return "{0}. {1}".FormatString(Id, Name);
}
}
The code above illustrates about defining POCO object to carry the values of each record line in the input file. First thing defines property for each record field with ChoJSONRecordFieldAttribute to qualify for JSON record mapping. JSONPath is a optional property. If not specified, framework automatically discover and load the values from JSON property. Id is decorated with RequiredAttribute, if the value is missing, it will throw exception. Name is given default value using DefaultValueAttribute. It means that if the Name JSON field contains empty value in the file, it will be defaulted to 'XXXX' value.
It is very simple and ready to extract JSON data in no time.
Listing 3.4.2 Main Method
foreach (var e in new ChoJSONReader<EmployeeRec>("emp.json"))
{
Console.WriteLine(e.Id);
Console.WriteLine(e.Name);
}
Sample fiddle: https://dotnetfiddle.net/GGd9uJ
We start by creating a new instance of ChoJSONReader object. That's all. All the heavy lifting of parsing and loading JSON data stream into the objects is done by the parser under the hood.
By default, ChoJSONReader discovers and uses default configuration parameters while loading JSON file. These can be overridable according to your needs. The following sections will give details about each configuration attributes.
It is as easy as setting up POCO object match up with JSON file structure, you can read the whole file as enumerable pattern. It is a deferred execution mode, but take care while making any aggregate operation on them. This will load the entire file records into memory.
Listing 4.1 Read JSON File
foreach (var e in new ChoJSONReader<EmployeeRec>("emp.json"))
{
Console.WriteLine(e.Id);
Console.WriteLine(e.Name);
}
or:
Listing 4.2 Read JSON file stream
foreach (var e in new ChoJSONReader<EmployeeRec>(textReader))
{
Console.WriteLine(e.Id);
Console.WriteLine(e.Name);
}
This model keeps your code elegant, clean, easy to read and maintain. Also leverages LINQ extension methods to perform grouping, joining, projection, aggregation, etc.
Listing 4.3 Using LINQ
var list = (from o in new ChoJSONReader<EmployeeRec>("emp.json")
where o.Name != null && o.Name.StartsWith("R")
select o).ToArray();
foreach (var e in list)
{
Console.WriteLine(e.Id);
Console.WriteLine(e.Name);
}
It is as easy as setting up POCO object match up with JSON file structure, you can read the whole file as enumerable pattern.
Listing 5.1 Read JSON File
var reader = new ChoJSONReader<EmployeeRec>("emp.json");
dynamic rec = (object)null;
while ((rec = reader.Read()) != null)
{
Console.WriteLine(rec.Id);
Console.WriteLine(rec.Name);
}
Sample fiddle: https://dotnetfiddle.net/ZTgdbV
Using ChoJSONRecordObjectAttribute, you can customize the POCO entity object declaratively.
Listing 6.1 Customizing POCO object for each record
[ChoJSONRecordObject]
public class EmployeeRec
{
[ChoJSONRecordField]
public int Id { get; set; }
[ChoJSONRecordField]
[Required]
[DefaultValue("ZZZ")]
public string Name { get; set; }
}
Here are the available attributes to carry out customization of JSON load operation on a file.
JSONPath - Optional. JSONPath expression used to pick the elements to load. If not specified, the member value will be discovered and loaded automatically.CultureName - The culture name (example, en-US, en-GB) is used to read and write JSON data.Encoding - The encoding of the JSON file.ColumnCountStrict - This flag indicates if an exception should be thrown if reading an expected field is missing.ErrorMode - This flag indicates if an exception should be thrown if reading and an expected field is failed to load. This can be overridden per property. Possible values are:
IgnoreAndContinue - Ignore the error, record will be skipped and continue with next.ReportAndContinue - Report the error to POCO entity if it is of IChoNotifyRecordRead typeThrowAndStop - Throw the error and stop the execution
IgnoreFieldValueMode - A flag to let the reader know if a record should be skipped when reading if it's empty / null. This can be overridden per property. Possible values are:
Null - skipped if the record value is nullDBNull - N/AEmpty - skipped if the record value is emptyWhiteSpace - skipped if the record value contains only whitespaces
ObjectValidationMode - A flag to let the reader know about the type of validation to be performed with record object. Possible values are:
Off - No object validation performed (Default)MemberLevel - Validation performed at the time of each JSON property gets loaded with valueObjectLevel - Validation performed after all the properties are loaded to the POCO object
For each JSON field, you can specify the mapping in POCO entity property using ChoJSONRecordFieldAttribute. Only use this attribute if you want to use custom JSONPath to map to this field.
Listing 7.1 Customizing POCO object for JSON fields
public class EmployeeRec
{
[ChoJSONRecordField]
public int Id { get; set; }
[ChoJSONRecordField]
[Required]
[DefaultValue("ZZZ")]
public string Name { get; set; }
}
Here are the available members to add some customization to it for each property:
JSONPath - Optional. JSONPath expression uses a path notation for addressing parts of an JSON document. If not specified, ChoJSONReader automatically discovers and loads the value from JSON object matching field name.
It is the value used and set to the property when the JSON value is empty or whitespace (controlled via IgnoreFieldValueMode).
Any POCO entity property can be specified with default value using System.ComponentModel.DefaultValueAttribute.
It is the value used and set to the property when the JSON value failed to set. Fallback value only set when ErrorMode is either IgnoreAndContinue or ReportAndContinue.
Any POCO entity property can be specified with fallback value using ChoETL.ChoFallbackValueAttribute.
Most of the primitive types are automatically converted and set them to the properties. If the value of the JSON field can't automatically be converted into the type of the property, you can specify a custom / built-in .NET converters to convert the value. These can be either IValueConverter or TypeConverter converters.
There are couple of ways you can specify the converters for each field:
- Declarative Approach
- Configuration Approach
This model is applicable to POCO entity object only. If you have POCO class, you can specify the converters to each property to carry out necessary conversion on them. Samples below show the way to do it.
public class EmployeeRec
{
[ChoJSONRecordField]
[ChoTypeConverter(typeof(IntConverter))]
public int Id { get; set; }
[ChoJSONRecordField]
[Required]
[DefaultValue("ZZZ")]
public string Name { get; set; }
}
Listing 7.3.1.2 IntConverter implementation
public class IntConverter : IValueConverter
{
public object Convert(object value, Type targetType,
object parameter, CultureInfo culture)
{
return value;
}
public object ConvertBack(object value, Type targetType,
object parameter, CultureInfo culture)
{
return value;
}
}
In the example above, we defined custom IntConverter class. And showed how to use it with 'Id' JSON property.
This model is applicable to both dynamic and POCO entity object. This gives freedom to attach the converters to each property at runtime. This takes the precedence over the declarative converters on POCO classes.
Listing 7.3.2.2 Specifying TypeConverters
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
ChoJSONRecordFieldConfiguration idConfig = new ChoJSONRecordFieldConfiguration("Id");
idConfig.AddConverter(new IntConverter());
config.JSONRecordFieldConfigurations.Add(idConfig);
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));
In the above, we construct and attach the IntConverter to 'Id' field using AddConverter helper method in ChoJSONRecordFieldConfiguration object.
Likewise, if you want to remove any converter from it, you can use RemoveConverter on ChoJSONRecordFieldConfiguration object.
ChoJSONReader leverages both System.ComponentModel.DataAnnotations and Validation Block validation attributes to specify validation rules for individual fields of POCO entity. Refer to the MSDN site for a list of available DataAnnotations validation attributes.
Listing 7.4.1 Using validation attributes in POCO entity
[ChoJSONRecordObject]
public partial class EmployeeRec
{
[ChoJSONRecordField(FieldName = "id")]
[ChoTypeConverter(typeof(IntConverter))]
[Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
[ChoFallbackValue(1)]
public int Id { get; set; }
[ChoJSONRecordField(FieldName = "Name")]
[Required]
[DefaultValue("ZZZ")]
[ChoFallbackValue("XXX")]
public string Name { get; set; }
}
In the example above, used Range validation attribute for Id property. Required validation attribute to Name property. ChoJSONReader performs validation on them during load based on Configuration.ObjectValidationMode is set to ChoObjectValidationMode.MemberLevel or ChoObjectValidationMode.ObjectLevel.
Sometime, you may want to override the defined declarative validation behaviors that come with POCO class, you can do with Cinchoo ETL via configuration approach. The sample below shows the way to override them.
static void ValidationOverridePOCOTest()
{
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
var idConfig = new ChoJSONRecordFieldConfiguration("Id");
idConfig.Validators = new ValidationAttribute[] { new RequiredAttribute() };
config.JSONRecordFieldConfigurations.Add(idConfig);
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));
using (var parser = new ChoJSONReader<EmployeeRec>("emp.json", config))
{
object rec;
while ((rec = parser.Read()) != null)
{
Console.WriteLine(rec.ToStringEx());
}
}
}
In some cases, you may want to take control and perform manual self validation within the POCO entity class. This can be achieved by inheriting POCO object from IChoValidatable interface.
Listing 7.4.2 Manual validation on POCO entity
[ChoJSONRecordObject]
public partial class EmployeeRec : IChoValidatable
{
[ChoJSONRecordField(FieldName = "id")]
[ChoTypeConverter(typeof(IntConverter))]
[Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
[ChoFallbackValue(1)]
public int Id { get; set; }
[ChoJSONRecordField(FieldName = "Name")]
[Required]
[DefaultValue("ZZZ")]
[ChoFallbackValue("XXX")]
public string Name { get; set; }
public bool TryValidate(object target, ICollection<ValidationResult> validationResults)
{
return true;
}
public bool TryValidateFor(object target, string memberName,
ICollection<ValidationResult> validationResults)
{
return true;
}
public void Validate(object target)
{
}
public void ValidateFor(object target, string memberName)
{
}
}
The sample above shows how to implement custom self-validation in POCO object.
IChoValidatable interface exposes the below methods:
TryValidate - Validate entire object, return true if all validation passed. Otherwise return false.Validate - Validate entire object, throw exception if validation is not passed.TryValidateFor - Validate specific property of the object, return true if all validation passed. Otherwise return false.ValidateFor - Validate specific property of the object, throw exception if validation is not passed.
ChoJSONReader offers industry standard JSON parsing out of the box to handle most of the parsing needs. If the parsing is not handling any of the needs, you can use the callback mechanism offered by ChoJSONReader to handle such situations. In order to participate in the callback mechanism, you can use either of the following models:
- Using event handlers exposed by
ChoJSONReader via IChoReader interface. - Inheriting POCO entity object from
IChoNotifyRecordRead / IChoNotifyFileRead / IChoNotifyRecordFieldRead interfaces - Inheriting
DataAnnotation's MetadataType type object by IChoNotifyRecordRead / IChoNotifyFileRead / IChoNotifyRecordFieldRead interfaces. - Inheriting
IChoNotifyRecordFieldConfigurable / IChoNotifyRecordFieldConfigurable configuration interfaces
Note: Any exceptions raised out of these interface methods will be ignored.
IChoReader exposes the below events:
BeginLoad - Invoked at the begin of the JSON file loadEndLoad - Invoked at the end of the JSON file loadBeforeRecordLoad - Raised before the JSON record loadAfterRecordLoad - Raised after JSON record loadRecordLoadError - Raised when JSON record load errors outBeforeRecordFieldLoad - Raised before JSON field value loadAfterRecordFieldLoad - Raised after JSON field value loadRecordFieldLoadError - Raised when JSON field value errors out- SkipUntil - Raised before the JSON parsing kicks off to add custom logic to skip record lines.
- DoWhile - Raised during JSON parsing where you can add custom logic to stop the parsing.
IChoNotifyRecordRead exposes the below methods:
BeforeRecordLoad - Raised before the JSON record loadAfterRecordLoad - Raised after JSON record loadRecordLoadError - Raised when JSON record load errors out
IChoNotifyFileRead exposes the below methods:
BeginLoad - Invoked at the begin of the JSON file loadEndLoad - Invoked at the end of the JSON file loadSkipUntil - Raised before the JSON parsing kicks off to add custom logic to skip record lines.DoWhile - Raised during JSON parsing where you can add custom logic to stop the parsing.
IChoNotifyRecordFieldRead exposes the below methods:
BeforeRecordFieldLoad - Raised before JSON field value loadAfterRecordFieldLoad - Raised after JSON field value loadRecordFieldLoadError - Raised when JSON field value errors out
IChoNotifyRecordConfigurable exposes the below methods:
RecondConfigure - Raised for JSON record configuration
IChoNotifyRecordFieldConfigurable exposes the below methods:
RecondFieldConfigure - Raised for each JSON record field configuration
This is more direct and the simplest way to subscribe to the callback events and handle your odd situations in parsing JSON files. The downside is that code can't be reusable as you do by implementing IChoNotifyRecordRead with POCO record object.
The sample below shows how to use the BeforeRecordLoad callback method to skip lines stating with '%' characters.
Listing 10.1.1 Using ChoJSONReader callback events
static void IgnoreLineTest()
{
using (var parser = new ChoJSONReader("emp.json"))
{
parser.BeforeRecordLoad += (o, e) =>
{
if (e.Source != null)
{
e.Skip = !((JObject)e.Source).ContainsKey("Name");
}
};
foreach (var e in parser)
Console.WriteLine(e.Dump());
}
}
Likewise, you can use other callback methods as well with ChoJSONReader.
The sample below shows how to implement IChoNotifyRecordRead interface to direct POCO class.
Listing 10.2.1 Direct POCO callback mechanism implementation
[ChoJSONRecordObject]
public partial class EmployeeRec : IChoNotifyRecordRead
{
[ChoJSONRecordField(FieldName = "Id")]
[ChoTypeConverter(typeof(IntConverter))]
[Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
[ChoFallbackValue(1)]
public int Id { get; set; }
[ChoJSONRecordField(FieldName = "Name")]
[Required]
[DefaultValue("ZZZ")]
[ChoFallbackValue("XXX")]
public string Name { get; set; }
public bool AfterRecordLoad(object target, int index, object source)
{
throw new NotImplementedException();
}
public bool BeforeRecordLoad(object target, int index, ref object source)
{
throw new NotImplementedException();
}
public bool RecordLoadError(object target, int index, object source, Exception ex)
{
throw new NotImplementedException();
}
}
The sample below shows how to attach Metadata class to POCO class by using MetadataTypeAttribute on it.
Listing 10.2 MetaDataType based callback mechanism implementation
[ChoJSONRecordObject]
public class EmployeeRecMeta : IChoNotifyRecordRead
{
[ChoJSONRecordField(FieldName = "Id")]
[ChoTypeConverter(typeof(IntConverter))]
[Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
[ChoFallbackValue(1)]
public int Id { get; set; }
[ChoJSONRecordField(FieldName = "Name")]
[Required]
[DefaultValue("ZZZ")]
[ChoFallbackValue("XXX")]
public string Name { get; set; }
public bool AfterRecordLoad(object target, int index, object source)
{
throw new NotImplementedException();
}
public bool BeforeRecordLoad(object target, int index, ref object source)
{
throw new NotImplementedException();
}
public bool RecordLoadError(object target, int index, object source, Exception ex)
{
throw new NotImplementedException();
}
}
[MetadataType(typeof(EmployeeRecMeta))]
public partial class EmployeeRec
{
public int Id { get; set; }
public string Name { get; set; }
}
The sample below shows how to attach Metadata class for sealed or third party POCO class by using ChoMetadataRefTypeAttribute on it.
Listing 10.2.3 ChoMetaDataRefType based callback mechanism implementation
[ChoMetadataRefType(typeof(EmployeeRec))]
[ChoJSONRecordObject]
public class EmployeeRecMeta : IChoNotifyRecordRead
{
[ChoJSONRecordField(FieldName = "id")]
[ChoTypeConverter(typeof(IntConverter))]
[Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
[ChoFallbackValue(1)]
public int Id { get; set; }
[ChoJSONRecordField(FieldName = "Name")]
[Required]
[DefaultValue("ZZZ")]
[ChoFallbackValue("XXX")]
public string Name { get; set; }
public bool AfterRecordLoad(object target, int index, object source)
{
throw new NotImplementedException();
}
public bool BeforeRecordLoad(object target, int index, ref object source)
{
throw new NotImplementedException();
}
public bool RecordLoadError(object target, int index, object source, Exception ex)
{
throw new NotImplementedException();
}
}
public partial class EmployeeRec
{
public int Id { get; set; }
public string Name { get; set; }
}
This callback is invoked once at the beginning of the JSON file load. source is the JSON file stream object. In here, you have a chance to inspect the stream, return true to continue the JSON load. Return false to stop the parsing.
Listing 10.1.1 BeginLoad Callback Sample
public bool BeginLoad(object source)
{
StreamReader sr = source as StreamReader;
return true;
}
This callback invoked once at the end of the JSON file load. source is the JSON file stream object. In here, you have a chance to inspect the stream, do any post steps to be performed on the stream.
Listing 10.2.1 EndLoad Callback Sample
public void EndLoad(object source)
{
StreamReader sr = source as StreamReader;
}
This callback invoked before each JSON node in the JSON file is loaded. target is the instance of the POCO record object. index is the JObject node index in the file. source is the JSON record object. In here, you have a chance to inspect the object, and override it with new values if want to.
TIP: If you want to skip the JObject from loading, set the source to null.
Return true to continue the load process, otherwise return false to stop the process.
Listing 10.5.1 BeforeRecordLoad Callback Sample
public bool BeforeRecordLoad(object target, int index, ref object source)
{
JObject obj = source as JObject;
return true;
}
This callback invoked after each JObject node in the JSON file is loaded. target is the instance of the POCO record object. index is the JObject node index in the file. source is the JSON record object. In here, you have a chance to do any post step operation with the JObject line.
Return true to continue the load process, otherwise return false to stop the process.
Listing 10.6.1 AfterRecordLoad Callback Sample
public bool AfterRecordLoad(object target, int index, object source)
{
JObject obj = source as JObject;
return true;
}
This callback is invoked if error is encountered while loading JObject node. target is the instance of the POCO record object. index is the JObject node index in the file. source is the JObject node. ex is the exception object. In here, you have chance to handle the exception. This method is invoked only when Configuration.ErrorMode is ReportAndContinue.
Return true to continue the load process, otherwise return false to stop the process.
Listing 10.7.1 RecordLoadError Callback Sample
public bool RecordLoadError(object target, int index, object source, Exception ex)
{
JObject obj = source as JObject;
return true;
}
This callback invoked before each JSON record field is loaded. target is the instance of the POCO record object. index is the JObject node index in the file. propName is the JSON record property name. value is the JSON field value. In here, you have a chance to inspect the JSON record property value and perform any custom validations, etc.
Return true to continue the load process, otherwise return false to stop the process.
Listing 10.8.1 BeforeRecordFieldLoad Callback Sample
public bool BeforeRecordFieldLoad(object target, int index, string propName, ref object value)
{
return true;
}
This callback is invoked after each JSON record field is loaded. target is the instance of the POCO record object. index is the JObject node index in the file. propName is the JSON record property name. value is the JSON field value. Any post field operation can be performed here, like computing other properties, validations, etc.
Return true to continue the load process, otherwise return false to stop the process.
Listing 10.9.1 AfterRecordFieldLoad Callback Sample
public bool AfterRecordFieldLoad(object target, int index, string propName, object value)
{
return true;
}
This callback is invoked when error is encountered while loading JSON record field value. target is the instance of the POCO record object. index is the JObject node index in the file. propName is the JSON record property name. value is the JSON field value. ex is the exception object. In here, you have a chance to handle the exception. This method is invoked only after the below two sequences of steps have been performed by the ChoJSONReader.
ChoJSONReader looks for FallbackValue value of each JSON property. If present, it tries to assign its value to it.- If the
FallbackValue value is not present and the Configuration.ErrorMode is specified as ReportAndContinue., this callback will be executed.
Return true to continue the load process, otherwise return false to stop the process.
Listing 10.10.1 RecordFieldLoadError Callback Sample
public bool RecordFieldLoadError(object target, int index, string propName,
object value, Exception ex)
{
return true;
}
This callback is invoked at the start of the JSON parsing with custom logic to skip nodes. index is the JObject node index in the file.
Return true to skip the line, otherwise return false.
Listing 10.11.1 SkipUntil Callback Sample
public bool SkipUntil(long index, object source)
{
return false;
}
This callback is invoked at the start of the JSON parsing with custom logic to skip nodes. index is the JObject node index in the file.
Return true to stop the parsing, otherwise return false.
Listing 10.12.1 DoWhile Callback Sample
public bool DoWhile(long index, object source)
{
return false;
}
ChoJSONReader automatically detects and loads the configured settings from POCO entity. At runtime, you can customize and tweak these parameters before JSON parsing. ChoJSONReader exposes Configuration property, it is of ChoJSONRecordConfiguration object. Using this property, you can customize them.
Listing 10.1 Customizing ChoJSONReader at run-time
class Program
{
static void Main(string[] args)
{
using (var parser = new ChoJSONReader<EmployeeRec>("emp.json"))
{
object row = null;
parser.Configuration.JSONPath = "$";
while ((row = parser.Read()) != null)
Console.WriteLine(row.ToString());
}
}
ChoJSONReader exposes AsDataReader helper method to retrieve the JSON records in .NET datareader object. DataReader are fast-forward streams of data. This datareader can be used in few places like bulk coping data to database using SqlBulkCopy, loading disconnected DataTable, etc.
Listing 11.1 Reading as DataReader sample
static void AsDataReaderTest()
{
using (var parser = new ChoJSONReader<EmployeeRec>("emp.json"))
{
IDataReader dr = parser.AsDataReader();
while (dr.Read())
{
Console.WriteLine("Id: {0}, Name: {1}", dr[0], dr[1]);
}
}
}
Sample fiddle: https://dotnetfiddle.net/yZrqwK
ChoJSONReader exposes AsDataTable helper method to retrieve the JSON records in .NET DataTable object. It then can be persisted to disk, displayed in grid/controls or stored in memory like any other object.
Listing 12.1 Reading as DataTable sample
static void AsDataTableTest()
{
using (var parser = new ChoJSONReader<EmployeeRec>("emp.json"))
{
DataTable dt = parser.AsDataTable();
foreach (DataRow dr in dt.Rows)
{
Console.WriteLine("Id: {0}, Name: {1}", dr[0], dr[1]);
}
}
}
Sample fiddle: https://dotnetfiddle.net/JYpII7
So far, the article explained about using ChoJSONReader with POCO object. ChoJSONReader also supports loading JSON file without POCO object. It leverages .NET dynamic feature. The sample below shows how to read JSON stream without POCO object.
If you have JSON file, you can parse and load the file with minimal/zero configuration.
The sample below shows it:
Listing 13.1 Loading JSON file
class Program
{
static void Main(string[] args)
{
dynamic row;
using (var parser = new ChoJSONReader("emp.json"))
{
while ((row = parser.Read()) != null)
{
Console.WriteLine(row.Id);
}
}
}
}
Sample fiddle: https://dotnetfiddle.net/3Y6OMn
The above example automatically discovers the JSON object members and parses the file.
You can override the default behavior of discovering fields automatically by adding field configurations manually and pass it to ChoJSONReader for parsing file.
The sample shows how to do it:
Listing 13.3 Loading JSON file with configuration
class Program
{
static void Main(string[] args)
{
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Id"));
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));
dynamic row;
using (var parser = new ChoJSONReader("emp.json", config))
{
while ((row = parser.Read()) != null)
{
Console.WriteLine(row.Name);
}
}
}
}
Sample fiddle: https://dotnetfiddle.net/nVGNi6
To completely turn off the auto field discovery, you will have to set ChoJSONRecordConfiguration.AutoDiscoverColumns to false.
It is the value used and set to the property when the JSON value is empty or whitespace (controlled via IgnoreFieldValueMode).
Any POCO entity property can be specified with default value using System.ComponentModel.DefaultValueAttribute.
For dynamic object members or to override the declarative POCO object member's default value specification, you can do so through configuration as shown below:
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Id"));
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name")
{ DefaultValue = "NoName" })
It is the value used and set to the property when the JSON value failed to set. Fallback value only set when ErrorMode is either IgnoreAndContinue or ReportAndContinue.
Any POCO entity property can be specified with fallback value using ChoETL.ChoFallbackValueAttribute.
For dynamic object members or to override the declarative POCO object member's fallback values, you can do through configuration as shown below:
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Id"));
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name")
{ FallbackValue = "Tom" });
In the typeless dynamic object model, the reader reads individual field value and populates them to dynamic object members in 'string' value. If you want to enforce the type and do extra type checking during load, you can do so by declaring the field type at the field configuration.
Listing 8.5.1 Defining FieldType
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Id")
{ FieldType = typeof(int) });
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));
The above sample shows to define field type as 'int' to 'Id' field. This instructs the ChoJSONReader to parse and convert the value to integer before assigning to it. This extra type safety alleviates the incorrect values being loaded to object while parsing.
Most of the primitive types are automatically converted and set to the properties by ChoJSONReader. If the value of the JSON field can't automatically be converted into the type of the property, you can specify a custom / built-in .NET converters to convert the value. These can be either IValueConverter or TypeConverter converters.
In the dynamic object model, you can specify these converters via configuration. See the below example on the approach taken to specify type converters for JSON fields.
Listing 13.4.1 Specifying TypeConverters
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
ChoJSONRecordFieldConfiguration idConfig = new ChoJSONRecordFieldConfiguration("Id");
idConfig.AddConverter(new IntConverter());
config.JSONRecordFieldConfigurations.Add(idConfig);
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));
In the above, we construct and attach the IntConverter to 'Id' field using AddConverter helper method in ChoJSONRecordFieldConfiguration object.
Likewise, if you want to remove any converter from it, you can use RemoveConverter on ChoJSONRecordFieldConfiguration object.
ChoJSONReader leverages both System.ComponentModel.DataAnnotations and Validation Block validation attributes to specify validation rules for individual JSON fields. Refer to the MSDN site for a list of available DataAnnotations validation attributes.
Listing 13.5.1 Specifying Validations
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
ChoJSONRecordFieldConfiguration idConfig = new ChoJSONRecordFieldConfiguration("Id");
idConfig.Validators = new ValidationAttribute[] { new RangeAttribute(0, 100) };
config.JSONRecordFieldConfigurations.Add(idConfig);
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));
In the example above, we used Range validation attribute for Id property. ChoJSONReader performs validation on them during load based on Configuration.ObjectValidationMode is set to ChoObjectValidationMode.MemberLevel or ChoObjectValidationMode.ObjectLevel.
P.S.: Self validation NOT supported in Dynamic object model
If you already have existing sealed POCO object or the object is in 3rd party library, we can use them with ChoJSONReader.
Listing 14.1 Existing sealed POCO Object
public sealed class ThirdPartyRec
{
public int Id
{
get;
set;
}
public string Name
{
get;
set;
}
}
Listing 14.2 Consuming JSON file
class Program
{
static void Main(string[] args)
{
using (var parser = new ChoJSONReader<ThirdPartyRec>("emp.json"))
{
object row = null;
while ((row = parser.Read()) != null)
Console.WriteLine(row.ToString());
}
}
}
In this case, ChoJSONReader reverse discovers the JSON fields from the JSON file and loads the data into POCO object. If the JSON file structure and POCO object matches, the load will succeed with populating all corresponding data to its properties. In case the property is missing for any JSON field, ChoJSONReader silently ignores them and continue on with the rest.
You can override this behavior by setting ChoJSONRecordConfiguration.ThrowAndStopOnMissingField property to false. In this case, the ChoJSONReader will throw ChoMissingRecordFieldException exception if a property is missing for a JSON field.
ChoJSONReader throws different types of exceptions in different situations.
ChoParserException - JSON file is bad and parser not able to recover.ChoRecordConfigurationException - Any invalid configuration settings are specified, this exception will be raised.ChoMissingRecordFieldException - A property is missing for a JSON field, this exception will be raised.
Cinchoo ETL works better with data annotation's MetadataType model. It is a way to attach MetaData class to data model class. In this associated class, you provide additional metadata information that is not in the data model. Its role is to add attribute to a class without having to modify this one. You can add this attribute that takes a single parameter to a class that will have all the attributes. This is useful when the POCO classes are auto generated (by Entity Framework, MVC, etc.) by automatic tools. This is why second class comes into play. You can add new stuff without touching the generated file. Also, this promotes modularization by separating the concerns into multiple classes.
For more information about it, please search in MSDN.
Listing 17.1 MetadataType annotation usage sample
[MetadataType(typeof(EmployeeRecMeta))]
public class EmployeeRec
{
public int Id { get; set; }
public string Name { get; set; }
}
[ChoJSONRecordObject]
public class EmployeeRecMeta : IChoNotifyRecordRead, IChoValidatable
{
[ChoJSONRecordField(FieldName = "id", ErrorMode = ChoErrorMode.ReportAndContinue )]
[ChoTypeConverter(typeof(IntConverter))]
[Range(1, 1, ErrorMessage = "Id must be > 0.")]
[ChoFallbackValue(1)]
public int Id { get; set; }
[ChoJSONRecordField(FieldName = "Name")]
[StringLength(1)]
[DefaultValue("ZZZ")]
[ChoFallbackValue("XXX")]
public string Name { get; set; }
public bool AfterRecordLoad(object target, int index, object source)
{
throw new NotImplementedException();
}
public bool BeforeRecordLoad(object target, int index, ref object source)
{
throw new NotImplementedException();
}
public bool RecordLoadError(object target, int index, object source, Exception ex)
{
throw new NotImplementedException();
}
public bool TryValidate(object target, ICollection<ValidationResult> validationResults)
{
return true;
}
public bool TryValidateFor(object target, string memberName,
ICollection<ValidationResult> validationResults)
{
return true;
}
public void Validate(object target)
{
}
public void ValidateFor(object target, string memberName)
{
}
}
In the above, EmployeeRec is the data class. Contains only domain specific properties and operations. Mark it very simple class to look at it.
We separate the validation, callback mechanism, configuration, etc. into metadata type class, EmployeeRecMeta.
If the POCO entity class is an auto-generated class or exposed via library or it is a sealed class, it limits you to attach JSON schema definition to it declaratively. In such case, you can choose one of the options below to specify JSON layout configuration:
- Manual Configuration
- Auto Map Configuration
- Attaching
MetadataType class
I'm going to show you how to configure the below POCO entity class on each approach.
Listing 18.1 Sealed POCO entity class
public sealed class EmployeeRec
{
public int Id { get; set; }
public string Name { get; set; }
}
Define a brand new configuration object from scratch and add all the necessary JSON fields to the ChoJSONConfiguration.JSONRecordFieldConfigurations collection property. This option gives you greater flexibility to control the configuration of JSON parsing. But the downside is the possibility of making mistakes and hard to manage them if the JSON file layout is large.
Listing 18.1.1 Manual Configuration
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Id"));
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));
This is an alternative approach and very less error-prone method to auto map the JSON fields for the POCO entity class.
First, define a schema class for EmployeeRec POCO entity class as below:
Listing 18.2.1 Auto Map class
public class EmployeeRecMap
{
[ChoJSONRecordField(FieldName = "Id")]
public int Id { get; set; }
[ChoJSONRecordField(FieldName = "Name")]
public string Name { get; set; }
}
Then, you can use it to auto map JSON fields by using ChoJSONRecordConfiguration.MapRecordFields method.
Listing 18.2.2 Using Auto Map configuration
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.MapRecordFields<EmployeeRecMap>();
foreach (var e in new ChoJSONReader<EmployeeRec>("emp.json", config))
Console.WriteLine(e.ToString());
This is one another approach to attach MetadataType class for POCO entity object. Previous approach simple care for auto mapping of JSON fields only. Other configuration properties like property converters, parser parameters, default/fallback values, etc. are not considered.
This model accounts for everything by defining MetadataType class and specifying the JSON configuration parameters declaratively. This is useful when your POCO entity is sealed and not partial class. Also, it is one of favorable and less error-prone approach to configure JSON parsing of POCO entity.
Listing 18.3.1 Define MetadataType class
[ChoJSONRecordObject]
public class EmployeeRecMeta : IChoNotifyRecordRead, IChoValidatable
{
[ChoJSONRecordField(FieldName = "Id", ErrorMode = ChoErrorMode.ReportAndContinue )]
[ChoTypeConverter(typeof(IntConverter))]
[Range(1, 1, ErrorMessage = "Id must be > 0.")]
public int Id { get; set; }
[ChoJSONRecordField(FieldName = "Name")]
[StringLength(1)]
[DefaultValue("ZZZ")]
[ChoFallbackValue("XXX")]
public string Name { get; set; }
public bool AfterRecordLoad(object target, int index, object source)
{
throw new NotImplementedException();
}
public bool BeforeRecordLoad(object target, int index, ref object source)
{
throw new NotImplementedException();
}
public bool RecordLoadError(object target, int index, object source, Exception ex)
{
throw new NotImplementedException();
}
public bool TryValidate(object target, ICollection<ValidationResult> validationResults)
{
return true;
}
public bool TryValidateFor
(object target, string memberName, ICollection<ValidationResult> validationResults)
{
return true;
}
public void Validate(object target)
{
}
public void ValidateFor(object target, string memberName)
{
}
}
Listing 18.3.2 Attaching MetadataType class
ChoMetadataObjectCache.Default.Attach<EmployeeRec>(new EmployeeRecMeta());
foreach (var e in new ChoJSONReader<EmployeeRec>("emp.json"))
Console.WriteLine(e.ToString()
This is a little nifty helper method to parse and load JSON text string into objects.
Listing 19.1 Using LoadText method
string txt = @"
[
{
"Id": 1,
"Name": "Jeanette"
},
{
"Id": 2,
"Name": "Giavani"
}
]";
foreach (var e in ChoJSONReader.LoadText(txt))
Console.WriteLine(e.ToStringEx());
Cinchoo ETL automatically parses and converts each JSON field values to the corresponding JSON field's underlying data type seamlessly. Most of the basic .NET types are handled automatically without any setup needed.
This is achieved through two key settings in the ETL system:
ChoJSONRecordConfiguration.CultureInfo - Represents information about a specific culture including the names of the culture, the writing system, and the calendar used, as well as access to culture-specific objects that provide information for common operations, such as formatting dates and sorting strings. Default is 'en-US'.ChoTypeConverterFormatSpec - It is global format specifier class holds all the intrinsic .NET types formatting specs.
In this section, I'm going to talk about changing the default format specs for each .NET intrinsic data types according to parsing needs.
ChoTypeConverterFormatSpec is singleton class, the instance is exposed via 'Instance' static member. It is thread local, means that there will be separate instance copy kept on each thread.
There are two sets of format specs members given to each intrinsic type, one for loading and another one for writing the value, except for Boolean, Enum, DataTime types. These types have only one member for both loading and writing operations.
Specifying each intrinsic data type format specs through ChoTypeConverterFormatSpec will impact system wide, i.e.,. by setting ChoTypeConverterFormatSpec.IntNumberStyle = NumberStyles.AllowParentheses will impact all integer members of JSON objects to allow parentheses. If you want to override this behavior and take control of specific JSON data member to handle its own unique parsing of JSON value from global system wide setting, it can be done by specifying TypeConverter at the JSON field member level. Refer to section 13.4 for more information.
NumberStyles (optional) used for loading values from JSON stream and Format string are used for writing values to JSON stream.
In this article, I'll brief about using NumberStyles for loading JSON data from stream. These values are optional. It determines the styles permitted for each type during parsing of JSON file. System automatically figures out the way to parse and load the values from underlying Culture. In odd situation, you may want to override and set the styles the way you want in order to successfully load the file. Refer to MSDN for more about NumberStyles and its values.
Listing 20.1.1 ChoTypeConverterFormatSpec Members
public class ChoTypeConverterFormatSpec
{
public static readonly ThreadLocal<ChoTypeConverterFormatSpec> Instance =
new ThreadLocal<ChoTypeConverterFormatSpec>(() => new ChoTypeConverterFormatSpec());
public string DateTimeFormat { get; set; }
public ChoBooleanFormatSpec BooleanFormat { get; set; }
public ChoEnumFormatSpec EnumFormat { get; set; }
public NumberStyles? CurrencyNumberStyle { get; set; }
public string CurrencyFormat { get; set; }
public NumberStyles? BigIntegerNumberStyle { get; set; }
public string BigIntegerFormat { get; set; }
public NumberStyles? ByteNumberStyle { get; set; }
public string ByteFormat { get; set; }
public NumberStyles? SByteNumberStyle { get; set; }
public string SByteFormat { get; set; }
public NumberStyles? DecimalNumberStyle { get; set; }
public string DecimalFormat { get; set; }
public NumberStyles? DoubleNumberStyle { get; set; }
public string DoubleFormat { get; set; }
public NumberStyles? FloatNumberStyle { get; set; }
public string FloatFormat { get; set; }
public string IntFormat { get; set; }
public NumberStyles? IntNumberStyle { get; set; }
public string UIntFormat { get; set; }
public NumberStyles? UIntNumberStyle { get; set; }
public NumberStyles? LongNumberStyle { get; set; }
public string LongFormat { get; set; }
public NumberStyles? ULongNumberStyle { get; set; }
public string ULongFormat { get; set; }
public NumberStyles? ShortNumberStyle { get; set; }
public string ShortFormat { get; set; }
public NumberStyles? UShortNumberStyle { get; set; }
public string UShortFormat { get; set; }
}
The sample below shows how to load JSON data stream having 'se-SE' (Swedish) culture specific data using ChoJSONReader. Also, the input feed comes with 'EmployeeNo' values containing parentheses. In order to make the load successful, we have to set the ChoTypeConverterFormatSpec.IntNumberStyle to NumberStyles.AllowParenthesis.
Listing 20.1.2 Using ChoTypeConverterFormatSpec in code
static void UsingFormatSpecs()
{
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.Culture = new System.Globalization.CultureInfo("se-SE");
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Id") { FieldType = typeof(int) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Name"));
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Salary") { FieldType = typeof(ChoCurrency) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("JoinedDate") { FieldType = typeof(DateTime) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("EmployeeNo") { FieldType = typeof(int) });
ChoTypeConverterFormatSpec.Instance.IntNumberStyle = NumberStyles.AllowParentheses;
using (var parser = new ChoJSONReader("emp.json", config))
{
object row = null;
while ((row = parser.Read()) != null)
Console.WriteLine(row.ToStringEx());
}
}
Cinchoo ETL provides ChoCurrency object to read and write currency values in JSON files. ChoCurrency is a wrapper class to hold the currency value in decimal type along with support of serializing them in text format during JSON load.
Listing 20.2.1 Using Currency members in dynamic model
static void CurrencyDynamicTest()
{
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Id"));
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Name"));
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Salary") { FieldType = typeof(ChoCurrency) });
using (var parser = new ChoJSONReader("emp.json", config))
{
object rec;
while ((rec = parser.Read()) != null)
{
Console.WriteLine(rec.ToStringEx());
}
}
}
The sample above shows how to load currency values using dynamic object model. By default, all the members of dynamic object are treated as string type, unless specified explicitly via ChoJSONFieldConfiguration.FieldType. By specifying the field type as ChoCurrency to the 'Salary' JSON field, ChoJSONReader loads them as currency object.
P.S.: The format of the currency value is figured by ChoJSONReader through ChoRecordConfiguration.Culture and ChoTypeConverterFormatSpec.CurrencyNumberStyle.
The sample below shows how to use ChoCurrency JSON field in POCO entity class.
Listing 20.2.2 Using Currency members in POCO model
public class EmployeeRecWithCurrency
{
public int Id { get; set; }
public string Name { get; set; }
public ChoCurrency Salary { get; set; }
}
static void CurrencyTest()
{
using (var parser = new ChoJSONReader<EmployeeRecWithCurrency>("emp.json"))
{
object rec;
while ((rec = parser.Read()) != null)
{
Console.WriteLine(rec.ToStringEx());
}
}
}
Cinchoo ETL implicitly handles parsing of enum field values from JSON files. If you want to fine control the parsing of these values, you can specify them globally via ChoTypeConverterFormatSpec.EnumFormat. Default is ChoEnumFormatSpec.Value.
FYI, changing this value will impact system wide.
There are three possible values that can be used:
ChoEnumFormatSpec.Value - Enum value is used for parsing.ChoEnumFormatSpec.Name - Enum key name is used for parsing.ChoEnumFormatSpec.Description - If each enum key is decorated with DescriptionAttribute, its value will be used for parsing.
Listing 20.3.1 Specifying Enum format specs during parsing
public enum EmployeeType
{
[Description("Full Time Employee")]
Permanent = 0,
[Description("Temporary Employee")]
Temporary = 1,
[Description("Contract Employee")]
Contract = 2
}
static void EnumTest()
{
ChoTypeConverterFormatSpec.Instance.EnumFormat = ChoEnumFormatSpec.Description;
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Id") { FieldType = typeof(int) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Name"));
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Salary") { FieldType = typeof(ChoCurrency) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("JoinedDate") { FieldType = typeof(DateTime) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("EmployeeType")
{ FieldType = typeof(EmployeeType) });
ChoTypeConverterFormatSpec.Instance.IntNumberStyle = NumberStyles.AllowParentheses;
using (var parser = new ChoJSONReader("emp.json", config))
{
object row = null;
while ((row = parser.Read()) != null)
Console.WriteLine(row.ToStringEx());
}
}
Cinchoo ETL implicitly handles parsing of boolean JSON field values from JSON files. If you want to fine control the parsing of these values, you can specify them globally via ChoTypeConverterFormatSpec.BooleanFormat. Default value is ChoBooleanFormatSpec.ZeroOrOne.
FYI, changing this value will impact system wide.
There are four possible values that can be used:
ChoBooleanFormatSpec.ZeroOrOne - '0' for false. '1' for trueChoBooleanFormatSpec.YOrN - 'Y' for true, 'N' for falseChoBooleanFormatSpec.TrueOrFalse - 'True' for true, 'False' for falseChoBooleanFormatSpec.YesOrNo - 'Yes' for true, 'No' for false
Listing 20.4.1 Specifying boolean format specs during parsing
static void BoolTest()
{
ChoTypeConverterFormatSpec.Instance.BooleanFormat = ChoBooleanFormatSpec.ZeroOrOne;
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Id") { FieldType = typeof(int) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Name"));
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Salary") { FieldType = typeof(ChoCurrency) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("JoinedDate") { FieldType = typeof(DateTime) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Active") { FieldType = typeof(bool) });
ChoTypeConverterFormatSpec.Instance.IntNumberStyle = NumberStyles.AllowParentheses;
using (var parser = new ChoJSONReader("emp.json", config))
{
object row = null;
while ((row = parser.Read()) != null)
Console.WriteLine(row.ToStringEx());
}
}
Cinchoo ETL implicitly handles parsing of datetime JSON field values from JSON files using system Culture or custom set culture. If you want to fine control the parsing of these values, you can specify them globally via ChoTypeConverterFormatSpec.DateTimeFormat. Default value is 'd'.
FYI, changing this value will impact system wide.
You can use any valid standard or custom datetime .NET format specification to parse the datetime JSON values from the file.
Listing 20.5.1 Specifying datetime format specs during parsing
static void DateTimeTest()
{
ChoTypeConverterFormatSpec.Instance.DateTimeFormat = "MMM dd, yyyy";
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Id") { FieldType = typeof(int) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Name"));
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Salary") { FieldType = typeof(ChoCurrency) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("JoinedDate") { FieldType = typeof(DateTime) });
config.JSONRecordFieldConfigurations.Add
(new ChoJSONRecordFieldConfiguration("Active") { FieldType = typeof(bool) });
ChoTypeConverterFormatSpec.Instance.IntNumberStyle = NumberStyles.AllowParentheses;
using (var parser = new ChoJSONReader("emp.json", config))
{
object row = null;
while ((row = parser.Read()) != null)
Console.WriteLine(row.ToStringEx());
}
}
The sample above shows how to parse custom datetime JSON values from JSON file.
Note: As the datetime values contains JSON separator, it is given with double quotes to pass the parsing.
ChoJSONReader exposes few frequent to use configuration parameters via fluent API methods. This will make the programming of parsing of JSON files quicker.
This API method sets the JSONPath expression to select the nodes to load using ChoJSONReader.
foreach (var e in new ChoJSONReader<EmployeeRec>("emp.json").WithJSONPath("$", true))
Console.WriteLine(e.ToString());
This API method specifies the list of JSON nodes (either attributes or elements) to be considered for parsing and loading. Other fields in the JSON nodes will be discarded.
foreach (var e in new ChoJSONReader<EmployeeRec>("emp.json").WithFields("Id", "Name"))
Console.WriteLine(e.ToString());
This API method used to add JSON node with JSONPath, data type and other parameters. This method is helpful in dynamic object model, by specifying each and individual JSON node with appropriate datatype.
foreach (var e in new ChoJSONReader<EmployeeRec>("emp.json").WithField
("Id", fieldType: typeof(int)))
Console.WriteLine(e.ToString());
This API method used to set the ChoJSONWriter to perform check on field countness before reading JSON file.
foreach (var e in new ChoJSONReader<EmployeeRec>("emp.json").ColumnCountStrict())
Console.WriteLine(e.ToString());
This API method is used to define the number of rows to be processed before generating a notification event. This property is designed for user interface components that illustrates the JSON loading progress. Notifications are sent to subscribers who subscribed to RowsLoaded event.
static void NotifyAfterTest()
{
using (var parser = new ChoJSONReader("emp.json")
.NotifyAfter(1000)
)
{
parser.RowsLoaded += (o, e) => Console.WriteLine(e.RowsLoaded);
foreach (var rec in parser)
{
Console.WriteLine(String.Format("Id: {0}", rec.Id));
Console.WriteLine(String.Format("Name: {0}", rec.Name));
Console.WriteLine(String.Format("Salary: {0}", rec.Salary));
}
}
}
This API method used to configure all configuration parameters which are not exposed via fluent API.
static void ConfigureTest()
{
using (var parser = new ChoJSONReader("emp.json")
.Configure(c => c.ErrorMode = ChoErrorMode.ThrowAndStop)
)
{
foreach (var rec in parser)
{
Console.WriteLine(String.Format("Id: {0}", rec.Id));
Console.WriteLine(String.Format("Name: {0}", rec.Name));
Console.WriteLine(String.Format("Salary: {0}", rec.Salary));
}
}
}
This API method used to setup the reader's parameters / events via fluent API.
static void SetupTest()
{
using (var parser = new ChoJSONReader("emp.json")
.Setup(r => r.BeforeRecordLoad += (o, e) =>
{
if (e.Source.CastTo<JObject>().ContainsKey("Name1"))
e.Skip = true;
}
)
{
foreach (var rec in parser)
{
Console.WriteLine(String.Format("Id: {0}", rec.Id));
Console.WriteLine(String.Format("Name: {0}", rec.Name));
Console.WriteLine(String.Format("Salary: {0}", rec.Salary));
}
}
}
ChoJSONReader implicitly handles the conversion of the enum text to enum value. The sample below shows how to load JSON with POCO object:
public enum Gender { Male, Female }
public class Employee
{
public int Age { get; set; }
public Gender Gender { get; set; }
}
static void EnumTest()
{
string json = @"{ ""Age"": 35, ""Gender"": ""Female"" }";
using (var r = ChoJSONReader<Employee>.LoadText(json))
{
foreach (var rec in r)
Console.WriteLine(rec.Dump());
}
}
Sample fiddle: https://dotnetfiddle.net/0E1YKl
The sample below shows how to parse the JSON with enum values in dynamic object model approach:
static void DynamicEnumTest()
{
string json = @"{ ""Age"": 35, ""Gender"": ""Male"" }";
using (var r = ChoJSONReader.LoadText(json)
.WithField("Age")
.WithField("Gender", fieldType: typeof(Gender))
)
{
foreach (var rec in r)
Console.WriteLine(rec.Dump());
}
}
Sample fiddle: https://dotnetfiddle.net/Rt2upb
With the custom JSONPath ($..^), you can load only the values from key-value pair JSON data.
public class Balance
{
public float amount { get; set; }
public float value { get; set; }
}
static void LoadDictValuesTest()
{
string json = @"{
""AE"": {
""amount"": ""0.00000000"",
""value"": ""0.00000000""
},
""AR"": {
""amount"": ""0.00000000"",
""value"": ""0.00000000""
},
""BC"": {
""amount"": ""0.09670332"",
""value"": ""3.74814004""
}
}";
using (var r = ChoJSONReader< Balance>.LoadText(json)
.WithJSONPath("$..^")
)
{
foreach (var rec in r)
Console.WriteLine(rec.Dump());
}
}
Sample fiddle: https://dotnetfiddle.net/yuWly8
Likewise, sample below shows how to load values using dynamic approach
public static void Main()
{
string json = @"{
""AE"": {
""amount"": ""1.00000000"",
""value"": ""2.00000000""
},
""AR"": {
""amount"": ""3.10000000"",
""value"": ""4.500000000""
},
""BC"": {
""amount"": ""0.09670332"",
""value"": ""3.74814004""
}
}";
using (var r = ChoJSONReader.LoadText(json)
.WithJSONPath("$..^")
)
{
foreach (var rec in r)
rec.Print();
}
}
Sample fiddle: https://dotnetfiddle.net/u0E5Mv
With the custom JSONPath ($..~), you can load only the values from JSON.
static void LoadDictKeysTest()
{
string json = @"{
""AE"": {
""amount"": ""0.00000000"",
""value"": ""0.00000000""
},
""AR"": {
""amount"": ""0.00000000"",
""value"": ""0.00000000""
},
""BC"": {
""amount"": ""0.09670332"",
""value"": ""3.74814004""
}
}";
using (var r = ChoJSONReader<string>.LoadText(json)
.WithJSONPath("$..^")
)
{
foreach (var rec in r)
Console.WriteLine(rec.Dump());
}
}
Sample fiddle: https://dotnetfiddle.net/ziAzHc
ChoJSONReader does this implicitly in the dynamic object model.
static void DynamicEnumTest()
{
string json = @"{ ""Age"": 35, ""Gender"": ""Male"" }";
using (dynamic r = ChoJSONReader.LoadText(json)
.WithField("Age")
.WithField("Gender", fieldType: typeof(Gender))
)
{
foreach (var rec in r)
{
Console.WriteLine(rec.Age);
Console.WriteLine(rec.Gender);
}
}
}
In the above, the parser loads the json file, constructs and returns dynamic object.
Cinchoo ETL provides ChoXmlWriter to generate XML file from objects. With ChoJSONReader along with ChoXmlWriter, you can convert JSON to XML format easily.
static void JSON2XmlTest()
{
string json = @"[
{
""Id"": 1,
""Name"": ""Mark""
},
{
""Id"": 2,
""Name"": ""Tom""
}
]
";
StringBuilder xml = new StringBuilder();
using (var r = ChoJSONReader.LoadText(json))
{
using (var w = new ChoXmlWriter(xml)
.WithRootName("Emps")
.WithNodeName("Emp")
)
w.Write(r);
}
Console.WriteLine(xml.ToString());
}
Output
<Emps>
<Emp>
<Id>1</Id>
<Name>Mark</Name>
</Emp>
<Emp>
<Id>2</Id>
<Name>Tom</Name>
</Emp>
</Emps>
Sample fiddle: https://dotnetfiddle.net/dgvc1D
With JSONPath, you can load selective nodes from JSON into objects using ChoJSONReader.
For a sample JSON below:
{
"user": {
"name": "asdf",
"teamname": "b",
"email": "c",
"players": ["1", "2"]
}
}
wanted to load User data into object. The sample code uses JSONPath "$.user" to pick the node and parse it.
public class UserInfo
{
public string name { get; set; }
public string teamname { get; set; }
public string email { get; set; }
public string[] players { get; set; }
}
static void ReadSelectNodeTest()
{
string json = @"
{
""user"": {
""name"": ""asdf"",
""teamname"": ""b"",
""email"": ""c"",
""players"": [""1"", ""2""]
}
}";
using (var r = ChoJSONReader<UserInfo>.LoadText(json)
.WithJSONPath("$.user")
)
{
foreach (var rec in r)
Console.WriteLine(rec.Dump());
}
}
Sample fiddle: https://dotnetfiddle.net/wyZWpG
With JSONPath at each member level, you can load selective subnodes from JSON into object members using ChoJSONReader.
For a sample JSON below:
{
"user": {
"name": "asdf",
"teamname": "b",
"email": "c",
"players": ["1", "2"]
}
}
Wanted to load User data into object. Sample code uses JSONPath on each field to pick the subnode and parse it.
public class UserInfo
{
[ChoJSONRecordField(JSONPath = "$.name")]
public string name { get; set; }
[ChoJSONRecordField(JSONPath = "$.teamname")]
public string teamname { get; set; }
[ChoJSONRecordField(JSONPath = "$.email")]
public string email { get; set; }
[ChoJSONRecordField(JSONPath = "$.players[1]")]
public string player { get; set; }
}
In the above, each member has been specified with JSONPath to choose the node value from. This way, you can pick and choose JSON node value with complex JSONPath, give you more control to choose the data from.
static void ReadSelectNodeTest()
{
string json = @"
{
""user"": {
""name"": ""asdf"",
""teamname"": ""b"",
""email"": ""c"",
""players"": [""1"", ""2""]
}
}";
using (var r = ChoJSONReader<UserInfo>.LoadText(json)
.WithJSONPath("$.user")
)
{
foreach (var rec in r)
Console.WriteLine(rec.Dump());
}
}
Sample fiddle: https://dotnetfiddle.net/6jq9uv
ChoJSONReader provides a little helper method to convert JSON to Datatable, AsDataTable():
static void ConvertToDataTableTest()
{
string json = @"
{
""user"": {
""name"": ""asdf"",
""teamname"": ""b"",
""email"": ""c"",
""players"": [""1"", ""2""]
}
}";
using (var r = ChoJSONReader<UserInfo>.LoadText(json)
.WithJSONPath("$.user")
.Configure(c => c.ArrayValueNamePrefix = String.Empty)
)
{
var dt = r.AsDataTable();
}
}
Sample fiddle: https://dotnetfiddle.net/hdOdvi
ChoJSONReader provides a little helper method to convert JSON to DataReader, AsDataReader().
static void ConvertToDataTableTest()
{
string json = @"
{
""user"": {
""name"": ""asdf"",
""teamname"": ""b"",
""email"": ""c"",
""players"": [""1"", ""2""]
}
}";
using (var r = ChoJSONReader<UserInfo>.LoadText(json)
.WithJSONPath("$.user")
)
{
var dt = r.AsDataReader();
}
}
Sample fiddle: https://dotnetfiddle.net/sCifj5
Cinchoo ETL provides ChoCSVWriter to generate CSV file from objects. With ChoJSONReader along with ChoCSVWriter, you can convert JSON to CSVformat easily.
JSON is a hierarchical object model, CSV is flat structure. Cinchoo handles them seamlessly and produces the output as expected.
static void Json2CSV2()
{
string json = @" {id: 1, name: ""Tom"", friends: [""Dick"", ""Harry""]}";
StringBuilder csv = new StringBuilder();
using (var r = ChoJSONReader.LoadText(json)
)
{
using (var w = new ChoCSVWriter(csv)
.WithFirstLineHeader()
.NestedColumnSeparator('/')
)
w.Write(r);
}
Console.WriteLine(csv.ToString());
}
Output
id,name,friends/0,friends/1
1,Tom,Dick,Harry
Sample fiddle: https://dotnetfiddle.net/2xrf62
This sample shows how to deserialize JSON to an object:
public class Account
{
public string Email { get; set; }
public bool Active { get; set; }
public DateTime CreatedDate { get; set; }
public IList<string> Roles { get; set; }
}
static void DeserializeObject()
{
string json = @"{
'Email': 'james@example.com',
'Active': true,
'CreatedDate': '2013-01-20T00:00:00Z',
'Roles': [
'User',
'Admin'
]
}";
Account account = ChoJSONReader.DeserializeText<Account>(json).FirstOrDefault();
Console.WriteLine(account.Email);
}
Sample fiddle: https://dotnetfiddle.net/CmeAfv
This sample shows deserialize JSON to objects in the inheritance hierarchy without using TypeNameHandling option.
public abstract class Person
{
public string FirstName { get; set; }
public string LastName { get; set; }
}
public class Employee : Person
{
public string Department { get; set; }
public string JobTitle { get; set; }
}
public class Artist : Person
{
public string Skill { get; set; }
}
JSON:
[
{
"Department": "Department1",
"JobTitle": "JobTitle1",
"FirstName": "FirstName1",
"LastName": "LastName1"
},
{
"Department": "Department2",
"JobTitle": "JobTitle2",
"FirstName": "FirstName2",
"LastName": "LastName2"
},
{
"Skill": "Painter",
"FirstName": "FirstName3",
"LastName": "LastName3"
}
]
Code:
static void DeserializeDifferentObjects()
{
using (var r = new ChoJSONReader<Person>("emp.json")
.WithCustomRecordSelector(o =>
{
var pair = (Tuple<long, JObject>)o;
var obj = pair.Item2;
if (obj.ContainsKey("Skill"))
return typeof(Artist);
return typeof(Employee);
})
)
{
foreach (var rec in r)
{
Console.WriteLine(rec.Dump());
}
}
}
Sample fiddle: https://dotnetfiddle.net/C8HQGS
This sample shows deserialize JSON to objects in the inheritance hierarchy using TypeNameHandling option.
static void InterfaceTest()
{
string json = @"
{
"$type": "ChoJSONReaderTest.Program+Person1, ChoJSONReaderTest,
Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
"Profession": {
"$type": "ChoJSONReaderTest.Program+Programming, ChoJSONReaderTest,
Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
"JobTitle": "Software Developer",
"FavoriteLanguage": "C#"
}
}";
foreach (var rec in ChoJSONReader<Person1>.LoadText(json)
.Configure(c => c.UseJSONSerialization = true)
.Configure(c => c.JsonSerializerSettings.TypeNameHandling = TypeNameHandling.All)
)
Console.WriteLine(rec.Dump());
}
This sample deserializes JSON to an collection:
static void DeserializeCollection()
{
string json = @"['Starcraft','Halo','Legend of Zelda']";
List<string> videogames = ChoJSONReader.DeserializeText<string>(json).ToList();
Console.WriteLine(string.Join(", ", videogames.ToArray()));
}
This sample deserializes JSON to a Dictionary:
static void DeserializeDictionary()
{
string json = @"{
'href': '/account/login.aspx',
'target': '_blank'
}";
Dictionary<string, string> htmlAttributes =
ChoJSONReader.DeserializeText<Dictionary<string, string>>(json).FirstOrDefault();
Console.WriteLine(htmlAttributes["href"]);
Console.WriteLine(htmlAttributes["target"]);
}
This sample deserializes JSON to a Dictionary:
public class Movie
{
public string Name { get; set; }
public int Year { get; set; }
}
static void DeserializeFromFile()
{
Movie movie1 = ChoJSONReader.Deserialize<Movie>("movie.json").FirstOrDefault();
}
This sample deserializes JSON with custom factory to instantiate Employee instance for Person type.
public class Person
{
public string FirstName { get; set; }
public string LastName { get; set; }
public DateTime BirthDate { get; set; }
}
public class Employee : Person
{
public string Department { get; set; }
public string JobTitle { get; set; }
}
static void CustomCreationTest()
{
string json = @"{
'Department': 'Furniture',
'JobTitle': 'Carpenter',
'FirstName': 'John',
'LastName': 'Joinery',
'BirthDate': '1983-02-02T00:00:00'
}";
ChoActivator.Factory = (type, args) =>
{
if (type == typeof(Person))
return new Employee();
else
return null;
};
Person person = ChoJSONReader.DeserializeText<Person>(json).FirstOrDefault();
Console.WriteLine(person.GetType().Name);
}
Let's say, you have JSON as below:
[
{
"email": "john.doe@sendgrid.com",
"timestamp": 1337966815,
"category": [
"newuser",
"transactional"
],
"event": "open"
},
{
"email": "jane.doe@sendgrid.com",
"timestamp": 1337966815,
"category": "olduser",
"event": "open"
}
]
In here, you will notice that category property for each item either can be simple string or array of strings. ChoJSONReader seamlessly handles this situation, and loads them properly without any using custom converters.
Define a POCO class as below:
public class Item
{
[JsonProperty("email")]
public string Email { get; set; }
[JsonProperty("timestamp")]
public int Timestamp { get; set; }
[JsonProperty("event")]
public string Event { get; set; }
[JsonProperty("category")]
public List<string> Categories { get; set; }
}
Here is how you can parse the JSON:
public static void SingleOrArrayItemTest()
{
foreach (var rec in new ChoJSONReader<Item>("items.json"))
{
Console.WriteLine(rec.Dump());
}
}
ChoJSONReader can automatically convert datetime value using current system culture. If JSON comes with custom datetime formatted value, you can set the custom datetime format to parse the JSON successfully.
Sample JSON with custom datetime format value:
{
'Department': 'Furniture',
'JobTitle': 'Carpenter',
'FirstName': 'John',
'LastName': 'Joinery',
'BirthDate': '30-12-2003'
}
Define POCO class as below to handle the custom datetime format:
public class Employee
{
public string Department { get; set; }
public string JobTitle { get; set; }
[DisplayFormat(DataFormatString = "dd-MM-yyyy")]
public DateTime BirthDate { get; set; }
}
or:
public class Employee
{
[ChoJSONRecordField]
public string Department { get; set; }
[ChoJSONRecordField]
public string JobTitle { get; set; }
[ChoJSONRecordField(FormatText = "dd-MM-yyyy")]
public DateTime BirthDate { get; set; }
}
Use the parser to load the JSON as below:
using (var r = ChoJSONReader<Employee>.LoadText(json))
{
foreach (var rec in r)
Console.WriteLine(rec.Dump());
}
In the dynamic model, you can set the custom datetime format as below:
using (var r = ChoJSONReader.LoadText(json)
.WithField("Department")
.WithField("JobTitle")
.WithField("BirthDate", fieldType: typeof(DateTime), formatText: "dd-MM-yyyy")
)
{
foreach (var rec in r)
Console.WriteLine(rec.Dump());
}
This section shows how to use Enumeration class while deserializing JSON using ChoJSONReader. For those who want to know what is Enumeration class, please visit this link.
There are a couple of ways in which you can handle to deserialize it successfully.
Method 1: Using Implicit / Explicit Operator Overloads
public class CardType : Enumeration
{
public static readonly CardType Amex = new CardType(1, "Amex");
public static readonly CardType Visa = new CardType(2, "Visa");
public static readonly CardType MasterCard = new CardType(3, "MasterCard");
public CardType(int id, string name)
: base(id, name)
{
}
public static explicit operator CardType(string name)
{
if (name == "Amex")
return Amex;
if (name == "MasterCard")
return MasterCard;
else
return Visa;
}
}
public class Dto
{
public string Name { get; set; }
public CardType CardType { get; set; }
}
static void DeserializeEnumClass()
{
string json = @"[
{
""Name"": ""Tom"",
""CardType"": ""Amex""
}
]";
var x = ChoJSONReader.DeserializeText<Dto>(json).FirstOrDefault();
Console.WriteLine(x.Dump());
}
Method 2: Using Value Convertor
public class CardType : Enumeration
{
public static readonly CardType Amex = new CardType(1, "Amex");
public static readonly CardType Visa = new CardType(2, "Visa");
public static readonly CardType MasterCard = new CardType(3, "MasterCard");
public CardType(int id, string name)
: base(id, name)
{
}
}
public class CardTypeConverter : IValueConverter
{
public object Convert
(object value, Type targetType, object parameter, CultureInfo culture)
{
var name = value as string;
if (name == "Amex")
return CardType.Amex;
if (name == "MasterCard")
return CardType.MasterCard;
else
return CardType.Visa;
}
public object ConvertBack
(object value, Type targetType, object parameter, CultureInfo culture)
{
throw new NotImplementedException();
}
}
public class Dto
{
public string Name { get; set; }
[ChoTypeConverter(typeof(CardTypeConverter))]
public CardType CardType { get; set; }
}
static void DeserializeEnumClass()
{
string json = @"[
{
""Name"": ""Tom"",
""CardType"": ""Amex""
}
]";
var x = ChoJSONReader.DeserializeText<Dto>(json).FirstOrDefault();
Console.WriteLine(x.Dump());
}
Let's say you have JSON as below, you wanted to deserialize only details having id > 0.
{
"id": 5,
"name": "test",
"details": [
{
"id": 12,
"data1": 0.25
},
{
"id": 0,
"data1": 0.0
}
]
}
First, define the object model matching JSON above:
public class CTest
{
public int Id { get; set; }
public string Name { get; set; }
public List<Class2> Details { get; set; }
}
public class Class2
{
public int Id { get; set; }
public int Data1 { get; set; }
}
Here is how deserialize only details having id > 0:
string json = @"[
{
""id"":5,
""name"":""test"",
""details"":[
{
""id"":12,
""data1"":0.25
},
{
""id"":0,
""data1"":0.0
},
]
}
]";
using (var r = ChoJSONReader<CTest>.LoadText(json)
.RegisterNodeConverterForType<List<Class2>>(o =>
{
dynamic x = o as dynamic;
var list = new List<Class2>();
while (x.reader.Read() && x.reader.TokenType != JsonToken.EndArray)
{
if (x.reader.TokenType == JsonToken.StartObject)
{
var item = x.serializer.Deserialize<Class2>(x.reader);
if (item.Id != 0)
list.Add(item);
}
}
return list;
})
)
{
foreach (var rec in r)
Console.WriteLine(rec.Dump());
}
- 22nd May, 2020: Initial version
