Mastering Type-Safe JSON Serialization in TypeScript

Introduction

Almost every web application requires data serialization. This need arises in situations like:

• Transferring data over the network (e.g., HTTP requests, WebSockets)
• Embedding data in HTML (for hydration, for instance)
• Storing data in a persistent storage (like LocalStorage)
• Sharing data between processes (like web workers or postMessage)

In many cases, data loss or corruption can lead to serious consequences, making it essential to provide a convenient and safe serialization mechanism that helps detect as many errors as possible during the development stage. For these purposes, it's convenient to use JSON as the data transfer format and TypeScript for static code checking during development.

TypeScript serves as a superset of JavaScript, which should enable the seamless use of functions like JSON.stringify and JSON.parse, right? Turns out, despite all its benefits, TypeScript doesn't naturally understand what JSON is and which data types are safe for serialization and deserialization into JSON. Let's illustrate this with an example.

The Problem with JSON in TypeScript

Consider, for example, a function that saves some data to LocalStorage. As LocalStorage cannot store objects, we use JSON serialization here:

interface PostComment {
  authorId: string;
  text: string;
  updatedAt: Date;
}

function saveComment(comment: PostComment) {
    const serializedComment = JSON.stringify(comment);
    localStorage.setItem('draft', serializedComment);
}

We will also need a function to retrieve the data from LocalStorage.

function restoreComment(): PostComment | undefined {
    const text = localStorage.getItem('draft');
    return text ? JSON.parse(text) : undefined;
}

What’s wrong with this code? The first problem is that when restoring the comment, we will get a string type instead of Date for the updatedAt field.

This happens because JSON only has four primitive data types (null, string, number, boolean), as well as arrays and objects. It is not possible to save a Date object in JSON, as well as other objects that are found in JavaScript: functions, Map, Set, etc.

When JSON.stringify encounters a value that cannot be represented in JSON format, type casting occurs. In the case of a Date object, we get a string because the Date object implements the toJson() method, which returns a string instead of a Date object.

const date = new Date('August 19, 1975 23:15:30 UTC');

const jsonDate = date.toJSON();
console.log(jsonDate);
// Expected output: "1975-08-19T23:15:30.000Z"

const isEqual = date.toJSON() === JSON.stringify(date);
console.log(isEqual);
// Expected output: true

The second problem is that the saveComment function returns the PostComment type, in which the date field is of type Date. But we already know that instead of Date, we will actually receive a string type. TypeScript could help us find this error, but why doesn't it?

Turns out, in TypeScript's standard library, the JSON.parse function is typed as (text: string) => any. Due to the use of any, type checking is essentially disabled. In our example, TypeScript simply took our word that the function would return a PostComment containing a Date object.

This TypeScript behavior is inconvenient and unsafe. Our application may crash if we try to treat a string like a Date object. For example, it might break if we call comment.updatedAt.toLocaleDateString().

Indeed, in our small example, we could simply replace the Date object with a numerical timestamp, which works well for JSON serialization. But in real applications, data objects might be extensive, types can be defined in multiple locations, and identifying such an error during development may be a challenging task.

What if we could enhance TypeScript's understanding of JSON?

Dealing with Serialization

To start with, let's figure out how to make TypeScript understand which data types can be safely serialized into JSON. Suppose we want to create a function safeJsonStringify, where TypeScript will check the input data format to ensure it's JSON serializable.

function safeJsonStringify(data: JSONValue) {
    return JSON.stringify(data);
}

In this function, the most important part is the JSONValue type, which represents all possible values that can be represented in the JSON format. The implementation is quite straightforward:

type JSONPrimitive = string | number | boolean | null | undefined;

type JSONValue = JSONPrimitive | JSONValue[] | {
    [key: string]: JSONValue;
};

First, we define the JSONPrimitive type, which describes all the primitive JSON data types. We also include the undefined type based on the fact that when serialized, keys with the undefined value will be omitted. During deserialization, these keys will simply not appear in the object, which in most cases is the same thing.

Next, we describe the JSONValue type. This type uses TypeScript's ability to describe recursive types, which are types that refer to themselves. Here, JSONValue can either be a JSONPrimitive, an array of JSONValue, or an object where all values are of the JSONValue type. As a result, a variable of type JSONValue can contain arrays and objects with unlimited nesting. The values within these will also be checked for compatibility with the JSON format.

Now we can test our safeJsonStringify function using the following examples:

// No errors
safeJsonStringify({
    updatedAt: Date.now()
});

// Yields an error:
// Argument of type '{ updatedAt: Date; }' is not assignable to parameter of type 'JSONValue'.
//   Types of property 'updatedAt' are incompatible.
//     Type 'Date' is not assignable to type 'JSONValue'.
safeJsonStringify({
    updatedAt: new Date();
});

Everything seems to function properly. The function allows us to pass the date as a number, but yields an error if we will pass the Date object.

But let's consider a more realistic example, in which the data passed to the function is stored in a variable and has a described type.

interface PostComment {
    authorId: string;
    text: string;
    updatedAt: number;
};

const comment: PostComment = {...};

// Yields an error:
// Argument of type 'PostComment' is not assignable to parameter of type 'JSONValue'.
//   Type 'PostComment' is not assignable to type '{ [key: string]: JSONValue; }'.
//     Index signature for type 'string' is missing in type 'PostComment'.
safeJsonStringify(comment);

Now, things are getting a bit tricky. TypeScript won't let us assign a variable of type PostComment to a function parameter of type JSONValue, because "Index signature for type 'string' is missing in type 'PostComment'".

So, what is an index signature and why is it missing? Remember how we described objects that can be serialized into the JSON format?

type JSONValue = {
    [key: string]: JSONValue;
};

In this case, [key: string] is the index signature. It means "this object can have any keys in the form of strings, the values of which have the JSONValue type". So, it turns out we need to add an index signature to the PostComment type, right?

interface PostComment {
    authorId: string;
    text: string;
    updatedAt: number;

    // Don't do this:
    [key: string]: JSONValue;
};

In fact, doing so would imply that the comment could contain any arbitrary fields, which is not typically the desired outcome when defining data types in an application.

The real solution to the problem with the index signature comes from Mapped Types, which allow to recursively iterate over fields, even for types that don't have an index signature defined. Combined with generics, this feature allows converting any data type T into another type JSONCompatible<T>, which is compatible with the JSON format.

type JSONCompatible<T> = unknown extends T ? never : {
    [P in keyof T]:
        T[P] extends JSONValue ? T[P] :
        T[P] extends NotAssignableToJson ? never :
        JSONCompatible<T[P]>;
};

type NotAssignableToJson =
    | bigint
    | symbol
    | Function;

The JSONCompatible<T> type is a mapped type that inspects whether a given type T can be safely serialized into JSON. It does this by iterating over each property in type T and doing the following:

1. The T[P] extends JSONValue ? T[P] : ... conditional type verifies if the property's type is compatible with the JSONValue type, assuring it can be safely converted to JSON. When this is the case, the property's type remains unchanged.
2. The T[P] extends NotAssignableToJson ? never : ... conditional type verifies if the property’s type isn't assignable to JSON. In this case, the property's type is converted to never, effectively filtering the property out from the final type.
3. If neither of these conditions is met, the type is recursively checked until a conclusion can be made. This way, it works even if the type doesn’t have an index signature.

The unknown extends T ? never :... check at the beginning is used to prevent the unknown type from being converted to an empty object type {}, which is essentially equivalent to the any type.

Another interesting aspect is the NotAssignableToJson type. It consists of two TypeScript primitives (bigint and symbol) and the Function type, which describes any possible function. The Function type is crucial in filtering out any values which aren't assignable to JSON. This is because any complex object in JavaScript is based on the Object type and has at least one function in its prototype chain (e.g., toString()). The JSONCompatible type iterates over all of those functions, so checking functions is sufficient to filter out anything that isn't serializable to JSON.

Now, let's use this type in the serialization function:

function safeJsonStringify<T>(data: JSONCompatible<T>) {
    return JSON.stringify(data);
}

Now, the function uses a generic parameter T and accepts the JSONCompatible<T> argument. This means it takes an argument data of type T, which should be a JSON compatible type. Now we can use the function with data types without an index signature.

The function now uses a generic parameter T that extends from the JSONCompatible<T> type. This means that it accepts an argument data of type T, which ought to be a JSON compatible type. As a result, we can utilize the function with data types that lack an index signature.

interface PostComment {
  authorId: string;
  text: string;
  updatedAt: number;
}

function saveComment(comment: PostComment) {
    const serializedComment = safeJsonStringify(comment);
    localStorage.setItem('draft', serializedComment);
}

This approach can be used whenever JSON serialization is necessary, such as transferring data over the network, embedding data in HTML, storing data in localStorage, transferring data between workers, etc. Additionally, the toJsonValue helper can be used when a strictly typed object without an index signature needs to be assigned to a variable of JSONValue type.

function toJsonValue<T>(value: JSONCompatible<T>): JSONValue {
    return value;
}

const comment: PostComment = {...};

const data: JSONValue = {
    comment: toJsonValue(comment)
};

In this example, using toJsonValue lets us bypass the error related to the missing index signature in the PostComment type.

Dealing with Deserialization

When it comes to deserialization, the challenge is both simpler and more complex simultaneously because it involves both static analysis checks and runtime checks for the received data's format.

From the perspective of TypeScript's type system, the challenge is quite simple. Let's consider the following example:

function safeJsonParse(text: string) {
    return JSON.parse(text) as unknown;
}

const data = JSON.parse(text);
//    ^?  unknown 

In this instance, we're substituting the any return type with the unknown type. Why choose unknown? Essentially, a JSON string could contain anything, not just the data what we expect to receive. For example, the data format might change between different application versions, or another part of the app could write data to the same LocalStorage key. Therefore, unknown is the safest and most precise choice.

However, working with the unknown type is less convenient than merely specifying the desired data type. Apart from type-casting, there are multiple ways to convert the unknown type into the required data type. One such method is utilizing the Superstruct library to validate data at runtime and throw detailed errors if the data is invalid.

import { create, object, number, string } from 'superstruct';

const PostComment = object({
    authorId: string(),
    text: string(),
    updatedAt: number(),
});

// Note: we no longer need to manually specify the return type
function restoreDraft() {
    const text = localStorage.getItem('draft');
    return text ? create(JSON.parse(text), PostComment) : undefined;
}

Here, the create function acts as a type guard, narrowing the type to the desired Comment interface. Consequently, we no longer need to manually specify the return type.

Implementing a secure deserialization option is only half the story. It's equally crucial not to forget to use it when tackling the next task in the project. This becomes particularly challenging if a large team is working on the project, as ensuring all agreements and best practices are followed can be difficult.

Typescript-eslint can assist in this task. This tool helps identify all instances of unsafe any usage. Specifically, all usages of JSON.parse can be found and it can be ensured that the received data's format is checked. More about getting rid of the any type in a codebase can be read in the article Making TypeScript Truly "Strongly Typed".

Conclusion

Here are the final utility functions and types designed to assist in safe JSON serialization and deserialization. You can test these in the prepared TS Playground.

type JSONPrimitive = string | number | boolean | null | undefined;

type JSONValue = JSONPrimitive | JSONValue[] | {
    [key: string]: JSONValue;
};

type NotAssignableToJson = 
    | bigint 
    | symbol 
    | Function;

type JSONCompatible<T> = unknown extends T ? never : {
    [P in keyof T]: 
        T[P] extends JSONValue ? T[P] : 
        T[P] extends NotAssignableToJson ? never : 
        JSONCompatible<T[P]>;
};

function toJsonValue<T>(value: JSONCompatible<T>): JSONValue {
    return value;
}

function safeJsonStringify<T>(data: JSONCompatible<T>) {
    return JSON.stringify(data);
}

function safeJsonParse(text: string): unknown {
    return JSON.parse(text);
}

These can be used in any situation where JSON serialization is necessary.

I've been using this strategy in my projects for several years now, and it has demonstrated its effectiveness by promptly detecting potential errors during application development.

I hope this article has provided you with some fresh insights. Thank you for reading!

Useful Links

• Making TypeScript Truly "Strongly Typed"
• Improving TypeScript’s Standard Library Types
• The superstuct library
• TS playground

History

• 26^th February, 2024: Initial version