This article explores the challenges of data serialization in TypeScript when using the JSON format. It particularly focuses on the shortcomings of JSON.stringify and JSON.parse functions. To address these issues, it suggests the use of JSONCompatible type to verify if a type T can be safely serialized to JSON. Furthermore, it recommends the Superstruct library for safe deserialization from JSON. This method improves type safety and enables error detection during development.
Introduction
Almost every web application requires data serialization. This need arises in situations like:
- Transferring data over the network (e.g., HTTP requests,
WebSocket
s) - Embedding data in HTML (for hydration, for instance)
- Storing data in a persistent storage (like
LocalStorage
) - Sharing data between processes (like web workers or
postMessage
)
In many cases, data loss or corruption can lead to serious consequences, making it essential to provide a convenient and safe serialization mechanism that helps detect as many errors as possible during the development stage. For these purposes, it's convenient to use JSON as the data transfer format and TypeScript for static code checking during development.
TypeScript serves as a superset of JavaScript, which should enable the seamless use of functions like JSON.stringify
and JSON.parse
, right? Turns out, despite all its benefits, TypeScript doesn't naturally understand what JSON is and which data types are safe for serialization and deserialization into JSON. Let's illustrate this with an example.
The Problem with JSON in TypeScript
Consider, for example, a function that saves some data to LocalStorage
. As LocalStorage
cannot store objects, we use JSON serialization here:
interface PostComment {
authorId: string;
text: string;
updatedAt: Date;
}
function saveComment(comment: PostComment) {
const serializedComment = JSON.stringify(comment);
localStorage.setItem('draft', serializedComment);
}
We will also need a function to retrieve the data from LocalStorage
.
function restoreComment(): PostComment | undefined {
const text = localStorage.getItem('draft');
return text ? JSON.parse(text) : undefined;
}
What’s wrong with this code? The first problem is that when restoring the comment, we will get a string
type instead of Date
for the updatedAt
field.
This happens because JSON only has four primitive data types (null
, string
, number
, boolean
), as well as arrays and objects. It is not possible to save a Date
object in JSON, as well as other objects that are found in JavaScript: functions
, Map
, Set
, etc.
When JSON.stringify
encounters a value that cannot be represented in JSON format, type casting occurs. In the case of a Date
object, we get a string
because the Date
object implements the toJson() method, which returns a string
instead of a Date
object.
const date = new Date('August 19, 1975 23:15:30 UTC');
const jsonDate = date.toJSON();
console.log(jsonDate);
const isEqual = date.toJSON() === JSON.stringify(date);
console.log(isEqual);
The second problem is that the saveComment
function returns the PostComment
type, in which the date field is of type Date
. But we already know that instead of Date
, we will actually receive a string
type. TypeScript could help us find this error, but why doesn't it?
Turns out, in TypeScript's standard library, the JSON.parse
function is typed as (text: string) => any
. Due to the use of any
, type checking is essentially disabled. In our example, TypeScript simply took our word that the function would return a PostComment
containing a Date
object.
This TypeScript behavior is inconvenient and unsafe. Our application may crash if we try to treat a string
like a Date
object. For example, it might break if we call comment.updatedAt.toLocaleDateString()
.
Indeed, in our small example, we could simply replace the Date
object with a numerical timestamp, which works well for JSON serialization. But in real applications, data objects might be extensive, types can be defined in multiple locations, and identifying such an error during development may be a challenging task.
What if we could enhance TypeScript's understanding of JSON?
Dealing with Serialization
To start with, let's figure out how to make TypeScript understand which data types can be safely serialized into JSON. Suppose we want to create a function safeJsonStringify
, where TypeScript will check the input data format to ensure it's JSON serializable.
function safeJsonStringify(data: JSONValue) {
return JSON.stringify(data);
}
In this function, the most important part is the JSONValue
type, which represents all possible values that can be represented in the JSON format. The implementation is quite straightforward:
type JSONPrimitive = string | number | boolean | null | undefined;
type JSONValue = JSONPrimitive | JSONValue[] | {
[key: string]: JSONValue;
};
First, we define the JSONPrimitive
type, which describes all the primitive JSON data types. We also include the undefined
type based on the fact that when serialized, keys with the undefined
value will be omitted. During deserialization, these keys will simply not appear in the object, which in most cases is the same thing.
Next, we describe the JSONValue
type. This type uses TypeScript's ability to describe recursive types, which are types that refer to themselves. Here, JSONValue
can either be a JSONPrimitive
, an array of JSONValue
, or an object where all values are of the JSONValue
type. As a result, a variable of type JSONValue
can contain arrays and objects with unlimited nesting. The values within these will also be checked for compatibility with the JSON format.
Now we can test our safeJsonStringify
function using the following examples:
safeJsonStringify({
updatedAt: Date.now()
});
safeJsonStringify({
updatedAt: new Date();
});
Everything seems to function properly. The function allows us to pass the date as a number, but yields an error if we will pass the Date
object.
But let's consider a more realistic example, in which the data passed to the function is stored in a variable and has a described type.
interface PostComment {
authorId: string;
text: string;
updatedAt: number;
};
const comment: PostComment = {...};
safeJsonStringify(comment);
Now, things are getting a bit tricky. TypeScript won't let us assign a variable of type PostComment
to a function parameter of type JSONValue
, because "Index signature for type 'string
' is missing in type 'PostComment
'".
So, what is an index signature and why is it missing? Remember how we described objects that can be serialized into the JSON format?
type JSONValue = {
[key: string]: JSONValue;
};
In this case, [key: string]
is the index signature. It means "this object can have any keys in the form of string
s, the values of which have the JSONValue
type". So, it turns out we need to add an index signature to the PostComment
type, right?
interface PostComment {
authorId: string;
text: string;
updatedAt: number;
[key: string]: JSONValue;
};
In fact, doing so would imply that the comment could contain any arbitrary fields, which is not typically the desired outcome when defining data types in an application.
The real solution to the problem with the index signature comes from Mapped Types, which allow to recursively iterate over fields, even for types that don't have an index signature defined. Combined with generics, this feature allows converting any data type T
into another type JSONCompatible<T>
, which is compatible with the JSON format.
type JSONCompatible<T> = unknown extends T ? never : {
[P in keyof T]:
T[P] extends JSONValue ? T[P] :
T[P] extends NotAssignableToJson ? never :
JSONCompatible<T[P]>;
};
type NotAssignableToJson =
| bigint
| symbol
| Function;
The JSONCompatible<T>
type is a mapped type that inspects whether a given type T
can be safely serialized into JSON. It does this by iterating over each property in type T
and doing the following:
- The
T[P] extends JSONValue ? T[P] : ...
conditional type verifies if the property's type is compatible with the JSONValue
type, assuring it can be safely converted to JSON. When this is the case, the property's type remains unchanged. - The
T[P] extends NotAssignableToJson ? never : ...
conditional type verifies if the property’s type isn't assignable to JSON. In this case, the property's type is converted to never
, effectively filtering the property out from the final type. - If neither of these conditions is met, the type is recursively checked until a conclusion can be made. This way, it works even if the type doesn’t have an index signature.
The unknown extends T ? never :...
check at the beginning is used to prevent the unknown
type from being converted to an empty object type {}
, which is essentially equivalent to the any
type.
Another interesting aspect is the NotAssignableToJson
type. It consists of two TypeScript primitives (bigint
and symbol) and the Function
type, which describes any possible function. The Function
type is crucial in filtering out any values which aren't assignable to JSON. This is because any complex object in JavaScript is based on the Object
type and has at least one function in its prototype chain (e.g., toString()
). The JSONCompatible
type iterates over all of those functions, so checking functions is sufficient to filter out anything that isn't serializable to JSON.
Now, let's use this type in the serialization function:
function safeJsonStringify<T>(data: JSONCompatible<T>) {
return JSON.stringify(data);
}
Now, the function uses a generic parameter T
and accepts the JSONCompatible<T>
argument. This means it takes an argument data
of type T
, which should be a JSON compatible type. Now we can use the function with data types without an index signature.
The function now uses a generic parameter T
that extends from the JSONCompatible<T>
type. This means that it accepts an argument data
of type T
, which ought to be a JSON compatible type. As a result, we can utilize the function with data types that lack an index signature.
interface PostComment {
authorId: string;
text: string;
updatedAt: number;
}
function saveComment(comment: PostComment) {
const serializedComment = safeJsonStringify(comment);
localStorage.setItem('draft', serializedComment);
}
This approach can be used whenever JSON serialization is necessary, such as transferring data over the network, embedding data in HTML, storing data in localStorage
, transferring data between workers, etc. Additionally, the toJsonValue
helper can be used when a strictly typed object without an index signature needs to be assigned to a variable of JSONValue
type.
function toJsonValue<T>(value: JSONCompatible<T>): JSONValue {
return value;
}
const comment: PostComment = {...};
const data: JSONValue = {
comment: toJsonValue(comment)
};
In this example, using toJsonValue
lets us bypass the error related to the missing index signature in the PostComment
type.
Dealing with Deserialization
When it comes to deserialization, the challenge is both simpler and more complex simultaneously because it involves both static analysis checks and runtime checks for the received data's format.
From the perspective of TypeScript's type system, the challenge is quite simple. Let's consider the following example:
function safeJsonParse(text: string) {
return JSON.parse(text) as unknown;
}
const data = JSON.parse(text);
In this instance, we're substituting the any
return type with the unknown
type. Why choose unknown
? Essentially, a JSON string
could contain anything, not just the data what we expect to receive. For example, the data format might change between different application versions, or another part of the app could write data to the same LocalStorage
key. Therefore, unknown
is the safest and most precise choice.
However, working with the unknown
type is less convenient than merely specifying the desired data type. Apart from type-casting, there are multiple ways to convert the unknown
type into the required data type. One such method is utilizing the Superstruct library to validate data at runtime and throw detailed errors if the data is invalid.
import { create, object, number, string } from 'superstruct';
const PostComment = object({
authorId: string(),
text: string(),
updatedAt: number(),
});
function restoreDraft() {
const text = localStorage.getItem('draft');
return text ? create(JSON.parse(text), PostComment) : undefined;
}
Here, the create
function acts as a type guard, narrowing the type to the desired Comment
interface. Consequently, we no longer need to manually specify the return type.
Implementing a secure deserialization option is only half the story. It's equally crucial not to forget to use it when tackling the next task in the project. This becomes particularly challenging if a large team is working on the project, as ensuring all agreements and best practices are followed can be difficult.
Typescript-eslint can assist in this task. This tool helps identify all instances of unsafe any
usage. Specifically, all usages of JSON.parse
can be found and it can be ensured that the received data's format is checked. More about getting rid of the any
type in a codebase can be read in the article Making TypeScript Truly "Strongly Typed".
Conclusion
Here are the final utility functions and types designed to assist in safe JSON serialization and deserialization. You can test these in the prepared TS Playground.
type JSONPrimitive = string | number | boolean | null | undefined;
type JSONValue = JSONPrimitive | JSONValue[] | {
[key: string]: JSONValue;
};
type NotAssignableToJson =
| bigint
| symbol
| Function;
type JSONCompatible<T> = unknown extends T ? never : {
[P in keyof T]:
T[P] extends JSONValue ? T[P] :
T[P] extends NotAssignableToJson ? never :
JSONCompatible<T[P]>;
};
function toJsonValue<T>(value: JSONCompatible<T>): JSONValue {
return value;
}
function safeJsonStringify<T>(data: JSONCompatible<T>) {
return JSON.stringify(data);
}
function safeJsonParse(text: string): unknown {
return JSON.parse(text);
}
These can be used in any situation where JSON serialization is necessary.
I've been using this strategy in my projects for several years now, and it has demonstrated its effectiveness by promptly detecting potential errors during application development.
I hope this article has provided you with some fresh insights. Thank you for reading!
Useful Links
History
- 26th February, 2024: Initial version