ES

Navigate back to the homepage

Dynamic Types Validation in Typescript

Ema Suriano
June 1st, 2020 · 8 min read

There is no doubt that Typescript has gained a huge adoption on the JavaScript ecosystem, and one of the great benefits it provides is the type checking of all the variables inside our code. It will check if performing any operation on a variable is possible given its type.

I gave a talk about the same topic in the Typescript Meetup of Berlin. This article and the talk cover the same content, so you pick any of them to learn about this topic!

Link to the talk

Most of the people think that by using Typescript as their application language, they are “covered” from any emptiness error, like the classic “undefined is not a function” or my favorite “can’t read property X of undefined”.

This assumption is wrong, and the best way to demonstrate it is with code!

Why will Typescript NOT always cover you? 🕵

The following example does not present any Typescript error.

1// Typescript definition
2type ExampleType = {
3 name: string,
4 age?: number,
5 pets: {
6 name: string,
7 legs: number,
8 }[],
9};
10
11// communicates with external API
12const fetchData = (): Promise<ExampleType> => {};
13
14const getBiped = async () => {
15 const data = await fetchData();
16 console.log(data);
17 // { name: 'John' }
18 return data.pets.find((pet) => pet.legs === 2); // Boom!
19};

The snippet contains:

  • ExampleType: A type definition with two properties required name and pets, and one optional age. The property pets is an array of objects with name and legs, both required.
  • fetchData: A function to retrieve data from an external endpoint.
  • getBiped: Another function that will call fetchData and then iterate over the pets, and return only the pets with two legs.

So, why my script will fail when I execute it? The reason is, the external API is returning an object which doesn’t contain pets inside, and then when you try to execute data.pets.find(), you will receive the error of Uncaught ReferenceError: Cannot read property 'find' of undefined.

Inside the React official Documentation you can find a very nice definition of what Typescript is:

TypeScript is a programming language developed by Microsoft. It is a typed superset of JavaScript and includes its compiler. Being a typed language, TypeScript can catch errors and bugs at build time, long before your app goes live.

Given that definition, it’s possible to formulate a new assumption:

Typescript performs static type validation. Developers should take of dynamic validations.

So then: Do you need to validate everything? 🤔

Simply, No 🎉

Checking all the variables of our application is time-consuming from a development and performance perspective. A nice rule of thumb you can follow is:

Validate all the external sources of your application.

External sources are everything that it’s external or doesn’t have access to your application. Some examples:

  • APIs responses
  • Content inside files
  • Input from the user
  • Untyped Libraries

An application will always present at least one external source, otherwise, it will be very useless. Therefore, let’s take a look at how you can write validations for your object in Typescript.

To keep things simple, the original snippet will be considered the base and on top, I will show how to implement each of the Validation Methods.


Manual Validation 👷‍♂️

Most basic validation, it’s a set of conditions that check if the structure is the expected one.

1const validate = (data: ExampleType) => {
2 if (!data.pets) return false;
3 // perform more checks
4
5 return true;
6};
7
8const getBiped = async () => {
9 const data = await fetchData();
10 console.log(data);
11 // { name: 'John' }
12
13 if (!validate(data))
14 throw Error('Validation error: data is not complete ...');
15
16 return data.pets.find((pet) => pet.legs === 2);
17};

As you can see a new function has been defined, called validate, which receives as a parameter an ExampleType object, with which is going to check if the property pets is defined or not. In case not, it will return false, which will end up throwing an Error with a description message. Otherwise, it will continue the execution and now when evaluating data.pets.find, it won’t throw an error.

Be aware that the implementation that the validate function is quite simple, and there is room for many more checks, such as:

  • name should exist
  • name should be a string
  • if age exists, it should be a number.
  • pets should be an array of object.
  • each pet Object should have a property name and legs.

The more checks you add, the more robust your application will be, but the more time you need to invest too.

The advantages of this method are:

  • No external library required: only pure Typescript.
  • Business-centric: you can add any business logic inside these validators, for example, you can check that propertyA shouldn’t exist if propertyB is present.

It also presents some disadvantages:

  • Manual work: every validation has to be manually coded and this can be quite time-consuming.
  • Duplication of code: in the example, ExampleType already defines that there is a pets property and that it is required, but again inside the validation code you should still check that it’s true.
  • Room for bugs: in the previous code, there were many “bugs” or places for improvement.

Using a Validation Library ✨

Why re-inventing the wheel, right? This method consists of using any validation library to assert the structure of the objects. To name some of the most used libraries:

The validation library used for this article is ajv, nevertheless, all the conclusions also apply to the other libraries.

1const Ajv = require('ajv');
2const ajv = new Ajv();
3
4const validate = ajv.compile({
5 properties: {
6 name: {
7 type: 'string',
8 minLength: 3,
9 },
10 age: { type: 'number' },
11 pets: {
12 type: 'array',
13 items: {
14 name: {
15 type: 'string',
16 minLength: 3,
17 },
18 legs: { type: 'number' },
19 },
20 },
21 },
22 required: ['name', 'pets'],
23 type: 'object',
24});
25
26const getBiped = async () => {
27 const data = await fetchData();
28 console.log(data);
29 // { name: 'John' }
30 if (!validate(data)) {
31 throw Error('Validation failed: ' + ajv.errorsText(validate.errors));
32 // Error: Validation failed: data should have required property 'pets'
33 }
34
35 return data.pets.find((pet) => pet.legs === 2);
36};

Many validation libraries force you to define a schema where you can describe the structure to evaluate. Given that schema, you can create the validation function which is going to be used in your code.

The declaration of your schema will always depend on the library you are using, therefore I always recommend checking the official docs. For the case of ajv, it enforces you to declare in an Object Style, where each property has to provide the type of it and, also, it’s possible to set custom checker for these values, like minLength for any array or string.

This method provides:

  • A standardized way to create validators and checks: the idea behind the schema is to have only one way to check for specific conditions inside your application. Especially in JavaScript where there are many ways to accomplish the same task, such as checking the length of an array. This quality is great to improve communication and collaboration inside a team.
  • Improvement of Error Reporting: in case there is a mismatch on some property the library will inform you which one is the property in a human-friendly way, rather than just printing the Stack Trace.

This new way of creating validations present the following drawbacks:

  • Introduction of new Syntax: when a team decides to add a new library, the difficulty degree to understand the whole codebase grows too. Every contributor has to know about the validator schema to understand how to make a change on it.
  • Validators and Types are not in Sync: the definition of the schema and ExampleType are disconnected, which means that every time you make a change inside the ExampleType, you have to manually reflect it inside the schema. Depending on how many validators, you have this task can be quite tedious.

One small comment regarding keeping in Sync Validators and Types, some open-source projects address this issue, such as json-schema-to-typescript, which can generate a Type definition from an existing Schema. Then this won’t be considered a problem.


Dynamic Types Validator 🔌

This is the method I want to talk about, and it presents a change of paradigm regarding how to create Validators and keep Types in sync.

In the two other methods, the Validator and the Type can be seen as different entities. The Validator will take the incoming Object and check its properties, and the Type statically belongs to the Object. Combining both entities, the result is a Validated Types Object.

Before
Before

The method of Dynamic Types Validator allows a Type to generate a validator from his definition. Now they have are related, where a Validator depends entirely on a Type, preventing any mismatch between structures.

Now
Now

Generation of Validators 🤖

To generate these Validators, I found an amazing Open Source project called typescript-json-validator, made by @ForbesLindesay. The description of the repository is:

Automatically generate a validator using JSON Schema and AJV for any TypeScript type.

For the test, lets re-use the ExampleType definition, which now has been moved to a separate file inside the types folder.

1// src/types/ExampleType.ts
2
3type ExampleType = {
4 name: string;
5 age?: number;
6 pets: {
7 name: string;
8 legs: number;
9 }[];
10};

This library exposes a handy CLI that can be called from anywhere, and given a file path and the name of the Type, it will generate in the same location as the file a new file with the Validator code.

1> npx typescript-json-validator src/types/ExampleType.ts ExampleType
2# ExampleType.validator.ts created!

The resulting validator can be a very long file, so let’s take a look piece by piece:

1. Creating the instance of ajv

It also sets some default configuration for ajv.

1/* tslint:disable */
2// generated by typescript-json-validator
3import { inspect } from 'util';
4import Ajv = require('ajv');
5import ExampleType from './ExampleType';
6
7export const ajv = new Ajv({
8 allErrors: true,
9 coerceTypes: false,
10 format: 'fast',
11 nullable: true,
12 unicode: true,
13 uniqueItems: true,
14 useDefaults: true,
15});
16
17ajv.addMetaSchema(require('ajv/lib/refs/json-schema-draft-06.json'));
18
19export { ExampleType };

2. Definition of the schema from the Type

This is the key of this approach.

1// Definition of Schema
2export const ExampleTypeSchema = {
3 $schema: 'http://json-schema.org/draft-07/schema#',
4 defaultProperties: [],
5 properties: {
6 age: {
7 type: 'number',
8 },
9 name: {
10 type: 'string',
11 },
12 pets: {
13 items: {
14 defaultProperties: [],
15 properties: {
16 legs: {
17 type: 'number',
18 },
19 name: {
20 type: 'string',
21 },
22 },
23 required: ['legs', 'name'],
24 type: 'object',
25 },
26 type: 'array',
27 },
28 },
29 required: ['name', 'pets'],
30 type: 'object',
31};

3. Export validation function using the generated schema

It takes care also of throwing an Exception in case there is an error.

1export type ValidateFunction<T> = ((data: unknown) => data is T) &
2 Pick<Ajv.ValidateFunction, 'errors'>;
3export const isExampleType = ajv.compile(ExampleTypeSchema) as ValidateFunction<
4 ExampleType
5>;
6
7export default function validate(value: unknown): ExampleType {
8 if (isExampleType(value)) {
9 return value;
10 } else {
11 throw new Error(
12 ajv.errorsText(
13 isExampleType.errors!.filter((e: any) => e.keyword !== 'if'),
14 { dataVar: 'ExampleType' },
15 ) +
16 '\n\n' +
17 inspect(value),
18 );
19 }
20}

To use the validator, you just need to import from the respective path and call it. Be aware that this function is already checking if there were any errors inside the object, therefore it’s not needed to add an if statement here, making the code much cleaner.

1import validate from 'src/types/ExampleType.validator';
2
3const getBiped = async () => {
4 const data = validate(await fetchData());
5
6 return data.pets.find((pet) => pet.legs === 2);
7};

Typescript ❤️ Ajv

This library uses ajv under the hood to create the Validator function, which means that you can make use of all nice features it provides, such as custom validation for types.

Let’s create a new definition type for ExampleType.

1interface ExampleType {
2 /**
3 * @format email
4 */
5 email?: string;
6 /**
7 * @minimum 0
8 * @maximum 100
9 */
10 answer: number;
11}

Above each property you find some annotations made inside comments bracket, these will be translated into ajv rules when the library generates the final Schema. This is the result:

1export const ExampleTypeSchema = {
2 $schema: 'http://json-schema.org/draft-07/schema#',
3 defaultProperties: [],
4 properties: {
5 answer: {
6 maximum: 100,
7 minimum: 0,
8 type: 'number',
9 },
10 email: {
11 format: 'email',
12 type: 'string',
13 },
14 },
15 required: ['answer'],
16 type: 'object',
17};

The property answer presents now two more attributes that will check if the number is between 0 and 100. In the case of email, it will check if the string value belongs to a valid email address.

As these annotations are wrapped inside comments, they don’t present any conflict with the Typescript compiler.

Making it part of your workflow

This method is based on the idea that the developer will run the CLI command and generate the validators, otherwise, it exists the possibility that the schema was generated with an older version of the Type, and then it can present mismatches.

Fixing this issue is quite easy, you have to simply add a script that will be executed before your code will run. You can call it prebuild or prestart, and this is how your package.json can look like:

1{
2 "scripts": {
3 "prebuild": "typescript-json-validator src/types/ExampleType.ts ExampleType",
4 "start": "yarn prebuild && ts-node start.ts",
5 "build": "yarn prebuild && tsc"
6 }
7}

One last recommendation, I recommend ignoring any validator.ts file from your project, because there is no point in committing these files to your repository as they are going to be generated every time you start your project.


My experience with this approach 🙋‍♂️

About 2 months ago, I open-sourced one of my side projects called gatsby-starter-linkedin-resume.

In summary, it’s a Gatsby Starter that can retrieve your information from Linked In, using a Linked In Crawler, and generate an HTML and PDF resume from it, using JSON Resume.

The project presents two main flows:

  1. Create the resume information: you will be asked to enter your Linked In credentials, then a crawler will open a new browser, read your profile values and finally save all this information inside a JSON file in your directory. After that, the project will transform the data extracted from the crawler into the structure for Json Resume.
  2. Build the project: once the resume information has been processed, Gatsby can generate HTML and PDF with it.

At the beginning of this article, I mentioned that it’s advisable to validate your external sources. For this project they are:

  1. Data coming from the Linked-in crawler: when dealing with crawlers you should always be very careful with their outcome because it’s highly attached to the website that they are getting the data from. In case there is a change on the website, the output from the crawler can be altered.
  2. Local file with the resume information: this project allows you to change the content of your resume manually, in reality in case you want to you can skip the creation of the resume information and create it by yourself. If the structure of the resume data is wrong, JSON Resume won’t be able to generate the resume properly.

These are the Type definition for each case:

1interface LinkedInSchema {
2 contact: ContactItem[];
3 profile: ProfileData;
4 positions: LinkedInPosition[];
5 educations: LinkedInEducation[];
6 skills: Skill[];
7 courses: Course[];
8 languages: LinkedInLanguage[];
9 projects: LinkedInProject[];
10}
11
12interface JsonResumeSchema {
13 basics: JsonResumeBasics;
14 work: JsonResumeWork[];
15 volunteer?: JsonResumeVolunteer[];
16 education: JsonResumeEducation[];
17 awards?: JsonResumeAward[];
18 publications?: JsonResumePublication[];
19 skills?: JsonResumeSkill[];
20 languages?: JsonResumeLanguage[];
21 interests?: JsonResumeInterest[];
22 references?: JsonResumeReference[];
23 projects?: JsonResumeProject[];
24}

Both Types present similarities in terms of variable names, but their internal structure differs. This is why it’s necessary to transform from one structure to the other on the first flow.

After I set up my project to generate the Validators from these types, checking the structure of the incoming object was a very easy task.

Validation of the Crawler result

1// src/index.ts
2import { RESUME_PATH, LINKED_IN_PATH } from './utils/path';
3import validateLinkedInSchema from './types/LinkedInSchema.validator';
4import { saveJson, readJson } from './utils/file';
5import { inquireLoginData, getLinkedInData } from './utils/linkedin';
6
7// ❗️❗️ IMPORT OF THE VALIDATOR ❗️❗️
8import mapLinkedInToJSONResume from './utils/mapLinkedInToJSONResume';
9
10export const main = async ({ renew }) => {
11 if (renew || !readJson(LINKED_IN_PATH)) {
12 const credentials = await inquireLoginData();
13 const linkedInData = await getLinkedInData(credentials);
14
15 saveJson(LINKED_IN_PATH, linkedInData);
16 }
17
18 // ❗️❗️ VALIDATION IN ACTION ❗️❗️
19 const linkedInParsed = validateLinkedInSchema(readJson(LINKED_IN_PATH));
20
21 const jsonResumeData = mapLinkedInToJSONResume(linkedInParsed);
22 saveJson(RESUME_PATH, jsonResumeData);
23};

Validation of the Resume Information

1// gatsby-config.js
2const { existsSync } = require('fs');
3
4// ❗️❗️ IMPORT OF THE VALIDATOR ❗️❗️
5const {
6 default: validateJsonResume,
7} = require('./lib/types/JsonResumeSchema.validator');
8
9if (!existsSync('./resume.json')) {
10 throw new Error(
11 'Please run "yarn generate-resume" to generate your resume information.',
12 );
13}
14
15// ❗️❗️ VALIDATION IN ACTION ❗️❗️
16const resumeJson = validateJsonResume(require('./resume.json'));
17
18module.exports = {
19 plugins: [
20 {
21 resolve: 'gatsby-theme-jsonresume',
22 options: {
23 resumeJson,
24 },
25 },
26 'gatsby-plugin-meta-redirect',
27 ],
28};

Closing Words 🗣

To sump up, I created this table comparing the three methods, in which the Dynamic Types approach shows that it grabs the best of the other methods, making it the recommended approach to validate your object.

ApproachManualLibraryDynamic Types
No additional Syntax
Validators and Types Sync
Standardization

In case you are working in a Typescript codebase, I recommend you to give a try to this new of validating your object. It’s very easy to make the set up for it, and in case you don’t find it useful removing from the codebase is as easy as removing an import from your files.

🚨 Get notified for my next article!

I tend to write about my challenges inside the weird, fast and hot Frontend world The challenges can be from learning a specific tool or framework to building a project from scratch.

I try to publish one article per month, but yeah sometimes life gets in the middle ... No SPAM, no hiring, no application marketing, just tech posts 👌

More articles from Ema Suriano

End to end testing in React Native with Detox

End-to-end testing is a technique that is widely performed in the web ecosystem with frameworks like Cypress, Puppeteer, or maybe with your own custom implementation.

October 9th, 2019 · 5 min read

Building a maintainable Icon System for React and React Native

Implementing a maintainable icon system for a React and React Native project can be a hard task, especially when it comes to achieving the same workflow to add/remove/use an icon in all the platform (Web, Android, and iOS). In this post, I will share how we implemented a consistent icon system inside our component library at Omio.

October 1st, 2019 · 5 min read
© 2018–2020 Ema Suriano
Link to $https://github.com/EmaSurianoLink to $https://twitter.com/EmaSurianoLink to $https://linkedin.com/EmaSurianoLink to $https://dev.to/emasurianoLink to $https://medium.com/@emasurianoLink to $https://www.youtube.com/c/EmaSuriano