Node Native Test Runner Benchmark
It is August 2023, version 20.5.1 was released five days back, and now we can test a stable version of this fantastic new native feature, the fabulous TestRunner.
Just to give it a brief intro, tests, what are they, what do they eat, where do they live ?!?!
We use unit tests to check for predictable behaviours, to see if after a change on our codebase, everything still working the way we expect, it is basically a way to protect us from ourselves, and when we have dependencies on outside resources like an API, a Service, Human Interaction (clicking on ‘ok’ or something small), we mock them all, we trust on them as we have no other option.
Now with a quick and small ChatGPT query, we can list a few of the JavaScript test frameworks out there:
Not planning to test against all of them on this first page so I will go with the ones I am more used to, let’s try and check how we could do the most basic test suite there is, for a comparison on how to do more complex stuff on all of them, I will build a new post about it in the future.
Now we need to have a quick look at what will be tested, let’s see the codebase:
export default function parseParams(baseObject, params = new URLSearchParams(window.location.search), arrSeparator = ',') {
if(!baseObject || typeof(baseObject) !== 'object') throw new Error('baseObject property must be declared and a object');
if(!params?.get) throw new Error('search params must be initialized like "new URLSearchParams(query)"');
if(!arrSeparator) throw new Error('arr separator must not be empty');
const obj = {};
for(const key in baseObject) {
const param = params.get(key);
if(param !== null)
obj[key] = parseByType(param, baseObject[key], arrSeparator);
else
obj[key] = null;
}
return obj;
}
export function parseByType (variable, property, arrSeparator) {
if (typeof(variable) !== 'string') throw new Error('variable and type must be strings');
let result;
switch(typeof(property)){
case 'string':
result = variable;
break;
case 'number':
result = +variable;
break;
case 'object':
if(Array.isArray(property))
result = variable.split(arrSeparator);
break;
case 'boolean':
result = stringToBoolean(variable);
break;
default:
console.log(`type ${typeof(property)} not mapped`)
}
return result;
}
export function stringToBoolean (stringValue) {
switch(stringValue?.toLowerCase()?.trim()){
case "true":
case "yes":
case "1":
return true;
case "false":
case "no":
case "0":
case null:
case undefined:
return false;
default:
return false;
}
}it is a simple urlQueryParser published here but simple or not we need to test it obviously!
And another quick look at how it will be tested
describe('Main parseParams method', () => {
test('should throw when there is no object passed', () => {
assert.throws(() => {
const result = parseParams('not an object', new URLSearchParams(''));
},
{
message: 'baseObject property must be declared and a object'
})
});
test('should throw when there is no URLSearchParams and is passed as null', () => {
assert.throws(() => {
const result = parseParams({ item1: 0}, null, null);
},
{
message: 'search params must be initialized like "new URLSearchParams(query)"'
});
});
test('should throw when the array separator is null', () => {
assert.throws(() => {
const result = parseParams({ item1: 0}, new URLSearchParams(''), null);
},
{
message: 'arr separator must not be empty'
});
});
});
describe('parseByType method', () => {
test('should throw when the variable type is not string', () => {
assert.throws(() => {
parseByType(true, 'name', ',');
},
{
message: 'variable and type must be strings'
});
});
test('should test the parsed value from query string value', () => {
const toBeParsed = [
{ value: 'toString', type: 'string', expected: 'toString' },
{ value: 'yEs', type: false, expected: true, },
{ value: '100', type: 0, expected: 100 },
{ value: 'var1,var2', type: [], expected: [ 'var1', 'var2' ] },
]
for(const toParse of toBeParsed) {
assert.deepEqual(
parseByType(toParse.value, toParse.type, ','),
toParse.expected,
`type (${typeof(toParse.type)}) is failing`
);
}
});
});
describe('stringToBoolean method', () => {
test('test variations of true values', () => {
const trueStrings = ['yes', 'YeS', '1', 'true', 'TrUe'];
for(const string of trueStrings) {
assert.strictEqual(stringToBoolean(string), true);
}
});
test('test variations of false values', () => {
const falseStrings = ['no', 'nO', '0', undefined, null];
for(const string of falseStrings) {
assert.strictEqual(stringToBoolean(string), false);
}
});
});
keep in mind that we are not looking for complexity, just the plain simple test for engine performance.
We have 3 test engines (Mocha, Jest and Node Test Runner), let’s figure them out and configure them as well.
Mocha
This one is simple, with no need to worry about ECMAScript or CommonJS at all, a simple straight forward npm i -D mocha will do the trick.
At the top of the file let’s directly import all the test tools we need
import { describe, test } from 'mocha';
import * as assert from 'node:assert';on Mocha, we can use the native node:assert or even a few other libraries like should.js expect.js chai , you may find the list here mocha docs.
to keep it small I will use the native one, which is available since the beta version of Node.js, as you can find here.
Now the package.json
{
"name": "nodejs-unity-tests-benchmark",
"version": "0.1.0",
"license": "ISC",
"engines": {
"node": "20.5.1"
},
"description": "",
"type": "module",
"main": "index.js",
"scripts": {
"test:mocha": "mocha index.mocha.spec.js"
},
"dependencies": {
"mocha": "^10.2.0"
}
}All set, let’s see how the result looks like
7ms, that looks like a lot, we will dive a bit deeper into that one later in our results review
Jest
This one requires just a bit more complexity, it is based on CommonJS, so we need to configure it a bit.
for ES modules we need to follow this doc, as I am using Windows, I will do the bigger path with npm i -D jest cross-env
Import the proper test suite again
import { describe, test, expect } from '@jest/globals'And configure the package.json
{
"name": "nodejs-unity-tests-benchmark",
"version": "0.1.0",
"license": "ISC",
"engines": {
"node": "20.5.1"
},
"description": "",
"type": "module",
"main": "index.js",
"scripts": {
"test:mocha": "mocha index.mocha.spec.js",
"test:jest": "cross-env NODE_OPTIONS=--experimental-vm-modules npx jest index.jest.spec.js"
},
"dependencies": {
"cross-env": "^7.0.3",
"jest": "^29.6.2",
"mocha": "^10.2.0"
}
}Now the result preview
Node.js
The last one is way more simple, no need to install anything, just import the proper suite
import { describe, test } from 'node:test';
import * as assert from 'node:assert';on Package.json
{
"name": "nodejs-unity-tests-benchmark",
"version": "0.1.0",
"license": "ISC",
"engines": {
"node": "20.5.1"
},
"description": "",
"type": "module",
"main": "index.js",
"scripts": {
"test:node": "node index.node.spec.js",
"test:mocha": "mocha index.mocha.spec.js",
"test:jest": "cross-env NODE_OPTIONS=--experimental-vm-modules npx jest index.jest.spec.js"
},
"dependencies": {
"cross-env": "^7.0.3",
"jest": "^29.6.2",
"mocha": "^10.2.0"
}
}the result
Results Review
Amazing, now we have something to work with, running them all 10 times each I ended up with this
Quite unfair, Mocha was showing results 20 times bigger, let’s change Mocha a bit and run it all in parallel, "test:mocha": "mocha index.mocha.spec.js --parallel" , now the results will be more satisfying as follows:
Amazing, now we can see stuff!
Let’s summarize!
1st — Node.js shows an average of 0.1388 ms
2nd — Mocha with 0.1694ms
3rd — Jest with almost twice as the other two, 0.2514ms
With this in hand, the first prize obviously goes to Node.js for performance, but if you think about it, it was unfair anyways as Node.js will come with this right out of the box, we don’t need to install anything, it is a simple action of using it really. Obviously, this was a pretty poor test with no complexity, no extra functionalities like spies or mocks, no async or promises, and not even a simple report for code coverage was shown there.
Pros and Cons
Cons :
- Mocha: it requires a simple config o
--parallelto be significantly faster, which is just fine as you only need to do this once; - Jest: it is natively compatible with CommonJS, so we need to configure node env vars, and for Windows, we need to download the
cross-envlibrary, not a good experience really; - Node: I didn’t find anything that bad honestly.
Pros :
- Mocha: it already has code coverage tools, you can filter tests based on test descriptions with the
--grepflag, here, didn’t find this for Node.js - Jest: we have it natively configured for many frameworks, if you use Next.js, (deprecated) Create React App, maybe you will find it on other frameworks.
- Node: out of the box feature with no need for dependencies, faster, no configuration.
Thank you a lot for reading this simple article, I am planning on doing a few other ones comparing applications, and how those libraries differ from each other when actually coding something.
