Security and Virtualization: Testing in Node.js

A test driven development cycle simplifies the thought process of writing code, makes it easier, and quicker in the long run. But just writing tests is not enough by itself, knowing the kinds of tests to write and how to structure code to conform to this pattern is what it's all about. In this article we will take a look at building a small app in Node.js following a TDD pattern.
Besides simple 'unit' tests, which we are all familiar with; We can also have Node.js'snc code running, which adds an extra dimension in that we don't always know the order in which functions will run or we may be trying to test something in a callback or checking to see how an async function is working.
In this article we will be building a Node app which can search for files that match a given query. I know there are already things for this (ack) but for the sake of demonstrating TDD I think it could be a well rounded project.
The first step is obviously to write some tests, but even before that, we need to choose a testing framework. You can use vanilla Node, as there is an assert library built-in, but it's not much in terms of a test runner, and is pretty much the bare essentials.
Another option and probably my favorite for general use is Jasmine. It's pretty self-contained, you don't have any other dependencies to add to your scripts and the syntax is very clean and easy to read. The only reason I am not going to use this today, is because I think Jack Franklin did an excellent job covering this in his recent Tuts+ series here, and it's good to know your options so you can pick the best tool for your situation.

What We'll Be Building

In this article we will be using the flexible 'Mocha' test runner along with the Chai assertion library.
Unlike Jasmine which is more like an entire test suite in one package, Mocha only takes care of the overall structure but has nothing to do with the actual assertions. This allows you to keep a consistent look and feel when running your tests, but also allows you to run whichever assertion library best fits your situation.
So for example, if you were going to use the vanilla 'assert' library, you could pair it with Mocha to add some structure to your tests.
Chai is a fairly popular option, and is also all about options and modularity. Even without any plugins, just using the default API you have three different syntaxes you can use depending on if you would like to use a more classic TDD style or a more verbose BDD syntax.
So now that we know what we are going to use, let's get into the installation.

The Setup

To get started, let's install Mocha globally by running:

1	`npm` `install` `-g mocha`

When that completes create a new folder for our project and run the following inside it:

1	`npm` `install` `chai`

This will install a local copy of Chai for our project. Next, create a folder named test inside our project's directory, as this is the default location Mocha will look for tests.
That's pretty much it for setup, the next step is to talk about how to structure your apps when following a test driven development process.

Structuring Your App

It's important to know, when following a TDD approach, what needs to have tests and what does not. A rule of thumb is to not write tests for other peoples already tested code. What I mean by this is the following: let's say your code opens a file, you don't need to test the individual fs function, it's part of the languge and is supposedly already well tested. The same goes when using third-party libraries, you shouldn't structure functions which primarily call these types of functions. You don't really write tests for these and because of this you have gaps in the TDD cycle.
Now of course, with every programming style there are a lot of different opinions and people will have different views on how to TDD. But the approach I use is that you create individual components to use in your app, each of which solves a unique functional problem. These components are built using TDD ensuring that they work as expected and you won't break their API. Then you write your main script, which is essentially all glue code, and does not need to be tested / can't be tested, in certain situations.
This also means that most of your components can be reused in the future as they do not really have much to do, directly, with the main script.
Following what I just said, it's common practice to create a folder named 'lib' where you put all the individual components. So up to this point you should have Mocha and Chai installed, and then a project directory with two folders: 'lib' and 'test'.

Getting Started With TDD

Just in case you are new to TDD I thought it would be a good idea to quickly cover the process. The basic rule is that you can't write any code unless the test runner tells you to.
Essentially, you're writing what your code is supposed to do before actually doing it. You have a really focused goal while coding and you never compromise your idea by getting side-tracked or thinking too far ahead. Besides that, since all of your code will have a test affiliated with it you can be certain you will never break your app in the future.
A test, in reality, is just a declaration of what a function is expected to do when run, you then run your test runner, which will obviously fail (since you haven't written the code yet) and then you write the minimum amount of code needed to pass the failing test. It's important never to skip this step, because sometimes a test will pass even before you add any code, due to other code you have in the same class or function. When this happens, you either wrote more code then you were supposed to for a different test or this is just a bad test (usually not specific enough).
Again according to our rule above, if the test passes right away you can't write any code, because it didn't tell you to. By continuously writing tests and then implementing the features you construct solid modules that you can rely on.
Once you're finished implementing and testing your component, you can then go back and refactor the code to optimize it and clean it up but making sure the refactoring doesn't fail any of the tests you have in place and more importantly, doesn't add any features that are untested.
Every testing library will have its own syntax, but they usually follow the same pattern of making assertions and then checking if they pass. Since we are using Mocha and Chai let's take a look at both their syntaxes starting with Chai.

Mocha & Chai

I will be using the 'Expect' BDD syntax, because as I mentioned Chai comes with a few options out of the box. The way this syntax works is you start by calling the expect function, passing it the object you want to make an assertion on, and then you chain it with a specific test. An example of what I mean could be as follows:

1	`expect(4+5).equal(9);`

That's the basic syntax, we are saying expect the addition of 4 and 5 to equal 9. Now this isn't a great test because the 4 and 5 will be added by Node.js before the function is even called so we are essentially testing my math skills, but I hope you get the general idea. The other thing you should note, is this syntax is not very readable, in terms of the flow of a normal English sentence. Knowing this, Chai added the following chain getters which don't do anything but you can add them to make it more verbose and readable. The chain getters are as follows:

to
be
been
is
that
and
have
with
at
of
same
a
an

Using the above, we can rewrite our previous test to something like this:

1	`expect(4+5).to.equal(9);`

I really like the feel of the entire library, which you can check out in their API. Simple things like negating the operation is as easy as writing .not before the test:

1	`expect(4+5).to.not.equal(10);`

So even if you have never used the library before, it won't be hard to figure out what a test is trying to do.
The last thing I would like to look over before we get into our first test is how we structure our code in Mocha

Mocha

Mocha is the test runner, so it doesn't really care too much about the actual tests, what it cares about is the tests structure, because that is how it knows what is failing and how to layout the results. The way you build it up, is you create multiple describe blocks which outline the different components of your library and then you add it blocks to specify a specific test.
For a quick example, let's say we had a JSON class and that class had a function to parse JSON and we wanted to make sure the parse function can detect a badly formatted JSON string, we could structure this like so:

describe("JSON", function() {
   describe(".parse()", function() {
       it("should detect malformed JSON strings", function(){
           //Test Goes Here
       });
   });
});

It's not complicated, and it's about 80% personal preference, but if you keep this kind of format, the test results should come out in a very readable format.
We are now ready to write our first library, let's begin with a simple synchronous module, to get ourselves better acquainted with the system. Our app will need to be able to accept command line options for setting things like how many levels of folders our app should search through and the query itself.
To take care of all this, we will create a module which accepts the command's string and parses all the included options along with their values.

The Tag Module

This is a great example of a module you can reuse in all your command line apps, as this issue comes up a lot. This will be a simplified version of an actual package I have on npm called ClTags. So to get started, create a file named tags.js inside of the lib folder, and then another file named tagsSpec.js inside of the test folder.
We need to pull in the Chai expect function, as that will be the assertion syntax we will be using and we need to pull in the actual tags file so we can test it. Altogether with some initial setup it should look something like this:

var expect = require("chai").expect;
var tags = require("../lib/tags.js");
 
describe("Tags", function(){
 
});

If you run the 'mocha' command now from the root of our project, everything should be passing like expected. Now let's think about what our module will do; we want to pass it the command arguments array that was used to run the app, and then we want it to build an object with all the tags, and it would be nice if we could also pass it a default object of settings, so if nothing get's overridden, we will have some settings already stored.
When dealing with tags, a lot of apps also provide shortcut options which are just one character, so let's say we wanted to set the depth of our search we could allow the user to either specify something like --depth=2 or something like -d=2 which should have the same effect.
So let's just begin with the long formed tags (for example, '--depth=2'), To begin with, let's write the first test:

describe("Tags", function(){
   describe("#parse()", function(){
       it("should parse long formed tags", function(){
           var args = ["--depth=4", "--hello=world"];
           var results = tags.parse(args);
 
           expect(results).to.have.a.property("depth", 4);
           expect(results).to.have.a.property("hello", "world");
       });
   });
});

We added one method to our test suite called parse and we added a test for long formed tags. Inside this test I created an example command and added two assertions for the two properties it should pickup.
Running Mocha now, you should get one error, namely that tags doesn't have a parse function. So to fix this error let's add a parse function to the tags module. A fairly typical way to create a node module is like so:

exports = module.exports = {};
 
exports.parse = function() {
 
}

The error said we needed a parse method so we created it, we didn't add any other code inside because it didn't yet tell us to. By sticking with the bare minimum you are assured that you won't write more then you are supposed to and end up with untested code.
Now let's run Mocha again, this time we should be getting an error telling us that it can't read a property named depth from an undefined variable. That is because currently our parse function isn't returning anything, so let's add some code so that it will return an object:

exports.parse = function() {
var options = {}
 
return options;
}

We are slowly moving along, if you run Mocha again, their shouldn't be any exceptions being thrown, just a clean error message saying that our empty object has no property called depth.

Now we can get into some real code. For our function to parse the tag and add it to our object we need to cycle through the arguments array and remove the double dashes at the start of the key.

exports.parse = function(args) {
   var options = {}
   for (var i in args) { //Cycle through args
       var arg = args[i];
       //Check if Long formed tag
       if (arg.substr(0, 2) === "--") {
           arg = arg.substr(2);
           //Check for equals sign
           if (arg.indexOf("=") !== -1) {
               arg = arg.split("=");
               var key = arg.shift();
               options[key] = arg.join("=");
           }
       }
   }
   return options;
}

This code cycles through the list of arguments, makes sure we are dealing with a long formed tag, and then splits it by the first equals character to create the key and value pair for the options object.
Now this almost solves our issue, but if we run Mocha again, you will see that we now have a key for depth, but it's set to a string instead of a number. Numbers are a bit easier to work with later on in our app, so the next piece of code we need to add is to convert values to numbers whenever possible. This can be achieved with some RegEx and the parseInt function as follows:

if (arg.indexOf("=") !== -1) {
    arg = arg.split("=");
    var key = arg.shift();
    var value = arg.join("=");
 
    if (/^[0-9]+$/.test(value)) {
        value = parseInt(value, 10);
    }
    options[key] = value;
}

Running Mocha now, you should get a pass with one test. The number conversion should arguably be in its own test, or at least mentioned in the tests declaration so you don't, by mistake, remove the number conversion assertion; so just add-on "add and convert numbers" to the it declaration for this test or separate it into a new it block. It really depends whether you consider this "obvious default behavior" or a separate feature.

Now like I have been trying to stress throughout this whole article, when you see a passing spec, it's time to write more tests. The next thing I wanted to add was the default array, so inside the tagsSpec file let's add the following it block right after the previous one:

it("should parse long formed tags and convert numbers", function(){
    var args = ["--depth=4", "--hello=world"];
    var results = tags.parse(args);
 
    expect(results).to.have.a.property("depth", 4);
    expect(results).to.have.a.property("hello", "world");
});
it("should fallback to defaults", function(){
    var args = ["--depth=4", "--hello=world"];
    var defaults = { depth: 2, foo: "bar" };
    var results = tags.parse(args, defaults);
 
    var expected = {
        depth: 4,
        foo: "bar",
        hello: "world"
    };
 
    expect(results).to.deep.equal(expected);
});

Here we are using a new test, the deep equal which is good for matching two objects for equal values. Alternatively, you can use the eql test which is a shortcut but I think this is more clear. This test passes two arguments as the command string and passes two defaults with one overlap, just so we can get a good spread on the test cases.
Running Mocha now, you should get a sort of diff, containing the differences between what is expected and what it actually got.

Let's now continue back to the tags.js module, and let's add this functionality in. It's a fairly simple fix to add, we just need to accept the second parameter, and when it's set to an object we can replace the standard empty object at the start with this object:

exports.parse = function(args, defaults) {
   var options = {};
   if (typeof defaults === "object" && !(defaults instanceof Array)) {
       options = defaults
   }

This will bring us back to a green state. The next thing I want to add is the ability to just specify a tag without a value and let it work like a boolean. For example, if we just set --searchContents or something like that, it will just add that to our options array with a value of true.
The test for this would look something like the following:

it("should accept tags without values as a bool", function(){
    var args = ["--searchContents"];
    var results = tags.parse(args);
 
    expect(results).to.have.a.property("searchContents", true);
});

Running this will give us the following error just like before:

Inside of the for loop, when we got a match for a long formed tag, we checked if it contained an equals sign; we can quickly write the code for this test by adding an else clause to that if statement and just setting the value to true:

if (arg.indexOf("=") !== -1) {
     arg = arg.split("=");
     var key = arg.shift();
     var value = arg.join("=");
 
     if (/^[0-9]+$/.test(value)) {
         value = parseInt(value, 10);
     }
     options[key] = value;
} else {
     options[arg] = true;
}

The next thing I want to add is the substitutions for the short-hand tags. This will be the third parameter to the parse function and will basically be an object with letters and their corresponding replacements. Here is the spec for this addition:

it("should accept short formed tags", function(){
    var args = ["-sd=4", "-h"];
    var replacements = {
        s: "searchContents",
        d: "depth",
        h: "hello"
    };
 
    var results = tags.parse(args, {}, replacements);
 
    var expected = {
        searchContents: true,
        depth: 4,
        hello: true
    };
 
    expect(results).to.deep.equal(expected);
});

The trouble with shorthand tags is that they are able to be combined in a row. What I mean by this is unlike the long formed tags where each one is separate, with short hand tags - since they are each just a letter long - you can call three different ones by typing -vgh. This makes the parsing a bit more difficult because we still need to allow for the equals operator for you to add a value to the last tag mentioned, while at the same time you need to still register the other tags. But not to worry, it's nothing that can't be solved with enough popping and shifting.
Here is the entire fix, from the beginning of the parse function:

exports.parse = function(args, defaults, replacements) {
   var options = {};
   if (typeof defaults === "object" && !(defaults instanceof Array)) {
       options = defaults
   }
 
   if (typeof replacements === "object" && !(defaults instanceof Array)) {
        for (var i in args) {
             var arg = args[i];
             if (arg.charAt(0) === "-" && arg.charAt(1) != "-") {
                  arg = arg.substr(1);
                  if (arg.indexOf("=") !== -1) {
                      arg = arg.split("=");
                      var keys = arg.shift();
                      var value = arg.join("=");
 
                      arg = keys.split("");
                      var key = arg.pop();
                      if (replacements.hasOwnProperty(key)) {
                           key = replacements[key];
                      }
 
                      args.push("--" + key + "=" + value);
                  } else {
                      arg = arg.split("");
                  }
 
                  arg.forEach(function(key){
                      if (replacements.hasOwnProperty(key)) {
                          key = replacements[key];
                      }
                      args.push("--" + key);
                  });
             }
        }
   }

It's a lot of code (in comparison) but all we are really doing is splitting the argument by an equals sign, then splitting that key into the individual letters. So for example if we passed -gj=asd we would split the asd into a variable called value, and then we would split the gj section into individual characters. The last character (j in our example) will become the key for the value (asd) whereas any other letters before it, will just be added as regular boolean tags. I didn't want to just process these tags now, just in case we changed the implementation later. So what we are doing is just converting these short hand tags into the long formed version and then letting our script handle it later.
Running Mocha again will take us back to our illustrious green results of four tests passing for this module.
Now there are a few more things we can add to this tags module to make it closer to the npm package, like the ability to also store plain text arguments for things like commands or the ability to collect all the text at the end, for a query property. But this article is already getting long and I would like to move on to implementing the search functionality.

The Search Module

We just went through creating a module step by step following a TDD approach and I hope you got the idea and feeling of how to write like this. But for the sake of keeping this article moving, for the rest of the article, I will speed up the testing process by grouping things together and just showing you the final versions of tests. It's more of a guide to different situations which may come up and how to write tests for them.
So just create a file named search.js inside the lib folder and a searchSpec.js file inside of the test folder.
Next open the spec file and let's setup our first test which can be for the function to get a list of files based on a depth parameter, this is also a great example for tests which require a bit of external setup for them to work. When dealing with outside object-like-data or in our case files, you will want to have a predefined setup which you know will work with your tests, but you also don't want to add fake info to your system.
There are basically two options to solve this problem, you can either mock the data, like I mentioned above if you are dealing with the languages own commands for loading data, you don't necessarily need to test them. In cases like that, you can simply provide the 'retrieved' data and continue on with your testing, kind of like what we did with the command string in the tags library. But in this case, we are testing the recursive functionality we are adding to the languages file reading capabilities, depending on the specified depth. In cases like these, you do need to write a test and so we need to create some demo files to test the file reading. The alternative is to maybe stub the fs functions to just run but not do anything, and then we can count how many times our fake function ran or something like that (check out spies) but for our example, I am just going to create some files.
Mocha provides functions which can run both before and after your tests, so you can perform these kinds of external setup and cleanup around your tests.
For our example, we will create a couple of test files and folders at two different depths so we can test out that functionality:

var expect = require("chai").expect;
var search = require("../lib/search.js");
var fs = require("fs");
 
describe("Search", function(){
   describe("#scan()", function(){
       before(function() {
           if (!fs.existsSync(".test_files")) {
               fs.mkdirSync(".test_files");
               fs.writeFileSync(".test_files/a", "");
               fs.writeFileSync(".test_files/b", "");
               fs.mkdirSync(".test_files/dir");
               fs.writeFileSync(".test_files/dir/c", "");
               fs.mkdirSync(".test_files/dir2");
               fs.writeFileSync(".test_files/dir2/d", "");
           }
       });
 
       after(function() {
           fs.unlinkSync(".test_files/dir/c");
           fs.rmdirSync(".test_files/dir");
           fs.unlinkSync(".test_files/dir2/d");
           fs.rmdirSync(".test_files/dir2");
           fs.unlinkSync(".test_files/a");
           fs.unlinkSync(".test_files/b");
           fs.rmdirSync(".test_files");
       });
   });
});

These will be called based on the describe block they are in, and you can even run code before and after each it block using beforeEach or afterEach instead. The functions themselves just use standard node commands to create and remove the files respectively. Next we need to write the actual test. This should go right next to the after function, still inside the describe block:

it("should retrieve the files from a directory", function(done) {
    search.scan(".test_files", 0, function(err, flist){
        expect(flist).to.deep.equal([
            ".test_files/a",
            ".test_files/b",
            ".test_files/dir/c",
            ".test_files/dir2/d"
        ]);
        done();
    });
});

This is our first example of testing an async function, but as you can see it's just as simple as before; all we need to do is use the done function Mocha provides in the it declarations to tell it when we are finished with this test.
Mocha will automatically detect if you specified the done variable in the callback and it will wait for it to be called allowing you to test asynchronous code really easily. Also, it's worth mentioning that this pattern is available throughout Mocha, you can for example, use this in the before or after functions if you needed to setup something asynchronously.
Next I would like to write a test that makes sure the depth parameter works if set:

it("should stop at a specified depth", function(done) {
    search.scan(".test_files", 1, function(err, flist) {
        expect(flist).to.deep.equal([
            ".test_files/a",
            ".test_files/b",
        ]);
        done();
    });
});

Nothing different here, just another plain test. Running this in Mocha you will get an error that the search doesn't have any methods, basically because we haven't written anything in it. So let's go add an outline with the function:

var fs = require("fs");
 
exports = module.exports = {};
 
exports.scan = function(dir, depth, done) {
 
}

If you now run Mocha again, it will pause waiting for this async function to return, but since we haven't called the callback at all, the test will just timeout. By default it should time out after about two seconds, but you can adjust this using this.timeout(milliseconds) inside of a describe or it block, to adjust their timeouts respectively.
This scan function is supposed to take a path and depth, and return a list of all the files it finds. This is actually kind of tricky when you start thinking about how we are essentially recursing two different functions together in a single function. We need to recurse through the different folders and then those folders need to scan themselves and decide on going further.
Doing this synchronously is fine because you can kind of step through it one by one, slowly completing one level or path at a time. When dealing with an async version it get's a bit more complicated because you can't just do a foreach loop or something, because it won't pause in between folders, they will all essentially run at the same time each returning different values and they would sort of overwrite each other.
So to make it work, you need to create a sort of stack where you can asynchronously process one at a time (or all at once if you use a queue instead) and then keep some order in that manner. It's a very specific algorithm so I just keep a snippet by Christopher Jeffrey which you can find on Stack Overflow. It doesn't apply just to loading files, but I have used this in a number of applications, basically anything where you need to process an array of objects one at a time using async functions.
We need to alter it a bit, because we would like to have a depth option, how the depth option works is you set how many levels of folders you want to check, or zero to recurs indefinitely.
Here is the completed function using the snippet:

exports.scan = function(dir, depth, done) {
   depth--;
   var results = [];
   fs.readdir(dir, function(err, list) {
       if (err) return done(err);
       var i = 0;
       (function next() {
           var file = list[i++];
           if (!file) return done(null, results);
           file = dir + '/' + file;
           fs.stat(file, function(err, stat) {
               if (stat && stat.isDirectory()) {
                   if (depth !== 0) {
                       var ndepth = (depth > 1) ? depth-1 : 1;
                       exports.scan(file, ndepth, function(err, res) {
                           results = results.concat(res);
                           next();
                       });
                   } else {
                       next();
                   }
               } else {
                   results.push(file);
                   next();
               }
           });
       })();
   });
};

Mocha should now be passing both tests. The last function we need to implement is the one which will accept an array of paths and a search keyword and return all matches. Here is the test for it:

describe("#match()", function(){
    it("should find and return matches based on a query", function(){
        var files = ["hello.txt", "world.js", "another.js"];
        var results = search.match(".js", files);
        expect(results).to.deep.equal(["world.js", "another.js"]);
 
        results = search.match("hello", files);
        expect(results).to.deep.equal(["hello.txt"]);
    });
});

And last but not least, let's add the function to search.js:

exports.match = function(query, files){
  var matches = [];
  files.forEach(function(name) {
      if (name.indexOf(query) !== -1) {
          matches.push(name);
      }
  });
  return matches;
}

Just to make sure, run Mocha again, you should have a total of seven tests all passing.

Putting It All Together

The last step is to really write the glue code which pulls all our modules together; so in the root of our project add a file named app.js or something like that and add the following inside:

# !/usr/bin/env node
 
var tags = require("./lib/tags.js");
var search = require("./lib/search.js");
 
 
var defaults = {
   path: ".",
   query: "",
   depth: 2
}
var replacements = {
   p: "path",
   q: "query",
   d: "depth",
   h: "help"
}
 
tags = tags.parse(process.argv, defaults, replacements);
 
if (tags.help) {
   console.log("Usage: ./app.js -q=query [-d=depth] [-p=path]");
} else {
   search.scan(tags.path, tags.depth, function(err, files) {
       search.match(tags.query, files).forEach(function(file){
           console.log(file);
       });
   });
}

No actual logic going on here really, we are just basically connecting the different modules together to get the desired results. I usually don't test this code as it's just glue code which has all been tested already.
You can now make your script executable (chmod +x app.js on a Unix system) and then run it like so:

1	`./app.js -q=".js"`

Optionally customizing some of the other placeholders we setup.

Conclusion

In this article we have built an entire file searching app, albeit a simple one, but I think it demonstrates the process as a whole fairly well.
Some personal advice moving forward; if you are going to do a lot of TDD, setup your environment. A lot of the overhead time people associate with TDD is due to them having to keep switching windows around, opening and closing different files, then running tests and repeating this 80 dozen times a day. In such a case it interrupts your workflow decreasing productivity. But if you have your editor setup, like you either have the tests and code side-by-side or your IDE supports jumping back and forth, this saves a ton of time. You can also get your tests to automatically run by calling it with the -w tag to watch the files for changes and auto run all tests. These kinds of things make the process more seamless and more of an aid then a bother.

Security and Virtualization

Pages

Saturday, 20 December 2014

Testing in Node.js