JavaScript Unit Testing in Visual Studio Team Services(VSO)

JavaScript unit testing can be a real struggle. Especially when you want to integrate unit testing into your continues delivery and/or continues deployment. This blog post is about enabling your JavaScript unit tests to run on Visual Studio Team Services.

Unit testing your web application is hard, you probably got to deal with one or multiple data connections, server side code such as ASP.net(C#) and front-end JavaScript code which is fully integrated into your HTML templates. Regardless of the code that has to be unit tested you got to mind several design principles, namely:

1. Separation of Concerns
2. Dependency injection
3. Build integration

This blog post isn't about the first and second principle but all about the 3rd one, namely Build Integration. This post therefore presumes that you have a working set of JavaScript unit test using a framework such as: QUnit, Jasmine and Mocha.

So why Build integration? Well having your unit tests integrated in your automated build flow is a must have for every web developer. The time that we manually build and unit test our code is over. If you are still doing that, then that's a big sign for you to critically look at your workflow. You are literally living in the past!

Most of you developers know how to unit test your .NET code. Microsoft got great tools in Visual Studio and TFS/VSTS(VSO) to automatically run your unit tests and flag the build as a failure if one of your tests fail. This will result in finding issues earlier in the development process and thus probably better quality and faster development.

So why are we not unit testing our JavaScript, and if we do why don't we integrate it in TFS/VSTS? Well because its a little bit more complicated than unit testing .NET code. JavaScript code obviously runs in a browser, TFS/VSTS should spin up a browser and run your unit tests and then check if they succeeded or failed. Well that's the exact problem, TFS/VSTS doesn't support that functionality and that leaves us hanging to automate our JavaScript unit tests.

But wait! what about all those server side JavaScript instances such as NodeJS, can't they tackle this problem for us? Well actually they can! NodeJS provides great functionality to run your JavaScript unit tests on the NodeJS runtime but the only problem is, is that NodeJS doesn't got a DOM. In fact it doesn't even support document.getElementById, which will cause a lot of unit tests to fail. Well luckily for us there's PhantomJS. PhantomJS is a headless in memory browser with a DOM. It actually renders the DOM but doesn't paint it. So that tackles the problem.

But how do we get PhantomJS to work in TFS/VSTS? Well you don't have to do anything. Someone at Microsoft fixed that for us and named it Chutzpah. Chutzpah is both a Visual Studio extension aswell as a NuGet package. It contains PhantomJS as a dependency and comes with 3 JavaScript unit test libraries, namely: Jasmine, QUnit and Mocha. Its able to run and debug your JavaScript unit tests via the Visual Studio and TFS/VSTS interface, that means that you get both your .NET as JavaScript test results in one unified standard. Which is great for build purposes.

So how does it work? Well just search for Chutzpah in the Visual Studio Extensions and you will find two extensions. The first is the Test adapter, its used to run tests in the Visual Studio Test Explorer. Once you've installed the Test adapter, you will see that your JavaScript unit tests will be picked up in the explorer. Just select your tests and spin it up, you will see that your tests will integrate perfectly within Visual Studio.

The second extension is a context menu extension that allows you to run your tests within two mouse clicks. This plugin also provides you additional functionality such as code coverage results, using blanket.js. It produces really nice code coverage files that exactly pinpoint the code that hasn't been hit.

It will be easy to get your unit tests to run on TFS/VSTS, once your unit tests run within Visual Studio. First crab the NuGet package. The NuGet package, as stated earlier, has the necessary files to run your unit tests, this also applies to TFS/VSTS. Secondly add a Visual Studio test build step to your build flow.

Once you've done that make sure that you adjust to test assembly to only match JavaScript test files. Normally I suffix these files with *.tests.js. This makes them easy to find and apply in the build step. Note that we have to exclude all files that are found in the obj folder. Otherwise you will have double tests. The screenshot contains the following line: **\*.tests.js;-:**\obj\**\*.tests.js

Lastly make sure that you point to the Chutzpah test adapter. This actually points to the NuGet package that you've installed for your project. Make sure that the $(Build.SourcesDirectory) has the packages folder or you might up ending with strange errors. 

Well and that's it. Hopefully you've had enough information. If you have any questions or problems feel free to contact me.

NuGet and JavaScript libraries are a bad marriage

Almost everyone who is using Visual Studio uses NuGet today. Its a powerful package manager for .NET based applications. It clearly does it job for maintaining and distribution of libraries. Its also widely used in Enterprise environments where companies setup their own NuGet servers for better application live cycle management of their code.

Well nothing but love... until you try to use NuGet for JavaScript libraries. JavaScript libraries are referenced differently than .NET libraries in Visual Studio projects. .NET libraries are always referenced in the project file, this results in a single list of referenced libraries and the NuGet package manager can always keep track of the downloaded libraries. JavaScript libraries however get dumped in a Scripts folder. Well I don't know about you guys but I easily use 10-20+ JavaScript libraries in a single Web application. So here is the pain in the ass. I'm frustrated about the fact that I can't structure my JavaScript in a more folder like structure. If I move the JavaScript libraries in a different folder than NuGet loses its track of the JavaScript libraries and as a result version upgrades or removal of libraries result in additional or unintentional left files. This problem gets even bigger when you use Typescript. With TypeScript the amount of files for a JavaScript library are almost alwayst doubled since you have to get the definition files. A lot of the most common libraries also contain a bunch of CSS. The exact same problem also applies on CSS, in fact it applies of every non .NET library.

Well thats enough hating, its time for some advise. I think NuGet should introduce a mapping scheme for non .NET libraries. You should be able to place the content of a NuGet package wherever you like in your project. NuGet then should remember those locations and keep track of those files. NuGet should also update their saved locations when a user moves those files. This will be huge for the usability of non .NET libraries in NuGet.

Thank you for reading and please share your thoughts.

Why MCSD certification is important

Recently I have started getting my MCSD Web Applications certification. As a .NET consultant this is quite important from a market value perspective however in my opinion it’s even more important from a personal development perspective.

Let me elaborate, as your company is trying to sell your expertise to a third party it’s important for the third party that they get an employee with the appropriate skills, in my case .NET framework skills. There is always a certain chance that the consultant exaggerates his skills and that he claims to be an expert on certain aspects while he only used the technology for a short amount of time or that he used it in a limited way. For instance, the .NET framework is a really big framework and stating to be expert in it might mean something else for you than for the hiring company. It’s a matter of perception. This resolves in a risk for the hiring company. This risk can be reduced by getting the right certifications. Certifications can’t be misinterpreted since the content of the examinations is fixed. So when the consultant has for instance a 70-487: Developing ASP.net MVC Web Applications certificate you can be sure that he has the appropriate knowledge to build a solid MVC Web application. This certainty reduces the risk of hiring an incompetent employees, which increases the market value of the consultant since hiring an employee with certifications has a higher change of success.

Secondly the personal development perspective. As a, not necessarily Microsoft, software developer you have to keep learning and keep expanding your knowledge. Software languages develop, frameworks expand, new libraries come and go, you simple can't learn one language or one framework and do your thing for 40 years. You need to keep track of new trends and developments. MCSD Certification helps to keep your Microsoft knowledge up to date. Since certifications expire you will have to keep track of new trends and developments or catch up when you retake your examination. The certification functions as a big stick to keep your knowledge up to date. Some developers think that just working experience is enough. I tend to think that that's not true. Working experience is often limited to the technology that applies to the application you are developing. Its pretty hard to keep your MVC knowledge up to date if you are for instance developing a Web Forms application for your work. You will have to catch up in your spare time and MVC is only a small part of the MCSD Web Applications stack. MCSD certifications basically force you to keep track of developments, some guys need it some guys don't. I like it.

Async Cordova Windows plugin

Recently I had to write a Cordova plugin for Windows Phone 8.1 and the Windows RT and desktop environment. I went to the Cordova plugin documentation and was disappointed by the lack of information. Hopefully this blog post will help you out.

So I assume that you know a little bit about a Cordova plugin structure. Please take a detailed look at existing plugins when you don't. One important part of a plugin are its platforms. Note that Windows and WP8 are two different platforms. Windows will apply on Windows Phone 8.1 and Windows RT/Metro while the WP8 platform will only apply on Windows Phone 8. This is because the Windows platform generates an Universal App which can be applied on almost every Microsoft Windows runtime.

Every plugin has a plugin.xml which contains information about the files of the plugin. Note that the example below is a stripped version of a plugin.xml file. Every plugin.xml should contain 1 or more platform elements with the supported platforms. Make sure that you create a separate xml element for every platform (ios, android, wp8, windows, etc).

<plugin id="com.foo.bar"
        version="1.0.0"
        xmlns:android="http://schemas.android.com/apk/res/android"
        xmlns="http://apache.org/cordova/ns/plugins/1.0">
    <name>FooBar</name>
    <platform name="windows">  
        <framework custom="true" src="src/windows/FooBar.dll"></framework>

        <js-module name="WebSqlProxy" src="src/windows/FooBarProxy.js">
            <runs />
        </js-module>
    </platform>
</plugin>

You can specify additional files inside the platform element. These files will be loaded into your application or executed in the Cordova.js context. The framework elements are files that will be added to your application and usually contain native libraries, for Windows usually DLL or WINMD files. WINMD files are commonly used for Windows Phone projects.

Next to the native library files there are JavaScript files inside the js-module elements. There are two kind of JavaScript files that you want to add, namely the proxy JavaScript files and functional JavaScript files. The proxy JavaScript files are used to build a bridge between your native library calls and the JavaScript calls. The functional JavaScript files on the other hand are commonly used to initialize the JavaScript or for example extend the window object with a certain namespace.

Inside element inside the js-module element states the mode of the execution of the JavaScript file. There are several options available but the two major ones are <runs \>, which just executes the JavaScript file and <merges target="x"> which gives you the power to merge a JavaScript object with a certain JavaScript object, such as window.

So there's the plugin.xml. So lets get back to those native libaries. I told you that the Windows platform generates a Universal App. The native library of your plugin should also be a library for Universal Apps. You can create one using Visual Studio > New Project > Templates > Store Apps > Universal Apps -> Windows Runtime Component project. Please choose your name carefully because your namspace is very important. I will use Nimrod.FooBar as namespace in my example.

using System;

namespace Nimrod
{
    // This class must be sealed because Cordova only recognizes sealed classes.
    public sealed class FooBar
    {
        public static string Echo(string value)
        {
            return value;
        }
    }
}

Compile the project and look for the WINMD file. This file should be added to the src\windows plugin directory. Also add it to the plugin.xml as a framework file. After that you can write a JavaScript proxy. It will look like this:

module.exports = {
    echo: function(success, fail, args) {
        var value = args.shift();
        var res = Nimrod.FooBar.echo(value);
        if (res != undefined) {
            success(res);
        } else {
            fail();
        }
    };
};
require("cordova/exec/proxy").add("Nimrod", module.exports);

The module.exports object is extended with an echo function. This function has a call to the native library; Nimrod.FooBar.echo(value). Please note that this syntax is confirm the Namespace.Class.Method syntax used in the native library. The Cordova proxy will map this JavaScript call to the native library.

So how do you use the plugin in your code? Well note that the module.exports is added with Nimrod as key. This will result in a Nimrod object on the window object. So use can use the plugin like this:

// This is a Cordova event that states that all plugins are loaded.
document.addEventListener("deviceready", function () {
    // window.Nimrod doesn't exist in iOS or Android.
    if(window.Nimrod) {
        window.Nimrod.echo(
            function(res) {
                console.log(res);
            }, function () {
                console.log("Something went wrong");
            }, 
            "Hello world"
        );
    }
});

So this is great isn't it? Well not exactly. What if my native library does a time consuming action, such as a database call or a mathematical calculation? My App will be blocked for the entire call. This is because the plugin isn't setup asynchronously. So lets fix that.

The .NET framework has excellent support for asynchronous programming. Especially .NET 4.5 or higher. Please take a look at this example:

using System;
using System.Threading.Tasks;
using Windows.Foundation;

namespace Nimrod
{
    // This class must be sealed because Cordova only recognizes sealed classes.
    public sealed class FooBar
    {
        public static IAsyncOperation Echo(string value)
        {
            return FooBar.doSomeTimeConsumingThings(value).AsAsyncOperation();
        }

        private static async Task doSomeTimeConsumingThings(string value)
        {
            string result = null;
            
            await Task.Run(() =>
            {
                Task.Delay(1000);

                result = value;
            });

            return result;
        }
    }
}

The echo method now returns an IAsyncOperation instead of the string. This AsyncOperation has as effect on the return value in the JavaScript proxy. Instead of being the string value it now will be an JavaScript promise. This will not be blocking the JavaScript. Take a look at the updated proxy code:

module.exports = {
    echo: function(success, fail, args) {
        var value = args.shift();
        var promise = Nimrod.FooBar.echo(value);
        promise.done(function (res) {
            success(res);
        });
        promise.fail(function () {
            fail();
        });
    };
};
require("cordova/exec/proxy").add("Nimrod", module.exports);

I hope this helped, if you have any questions feel free to ask. I might be able to help you out any further.

Combo breaker reloaded

In March 2014 I wrote a blog post about some issues I ran into while developing an HTML5 App using Cordova and JQuery Mobile for Windows 8. Well it has been 10 months and I would like to share some additional solutions to those problems. Especially the "dynamic content" and "usage of unsafe HTML" issues.

As you probably know, jQuery Mobile uses AJAX navigation to switch between pages. This creates a richer user experience but also causes some issues on the Windows platform(both metro and phone). Your page might contain some unsafe HTML which will cause the following error:

Unhandled exception at line 5472, column 5 in ms-appx://<app identifier>/www/js/jquery/jquery-2.0.2.js

0x800c001c - JavaScript runtime error: Unable to add dynamic content. A script attempted to inject dynamic content, or elements previously modified dynamically, that might be unsafe. For example, using the innerHTML property to add script or malformed HTML will generate this exception. Use the toStaticHTML method to filter dynamic content, or explicitly create elements and attributes with a method such as createElement.  For more information, see http://go.microsoft.com/fwlink/?LinkID=247104.

If there is a handler for this exception, the program may be safely continued.

In the previous blog post I created a hack in jQuery with a MSApp.execUnsafeLocalfunction wrapper which allows adding dynamic content. Well this hack was highly volatile and probably incomplete. Luckily for you and me we have some smart guys at MSOpenTech which solved this problem for us. They created a "JavaScript Dynamic Content shim for Windows Store apps".

The shim basically wraps all unsafe properties and functions in a MSApp.execUnsafeLocalfunction call. This will prevent WinJS from throwing exceptions when you dynamically add unsafe HTML. Please take a look at their Github page for more information.

Hope this helps!