Programming (back-end), Programming (front-end)

Czy można ignorować testy?

Wiele razy słyszałem, że jak ma się ignorować testy to już lepiej je usunąć. Dlaczego? Bo bardzo łatwo podjąć decyzję o zignorowaniu testu (dodaniu jednego atrybutu czy tam literki “x” w przypadku JavaScript) a trudniej podjąć decyzję o od-ignorowanie testu. Tutaj znajdzie się już milion powodów, żeby tego nie robić: a bo nie ma czasu, a bo to trzeba zmienić coś innego co już działa, a bo ten test to od początku pisała jakaś ciamajda i trzeba go przepisać i to potrwa.
A więc czemu usunąć? Bo i tak nie są uruchamiane i starzeją się z każdą chwilą. To jest “Dead code” który przy okazji maskuje inny “dead code” (normalnie na klasie dostalibyśmy “0 references”, ale przecież jest używana w tym martwym teście).

Continue reading

Programming (back-end)

WebBrowser.InvokeScript case study – running custom JavaScript

Last time I played with WebBrowser control (in WindowsPhone) as I wanted to automate navigation in some external web application. The first thought was to use InvokeScript method as it allows to run custom script. But the thing was not so obvious, that’s why I’d like to share my study here. Let’s start with a simple code snippet (WebBrowser is an instance of WebBrowser control):


This code will throw System.SystemException: An unknows error has occurred. Error: 80020006. It appears that InvokeScript method can execute only functions that already exist. Weird, especially if you played with ExecuteScript in Selenium. So what, am I not able to execute my own code? Hopefully there is Javascript eval function:


OK, now it’s better. Alert appeared. Let’s try to do something more than a single expression:


Whew! Another success. What about returning values?  Failed! It returned empty string. Why not 2? Why empty string and not null? InvokeScript return type is object. Maybe I missed “return” keyword. Let’s check:


Now there is: System.SystemException: An unknows error has occurred. Error: 80020101. After reading the Internet, I’ve found enigmatic: jQuery selector not working on Windows Phone 7. It seems that InvokeScript has problems with returning values from eval function. The solution here is to cast to string on JavaScript’s side


Unfortunately it’s more complicated when we have more JavaScript to run as it should always cast to string:


My approach to solve it was to use self executing function encapsulating the custom script. For better use of it let’f extract a method:


Finally we have to remember about adding return keyword but that’s the price worth to pay.

Programming (back-end)

Debugging NSpec with VisualStudio/Resharper – simple trick

NSpec is purely awesome, although coming with little painful disadvantage – debugging. The default approach (DebuggerShim.cs) is weak because forces me to always navigate to a separate file and change some code. Authors of this great library claim in the documentation that they already work on “fully integrated support”, but it’s not there yet. Fortunately you can apply one simple trick to make it work far better.

1. Get Nspec: install-package nspec

2. Get NUnit (in version referenced by NSpec): install-package nunit -Version

3. Go to DebuggerShim.cs file and change it to look like that:

nspec - DebuggerShim.cs

And that’s it! You can go back to your specs and be delighted by the “unit test” icon beside your spec class.

NSpec - test result

…and test runner:

NSpec - Run spec

What has happened?

DebuggerShim was a tool (nunit TestFixture) that run some given specification. By renaming it to “nspec” class it’s become a wrapper for all specifications. For NSpec it doesn’t matter because it doesn’t care about NUnit attributes nor abstract classes. If it’s in the same namespace as other specifications, then we do not have to change anything and Resharper starts to detect it as NUnit test fixture (as it derives from our custom “nspec” test fixture).


  • Simple – no need to install any plugin/adapter
  • Unobtrusive – no need to change existing specifications
  • Easy to update – if NSpec gets fully integrated debugging support, then just delete custom abstract nspec wrapper
  • Additionaly as a test output we see specification generated by NSpec runner


  • It runs the whole specification, it’s not possible to debug single context. This is particularly painful in case of big (and slow) UI testing scenarios. In that case I try to split them into several more specific scenarios.

You can download the test sample here:

Programming (back-end)

Improve your testing experience with NSpec, best BDD test framework

Amongst long list of benefits that automated testing gives our projects we have ‘documentation’ feature. I think it takes even one of the first places. Many times I was looking at unit tests trying to decipher what they did and why they were there. Ask yourself a question if your unit test project is a mixture of test methods or it shapes a specification that you can share with your client. Saying ‘specification’ I meant something like that:


This is a test result generated by NSpec for a sample project. NSpec is a beautiful test framework that I’ve focused on for the last several months and probably will stay for a longer time.

If you really want to change your testing experience it’s more often about the approach not a framework. I hate when people see a goal in moving from NUnit to xUnit because it’s almost only changing attributes used for testing. NSpec also allows you to omit class/method attributes but the key concept is to create scenarios that are cool to maintain. For me the testing experience is close to Jasmine (JavaScript) but here we can use lambdas instead of long “function(){}” syntax. The image is worth a thousand words so:


It generates the following result:


Why better than: MSTest/NUnit/xUnit

These are just TDD frameworks with flat structure. No way to define scenarios. NSpec enables you to build real scenarios. It also simplifies the way we organise tests (far easier to add new test, find the proper place). Contexts decrease the size of Arrange/Given part.

Why better than MSpec:

Both are BDD frameworks, but NSpec has more user-friendly syntaxt. It’s possible to avoid hard string concatenation (like ShouldDoSomething, or should_do_something_else). Test result for complex scenarios looks better, you can have more levels. MSpec also takes advantage of lambdas. If you don’t like it, with NSpec you can write tests without them (using conventions).

Why better than SpecFlow:

I don’t believe that the client can develop SpecFlow*.feature files. Because of that I don’t like the workload involved with maintaining *.feature files and their mappings. I’m a developer and what I really want to do is to write code. Although what’s good in SpecFlow is creating specification-by-example.

NSpec disadvantages:

* debugging – see this official tutorial or install additional test adapter
* no resharper support – still in progress
* lot of lambdas (non-mandatory) – for some people it’s an advantage but maybe not for all

Regarding debugging with resharper I have my own solution.Check this out here

Keep in touch.

Programming (front-end)

JavaScript/TypeScript unit testing –11 sample projects for Resharper, Chutzpah and Karma


An example is worth a thousand words. But seriously it’s a lot easier to follow some given scenario than searching for tutorials for some tools and then trying to apply them to a project that is not supported in the end. Moreover unit testing the client side of the project is more complicated than the back-end part. Maybe it’s because we’ve got used to creating test projects based on nunit/xunit/mstest/mspec etc. In Javascript world it’s a little bit more tangled: we have Javascript but also move forward to TypeScript. We have plain code without modules or use requirejs to benefit from AMD. For each configuration the setup of unit test project is different. I think having all these samples in one place could help us to start TDD in client-side.
All the sample are based on the sample application from my previous posts about unit testing JavaScript. Actually these are not 11 different sample projects, but 11 versions of the same project so that you can easily catch the difference between the configurations.

Samples (links)


JavaScript JavaScript TypeScript TypeScript
Resharper link link* link** (none)**
Chutzpah link link link link
Karma link link link link

** Resharper 8.2 should support support TypeScript testing [RSRP-394249]

* AMD support in this sample is artificial, code must be based on named modules that are loaded staticaly.

Tools review




Resharper lets you run unit test for JavaScript. Actually it could be enough to just start using it as a test runner. It’s awesome to right-click on solution, choose “run unit tests” and watch the solution passing all the tests: back-end and front-end in one go. Moreover it’s also super-easy to set up. Unfortunately the practice is more cruel – it’s ok for simple Javascript project without AMD.  TypeScript support is not yet in live and AMD support is not yet even planned. And there is no command line runner to let the tests run in buildserver. Otherwise it would be the perfect choice.



As an alternative we have Chutzpah. To integrate it with Visual Studio  you can install “Chutzpah Test Runner Context menu Extension” or “Chutzpah Test Explorer Adapter” (default one for Visual Studio). You can also download command line tool that will run the tests in console or in buildserver (it even generates code coverage report). It has builtin support for requirejs and TypeScript. Unfortunately tests discovery is a little bit too slow to run it spontaneously after each change.



This runner is a must-have in AngularJS world and is a JavaScript alternative to NCruch (runs tests continuously after saving changes). It’s a little bit tricky to configure but after that it is super cool to play with. It doesn’t have TypeScript support out of the box, but because TypeScript is compiled to javascript it deals with it successfuly. It’s possible to configure karma to use Chrome (or any other favourite browser) to be the test runner and it makes it very fast and natural to debug. It’s also possible to set the “run once” mode to run it on buildserver using phantomJS. The dark side of the tool is that it’s not integrated with VisualStudio and you have to stuck with console window (actually there is one plugin that runs karma in VS output window but it’s still a hidden console and not Test explorer adapter).

Happy testing folks!

BTW: if you like the samples, remember to leave a github star 😉

Programming (front-end)

Unit testing JavaScript with VisualStudio and Resharper – running on buildserver + code coverage using Chutzpah


Unit tests are priceless but only when run automatically. Otherwise they can become forgotten and stay red (or ignored) for the rest of their lifetime. In the last two posts about unit testing in Javascript [1, 2] we have prepared our solution. Now it’s a very good time to tell our buildserver to run them during the build process. I chose Chutzpah as the test runner.


Chutzpah, pronounced [hutz-pah] from Yiddish/Hebrew, means “shameless audacity, utter nerve”. It’s a JavaScript tests runner that uses internally PhantomJS, Jasmine/QUnit to run the tests and BlanketJS to generate code coverage. The package contains both VisualStudio extension to run the tests (but I prefer Resharper) and console runner used by buildservers (especially TeamCity). Beside its core functionality chutzpah can also generate report with code coverage of JavaScript. Among all known approaches to run javascript test, in my opinion using Chutzpah is the simplest.

Getting started with Chutzpah

  1. Download Chutzpah and extract it to some well-known place (in my case C:\BuildTools\chutzpah)
  2. Add chutzpah.json configuration file to the test project folder. This step is optional for chutzpah, but in our case we use root path while referencing files in unit tests and this needs to be configured specially in the config file (there is also some additional information about code coverage pattern):3
  3. Run chutzpah console runner with parameters:
    /path – place where all the testes are
    /codecoverage – tell chutzpah to generate html report with code coverage


There is also an extra output file with code coverage information:


Preparing package for build server

Now we are able to run tests on build server – just run chutzpah.console.exe with specific parameters. the build should fail if any test fails. The other thing is the reporting to the build server about the test result. Fortunately Chutzpah has built-in support for TeamCity (/teamcity command line swith) but also serves XML output (/junit switch).

To have it on Teamcity I needed only to prepare MSBUILD file with the build procedure (I prefer to have everything in source control, especially build procedure, that’s why I don’t believe in TeamCity build steps):1

After adding this file to TeamCity we should see:


And after configuring report tab for “JS Code Coverage”:


Have fun!


Download and play with the demo solution:

Programming (front-end)

Unit testing JavaScript in VisualStudio with ReSharper – improvements

This is another approach to the previous subject: Unit testing JavaScript in VisualStudio with ReSharper. I kindly encourage you to get back to it before further reading.

Referencing production code in Test project

Last time I suggested “link-referencing” files with production code. Then during the build process files were copied to the test project. This approach was a bit cumbersome:

  • we had code duplication in our solution structure
  • sometimes it wasn’t obvious which file was edited – original file or the file copied to the test project
  • R# was complaining about missing references before the build process
  • You need to prepare test project file (.csproj) to make it work
  • You need to reproduce reference structure and keep it up-to-date while working with tests

Fortunately there is a cleaner approach to referencing production code. The solution is to reference files using root reference path:image

In this case we get the root of the test project, go to parent folder (..) and take the files directly from the project with production code (FrontEndTools). Now the only disadvantage is that each test file should know what’s the name of the folder of the tested project and change it when the folder name is changed – but this is very rare case, isn’t it?

If your Resharper marks the referenced file red, add reference to the tested project (FrontEndTools in my case).

From now on the solution structure looks much cleaner:


Switching to Jasmine

Previously I mentioned that I prefer QUnit with its TDD style, but after some time using both I changed my mind. Here are some of my thoughts:

  • Jasmine is more popular – I don’t have any statistics, this is just my feeling after browsing some open source projects and mixing with angularJS communities – I’ve just found Jasmine as a standard
  • Jasmine with its beforeEach/afterEach is more similar to NUnit SetUp/TearDown
  • Jasmine has own (extendable) matchers that simplifies doing even complex assertions
  • Built-in mocking framework is awesome – and this is actually killer feature – in QUnit you should build mocks manually or add external libraries (like SinonJS)
  • Both are supported by default (and only these two) by Resharper
  • Both have very good support and community background (with plugins and extensions)

Other demo solution improvements

  • jasmine.js referenced to test scenarios – Jasmine is by default supported by Resharper, but to get IntelliSense and get rid of Resharper annotations it’s good to reference proper Jasmine.js file explicitly
  • added tests for ColorBoxViewModel with a sample of Jasmine mocking


You can download the demo here: