Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,128 | Comments: 45,550

filter by tags archive

Unit testing frameworks extensibility

time to read 2 min | 208 words

I just run into a road block, and I am posting before lunch, hopefully I'll get some ideas about how to solve this.

I have a directory full of source files, which contains tests. What I want to do is to write something like this:

public class MyTest
	public void Validators()

The typical scenario for each source file is something like:

public void ExecuteSingleTest(string path)
      var test = CompileAndGetInstance(path);
ExecuteTestInUnitTestingFramework(test); }

A source file can contain more than a single test, and I want all those individual tests to be reported as part of the overall test run.

The problem is that I took a look at the extensibility mechanisms for both NUnit and MbUnit, and they are so big. It is not that I can't see how I would solve the issue, but it would require a lot of work to do so.

I know how I can do this with xUnit, where it would be a truly trivial affair.

Am I missing something? I never really had to deal with the extensibility mechanisms in NUnit or MbUnit before.


Rodrigo B. de Oliveira

I think code generation for that particular scenario works best specially if you are interested in tool integration (think running a particular test case from VS using ReSharper).

The idea is to have a simple boo script somewhere that's executed as part of the build process to generate a partial class definition for the fixture with all the test methods.The meat of test execution can go to another partial definition or to a base type.

OR, code the test assembly in boo and write a ExecuteAllTestsIn attribute. ;)

Neil Mosafi

I have actually worked on this a week ago for orchestrating JSUnit tests withiin NUnit

I used a custom NUnit extension for this (http://www.taumuon.co.uk/rakija/index.html) which can dynamically generate fixtures, and then run parameterised tests from the fixtures. I wrote some code to scan an HTML file for all tests and then create a fixture for each one.

If you want more info let me know.

Neil Mosafi

Actually an alternative one is IterativeTest, if you just want one fixture with all your tests in it...


Oh, also I have no idea about MbUnit, and this doesn't work in Resharper (you need to use the NUnit GUI or Console to run the tests) as I don't think it supports NUnit extensions

Ben Hall

Time for everyone to post their how to extend unit testing framework articles :)

Here is mine:


Look at the "Test Case Builders" section as this can build you a series of tests (one for each directory) using NUnit. But you are correct, NUnit is very heavy when it comes to extensions.

XUnit on the other hand is much easier!

Take a look at my post:


Use a PropertyData attribute.



Then have that a property which will yield you all the directories to run the unit test in. You would add this attribute to ExecuteSingleTest

public static IEnumerable<object[]> MyTestData




   yield return new object[] { "hello", 5 }; 

   yield return new object[] { "hello world", 11 }; 

   yield return new object[] { "failing", 0 }; 



That's just out the box with xUnit - very quick and easy for scenarios like this I think :)

MbUnit v2 sits somewhere in the middle, you don't have to compile your attribute into a separate assembly but there is still a lot of lifting to do.

Hope this helps.


Neil Mosafi

That seems to be the way most frameworks are going - an attribute which points to a property which returns the parameters to the tests.

Nice to see that it's OOTB with XUnit but I would add that XUnit is the newest framework so is probably going to try and include these sorts of things in there. The IterativeTest extension comes built into the later versions of NUnit and enables the same thing.

Good luck Ayende

Jeff Brown

This isn't too hard in MbUnit v3. You define a new data source attribute as a subclass of DataAttribute then implement its PopulateDataSource method.

The next major release (MbUnit v3 Alpha 4) will have a "Factory" mechanism for defining simple data sources as IEnumerables within your test fixture.

Simple things should be simple...

Comment preview

Comments have been closed on this topic.


  1. The worker pattern - about one day from now

There are posts all the way to May 30, 2016


  1. The design of RavenDB 4.0 (14):
    26 May 2016 - The client side
  2. RavenDB 3.5 whirl wind tour (14):
    25 May 2016 - Got anything to declare, ya smuggler?
  3. Tasks for the new comer (2):
    15 Apr 2016 - Quartz.NET with RavenDB
  4. Code through the looking glass (5):
    18 Mar 2016 - And a linear search to rule them
  5. Find the bug (8):
    29 Feb 2016 - When you can't rely on your own identity
View all series


Main feed Feed Stats
Comments feed   Comments Feed Stats