Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,007 | Comments: 44,761

filter by tags archive

Opening seams for testing

time to read 14 min | 2722 words

While testing Rhino Service Bus, I run into several pretty annoying issues. The most consistent one is that the actual work done by the bus is done on another thread, so we have to have some synchronization mechanisms build into the bus just so we would be able to get consistent tests.

In some tests, this is not really needed, because I can utilize the existing synchronization primitives in the platform. Here is a good example of that:

   1: [Fact]
   2: public void when_start_load_balancer_that_has_secondary_will_start_sending_heartbeats_to_secondary()
   3: {
   4:     using (var loadBalancer = container.Resolve<MsmqLoadBalancer>())
   5:     {
   6:         loadBalancer.Start();
   8:         Message peek = testQueue2.Peek();
   9:         object[] msgs = container.Resolve<IMessageSerializer>().Deserialize(peek.BodyStream);
  11:         Assert.IsType<HeartBeat>(msgs[0]);
  12:         var beat = (HeartBeat)msgs[0];
  13:         Assert.Equal(loadBalancer.Endpoint.Uri, beat.From);
  14:     }
  15: }

Here, the synchronization is happening in line 8, Peek() will wait until a message arrive in the queue, so we don’t need to manage that ourselves.

This is not always possible, however, and this actually breaks down for more complex cases. For example, let us inspect this test:

   1: [Fact]
   2: public void Can_ReRoute_messages()
   3: {
   4:     using (var bus = container.Resolve<IStartableServiceBus>())
   5:     {
   6:         bus.Start();
   7:         var endpointRouter = container.Resolve<IEndpointRouter>();
   8:         var original = new Uri("msmq://foo/original");
  10:         var routedEndpoint = endpointRouter.GetRoutedEndpoint(original);
  11:         Assert.Equal(original, routedEndpoint.Uri);
  13:         var wait = new ManualResetEvent(false);
  14:         bus.ReroutedEndpoint += x => wait.Set();
  16:         var newEndPoint = new Uri("msmq://new/endpoint");
  17:         bus.Send(bus.Endpoint,
  18:                  new Reroute
  19:                  {
  20:                      OriginalEndPoint = original,
  21:                      NewEndPoint = newEndPoint
  22:                  });
  24:         wait.WaitOne();
  25:         routedEndpoint = endpointRouter.GetRoutedEndpoint(original);
  26:         Assert.Equal(newEndPoint, routedEndpoint.Uri);
  27:     }
  28: }

Notice that we are making explicit synchronization in the tests, line 14 and line 24. ReroutedEndpoint is an event that we added for the express purpose of allowing us to write this test.

I remember several years ago the big debates on whatever it is okay to change your code to make it more testable. I haven’t heard this issue raised in a while, I guess that the argument was decided.

As a side note, in order to get rerouting to work, we had to change the way that Rhino Service Bus viewed endpoints. That was a very invasive change, and we did it in less than two hours, but simply making the change and fixing the tests where they broke.


Matt Heaton

I've been working on some code for a socket library we need here that sits on top of the .NET Async sockets. Testing that's been interesting because allowing compression, SSL encryption/decryption and packing/unpacking of our protocol units on background threads is a goal. Not the real issue though.

I noticed that you put no timeout on your WaitOne call.

I found that sometimes in a multi-threaded environment I could break stuff well enough to never have the wait.Set() called. In that instance the timeout was what stops the suite from stalling forever and allows an error to be logged. Just a thought.

Jacob Stanley

Have you guys played with CHESS yet for unit testing concurrency situations? Microsoft demo'ed it at the PDC.

It only works with MSTest at the moment, or standalone exe's, but it's really quite amazing. It tests many likely interleavings of threads by taking control of the thread scheduler.


Omer Mor

I second Matt's suggestion for timeout-ing your waits.

In our multihreaded tests we use our own Assert.EventWasSet(), and Assert.EventWasNotSet() with default or optional explicit timeouts.

Comment preview

Comments have been closed on this topic.


No future posts left, oh my!


  1. Speaking (3):
    23 Sep 2015 - Build Stuff 2015 (Lithuania & Ukraine), Nov 18 - 24
  2. Production postmortem (11):
    22 Sep 2015 - The case of the Unicode Poo
  3. Technical observations from my wife (2):
    15 Sep 2015 - Disk speeds
  4. Find the bug (5):
    11 Sep 2015 - The concurrent memory buster
  5. Buffer allocation strategies (3):
    09 Sep 2015 - Bad usage patterns
View all series



Main feed Feed Stats
Comments feed   Comments Feed Stats