Assume that I have the following interface:

public interface IMessageHandler<T> where T : AbstractMessage
{
	void Handle(T msg);
}

How would you write this method so dispatching a message doesn't require reflection every time:

public void Dispatch(AbstractMessage msg)
{
	IMessageHandler<msg.GetType()> handler = new MyHandler<msg.GetType()>();
	handler.Handle(msg);
}

Note that you can use reflection the first time you encounter a message of a particular type, but not in any subsequent calls.

Here is another thing to note, computers are fast. I can't tell you how fast because it would take too long. Thinking about micro performance is a losing proposition. Sasha has asked an important question:

Now assume you have a non-functional requirement saying that you must support 1,000,000 messages per second inserted into this queue. Would you still disregard the fact using an interface is a BAD decision?

My answer, yes. The reason for that? Let us take a look at the slowest method I could think of to do in process queue:

public static void Main(string[] args)
{
	Queue<string> queue = new Queue<string>();
	ThreadPool.QueueUserWorkItem(delegate(object state)
	{
		Stopwatch startNew = Stopwatch.StartNew();
		for (int i = 0; i < 100000000; i++)
		{
			lock(queue)
			{
				bool queueEmpty = queue.Count == 0;
				queue.Enqueue("test");
				if(queueEmpty)
					Monitor.Pulse(queue);
			}
		}
		Console.WriteLine("Done publishing in: " + startNew.ElapsedMilliseconds);
	});
	ThreadPool.QueueUserWorkItem(delegate(object state)
	{
		Stopwatch startNew = Stopwatch.StartNew();
		for (int i = 0; i < 100000000; i++)
		{
			lock (queue)
			{
				while (queue.Count == 0) 
					Monitor.Wait(queue);
                                queue.Dequeue();
			}
		}
		Console.WriteLine("Done reading in: " + startNew.ElapsedMilliseconds);
	});
	Console.ReadLine();
}

On my machine, this outputs:

Done publishing in: 23044 (ms)
Done reading in: 26866 (ms)

Which means that it processed one hundred million items in just 26 seconds or so.

This also puts us at close to 4 million messages per second, using the most trivial and slow performing approach possible.

In fact, using a LockFreeQueue, we get significantly worse performance (3 times as slow!). I am not quite sure why, but I don't care. Pure pumping of messages is quick & easy, and scaling to millions of messages a second is trivial.

Yes, dispatching the messages is costly, I'll admit. Changing the reader thread to use:

instance.Handle(queue.Dequeue());

Has dropped the performance to a measly 2 million messages a second. Changing the message handler to use late bound semantics, like this:

IHandler instance = Activator.CreateInstance<Handler>();
instance.Handle(queue.Dequeue());

Would drop the performance down to 300,000 per second. So it is far more interesting to see what you are doing with the message the the actual dispatching.

And yes, here, in the dispatching code, I would probably do additional work. Using GetUninitializedObject() and cached constructor I was able to raise that to 350,000 messages per second.

And yes, this is a flawed benchmark, you need to test this with multiple readers & writers to get a more real understanding what what would actually happen. But initial results are very promising, I think.

And again, computers are fast, don't look for microseconds.

Sasha pointed out that I should also test what happens when you are using multiply implementation of the interface, vs. direct calls. This is important because of JIT optimizations with regards to interface calls that always resolve to the same instance.

Here is the code:

class Program
{
	public static void Main(string[] args)
	{
		//warm up
		PerformOp(new List<string>(101));
		PerformOp(new NullList());
		List<long> times = new List<long>();

		Stopwatch startNew = new Stopwatch();
		for (int i = 0; i < 50; i++)
		{
			startNew.Start();
			PerformOp(new List<string>(101));
			PerformOp(new NullList());
			times.Add(startNew.ElapsedMilliseconds);
			startNew.Reset();
		}
		Console.WriteLine(times.Average());
	
	}

	private static void PerformOp(List<string> strings)
	{
		for (int i = 0; i < 100000000; i++)
		{
			strings.Add("item");
			if(strings.Count>100)
				strings.Clear();
		}
	}

	private static void PerformOp(NullList strings)
	{
		for (int i = 0; i < 100000000; i++)
		{
			strings.Add("item");
			if (strings.Count > 100)
				strings.Clear();
		}
	}
}

And the results are:

• IList<T> - 5789 milliseconds
• List<string>, NullList - 3941 millseconds

Note that this is significantly higher than the previous results, because no we run the results two hundred million times.

Individual method call here costs:

• IList<T> - 0.0000289489
• List<string>, NullList - 0.0000197096

We now have a major difference between the two calls. When we had a single impl of the interface, the difference between the two was: 0.00000406 milliseconds.

With multiple implementations, the difference between the two methods is: 0.0000092393 milliseconds. Much higher than before, and still less than a microsecond.

Sasha commented on my perf post, and he mentioned the following:

But what if you're designing a messaging infrastructure for intra-application communication?  It is no longer fine to hide it behind a mockable IQueue interface because that interface call is more expensive than the enqueue operation!

Now, I know that an interface call is more expensive than a non virtual call. The machine needs to do more work. But exactly how much more work is there to do? I decided to find out.

class Program
{
	public static void Main(string[] args)
	{
		//warm up
		PerformOp(new List<string>(101));
		List<long> times = new List<long>();

		Stopwatch startNew = new Stopwatch();
		for (int i = 0; i < 50; i++)
		{
			startNew.Start();
			PerformOp(new List<string>(101));
			times.Add(startNew.ElapsedMilliseconds);
			startNew.Reset();
		}
		Console.WriteLine(times.Average());
	
	}

	private static void PerformOp(IList<string> strings)
	{
		for (int i = 0; i < 100000000; i++)
		{
			strings.Add("item");
			if(strings.Count>100)
				strings.Clear();
		}
	}
}

I run this program with first with PerforOp(List<string>) and then with PerformOp(IList<string>). In the first case, we are doing non virtual method call, and in the second, we are going through the interface.

Running this has produced the following results:

• IList<T> - 2786 milliseconds
• List<T> - 2380 milliseconds

Well, that is a significant, right?

Not really. Remember that we run a tight loop of a hundred million times.

Let us see individual call times:

• IList<T> - 0.0000278600 milliseconds
• List<T>  - 0.0000238000 milliseconds

The difference between the two is 0.00000406 milliseconds. In other words, it is an insignificant part of a microsecond.

The difference between a non virtual call and an method call is significantly less than a microsecond.

I don't think that I am going to worry about that, thank you very much.

A while ago I posted about upfront optimizations. Sasha Goldshtein commented (after an interesting email conversation that we had):

What truly boggles me is how the argument for correctness in software is not applied to performance in software.  I don't understand how someone can write unit tests for their yet-unwritten code (as TDD teaches us) and disregard its performance implications, at the same time.  No one in their right mind could possibly say to you, "Let's define and test for correctness later, first I'd like to write some code without thinking about functional requirements."  But on the other hand, how can you say to someone, "Let's define and test for performance later, first I'd like to write some code without thinking about non-functional requirements?"

It fell into my blog backlog (which is a tame black hole) and just now came up.

The reason for this attitude, which I subscribe to, is very simple. A lot of developers will do a lot of really bad things in the name of performance. But I did see, more than once, whole teams dedicate inordinate amount of time to performance, contorting the design so it resembles a snake after a fit of the hiccups and leave an unholy mess in their wake.

Add to that that most of the time performance bottlenecks will not be where you think they would, you get into a really big problem with most developers (myself included, by the way). As a result, the push to "don't think about performance" has very real reasoning behind it. As I mentioned, you should think about performance when you are designing the system, but only in order to ensure that you aren't trying to do stupid things. (Like running a server off a single user embedded database, my latest blunder). Most of the time, and assuming this is not a brand new field, you already know that calling the DB in a tight loop is going to be a problem, so you don't even get into that situation.

Another issue is that I can refactor for performance, but (by definition), I can't refactor to be correct.

Recently I had several discussions about the usage of patterns, and application thereof. Usually, there are recommended implementation for a pattern, often this is the way it is presented in its pattern catalog.

The discussion had to do with the amount of fidelity that you had in implementing the pattern. My view on that is that the usage of a pattern doesn't require a one to one match with the common implementation, but leave room for change while keeping the concepts of the pattern in use.

Patterns are the overall shape of the solution, they are not a blueprint, to be slavishly followed.

Since people seems to really enjoy posts like this, here is another one. This time it is an interesting issue that I dealt with today.

Given a set of versioned file, you need to cache them locally. Note that IPersistentCache semantics means that if you put a value in it, is is always going to remain there.

Here is the skeleton:

public interface IPersistentCache
{
	void Set(string key, params string[] items);
	string[] Get(string key);
}

public enum Recursion
{
	None,
	OneLevel,
	Full
}

public class VersionedFile
{
	public int Version;
	public string Name;
}

public class FileSystemCache : IFileSystemCache 
{
	IPersistentCache cache;
	
	public FileSystemCache(IPersistentCache cahce)
	{
		this.cache = cache;
	}
	
	public void Add(VersionedFile[] files)
	{
		// to do
	}

	public string[] ListFilesAndFolders(string root, int version, Recursion recursion)
	{
		// to do
	}
	
}

How would you implement this? Note that your only allowed external dependency is the ICache interface.

The usage is something like this:

// given 
var fsc = new FileSystemCache(cache);
fsc.Add(new []
{
      new VersionFile{Version = 1, Name = "/"},
      new VersionFile{Version = 1, Name = "/foo"},
      new VersionFile{Version = 1, Name = "/foo/bar"},
      new VersionFile{Version = 1, Name = "/foo/bar/text.txt"},
});
fsc.Add(new []
{

      new VersionFile{Version = 2, Name = "/"},
      new VersionFile{Version = 2, Name = "/foo"},
      new VersionFile{Version = 2, Name = "/foo/bar"},
      new VersionFile{Version = 2, Name = "/foo/bar/text.txt"},
      new VersionFile{Version = 2, Name = "/test.txt"},
});

// then 
fsc.ListFilesAndFolders("/", 1, Recursion.None) == { "/" } 

fsc.ListFilesAndFolders("/", 1, Recursion.OneLevel) == { "/", "/foo", } 

fsc.ListFilesAndFolders("/", 2, Recursion.OneLevel) == { "/", "/foo", "test.txt" } 

fsc.ListFilesAndFolders("/", 1, Recursion.Full) == { "/", "/foo", "/foo/bar", "/foo/bar/text.txt"}  

You can assume that all paths are '/' separated and they always starts with '/'.

I am having an interesting discussion in email with Evan Hoff, about the benefits of pub/sub architecture vs. request/reply. I want to state upfront that I see a lot of value in async one way messaging for a lot of the interactions in an application and across services. However, I am still skeptic about the push for pub/sub all the way.

To me, it seems like the request/reply model is a very natural one in many common scenarios. In most backend processing systems, I would tend to use pub/sub with sagas as both the easiest and most scalable solution. But in almost all cases that end up in the UI, this just doesn't make sense. Even if I want to fire several requests and then wait for all their reply, the model is still more natural than a pub/sub approach.

Note that the following code sample use imaginary API.

Let us go back to the displaying the available candidates. Using the request/reply model, I would have:

public void AvailableCandidates()

{
     var specifications = BuildSpecifications();
     var msg = bus.Process(new FindAvailableCandidateMessage(specifications));
     DisplayResults(msg.Results);
}

All of those are using the request / reply model.

The pub/sub model will look somewhat like this:

public void AvailableCandidates()
{
	var specifications = BuildSpecification();
	
	bus.Publish(new FindAvailableCandidatesMessage(specifications))
		.WaitForCallback(delegate(AvailableCandidatesMessage msg)
		{
			DisplayResults(msg.Results); 
		});
	
}

Or probably something like:

public void AvailableCandidates()
{
	var specifications = BuildSpecification();
	Guid correlationId = Guid.NewGuid();
	ManualResetEvent waitForReply = new ManualResetEvent(false);
	
	bus.SubscribeOnce<AvailableCandidatesMessage>(correlationId, delegate(AvailableCandidatesMessage msg)
		{
			DisplayResults(msg.Results); 
			waitForReply.Set();
		});
	bus.Publish(new FindAvailableCandidatesMessage(correlationId, specifications));
	waitForReply.WaitOne();
}

Notice that I am not saying anything about the way those are handled once they leave the client. In fact, some of the reasons that were mentioned to favor the pub/sub model is the flexibility that you get when adding more functionality. Such as adding auditing, or redirecting messages to different services based on the type of candidate, etc.

The problem is that I see not inherent need for pub/sub to make this work. In fact, I think that anything that publish a message and than waits for the callback message is a big API issue, because this is request/reply, but with harder to use API. Even if you are doing the wait in async manner, I still think that this is a bad idea.

In the context of a request/reply interaction, we can handle everything the way we would handle any other message:

In fact, from the perspective of the infrastructure, there is little need to distinguish those. A message going out will go out either via a request/reply channel or a one way output channel, with a reply maybe arrived through a one way input channel. We don't really care about that.

From my point of view, it makes a lot more sense than to wait for a callback.

Thoughts?

Yesterday I posted about designing an SOA system. I identified the following service:

Available candidates - given a set of requirements, will attempt to find all verified candidates that match the requirements. Additional responsibilities include logging search history (important for noticing what requirements are common and desirable), and recording which requirements has no matching physicians in the system (should be passed for human inspection, to decide if head hunting should begin, or this should be ignored).

BIll Poole has left a comment regarding this post, which included this statement:

For example, the "Available Candidates" service appears highly data centric, likely exposing search/data retrieval operations. CRUD interfaces are bad.

This is something that I have often heard, but I still don't understand. I understand why CRUD interfaces are discourage, you get better results by just using a DB. But in this case?

How do you get the data out of an SOA system? It seems to me like this is a natural requirement for the problem at hand, but I feel that I am missing something.

This has me very confused.

The answer, however, turned out to be that I am an idiot: