Ayende @ Rahien

Hi!
My name is Ayende Rahien
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

@

Posts: 5,947 | Comments: 44,539

filter by tags archive

Digging into MSMQ


I got into a discussion online about MSMQ and its performance. So I decided to test things out.

What I want to do is to check a few things, in particular, how much messages can I push to and from MSMQ in various configurations.

I created a non transactional queue, and then I run the following code:

var sp = Stopwatch.StartNew();
int count = 0;
while (sp.Elapsed.TotalSeconds < 10)
{
var message = new Message
{
BodyStream = new MemoryStream(data)
};
queue.Send(message);
count++;
}

Console.WriteLine(sp.Elapsed);
Console.WriteLine(count);

This gives me 181,832 messages in 10 seconds ,or 18,183 messages per second. I tried doing the same in a multi threaded fashion, with 8 threads writing to MSMQ, and got an insufficient resources error, so we’ll do this all in a single threaded tests.

Next, the exact same code, but for the Send line, which now looks like this:

queue.Send(message, MessageQueueTransactionType.Single);

This gives me 43,967 messages in 10 seconds, or 4,396 messages per second.

Next I added DTC, which gave me a total of 8,700 messages in ten seconds, or 870 messages per second! Yeah, DTC is evil.

Now, how about reading from it? I used the following code for that:

while (true)
{
try
{
Message receive = queue.Receive(TimeSpan.Zero);
receive.BodyStream.Read(data, 0, data.Length);
}
catch (MessageQueueException e)
{
Console.WriteLine(e);
break;
}
}

Reading from transactional queue, we get 5,955 messages per second for 100,000 messages. And using non transaction queue it can read about 16,000 messages a second.

Note that those are pretty piss poor “benchmarks”, they are intended more to give you a feel for the numbers than anything else.  I’ve mostly used MSMQ within the context of DTC, and it really hit the performance hard.


Comments

Edo van Asseldonk

Insufficient resources error can mean that storage limit of msmq is exceeded.

Go to the properties of MSMQ. Click to select the Limit message storage to (KB) check box. In the Amount of space box, enter the limit of kilobytes that you want to enable the Message Queuing resource to retain.

Andrew McClenaghan

Would be interesting to see how this compares to something like RabbitMQ or Azure or AWS SQS.

Clemens Vasters

Those results are not at all surprising. Using MSMQ with DTC doesn't "hit the performance hard" - it's a completely different scenario from trivial enqueue/dequeue where you're simply doing a ton more work because, presumably, you are using DTC to enlist a secondary transactional resource in the same context, which is the only reason why you'd ever use DTC with MSMQ. Similarly, using a transactional queue is different from a regular queue in that it gives the client control over when/if the messages are committed and that's extra cost. More features, more cost.

Jimmy Zimms

Less that DTC is evil and more that like all things engineering related, there are trade offs so profile, measure and think about consequences.

Chris Marisic

https://www.google.com/search?q=msmq+tuning

Many things can be done to increase performance to MSMQ.

I do find it quite interesting that even the most vanilla of all tests shows THOUSANDS and TENS OF THOUSANDS per second.

I'm sure this post was either directly caused by or atleast heavily influenced by my statement that MSMQ can operate at 100K req/s. Seeing your initial numbers my statements certainly seem reasonable. With the right tuning it would be reasonable to expect to be able to achieve a single order of magnitude increase in performance.

Chris Marisic

@Clemens regarding your statements about transactional queues. Can i read that to mean even if a queue is not transactional, that it is still durable in the sense that all messages sent to the queue are crash proof? That if the MSMQ server went down in the middle of a enqueue operation the client would error.

That the only purpose of a transactional queue is to allow the client to undo enqueuing a message? As opposed to an impossible to stop enqueue to a non-transactional queue?

Clemens Vasters

Transactions and durable vs. express are different aspects http://msdn.microsoft.com/en-us/library/ms704130(v=vs.85).aspx

Dan
Dan

In my experience, any time a DTC transaction gets promoted to cross remote machines, it incurs a performance hit that is enough to be easily noticed by a human. For example, I've tried initiating DTC transactions from a WCF client through to a WCF server and ultimately to a sql server database. The transaction can take seconds to initialize or rollback.

And don't get me started on that damn "Insufficient Resources" error. Of course, there's the well known "checklist" of somewhere around 10 possible causes it may happen. But if you're unfortunate enough to get the error for other reasons, you are SOL. The log files for MSMQ are a proprietary format, and only MS support has a tool to read them. And once MS support reads the log file, there's no guarantee they will even know what to do with that info.

Alexander

It would be nice to know the payload size. It makes a huge difference when assessing throughput.

Comment preview

Comments have been closed on this topic.

FUTURE POSTS

  1. RavenDB Sharding - Adding a new shard to an existing cluster, splitting the shard - 32 minutes from now

There are posts all the way to May 22, 2015

RECENT SERIES

  1. RavenDB Sharding (2):
    21 May 2015 - Adding a new shard to an existing cluster, the easy way
  2. The RavenDB Comic Strip (2):
    20 May 2015 - Part II – a team in trouble!
  3. Challenge (45):
    28 Apr 2015 - What is the meaning of this change?
  4. Interview question (2):
    30 Mar 2015 - fix the index
  5. Excerpts from the RavenDB Performance team report (20):
    20 Feb 2015 - Optimizing Compare – The circle of life (a post-mortem)
View all series

RECENT COMMENTS

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats