Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,026 | Comments: 44,842

filter by tags archive

Rhino Queues, Take 6

time to read 3 min | 521 words

As I stated before, I started writing a queuing system a few days ago, loosely based on my previous efforts in this area, but aimed to create a production ready queuing infrastructure that I can use in my own applications. The decision to build this was not made lightly, but I wanted a queuing system that met my needs, and was flexible enough to extend at needs.

The design goals stated for Rhino Queues were:

  • XCopy deployable
  • Zero configuration
  • Durable
  • Supports System.Transactions
  • Works well with Load Balancing hardware
  • Supports sub queues
  • Support arbitrarily large messages

It is now publically available, and I think it deserve some discussion.

Here is a very simple example of using it:

var queueManager = new QueueManager(new IPEndPoint(IPAddress.Loopback, 2200), "queues.esent");
var queue = queueManager.GetQueue("Web");

    using(var tx = new TransactionScope())
        var msg = queue.Receive();
        Console.WriteLine("Message from {0}:", msg.Headers["source"]);

This example is merely to show the API.

The actual software is pretty interesting. Communication between different queues are done using TCP, with a protocol that works well with load balancing hardware, making load balancing queued application as easy as balancing any HTTP based app.

You cannot see it in the API example, but the system is supporting System.Transactions fully, so sending a message is delayed until a transaction is committed, and receipt of a message would be rolled back on transaction rollback. We even support recovery after a hard crash, by plugging into the recovery mechanism that MSDTC offers us.

All messages are durable, so a system reboot will not remove them. While the default mode for Rhino Queues is an embedded component in your application (XCopy deployable), we still support the ability to deliver a message to a server that is down, by implementing a fairly flexible message retry mechanism.

Because Rhino Service Bus makes such a heavy use of this, we are also support sub queues, and we have no hard limit on the size of messages that we can deliver.

I delayed announcing this until I could finish integrating this completely into Rhino Service Bus, but this is now done, and should just work. Rhino Service Bus will still supports MSMQ, but I think that a lot of the work that we are going to do now on Rhino Service Bus would be with the new queuing infrastructure.



"It is not publically available,"

Did you mean it was 'now publicly available'?

I assume that it was a misprint what with the link and all, but since I'm currently looking at Rhino Service Bus, wanted to make sure.


Ayende Rahien

Yes, that is a misprint, it IS available.

I fixed the post


Very, very exciting stuff...can't wait to integrate it into our messaging project. Have you done any performance comparisons?


Perhaps I'm being facetious, but when you say "arbitrarily large messages", don't you mean messages smaller than 2GB? The length used for message data in SerializationExtensions is only an Int32...


Great, will check it out.

Hopefully it will make Rhino Service bus easier / more accessible to work with.

Microsoft should hire you, they could then retire most their .NET BCL developers :)


Congratulations, hope you're not working on your own operating system yet? Question: can subqueues be freely defined, or are they already pre-defined? Thanks for keeping the API simple... with this addition I'll definitely have a look at Rhino SB.

James L

I look forward to playing with this. From the code sample, something strikes me right away, the use of the word 'Manager' in classes is a pet hate of mine as it doesn't mean anything. What does the QueueManager really do?

Everyone's a critic right?!


I guess TCP communication endpoint should be rather named TcpEndpoint, not IPEndpoint, especially since port numbers have nothing to do with raw IP.

Ayende Rahien


Not only didn't I do any perf comparisons, I didn't even try to think about perf yet :-)

Ayende Rahien


Well, yes. It is limited to 2GB, for more than one reason.

I consider this to be arbitrarily large :-)

Ayende Rahien


subqueues are freely defined, no need to explicitly create them upfront.


Me too, feel free to suggest a better name.


Talk to MS about that, IPEndpoint is their name.


Hi ayende thanks for this code, im trying to run the tests and get an error on the chemaCreator.Create

Error TaggedNotNULL (JET_errTaggedNotNULL, No non-NULL tagged columns)

i found that changing

ColumndefGrbit.ColumnNotNULL to ColumndefGrbit.None

for columns JET_coltyp.LongText, works fine. im using Windows XP SP3.


Just finished reading the post... i dont know if I like it yet... lol

How do you check how many messages are in the queue? Using MSMQ, the operational staff can just check the message count (computer mgmt/services/queues).

Are you planning to create services to monitor your queues? Maybe add them to the computer mgmt. just like MSMQ...

I never got why you must create your own stuff... looks like everything else has lots of problems... It's your time, I'm not here to judge. I just would like to have a better understanding of your reasons. Maybe you could write a blog post about that. (;

A good one woule be a comparison...

nServiceBus x Rhino Service Bus x Mass Transit.

Rhino Queues x MSMQ x ActiveMQ.

I know you have some posts about that, but its never enough.

Keep up the good work.

Ayende Rahien


One of the planned features is a full stats report from the queue, yes.

As for why I create my own, the answer is quite simple, whatever is out there doesn't meet the requirements that I have.

RSB was created to be super simple to use, highly opinionated, developer friendly ESB.

RQ was created to be xcopy deployed with load balancing friendliness.


interesting... good stuff

a couple of questions

well really one question and one note

first the note:

i noticed in one of your tests that you are newing up a new Reciever object in a "using" statement. didn't you have a post about not too long ago stating that it was bad to do that? just curious

now the question:

i feel like i am missing something when looking at the code

i think that i understand that you have to create a Reciever object and handle the CompletedRecievingMessages event in order to know if a message has been sent to the queue or not

i am missing how or where the information is to send a message back

is there something in the Message that tells me where the message came from?

is that something that i have to add in the Headers?

hope the question makes sense


I guess you are using a patched version of the managed Esent API? JetEscrowUpdate doesn't seem to exist on the codeplex version...


I know it's bad to optimize the code too early, but I think Reciever should be spelled Receiver

Ayende Rahien


Yes, I am using a private build.

But I spoke with Laurion, and he said he would add it to the public build

Frank Quednau


With 2GB being arbitrarily large, my customer would beg to differ. They like to send around 4GB avi files in their intranet as if there is no tomorrow. :) Well, no messaging for that one, that project became more of a streaming thing anyway...

Ayende Rahien


I wouldn't call an AVI file a message.

And stream is a much better alternative for that.

Ayende Rahien


Thanks, I fixed that.

Ayende Rahien


using for receiver - I think you were talking about my post regarding using and ISession.

Different things all together.

No, the receiver get a Func{Messages[], IMessageAcceptance}, which is how it let the rest of the app know about new messages.

The event is just to facilitate easier testing.

The information about the source message is located in the message headers.

message.Headers["source"], to be exact.

That is beyond the scope of Rhino Queues, it is something that you need to add.


Ayende, as far as I remember Rhino Service Bus required windows 2008 because of some functions available only in MSMQ 4.0. With the introduction of Rhino queues will it work on windows 2003, xp or vista?

Ayende Rahien


RSB can run on all of those for a while now.

MSMQ 3.0 is supported as well.

And yes, RQ runs on XP, so that is a positive.


thanks for the info... guess i am too stuck in the MSMQ world


hi Ayende im doing some tests with the latest src on svn and i have a prob

first i start a server with:

using (var tx = new TransactionScope())


            receiver = new QueueManager(new IPEndPoint(IPAddress.Loopback, 4545), "werwer.esent");




while (true)



            using (var tx = new TransactionScope())


                   var msg = receiver.Receive("uno", null, new TimeSpan(0,0,10));                      

                    if (msg != null)


                        Console.WriteLine(Encoding.ASCII.GetString(msg.Data) +"   " + msg.Id);                            


now in a client i do:

        using (var tx = new TransactionScope())


            sender = new QueueManager(new System.Net.IPEndPoint(IPAddress.Loopback, 4546), "client.esent");



using (var tx = new TransactionScope())


        sender.Send(new Uri("rhino.queues://localhost:4545/uno"),

                    new MessagePayload


                        Data = Encoding.ASCII.GetBytes("Message "  + x.ToString())






now, i send one msg and receive one in the server

then i run it again and receive 3 times the new msg.

maybe im missing something there?

Ayende Rahien


Please post this to the mailing list

Comment preview

Comments have been closed on this topic.


No future posts left, oh my!


  1. Technical observations from my wife (3):
    13 Nov 2015 - Production issues
  2. Production postmortem (13):
    13 Nov 2015 - The case of the “it is slow on that machine (only)”
  3. Speaking (5):
    09 Nov 2015 - Community talk in Kiev, Ukraine–What does it take to be a good developer
  4. Find the bug (5):
    11 Sep 2015 - The concurrent memory buster
  5. Buffer allocation strategies (3):
    09 Sep 2015 - Bad usage patterns
View all series


Main feed Feed Stats
Comments feed   Comments Feed Stats