Ayende @ Rahien

It's a girl

Raven MQ – Principles

Originally posted at 11/9/2010

Raven MQ is a new project that I am working on. As you can guess from the name, this is a queuing system, but it is a queuing system with a few twists.  I already wrote a queuing system in the past (Rhino Queues), why write another one?

Raven MQ builds upon the experience in building Rhino Queues, but it also targets a different set of usage scenarios. Like Rhino Queues, Raven MQ can be xcopy deployed, but it is not usually used in a traditional point to point messaging system. Instead, Raven MQ is a queuing system for the web. What do I mean by that? Raven MQ has a different set of design decisions, focused on making some things that are traditionally expensive in queuing systems cheap:

  • Unlike in most queuing systems, queues are cheap. That allows you to create an unlimited amount of queues. Typical deployment of Raven MQ will have at least one queue per client.
  • Which leads to the next point, Raven MQ is designed to support literally thousands of clients.

The model isn’t the traditional queuing one you might be familiar with from MSMQ:

image

Instead, the model uses a central server to hold all the information:

image

 

The reasoning behind this is actually pretty simple. Unlike in traditional queuing systems, where we have a node of the queuing system running on each end point, Raven MQ makes the assumption that most of the clients connect to it are actually web clients, using JavaScript on the page or maybe Silverlight applications.

The decision to directly support those clients is what makes Raven MQ unique.

Transport models

Raven MQ offers two distinct models for transporting messages. The first is the traditional queue model, where each message can only be consumed by a single consumer. This is not a very interesting model.

A much more interesting model is the message stream. A message stream in Raven MQ is a set of messages sent to a particular queue. But unlike a queue, reading a message from the stream does not consume it. That means that multiple consumers can read the messages on the stream. Moreover, clients that arrive after the message was sent can still read the message (as long as its time to live is in effect).

Usage model

The previous section is probably hard to understand. As usual, an example will makes all the difference in the world.

Let us imagine that we are building a CRM system, and we are currently viewing a customer screen. At that point, we are subscribe to the following streams:

  • /streams/system/notifications – Global system notifications
  • /streams/customers/1234 – Updates about customer 1234
  • /streams/users/4321 – Updates about our logged on user

And the following queue:

  • /queues/mailboxes/1234 – Replies to our particular client

The idea is pretty simple, actually. When we read the customer data, we are loading it from the view model store, but we also need to be able to efficiently get updates about changes that happen to the customer when we are looking at it. We are doing that by subscribing to the appropriate stream. Another user who is also looking at the same user is also subscribed to the same stream. Even more importantly, a user that opened the customer after some changes have been made (but before they were written to the view model store) will also get those updates, and will be able to reconstruct the current state in an seamless manner.

This approach drastically simplifies the update problem in complex systems.

Why call them streams and not topics?

Topics are a routing mechanism, but with Raven MQ, streams aren’t used for routing. They are used to hold a set of messages, that is all. The problem with routing is that you can’t join up later and receive previously sent messages, and (much worse) you can’t really use routing on the web, because when you have potentially thousands of clients, all coming & going at will, you can’t setup a queue for each of them, it is too expensive.

The stream/notification model solve that problem rather neatly, even if I say so myself.

What I did not discussed?

Please note that I am discussing the system at a very high level right now. I didn’t talk about the API or the actual distribution model. That is intentional, I’ll cover that in a future post.

Comments

Ayende Rahien
11/09/2010 10:53 AM by
Ayende Rahien

Mogens,

Actually, no, I am not familiar with it.

Looking at this, I can tell you that I am going to support:

  • Transactions

  • Push notifications

  • Authentication

  • Replication / scale out

  • Streams are using meaningful strings, rather than ids

  • We persist to disk

  • Support for hierarchical topics/streams

GeeBee
11/09/2010 11:59 AM by
GeeBee

I see you're calling the project Raven MQ and not Rhino MQ, does that mean that it will be open sourced but with the restriction that if we want to use it in a commercial product there will be a license fee? I like the idea that anything named Rhino.* is a free license, but Raven.* is opensource with commercial fee applicable

Ayende Rahien
11/09/2010 12:04 PM by
Ayende Rahien

GeeBee,

Yes, the same as RavenDB.

And yes, that is more or less my reasoning.

I
11/09/2010 12:46 PM by
I

Would this model have one server queue per stream?

/streams/system/notifications – Global system notifications

/streams/customers/1234 – Updates about customer 1234

/streams/users/4321 – Updates about our logged on user

would be 3 server side queues

No. of queues could grow massively as customers / users grow. If you had client queues messages would only be held on the server until they were delivered to the client.

It sounds like a very specific problem. Would it not be better to just have the queuing system support time to live, and then the actual deployment is left to consumers?

Ayende Rahien
11/09/2010 01:42 PM by
Ayende Rahien

I,

Except that you can't assume that they will be delivered.

And you can't assume much about the client at all.

The actual implementation isn't really important, but there isn't an actual queue object to take space, not for real.

Reshef
11/09/2010 04:08 PM by
Reshef

Doesn't cqrs already solves this problem?

And, cqrs consolidates the state of the changes to the state instead of letting each client do it for itself.

Richard Dingwall
11/09/2010 11:49 PM by
Richard Dingwall

Reminds me of XMPP. Effectively a message bus but with a more instant-messaging-inspired architecture for clients across the web.

Luke Schafer
11/10/2010 03:15 AM by
Luke Schafer

Reshef - CQRS is irrespective of messaging, let alone a particular messaging implementation (which this is). Instead, try thinking about Raven MQ in a way that directly supports a CQRS implementation (as I think Ayene may have been alluding to)

Darius Damalakas
11/10/2010 09:53 AM by
Darius Damalakas

Unlike in most queuing systems, queues are cheap.

By " most queuing systems", what do you mean?

Sound like in most of the products queues are expensive from your point of saying. (Which in my experience is not true)

Ayende Rahien
11/10/2010 09:58 AM by
Ayende Rahien

Darius,

With MSMQ, RabbitMQ, etc - it is expensive to create a queue. You wouldn't create thousands of them or create/destroy queues on the fly.

Igor Loginov
11/10/2010 10:19 AM by
Igor Loginov

Ayende, it looks more like a new type of database engine with select an insert but without update and delete (which should be fast), extended with publisher-subscriber mechanism. Isn't it? And what about non-web .NET client?

Ayende Rahien
11/10/2010 10:23 AM by
Ayende Rahien

Igor,

I think that I need more information to understand what you mean.

And there will be a .NET Client API

Igor Loginov
11/10/2010 10:46 AM by
Igor Loginov

Let me explain. Your stream concept actually means that you keep all the messages in your centralized storage (i.e., you don't delete them). Messages are not being updated as well - a new message is placed instead. And finally, your storage is centralized. So, it looks like an event triggered DB without update and delete operations. Anyway, it's just another point of view, never mind.

Now I am interested in something like you described. I tried Laharsub mentioned above, and another promising project - nvents.org. But your idea looks more complete

Ayende Rahien
11/10/2010 10:49 AM by
Ayende Rahien

Igor,

I suppose so. Take note that messages are deleted eventually.

And we will probably have to do replication and point to point at some point, so it is a bit more complex than that.

Igor Loginov
11/10/2010 11:28 AM by
Igor Loginov

Generally speaking, you can avoid deletion by ID, Then you will need only a truncate operation (or kind of archiving). Actually, the source of my association is how database for SMS processing works - no updates and deletes... Again, never mind.

Periop IT
11/10/2010 01:54 PM by
Periop IT

Do you think this is something that could be integrated into NServiceBus? When to you plan to have a working version of this Raven MQ?

Ayende Rahien
11/10/2010 02:01 PM by
Ayende Rahien

Periop,

Probably, and I would like to keep options open on that.

I'll say that I have passing tests right now :-)

Igor Loginov
11/10/2010 02:49 PM by
Igor Loginov

Ayende, and will it be possible to query not only full historical set of messages, but also a subset starting from particular time or particular message ID ?

Ayende Rahien
11/10/2010 02:54 PM by
Ayende Rahien

Igor,

Of course.

In fact, the way it works, the client keeps track of its last seen ID.

That is how you ensure that you aren't getting the same message twice.

Igor Loginov
11/10/2010 03:20 PM by
Igor Loginov

Great! Exactly what I wanted to hear. Eager to see it in live.

Periop IT
11/10/2010 03:57 PM by
Periop IT

I can not wait to see this product in action. To me it seems like the answer to using messaging with Smart Clients especially when you need to publish events to smart clients not located on the same subnet. Could this be used with Windows Phone 7?

Jesús López
11/10/2010 06:14 PM by
Jesús López

Ayende,

Can it be used as a kind of Comet server where javascript clients (browsers) long poll queues for events (or messages)?

Luke Schafer
11/10/2010 11:45 PM by
Luke Schafer

@Jesus - It seems very likely that this is the case, as I certainly get that feeling from this short post. Even if, in the off chance, that it isn't exposed as such (which really, I'm pretty sure is half the point), a light wrapper would serve to expose a stream ala comet server.

Luke Schafer
11/11/2010 02:00 AM by
Luke Schafer

Ayende,

Would I be somewhere near correct in assuming the usage (or one usage pattern) would be similar to:

Publish: Post to Stream /stream/service/identifier

Subscribe: Get (instant or long-poll) from /stream/service/identifier

So, the short question, is it modelled on Rest, or is that just the format you decided on for naming streams.

Ayende Rahien
11/11/2010 10:23 AM by
Ayende Rahien

Jesus,

I plan to use WebSockets, not long polling, but essentially, yes.

Ayende Rahien
11/11/2010 10:24 AM by
Ayende Rahien

Luke,

I really like use naming conventions, and it works pretty well in this regard.

It just happens to fall nicely into the REST pattern too.

Jesús López
11/11/2010 10:54 AM by
Jesús López

Ayende,

And are you considering to fall back to long polling when WebSockets is not available at the client side?

Uriel Katz
11/11/2010 01:46 PM by
Uriel Katz

This is basically to support real-time web apps(ala COMET) right?

A message bus with history and WebSockets support.

looks nice will it support Mono? and will it have a HTTP API?

Ayende Rahien
11/11/2010 01:48 PM by
Ayende Rahien

Jesus,

I might, it depends on too many factors. Long polling doesn't really work that well in the scenarios that I have in mind (multi level subscriptions)

Ayende Rahien
11/11/2010 01:48 PM by
Ayende Rahien

Uriel,

That is pretty much it, yes.

It will support Mono, and it has HTTP API

Uriel Katz
11/11/2010 01:54 PM by
Uriel Katz

why you are not supporting long-polling? ,it is pretty much the only cross-browser way of doing real time updates,and it is very easy to implement using async networking.

Ayende Rahien
11/11/2010 02:02 PM by
Ayende Rahien

Uriel,

I didn't say that I am not supporting them, I said that I don't know if I will

Uriel Katz
11/11/2010 02:05 PM by
Uriel Katz

Ha ok sorry about that :)

It will be really nice if you do :) you are going to use async networking either way right?,because you plan to handle a lot of clients(in this case clients can be web browser and not just a app like in normal queuing systems).

Sven
11/14/2010 10:29 PM by
Sven

Ayende, I learned a lot from reading your recent articles in msdn magazine.

How does RavenMQ fit into the ideas you presented in the articles ?

Is it "just" a replacement for Rhino Queues that is better suited to the scenarios you talked about in the articles ? Does anything else change ? Is there going to be a sequel (series) ? :-)

Ayende Rahien
11/15/2010 08:27 AM by
Ayende Rahien

Sven,

It can serve as a replacement for Rhino Queues, but it is also going to serve as a notification server for applications.

I am probably going to write an article or two on that.

Comments have been closed on this topic.