Ayende @ Rahien

It's a girl

Solving problems with messaging: Creating a new user

Another example, which is naturally asynchronous, is the way most web sites create a new user. This is done in a two stage process. First, the user fills in their details and initial validation is done on the user information (most often, that we have no duplicate user names).

Then, we send an email to the user and ask them to validate their email address. It is only when they validate their account that we will create a real user and let them login into the system. If a certain period of time elapsed (24 hours to a few days, usually), we need to delete any action that we perform and make that user name available again.

When we want to solve the problem with messaging, we run into an interesting problem. The process of creating the user is a multi message process, in which we have to maintain the current state. Not only that, but we also need to deal with the timing issues build into this problem.

It gets a bit more interesting when you consider the cohesiveness of the problem. Let us consider a typical implementation.

First, we have the issue of the CreateUser page:

image

Then we have the process of actually validating the user:

image

And, lest us forget, we have a scheduled job to remove expired user account reservations:

image

We have the logic for this single feature scattered across three different places, which execute in different times, and likely reside in different projects.

Not only that, but if we want to make the experience pleasant for the user, we have a lot to deal with. Sending an email is slow. You don’t want to put this as a part of synchronous process, if only because of the latency it will add to showing a response to the user. It is also an unreliable process. And we haven’t even started to discuss error handling yet.

For that matter, sending an email is not something that you should just new an SmtpClient for. You have to make sure that someone doesn’t use your CreateUser page to bomb someone else’s mailbox, you need to keep track of emails for regulatory reasons, you need to detect invalid emails (from SMTP response), etc.

Let us see how we can do this with async messaging, first we will tackle the register user and send an email to validate their email:

image

When the user click on the link in their email, we have the following set of interactions:

image

And, of course, we need to expire the reserved username:

image

In the diagrams, everything that happens in the App Server is happening inside the context of a single saga. This is a single class that manages all the logic relating to the creation of a new user. That is good in itself, but I gain a few other things from this approach.

Robustness from errors and fast response times are one thing, of course, but there are other things that I am not touching here. In the previous example, I have shown a very simplistic approach to handling the behavior, where everything is happening inline. This is done because, frankly, I didn’t have the time to sit and draw all the pretty boxes for the sync example.

In the real world, we would want to have pretty much the same separation in the sync example as well. And now we are running into even more “interesting” problems. Especially if we started out with everything running locally. The sync model make it really hard to ignore the fallacies of distributed computing. The async model put them in your face, and make you deal with them explicitly.

The level of complexity that you have to deal with with async messaging remains more or less constant when you try to bring the issues of scalability, fault tolerance and distribution. They most certainly do not stay the same when you have sync model.

Another advantage of this approach is that we are using the actor model, which make it very explicit who is doing what and why, and allow us to work on all of those in an independent fashion.

The end result is a system compromised of easily disassembled parts. It is easy to understand what is going on because the interactions between the parts of the system are explicit, understood and not easily bypassed.

Comments

Victor Kornov
02/22/2009 08:49 PM by
Victor Kornov

Both "(click to enlarge)" links open the same small images.

Rafal
02/23/2009 09:26 AM by
Rafal

I think workflow model is a better abstraction of such design problems. Interaction diagrams, as above, go too much into details and it's harder to see the 'logic' of this scenario.

Gavin
02/23/2009 11:44 AM by
Gavin

Rafal, Isn't the goal of a sequence diagram to model the sequence interactions between types? For logic you should have probably asked Ayende for use cases ... I still dont get the relevance of what your saying for this article.

Rafal
02/23/2009 12:03 PM by
Rafal

Gavin, Ayende has given enough information about business scenario, I don't need use cases to know what he's talking about. My point is that issues like this one can be easily addressed by a workflow engine. Workflow centralizes the logic, coordinates all the parties involved and takes care of timing and synchronization issues. Of course there is a very interesting subject of what happens underneath - what messages are sent, how these components interact and so on - but that's an implementation detail and not the logic our customers or users are talking about.

Ayende Rahien
02/23/2009 12:46 PM by
Ayende Rahien

Rafal,

That is not a diagram that I intend to give to a customer or a user.

This is a developer diagram

Rafal
02/23/2009 01:31 PM by
Rafal

Ayende, you have written a very nice piece about system design and I agree with you that async communication mechanics are very difficult to understand and implement properly. Simple case of user registration results in a very complex interaction between several components and it's great you show how to approach it with sagas and messaging. But at the same time I wish we, application developers, knew the design principles, understood the architecture, and then used some higher-level tools so we don't get overwhelmed with the complexities. We need a kind of language that encapsulates async messaging, state persistence, sagas, correlations and component communication, so we can concentrate on user's interaction with the application. Hope you touch this subject in future.

Stephen
02/23/2009 01:50 PM by
Stephen

Still can't get over the concept of using javascript to spin waiting for something to happen, I understand you want to free a servers request threads so they can forefil other requests, but when I have a system that uses javascript, I almost always have that to be a 'neat' version of a vanilla http process.. and I can only imagine the experience for vanilla http would be... weird.

Here's how it would go:

visit site, fill in registration form, hit submit, instantly get a response telling me to click a button to what? check how my registration is going?

I couldn't do that, the javascript one is fine, once filling in the form I'd just run an ajax request to start the registration, and keep it 'pinging' for success while I spin with 'please wait'.

It seems like a 'hack' to a problem of locking request threads up, is it really needed? surely there must be a better way to handle waiting threads in IIS / ASP.NET.

Ayende Rahien
02/23/2009 02:35 PM by
Ayende Rahien

Rafal,

You can assume that business level scenarios are presented quite differently.

I usually talk about user & system, without going into the details about the actual system.

Ayende Rahien
02/23/2009 02:36 PM by
Ayende Rahien

Stephen,

meta refresh can do the work as well, and it is quite easy to set it up.

No need for user interaction at all.

Ayende Rahien
02/23/2009 02:37 PM by
Ayende Rahien

Stephen,

async pages requires that the developer manage threading explicitly.

It also result in poorer UX in many cases, because page load times are significantly higher.

Gavin
02/23/2009 02:42 PM by
Gavin

@Rafal

I was not being facetious when I made the remark about use cases. Your comment lacked information. Isn't there a fundamental difference in how the state is managed with a workflow engine? The clue here is the word 'saga' which by definition is wrapping these functions up into a single context. I think workflow might be off the mark as the concept is very different to what is being stated in this article.

Rafal
02/23/2009 03:16 PM by
Rafal

Gavin, 'facetious' is certainly a new word in my dictionary, thanks. Persisting saga state and persisting workflow state is the same thing. I said 'workflow engine' but should have said 'orchestration engine' - service responsible for coordinating an interaction between several components. Something one level of abstraction higher than sagas with built in state management, messaging and reusable process building blocks. For me, these are very closely related concepts.

Stephen
02/23/2009 03:34 PM by
Stephen

Ayende I guess this is just differing opinions, the load times should be the same (if not faster with async ones) if you consider the load time to be the time from the user hitting submit and getting to a 'check your mail'.

I'd never use meta-refresh as it can freak people out and isn't very accessible nor friendly to browser history.

Gavin
02/23/2009 03:48 PM by
Gavin

@Rafal

I am still not sure, ..., state persistance is the same for most things no matter which way you look at it. The management thereof might be a different story. I might be wrong but this article is about yielding the benefit of async design upfront and the interesting problems along the way that get solved. You might just be using a hammer to crack a wall nut for the sake of this article :)

Rafal
02/23/2009 07:47 PM by
Rafal

Gavin, you're right. For such case, using a workflow engine would be like using a cannot to shoot a sparrow (a polish proverb). But user registration is only the beginning, probably the system will grow to contain hundreds of functions, use cases, procedures, rules and exceptions to rules and such detailed approach quickly becomes unmanageable. Even in this simple example there are 3 sub-procedures - registration, confirmation and expiration. I know I'm biased because i'm working on a project with hundreds of 'business cases' and there's no single person that can understand it, further development is like building a hut from cow dung - you know, slap some here and some there until it sticks together. We had to resort to extreme programming in pairs - one developer holds the walls together while another one goes to get the pay from customer. To address such problems, I'm developing a workflow engine and use Ayende's blog as an inspiration.

Mr_Simple
02/23/2009 09:43 PM by
Mr_Simple

@Rafal

Umm just what I want to maintain, a homegrown workflow engine.

What was the name of the place you work? Oh never mind, they'll be out of money before ya'll get the project finished.

Rafal
02/24/2009 06:24 AM by
Rafal

Thanks Simple, if ya wanna help I'll remember to have a bucket of dung for you.

Mr_Simple
02/24/2009 11:11 PM by
Mr_Simple

@Rafal

Yup that's the bucket of goo that I pulled out of the codebase to simplify things.

Ivan K
02/27/2009 04:35 PM by
Ivan K

The end result is the system COMPROMISED...

What a Freudian slip of tongue :-)

On a more serious note, after a brief reading, frankly I am missing the point here. How exactly the whole async messaging saga falls into one class? Would it be different or same in the sync scenario?

Comments have been closed on this topic.