+972 52-548-6969
Solving problems with messaging: Creating a new user
Another example, which is naturally asynchronous, is the way most web sites create a new user. This is done in a two stage process. First, the user fills in their details and initial validation is done on the user information (most often, that we have no duplicate user names).
Then, we send an email to the user and ask them to validate their email address. It is only when they validate their account that we will create a real user and let them login into the system. If a certain period of time elapsed (24 hours to a few days, usually), we need to delete any action that we perform and make that user name available again.
When we want to solve the problem with messaging, we run into an interesting problem. The process of creating the user is a multi message process, in which we have to maintain the current state. Not only that, but we also need to deal with the timing issues build into this problem.
It gets a bit more interesting when you consider the cohesiveness of the problem. Let us consider a typical implementation.
First, we have the issue of the CreateUser page:
Then we have the process of actually validating the user:
And, lest us forget, we have a scheduled job to remove expired user account reservations:
We have the logic for this single feature scattered across three different places, which execute in different times, and likely reside in different projects.
Not only that, but if we want to make the experience pleasant for the user, we have a lot to deal with. Sending an email is slow. You don’t want to put this as a part of synchronous process, if only because of the latency it will add to showing a response to the user. It is also an unreliable process. And we haven’t even started to discuss error handling yet.
For that matter, sending an email is not something that you should just new an SmtpClient for. You have to make sure that someone doesn’t use your CreateUser page to bomb someone else’s mailbox, you need to keep track of emails for regulatory reasons, you need to detect invalid emails (from SMTP response), etc.
Let us see how we can do this with async messaging, first we will tackle the register user and send an email to validate their email:
When the user click on the link in their email, we have the following set of interactions:
And, of course, we need to expire the reserved username:
In the diagrams, everything that happens in the App Server is happening inside the context of a single saga. This is a single class that manages all the logic relating to the creation of a new user. That is good in itself, but I gain a few other things from this approach.
Robustness from errors and fast response times are one thing, of course, but there are other things that I am not touching here. In the previous example, I have shown a very simplistic approach to handling the behavior, where everything is happening inline. This is done because, frankly, I didn’t have the time to sit and draw all the pretty boxes for the sync example.
In the real world, we would want to have pretty much the same separation in the sync example as well. And now we are running into even more “interesting” problems. Especially if we started out with everything running locally. The sync model make it really hard to ignore the fallacies of distributed computing. The async model put them in your face, and make you deal with them explicitly.
The level of complexity that you have to deal with with async messaging remains more or less constant when you try to bring the issues of scalability, fault tolerance and distribution. They most certainly do not stay the same when you have sync model.
Another advantage of this approach is that we are using the actor model, which make it very explicit who is doing what and why, and allow us to work on all of those in an independent fashion.
The end result is a system compromised of easily disassembled parts. It is easy to understand what is going on because the interactions between the parts of the system are explicit, understood and not easily bypassed.
Comments
Both "(click to enlarge)" links open the same small images.
I think workflow model is a better abstraction of such design problems. Interaction diagrams, as above, go too much into details and it's harder to see the 'logic' of this scenario.
Rafal, Isn't the goal of a sequence diagram to model the sequence interactions between types? For logic you should have probably asked Ayende for use cases ... I still dont get the relevance of what your saying for this article.
Gavin, Ayende has given enough information about business scenario, I don't need use cases to know what he's talking about. My point is that issues like this one can be easily addressed by a workflow engine. Workflow centralizes the logic, coordinates all the parties involved and takes care of timing and synchronization issues. Of course there is a very interesting subject of what happens underneath - what messages are sent, how these components interact and so on - but that's an implementation detail and not the logic our customers or users are talking about.
Rafal,
That is not a diagram that I intend to give to a customer or a user.
This is a developer diagram
Ayende, you have written a very nice piece about system design and I agree with you that async communication mechanics are very difficult to understand and implement properly. Simple case of user registration results in a very complex interaction between several components and it's great you show how to approach it with sagas and messaging. But at the same time I wish we, application developers, knew the design principles, understood the architecture, and then used some higher-level tools so we don't get overwhelmed with the complexities. We need a kind of language that encapsulates async messaging, state persistence, sagas, correlations and component communication, so we can concentrate on user's interaction with the application. Hope you touch this subject in future.
Still can't get over the concept of using javascript to spin waiting for something to happen, I understand you want to free a servers request threads so they can forefil other requests, but when I have a system that uses javascript, I almost always have that to be a 'neat' version of a vanilla http process.. and I can only imagine the experience for vanilla http would be... weird.
Here's how it would go:
visit site, fill in registration form, hit submit, instantly get a response telling me to click a button to what? check how my registration is going?
I couldn't do that, the javascript one is fine, once filling in the form I'd just run an ajax request to start the registration, and keep it 'pinging' for success while I spin with 'please wait'.
It seems like a 'hack' to a problem of locking request threads up, is it really needed? surely there must be a better way to handle waiting threads in IIS / ASP.NET.
Ayende, what about async pages in asp.net 2:
msdn.microsoft.com/en-us/magazine/cc163725.aspx
The threads are free'd back to the pool (to work) whilst they wait on an async event.
Rafal,
You can assume that business level scenarios are presented quite differently.
I usually talk about user & system, without going into the details about the actual system.
Stephen,
meta refresh can do the work as well, and it is quite easy to set it up.
No need for user interaction at all.
Stephen,
async pages requires that the developer manage threading explicitly.
It also result in poorer UX in many cases, because page load times are significantly higher.
@Rafal
I was not being facetious when I made the remark about use cases. Your comment lacked information. Isn't there a fundamental difference in how the state is managed with a workflow engine? The clue here is the word 'saga' which by definition is wrapping these functions up into a single context. I think workflow might be off the mark as the concept is very different to what is being stated in this article.
Gavin, 'facetious' is certainly a new word in my dictionary, thanks. Persisting saga state and persisting workflow state is the same thing. I said 'workflow engine' but should have said 'orchestration engine' - service responsible for coordinating an interaction between several components. Something one level of abstraction higher than sagas with built in state management, messaging and reusable process building blocks. For me, these are very closely related concepts.
Ayende I guess this is just differing opinions, the load times should be the same (if not faster with async ones) if you consider the load time to be the time from the user hitting submit and getting to a 'check your mail'.
I'd never use meta-refresh as it can freak people out and isn't very accessible nor friendly to browser history.
@Rafal
I am still not sure, ..., state persistance is the same for most things no matter which way you look at it. The management thereof might be a different story. I might be wrong but this article is about yielding the benefit of async design upfront and the interesting problems along the way that get solved. You might just be using a hammer to crack a wall nut for the sake of this article :)
Gavin, you're right. For such case, using a workflow engine would be like using a cannot to shoot a sparrow (a polish proverb). But user registration is only the beginning, probably the system will grow to contain hundreds of functions, use cases, procedures, rules and exceptions to rules and such detailed approach quickly becomes unmanageable. Even in this simple example there are 3 sub-procedures - registration, confirmation and expiration. I know I'm biased because i'm working on a project with hundreds of 'business cases' and there's no single person that can understand it, further development is like building a hut from cow dung - you know, slap some here and some there until it sticks together. We had to resort to extreme programming in pairs - one developer holds the walls together while another one goes to get the pay from customer. To address such problems, I'm developing a workflow engine and use Ayende's blog as an inspiration.
@Rafal
Umm just what I want to maintain, a homegrown workflow engine.
What was the name of the place you work? Oh never mind, they'll be out of money before ya'll get the project finished.
Thanks Simple, if ya wanna help I'll remember to have a bucket of dung for you.
@Rafal
Yup that's the bucket of goo that I pulled out of the codebase to simplify things.
The end result is the system COMPROMISED...
What a Freudian slip of tongue :-)
On a more serious note, after a brief reading, frankly I am missing the point here. How exactly the whole async messaging saga falls into one class? Would it be different or same in the sync scenario?
Comment preview