Ayende @ Rahien

Hi!
My name is Ayende Rahien
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

@

Posts: 5,947 | Comments: 44,540

filter by tags archive

Macto, Authorization decisions


image

Authorization is one of the few ubiquitous requirements, you are going to have to handle them in pretty much every system that you are going to build.

The users are the staff, and the securables are the inmates. The problem is that we have fairly different authorization requirements for different parts of the system.

For example, any authenticated user can lookup pretty much any inmate’s data (except for medical records, of course), but changing release date is something that only Legal can do. Only the staff on the enclosure that the inmate is located in can Sign Out the inmate. Actually releasing an inmate requires Legal and On Duty Officer approval, etc.

However, during weekends, the on duty staff assume responsibility for the entire prison. That is, an officer from enclosure C can Sign Out an inmate from enclosure A if that officer is the on duty officer.

There are a few more complications, but we will ignore them for now. One thing that is fairly clear, we have fairly complex authorization requirements, and they are different for each part of the system. For that matter, the way we make security decisions itself is different.

And since authorization decisions are synchronous (you can make them async, sort of, but at very high cost), performance is a critical concern. This is especially true because there is a strong tendency to call authorization decisions many times.

Given that, and given the complexity inherit to authorization, I think that we can skip the entire problem entirely by changing the rules of the game.

Most authentication systems would have you do something like:

auth.IsAllowed(CurrentUser, "/Inmate/Move", inmate);

And rely on the system to do its magic this way. The problem in this manner is that we provide the authorization system with very little information with which it can work. That means that in order to make authorization decisions, the system has to have a way to access other data (such as in what enclosures the current user is in charge of, where the inmate is, what is its status, etc).

The problem then become more an issue of data retrieval complexity rather than the authorization rules complexity. I think that we can avoid this by designing the system with more flexibility by providing the required data to the authorization system explicitly.

What do I mean? Well, just take a look:

auth.IsAllowed(
	new SignOutAuthorization
	{
		OfficerRank = CurrentUser.Rank,
		OfficerRoles = CurrenUser.GetAllCurrentRoles(),
		OfficerEnclosures = CurrentUser.GetEnclosuresUserIsResponibleFor(),
		InmateEnclosure = inmate.Enclosure,
		InmateStatus = inmate.Status,
	}
);

In other words, we are explicitly providing the authorization system with all the data that it needs for a particular task. That means, in turn, that we can now execute the authorization decision completely locally, without having to go somewhere to fetch the data. It also open the option of using a DSL to build the authorization rules, which will make things more dynamic and easier to work with.

Reviewing NerdDinner: The Select N+1 pitfall


During my review of NerdDinner, I came across the following piece of code:

image

And I knew that there is a Select N+1 problem here. This is quite similar to the problem that I described here. RSVPs is a lazily loaded collection. As such, calling Any() on it will result in a database query.

Now the question was whatever or not it is used somewhere in a loop. ReSharper will help us figure that one out:

image

The first usage location is here:

image

And this certainly doesn’t look like it can cause a Select N+1, right. Just to be sure, I checked where it is called from the client side as well:

image

This looks all right, we will have a query, but it is only one per request.

Great, now let us take a look at the second usage. You can tell that this is going to be a problem just by the ascx suffix. The code is pretty simple:

image

I have a moral issue with the view generating a call to the database, but at least it isn’t being done in a loop.

I am certain, however, that we are going to find this .ascx file in a grid, which will cause a loop. This .ASCX file is being used in Details.aspx and is used like this:

image

However, it is not actually being used in a loop.

There is not a Select N+1 issue, which renders this entire post moot.

However, while it is not an active Select N+1, it is a dormant one. It would be very easy to have a new requirements that would make use of the RSVPStatus.ascx file in a loop and cause the issue.

Reviewing NerdDinner


Scott Hanselman has asked me to see if I can port NerdDinner to NHibernate, now that Linq for NHibernate has made RTM. I thought that I might as well take the time to look at the code and do a code review.

I would like to say a few things before actually even starting to look at the code. I am not going to review the code as a sample application, I am going to review it as if it was a standard production project. I am assuming that (remember, haven’t seen the code yet) there are things there that are done that way explicitly because they are easier to explain as a sample app than a more complex project would be.

The first thing that I opened was HomeController, which I am going to show you in full.

image

There are two things that I don’t like here.

First, the empty actions (return View() is empty as far as I am concerned) resulting in what is, for all intents and purposes, an empty controller (one that contains empty actions). In other words, the only reason that this controller exists is because it is the way ASP.Net MVC want it to. A brief overview of the codebase show me that there are more than a few of method like that.

I would deal with something like that completely on the view side, having a routing action that would check the view directly and just render the view, rather than creating empty actions and controllers.

Second, [HandleErrorWithELMAH], such things should not be on controllers (and that attribute appears on all controllers). It should be on a base controller, because right now, it is a violation of DRY, and it is easy to forget.

Then there are the tests for HomeController, which are… less than useful, shall we say?

image

Next, let us take a look at the following interface:

image

I have a serious problem with the Save() method. I had to look at the implementation to figure out what it is doing. It should be called FlushToDatabase, or SubmitChanges. Save() is a totally unexpected name for the method who implementation is:

image

You are not, actually, saving anything.

As an aside, I don’t like seeing things like this one:

image

Linq to SQL actually have a way of handling cascading deletes, and it should have been used.

For that matter, coming back to the tests, there aren’t any persistence tests. Since there are some complexity involved in the queries (geo located queries), I would have liked to see at least those queries being covered.

I don’t want to rehash the poor man’s IoC story, but this is really pissing me off:

image

I mean, if you want to do poor man’s IoC, go ahead. But please don’t create this bastard child.

This is more about code style than hygiene, but I don’t like it:

image

Those classes should be in separated files, they should not be in AccountController.

Overall, I don’t see too many issues with the codebase. The only thing that pissed me off was the Mad Man IoC.

Macto, defining Centralized Service, Distributed Service and Localized Component


I lately come into the conclusion that I need a few new terms to describe a few common ways to talk about the way that I structure different components in my applications.

Those are Centralized Service, Distributed Service and Localized Component. Mostly, I use them as a way to express the distribution semantics of the item in question.

As you can probably guess, I am using the term service to refer to something that we make remote calls to, while I am using the term component to refer to something that is running locally.

Centralized Service is probably the classic example of a web service. It is a server that is running somewhere to which we make remote calls to. As far as the system is built, there is only one such server. It may be implemented with clustering or load balancing, but logically (and quite often, physically), it is a single server that is processing requests. This is probably the easiest model to work with, since it more or less remove the entire question of concurrency conflicts from the system. Internally, the Centralized Service is using transactions or locks to ensure coherency in the face of concurrency.

Distributed Service is a built upfront to run on a set of server, and the need to handle concurrency conflicts is built into the design of the system. That may be done using sharding, Paxos or other methods. Usually, we build Distributed Service for very high scalability / reliability cases, since it tend to be the more complex solution. An example of a Distributed Service would be DNS, where we explicitly design the system to be resilient to failure, but accept the more complex concurrency issues (slow updates).

Localized Component is a solution to the chatty interface problem. There are quite a few scenarios where we need to make calls to a separated subsystem, but the cost of network traffic completely outweigh the cost of actually performing the operation on the other side. In this case, we may switch from a Centralized Service to a Localized Component. What this means is that instead of executing the operation on the other side, we perform it locally.

In practice, this means that we need to design our system in a way that any data that we would like to have is structured in such a way that it can be brought locally or retrieved very cheaply. An example of such a system appears in this post, although that is a fairly complex one. A more common situation is a component that deals with a set of rule, and we simply need to get the rule from the rule repository and execute it locally.

Another alternative for Localized Components is to structure it in such a way that retrieving and persisting the data is cheap, and processing it is done on locally. That way, the sharing of the data and the actual processing of the data are two distinct issues, which can be resolved separately. A common issue that needs to be resolved with Localized Components is consistency, if the component allow writes, how do other instances of the component, running on different machine get notified about it.

I tend to avoid Distributed Services in favor of Centralized Services and Localized Components, which tend to be easier to work with. It is also easier to lean on existing infrastructure then write an implementation from scratch. For example, I am using Rhino DHT (which I consider to be a Distributed Service) to handle a lot of the complexity inherit to one of those.

Macto, a module features spec sheet for authentication


imageI am going to talk about a few ways of trying to organize a project, mostly as a way to lay out the high level requirements for a feature or a module.

I consider filling in one of those to be the act of actually sitting down and thinking about the concrete design of the system. It is not final design, and it is not set in stone, but in general it forces me to think about things in a structured way.

It is not a very hard to do, so let us try to do this for the authentication part of the application. Authentication itself is a fairly simple task. In real corporate environment, I’ll probably need integration with Active Directory, but I think that we can do with simple username & pass in the sample.

Module: Authentication

Tasks: Authenticate users using name/pass combination

Integration: Publish notifications for changes to users

Scaling constraints:

Up to 100,000 users, with several million authentication calls per day.

Physical layout:

Since the system need to handle small amount of users, we have to separate deployment options, Centralized Service* and Localized Component*. Both options are going to be developed, to show both options.

Feature

Description

SLA

authenticate user

Based on name & password
Lock a user if after 5 failed logins

less than 50 ms per authentication request 99.9% of the time, for 100 requests per second per server

create new user

User name, password, email

less than 250 ms per change password request 99.9% of the time, for 10 requests per second globally

change password

 

less than 250 ms per change password request 99.9% of the time, for 10 requests per second globally

reset password

 

less than 250 ms per change password request 99.9% of the time, for 10 requests per second globally

enable / disable user

disable / enable the option to login to the system

less than 250 ms per change password request 99.9% of the time, for 10 requests per second globally

You should note that while I don’t expect to have that many users in the system, or have to handle that load, for the purpose of the sample, I think it would be interesting to see how to deal with such requirements.

The implications of this spec sheet is that the system can handle about 8.5 million authentication requests per day, and about a 3/4 of a million user modifications requests.

There are a few important things to observe about the spec sheet. It is extremely high level, it provide no actual implementation semantics but it does provide a few key data items. First, we know what the expected data size and load are. Second, we know what the SLAs for those are.

* Centralized Service & Localized Component are two topics that I’ll talk about in the future.

What is maintainable?


Frans Bouma left the following in this post, which I found quite interesting:

The person who spend a lot of time inside the code base obviously finds it logical and knows his way through the codebase and where to make changes what to avoid etc. For that person, the codebase is maintainable. It is different for a person who's new to the codebase. If that person has to spend a lot of time figuring out where what is and most importantly: why, it's already a sign things aren't as maintainable as it should.

I have to say that I disagree quite strongly with this definition.

Maintainable is a value that can only be applied by someone who is familiar with the codebase. If that someone find it hard to work on the codebase, it is hard to maintain. If someone with no knowledge of a codebase find it hard to work with it, tough luck, but that doesn’t say anything about the maintainability of a code base.

A codebase is more than just a code sample, it is conventions, ways of doing things, overall architecture and a sense of how to do things.

Now, there is a separate value, related to how easy it is to bring someone up to speed on the codebase, but that does not relate directly to maintainability.

A short note about NHibernate and Silverlight


I got a few questions about NHibernate and Silverlight. That is actually a very easy thing to answer.

Don’t even try. They don’t get along. In fact, they aren’t even going to get along.

Silverlight doesn’t have System.Data.IDbConnection, and you can safely assume that that it somewhat important to NHibernate.

So, running NHibernate inside a Silverlight application, presumably in order to access a local database is out. But I don’t think that this is what most people actually had in mind when they ask about NHibernate and Silverlight. They want to know about NHibernate on the server and Silverlight on the client.

And that is easy enough to answer as well, it is going to work just like any client / server system. All the same rules apply.

Macto, or How To Build a Prison


image

The sample application that I am going to build is going to be a prison management application. I am going to take this post as a chance to talk about it a bit, discuss the domain and then I’ll talk about the overall architecture in more details.

The domain of a prison is actually fairly simple, you have an inmate, and the sole requirement is that you would keep him (it tend to be overwhelmingly him, rather than her) in lawful custody.

The term lawful custody has a lot of implications, which are, in more or less their order of importance:

  • The inmate is in custody, that is, he didn’t manage to run away.
  • Custody is lawful, that is, you have legal authorization to keep him in jail. Usually that means an order by a judge, or for the first 24 hours, by a police officer.
  • Lawful custody itself means that you:
    • keep the inmate fed
    • in reasonable conditions (sleeping quarters, sanitation, space)
    • access to medical facilities. Indeed, in most prisons the inmates get better health care, especially for emergencies, than the people living in most big cities.
    • ability to communicate with lawyers and family

The devil, however, is in the details. I am pretty sure that I could sit down and write about 250 pages of high level spec for things that are absolutely required for a system that run a prison and still not get everything right.

In practice, at least in the prisons I served at, we did stuff using paper, some VB6 apps & Access, and in one memorable occasion, an entire set of small prisons where running on what amounted to a full blown application written using Excel macros.

Anyway, what I think that I’ll do is start with a few modules in the system, not try to build a full blown system.

The modules that I‘ll start with would be:

  • Staff – Managing the prison’s staff. This is mostly for authentication & authorization for now.
  • Roster – Managing the roster of the prisoners, recording Countings, etc.
  • Legal – Managing the legal side of the prisoners, ensuring that there are authorizations for all the inmates, court dates, notifications, etc.
  • Escort – Responsible for actually taking the inmates out for court, medical evacs, releasing inmates, etc.

That is enough for now, for that matter, it is a huge workload already, but that is about the only way in which I can actually have a chance to show a big enough system and the interactions between all the parts.

Unethical behavior


Ben is pointing out something that I find flat out infuriating, a TFS MVP had removed a comment talking about SVN & Git from his blog with the following explanation:

“No offense, but I deleted your comment.  I make way too much $$ on Team System training & consulting to go publicly plugging alternative options.”

I am… disappointed. I started to write shocked, but it is not the first time that I have seen stuff like that happen.

What bothers me even more, if you can’t deal with critics on something that you are doing for a living, how can you call yourself an consultant in the field? How can you actually point out the options if you refuse to even look at them or engage in conversation about them.

This make me seriously doubt the professionalism of the person in question, to say the least.

image

Independent expert? I don’t think so.

Real world answers? Hardly.

Update: Ben Day has posted a reply about this.

Pitfalls


Mike Scott has pointed out a bug in this code, relating to the use of DateTime.Now vs. DateTime.UtcNow, which can cause inaccurate wait durations during daylight change time.

That made me think for a bit, there are things that just naturally make me highly suspicious, because they are a common source for bugs.

Using a WCF service in a using block, creating a session factory, a foreach loop with database calls, a select without a limit clause.

What are the things that cause a flashing neon sign to go through your head when you see them in code? The stuff that you know will have to be re-read twice to make sure it is bug free.

FUTURE POSTS

No future posts left, oh my!

RECENT SERIES

  1. RavenDB Sharding (2):
    21 May 2015 - Adding a new shard to an existing cluster, the easy way
  2. The RavenDB Comic Strip (2):
    20 May 2015 - Part II – a team in trouble!
  3. Challenge (45):
    28 Apr 2015 - What is the meaning of this change?
  4. Interview question (2):
    30 Mar 2015 - fix the index
  5. Excerpts from the RavenDB Performance team report (20):
    20 Feb 2015 - Optimizing Compare – The circle of life (a post-mortem)
View all series

RECENT COMMENTS

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats