Ayende @ Rahien

It's a girl

A vision of enterprise platform: Security Infrastructure

I have been asked how I would design a security infrastructure for my vision of an enterprise platform, and here is an initial draft of the ideas.

As anything in this series, no actual code was written down to build them. What I am doing is going through the steps that I would usually go before I actually sit down and implement something.

While most systems goes for the Users & Roles metaphor, I have found that this is rarely a valid approach in real enterprise scenarios. You often want to do more than just the users & roles, such as granting and revoking permissions from individuals, business logic based permissions, etc.

What are the requirements for this kind of an infrastructure?

  • Performant
  • Human understandable
  • Flexible
  • Ability to specify permissions using the following scheme:
    • On a
      • Group
      • Individual users
    • Based on
      • Entity Type
      • Specific Entity
      • Entity group

Let us give a few scenarios and then go over how we are going to solve them, shall we?

  1. A helpdesk representative can view account data, cannot edit it. The helpdesk representative also cannot view the account's projected revenue.
  2. Only managers can handle accounts marked as "Special Care"
  3. A team leader can handle all the cases handled by members in the team, team members can handle only their own cases.

The security infrastructure revolves around this interface:

image

The IsAllowed purpose should be clear, I believe, but let us talk a bit about the AddPermissionsToQuery part, shall we?

Once upon a time, I built a system that had a Security Service, that being a separate system running on a different machine. That meant that in order to find out if the user had permission to perform some action, I had to send the security service the entity type, id and the requested operation. This worked, but it was problematic when we wanted to display the user more than a single entity at a time. Because the system was external, we couldn't involve it in the query directly, which meant that we had to send the entire result set to the external service for filtering. Beyond the performance issue, there is another big problem, we had no way to reliability perform paged queries, the service could decide to chop up 50% of the returned results, and we would need to compensate for that somehow. That wasn't fun, let me tell you that.

So, the next application that I built, I used a different approach. Instead of an external security service, I had an internal one, and I could send all my queries through it. The security service would enhance the query so permissions would be observed, and everything just worked. It was very good to observe. In that case, we had a lot of methods that did it, because we had a custom security infrastructure. In this case, I think we can get away with a single AddPermissionsToQuery method, since the security infrastructure in place is standardize.

Now, why do we have a Why method there? Pretty strange method, that one, no?

Well, yes, it is. But this is also something that came up through painful experience. In any security system of significant complexity, you would have to ask yourself questions such as: "Why does this user see this information" and "Why can't I see this information" ?

I remember once getting a Priority Bug that some users were not seeing information that they should see, and I sat there and looked at it, and couldn't figure out how they got to that point. After we gave up understanding on our own, we started debugging it, and we reproduced the "error" on our machines. After stepping through it for ten or twenty times, it suddenly hit me, the system was doing exactly what it was supposed to do. I stepped over the line that did it in each and every one of the times that I debugged it, but I never noticed it.

You really want transparency in such a system, because "Access Denied" is about the second most annoying error to debug, if the system will give you no further information.

Now, I am going to show you the table structure, this is not fixed in stone, and don't try to read too much into seeing a table model here. It simply make it easier to follow the connections that a class diagram would.

image

Let us go over some of the concepts that we have here, shall we?

Users & Groups should be immediately obvious, let us focus for a moment on the Operations and Permissions. What is an operation? Operation is an action that can happen in the application. Examples of operations are:

  • Account.View
  • Account.Edit
  • Account.ProjectedRevenue.View
  • Account.ProjectedRevenue.Edit
  • Account.Assign
  • Account.SendEmail

As you can see, we have a fairly simple convention here. [Entity].[Action] and [Entity].[Field].[Action], this allows me to specify granular permissions in a very easy to grok fashion. The above mentioned operations are entity-based operations, they operate on a single entity instance at a time. We also have feature-based operations, such as:

  • Features.HelpDesk
  • Features.CustomerPortal

Those operate without an object to verify on, and are a way to turn on/off permissions for an entire section of the application. Since some operations are naturally grouped together, we also have relations between operations, so we will have the "Account" operation, which will include the "Account.Edit", "Account.View" as children. If you are granted the Account operation on an entity, you automatically get the "Account.Edit" and "Account.View" on the entity as well.

This makes the design somewhat more awkward, because now we need to go through two levels of operations to find the correct one, but it is not a big deal, since we are going to be smart about how we do it.

Permissions are the set of allowed / revoked permissions for an operation on an EntitySecurityKey (will be immediately explained) which is associated with Group, User or EntityGroup.

A simple example may be something like:

  • For User "Ayende", Allow "Account" on the "Account Entity" EntitySecurityKey, Importance 1
  • For Group "Managers", Revoke "Case.Edit" on "Case Entity" EntitySecurityKey, Importance 1
  • For Group "Users", Revoke "Account.Edit" on "Important Accounts Entity Group" EntitySecurityKey, Importance 1
  • For Group "Managers", Allow "Account.Edit" on "Important Accounts Entity Group" EntitySecurityKey, Importance 10
  • For User "Bob from Northwind", Revoke "Account" on "Northwind Account"  EntitySecurityKey, Importance 1

The algorithm for IsAllowed(account, "Account.Edit", user) is something like this, get all the operations relevant to the current entity, default to deny access, then check operations. Revoke operation gets a +1, so it is more important than an Allow operation in the same level. Or in pseudo code (ie, doesn't really handle all the complexity involved):

bool isAllowed = false;
int isAllowedImportance = 0;
foreach(Operation operation in GetAllOperationsForUser(user, operationName, entity.EntitySecurityKey))
{
	bool importance = operation.Importance;
	if(operation.Allow == false)
		importance + 1; 
	if ( isAllowedimportance <  )
	{
		isAllowed = operation.Allow;
		isAllowedimportance = operation.Importance;
	}
}
return isAllowed;

As you had probably noticed already, we have the notion of an Entity Security Key, what is that?

Well, when you define an entity you also need to define its default security, this way, you can specify who can view and edit it. Then, we we create an entity, its EntitySecurityKey is copied from the default one. If we want to set special permissions on a specific entity, we will create a copy of all the current permissions on the entity type, and then edit that, under a different EntitySecurityKey, which is related to its parent.

All the operations in the child EntitySecurityKey are automatically more important then the ones in the parent EntitySecurityKey, regardless of the important score that the parent operations has.

In addition to all of that, we also have the concept of an EntityGroup to consider. Permissions can be granted and revoked on an Entity Group, and those are applicable to all the entities that are member in this group. This way, business logic that touches permissions doesn't need to be scattered all over the place, when a state change affects the permissions on an entity, it is added or removed to an entity group, which has a well known operations defined on it.

Now that you probably understand the overall idea, let us talk about what problem do we have with this approach.

Performance

The security scheme is complex, and of the top of my head, given all the variables, I can't really think of a single query that will answer it for me. The solution for that, like in all things, it to not solve the complex problem, but to break it down to easier problems.

The first thing that we want to consider is what kind of question are we asking the security system. Right now, I am thinking that the IsAllowed method should have the following signatures:

public bool IsAllowed(Operation, User, Entity);
public bool IsAllowed(Operation, User);

This means that the question that we will always ask is "Does 'User' have 'Operation' on 'Entity'?", and "Does 'User' have 'Operation'?". The last is applicable for feature based operations only, of course.

So, given that this is the question we have, how can we answer this efficiently? Let us try to take the above mentioned table structure and de-normalize it to make queries more efficient. My first attempt is this:

image

This allows you to very easily query by the above semantics, and get all the required information in a single go.

A lot of the rules that I have previously mentioned will already be calculated in advance when we write to this table, so we have a far simpler scenario when we come to check the actual permissions.

For instance, the EntitySecurityKey that we send is always the one on the Entity, so the DenormalizedPermissions table will always have the permissions from the parent EntitySecurityKey copied with pre calculated values.

Since everything is based around the EntitySecurityKey, we also have a very simple time when it comes to updating this table.

All we need to do it rebuilt the permissions for this particular EntitySecurityKey.

This makes things much easier, all around.

 

Querying

What this means, in turn, is that we have the following query to issue when we come to check permissions:

SELECT dp.Allow, dp.Importance FROM DenormalizedPermission dp
WHERE       dp.EntitySecurityKey = :EntitySecurityKey
AND         dp.Operation = :Operation
AND         (dp.User = :User OR dp.Group IN (@UserGroups)
                  OR EntityGroup IN (@EntityGroups) )

All we need to do before the query is to find out all the groups that the user belongs to, directly or indirectly, and all the Entity Groups that the entity belongs to.

When it comes down to check a feature-base operation, we can issue the same query, sans the EntitySecurityKey, and we are done.

Another important consideration is the ability to cache this sort of query. Since we will probably make a lot of those, and since we are probably also going to want to have immediate response to changes in security, caching is important, and write-through caching layer can do wonder for making this optimized.

What is missing

Just to note: this is not complete, I can think of several scenarios that this has no answer for, from the Owner can do things other cannot to supporting permissions if the organization unit is identical for the entity and the user. However, adding those is fairly easy to build within the system, all we need to do is define an action that would add the owner's permissions explicitly to the entity, and remove it when they are changed. The same can be done for entities in an organization unit, you would have the group of users in Organization Unit Foo and the Entity Group of entities in Organization Unit Foo, which will have a permission set for that group.

Final thoughts

This turned out to be quite a bit longer than anticipated, waiting expectantly for you, dear reader, to tell me how grossly off I am.

Next topics:

  • Hot deployments and distributed deployments
  • A database that doesn't make you cry
  • Supporting upgrades
  • Platform I/O - integration with the rest of the enterprise

Comments

Thomas Krause
11/17/2007 06:17 PM by
Thomas Krause

"If we want to set special permissions on a specific entity, we will create a copy of all the current permissions on the entity type, and then edit that, under a different EntitySecurityKey"

This means that changes to the permissions on an EntityType after copying will no longer affect the specific entity. Why this may be useful in some situations, I believe that in most cases you want to use the default entity type permissions (even when they change) and only add or change a few of them. So why not use some sort of dynamic inheritance instead, like you can do in the NTFS file system.

Also have you thought about merging the EntitySecurityKey and the EntityGroup concept together?

They both define a set of one or more entities and since you have to determine all entity groups an entity belongs to anyway, it wouldn't hurt performace that much, would it?

Also, since entity groups are not mutually exclusive like EntitySecurityKeys are, you get some form of permission inheritance thrown in for free...

Anyway, this is really a great series so far, so keep the posts coming!

Marco
11/17/2007 07:16 PM by
Marco

Thanks for this well explained security layer! I'm very interested to see how you would implement this with real source-code, especially how to handle the DenormalizedPermissions with write-through caching layer

Another idea for your next topics:

  • Implement a plugin system for the UserInterface (and/or database to allow db changes for each client)

Thanks

Marco

Scott Bellware
11/17/2007 07:31 PM by
Scott Bellware

Cool. Some similar ideas to what we do in our Rails app.

Ayende Rahien
11/17/2007 08:01 PM by
Ayende Rahien

Thomas,

Did you notice that the EntitySecurityKey has a Parent? That is where the parent goes, so you do get dynamic inheritence of those.

Thomas Krause
11/17/2007 08:12 PM by
Thomas Krause

Yeah I noticed, but since your were speaking of copying the permissions when a new EntitySecurityKey is created and your query at the end only selects based on a single EntitySecurityKey, I thought you were just tracking the parent key for some other purpose.

Thanks for clarifying this.

Any thoughts about merging EntitySecurityKeys and EntityGroups?

Ayende Rahien
11/17/2007 09:12 PM by
Ayende Rahien

I think that I meant copying them to the denormalized table.

About the merge, I don't think so, they have distinct resposabilities.

An Entity Group is a way to specify permissions for a set of entities, while the key is the set of permissions on a specific entity

Mark
11/17/2007 09:58 PM by
Mark

wow nice write up.

I dont suppose you would be interested in providing a sample project and db to demonstrate all this? It would be great to be able to see this working, and help understand it better.

I think entity security in an application is one of those hardly mentioned topics some many dont know the true way of implementing it.

Steve Campbell
11/17/2007 10:10 PM by
Steve Campbell

I think your model is severely flawed (due to being entity-based).

It is better to define security very close to the views and controllers, rather than the model (entities). This is mostly because that is how the user wants to define security. They may legitimately have reason to access to an entity from one screen and not another, or during one process but not another.

Ayende Rahien
11/17/2007 10:44 PM by
Ayende Rahien

Steve,

Account.SomeOperation

Account.AnotherOperation

You define another operation for that.

Think in terms of operations on data, not in terms of whatever it is that allows it to do it.

KnaveT
11/18/2007 06:42 AM by
KnaveT

As a suggestion, you can make use of Enum flags to combine these operations into a single line.

e.g.

public enum AccountOps //values must be multiples of 2.

{

CanView = 1,

CanEdit = 2,

CanViewSpecialData = 4,

CanViewTotalRevenue = 8

...

}

And when storing the configurations as one, you only need to combine them.

AccountOps ops = AccountOps.CanView | AccountOps.CanEdit;

When persisting into your entity, just use an Int32.

When checking for permission, you can just take out and compare them like this:

bool canEdit = false;

AccountOps ops = AccountPermissionController.Get( myUser );

if( (ops & AccountOps.CanEdit ) > 0 )

canEdit = true;

return canEdit;

Shadi Mari
11/18/2007 10:47 AM by
Shadi Mari

Knavet,

If i'm not mistaken, depending on the operation name is more maintainable than having an Enum inside your code, where you need to update everytime you add a new method to the entity in question. Besides that you need to maintain a seperate enum per entity. In addition, implementing an admin console is easier through reflection and custom attributes.

Just thoughts.

Shadi Mari
11/18/2007 10:56 AM by
Shadi Mari

Ayende,

Excellent post. I'm just thinking loudly on how you can support partial entity view permissions. In other words, one user can view certain entity information that others can't in cases where you have your entity properties on one screen. Is it by having the entity property name in the database and you filter your results according to it.

Also, what would be the case in screens where you have more than one entity.

KnaveT
11/18/2007 03:17 PM by
KnaveT

Shadi,

I concur with the maintenance issue. However since the permissions module will be very heavily stressed, we often need to consider performance factors into it. Having 1 db row versus 6-7 rows per user to store a single module/entity's permission will definitely give you better performance during read/write operations.

But this is just the easy part. In each of our modules, we often execute check permissions code for that particular function. But when coding for the UI layer, you will often need to pull out majority of the permissions data, i.e. you should not see that delete button if you're not suppose to see it.

Even with caching applied that will take a very heavy toll on the overall system response.

Just my two cents.

KnaveT

comlin
11/18/2007 05:12 PM by
comlin

@Ayende, What would you say if I give you a real world scenario:

  1. Server running Microsoft Windows hosts a web site which includes your security features.

  2. Separate server hosts database.

Well, common thing isn't it?

We are pretty sure that the first server(it's on the Internet) would be hacked sooner or later and the hacker will acquire admin privileges. And all we have is your "entity security" inside the BL, who needs it anyway, you will take the connection string and perform all the malicious actions inside the database!

I see two ways to handle this situation: developing an app server to process queries or building security infrastructure inside DB using stored procedures or row-level security. What do you think?

Ayende Rahien
11/18/2007 05:16 PM by
Ayende Rahien

Knave, the enum is an issue for maintainability, as already mentioned, but it also suffers from another problem, the inability of a user to define their own operations and extend the system.

If performance is a goal, there are very few things that can beat caching. Especially if you want to cache just the result, not the way.

So you can cache if I can look at an account, for instance.

Ayende Rahien
11/18/2007 05:18 PM by
Ayende Rahien

Shadi,

Partial views are already handled in this scheme, the UI needs to check for operations such as Account.ExpectedRevenue.View and use this information to display or hide fields.

Ayende Rahien
11/18/2007 05:32 PM by
Ayende Rahien

comlin,

Well, how are he going to get the connection string, first of all?

He would need to be able to break DPAPI, since they are encrypted using it.

But even assuming a nefarious person managed to get to this level, why the hell am I trying to close the barn door after the horse went away.

I'll have a full post about it shortly.

Luke Breuer
11/18/2007 07:05 PM by
Luke Breuer

Your Why() method is, sadly, brilliant. I understand the need to not output sensitive information which could make it into a UI element or log on a client's machine, but simply doing nothing is a terrible choice that has cost software developers and IT pros countless hours of unnecessary frustration.

One thing you do not take into account is what to show the user if he/she is not allowed to view particular data. Often the answer is "nothing" or "hide the field", but sometimes you might want to put "***" or something like that. One could argue that this is information for the view, but it would be good to be able to see security information closely linked with what the user sees, at least in the security admin UI. This might even tie into your Why() method above -- it would be good to have technical reasons and business reasons for why something cannot happen. Perhaps this would have helped you in the story you mentioned, where the code was doing exactly what it wanted to do, something that was in contrast with the business needs.

... all we need to do is define an action that would add the owner's permissions explicitly to the entity ...

This bothers me. It seems you would need a potentially very large number of custom EntitySecurityKeys, as I see no way to attach permissions without it. If one thousand users can do special things to items they own in a hundred different tables, that's up to 100,000 records that are potentially very simple calculations. Let's say that users can, on average, see 10 more columns on records they own. Now we're up to a million security records. Perhaps I'm overestimating by an order of magnitude, but the idea of having so much duplication bugs me.

Ayende Rahien
11/19/2007 02:56 AM by
Ayende Rahien

Luke,

My main concern here is the simplicity of the model, so I want to have as few special cases as possible. I don't think that you are an order of magnitude higher, no. It is entirely possible to have million or more security records. The nice thing about it is that we don't care.

DB are really fast in making sense of all that data, especially with predictable queries and good indexes, and caching on top of that should make it easier still.

Bob
11/19/2007 02:22 PM by
Bob

Ayende, thanks for all your great information....quick question for you: Can you explain the "Importance" attribute, and what your using it for? And I echo Marco's comment that a sample/demo app would be much appreciated!

Ayende Rahien
11/19/2007 05:07 PM by
Ayende Rahien

Bob,

Importance is for deciding whatever to allow or revoke base on several permissions.

Check the algorithm for that

Luke Breuer
11/19/2007 07:00 PM by
Luke Breuer

My worry wasn't so much about performance as duplication. It's like having a clunky lookup table instead of an elegant equation. Wouldn't things get complicated if you decided to change the rules of ownership after multiple EntitySecurityKeys originally created for ownership rules got their rules modified for reasons other than ownership rules? Getting the two rule sets (the one-off vs. the equation-based) mixed could get messy.

Ayende Rahien
11/19/2007 07:16 PM by
Ayende Rahien

The problem is that the equation requires changing frequently, while the lookup table is a more solid approach to changes.

Luke Breuer
11/19/2007 07:38 PM by
Luke Breuer

Can you elaborate on that? If the equation changes, the lookup table needs to change as well...

Ayende Rahien
11/19/2007 07:48 PM by
Ayende Rahien

Changing a lookup table is easy, it is just data.

Changing the equation means changing code and has further implications

Ayende Rahien
11/19/2007 07:49 PM by
Ayende Rahien

In other words, if I have a solution that I can implement externally vs. solution that I need to mess around with the code with, I would rather go with the first.

This way, there is only a single set of rules, and the only thing you need to do is operate on the data.

This make it simple to build things like Why(), and it make it simple to understand what is going on.

Ayende Rahien
11/19/2007 07:49 PM by
Ayende Rahien

In other words, if I have a solution that I can implement externally vs. solution that I need to mess around with the code with, I would rather go with the first.

This way, there is only a single set of rules, and the only thing you need to do is operate on the data.

This make it simple to build things like Why(), and it make it simple to understand what is going on.

Luke Breuer
11/19/2007 09:05 PM by
Luke Breuer

Let's say we need to change ownership rules. We first need to find all the EntitySecurityKeys that were created for the old rule. Then we test to ensure that all the security info created from the old rule exists for all these keys (otherwise, throw an exception). Then we find any additional security info linked to these EntitySecurityKeys, which was used to augment the rule-based security info. We either store this info in X, or throw an exception if any such info is found. Then we wipe out all those EntitySecurityKeys, create new ones based on the new ownership rule, and optionally merge X into these new keys. Does this make sense? If so, are you happy with this approach?

ParanoidPenguin
11/20/2007 11:25 AM by
ParanoidPenguin

Great article - thanks! Sorry if this is a poor question, but how would you deal with scenarios like certain users only being able to view customers over a certain age threshold? Would you use the EntityGroup?

Ayende Rahien
11/20/2007 11:26 AM by
Ayende Rahien

ParanoidPenguin,

Yes, EntityGroup is the way to go here.

Comments have been closed on this topic.