Oren Eini

CEO of RavenDB

a NoSQL Open Source Document Database

Get in touch with me:

oren@ravendb.net +972 52-548-6969

Posts: 7,597
|
Comments: 51,224
Privacy Policy · Terms
filter by tags archive
time to read 10 min | 1850 words

I posted a riddle about how to use Active Record as a Rule Engine, here is the answer.

First, just to remind you of the scenario:

A rule engine that can process big amount of data. The engine will run as a Windows service, and not as a part of another application.
Throughput is important, as well as flexibility in the rules and what they can do. The processing of each rule can be quite complex, so it is important that the rule engine will not limit what the rule can do. The rules run on varied pieces of data, which reside in a database.

Okay, so we have a database and we have a lot of rules. Let us consider first what a rule is, shall we?

A rule is usually composed of two distinct parts:

  1. What this rule applies to?
  2. What this rule should do?

For instnace, consider this rule:

Increase the salary by 10% of all employees that have over 5 years senority.
  • What this rule applies to: Employee with over 5 years senority.
  • What this rule should do: Increase salary by 10%

Do you see where I am heading? Let us first look at the object model first:

RuleDiagram

I have got a Rule<T> class, which manages the rules. It has either an HQL Query which select the objects to works on, or a set of predicates that it uses.

The predicates are very thin wrappers over NHibernate.Expression namespace, and they mainly provide persistance support. The reason that I am using both is that is it very easy to provide a nice drag & drop interface for the predicates, which allow a business analyst to define the set of objects that this rule will run on. The HQL Query is for the more complex issues, where a developer will be needed.

I hope you noticed the IRuleAction<T> interface. This allows to write actions that accept an object, and do something to it. This can be something like Increase Salary, Fire Employee, etc.

Note that both the Rule<T> and the IRuleAction<T> has a collection of parameters. Those are strongly typed parameters. Which means that I don't deal with hash tables, or things like that. I get a strongly typed object that I can use in compile time.

For reference, here is the table structure that backs this up:

(Image from clipboard).png

How this is working? Check out the Rule<T>.Execute() method:

public void Execute()

{

       using (new SessionScope())

       {

              T[] items = GetItemsToWorkOnFromQueryOrPredicates();

              foreach (IRuleAction<T> action in Actions)

              {

                     foreach (T item in items)

                     {

                           action.Act(item);

                     }

              }

       }// all changes to the items are persisted here.

}

And here is how an action looks like (I omitted the uninteresting properties):

public class IncreaseEmployeeSalaryAction : IRuleAction<Employee>

{

      

       public void Act(Employee item)

       {

              item.Salary.IncreaseBy(Parameter.Amount);

       }

}

NHibernate can handle generic types quite well (but not generic type inferencing, thanks goodness), but it require the full generic type name, which can be a pain to write. Active Record does even that for me, so it is very easy to work with the system.

Hope you like the idea. An additional extention of this can be further processing of complex business rules that cannot be expressed in an HQL Query alone.

Update: Just to clarify, NHibernate is able to tell based on the XxxType column in each table which object it is, and it construct the object graph accordingly.

time to read 12 min | 2343 words

I am using the code below to run work in multiply threads. A client registers a callback and an input, and it is executed on the thread pool.

I have some notes about this code below:

public class WorkManager<T> : IWorkManager<T>

{

       static ILog logger = LogManager.GetLogger(typeof(WorkManager<T>));

 

       /// <summary>

       /// I hate XML Comments, can't use generics in them.

       /// </summary>

       /// <param name="state">State is KeyValuePair&lt;Action&lt;T&gt;,T&gt;</param>

       private void ExecuteWorkThreaded(object state)

       {

              KeyValuePair<Action<T>, T> pair = (KeyValuePair<Action<T>, T>)state;

              Action<T> action = pair.Key;

              logger.DebugFormat("Executing action: {0}.{1}", action.Method.DeclaringType, action.Method.Name);

              // this should prevent parallel execution of the same method from the same object

              // and help prevent re-entracy issues.

              lock (action)

              {

                     try

                     {

                           action(pair.Value);

                     }

                     catch (Exception e)

                     {

                           logger.Error(string.Format("Exception when executing work on {0}.{1}",
                                 action.Method.DeclaringType, action.Method.Name), e);

                           EventHandler<EventArgs<Exception>> temp = ExceptionRaised;

                           if (temp != null)

                                  temp(this, new EventArgs<Exception>(e));

                     }

              }

       }

 

 

       public void ExecuteWork(Action<T> action, T item)

       {

              KeyValuePair<Action<T>, T> pair = new KeyValuePair<Action<T>, T>(action, item);

              AbortableThreadPool.QueueUserWorkItem(ExecuteWorkThreaded, pair);

 

       }

 

       public event EventHandler<EventArgs<Exception>> ExceptionRaised;

}

I am using the command patten here, where an object perform a single task, so I don't run into issues with two threads running actions from the same object. The object may be registered twice, but that is okay, we are serializing access to the action, so it wouldn't run in parallel. The only requirement that I have from my actions is that they can be called more than once, which is a lot less than require them to be thread safe.

The whole idea here is to save threads. I don't want my threads to be busy waiting for something, and I do want centralized control over how they are executed and why. As I said, the logging is extremely important to me, but it also something that I would like to free the actions from (except action related logging, of course.

time to read 2 min | 354 words

After reading this, I was reminded how much I want the null method operator. Here is the scenario:

string name = user != null ? user.Name : "no-name";

The Null Method Operator work like this:

string name = user.Name ?! "no-name";

The rules:

  • The first object reference is checked, and if it is null, it skips the method call and goes to the other line.
  • Only the first object reference is checked. Chained method calls are not supported, this should throw an NullReferenceError, as usual.
    string name = user.ReturnNull.ToString() ?! "no-name";
  • You can chain the calls, like this:
    string name = user.Name ?! Context.User.Identity.Name ?! "really-no-name";

I really can't express how much I would like this feature. And have the null coalesing operator is like a thorn in my foot, everytime I think about it, I get annoyed that I don't hve the Null Method Operator.

Feel free to come with a better name.

time to read 3 min | 460 words

I mentioned that it can take me quite a bit of code to read a file. After carefully consulting the documentation, I have come to realize that INT21 calls are not necessary in .Net. A bit amazed, I took a look at the code, which grew in leaps and bounds and is now self sustaining, according to my research.

I don’t believe in UML, but I do believe in pretty pictures, so I put together the following picture.

 

 

 

I left out several pieces, like the file parser, to focus on the important issue, which is reading a file. The big fun was to listen to a list of requirements and say “Yes, I can do that”, “Yes, should be easy”, “Already there”, etc. Building the configuration is a bit like playing puzzles, each part of the application is truly separated.  

You get nice configuration bugs this way. I accidently put the mark done action before the parse file action, and couldn’t understand why it wasn’t working. J

This is the first project in a long time where I did not use NHibernate. I now regret this position bitterly, ADO.Net is just a PITA to work with. The amount of repeating code that you need to write is amazing, the amount of strings is also frightening. I’ll probably put NHibernate there anyway, since that would significantly reduce the amount of stuff that I need to do.  I am not sure that this is an overkill or not. Right now I start to get into the data access for the application itself, and it looks like it will be needed anyway.

There is only so much datasets that I can stand.

 

 Anyway, most of those pieces were developed TDD`ed and separately; I had major worries about integration, but it passed rather smoothly. I love Windsor, and in this case, it is the application. The above should be the simplified live object graph, as it was generated by Windsor. Call a single method on the coordinator, and the system starts rolling. Input comes in, and the system responds, often in fairly complex ways.

The only output that I have at the moment is log4net output, which is a great help in general, but thought me quite a bit about what is considered good logging.

“Executing action” is generally not a good log message.

 

By the way, anyone knows of a good way to draw complex object graphs?  The example above is nice, but it is one dimensional and show just a single aspect of the object graph. The case in point, under both the clock and the file watcher, I’ve many sub trees, and it is just not possible to get them all there in the  structure above. You would lose sight of the forest from all the trees. J

Elegant

time to read 1 min | 84 words

I would let this piece of code to stand on its own. I like it.

 

UPDATE TOP (100) Messages

SET Status = 'Sending'

OUTPUT Inserted.*

WHERE Status  IN (

      'New',

      'Sending') -- we previously failed to send this message

 

time to read 4 min | 729 words

You write code like this:

public bool IsInPeriod(DateTime dateTime)
{
    bool withinHoursRange = startHour >= dateTime.Hour && dateTime.Hour <= endHour;
    bool matchDayInMonth = dateTime.Day == ( dayInMonth ?? dateTime.Day);
    bool matchDayOfWeek = dateTime.DayOfWeek == ( dayOfWeek ?? dateTime.DayOfWeek);
    bool matchMonth = dateTime.Month == ( month ?? dateTime.Month);
    return withinHoursRange &&
           matchDayInMonth &&
           matchDayOfWeek &&
           matchMonth;
}

Ten points for the first guy that will tell me why this is not good for DBs.

Update: Just to clarify, this is not SQL CLR code, it is an instance method in a business object

time to read 1 min | 93 words

It was too late in the night, and I debugged multi threaded code with ease.

It was too late in the night, and I struggled with regular expressions.

It was too late in the night, and I downloaded the Regulator.

It was too late in the night, and I found in two minutes that I forgot to add a #@^$@# (no, that is not the pattern) ? for the regex.

Thanks God for the Regulator.

And thanks to Roy.

time to read 1 min | 200 words

I mentioned in a previous post that I used Active Record as a Rule Engine, and a couple of people asked just how. Before I will post the answer, I want to see if you can come up with the same idea.

The scenario:

A rule engine that can process big amount of data. The engine will run as a Windows service, and not as a part of another application.
Throughput is important, as well as flexibility in the rules and what they can do. The processing of each rule can be quite complex, so it is important that the rule engine will not limit what the rule can do.

Hints:

  • A database is really good in filtering data.
  • Active Record (well, NHibernate, but it is the same thing) can take inheritance to the database level.
time to read 2 min | 214 words

It felt like I was posting a lots of posts lately, but checking the numbers, I didn't come close to scratching my daily record in the entire weekend, and I still have items in my blog scratch pad that I need to post (down to five items, though). Anyway, this post is about interfaces, not about my blogging habits.

I mentioned that it takes quite a bit of interfaces to write a file to database (8 interfaces and 15 classes, on last count). I found it useful to make all the interfaces inherit from IDisposable. There is quite a bit of composition there, and even if currently their implementor do not hold disposable resources, they will probably will.

I don't have a good defination for an acting interface, but I have a good defination for one that isn't. A non acting interface is one that can be used only to get information or to change the state of an object. An acting interface is one that can change the state of the system.

This allows me to handle resources in the top level class by simply disposing a single interface, which will pass the call to the rest of the system.

FUTURE POSTS

  1. Replacing developers with GPUs - 3 hours from now
  2. Memory optimizations to reduce CPU costs - 2 days from now
  3. AI's hidden state in the execution stack - 5 days from now

There are posts all the way to Aug 18, 2025

RECENT SERIES

  1. RavenDB 7.1 (7):
    11 Jul 2025 - The Gen AI release
  2. Production postmorterm (2):
    11 Jun 2025 - The rookie server's untimely promotion
  3. Webinar (7):
    05 Jun 2025 - Think inside the database
  4. Recording (16):
    29 May 2025 - RavenDB's Upcoming Optimizations Deep Dive
  5. RavenDB News (2):
    02 May 2025 - May 2025
View all series

Syndication

Main feed ... ...
Comments feed   ... ...
}