A few hours ago I finished writing Chapter 10. Looking at the time on the calendar, it didn't take too long, about a month. Looking at it from the point of view of effort involved, I feel about as tired as if it took three years.

Other chapters took longer, but didn't take out quite as much effort. Not sure what this means... but I am happy I am done with it (for a given value of done, I will still need to review and read it at least three dozen times).

Probably one of the hardest challenges that I am facing with writing the book is to know what to say and what to leave unsaid.

Phrasing it another way, it is choosing at what level to talk to the reader. On the one hand, I really want the reader to be able to make immediate use of the concepts that I am talking about, which drive me to do more practical demonstrations, code samples and covering more common situations. On the other hand, those take up a lot of room, and they tend to be boring if you don't need exactly what you need right this moment.

High level concepts, open ended possibilities and assuming a bit about the reader knowledge level makes for a book that is much more narrowly focused, and I think that it more valuable. However, it also tend to leave readers unsatisfied, because not everything is explained.

Currently I am writing a UI focused chapter, and to get a good experience from the UI you need to invest a lot of time. Metric tons of it. I am trying to chart the way and show how this can be done, but without getting mired in all the actual minute details.

This is a tough balancing act, and I am not sure if I am succeeding.

Going back to the theme of visually editing a DSL, we have something like this:

The backend to this cutey is pattern recognition with custom UI on top. You have to teach it about each pattern that you use with it, but you can get pretty smart with how it works while using a fairly brute force (and simple) techniques.

Thoughts?

One of the more annoying problems with  building rules that are also code is that you have to deal with code related issues. One of the more common ones is NullReferenceException.

For example, let us say that we have the following rule:

when Order.Amount > 10 and Customer.IsPreferred:
      ApplyDiscount 5.precent 

We also support a mode in which a customer can create an order without actually registering on the site (anonymous checkout).

In this scenario, the Customer property is null. We can rewrite the rule to look like this:

when Order.Amount > 10 and Customer is not null and Customer.IsPreferred:
      ApplyDiscount 5.precent 

But I think that this is extremely ugly. We can also decide to return a default instance of the customer when it is not there, but here I want to show you another way to handle this. We define the rule as invalid when Customer is not there, so it should not be run. The question is how we can know that.

The dirty way is to do something like this:

var referencesCustomer = File.ReadAllText(ruleName).Contains("Customer");
if(referencesCustomer && Customer == null)
   return;

If you gagged when seeing this code, that is a good sign. Let us solve this properly. First, we want some help from the compiler, so let us inspect the when() meta method that we have seen in the previous post a little closer.

[Meta]
public static ExpressionStatement when(Expression condition, BlockExpression action)
{
	var ctor = new BlockExpression();

	var conditionFunc = new Block();
	conditionFunc.Add(new ReturnStatement(condition));

	ctor.Body.Add(
		new BinaryExpression(
			BinaryOperatorType.Assign,
			new ReferenceExpression("Condition"),
			new BlockExpression(conditionFunc)
			)
		);

	Expression serialize = new CodeSerializer().Serialize(condition);
	Builder.Revisit(serialize);

	ctor.Body.Add(
		new BinaryExpression(
			BinaryOperatorType.Assign,
			new ReferenceExpression("ConditionExpression"),
			serialize
			)
		);

	ctor.Body.Add(
		new BinaryExpression(
			BinaryOperatorType.Assign,
			new ReferenceExpression("Action"),
			action
			)
		);

	return new ExpressionStatement(
		new MethodInvocationExpression(
			ctor
			));
}

We take the cal to the when method and transform it to the following code:

delegate
{
	Condition = () => Order.Amount > 10 && Customer.IsPreferred;
	ConditionExpression = (Expression<Func<bool>>)() => Order.Amount > 10 && Customer.IsPreferred;
	Action = delegate
	{
		// not interesting for this post
	};
}();

I am translating to C# 3.0 here in order to make it easier to grasp the concept. The real code is in Boo, of course, and is more interesting. The most fascinating concept here is the use of CodeSerializer, which will turn the condition that we passed into an AST that we can access at runtime. I tried to simulate that by doing an explicit cast to expression tree, which would give similar result in C#).

Having the AST of the code at runtime, even if we don't want to change it (a totally different concept) is incredibly powerful. In this case, we are going to use this to detect when we are referencing a null property and marking the rule as invalid.

Here is the code:

public void Evaluate()
{
	var references = new List<string>();
	new InlineVisitor
	{
		OnReferenceExpression = r => references.Add(r.Name);
	}.Visit(ConditionExpression);
	if(references.Contains("Customer") && Customer == null)
		return;// rule invalid
	if(Condition())
		Action();
}

This is a very simple example of how you can add smarts to the way that your code behaves. This technique is the foundation for a whole host of options. I am using similar approaches for adaptive rules and for auditable actions. Fun stuff, if I say so myself.

One of the most common problems with using Boo's Meta Methods is that they can only return expressions. This is not good if what you want to return from the meta method is a set of statements to be executed.

The most common reason for that is to initialize a set of variables. Obviously, you can call a method that will do it for you, but there is a simpler way.

Method invocation is an expression, and anonymous delegate definition is also an expression. What does this tells you? That this is an expression as well:

// the whole thing is a single expression.
delegate (int x)
{
	Console.WriteLine(x);
	Console.WriteLine("another statement");
}(5);

You generally won't use something like that in real code, but when you are working with the AST directly, expressions vs. statements require a wholly different world view.

Anyway, let us see how we can implement this using Boo.

[Meta]
public static ExpressionStatement when(Expression condition, BlockExpression action)
{
	var func = new BlockExpression();

	var conditionFunc = new Block();
	conditionFunc.Add(new ReturnStatement(condition));

	func .Body.Add(
		new BinaryExpression(
			BinaryOperatorType.Assign,
			new ReferenceExpression("Condition"),
			new BlockExpression(conditionFunc)
			)
		);

	Expression serialize = new CodeSerializer().Serialize(condition);
	RuleBuilder.Revisit(serialize);

	func .Body.Add(
		new BinaryExpression(
			BinaryOperatorType.Assign,
			new ReferenceExpression("ConditionExpression"),
			serialize
			)
		);

	func .Body.Add(
		new BinaryExpression(
			BinaryOperatorType.Assign,
			new ReferenceExpression("Action"),
			action
			)
		);

	return new MethodInvocationExpression(func);
}

This trick lets you use the meta method to return several statements, which allows to do several property assignments (something that you generally cannot do in a single expression). I'll go over the actual meaning of the code (rather than the mechanics) in a future post.

One of the most common chores when working with compilers is the need to create special visitors. I mean, I just need to get a list of all the variables in the code, but I need to create a visitor class and execute it in order to get the information out. This is not hard, the code is something like this:

public class ReferenceVisitor : DepthFirstVisitor
{
     public List<string> References = new List<string>();
 
     public void OnReferenceExpression(ReferenceExpression re)
     {
            References.Add(re.Name); 
     }
}
public bool IsCallingEmployeeProperty(Expression condition)
{ 
    var visitor = new ReferenceVisitor();
    visitor.Visit(condition);
    return visitor.References.Contains("Employee"); 
}

Doing this is just annoying. Especially when you have to create several of those, and they make no sense outside of their call site. In many ways, they are to compilers what event handlers are to UI components.

What would happen if we could create a special visitor inline, without going through the "create whole new type" crap? I think that this would be as valuable as anonymous delegates and lambdas turned out to be. With that in mind, let us see if I can make this work, shall we?

public bool IsCallingEmployeeProperty(Expression condition)
{
	var references = new List<string>();
	new InlineVisitor
	{
		OnRefefenceExpression = re => references.Add(re.Name) 
	}.Visit(condition);
	return references.Contains("Employee"); 
}

Especially in the more moderately complex scenarios, such a thing is extremely useful.

  So, Google is coming out with a new browsers, while at the same time they are also responsible for a large part of both FireFox and Opera's budgets.

I think that this is a very interesting development. In particular, because Google thinks about the browser as a complementary offer to what it does, or as a baseline platform, not as the actual end result.

This get really interesting when you think that in this scenario, Google can leverage what are effectively Killer Applications in order to migrate people from one browser to another. If YouTube worked better with Chrome than with IE, I think it would be a very powerful motivator to move. And Google has dozens of such high value assets.

I am pretty sure that Google will produce plugins for anything new it creates, so other browsers can work with it as well (to do otherwise is to risk monopoly charges), but if used properly, it will allow Google to leverage its own power to produce its own de facto standards, which browsers will have to follow.

The focus on creating a browser which is focused primarily on allowing application development and hosting (vs. mere browsing) is likely to aid in setting the new baseline standard of what a browser platform should provide for the web applications that are hosted in it.

From my point of view, I think that this is going to allow Google to take a much more active role in shaping the environment in which their applications are living. From that perspective, it seems like a very natural step for them.

I just finished reading Hibernate Search in Action, and I loved it. I should point out that I was the porter of Hibernate Search to NHibernate Search, so I had some previous expertise in the topic. In addition to that, I approached this book at an angle completely orthogonal to the expected audience. Unlike most "in Action" books, I did not intend to make immediate use of the code and approaches suggested in the book. Instead, I looked to the book as a way to deepen my understanding of the tool and how it works.

I am impressed, massively so, that it did so well in this regard for someone who has gone through the entire source code of the project several times.

I'll not bore you with the actual details, you can get the actual content summary of off the site. From my perspective, after reading this book I know that I am going to take a completely different approach for most complex search scenarios, and I think that I have the practical theoretical knowledge to deal with it.

I highly recommend the book if you actually need to deal with Hibernate Search, but I would recommend it to people who are not using it, because it contains some important eye opening concepts if you are not used to full text search tools capabilities. As a nice bonus, I was able to take the information in the book and use it to discuss a problem the customer was having, ending up with something that I consider far superior of the solution that they currently employ.

It is not out yet, and I reviewed a non final copy, but you can order the PDF right now, and just reading the freely available first chapter is valuable in itself.