I have just finished 2nd pass editing all the chapters and appendixes for my DSL book. This has been, I think, the 1,390th time that I had gone through the book since I finished writing it. Luckily for me, this appears to be the end, there is the final printer proofs stage, and then I’ll truly be free.

One of the things that I wasn’t told about when I agree to write the book is just how many times I would have to read it. I think that I am good writer (God knows that I get enough practice), but for crying out load, there is only so many times that I can read the same piece (especially if I wrote it) before it gets pretty boring.

Now, to be absolutely clear, the publisher is doing great job, and I think that each pass has been extremely valuable. We found problems in grammar, spelling, fuzzy speak, missing concepts, bugs in code, etc. This is important, valuable and necessary.

One of the things that some people have hard time understanding about the way that I think is that I can accept that some things are necessary even if they aren’t fun. That mean that I’ll do them, but I’ll complain about it. Just like taxes. I’ll pay them, but I reserve the right to bitch about it.

It took me a while to understand why I hate editing so much. I know how to handle boring work, and I know how to handle creative work. What I don’t know how to handle is boring creative work. I split my attention when I am doing boring stuff, usually having something in the background that it interesting while I am doing the boring stuff. For creative stuff, I usually enjoy it.

But boring creative stuff? I can’t split my attention then, and it is boring. It is annoying, and I do think about it like a tax.

Consider this statement, that was flagged by my editor:

The MacroMacro passes the MacroStatement that the compiler hands to the macro using the name of the macro

Can you figure out what I meant here? It took me a while to do that :-)

Editing the book has been a huge weight on my mind, having finished that is such a huge relief…

Building Domain Specific Languages with Boo

Another big milestone, early access subscribers can now read the entire book, all 13 chapters and both appendixes.

Not edited yet, but much closer to completion.

Jeffery Olson has just made the first release of Boo Lang Studio available.

This one comes with a "Yes, Dear" installer.

Yeah, we have Boo installed!

Let us create a new project:

And take a look at the code:

And intellisense works as well, whew!

Jeffery Olson and James Gregory: THANKS!

A few days ago I posted about Ugly Linq. Ever since then, I kept thinking about how ugly it is to handle this by hand. Suddenly, it hit me that I don't have to do it that way.

Boo already has the facilities to take a compiler AST and translate that into the code that would recreate this AST. In particular, this makes the code we previously had to write to this:

public class ConditionMacro : AbstractAstMacro
{
	public override Statement Expand(MacroStatement macro)
	{
		Expression serialize = new CodeSerializer().Serialize(macro.Arguments[0]);
		var body = new Block();
		body.Statements.Add(new ReturnStatement(macro.Arguments[0]));
		return new ExpressionStatement(
			new MethodInvocationExpression(
					AstUtil.CreateReferenceExpression(typeof(Condition).FullName),
					new BlockExpression(body),
					serialize
				)

			);
	}
}

And what that means is that given this code:

condition a > 10

We can get this result:

And that is it.

You get both the actual compiled expression and the AST that describes this. This is critically important because you can now take this piece of AST and do transformations / views on it.

And that is important if you want to have reliable graphical representation on top of a textual DSL, which is what my chapter 10 is going to cover.

Damn, this is simple! Thanks Rodrigo!

One of the things that always bothered me with Linq was that it is actually not an interesting idea from the compiler perspective. I just had to implement a very simple expression to expression tree converter, which only served to strengthen my opinion. Here is the (ugly, proof of concept, horrible) implementation, using the Boo AST:

private static Block Linqify(Expression expr)
{
	var block = new Block(expr.LexicalInfo);
	ReferenceExpression condition = AddCondition(block);
	Parse(block, condition, expr);
	block.Add(new ReturnStatement(condition));
	return block;
}

private static ReferenceExpression AddCondition(Block block)
{
	var condition = new MethodInvocationExpression(new ReferenceExpression("Condition"));
	var expression = new ReferenceExpression("condition_" + CompilerContext.Current.AllocIndex());
	block.Add(
		new BinaryExpression(
			BinaryOperatorType.Assign,
			expression,
			condition)
		);
	return expression;
}

private static void Parse(Block block, Expression condition, Expression expr)
{
	var be = expr as BinaryExpression;
	if (be != null && (be.Operator == BinaryOperatorType.Or || be.Operator == BinaryOperatorType.And))
	{
		block.Add(new BinaryExpression(
			BinaryOperatorType.Assign,
			new MemberReferenceExpression(condition, "Operator"),
			new StringLiteralExpression(be.Operator.ToString().ToLowerInvariant()))
			);
		ReferenceExpression left = AddCondition(block);
		block.Add(
			new MethodInvocationExpression(
				new MemberReferenceExpression(new MemberReferenceExpression(condition, "Expressions"), "Add"), left));
		ReferenceExpression right = AddCondition(block);
		block.Add(
			new MethodInvocationExpression(
				new MemberReferenceExpression(new MemberReferenceExpression(condition, "Expressions"), "Add"), right));
		Parse(block, left, be.Left);
		Parse(block, right, be.Right);
		return;
	}
	var fragment = new MethodInvocationExpression(new ReferenceExpression("Fragment"));
	if (expr is UnaryExpression)
	{
		fragment.NamedArguments.Add(
			new ExpressionPair(new ReferenceExpression("Modifier"),
				new StringLiteralExpression("not"))
			);
		be = (BinaryExpression)((UnaryExpression)expr).Operand;
	}
	var func1 = (MethodInvocationExpression)be.Left;
	fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Func1"), new StringLiteralExpression(func1.Target.ToString())));
	fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Prop1"), GetStringArgument(func1.Arguments[0])));
	var func2 = be.Right as MethodInvocationExpression;
	if (func2 != null)
	{
		fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Func2"), new StringLiteralExpression(func2.Target.ToString())));
		fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Prop2"), GetStringArgument(func2.Arguments[0])));
	}
	else
	{
		fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Func2"), new StringLiteralExpression("literal")));
		fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Prop2"), GetStringArgument(be.Right)));
	}
	fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Operator"),
		new StringLiteralExpression(GetOperator(be))));

	block.Add(
		new MethodInvocationExpression(
			new MemberReferenceExpression(new MemberReferenceExpression(condition, "Expressions"), "Add"), fragment));

}

private static Expression GetStringArgument(Expression expr)
{
	if (expr is StringLiteralExpression)
		return expr;
	return new StringLiteralExpression(expr.ToString());
}

private static string GetOperator(BinaryExpression be)
{
	switch (be.Operator)
	{
		case BinaryOperatorType.Equality:
			return "==";
		case BinaryOperatorType.Inequality:
			return "!=";
		case BinaryOperatorType.Member:
			return "in";
		case BinaryOperatorType.LessThan:
			return "<";
		case BinaryOperatorType.LessThanOrEqual:
			return "<=";
		case BinaryOperatorType.GreaterThan:
			return ">";
		case BinaryOperatorType.GreaterThanOrEqual:
			return ">=";
		default:
			throw new NotSupportedException(be.Operator.ToString());
	}
}

This takes a Boo expression and transform that into the code that creates an object model that represents this expression. Linq is simply an extension to this implementation.

Nathan has posted Simple State Machine to CodePlex, it is the first project that I am aware of that uses Rhino DSL and the techniques that I am talking about in the book.

What is impressive about this is the level of professionalism that is involved in the project. It is a full scale DSL, with all the supporting infrastructure. I spent half an hour or so going through the entire thing, and I am impressed.

Put simply, this is how I think state based work flows should be defined. I could easily see myself extending this a bit to add persistence support & integration with NServiceBus, and be done with it.

Like most state machines, it has the ideas of states, events that can cause the state to be changed, and legal transitions from state to state. You can define tasks which will be executed upon changing a state, or upon entering / leaving a certain state.

Enough talking, let us look at a reasonably complex work flow:

workflow "Order Lifecycle"

#Event & State Identifier Targets.
#This section controls which Types will be used
#to resolve Event or State names into strongly typed CLR objects.
#--------------------------------------------------------
state_identifier_target @OrderStatus
event_identifier_target @OrderEvents

#Global Actions
#--------------------------------------------------------
on_change_state      @WriteToHistory, "on_change_state"
on_workflow_start    @WriteToHistory, "on_workflow_start"
on_workflow_complete @WriteToHistory, "on_workflow_complete"

#Event Definitions
#--------------------------------------------------------
define_event  @OrderPlaced
define_event  @CreditCardApproved
define_event  @CreditCardDenied
define_event  @OrderCancelledByCustomer
define_event  @OutOfStock
define_event  @OrderStocked
define_event  @OrderShipped
define_event  @OrderReceived
define_event  @OrderLost

#State & Transition Definitions
#--------------------------------------------------------
state @AwaitingOrder:
       when @OrderPlaced              >> @AwaitingPayment

state @AwaitingPayment:
       when @CreditCardApproved       >> @AwaitingShipment
       when @CreditCardDenied         >> @OrderCancelled
       when @OrderCancelledByCustomer >> @OrderCancelled

state @AwaitingShipment:
       when @OrderCancelledByCustomer >> @OrderCancelled
       when @OutOfStock               >> @OnBackorder
       when @OrderShipped             >> @InTransit

       #Individual states can define transition events as well
       on_enter_state @WriteToHistory, "on_enter_state(AwaitingShipment)"

state @OnBackorder:
       when @OrderCancelledByCustomer >> @OrderCancelled
       when @OrderStocked             >> @AwaitingShipment

state @InTransit:
       when @OrderReceived            >> @OrderComplete
       when @OrderLost                >> @AwaitingShipment

#NOTE: State definitions without any transitions will cause
#the state machine to Complete when they are reached.
#------------------------------------------------------------
state @OrderComplete
state @OrderCancelled

Here is the demo application UI, for the order processing life cycle:

As I said, impressive.

A few days ago, the BooLangStudio was announced in the Boo mailing list, bringing Boo support into Visual Studio.

Below you can see several screen shots. And you can find out more about it here.

This is a very promising move, especially since I soon have to write my tooling chapter :-)

Of course, this is still very early in the game, but it is good to see progress in this area again.

when I am writing DSL, I keep hitting one pain point. The CLR naming conventions, which are more or less imprinted on my eyelids, are not really conductive to clear reading in a DSL.

Let us take these entities, and see what we get when we try to build a DSL from them:

The DSL is for defining business rules, and it looks like this:

when User.IsPreferred and Order.TotalCost > 1000:
	AddDiscountPrecentage  5
	ApplyFreeShipping
when not User.IsPreferred and Order.TotalCost > 1000:
	SuggestUpgradeToPreferred 
	ApplyFreeShipping
when User.IsNotPreferred and Order.TotalCost > 500:
	ApplyFreeShipping

The main problem with this style of writing is that it is visually condense. I can read it pretty much as easily as I read natural English, but anyone who is not a developer really have to make an effort, and even for me, trying to read ruby styled code is easier. Here is how this would look like when using the ruby style conventions:

when User.is_preferred and Order.total_cost > 1000:
    add_discount_precentage 5
    apply_free_shipping
when not User.is_preferred and Order.total_cost > 1000:
   suggest_upgrade_to_preferred 
    apply_free_shipping
when User.is_not_preferred and Order.total_cost > 500:
   apply_free_shipping

This is much easier to read, in my opinion. The problem is that I consider this extremely ugly.

Obviously a different solution is needed...

Wait a minute! Boo has an open compiler. Why not just change the way it handle references? And that is what I did:

///<summary>
/// Allow to use underscore separated names, which will be translated to pascal case names.
/// pascal_case -> PascalCase.
/// All names that contains an underscores will go through this treatment.
///</summary>
/// <example>
/// You can  enable this behavior using the following statement
/// <code>
/// compiler.Parameters.Pipeline
///		.Replace(typeof (ProcessMethodBodiesWithDuckTyping),
/// 				 new ProcessMethodBodiesWithDslNamesAndDuckTyping());
/// </code>
/// </example>
public class ProcessMethodBodiesWithDslNamesAndDuckTyping : ProcessMethodBodiesWithDuckTyping
{
	/// <summary>
	/// Called when we encounter a reference expression
	/// </summary>
	/// <param name="node">The node.</param>
	public override void OnReferenceExpression(ReferenceExpression node)
	{
		if(node.Name.Contains("_"))
			SetNodeNameToPascalCase(node);
		base.OnReferenceExpression(node);
	}

	/// <summary>
	/// Called when we encounters a member reference expression
	/// </summary>
	/// <param name="node">The node.</param>
	public override void OnMemberReferenceExpression(MemberReferenceExpression node)
	{
		if (node.Name.Contains("_"))
			SetNodeNameToPascalCase(node);
		base.OnMemberReferenceExpression(node);
	}

	/// <summary>
	/// Sets the node name to pascal case.
	/// </summary>
	/// <param name="node">The node.</param>
	private static void SetNodeNameToPascalCase(ReferenceExpression node)
	{
		string[] parts = node.Name.Split(new char[] { '_' },StringSplitOptions.RemoveEmptyEntries);
		StringBuilder name = new StringBuilder();
		foreach (var part in parts)
		{
			name.Append(char.ToUpperInvariant(part[0]))
				.Append(part.Substring(1));
		}
		node.Name = name.ToString();
	}
}

I love Boo, with cause.

This is the entire Binsor config file for a real application:

import Castle.MonoRail.Framework
import Castle.MonoRail.WindsorExtension
import Rhino.Commons.Facilities from Rhino.Commons.ActiveRecord

facility MonoRailFacility
facility RhinoTransactionFacility
facility ActiveRecordUnitOfWorkFacility:
	assembly = "HibernatingRhinos"

for type in AllTypesBased of IController("HibernatingRhinos"):
	component type.Name, type
	
for type in AllTypes("HibernatingRhinos").WhereNamespaceEq("HibernatingRhinos.Services"):
	component type.GetServiceInterface(), type

And I am pretty confident that I am not going to have to do much in the future with those.

And yes, you can do it with the fluent registration API as well.

Here is the syntax that I am getting at...

for type in AllTypesBased of IView("Rhino.Commons.Test"):
	component type
	
for type in AllTypesWithAttribute of ControllerAttribute("Rhino.Commons.Test"):
	component type
	
for type in AllTypes("Rhino.Commons.Test") \
	.WhereNamespaceEq("Rhino.Commons.Test.Binsor"):
	component type

for type in AllTypes("Rhino.Commons.NHibernate") \
	.Where({ t as System.Type | t.Name.Contains("NHRepository") }):
	component "nh.repos", type.GetSeriveInterface(), type

And this seems to cover just about any scenario that I can think of. Combine that with Binsor's extend facility, and we are more or less done.