Ayende @ Rahien

Refunds available at head office

Why I HATE editing?

I have just finished 2nd pass editing all the chapters and appendixes for my DSL book. This has been, I think, the 1,390th time that I had gone through the book since I finished writing it. Luckily for me, this appears to be the end, there is the final printer proofs stage, and then I’ll truly be free.

One of the things that I wasn’t told about when I agree to write the book is just how many times I would have to read it. I think that I am good writer (God knows that I get enough practice), but for crying out load, there is only so many times that I can read the same piece (especially if I wrote it) before it gets pretty boring.

Now, to be absolutely clear, the publisher is doing great job, and I think that each pass has been extremely valuable. We found problems in grammar, spelling, fuzzy speak, missing concepts, bugs in code, etc. This is important, valuable and necessary.

One of the things that some people have hard time understanding about the way that I think is that I can accept that some things are necessary even if they aren’t fun. That mean that I’ll do them, but I’ll complain about it. Just like taxes. I’ll pay them, but I reserve the right to bitch about it.

It took me a while to understand why I hate editing so much. I know how to handle boring work, and I know how to handle creative work. What I don’t know how to handle is boring creative work. I split my attention when I am doing boring stuff, usually having something in the background that it interesting while I am doing the boring stuff. For creative stuff, I usually enjoy it.

But boring creative stuff? I can’t split my attention then, and it is boring. It is annoying, and I do think about it like a tax.

Consider this statement, that was flagged by my editor:

The MacroMacro passes the MacroStatement that the compiler hands to the macro using the name of the macro

Can you figure out what I meant here? It took me a while to do that :-)

Editing the book has been a huge weight on my mind, having finished that is such a huge relief…

Boo Lang Studio 1.0 Alpha it out!

Jeffery Olson has just made the first release of Boo Lang Studio available.

This one comes with a "Yes, Dear" installer.

image

Yeah, we have Boo installed!

image

Let us create a new project:

image

And take a look at the code:

image

And intellisense works as well, whew!

image

Jeffery Olson and James Gregory: THANKS!

Nicer Linq

A few days ago I posted about Ugly Linq. Ever since then, I kept thinking about how ugly it is to handle this by hand. Suddenly, it hit me that I don't have to do it that way.

Boo already has the facilities to take a compiler AST and translate that into the code that would recreate this AST. In particular, this makes the code we previously had to write to this:

public class ConditionMacro : AbstractAstMacro
{
	public override Statement Expand(MacroStatement macro)
	{
		Expression serialize = new CodeSerializer().Serialize(macro.Arguments[0]);
		var body = new Block();
		body.Statements.Add(new ReturnStatement(macro.Arguments[0]));
		return new ExpressionStatement(
			new MethodInvocationExpression(
					AstUtil.CreateReferenceExpression(typeof(Condition).FullName),
					new BlockExpression(body),
					serialize
				)

			);
	}
}

And what that means is that given this code:

condition a > 10

We can get this result:

image

And that is it.

You get both the actual compiled expression and the AST that describes this. This is critically important because you can now take this piece of AST and do transformations / views on it.

And that is important if you want to have reliable graphical representation on top of a textual DSL, which is what my chapter 10 is going to cover.

Damn, this is simple! Thanks Rodrigo!

Ugly Linq

One of the things that always bothered me with Linq was that it is actually not an interesting idea from the compiler perspective. I just had to implement a very simple expression to expression tree converter, which only served to strengthen my opinion. Here is the (ugly, proof of concept, horrible) implementation, using the Boo AST:

private static Block Linqify(Expression expr)
{
	var block = new Block(expr.LexicalInfo);
	ReferenceExpression condition = AddCondition(block);
	Parse(block, condition, expr);
	block.Add(new ReturnStatement(condition));
	return block;
}

private static ReferenceExpression AddCondition(Block block)
{
	var condition = new MethodInvocationExpression(new ReferenceExpression("Condition"));
	var expression = new ReferenceExpression("condition_" + CompilerContext.Current.AllocIndex());
	block.Add(
		new BinaryExpression(
			BinaryOperatorType.Assign,
			expression,
			condition)
		);
	return expression;
}

private static void Parse(Block block, Expression condition, Expression expr)
{
	var be = expr as BinaryExpression;
	if (be != null && (be.Operator == BinaryOperatorType.Or || be.Operator == BinaryOperatorType.And))
	{
		block.Add(new BinaryExpression(
			BinaryOperatorType.Assign,
			new MemberReferenceExpression(condition, "Operator"),
			new StringLiteralExpression(be.Operator.ToString().ToLowerInvariant()))
			);
		ReferenceExpression left = AddCondition(block);
		block.Add(
			new MethodInvocationExpression(
				new MemberReferenceExpression(new MemberReferenceExpression(condition, "Expressions"), "Add"), left));
		ReferenceExpression right = AddCondition(block);
		block.Add(
			new MethodInvocationExpression(
				new MemberReferenceExpression(new MemberReferenceExpression(condition, "Expressions"), "Add"), right));
		Parse(block, left, be.Left);
		Parse(block, right, be.Right);
		return;
	}
	var fragment = new MethodInvocationExpression(new ReferenceExpression("Fragment"));
	if (expr is UnaryExpression)
	{
		fragment.NamedArguments.Add(
			new ExpressionPair(new ReferenceExpression("Modifier"),
				new StringLiteralExpression("not"))
			);
		be = (BinaryExpression)((UnaryExpression)expr).Operand;
	}
	var func1 = (MethodInvocationExpression)be.Left;
	fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Func1"), new StringLiteralExpression(func1.Target.ToString())));
	fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Prop1"), GetStringArgument(func1.Arguments[0])));
	var func2 = be.Right as MethodInvocationExpression;
	if (func2 != null)
	{
		fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Func2"), new StringLiteralExpression(func2.Target.ToString())));
		fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Prop2"), GetStringArgument(func2.Arguments[0])));
	}
	else
	{
		fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Func2"), new StringLiteralExpression("literal")));
		fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Prop2"), GetStringArgument(be.Right)));
	}
	fragment.NamedArguments.Add(new ExpressionPair(new ReferenceExpression("Operator"),
		new StringLiteralExpression(GetOperator(be))));

	block.Add(
		new MethodInvocationExpression(
			new MemberReferenceExpression(new MemberReferenceExpression(condition, "Expressions"), "Add"), fragment));

}

private static Expression GetStringArgument(Expression expr)
{
	if (expr is StringLiteralExpression)
		return expr;
	return new StringLiteralExpression(expr.ToString());
}

private static string GetOperator(BinaryExpression be)
{
	switch (be.Operator)
	{
		case BinaryOperatorType.Equality:
			return "==";
		case BinaryOperatorType.Inequality:
			return "!=";
		case BinaryOperatorType.Member:
			return "in";
		case BinaryOperatorType.LessThan:
			return "<";
		case BinaryOperatorType.LessThanOrEqual:
			return "<=";
		case BinaryOperatorType.GreaterThan:
			return ">";
		case BinaryOperatorType.GreaterThanOrEqual:
			return ">=";
		default:
			throw new NotSupportedException(be.Operator.ToString());
	}
}

This takes a Boo expression and transform that into the code that creates an object model that represents this expression. Linq is simply an extension to this implementation.

Simple State Machine

Nathan has posted Simple State Machine to CodePlex, it is the first project that I am aware of that uses Rhino DSL and the techniques that I am talking about in the book.

What is impressive about this is the level of professionalism that is involved in the project. It is a full scale DSL, with all the supporting infrastructure. I spent half an hour or so going through the entire thing, and I am impressed.

Put simply, this is how I think state based work flows should be defined. I could easily see myself extending this a bit to add persistence support & integration with NServiceBus, and be done with it.

Like most state machines, it has the ideas of states, events that can cause the state to be changed, and legal transitions from state to state. You can define tasks which will be executed upon changing a state, or upon entering / leaving a certain state.

Enough talking, let us look at a reasonably complex work flow:

workflow "Order Lifecycle"

#Event & State Identifier Targets.
#This section controls which Types will be used
#to resolve Event or State names into strongly typed CLR objects.
#--------------------------------------------------------
state_identifier_target @OrderStatus
event_identifier_target @OrderEvents

#Global Actions
#--------------------------------------------------------
on_change_state      @WriteToHistory, "on_change_state"
on_workflow_start    @WriteToHistory, "on_workflow_start"
on_workflow_complete @WriteToHistory, "on_workflow_complete"

#Event Definitions
#--------------------------------------------------------
define_event  @OrderPlaced
define_event  @CreditCardApproved
define_event  @CreditCardDenied
define_event  @OrderCancelledByCustomer
define_event  @OutOfStock
define_event  @OrderStocked
define_event  @OrderShipped
define_event  @OrderReceived
define_event  @OrderLost

#State & Transition Definitions
#--------------------------------------------------------
state @AwaitingOrder:
       when @OrderPlaced              >> @AwaitingPayment

state @AwaitingPayment:
       when @CreditCardApproved       >> @AwaitingShipment
       when @CreditCardDenied         >> @OrderCancelled
       when @OrderCancelledByCustomer >> @OrderCancelled

state @AwaitingShipment:
       when @OrderCancelledByCustomer >> @OrderCancelled
       when @OutOfStock               >> @OnBackorder
       when @OrderShipped             >> @InTransit

       #Individual states can define transition events as well
       on_enter_state @WriteToHistory, "on_enter_state(AwaitingShipment)"

state @OnBackorder:
       when @OrderCancelledByCustomer >> @OrderCancelled
       when @OrderStocked             >> @AwaitingShipment

state @InTransit:
       when @OrderReceived            >> @OrderComplete
       when @OrderLost                >> @AwaitingShipment

#NOTE: State definitions without any transitions will cause
#the state machine to Complete when they are reached.
#------------------------------------------------------------
state @OrderComplete
state @OrderCancelled

Here is the demo application UI, for the order processing life cycle:

image

As I said, impressive.

BooLangStudio: Boo in Visual Studio

A few days ago, the BooLangStudio was announced in the Boo mailing list, bringing Boo support into Visual Studio.

Below you can see several screen shots. And you can find out more about it here.

This is a very promising move, especially since I soon have to write my tooling chapter :-)

Of course, this is still very early in the game, but it is good to see progress in this area again.

image

image

The magic of boo - Flexible syntax

when I am writing DSL, I keep hitting one pain point. The CLR naming conventions, which are more or less imprinted on my eyelids, are not really conductive to clear reading in a DSL.

Let us take these entities, and see what we get when we try to build a DSL from them:

image

The DSL is for defining business rules, and it looks like this:

when User.IsPreferred and Order.TotalCost > 1000:
	AddDiscountPrecentage  5
	ApplyFreeShipping
when not User.IsPreferred and Order.TotalCost > 1000:
	SuggestUpgradeToPreferred 
	ApplyFreeShipping
when User.IsNotPreferred and Order.TotalCost > 500:
	ApplyFreeShipping

The main problem with this style of writing is that it is visually condense. I can read it pretty much as easily as I read natural English, but anyone who is not a developer really have to make an effort, and even for me, trying to read ruby styled code is easier. Here is how this would look like when using the ruby style conventions:

when User.is_preferred and Order.total_cost > 1000:
    add_discount_precentage 5
    apply_free_shipping
when
not User.is_preferred and Order.total_cost > 1000:
   suggest_upgrade_to_preferred 
    apply_free_shipping
when User.is_not_preferred and Order.total_cost > 500:
   apply_free_shipping

This is much easier to read, in my opinion. The problem is that I consider this extremely ugly.

image

Obviously a different solution is needed...

Wait a minute! Boo has an open compiler. Why not just change the way it handle references? And that is what I did:

///<summary>
/// Allow to use underscore separated names, which will be translated to pascal case names.
/// pascal_case -> PascalCase.
/// All names that contains an underscores will go through this treatment.
///</summary>
/// <example>
/// You can  enable this behavior using the following statement
/// <code>
/// compiler.Parameters.Pipeline
///		.Replace(typeof (ProcessMethodBodiesWithDuckTyping),
/// 				 new ProcessMethodBodiesWithDslNamesAndDuckTyping());
/// </code>
/// </example>
public class ProcessMethodBodiesWithDslNamesAndDuckTyping : ProcessMethodBodiesWithDuckTyping
{
	/// <summary>
	/// Called when we encounter a reference expression
	/// </summary>
	/// <param name="node">The node.</param>
	public override void OnReferenceExpression(ReferenceExpression node)
	{
		if(node.Name.Contains("_"))
			SetNodeNameToPascalCase(node);
		base.OnReferenceExpression(node);
	}

	/// <summary>
	/// Called when we encounters a member reference expression
	/// </summary>
	/// <param name="node">The node.</param>
	public override void OnMemberReferenceExpression(MemberReferenceExpression node)
	{
		if (node.Name.Contains("_"))
			SetNodeNameToPascalCase(node);
		base.OnMemberReferenceExpression(node);
	}

	/// <summary>
	/// Sets the node name to pascal case.
	/// </summary>
	/// <param name="node">The node.</param>
	private static void SetNodeNameToPascalCase(ReferenceExpression node)
	{
		string[] parts = node.Name.Split(new char[] { '_' },StringSplitOptions.RemoveEmptyEntries);
		StringBuilder name = new StringBuilder();
		foreach (var part in parts)
		{
			name.Append(char.ToUpperInvariant(part[0]))
				.Append(part.Substring(1));
		}
		node.Name = name.ToString();
	}
}

I love Boo, with cause.

Zero friction IoC: Auto registration is mandatory

This is the entire Binsor config file for a real application:

import Castle.MonoRail.Framework
import Castle.MonoRail.WindsorExtension
import Rhino.Commons.Facilities from Rhino.Commons.ActiveRecord

facility MonoRailFacility
facility RhinoTransactionFacility
facility ActiveRecordUnitOfWorkFacility:
	assembly = "HibernatingRhinos"

for type in AllTypesBased of IController("HibernatingRhinos"):
	component type.Name, type
	
for type in AllTypes("HibernatingRhinos").WhereNamespaceEq("HibernatingRhinos.Services"):
	component type.GetServiceInterface(), type

And I am pretty confident that I am not going to have to do much in the future with those.

And yes, you can do it with the fluent registration API as well.

Binsor & Auto Registration - Making it even simpler

Here is the syntax that I am getting at...

for type in AllTypesBased of IView("Rhino.Commons.Test"):
	component type
	
for type in AllTypesWithAttribute of ControllerAttribute("Rhino.Commons.Test"):
	component type
	
for type in AllTypes("Rhino.Commons.Test") \
	.WhereNamespaceEq("Rhino.Commons.Test.Binsor"):
	component type

for type in AllTypes("Rhino.Commons.NHibernate") \
	.Where({ t as System.Type | t.Name.Contains("NHRepository") }):
	component "nh.repos", type.GetSeriveInterface(), type

And this seems to cover just about any scenario that I can think of. Combine that with Binsor's extend facility, and we are more or less done.

Course: Building Domain Specific Languages in Boo

You can register here for a two days course in building DSL with Boo.

It is going to take place two weeks from today, in Austin. (19 - 20 May)

I know that this is short notice, but it wasn't something that was planned well in advance. It came out of the ALT.Net conference.

Topics:

  • Creating Domain Specific Languages
  • The Boo Language
  • Flexible compiler and malleable language
  • Creating applications with embedded DSL
  • Management, tracing and debugging
  • Tooling support
  • Testing and maintainability concerns

There are ten seats open for that.

I hope we would have fun.

I would also like to thank Jeffrey Palermo and Headspring for hosting the course.

Loop unrolling in Boo

// We create a class for the macro, the class name is
// meaningful, [macro name]Macro allows us to later refer
// to the macro using just [macro name].
// Note that we inherit from AbstractAstMacro
class UnrollMacro(AbstractAstMacro):

	// Here we perform the actual compiler manipulation
	// the compiler hands us a macro statement, and we have
	// to return a statement back, which will replace it.
	override def Expand(macro as MacroStatement) as Statement:
	
		// define a block of code
		block = Block()
		
		// extract the second parameter value
		end = cast(IntegerLiteralExpression, macro.Arguments[1]).Value
		
		for i in range(end):
			// create assignment statement, using the block: trick 
			// and add it to the output
			statements = [|
				block:
					$(macro.Arguments[0]) = $i
			|].Block
			block.Add(statements)
			
			// add the original contents of the macro
			// to the output
			block.Add(macro.Block)
			
		return block
And usage:
unroll i, 5:
	print i

Which will produce:

i = 0
print i
i = 1
print i
i = 2
print i
i = 3
print i
i = 4
print i

The quote generation DSL

I am doing some work on the DSL book right now, and I run into this example, which is simple too delicious not to post about.

Assume that you have the following UI, which you use to let a salesperson generate a quote for your system.

image

This is much more than just a UI issue, to be clear. You have fully fledged logic system here. Calculating the total cost is the easy part, first you have to understand what you need.

Let us define a set of rules for the application, is will be clearer when we have the list in front of us:

  • The Salary module requires a machine per every 150 users.
  • The Taxes module requires a machine per 50 users.
  • The Vacations module requires the Scheduling Work module.
  • The Vacations module requires the External Connections module.
  • The Pension Plans module requires the External Connections module.
  • The Pension Plans module must be on the same machine as the Health Insurance module.
  • The Health Insurance module requires the External Connections module.
  • The Recruiting module requires a connection to the internet, and therefore requires a fire wall of the recommended list.
  • The Employee Monitoring module requires the CompMonitor component

Of course, this fictitious sample is still too simple, we can probably sit down and come up with fifty or so more rules that we need to handle. Just handling the second level dependencies (External Connections, CompMonitor, etc) would be a big task, for example.

Assume that you have not a single such system, but 50 of them. I know of a company that spent 10 years and has 100,000 lines of C++ code (that implements a poorly performing Lisp machine, of course) to solve this issue.

My solution?

specification @vacations:
	requires @scheduling_work
	requires @external_connections
	
specification @salary:
	users_per_machine 150
	
specification @taxes:
	users_per_machine 50

specification @pension:
	same_machine_as @health_insurance

Why do we need a DSL for this? Isn’t this a good candidate for data storage system? It seems to me that we could have expressed the same ideas with XML (or a database, etc) just as easily. Here is the same concept, now express in XML.

<specification name="vacation">
	<requires name="scheduling_work"/>
	<requires name="external_connections"/>
</specification>

<specification name="salary">
	<users_per_machine value="150"/>
</specification>

<specification name="taxes">
	<users_per_machine value="50"/>
</specification>

<specification name="pension">
	<same_machine_as name="health_insurance"/>
</specification>

That is a one to one translation of the two, why do I need a DSL here?

Personally, I think that the DSL syntax is nicer, and the amount of work to get from a DSL to the object model is very small compared to the work required to translate to the same object model from XML.

That is mostly a personal opinion, however. For pure declarative DSL, we are comparable with XML in almost all things. It gets interesting when we decide that we don’t want this purity. Let us add a new rule to the mix, shall we?

  • The Pension Plans module must be on the same machine as the Health Insurance module, if the user count is less than 500.
  • The Pension Plans module requires distributed messaging backend, if the user count is great than 500.

Trying to express that in XML can be a real pain. In fact, it means that we are trying to shove programming concepts into the XML, which is always a bad idea. We could try to put this logic in the quote generation engine, but that is complicating it with no good reason, tying it to the specific application that we are using, and in general making a mess.

Using our DSL (with no modification needed), we can write it:

specification @pension: 
	if information.UserCount < 500: 
		same_machine_as @health_insurance 
	else: 	
		requires @distributed_messaging_backend

As you can imagine, once you have run all the rules in the DSL, you are left with a very simple problem to solve, with all the parameters well known.

In fact, throughout the process, there isn't a single place of overwhelming complexity.

I like that.

A web server in 30 lines of code

Just found myself writing that, and it was amusing.

import System.Net
import System.IO

if argv.Length != 2:
	print "You must pass [prefix] [path] as parameters"
	return

prefix = argv[0]
path = argv[1]

if not Directory.Exists(path):
	print "Could not find ${path}"
	return

listener = HttpListener()
listener.Prefixes.Add(prefix)
listener.Start()

while true:
	context = listener.GetContext()
	file = Path.GetFileName(context.Request.RawUrl)
	fullPath = Path.Combine(path, file)
	if File.Exists(fullPath):
		context.Response.AddHeader("Content-Disposition","attachment; filename=${file}")
		bytes = File.ReadAllBytes(fullPath)
		context.Response.OutputStream.Write(bytes, 0, bytes.Length)
		context.Response.OutputStream.Flush()
		context.Response.Close()
	else:
		context.Response.StatusCode = 404
		context.Response.Close()

Short yet interesting: in search of a sample

One of the feedback points that I got about the book was that there need to be a fully working Boo program, before we start to introduce the DSL ones. The problem? I can't think of anything that will be both interesting and short enough to put into a book.

My limit is at about two pages, and I don't want a tedious example. Any suggestions?

One major point, it should have as little background assumptions as possible.

From BooBS to Bake

Okay, I renamed Boo Build System to Bake. Now you can cut the jokes and actually integrate it into a PC environment.

The repository is here, although you can just grab the binaries.

image

Boo: Design By Contract in 20 lines of code

Now, before Greg hurls a modopt on me, I want to be clear that this isn't the same thing that Spec# is doing. But it is a very cool way to specify constraints that must always be valid when a method exists.

Here is the code:

[AttributeUsage(AttributeTargets.Class)]
class EnsureAttribute(AbstractAstAttribute):
	
	expr as Expression
	
	def constructor(expr as Expression):
		self.expr = expr
		
	def Apply(target as Node):
		type as ClassDefinition = target
		for member in type.Members:
			method = member as Method
			continue if method is null
			block = method.Body
			method.Body = [|
				block:
					try:
						$block
					ensure:
						assert $expr
			|].Block

And the usage:

[ensure(name is not null)]
class Customer:
	name as string
		
	def constructor(name as string):
		self.name = name
	
	def SetName(newName as string):
		name = newName

Now, any attempt to set the name to null will cause an assertion exception. This technique is quite powerful, and very easy to use. A few years ago I wrote a design by contract implementation for boo that was far more ambitious (handling inheritance, etc). I remember it being much more complicated, and while things like quasi quotation do make it easier, it is not that big a change.

I think that mostly it is the way I write code now, striving to simplicity is something that I am trying to apply recently, and I think it works.

Meta Methods

A meta-method is a shortcut into the compiler; it is a method that accepts AST nodes[1] and returns an AST node.

Let us implement this very simple scenario, the assert statement. Now, because Boo already has that, we will use “verify” as the method name. Here is the full method implementation:

[Meta]
static def verify(expr as Expression):
	return [|
		unless $expr:
			raise $(expr.ToCodeString())
	|]

We are using quasi quotation to save us typing. This is a static method decorated with the [Meta] attribute, and accepting an AST expression. This is all you need in order to create a meta-method. When you have a meta-method, you can call it, like this:

verify 1 == 2

Now the interesting tidbit happens. When the compiler sees a call to a meta-method, it doesn’t emit the code to call this method at runtime. Instead, during compilation, the meta-method is executed. We pass it the AST of the arguments of the method code (including anonymous blocks), and then we replace this method call with the result of calling the meta-method.

It is important that you’ll understand that after compilation, where in the code we had this:

verify 1 == 2

The actual compiled bits will have this:

unless 1 == 2:
	raise “1 == 2”

Please go over it again, to make sure that you understand how it works. It is similar to text substitution macros in C and C++, but this is actual code that is running during compilation that gets to output any code that it wants back into the compilation process, not mere text preprocessing. In addition to that, we are dealing directly with the compiler’s AST, not just copying lines of text.

This seems to be something that a lot of people have a hard time grasping. The compiler will ask you, at compilation time, what kind of transformation you want to do on the code. It will then take the result of the transformation (the method return value) and put it where the method call used to be.

The Boo code above can also be translated to the following C#, which is a bit more explicit about what is going on:

[Meta]
public static UnlessStatement verify(Expression expr)
{
	UnlessStatement unlessS = new UnlessStatement();
	unless.Condition = Expression.Lift(expr);
	RaiseStatement raise = new RaiseStatement();
	raise.Exception = Expression.Lift(expr.ToCodeString());
	unless.Statements.Add(raise);
	return unless;
}

Both have the same exact semantics.

We have actually used meta-methods before, when we implemented the “when” keyword for the scheduling DSL. Meta-methods are used in DSL quite often. They are usually the first step that we need to take into the compiler when we run into the limits of what the compiler gives us out of the box.


[1] An ast node is generic term to all the types that compose the abstract syntax tree of the language.

If it walks like a duck and it quacks like a duck

Then it must be an IQuackFu.

IQuackFu is Boo’s answer to the Method Missing / Message Not Understood from dynamic languages. Since Boo is a statically typed language[1], and since method missing is such a nice concept to have, we use this special interface to introduce this capability.

You are probably confused, because I didn’t even explain what method missing is. Let us go back and look at an example, shall we? We want to look at the following xml:

<People> 	<Person> 		<FirstName>John</FirstName> 	</Person> 	<Person> 		<FirstName>Jane</FirstName> 	</Person> </People> 

Now we want to display the first names in the xml. We can do it using XPath, but the amount of code required makes this awkward. We can also generate some sort of strongly typed wrapper around it, assuming that we have a schema for this, we can use a tool to generate the schema, if we don’t have it already…

Doesn’t it look like a lot of work? We can also do this:

doc = XmlObject(xmlDocument.DocumentElement)
for person as XmlObject in doc.Person:
print person.FirstName

But we are using a generic object here, how can this work? This works because we intercept the calls to the object and decide how to answer them at runtime. This is the meaning of the term “method missing”. We “catch” the method missing and decide to do something smart about it (like returning the data from the xml document).

At least, this is how it works in dynamic languages. For a statically typed language, the situation is a bit different; all method calls must be known at compile time. That is why Boo introduced the idea of IQuackFu. Let us check the implementation of XmlObject first, and then we will discuss how it works:

class XmlObject(IQuackFu):
_element as XmlElement

def constructor(element as XmlElement):
_element = element

def QuackInvoke(name as string, args as (object)) as object:
pass # ignored

def QuackSet(name as string, parameters as (object), value) as object:
pass # ignored

def QuackGet(name as string, parameters as (object)) as object:
elements = _element.SelectNodes(name)
if elements is not null:
return XmlObject(elements[0]) if elements.Count == 1
return XmlObject(e) for e as XmlElement in elements

override def ToString():
return _element.InnerText

We didn’t implement the QuackInvoke and QuackSet, because they are not relevant to the example at hand, I think that QuackGet will make the point. Now, just to complete the picture, we will write the first code sample, the use of XmlObject, as the compiler will output it.

doc = XmlObject(xmlDocument)
for person as XmlObject in doc.QuackGet(“Person”):
print person.QuackGet(“FirstName”)

The way it works, when the compiler finds that it can’t resolve a method (or a property) in the usual way, it then check if the type implements the IQuackFu interface. If it does implement IQuackFu, it translates the method call into the equivalent method call.

The example of the Xml Object is a really tiny one of the possibilities. Convention based methods are an interesting idea[2] that is widely used in Ruby. Here is an example that should be immediately familiar to anyone who dabbled in Rails’ ActiveRecord:

user as User = Users.FindByNameAndPassword(“foo”, “bar”)

Which will be translated by the compiler to:

user as User = Users.QuackInvoke(“FindByNameAndPassword”, “foo”, “bar”)

The Users’ QuackInvoke method will parse the “method name” and issue a query by name and password.

You can do some very interesting things with IQuackFu...


[1] Well, it is statically typed unless you explicitly tell the compiler that you want late bound semantics. Aside from working against IDispatch COM interfaces, I have rarely found that ability useful. One case I did find it useful, however, was when I wanted to introduce Context Parameters, which we will discuss in a few pages.

[2] For the adventurous sorts, you can also do something called Lazy Methods, in which you generate a method if and only if it is being called. This is an interesting exercise in extending the compiler, but for all intents and purposes, IQuackFu answers this need very well.

How to get around in Boo

When you are not sure how to do something in Boo, try doing it like you would with C# (with the obvious syntax changes), in most cases, it would work. It may not be the best way to do something, however.

Keep this a secret, I may get thrown out of the Boo Associated Hackers community if that would happen, and where I would be without my BAH! membership?

What makes Boo a great language

I really like the CLR. It is a great platform, it has a rich set of libraries, it has a lot of power and flexibility, and it was designed with multi language support in mind, which means that what you can do on the CLR vs. what you can do in your language are two different things.

It is just too bad that the default CLR language is C#. Now, that probably raised a few brows, and definitely some hackles. C# is a great language, I can hear you saying. Well, yes, sort of. If you thinks that holding the compiler’s hands is a useful thing to do. Leaving aside the ability to extend the compiler, Boo has the following to offer us:

Syntactic sugar for common programming patterns - List, hash and array literals, object initialization, string formatting and regular expression matching are all first class concepts in Boo, with direct support for all of them in a natural manner.

Automatic Variable Declaration together with Automatic type inference – the compiler takes care of things for you, instead of having to type the same thing over and over and over again. Some would say that this is bad, for them I would reply that they should try it first. It works.
Take a look at this:

def random():
	return 4 # selected by dice roll, guaranteed to be random
val = random() 

Is there are reason that I would need to specify the type over and over again? The compiler can figure it out for itself and not bother you with it. If you want to get a compiler error, you can:

val as string = random() # will error about type mismatch 

Automatic type casting – don’t make me explicitly say it, figure it out for me. I’ll have unit tests to check to cover me.

Duck typing – Boo is a strongly typed language, but you can ask the compiler to relax those constraints at certain scenarios. It makes some things much more natural, especially since you have a way to get into this infrastructure and decide what to do at runtime.
If we will take a simple example, let us look at the XmlObject and what we can do with it:

person = XmlObject(xmlDocument)
print person.FirstName
print person.LastName 

What we just did was resolve, at runtime, that we were asked to get the value of a property called “FirstName”, and we did. This is a trivial piece of code to implement.

Just those benefits present a significant improvement in the language experience, and we haven’t even touched the extensible compiler yet.

The Boo Language

So, what is this Boo language anyway? I am talking about it fairly often, but I never explained, and I am going to assume that people aren't following links :-)

image Boo is an object oriented, statically typed language for the common language runtime with a Python inspired syntax and focused on language and compiler extensibility.

Boo is an Open Source project, released under the BSD license. This means that you are free to take, modify and use the language and its products in any way you want, without any limitation[1]

. Rodrigo B. de Oliveira started the project, and since then grown in popularity. The project is active and continually evolving. At the time of the writing, the released version of Boo is 0.8.

However, Boo seems to be following the trend of many open source projects, in that that it is not really rushing toward 1.0 release. Boo has been around for a number of years, and I have been using it for production for the last two years.

Production, in this case, means systems that move money around with my neck on the line. I have a high degree of trust in it, obviously.

Beyond being a really malleable language, suitable for DSL. Boo is also a nice language to work with for day-to-day tasks. The SharpDevelop IDE project has support for Boo, so you get the usual benefits. There is an ongoing project to support Boo in Visual Studio, and I have an active interest in seeing support for that being established. I don't much care for which IDE, but I want a standard, checklist driven way of getting a language running.

Here is a secret, I tend to think and write in Boo, translating down to C# because I have to. Boo is a far more flexible language, and the ability to reprogram the compiler makes tedious tasks in other languages an amusing challenge. Those same tedious problems are usually overcome, to some degree, by throwing code generation at the problem until it goes away.image

In Boo, we would usually tell the compiler how we want to handle this, and let the compiler take over from there. This is similar to code generation, except that this is done at the compiler level, which means that we have far more knowledge about what is going on.

For example, let us say that we want to implement the singleton pattern. We need to do quite a bit of work to make it happen. Ensure that creating the singleton is thread safe, lazy initialization, make all the constructors private, ensure that serializable singleton remain singleton through serialization/deserialization, create static Instance property, have no static members on the singleton class.

To do this in most language we would have to follow an implementation pattern and write fairly repetitive code. In Boo, we can capture this usage pattern in an AstAttribute (more on that later) that will do all of that for us, at the cost of a single attribute declaration.

Similar capabilities allow correct usage of disposable, observable, and more advance techniques such as aspect oriented programming.

Some of the capabilities that Boo has can be had using IL Weaving tools such as PostSharp. The problem with those is that using them is something that happens after compilation. As a simple example, you would not be able to create a singleton in this manner, because you would not be able to generate the Instance property at the appropriate time.

Assuming that you want to use the singleton in the same assembly as it was defined, you will not be able to reference the Instance property, it will not exist until the post compile step has run. Boo, however, does this at compile time, so by the time the compiler get to the point where it is resolving the property, it has already been added.

In short, I really like the language and what it allows me to do.


[1] Well, if you are interested in creating a new language called Ba, you probably would want to make it clear where the project ancestry came from…

Domain Specific Language: Losing the original language

Here is an interesting question, at what point you drive away so far from the original language that you lose the benefits of an internal DSL?

The following examples are several ways to represent the same thing, going from the extreme DSL to the no DSL to C#. I think that they all have value, and neither cross that line, but I believe that they will help make the point. The example here is a a simple MonoRail Controller:

#1:

controller Field:
    action Tag:
           @defaultTags = [ "green", "trees", "growth" ]

#2:

controller Field:
    def Tag():
         @defaultTags = [ "green", "trees", "growth" ]

#3:

controller Field:
    def Tag():
          PropertyBag["defaultTags"] = [ "green", "trees", "growth" ]

#4:

class FieldController(Controller):
   def Tag():
          PropertyBag["defaultTags"] = [ "green", "trees", "growth" ]

#5:

public class FieldController : Controller
{
	public void Tag()
	{
		PropertyBag["defaultTags"] = new string[] { "green", "trees", "growth" };
	}
}

Any comments?