Today, around 3 AM, I finally submitted the last chapter in the book.

I was surprised to realize that it took over a year to write it, and I am happy beyond words that the action of writing the book is behind me.

That said, it is not actually the end of the work. All it means is that I am done writing new content, now we are going to get into a cycle of editing once more, and I really don't look forward to re-reading what I wrote five to ten times, over and over and over again.

Nevertheless, I am feeling GREAT!

User interface is often seen as an external piece of the DSL, something that is completely separated from the language. While this is one options to handle things, it is not necessarily the only one.

During the design of the DSL, we should also think about the user interface that we intend to give our users. That will change the way that we build the language, usually, but not only because we need to give the UI more information, but because we can also utilize the UI to make things easier.

In particular, a good usage of the UI will allow us to give the users more information. Using the Shopping Cart DSL, we can utilize the UI to show the users what are the actions that will be taken on bad credit for all types of customers, or what it means to be a preferred customer. This is a very efficient way of writing documentation, once that is also guaranteed to be always updated.

A mockup of a screen reporting on the behavior of the system, as extracted from the DSL scripts:

This is a very important concept because it is showing the user data that the DSL isn't actually able to show. This is aggregated data across all behaviors and in all states, something that in the past you would have to either remember or research every time that you wanted to find out what is actually happening.

In short, don't try to limit yourself to just the typical IDE option, a DSL can and should give you a lot more information, and using this information can give you a lot of insight into your business.

As the final example in the book, I am showing off a DSL for processing a shopping cart. Here is how it looks like:

behavior of preferred_customer

upon bad_credit:
	authorize_funds cart.Total * 0.5,  "For preferred customers we only authorize half the amount"

upon cart_update:
	when cart.Total > 1000:
  		add_cart_discount 5, "Preferred members gets 5% discount for orders over 1,000$"

And this:

behavior of default_customer

upon bad_credit:
      authorize_funds cart.Total,  "We require full authorization of the amount in low credit rating scenarios"

Where we defined preferred customer and default customer as:

define preferred_customer:
	customer.TotalPurchaseAmount > 5000
	
define default_customer:
	customer.TotalPurchaseAmount <= 5000

The implementation is surprisingly easy, and I was able to walk through the reasoning for this implementation rather than the usual when [condition] in a way that I hope would make sense.

You can look at the implementation here: https://rhino-tools.svn.sourceforge.net/svnroot/rhino-tools/BDSLiB/trunk

Yesterday I gave two workshops, Advanced NHibernate and Building DSL with Boo. I finished the day absolutely crushed, but I think they went very well.

Both were recorded, although I am not sure when they will be online.

Ryan Kelley has a blow by blow description of the NHibernate talk, and you can get the code for that here:

https://rhino-tools.svn.sourceforge.net/svnroot/rhino-tools/trunk/SampleApplications/ORM+=2/

I'll post the code for the DSL talk shortly afterward.

I got some really positive feedback about the NHibernate Profiler, and I am very interested in demoing that and getting additional feedback when the real conference starts.

I had this though in my head for a while now. I built an IDE for a DSL, and somewhere toward the end of the first revision I understood something very interesting. My IDE wasn't actually using the textual representation of the language. The scripts that the user was editing were actually live instances, and they were fully capable of validating and saving themselves.

The IDE worked with those instances, used them to do its operations, and allowed to edit them on the fly. It was quite challenging to do, I must say, and I kept thinking about the image model of smalltalk, where everything is a live instance.

This brings to mind Greenspan's tenth rule, which state: Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

You can head to the Book Site and get a new version, because Chapters 1 - 11 (and appendix B) are available.

The only remaining chapters are #12 (DSL implementation patterns) and #13 Implementing a Real World DSL. Chapter 12 goes really well, so far, and I hope to have it ready by the end of the weekend.

Here it the table of content:

• Writing the Getting Started Guide
• Create Low Hanging Fruits
• The User Guide
• Documenting the Language Syntax
• The Language Reference
• Debugging for business users
• Creating developer documentation
• Outlining DSL structure
• The syntax implementation
• Keywords
• Behaviors
• Conventions
• Notations
• External Integration
• Documenting AST transformations
• Executable documentation
• Summary

The chapter starts with...

Documentation is a task that most developers strongly dislike. It is treated as a tedious, annoying task and often falls on the developer who protests the least. One additional problem is that a developer trying to document his own work is often not going to do a good work.

This is not an aspiration against developers in general; the problem is that there are too many things that people who actually write the code takes for granted. And even in we ignore that, developers tend to write to developers, in a way that makes little sense to non developers.

That is true for me as well. And trying to document how developers should write documentation is... hard.

At least I can look forward for a really interesting Chapter 12.

I am writing about documentation at the moment, and I found myself writing the following:

I don’t think that I can emphasize enough how important it is to have a good first impression in the DSL. It can literally make or break your project. We should make above reasonable efforts to ensure that the first impression of the user from our system would be positive.

This includes investing time in building good looking UI, and snappy graphics. They might not actually have a lot of value for the project from a technical perspective, not even from the point of day to day usage in some cases, but they are crucially important from social engineering perspective.

A project that looks good is pleasant to use, easier to demo and in general easier to get funding for.

This also includes the documentation, if we can do something in a short amount of time; we get a level of trust from the users. “Hey, I can make it go bang!” is important to gain acceptance. The first stage should be a very easy one, even if you have to design to enable that specifically.

After reading that, I quickly added this as well:

Note, however, that you should be wary of creating a demoware project, one that is strictly focused on demoing well, and not actually add value in real world conditions. Such projects may demo well, and get funding and support, but they tend to fall into the land of tortureware very rapidly, making things harder to do, instead of easier.

Beware of the demoware.

A while ago I talked about persistent DSL caching. I was asked why my solution was not a builtin part of Rhino DSL.

The reason for that is that this is actually a not so simple problem. Let me point out a few of the issues that are non obvious.

• Need to handle removal of scripts
• Need to handle updating scripts
• Need to handle new scripts

Those are easy, sort of, but what about this one?

• Need to handle DSL updates

When you are in development mode, you really need to know that changing the way the DSL behaves would also invalidate any cache.

I like to keep a very high bar of quality on the software I make, and there is a fine distinction between one off attempts and reusable ones. One off attempts can be hackish and stupid. Reusable implementations should be written properly.

And no, there isn't anything overly complex here. Just time to test all bases.

Anyone feels like sumbiting a patch?

I said that I would post about it, so here is the high level design for generic implementation of natural language looking parsing. Let us explore the problem scenario first. We want to be able to build this language, without having to build a full blown language from scratch:

open http://www.ayende.com/
click on link to Blog
click on link to first post
enter comment with name Ayende Rahien and email foo@example.org and url http://www.ayende.com/Blog/
enter comment text This is an awesome post.
click on submit
comment with This is an awesome post should appear on page

And to prove that we are not focusing on a single language, let us try this one as well:

when account balance is 500$ and withdrawal is made of 400$ we should get a low funds alert
when account balance is 500$ and withdrawal is made of 501$ we should deny the transaction
when weekly international charge is at 3,500$ and max weekly international charge is of 5,000$ and new charge arrives for amount 2,230$ we should deny the transaction

I think that those are divergent enough to show that the solution is a generic one.

And now, to the solution. Each type of language is going to have its own DSL engine, which know how to deal with the particular dialect that we are using. The default parsing is a three steps solution. First, split the text into sentences, then, split each sentence to tokens by whitespace. Now, for each statement, we search for the appropriate statement resolve, which is a class that knows how to deal with it. The statement resolver methods are then called to process the statement.

There are two key principal to the design. First, turning something like 'click on link' to an invocation of the ClickOnLink statement resolver and lazy parameter evaluation.

This is going to be interesting, the time right now is 19:38, and I am going to start implementing this.

It is now 22:04, and I finished the first language.

Working on the second now. It is 22:10 and I am done with the second one.

What did I do?

I took the text we had and turn that into executable commands. Now, this isn't flexible at all. If you make a modification in the way it is structured, it will fail, coming back to why natural language is a bad choice here, but it had quite a bit of flexibility in it.

You can get the code for this, including tests, here: https://rhino-tools.svn.sourceforge.net/svnroot/rhino-tools/experiments/natrual-language

But let us talk for a bit about how this is implemented. I'll show the bank example, because it is easier.

We start by defining the BankParser, which looks like this:

The bank parser merely define what the statement resolvers are, and any special parsers that are needed (in this case, we need to handle dollar values).

A statement parser is trivial:

And yes, those are pure POCO classes.

The whole idea here was that I can implement some smarts into the default engine about how it recognize methods and resolve parameters. I will admit that overloading caused some issues, but I think that this is pretty simple implementation.

It also does a good job in demonstrating the problems in such a language. Go ahead and try to build operator precedence into it. Or implement an if statement. You really can't, not without introducing a lot more structure into it. And that would turn it into yet another programming language.

What about the tooling? Intellisense and syntax highlighting?

Well, since we have the structure of the code, and we know the conventions, you shouldn't have a problem taking my previous posts about this and translating them directly into supporting this.

And yes, I can create a language in this in a few minutes, As BankParser has proven.