Oren Eini

CEO of RavenDB

a NoSQL Open Source Document Database

Get in touch with me:

oren@ravendb.net +972 52-548-6969

Posts: 7,573
|
Comments: 51,188
Privacy Policy · Terms
filter by tags archive
time to read 4 min | 642 words

Tim Wilde asked how I would build a DSL, given the example that I gave:

task "warn if website is not alive":
	every 3.Minutes() 
	starting now
	when WebSite("http://example.org").IsAlive == false
	then:
		notify "admin@example.org", "server down!"

Now, I have a small confession to make, I didn't build the DSL for the post, I just thought it up and posted it. Tim has spiked a somewhat nicer implementation of the C# fluent interface, and it was about 250 lines of code, in 8 classes, 6 interfaces.

Face with that, I felt that I had no other choice than to go on and build a DSL implementation. If fact, I think that this is a good exercise of DSL building. We have started from the required syntax, and then we see how we can implement it.

I had to make two modifications to the DSL in order to make it really work, I marked them in red. One was that I wanted to implement the DSL in C#, and it has no way to write an indexed property, which I need in order to create extension properties in Boo, something that Boo completely supports. It was an interesting insight. The second was that the is operator is for reference types, not for values types, so I changed the IsAlive is false to IsAlive == false.

Most of my DSL use the Anonymous Base Class pattern. This means that you need to imagine a base class that wraps the code in this class. This is a really simple DSL, and I wanted to introduce as little complexity as possible, so we just use the basic capabilities of the language.

There are several interesting things that are going on in the DSL. For a start, we are using Boo's ability to drop parenthesizes if needed, which gives us a more "keyword" feeling. Another interesting approach is that when the last parameter of a method is a delegate, we can use just create a block and use that as a way to create the delegate.

So this code:

task "foo":
	print "bar"	

Is the equivalent of this C# code:

task("foo", delegate
{
	Console.WriteLine("bar");
});

This is how "task" and "then" keywords works.

public void task(string name, ActionDelegate taskDelegate)
{
	this.taskName = name;
	taskDelegate();
}
public void then(ActionDelegate actionDelegate)
{
	this.action = actionDelegate;
}

The "when" keyword is a lot more interesting. We want to do late evaluation of the expression that we pass to the when keyword, but the language doesn't allow us to do so. This is where the real power of Boo comes into place, we simply change the language.

[Meta]
public static Expression when(Expression expression)
{
	BlockExpression right = new BlockExpression();
	right.Body.Add(new ReturnStatement(expression));
	return new BinaryExpression(
		BinaryOperatorType.Assign,
		new ReferenceExpression("condition"),
		right
	);
}

This code (which looks much better when it is in Boo, by the way) means that whenever the compiler encounters the result of this statement:

when WebSite("http://example.org").IsAlive == false

Is actually this code:

condition = { return (WebSite("http://example.org").IsAlive == false) }

To my knowledge, this type of transformations is not available in other languages. In fact, while I am certainly not a Ruby expert, I do not believe that this is possible in Ruby. (Let us see if this brings the mob out of the woodworks, shall we?)

For the compiler, the code that we have above will be translated to:

public class validateWebSiteUp(Rhino.DSL.Tests.SchedulingDSL.BaseScheduler):

	public override def Prepare() as void:
		self.task('warn if website is not alive', def ():
			self.every(Rhino.DSL.Tests.SchedulingDSL.BaseScheduler.Minutes(3))
			self.starting(self.now)
			self.condition = { return (WebSite("http://example.org").IsAlive == false) }
			self.then({ self.notify('admin@example.org', 'server down!') })

You can get the source for the DSL here:

https://rhino-tools.svn.sourceforge.net/svnroot/rhino-tools/trunk/rhino-dsl/Rhino.DSL.Tests/SchedulingDSL/BaseScheduler.cs

A sample DSL is here:

https://rhino-tools.svn.sourceforge.net/svnroot/rhino-tools/trunk/rhino-dsl/Rhino.DSL.Tests/SchedulingDSL/ValidateWebSiteUp.boo

Tests:

https://rhino-tools.svn.sourceforge.net/svnroot/rhino-tools/trunk/rhino-dsl/Rhino.DSL.Tests/SchedulingDSL/SchedulingDSLFixture.cs

time to read 1 min | 98 words

I just run into this problem, and I came up with a different reason than the usual one. C# simply doesn't have the concept of indexed properties. This is not legal C# code:

public static string Items[string something]
{
    get { return something; }
}

I think you can do that with VB.Net, and I am certain that C++ supports it.

The interesting part is that I run into it while building a DSL. The limit of the implementation language has actually limited the DSL itself.

time to read 2 min | 295 words

Let us take a simple example, scheduling tasks, and see how we can demonstrate the difference between the two. From the perspective of the client, what do we want from a scheduling tasks?

  • Define named tasks
  • Define what happens when the task is executed
  • Define when a task should execute
  • Define conditions to execute the task
  • Define recurrence pattern

Now, let us look at the fluent interface for this:

new FluentTask("warn if website is not alive")
	.Every( TimeSpan.FromMinutes(3) )
	.StartingFrom( DateTime.Now )
	.When(delegate
	{
		return WebSite("http://example.org").IsAlive == false;
	})
	.Execute(delegate
	{
		Notify(“admin@example.org”, “server down!”);
	});

And contrast that with the DSL sample:

task "warn if website is not alive":
	every 3.Minutes
	starting now
	when WebSite("http://example.org").IsAlive is false
	then:
		notify "admin@example.org", "server down!"

The second version has far less line noise, it a lot more clearer, and allows me to grow my DSL as I get more information, without hitting the language limits.

It also brings to mind another issue, who define these tasks and when?

Most of the time, I use a DSL as a way to setup and configure a runtime engine. I might talk later about imperative vs. declarative DSL. I tend to write my DSL at a different time than the code.

In fact, if you have no need to ever do something externally, and a fluent interface manages to express the ideas that you have clearly enough, than you probably should just use that.

If you need to do things externally, than a DSL is the place to look for. In fact, if you feel the urge to put XML into place, as anything except strict data storage mechanism, that is a place to go with a DSL.

time to read 6 min | 1015 words

I speak quite often about DSL and how to build them, but so far I did not do was explain why you need a DSL at all. After all, since you are reading this blog, you already know how to program. Can’t we just use “normal” programming languages to do the same job? We can even do with a dash of fluent interfaces and Domain Driven Design to make the code easier to read.

We need to inspect the different needs that lead the drive toward a DSL.

A Technical DSL

A technical DSL is supposed to be consumed by someone that understands development. It is meant to express matters in a more concise form, but it is still very much a programming language at heart. The main difference is that it is a language focused on solving the specific problem at hand. As such, it has all the benefits (and drawbacks) of single purpose languages. Examples for technical DSL include Rake, Binsor, Rhino ETL, etc.

The driving force around that is that you want to have richer expressiveness in specifying what you want to happen. Technical DSL are usually easier to write, because your target audience already understands programming.

In fact, the use of programming features can make a DSL very sweet indeed. We have already seen a Rake sample, so let us see a Binsor sample:

for type in Assembly.Load(“MyApplication.Web”).GetTypes():
     continue unless type.IsAssignableFrom(Controller)
     component type.FullName, type

So we take three lines to register all the controllers in the application. That is quite a bit of expressiveness. Of course, it assumes that you are familiar with the API and its usage, which is not always true, which leads nicely to business focused DSL

A Business DSL

A business DSL is focused on being (at least) readable to a businessperson with no background in programming.

This type of DSL is mainly expressive in the terms of the domain, and it has a lot less emphasis on the programming features that may still exists. It is also tend to be much more declarative than technical DSL in general and a lot more emphasis is placed on the nature of the DSL so those would not really be necessary.

I can’t really think of a good example of a business DSL in the open. A good example of DSL that I have run into include a cellular company that had to have some way to handle all the myriad of different contracts and benefits that it had. It also needed to handle this with a fairly rapid time to market, since the company needed to respond quickly to market conditions.

The end result was a DSL in which you could specify the different conditions and their results. For instance, to specify that you get 300 minutes free if you speak over 300 minutes a month, you would write something similar to this:

if CallMinutesForCurrentMonth > 300 and HasBenefit “300 Minutes Free!!!”:
          AddCallMinutes -300, "300 Minutes Free!!!"

It was fairly simple to define a small language that could describe most of the types of benefits that the company wanted to express. The rest was a matter of naming conventions and dropping files in a specified folder, to be picked up and processed at regular intervals. The structure that surrounds a DSL is a subject that deserves quite a bit of attention on its own.

A businessperson may not always be able to write actions using a business DSL (more on that later), but they should be able to read and understand it. After all, it is their business and their domain that you are trying to describe.

Now, why shouldn’t a businessperson be able to write actions using a business DSL?

The main reason, as I see it, is of error handling. No, I don’t mean in the actual running of the DSL action, I mean when writing it.

A DSL is supposed to be read like a language, but it is still a programming language, and those have little tolerance for such thing as bad casing for keywords, for instance. Certain types of users will simply be unable to go over the first hurdle in the road they face, because of this.

It is important to know your audience, and it is even more important not to be contemptuous toward that mythical businessperson. You may not think that this person can understand programming, only to discover that there are quite a bit of automation going on in their life already, powered by VBScript and excel macros.

If you can leverage this knowledge, you have a very powerful combination in your hand, because you can provide that businessperson the tools, and he can provide the knowledge and the required perspective.

Automation DSL

I am not quite sure about this classification, but it certainly has its place. Another name for this may be the IT DSL. This type of a DSL it often used to expose the internal of an application to the outside world.

Modern games are usually engines that are being configured using some sort of a DSL. In fact, I fondly remember building levels in Neverwinters Nights.

More serious usage for this can certainly be found, such as a DSL that lets you go into the internals of an application and mange it. Think about the ability to run a script that will take re-route all traffic from a server, wait for all current work to complete, and then take the server down, update it and bring it up again.

Right now, it is possible to do this with shell scripts of various kinds, but most enterprise application can certainly have more visibility into them than already exist, and a DSL that will allow me to inspect and modify the internal state would be very welcome.

I can certainly think of a few administrators who would be grateful to have more power in their hands.

 

To conclude, can you think of other types of DSL in use?

time to read 2 min | 235 words

Andrey Shchekin made a good point when he commented on my post about graphical DSL:

I do not use DSLs that are purely graphical. But talking about documentation, I think it is very useful to have a graphical _representation_ for some DSLs that is synchronized with (or generated from) the DSL itself.
For example, any workflows or dataflows (think Rhino ETL) are much easier to see at a glance on visual surface. Editing them on visual surface is also an option, but not a requirement.

I certainly am a believer that a DSL should be mostly declarative, and indeed, taking Rhino ETL as an example, it is something that can have a visual representation.

The only remaining question is how?

Long ago there was Lithium and Netron projects, which were good for building the kind of UI that a DSL is composed of. Those have gone commercial now, and while I can probably find the originals somewhere, that is not a very promising concept. There is always the Visual Studio DSL toolkit, but I have a strong suspicion that I would not like it (COM involved), and anyway, I want to do it externally to Visual Studio.

Anyone familiar with useful libraries that can make it happen? Ideally, it would allow me to generate UI similar to the class designer. It should also be simple, but I am not sure how to really quantify that.

time to read 1 min | 183 words

Here is an interesting question, at what point you drive away so far from the original language that you lose the benefits of an internal DSL?

The following examples are several ways to represent the same thing, going from the extreme DSL to the no DSL to C#. I think that they all have value, and neither cross that line, but I believe that they will help make the point. The example here is a a simple MonoRail Controller:

#1:

controller Field:
    action Tag:
           @defaultTags = [ "green", "trees", "growth" ]

#2:

controller Field:
    def Tag():
         @defaultTags = [ "green", "trees", "growth" ]

#3:

controller Field:
    def Tag():
          PropertyBag["defaultTags"] = [ "green", "trees", "growth" ]

#4:

class FieldController(Controller):
   def Tag():
          PropertyBag["defaultTags"] = [ "green", "trees", "growth" ]

#5:

public class FieldController : Controller
{
	public void Tag()
	{
		PropertyBag["defaultTags"] = new string[] { "green", "trees", "growth" };
	}
}

Any comments?

Binsor 2.0

time to read 4 min | 671 words

All credit should go to Craig Neuwirt, for some amazing feats of syntax. He has managed to extend Binsor to include practically all of Windsor's configuration schema. This is important because previously we had to resort to either manual / ugly stuff or go back to XML (see: manual & ugly).

This is important because some of the more interesting things that you can do with Windsor are done using the facilities, and Craig has made sure that Binsor will support the main ones in a natural manner, including out of the box support for the standard configuration model.

Let us start by using the standard configuration model for a moment, the Active Record Integration Facility is expecting to be configured from XML, but we can configure it like this:

facility arintegration, ActiveRecordFacility:
	configuration:
		@isWeb = true, isDebug = true
		assemblies = [ Assembly.Load("My.Assembly") ]
		config(keyvalues, item: add):
			show_sql = true
			command_timeout = 5000
connection.isolation = 'ReadCommitted' cache.use_query_cache = false dialect = 'NHibernate.Dialect.MsSql2005Dialect' connection.provider = 'NHibernate.Connection.DriverConnectionProvider' connection.driver_class = 'NHibernate.Driver.SqlClientDriver' connection.connection_string_name = 'MyDatabase' end end

To compare, here is the equivalent  XML configuration.

<facility id="arintegration" 
	type="Castle.Facilities.ActiveRecordIntegration.ActiveRecordFacility, Castle.Facilities.ActiveRecordIntegration" 
	isWeb="true" 
	createSchema="true">
 <assemblies>
	<item>My.Assembly</item>
 </assemblies>
 <config>
	<add key="hibernate.cache.use_query_cache" value="true" />
	<add key="hibernate.connection.isolation" value="ReadCommitted" />
	<add key="hibernate.show_sql" value="false" />
	<add key="hibernate.dialect" value="NHibernate.Dialect.MsSql2005Dialect" />
	<add key="hibernate.connection.driver_class" value="NHibernate.Driver.SqlClientDriver" />
	<add key="hibernate.connection.connection_string_name" value="MyDatabase" />
	<add key="hibernate.connection.provider" value="NHibernate.Connection.DriverConnectionProvider" />
 </config>
</facility>

We don't have any significant reduction in the number of lines. This is mostly because we need to specify a lot of items for the facility to work, but we do have significant reduction in the noise that we have to deal with.

But, frankly, this isn't the best sample, let us take a look at the shortcuts that Binsor now provides, shall we?

Event Wiring

XML configuration:

<component 
	id="SimpleListener" 
      type="Castle.Facilities.EventWiring.Tests.Model.SimpleListener, Castle.Facilities.EventWiring.Tests" />

<component 
	id="SimplePublisher" 
      type="Castle.Facilities.EventWiring.Tests.Model.SimplePublisher, Castle.Facilities.EventWiring.Tests" >
      <subscribers>
      	<subscriber id="SimpleListener" event="Event" handler="OnPublish"/>
	</subscribers>
</component>

And the Binsor configuration to match it:

component simple_listener, SimpleListener

component simple_publisher, SimplePublisher:
	wireEvent Publish:
		to @simple_listener.OnPublish

Now that is far clearer, isn't it?

Factory Support

XML configuration:

<component id="mycompfactory" 
      type="Company.Components.MyCompFactory, Company.Components"/>

<component id="mycomp" 
	type="Company.Components.MyComp, Company.Components"
	factoryId="mycompfactory" 
	factoryCreate="Create" />

Binsor configuration:

component mycompfactory, MyCompFactory

component mycomp, MyComp:
	createUsing @mycompfactory.Create

Other new features:

  • Integrating Castle Resources, so now we can use Binsor over several files, which can be located anywhere, including assembly resources, for instance.
  • Easy support for startable and life styles.
  • Extension points for future needs.
time to read 2 min | 391 words

You were just handed a strange DSL implementation, it does stuff, and it may be cool, but you have no idea how it works.

Craig Neuwirt recently did a major overhaul of Windsor ( I am going to post soon with the details of how cool it is now ), but I am now faced with the question, how do you grok such a thing? I thought that it would be useful to put a list of what I am doing to understand how the internals now works.

It goes without saying, but the nitpickers will ask, that a DSL implementation is code like any other, and as such, it can have good & bad implementations. I think that what Craig has done was amazing. It makes for an interesting reading, I actually had to take notes, to make sure that I will not miss any of the cool stuff.

  • Understand the domain.
    This is critical, because if the DSL is chicken scratching to you, you won't be able to get the implementation.
  • Get the tests.
    First things first, make sure that you have tests that ensure that you can understand what the DSL is supposed to do. In addition to that, it allows you to debug a simplified scenario repeatedly, so you can walk through that and figure out what is going on.
  • Take the simplest scenario and run it, then open the resulting DLL in Reflector
    This is important, because you need to be able to see what the differences are between the DSL and the actual executed code are. This allows you to understand in a deep level what the various elements of the language do.
  • Identify a transformation, and follow how it is being built throughout the code
    It will allow you to understand the concepts and patterns used through the code.
  • Follow each transformation throughout its own code path only
    Fairly often you can get side tracked by trying to put the entire program in your head, this is usually not possible, track a single path through the code to completion. After that, you can start tracking other paths, but it is important to have an understanding of the full cycle, and then expand from that.

Hm, not really that different from how I would approach an unfamiliar code base, come to think about it.

FUTURE POSTS

  1. RavenDB Storage Provider for Orleans - 41 minutes from now
  2. Making the costs visible, then fixing them - 2 days from now
  3. Scaling HNSW in RavenDB: Optimizing for inadequate hardware - 4 days from now
  4. Optimizing the cost of clearing a set - 7 days from now

There are posts all the way to May 12, 2025

RECENT SERIES

  1. RavenDB News (2):
    02 May 2025 - May 2025
  2. Recording (15):
    30 Apr 2025 - Practical AI Integration with RavenDB
  3. Production Postmortem (52):
    07 Apr 2025 - The race condition in the interlock
  4. RavenDB (13):
    02 Apr 2025 - .NET Aspire integration
  5. RavenDB 7.1 (6):
    18 Mar 2025 - One IO Ring to rule them all
View all series

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats
}