Ayende @ Rahien

Refunds available at head office

What is wrong with this code?

There is a huge bug in this code, resulting in data corruption. Can you spot what it is?

public byte[] DownloadBytes(string url,
                            ICredentials credentials)
{
    WebRequest request = Util.SetupWebRequest(WebRequest.Create(url), credentials);

    using (WebResponse response = request.GetResponse())
    {
        using (Stream stream = GetResponseStream(response))
        {
            byte[] buffer = new byte[response.ContentLength];
            int current = 0;
            int read;
            do
            {
                read = stream.Read(buffer, current, buffer.Length - current);
                current += read;
            } while (read != 0);
            return buffer;
        }
    }
}

Hint, this has nothing to do with exception handling. Assumes that nothing goes wrong.

Find out the right directory

Another interesting challenge. Given the following API:

public interface IFileRepository
{
	void Save(IFile file);

	IFile[] GetFiles(string path, Recursion recursion);
}

public enum Recursion
{
	None,
	OneLevel,
	Full
}

And assuming that you are saving to a relational database, how are you going to build the GetFiles method?

Oh, and just to make things interesting, Save(IFile) may only perform a single SQL INSERT statement.

Challenge: Strongly typing weakly typed code

How would you make the following code work?

public static class Security
{
	public static string GetDescription(Type entityType, Guid securityKey)
	{
		Guard.Against<ArgumentException>(securityKey == Guid.Empty, "Security Key cannot be empty");
		IEntityInformationExtractor<TEntity> extractor = IoC.Resolve<IEntityInformationExtractor<TEntity>>();
		return extractor.GetDescription(securityKey);
	}
}

You can't change the entity type parameter to a generic parameter, because you only know about it at runtime. This is usually called with:

Security.GetDescription(Type.GetType(permission.EntityTypeName), permission.EntitySecurityKey.Value);

Count the interfaces...

Without running this code, what does this print?

public interface IStartable : IDisposable
{
   void Start();
}

public class Pipeline : IStartable
{ 
  // ...
}

Console.WriteLine(typeof(Pipeline).GetInterfaces().Length);

Riddle me this...

What is the result of this piece of code?

application.Startup += delegate
{
    Console.WriteLine("will it");
    application.Startup += delegate
    {
        Console.WriteLine("work?");
    };
};

As usual, I am raffling a Rhino Mocks license among those who will answer correctly.

Scheduled Tasks in MonoRail: The Quick & Dirty solution

I need to develop a set of tasks that would run in internals. Now, we need to do several dozens of those, and at that point, it is not important how we actually schedule them, we can defer that decision. But we really need to be able to start develop them soon.

So, I came up with this idea. The basic structure is this:

[OccuresEvery(Occurances.Day)]
public class SendBirthdayEmails : ScheduledTask
{
	public override void Execute()
	{
		foreach(Employee emp in Repository<Employee>.FindAll(Where.Employee.Birthday == DateTime.Today)
		{
			Email
				.From(Settings.Default.HumanResourcesEmail)
				.To(emp.Email)
				.Template(Templates.Email)
				.Parameter("employee", emp)
			.Send();
		}
	}
}

This is not really interesting, but the rest is. Remember that I don't want to deal with deciding how to actually schedule them, but we need to be able to run them right now for test / debug / demo purposes.

In my windsor.boo:

//Controllers
controllersAssembly = Assembly.Load("MyApp.Web")
for type in controllersAssembly.GetTypes():
	continue if type.Name == "ScheduledTasksController"
	continue if not typeof(Controller).IsAssignableFrom(type)
	IoC.Container.AddComponent(type.FullName, type)

//register scheduled tasks
scheduledTasksAssembly = Assembly.Load("MyApp.ScheduledTasks")
scheduledTasks = []
for type in scheduledTasksAssembly.GetTypes():
	continue if not typeof(ScheduledTask).IsAssignableFrom(type)
	IoC.Container.AddComponent(type.FullName, type)
	scheduledTasks.Add(type)
//register scheduled tasks controller independently, since it requires special configuration 
Component("ScheduledTasksController", ScheduledTasksController,
	scheduledTasks: scheduledTasks.ToArray(Type) )

So it will automatically scan the scheduled tasks assembly, and the only thing that I have left is to write the ScheduledTasksController. This is a very simple one:

image

It has just two methods, one to list all the tasks, and the second to execute them. This is strictly a dev only part of the application, so I took most of the available shortcuts that I could. So the UI looks like this:

image

And the view code is:

<% for task in tasks: %>
<tr>
	<td>
	${Text.PascalCaseToWord(task.Name)}
	</td>
	<td>
	Every ${task.OccuranceEvery}
	</td>
	<td>
	${Html.LinkTo("Execute", "ScheduledTasks", "Execute", task.FullName)}
	</td>
</tr>
<% end %>

I really like the PascalCaseToWord helper, really nice.

On the controller's side of things, I have this:

public ScheduledTasksController(Type[] scheduledTasks)
{
	scheduledTaskDescriptions = new ScheduledTaskDescription[scheduledTasks.Length];
	for (int i = 0; i < scheduledTasks.Length; i++)
	{
		scheduledTaskDescriptions[i] = new ScheduledTaskDescription(scheduledTasks[i]);
	}
}

public void Index()
{
	PropertyBag["tasks"] = scheduledTaskDescriptions;
}

Not a best practice code, but I did knock the whole thing very quickly. ScheduledTaskDescription just takes a type and unpack it in terms of attributes, name, etc.

The end result is that the other developers on my team can add a new scheduled task by simply adding a class that inherits from ScheduledTask, go to the browser, hit F5 and start executing it.

Now that is RAD.

Anti Corruption Layers: Striving for FizzBuzz level

I think that I mentioned that I don't really like Microsoft CRM development options. Considering the typical quality of the code that I see online when I search for samples, I certainly see the CRM as corrupting influence.

That is why I pulled the big guns and built a whole new layer on top of it. I assume that you are already aware of my... reservations for leaky abstractions, and considering my relative lack of expertise on the CRM itself, I don't think that it would have been wise to diverge too far from the model that the CRM has, only to break the way we are handling it.

As a result of that, we have this piece of code, which is how I handles tasks that are usually handled by callouts. The major advantage is with the ability to develop on the local machine without being tied to the server, and a lot of smarts with regards to banishing XML parsing from the core business logic. The end result, as far as I am concerned, is that I should be able to write business logic using Fizz Buzz level of code, if that. It is a bit mind numbing, but that is what the client wants, not to struggle mightily with the underlying platform.

[Handles(CrmOperations.PreCreate)]
public class OnCreateFriend_SetFullName : NorthwindCommand<new_friend>
{
	public override void Execute()
	{
		if(string.IsNullOrEmpty(PreState.new_name)==false)
			return;
		PreState.new_name = PreState.new_firstname + " " + PreState.new_lastname;
	}
}

 And to foretell questions, the basis for that is not Open Source, although we may make a product out of it at some point. (Drop me a line if you are interested).

Can you figure it out? Functional caching

Okay, here is a challenge, without running the code, what do you think should be the result of this program, and why?

public class Program
{
	private static void Main(string[] args)
	{
		Fetch<int> method = Get;
		method = AddCaching(method);

		method();
		method();
		method();
	}

	public static int Get()
	{
		Console.WriteLine("Called");
		return 1;
	}

	public delegate T Fetch<T>();

	public static Fetch<T> AddCaching<T>(Fetch<T> fetch)
	{
		T cachedObject = default(T);
		bool isCached = false;
		return delegate
		{
			if (isCached == false)
			{
				cachedObject = fetch();
				isCached = true;
			}
			return cachedObject;
		};
	}
}

Challenge: Windsor Null Object Dependency Facility

In Scott Bellware's post about dependency injection, he has a comment about optional dependencies:

Setter dependencies are optional.  Their types should have a Null Object pattern implementation, and if not it's often a good idea to decorate them with one.  For example, if an optional dependency hasn't been set, it might not be desirable to have a null reference exception when a method is invoked on it

The challenge is to provide a facility for Windsor that will detect an optional dependency and fill it with a Null Object (i.e, an implementation that doesn't do anything). It is safe to assume that all such dependencies are using interfaces.

Can you do it?

Answering Mats' Challenge

Mats Helander has a challenge for OR/M developers, and Mats should know, since he is behind NPersist.

Go for the post for details, but basically it is loading all Customer->Orders->OrderLines graph in 3 statements or less.

Because I am a sucker for challenges, I implemented it with ActiveRecord. In Mats' terms, the code is so simple it hurts, and yes, I cheated :-)

internal static IList<Customer> LoadCustomersOrdersAndOrderLines()

{

    Customer[] customers = Customer.FindAll();

    Order[] orders = Order.FindAll();

    OrderLine[] orderLines = OrderLine.FindAll();

    foreach (Customer customer in customers)

    {

        customer.Orders = new List<Order>();//avoid lazy load when adding

    }

 

    foreach (Order order in orders)

    {

        order.OrderLines = new List<OrderLine>();//avoid lazy load when adding

        order.Customer.Orders.Add(order);

    }

 

    foreach (OrderLine line in orderLines)

    {

        line.Order.OrderLines.Add(line);

    }

    return customers;

}

 

Understanding Bad Code

Frans' contribution to the conversation about maintainable code deserve its own post, but I would like to mention this part in particular:

you will not [understand the code]. Not now, not ever. And not only you, but everyone out there who writes code, thus that includes me as well, will not be able to read code and understand it immediately.

You know what? That is not limited to bad code. I had hard time grokking good code bases, simply because of their size and complexity (NHibernate and Windsor comes to mind). Other code bases are as large, but they have easier approachability, probably because they are dealing with less complex domain an tend to have a wide coverage rather than deep (MonoRail comes to mind).

A while ago I was involved with an effort to migrate an ancient system to SQL Server 2005. The system compromised of over 100,000 lines of code, spread over some thousands of files scattered randomly in a case sensitive file system (you can guess why this is significant). The code base was about 85% SQL and 15% bash shell scripts. The database in question was a core system and contained slightly over 4,000 tables. One of the core tables was called tmp1_PlcyDma and was used to do business critical processing. That code base took data driven code generation to a level I have never seen before. I gave up trying to track down 7(!) levels of code->generating code->execute code->generating code->rinse->repeat

To say that the code base was bad is quite an understatement. To mention that the only place where I could run the code was by using a telnet console into a test environment that was not identical to production is only the start. I could mention no debugging, runtime of ~5 hours, test time of ~3 hours, etc. The code grew organically over a ten years period and you could track the developer progress from merely annoying to criminally insane (he invented his own group by construct, using triple nested cursors and syntax so obscure that even the DBA that worked with the system for the last 5 years had no idea what was going on).

Perhaps the thing that I remember most from this project that we had a bug that kept two people hunting after it for three weeks. The issue was a missing ';'. Oh, and the criteria for success in this project was successful migration, with bug-per-bug compatibility, and no one really knew what it did, including the authors(!).

But, you know what, after a month or so of looking at the code, it got to the point where I could look at something like pc_cl_mn.sql and know that it would contains the monthly policy calculation, and that this piece of code was doing joins manually via cursors again, that plcy_tr_tmp.sql was the "indexes priming" script, etc, etc.

The code was still a horror, but once you understood that the authors of this code had a... "special" way of looking at databases, you got to the point where you could get the point of the code in an hour instead of a day, and then move that to a saner approach.

So, what does this horrifying story has to do with Frans' point above?

The premise that you can read and understand code immediately is highly dependant on what you are familiar with. I know of no one that can just sit in front of an unfamiliar  code base and start producing value within the first ten minutes. But on a good code base, you should be able to be able to start producing value very quickly.

The right UI metaphor

imageAfter some discussion about whatever a tree is the correct UI to show the use for my permissions issue, I decided to see if there is another way to handle that.

This is not the way the UI looks like, obviously, but it should give you an indication about how it works. Gray check mark means that something below this level is checked, a check mark means that it has permission on this node. Note that I can have permission on a node, and permission on sub nodes, and it has different meanings. If I have a permission on a node I have cascading permission to all its children, but a child may be associated with multiply parents (and not always at the same level of the tree, sigh).

In this case, we have Baron that has permission to schedule work for the Tel Aviv's help desk stuff. He also has permissions to schedule work to John & Barbara, no matter in what capacity.

It other words, even thought Baron can assign work to Jet, he can only do so when Jet works for the Tel Aviv Help Desk,  he cannot assign Jet to work in the Pesky Programmers role. He can do that to John & Barbara (assuming he has the rights to do assign work on the Pesky Programmers department, of course, which is another tree.

The idea is that you can assign detailed permissions to any parts of the tree that you are interested in. There is another screen that allows you to find the hierarchy of objects if you are really interested (not shown here).

Naturally, permissions are many to many, the tree is many to many and I have a headache just try to figure it out. Just to point out, this is done on a web application, and the complexity is that the real tree has about two thousands entries at the lowest level (and ~7 at the top most level), so you need to get data lazily from the server, but, you also need to display the grayed check box, so the user will know that a child node is marked, that was the main difficulty, actually.

So, I am open for ideas about how to design this better.

How complex can this be?

TriStateTree.png How long would it take you to implement the functionality to the left?

I have already spent over a day implementing a tri state tree. The scary part is that I have not even tried to build the tree myself, it was a completely innocent chain of events that led to this stage.

Let me explain a bit about what I am trying to do first. It is a tree that shows the assoication of an entity to a hierarchy. To make a concrete example, let us say that we have an employee entity that should have responsability on a part of the hierarchy. The hierarchy is 5 levels deep, and fairly big.

At first, I tried to simply use the ASP.Net Tree View. That didn't work very well. Too much data was sent to the client. So I needed to do lazy load of the tree, using Ajax. The ASP.Net tree view supports that, but, to my utter lack of amazement, not in a way that allows me to plug in easily and handle additional concerns after the load has happened.

Time to write my own tree :-(, that isn't very hard, actually, there are numerous examples of tree around, and it was fairly easy to turn MkTree into an ajaxian one. Now I had the proper extension points, and I could turn my mind to the other concerns. As you can see in the picture, there is a checkbox that indicate whatever the user is associated with this node or not. So far, not so hard, I can handle that with ease.

The problem is that I need to handle the third state, where the assoication is in a deeper level, and still show it to the user. So, in a 5 level hierarchy, where the user can be associated to any of the levels (many to many, btw), I need to find out if it is associated with this node or any of its children.

17 left outer joins later (and no, this isn't just a number, I really got to that), I realized that I have no idea what I am doing and that I probably want to stop before I reach the triple digit joins statement (don't ask, but it was a hand written query, to stem the OR/M sucks complaints) again.

Solved that (with MutliQuery and by thinking about it from the other end), and found out that the hierarchy is so nicely real world, so you can have a 4th level node that doesn't have a 3rd level node, which should still go under its grandfather on the 2nd level. And if that is not enough the next issue in line is solving the issue of editing a partially loaded tree, what do I do with the unloaded state? Yuck!

Right now, it is compromised of:

  • User Control
  • Two custom server controls
  • Web Service
  • JavaScript on the client side
  • Controller
  • Create querying with NHibernate
  • Complex UI interactions

It is a %$#@$& tri state tree view, for crying out load, and I can't believe how complex this has turned out to be. And it is not over yet.

Web Forms are killing me

I would preface by saying that I am using Web Forms at my current project because the client insisted. They are paying, they get to make the decisions, end of story.

This is a rant, plain and simple. I don't do those often, but I have too much annoyances lately to not do it.

I have several years of working with Web Forms, in projects of various sizes, I have committed my Sins Against the GridView, and have memorized the page lifecycle, I have walked the path of Page.ProcessRequest, and learnt at a distant monastery about the mysteries of the ViewState, I have studied the intricate play of control's events and the secrets of Data Binding.

I get Web Forms. I know how they work.

I have detailed elsewhere what I don't like about WebForms (trying to be stateful, complex, lying, etc). Today it had gotten to the point when I have given up on trying anymore. This was after we have been hitting, repeatedly, obstacle after obstacle with using Web Forms.

Today we had no less than three such incidents, all of them related to various intricate ways that the whole messy pipeline is working together to create a results that is more complex than the sum of all its parts and the legacy systems on the next door. Today was far from being an exception.

We didn't even try to do anything complex. Take a bunch of objects, bind them to a GridView, rinse-repeat. There was a set of pages that were almost literally forms over data, exactly the use case for WebForms, or so I have thought. In a set of about a dozen pages, with really minor differences between one another, we had at least one serious problem at each page. Invariably it was related to some weird way WebForms expected me to work.

In order to get the data from the request, I need to bind the data again in page load, and in a later event, I need to process it, after the Web Forms engine had magically filled the values from the request. And woe upon you if you dare to mix a data source with a call to this.DataBind(); for you are heretic and deserve to lose the user's input without warning, and only by the grace of the good lord will you be saved.

At one point today I resorted to an http handler and a set of calls to Response.Writer(), because that was easier than trying to get the functionality from the WebForms engine.

At this point, I feel like I am trying to build on top of a house of cards, with the slightest movement will send everything crumbling down. I found a bug that was totally my fault today, and we had about five minutes of hilarity about finally finding something that we could easily fix. While this is a very instructive in terms of knowing how the internals of the WebForms engine works, it is a complete waste of time otherwise. When I need to be absolutely aware, at all times, about all the implications of what is going on all around me, I can't really do anything at all.

I have learned my lessons from past projects, thou shall not try to be smart with the WebForms framework, it will be smart right back at you and bit you in the ass. I am not trying to do anything smart here. I am literally scaling down everything that I do on the UI layer to the set of what is wholly approved and blessed by Microsoft. I am not even trying to workaround technical limitation, I am just trying to build working software.

At this point, I am more willing to debug Dynamic Proxy's IL generation engine than my pages, because at least with Dynamic Proxy, I have some control over what is going on, hard as it may be. The WebForms engine gives the illusion of control, but it does things in completely arbitrary ways, contrary to the principal of least surprised and common sense.

I am sick of it.

Dynamic Proxy 2: Mixins

I needed a rest from dealing with SSIS and data migrations issues, so I decided to put some time in real code. I just finished adding support for Mixins to Dynamic Proxy 2. Here is the first test that passed, the name should tell you quite a bit about what is required to make it work.

[Test]

public void CanCreateSimpleMixinWithoutGettingExecutionEngineExceptionsOrBadImageExceptions()

{

       ProxyGenerationOptions proxyGenerationOptions = new ProxyGenerationOptions();

       proxyGenerationOptions.AddMixinInstance(new SimpleMixin());

       object proxy = generator.CreateClassProxy(

              typeof(object), proxyGenerationOptions, new AssertInvocationInterceptor());

 

       Assert.IsTrue(proxy is ISimpleMixin);

 

       ((ISimpleMixin)proxy).DoSomething();

}

The generated proxy looks something like this one:

public class ObjectProxyefb7dccd21fe43b5b2d13c788dce3bdb : ISimpleMixin

{

    public IInterceptor[] __interceptors;

    public ISimpleMixin __mixin_Castle_DynamicProxy_Test_Mixins_ISimpleMixin;

    public static MethodInfo tokenCache1 = ((MethodInfo) methodof(ISimpleMixin.DoSomething, ISimpleMixin));

    public static Type typeTokenCache = typeof(object);

 

    public ObjectProxyefb7dccd21fe43b5b2d13c788dce3bdb(ISimpleMixin mixin1, IInterceptor[] interceptorArray1)

    {

        this.__mixin_Castle_DynamicProxy_Test_Mixins_ISimpleMixin = mixin1;

        this.__interceptors = interceptorArray1;

    }

 

    public override int DoSomething()

    {

        object[] objArray = new object[0];

        InvocationDoSomething_1 g_ = new InvocationDoSomething_1(
            
this.__mixin_Castle_DynamicProxy_Test_Mixins_ISimpleMixin,
             this.__interceptors, typeTokenCache, tokenCache1, objArray, this);

        g_.Proceed();

        return (int) g_.ReturnValue;

    }

 

    [Serializable]

    public sealed class InvocationDoSomething_1 : AbstractInvocation

    {

        public ISimpleMixin target;

 

        public InvocationDoSomething_1(ISimpleMixin mixin1, IInterceptor[] interceptorArray1, Type type1, MethodInfo info1,
              
object[] objArray1, object obj1) : base(mixin1, obj1, interceptorArray1, type1, info1, objArray1)

        {

            this.target = mixin1;

        }

 

        public sealed override void InvokeMethodOnTarget()

        {

            int num = this.target.DoSomething();

            base.ReturnValue = num;

        }

    }

}

 

The code is in the repository, if you feel like taking it out for a spin.

A Challange: Simple HR Model + Rules

It seems like no one is ready to take my Linq Challange, so I decided to expand it a bit and meet my own standards. I decided to implement the challange in Active Record, since it is much faster to work with than NHibernate. Note that this is just the object model and the mapping, nothing more.

Overall, this is 800 lines of code and 20 lines of configuration, and I think that it took about three to four hours to build this demo.

Here is the object diagram:

OhMy.png

And the database model:

database.PNG

You can download the sample project here, I included a SQL Script to create the database in the file.

Just to note, this is not a toy sample, this is a complex model that is capable of expressing very annoying business requirements. I added a bit of recursive rules, just to make it a bit more realistic. The real project has a lot more stuff to make it easier to work with the model, but in essense, this is the way that I built the project.

Just to clarify, I am truly interested in seeing other solutions to this problem, preferably not ones in NHibernate / Active Record. Mainly because I think that this is complex enough to be a real world problem and not something that can be solved with demo code.

Linq Challange: HR Sample Model

Okay, here is an interesting challange spurred by a a comment Alex has left on my previous post about linq.

Given this model, how would you build it using Linq for SQL (is it possible?) and Linq for Entities? I don't really care about the table layout (in other words, feel free to build something that can make your life easier), only that the model will be able to express the required complexity cleanly. A couple of notes, this is a temporal model, which include most of the usual database semantics (1:M, N:M, M:1).

The two gotchas for the OR/M implementation is that Rule is an abstract class that has several implementations, and that a rule is always attached to an entity (which may be of several urelated type Contract/Employee/Department, etc).

I would be interested to see how both Linq implementations handle this task...

Any takers?