Ayende @ Rahien

It's a girl

Slicing & Dicing Queries with NHibernate

This was how I started the day, interesting problem, and I like the solution:

DetachedCriteria getMaxScanDateForScan = DetachedCriteria.For<ScanResult>()
	.SetProjection(Projections.Max("ScanDate"))
	.Add(Property.ForName("ScanUri").EqProperty("scan.Id"));
DetachedCriteria hasResults = DetachedCriteria.For<ScanResult>()
   .SetProjection(Projections.Id())
   .Add(Property.ForName("ScanUri").EqProperty("scan.Id"));

criteria
	.CreateCriteria("Results", "result", JoinType.LeftOuterJoin)
	.SetProjection(
	Projections.ProjectionList()
		.Add(Projections.Property("scan.Id"), "Id")
		.Add(Projections.Property("scan.Uri"), "Uri")
		.Add(Projections.Property("scan.DaysNotice"), "DaysNotice")
		.Add(Projections.Property("scan.EnableScan"), "EnableScan")
		.Add(Projections.Property("result.ScanDate"), "LastScan")
		.Add(Projections.Property("result.Status"), "Status")
		.Add(Projections.Property("result.Message"), "Message")
	)
	.Add(Expression.Disjunction()
		.Add(Subqueries.PropertyEq("result.ScanDate", getMaxScanDateForScan))
		.Add(Subqueries.NotExists(hasResults))
	);

IList list = criteria
		.GetExecutableCriteria(session)
		.SetResultTransformer(new AliasToBeanResultTransformer(typeof (ScanURIReport)))
		.List();

Mixed Mode Authentication

I just finished spending half a day implementing support for mixed mode authentication in my application. I am putting it here mostly to remind me how it is done, since it was a pretty involved process.

As usual, the requirement was that most of the users would use Windows Authentication in a single sign on fashion, and some users would get a login screen.

I am using Forms Authentication, and I want to keep is as simple as possible. After some searching, it seems that most of the advice on the web seems to include building two sites, and transfering credentials between the sites.

Perhaps the best places to look for it is this image, that explains how Forms Authentication works, and these two posts from Craig Andera #1, #2. After reading those, I had a much better picture of what I needed to do.

This requires several steps that are supposed to be self coordinated in order for it to work:

  • Setup IIS for Anonmous + Integrated security.
  • In the web.config, specify forms authentication.
  • In the Login Controller, you need to check whatever the user is a candidate or windows authentication. In my case, it is decided according to IP ranges, but your case may be different.
  • If the user can do windows authentication, you need to send back a 401 status code.
  • Here is where it get a bit hairy. You can't do it from the login controller, because the FormsAuthentication Module will intercept that and turn it into a redirect to the login page.
  • You need to create a http module and register it last in the HttpModule sections, and there you can specify the 401 status code safely. I used the HttpContext.Items to transfer that request.
  • After the request has returned, if Windows Authenticaton has been successful, you can access the user name in the ServerVariables["LOGON_USER"].
  • Create a Form Authentication cookie as usual, and carry on with your life.

The main issue here, as far as I am concerned was to make sure that I will do it in a way that is maintainable, there are several disparate actions that are taking place that are all needed to make it work.

It took a while before I could get to grip with what was going on the wire, so here is the code for this:

private void HandleInternalUsersAndSingleSignOn()
{
	//semi-internal users and the like
	if(Context.ClientIpIsIn(Settings.Default.ExcludeFromWindowsAuthetication))
		return;
	string logonUser = Context.GetLogonUser();
	if(string.IsNullOrEmpty(logonUser))
	{
		//Internal installation and an empty user means
		//that we have not yet auttenticated, we register a request
		//to send a 401 (Unauthorized) to the client, so it will provide
		//us with their windows credentials.
		//we have to register a request for 401, which will be handled by the
		//Add401StatusCodeHttpModule on the EndRequest because we are also using
		//FormsAuthentication, and we need to bypass the FormAuthentication interception
		//of 401 status code.
		Context.SetContextVaraible(Constants.Request401StatusCode, true);
		Context.EndResponse();
	}
	// will redirect to destination page if successful
	LoginAuthenticatedUser(logonUser);
}

Placing complexity: Localized Complexity Zone

Another semi-post morten talk that we had today had to do about complexity. A co-worker has just found my windsor.boo file, and commented on the complexity that it invovles. We had a discussion about this. I aggreed with her that this is indeed complex, but, in over eight months of the  project, it was the irst time that someone other than me had to deal with it, and for the rest of the team it Just Worked.

Side note: one of the things that I am going to make sure for the next project is make sure that the entire team has at least a rudimetry understanding of all the important infrastructure pieces.

It got us to a discussion of whatever we can reduce the complexity. Considering what we need to handle in this project, we both agreed that we could not significantly reduce the complexity of the system and that the alternative were either going with a localized complexity zone or spreading it around the entire code base if we removed this functionality.

We couldn't reduce complexity, we could only move it around. By certralizing it in a single place we increased the local complexity of a single part of the application (not the complexity of the code, but the cmpliexity in understanding what is going there, IoC & DI ). But trade off is that we made the rest of the code of the system much easier to work with and understand.

What do you think?

Developing on dotOren

I mentioned that we are on the closing of a project, and we have already started to work on side projects and preparing for the next one.

In the current project, I have setup a lot of the inrasturcture myself, and then the team started to work on it. I had a discussion with a co worker today about it, she had a lot of frustration because she couldn't work on the side project the way she was used to in this project.

We setup most of everything that she needed, but one thing that she said stuck with me:

I don't know how to develop on .NET, I know how to develop on .OREN

I found it funny, but we still scheduled training sessions...

The cost of adding a field

I was asked to add a few fields to a report that we are showing, I had to touch the following parts of the system:

  • Database table
  • SSIS
  • Entity
  • Report page
  • Export to excel

I spent about 20% of the time in SSIS and another 70% in trying to fit the new fields into the current page without causing horizional scroll bar.

Pretty good balance, I would say.

From my mail box, Rhino Mocks bugs

I need about 3 hours to sit on those, and I can't seem to find the time.

image

I take comfort that at least they are interesting bugs (my keyword for "I don't have a clue what to do here").

Microsoft CRM: Error reporting from the stone age

image

I have said, more than once, that the #1 reason that I like .NET is that I get good error handling.

Here is the current error that I have to deal with from the CRM.

I had to dig into the three separate log files, the event log, restart services and howl at the moon to figure out what is wrong.

 

 

 

BLAH!

Lazy Property Loading In NHibernate

Well, I am not going to give the implementation here, but here are some notes for the implementation. What you basically need is actually quite simple.

  • Start with a partial query, as shown here.
  • Instead of returning the object, return a proxy to the object, that is aware of which properties were loaded.
  • When a property is accessed that is not in the loaded properties list, do the following:
    • Load the real object using NHibernate.
    • Copy the existing values from the current object to the newly loaded one. (Preserving state)
    • Redirect all calls to the newly loaded object.

The whole thing is quite simple, once you think about it from the right angle. Then again, here talks the guy who spent two hours debugging why calling the wrong machine would fail.

Partial Object Queries With NHibernate

Aaron still wants partial object queries, so I set up to build them using NHibernate. Here is the implementation, notice that this query will result in a list of Blog instances, but the select will only include their titles & subtitles.

using (ISession session = sessionFactory.OpenSession())
{
	TupleToPropertyResultTransformer transformer = 
		new TupleToPropertyResultTransformer(typeof(Blog),"Title", "Subtitle");
	IList list = session.CreateQuery("select b.Title, b.Subtitle from Blog b")
		.SetResultTransformer(transformer)
		.List();
	foreach (Blog b in list)
	{
		System.Console.WriteLine("Blog: {0} - {1}", b.Title, b.Subtitle);
	}
}

But where does TupleToPropertyResultTransformer comes from, well, that is where the magic comes in, here is my implementation for it:

public class TupleToPropertyResultTransformer : IResultTransformer
{
	private Type result;
	private PropertyInfo[] properties;

	public TupleToPropertyResultTransformer(Type result, params string[] names)
	{
		this.result = result;
		List<PropertyInfo> props = new List<PropertyInfo>();
		foreach (string name in names)
		{
			props.Add(result.GetProperty(name));	
		}
		properties = props.ToArray();
	}

	public object TransformTuple(object[] tuple, string[] aliases)
	{
		object instance = Activator.CreateInstance(result);
		for (int i = 0; i < tuple.Length; i++)
		{
			properties[i].SetValue(instance, tuple[i], null);
		}
		return instance;
	}

	public IList TransformList(IList collection)
	{
		return collection;
	}
}

This isn't the most optimized version that you can think of, but it does the job.

I want to make it clear, however, that I feel that doing stuff like this is not something that I would consider to be a best practice. Quite the opposite, frankly. What we have here is an object in a state that it was never intended to be, with only part of its fields filled, and certainly not based on any logic. I would much rather see a DTO class take its place, because that has a clear responsibility in the application, reusing your entities as dumb data container is not something that I would recommend.

In the authentication maze

image

Well, it looks like I have once again managed to put myself in a tricky spot. I have the following scenario:

  • Domains: A1 & A2 - no trust or any association between the two.
  • A user access a Website on A1, using windows authentication, which makes a web service call to a machine on domain A2 (anonymous security, at the moment).
  • As a result of the web service call, the machine on A2 needs to make another web service call to A1, and it needs to do it with windows authentication, with the credentials of the original user.

Now, I have control on the machine in A1 (.Net 1.1 ASMX WebServices) and I can do whatever I want to the machine in A2 (WCF). No trust between the domains, as I said, so I don't think that I can make Windows Authentication works between the two. Frankly, I don't care about authenticating users, I just need their credentials when I am going back to the machine in A1.

Any suggestions?

Optimizing NHibernate

Aaron (Eleutian) is talking about some issues that he has with optimizing with NHibernate.

So in short, I feel NHibernate (and any ORM for that matter) needs the following features to really be optimization friendly:

  • Lazy field initialization
  • Querying for partial objects: select u(Username, Email) from User u
  • Read-only queries that do not get flushed.
  • Join qualifiers (on in T-SQL)

Let me try to take this in order.

Lazy Field Initialization:

On the surface, it looks very good, because you can do something like:

Customer customer = session.Load<Customer>(15);
Console.Write(customer.Name);

And the OR/M would generate this SQL:

SELECT Name FROM Customers where Id = @p0; @p0 = 15;

That sounds fine, until you realize that the database roundtrip is far more expensive than loading a single row, even if you load all its columns (leaving BLOB aside for now). This means that it is usually much more efficient to load the entire row at a single go, rather than piecemeal.

I tried to toy a bit with the way you would do that, and I can't really think of a good way to handle it without causing major management issues for the OR/M user.

That said, it is a feature that can be very valuable if you make it optional, per field basis. That way, you can have a customer object that contains a Photo BLOB column, and have it accessible only when needed, yet keep the natural programming model.

Partial object queries

Well, NHibernate has that:

select new UserSummaryDetails(u.Username, u.Email) from User u

It will return a list of UserSummaryDetails that you can use. As an aside, they are also not tracked by NHibernate.

Real Only Queries

Those are actually fairly simple to implement, you just need something like:

ISession tempSession = factory.OpenSession( currentSession.Connection );
tempSession.FlushMode = FlushMode.Never;
results = tempSession.CreateCriteria(typeof(Customer)).List();
tempSession.Dispose();

Abstract that to a helper function and you are set.

Another option is to use SetResultTranformer to inject an modifier that will evict the instance from the session (which has its own issues).

Join Qualifiers

From a few experiments that I have done, there is not difference on the query plan if you are using ANSI joins or where clause joins. This is one solution to the problem. Another issue would be what syntax to choose. NHibernate would need to map that to all the relevant databases, which may not always support ANSI joins.

Not simple answer there, but the HQL Parser that we are building should make it more accessible to developers to go in and change it.

HTML & Assembly

Scott McMaster equates HTML to assembly. I find it amusing because one of the design goals of HTML is to be accessible to anyone, and I do believe that it has served that job in a very good manner.

Scott suggests that higher level abstractions such as control toolkits will take the place of HTML in the future. I do not agree, mainly because I have seen how painful such approaches can be when you want to simply generate HTML. Remember, for an abstraction to be useful, it has to give me a significant advantage over the existing approach, and it shouldn't limit me overmuch.

HTML is dead simple, so I don't see much value with that, and since no control toolkit can give me what I want, and since when I need to add additional stuff, I want it to be simple, I don't really see much value in going in that direction.

What is possible is that technologies such as Flash / Silverlight would become much more pervasive and allow us to move from HTML documents to more elaborate methods of communicating with the user.

In Defense of WebForms: Well, maybe not...

Mats Helander is in love with WebForms, and he explains why. I think that I have already explained why I don't agree with that, but allow me to go over his points. One important thing to notice is that Mats ignore the problem of "WebForms tries to be WinForms", because, as he says, that is usually an issue raised to explain the root cause WebForms problems.

I think that this is a mistake, because the rest of the problems in WebForms are non as critical as the issue of trying to be WinForms. And the main problem there is state. The web is stateless, windows is stateful. Once you break that basic concept, you already in a world of hurt.

I am much more in love with the WebForms model, which in my opinion is beautiful, efficient, powerful, flexible, scalable, portable, un-obtrusive, perfect for spitting out html, has a really well designed page lifecycle and it jives very nicely with MVC thank-you-very-much

image I was going to try to formulate an answer to that myself, but I think that I will let Mats' own word handle that:

The WebForms page lifecycle model is too complex to wrap your head around

Well, ok, it is.

But it is also really, really good. I freely admit that whenever I need to do something more complex than implement Page_Load and the event handlers of controls, I get totally lost. But when I eventually get it to work – by luck, swearing and occult sacrifice – I must admit I am impressed with the results.

And after saying that, you still think that you can call it good?! That is my definition of Hell. Building software should be predictable, maintainable and obvious. I don't have an issue with learning complex stuff, but needing occult sacrifice (which I have felt the need as well) to get it to work is out of the question. To quote Paul Graham:

It's pretty easy to say what kinds of problems are not interesting: those where instead of solving a few big, clear, problems, you have to solve a lot of nasty little ones. One of the worst kinds of projects is writing an interface to a piece of software that's full of bugs. Another is when you have to customize something for an individual client's complex and ill-defined needs. To hackers these kinds of projects are the death of a thousand cuts.

Sorry, just that is enough to keep me away from WebForms.

Mats then talks about using the DOM that WebForms provides:

Say that you were asked to implement the following method:

     public string SerializeToXml(Employee employee);

There are two obvious ways to do this:

StringBuilder

Using a StringBuilder object (or a plain string) you simply build up the xml as a string, doing things like stringBuilder.Append(“[name]” + employee.Name + “[/name]”);

XmlDocument

Using an XmlDocument object you build up your xml as a Document Object Model (DOM), doing things like xmlNodeEmployee.ChildNodes.Append(xmlNodeName);

When asked to implement the following method, you have the same basic choice:

     public string SerializeToHtml(Employee employee);

You can either use a StringBuilder and just build up the html as a string, or you can build up a DOM representation of your html.

imageMu! I can also do the smart thing and use a templating language. Using XmlDocument to generate XML is tedious in the extreme, because the DOM is built to the document model, not to ease the way we build XML. There is a reason why Orcas has a whole new set of ways to work with XML, because going the DOM approach is just too awkward.

As anyone who has ever worked with AST / DOM can tell you, just because it is the way the computer thinks that way, it does not means that this is the best way to generate it. If I wanted to generate XML, I would much prefer to use something like BooML than XmlDocument. Take a look at the differences:

[XmlLiteral()]
def GetBooksXml(books):
    books:
        for book as Book in books:
            book:
                @id = book.Id
                title book.Title
                author book.Author

The same goes for generating HTML. The DOM is a very awkward way to handle it.

 Mats then raise several cases where having a DOM is useful. Multiply outputs and wanting to delete an existing node after the fact are two of the issues he raises, he then says:

The problem, in my view, with going with a platform that doesn’t offer the flexibility of a DOM is that you don’t have the option the day it would become useful.

I would say YAGNI to that, while adding that if you want it, it is already there. In MonoRail it is DSL Support for Brail, in Ruby on Rails it is Haml. Here is how it looks like:

<?brail
dsl Html:
	body:
		component SmartGridComponent, {"source" : names}
	end
end
?>

Right now it does direct calls to HTML, but it is trivially possible to add HtmlDOM implementation, and handle that.

Mats has this to say on the page lifecycle complexity issue:

So my response to this accusation is to say that the complexity is usually hidden from the application developer.

Really? Take a repeater and put a button in the ItemTemplate, with an OnClick handler to a method. Now DataBind it, send it to the client, and press on one of those buttons. A postback occurs, but the event is not raised. Why?

This is just one of several hundreds of examples that I have where, as an application developer, I need to intimately understand the page lifecycle model and how it affects me. I don't really think that you can say that that an app dev is free from having to always deal with that on all but the simplest page.

imageHere is another telling statement:

Since it is too complex for me to understand, I really couldn’t make a judgment

Mats wrote an OR/M  implementation, which is one complex piece of software. I have a lot of personal respect for that, since I know what is involved in making such an effort. If he can't grasp the page lifecycle, what is the chance that most developers would be able to?

 WebForms and MVC

I have talked about the issues that WebForms MVC has extensively here, but I can surmise it simply as: There is no way for the controller to know that a user has pressed a button without the view being involved, or to build a Controller that doesn't need to be modified to fit the page lifecycle.

You can see how I think about the flow of a request using an MVC platform in the image. Notice that the view is the last thing that is involved, since it is not an important player until the very very end.

One thing that come up as a result of that is that a lot of the DOM manipulation games that Mats talks so much about are simply not necessary when you are using an MVC architecture, it is not needed because you know, in advance of building the view, exactly what you need to do.

I have done extensive work in this area, and my conclusion is that it is simply not possible to get separation from the view layer in WebForms, it is built into the platform, and cannot be changed.

Conclusion, while Mats may have intended to support the WebForm model, but I think that he had managed to bring up the points where using WebForms hurts.

And, as the joke says, stop doing that.

Plain Old Code

When J2EE came around, the notion of a container and managed objects / entities came into the fray, but those had such a high cost associated with them, that the Java community rebelled and now mainly advocates the Plain Old Java Objects. The .NET community has taken the same approach recently, with POCO or PONO being the term used.

This post is driven by my experiences with several of Microsoft's tools that tend to focus on "No Code Required". WebForms' 2.0 had a huge marketing focus on that, and MS CRM is being sold with the premise that you can extend it with just a GUI, so does many others tools from Microsoft (SSIS comes to mind).

I outlined the problems that I see with this model here, so I will not repeat it here, but I want to focus on code vs. GUI right now. I am aware of only a single such tool that provide a really good service, and that is the WinForms designer, and even that fails occasionally.

But what really gets me is the CRM's style of designers, where the customization is locked tight somewhere, where I cannot access it in any meaningful way.

Given the chance, I would define customization with code, probably a DSL, and probably something like this:

entity MyCustomerCustomer:
	CustomerNumber as int
	[Length(255)]
	Name as string
	Email as Email

screen MainView, for: MyCustomerCustomer:
	tab "Customer Properties":
		Display Name
		Display Email
		Display CustomerNumber, readonly

Hm, I seem to getting back to DSL no matter which way I go recently.

Ergonomic Hardware

imageLike many people who work with computers a lot, I suffer from wrist pain. I recently switched to using a Microsoft Natural Keyboard 4000, and it reduced those issues very quickly. The main benefit of the keyboard, aside from the ergonomic, split, design, is that it is  raised, which mean that my wrists doesn't have to bend in order to use the keyboard.

I have found that this to be a great product, with the single complaint that the wrist rest hand doesn't stretch enough to the left, so I have no easy way to access the left most keys column of the keyboard. I literally need an extra inch or two and it would be perfect.

Now I am looking for a mouse that I can use that will not reduce the effort even more. I would like to be able to go totally mouse free, and while I can do it, the problem is that this is simply not possible in many applications.

In Windows Live Writer, for instance, I don't have any way to access the right option bar without the mouse (at least non that I could find), and that is the case in many applications / sites. Selecting categories or setting publish date requires the use of the mouse as well.

I have tried using a trackball for a while, and that didn't reduce any wrist pain, if anything, it simply moved them from one place to another.

Any recommendation on a good mouse to use?

I have heard about data-hands, and I am interesting in taking them for a spin, but the cost is really prohibitive, and I am not going to even try that without seeing clear benefits out of this.

Prioritization, the Dilbert way

imageThis strip had me LOL, because it is such a true description of the way most customers want to think.

I am dealing with this issue by telling the customer that if they don't give me a priority scale that I can work with, I am going to implement the system using my own prioritization method, which is based on the technical fun factor of a particular feature, and that I don't consider UI to be fun, ever.

That tends to get me a list of features for the next week or two :-)

More seriously, though, I am posting this mostly as a way to remember to send this to the next customer that tries to take this approach.

My last customer was a pleasure to work with, because he was the rare combination of a technical guy with a good sense of the business, so we could discuss anything from implementation strategy to feature prioritization based on internal dependencies and the business need.

I think I need the non PHB award.

Developing on Microsoft CRM

I am currently in the process of leading a team in a project that is built around Microsoft CRM. A while ago I posted what I consider essential requirements for working effectively with business platforms. Since then, I had had a lot of time to play with MS CRM and see what the development story is.

Please remember, this is an evaluation of Microsoft CRM from a developer perspective. I don't touch on any of the other aspects that it may have.

I have already started to dislike it, and I have a feeling that it would only grow more acute in the future. The executive summary of this post is here:

Developing on MS CRM is building on a CRUD platform where all your business logic reside inside triggers.

Let me go over the points that I have outlined in my earlier post, again, those points are brought up without any particular order.

Source Control

MS CRM supports the creation of new entities and customizing existing ones. Those customizations are kept in a dark cave, to be let out at irregular interval for a slow walk around the block. The source control story on MS CRM is similar to the source control story of the average DB, except that you need to imagine that any DDL operation takes between one to thirty minutes to complete. And there is not textual representation that you can keep on the side.

Further extension points are available as plugins (callouts), which are triggers for CRUD operations on the CRM, and JavaScript customizations which you are supposed to write in a small text area inside the browser and allows you to respond to such events as changed / load. The SCM story for the plugins is a standard plugin issue, but for the JS customization there is literally no option except full import / export of the changes made from the CRM baseline image. See the previous paragraph about the performance implications of this, unacceptable.

Let me make it simple, if you want me to develop on your platform, source control is not a negotiable attribute. It should be as natural as breathing. If I have to do extra work to get it working, this is a critical issue.

Ease of deployment

You deploy a CRM installation by export your changes from the dev machine and importing to the staging/production installation. This is a lengthy process, and it doesn't capture all the changes that were made, you need to make several others (moving callouts, configuration, etc) from one environment to another.

Oh, and get ready to be intimately familiar with IISReset, and apparently resetting three other services is mandatory as well. And you need to do those on a regular basis just to get it deployed.

Debuggable

Well, you can attach to the CRM process, if you really feel like it, but that is frustrating, because you get to control only a small part of what is going on. It also looks like the CRM is throwing a lot of exceptions in normal operation, since I am unable to keep debugging in a "stop on all thrown exception", because it keeps throwing a lot of those.

Testable

There isn't any real way to do testing on the CRM without resorting to UI level testing, which is painful, to say the least. A lot of the methods in the extension API available expose such things as "string preEntityXml", which make writing the code hard, and testing harder.

Automation of deployment

As far as I can see, that is not doable easily, and it is a painful process (see deployment above) that really should be done with a click of a button.

Separation of Concerns & Don't Repeat Yourself

One of the issues that I have run into so far is that the entire development model for the CRM is reactive, and that doesn't really lend itself to keeping concerns separated and duplication minimized. The main problem is that the exposed API is primitive in the extreme and doesn't really lend itself to good practices (see extension options below).

In particular, the decision to base the extension options on non-portable Web Services and XML is a big mistake in this regard.

Doesn't shoot me in the foot

No comment on that so far, I haven't dug into it enough to know whatever this is an issue yet.

Make sense

This refers to what goes under the covers, and how I need to interact with it. So far, I had been told countless times "That is the way the CRM does it" in response to "WTF is this doing?!"

Possible to extend

Hacks are not something that I enjoy doing, and the more I dig into MS CRM, the more I feel like I need to resort to black magic hacks just to get things working. Let us go over some of the issues that I have run into so far.

Programming Model

If I were to suggest building a platform where users will have direct access to the DB tables' and all the logic for the system would be implemented using triggers, I would be called a madman. Nevertheless, this is the programming model that you have when you work in the CRM. Responding to actions in the CRM using plugins (called callouts).

Those callouts have signatures such as this one:

public override PreCalloutReturnValue PreCreate(
	CalloutUserContext userContext, 
	CalloutEntityContext entityContext, 
	ref string entityXml, 
	ref string errorMessage)

Then, you need to configure them in a config file, which lets you limit what this function will get. In that function, you can use the provided information to do something, but there is a lot of boiler plate code that you need to handle just to get out of the ugliness of this.

Configuration

Let us assume that you had the unmitigated goal to actually do try to develop a non trivial plugin to the CRM. You now need to supply it with some configuration, but the standard app.config / web.config approach will not work for you. The plugins are loaded into IIS's process, and I am not about to change w3wp.exe.config, that is taking it too far, and it violate the configuration / code should be a single unit issue.

Public Interface & API

The decision to base the entire extensibility model on web services and XML was a big mistake. It is mistake because the API that you get into the CRM is about the worst you can think of. Procedural, generated code, with very little flexibility or any way to supply layers on top of it that would allow you to present a nicer interface.

The generated code from the web service reference suffers from the usual awkwardness of WS code everywhere. Let us take this example, I want to change the customer preferred restaurant type and whatever he has a discount in it. Un order to do that, I need to write something like:

//preparing to call to the CRM
CrmService svc = new CrmService();
svc.Credentials = CredentialsHelper.GetCredentials();
svc.CallerIdValue = new CallerId();
svc.CallerIdValue.CallerGuid  = currentUserId;
ColumnSet columnsNeeded = new ColumnSet();
columnsNeeded.Attributes = new string[] {"name","new_customerpreferredresturanttype"};

//calling to the CRM
customer customer = (customer)svc.Retrieve("customer", customerGuid, columnsNeeded);

//updating customer value
customer.new_customerpreferredrestauranttype = new Picklist();
customer.new_customerpreferredrestauranttype.Value = 13;// Japanese 
customer.new_customerhasrestaurantdiscount = new CrmBoolean();
customer.new_customerhasrestaurantdiscount.Value = true;

svc.Update(customer);

Let us count the number of issues in this example:

  • It take 15 lines to do something as simple of that, and only two of those lines are actually doing anything useful, the rest are just there to make the API happy.
  • customer as a type name. .Net follows the PascalCase convention, and it is annoying in the extreme to see this in the middle of my code.
  • Magic strings all over the place. You don't have any way to avoid that.
  • Awkward API - CrmBoolean, anyone? Yes, they needed that to support nullable value types in 1.1, but it goes back to the problems with relying on the Web Service generated API to handle this scenario.
  • Magic values. Do you see the restaurant type constant there, it is a magic number that was defined in the CRM and it has no textual representation. You have the ID and the display value, and nothing else.

I couldn't get Microsoft's flagship web services' product to talk to the CRM successfully, so I am going to assume that the CRM is only accessible to ASMX platform (feel free to tell me if you can get Axis to successfully interact with MS CRM). That being the case, and since .Net is the obviously technology to work with the CRM, it would have been much better if MS would have simply supplied us with the ability to generate a DLL from the CRM with all the entities definitions and useful approach for interfacing with it. CodeDOM make it about as easy as providing a web service, so I have no idea why they choose not to do it.

Being able to ship an actual development platform, rather than anemic set of Web Service would have significantly improve the possibilities of improvements.

This shows itself in many other places, where the API hands me a string of XML and expect me to do something with it. Well, what am I supposed to do with it? Why not give me an object instance that I can actually work with?

The problem is that it can't do that, because there is no way for me to get the type that it is using for the entity. The best I can do is use the generated class from the generated proxy, and obviously the CRM has no way to give me that type, since it exists in my plugin assembly alone.

Subverting Intent

You may have noticed that the field names are stuff like "new_customerpreferredrestauranttype", that is because the CRM, in its wisdom, has decided that you don't really need to have readable (PascalCase) names, even if you took the care to put them into the CRM in this manner, it will lower case them and make your life harder.

Team Development Story

This is something that is at least as big as the source control issues, and for the same reason. The CRM doesn't support any sort of team development on a project. The main issue is that you don't have a way to work locally on a CRM instance, and then commit those changes to the trunk. The entire is forced to work on the same instance, with all the associated problems that this entails.

This has a lot of scary implications, some of which are:

  • I have no control over what goes into the system, no way to review changes, no way to revert or selectively modify what goes in.
  • I can't just pull out a debugger and try to find an issue in an errant plugin, because doing so will send the entire team into a screeching halt.

 

Performance

So far I had issue with it from a developer perspective, it moves from a simple request causing the DB to take 100% for minutes to stuttering under the load of a single user. I understand that there are several fixes for those issues, but that is on a clean machine, just trying to get to grips with the way that the CRM works.

Conclusion

As you can imagine, that is not a situation that sits very well with me, and I am actively working to overcome as much of those limitations as I can. I am making good inroads, but I can't believe how bloody hard the platform make it to use it.

Dependency Injection in a Dynamic Environment

Scott Bellware joins the DI discussion from a completely different angel, the dynamic language point of view:

The level of abstraction that we work at in .NET is appropriate to explicit inversion of control and thus dependency injection.  To make type mocking your main method of achieving testability in .NET development will drive your designs increasingly sap the maintainability of your code.  Maintainability and agility in a static environment are mostly the effect of applying appropriate static design patterns for static development. 

[...snip...]

The difference in my comfort level in doing this has a lot to do with the fact that I'm usually coding (and thus mocking) at the abstraction level of the Rails framework’s DSLs rather than lower down at the level of an API or a static Domain Model.

I think that the key here is the level in which you are working at a given moment. It is fairly easy to add cross cutting support for DI in my domain entities, but that is something that I wouldn't want to do. I am using DI mainly in controllers, services and commands, because injecting services to entities is something that looks... strange to me. I like my entities to be able to stand up on their own.

Minor note: I have something that I don't know how to call, which bridge the entities business logic and the required services. It is usually expressed in the terms of the domain, and is directly exposed to the entities, but usually as a parameter for methods, and the like.

I have written a few DSL and my share of fluent interfaces, and I think that while there is a lot of value in having a DSL (and a dynamic environment), a good API can cover a lot of ground.

As a side note, he also mentions this:

Beneath the DSL is an API and a framework that has all kinds of OO design goodness, but unlike working with a Rails clone in .NET, I don't often have to have to see the vestiges of the framework's uses of dependency injection in my application code.

I usually divide my applications to several layers: infrastructure, entities, services, controllers, etc. Of those, the layer that is most likely to be exposed to the messy details of real world coding is the infrastructure layer. The rest of those are expressed in terms of the current context, and that works fairly well. The main use of DI in those layers is for constructor injection (I am not fond of setters injection in general) and not much more. In fact, as someone who is part of a Rails-inspired project, I disagree with this statement. I find that I often can write the code in a way that make sense for the layer and context that I find myself in.

Here is a small example for your critique:

public virtual bool AddNewPolicy()
{
	using (RhinoTransaction transaction = UnitOfWork.Current.BeginTransaction())
	{
		Policy policy = BuildPolicy();

		if (ValidatorRunner.IsValid(Policy))
		{
			if (!ValidateThatUserIsAuthorizedToSavePolicy(policy))
				return false;
			if (AddValidPolicy(policy))
			{
				transaction.Commit();
				SetupRedirect(Policy);
				return true;
			}
		}
		else
		{
			Scope.ErrorSummary = ValidatorRunner.GetErrorSummary(policy);
		}
		return false;
	}
}

private bool AddValidPolicy(Policy policy)
{
	Repository<Policy>.Save(policy);
	UnitOfWork.Current.Flush(); //ensure that it will have an id
	try
	{
		CallWebServiceInsert(policy);
	}
	catch (Exception e)
	{
		Log.Error("Failed to insert policy using web service", e);
		Scope.ErrorMessage = string.Format(
			Resources.CouldNotInsertPolicyToBackEndService,
			e.Message
			);
		return false;
	}
	Usage.NewPolicyCreated(policy);
	return true;
}

I don't like the explicit transaction management here, but I had a special case of needing to coordinate an explicit (and expected) failure case, and that was the simplest solution to that. This is a controller's method, and the reason that it is returning a bool is for a check in the UI layer that would show the error message if necessary. Exception semantics usually stops at the controller, as far as I am concerned.

By the way, there is a line here that I didn't written, can you spot it?

I know that CRUD based example is not the best I could give, but I don't think that even this is encumbered by infrastructure concerns. Then again, I never did more than a sample app in rails, so it is entirely possible that I missing things (and I would like to hear about that).

He then goes on to say:

As for dependency injection, I'm simply bloody sick of it.  The vestiges of dependency wiring have no business in application code.

I would agree, except that my term for application code are the entities and controllers (maybe services). Like threading, DI is something that I would like to keep to infrastructure code only.

I would like to offer some educated guesses that may turn to be completely wrong, but I am guessing that while the need for DI is lessened in dynamic environments (I want method missing too, damn it!), you still need some sort of dependencies management, if only to provide a way to handle this as applications grow.

The main problem is that when you have several ways to tackle the same problem in different ways (handling logins for dev, staging, production is a good sample), or need to handle life cycle.

Dependency Injection: Separating the Container

In a previous post, I made the statement that I believe that using an IoC container should be transparent to the application code, and that a good IoC can and should be invisible to anything but the Main() method.

I would like to expand a bit on this topic, cover the sample problem that Tobias had brought up:

I never could achieve this and I'm wondering, what you mean by this statement. There a are always a lot of references to the DI/IoC container throughout the application. E.g. each IView that needs to be created to show a WinForm requires to ask the Container to give me the instances:

                ObjectFactory.GetInstance<IMyView>()

It might be an IController as well and IView is automatically resolved by the container, but as long as I need to instantiate new objects dynamically, I need to reference the IoC container at serveral places outside Main(). Maybe in very simple cases, the whole object tree required during the lifecycle of the application can be set up in Main().

Tobias has indeed hit the nail on the head, lifecycle issues are the one thing that can cause a lot of issues there. I would preface this by saying that I do not always follow this advice fully, and I have no compunctions about using the IoC as a service locator where appropriate, but I would try to limit it to the infrastructure part of the application only.

In this scenario, what I would generally do is define something like this:

// infrastructure service
public interface IRedirectionService
{
	IRedirectionOperation To<TController>();
}

And then use it like this:

Redirection
  .To<OrderController>()
  .ActivateAndWait();

It depends on the given scenario, but in this case, I am handing control from one controller to the other, and waiting until the second controller is done doing its job (think about it like modal dialog).

The Redirection Service is responsible for creating the controller / view (or find them if they are already active), and activate them.

IoC & DI are under the cover, but I don't have it in my face, and I have nice API against which I can work.

Shattered Aspects

Aspect Orientation is a way to structure your application so that some concerns (usually cross cutting, mostly infrastructure) are handled transparently by the framework you are running on. The classic examples are logging & auditing, but I am using it for transactions, thread safety, mocking, resource cleanup, validation and more. It is a good tool to have, and it can make some things very easy to handle.*

I am writing this post because I have dealt with AoP and derivatives quite often in the recent past, and I had some minor part in building some tools which are key for  Aspect Orientation in .Net (Dynamic Proxy). The problem with AOP on the CLR is that it requires extra steps on the part of the developer to get to it.

The CLR doesn't support AOP, and considering the need ** for that, it is fairly surprising that it isn't there. Furthermore, it seems like there is no intention to add that to the CLR. This is especially problematic in light of the non-virtual by default that we have in most CLR languages.

Since the platform doesn't give us AOP out of the box, we are left with hacks & half measures. Dynamic Proxy, the Policy Injection Block, partial methods and the like are all things that have been built because AOP is the correct answer to a certain class of problems. Considering that two of those are coming from Microsoft, I would like to see this being turned into giving us true AOP capabilities baked into the platforms.

There are 7 overall approaches to AOP in .Net, all of them has some serious limitations, and the ones that do give me what I would consider true AOP require that I would use the profiling / debugging API (special launcher, COM API, complex) or do IL re-weaving, which is simply hard & unproductive.  I am doing some work on Dynamic Proxy, and I can tell you up front that trying to workaround some of the issues there is very hard. That should be a service offered by the platform, not something that I would need to supply.

Nitpicker corner:

* Nevertheless, Aspect Orientation is not a Silver Bullet.

** You might not feel the need in your application, but I do. Rhino Mocks is entirely built on the idea of method interception, which is a core part of what AOP needs.

Useful lies about JavaScript Prototype Model

Note: This is not how it works, but it is close enough that it is a valid mental model for most scenarios.

When I think about JavaScript and its type system, I usually envision objects as hash tables that can carry values or functions. The prototype idea is just an associated hash table, and so on. The moment that I settled on this mental model, it was much easier to grok javascript.

Now, that is a vague explanation if I ever heard one, so let us speak in code:

public class Object
{
	private readonly Dictionary<string, Procedure> functions 
		= new Dictionary<string, Procedure>();
	private Object prototype;

	public Object Prototype
	{
		get
		{
			if(prototype==null)
				prototype = new Object();
			return prototype;
		}
		set { prototype = value; }
	}

	public Procedure this[string name]
	{
		get
		{
			if (functions.ContainsKey(name))
				return functions[name];
			if (prototype != null && prototype.functions.ContainsKey(name))
				return prototype[name];
			throw new InvalidOperationException("No function called " + name);
		}
		set { functions[name] = value; }
	}
}

This is a simple example of the matter, and here is how you can use it, if you ignore the fact that C# doesn't have any useful duck typing, then this code looks very much like the one that you would write in JavaScript:

Object obj = new Object();
obj["on_change"] = delegate { Console.WriteLine("changed"); };

obj.Prototype["on_load"] = delegate { Console.WriteLine("loaded from prototype"); };


obj["on_change"]();
obj["on_load"]();

//"overriding" the prototype method
obj["on_load"] = delegate{ Console.WriteLine("loaded from object"); };

obj["on_load"]();

obj["missing_method"]();//will throw

Ayende vs. Microsoft CRM: Round 3 goes to Ayende!

After a heoric struggle, I managed to get the local development story against MS CRM working. This means that I can use the normal semantics of develop locally and then deploy, instead of centralized development.

That said, there is something very wrong in a solution that requires me to take action in the TCP stack level to get a normal development story going.

Tags:

Published at

Searching for DDD code for reading

This may sound strange, but I want to read some DDD code, and I am sure where I can find some. There frameworks code is available all over the place, but DDD is often practiced inside a business app, which rarely has the source available. Preferably, it is something more than an example application.
Any suggestions?

Interviewing developers

My post about two (PHP) code samples that I got from people that I interviewed got a lot of comments, so instead of answering them one by one, I am going to answer them here. The post had a two code samples, one from a coder with 8 years experience, the second from a sysadmin that wants to move to development. Suffice to say that the sample from the coder with 8 years under his belt is atrocious.

Mostly, the comments tended to be:

  • Why am I interviewing PHP programmers?

    My company is also doing head hunting / recruitment, interviewing people is part of the job.
  • The sysadmin code looks good on the surface, but it is has problems. (and then listing the problems, in many detailed and interesting ways).

    The sysadmin code is not perfect by any means, but it is structured, it shows basic understanding of how things work, and the rest can be handled. That guy can learn to write good code. That isn't code that I would write, or would accept in my projects, but that is a very good start for the first steps.

image But a lot of people said that the first code sample is a quick & dirty solution to the problem at hand, some actually took offence at the idea.

If I interview someone for a position that include working with code (in fact, if the CV include any programming language whatsoever), you write code, period. The reasoning is that there are plenty that can talk the talk, but literally can't code. And looking at your code means that you get a lot of insight on how that person write code and how they think.

Now, here is an observation, you tend to write code in the same way no matter in what scenarios you are. Writing sloppy code in an interview is something that is a strong indication for a sloppy coder. That is not the only indication that I use, but it is certainly a significant one.

Given a choice between an experienced sloppy coder and an inexperienced one, I know who I would rather hire.

Now, to those who claimed that such questions are insulting, I am usually trying to fit the question to the person that I am interviewing (so a team lead would get something more interesting), but the key thing to remember is that I am not looking at the solution itself, I am looking at everything that they have around that.

I want to say that code style isn't something that I considering, but that would be a lie. I expect that the candidate would have some sort of a code style, and that it would be internally consistent. Beyond that, I don't really care if it spaces vs. tabs and where the curly go. But if there isn't a coding style that the candidate adheres to, that is something quite alarming.

Oh, and here is a secret that would make the argument more interesting, I don't program in PHP. The last time that I had to use that was when I tried to modify a bug tracker in PHP, and that was about two and a half years ago (the dot operator for string concat drives me crazy).

Just to remind people, my purpose in posting that was to talk a bit the difference between "experience" and quality.

How to test for SQL Injections

Not only does this snippet effective in finding simple SQL Injection attacks:

image

 The side effects are highly reduced ratio of second offences, and a sudden improvement in backup practices.

Now if I could fix the weeping issue...