Well, it is about that time again :-)

In about a month I’ll be returning to the UK to give another round of my NHibernate Course. It has been a while since I gave that in London, but the previous two runs were very successful, and I had great time teaching it.

This course is meant to give you working knowledge how to effectively use NHibernate in your applications, based on real world expertise.

You can register here: http://skillsmatter.com/course/open-source-dot-net/core-persistence-with-nhibernate

I am having a friendly competition with a friend about the stupidest support questions that we get from random people we never met. I posted about this previous, but I really can’t resist posting the content of something that I just received, it is either that or figure out how to send a nuke via email.

Hi,

Im f[removed] from Malaysia…. I has look at ur website about SOA. Did u do SOA application? Actually im a new learner, my knowledge are 0 about programming. I take an e-commerce course, at here I just do like a practical for 6 month and now I need to do on SOA. If u can help, I need some information about SOA, how to integrate the application, service and database. I help u can help me…

Thank you,

Best Regards;

F[removed]

This is the actual email text, I merely removed this guy name.

Therefore, I unilaterally declare myself the winner of the stupidest support emails contest.

I don’t actually have an opinion about the actual feature, but I felt that I just have to comment on this post, from Brad Wilson, about the [Required] attribute in ASP.Net MVC 2.

Approximately once every 21.12 seconds, someone will ask this question on the ASP.NET MVC forums

…

The answer is the title of this blog post. ([Required] Doesn’t Mean What You Think It Does)

If this is the case, I have to say that the design of [Required] is misleading, and should be change to match the expectations of the users.

We have a pretty common case of plenty of users finding this behavior problematic, the answer isn’t to try to educate the users, the answer is to fix the design so it isn’t misleading.

I am pretty sure that when the spec for the feature was written, it made sense, but that doesn’t mean that it works in the real world. I think it should either be fixed, or removed. Leaving this in would be a constant tripwire that people will fall into.

I’m currently on the way to several days of Army Reserve Duty, with limited to none internet connectivity.

I am making this announcement because the last few times I dropped offline for some time people started speculating that I am dead, which I sort of resent.

One of the things that seems to pop up frequently is people wanting to load an entity with all of its associations eagerly. That is pretty easy to do when the associations are many-to-one (that is, there is only one of them for each root entity). Example of those would be things like Owner, Site, etc.

Here is an HQL query that would load a blog with its owner and site as well:

from Blog b left join fetch b.Owner left join fetch b.Site

The problem starts when you try to do the same for multiple collection associations. NHibernate allows you to do so, but the result is probably not what you would initially expect. This query, for example:

from Blog b left join fetch b.Posts left join fetch b.Users where b.Id = :id

Will result in the following SQL statement:

select blog0_.Id             as Id7_0_,
       posts1_.Id            as Id0_1_,
       user3_.Id             as Id5_2_,
       blog0_.Title          as Title7_0_,
       blog0_.Subtitle       as Subtitle7_0_,
       blog0_.AllowsComments as AllowsCo4_7_0_,
       blog0_.CreatedAt      as CreatedAt7_0_,
       posts1_.Title         as Title0_1_,
       posts1_.Text          as Text0_1_,
       posts1_.PostedAt      as PostedAt0_1_,
       posts1_.BlogId        as BlogId0_1_,
       posts1_.UserId        as UserId0_1_,
       posts1_.BlogId        as BlogId0__,
       posts1_.Id            as Id0__,
       user3_.Password       as Password5_2_,
       user3_.Username       as Username5_2_,
       user3_.Email          as Email5_2_,
       user3_.CreatedAt      as CreatedAt5_2_,
       user3_.Bio            as Bio5_2_,
       users2_.BlogId        as BlogId1__,
       users2_.UserId        as UserId1__
from   Blogs blog0_
       left outer join Posts posts1_
         on blog0_.Id = posts1_.BlogId
       left outer join UsersBlogs users2_
         on blog0_.Id = users2_.BlogId
       left outer join Users user3_
         on users2_.UserId = user3_.Id
where  blog0_.Id = 1 /* @p0 */

Something that may not be apparent immediately is going to result in a Cartesian product. This is pointed out in the documentation, but I think that we can all agree that while there may be reasons for this behavior, it is far from ideal.

Let us look at what other OR/Ms are doing, shall we?

The comparable query using Entity Framework would look like this:

db.Blogs
    .Include("Posts")
    .Include("Users")
    .Where(x => x.Id == i)
    .ToArray();

And the resulting SQL would be:

SELECT   [UnionAll1].[Id]             AS [C1],
         [UnionAll1].[Title]          AS [C2],
         [UnionAll1].[Subtitle]       AS [C3],
         [UnionAll1].[AllowsComments] AS [C4],
         [UnionAll1].[CreatedAt]      AS [C5],
         [UnionAll1].[C2]             AS [C6],
         [UnionAll1].[C1]             AS [C7],
         [UnionAll1].[C3]             AS [C8],
         [UnionAll1].[Id1]            AS [C9],
         [UnionAll1].[Title1]         AS [C10],
         [UnionAll1].[Text]           AS [C11],
         [UnionAll1].[PostedAt]       AS [C12],
         [UnionAll1].[BlogId]         AS [C13],
         [UnionAll1].[UserId]         AS [C14],
         [UnionAll1].[C4]             AS [C15],
         [UnionAll1].[C5]             AS [C16],
         [UnionAll1].[C6]             AS [C17],
         [UnionAll1].[C7]             AS [C18],
         [UnionAll1].[C8]             AS [C19],
         [UnionAll1].[C9]             AS [C20]
FROM     (SELECT CASE 
                   WHEN ([Extent2].[Id] IS NULL) THEN CAST(NULL AS int)
                   ELSE 1
                 END AS [C1],
                 [Extent1].[Id]             AS [Id],
                 [Extent1].[Title]          AS [Title],
                 [Extent1].[Subtitle]       AS [Subtitle],
                 [Extent1].[AllowsComments] AS [AllowsComments],
                 [Extent1].[CreatedAt]      AS [CreatedAt],
                 1                          AS [C2],
                 CASE 
                   WHEN ([Extent2].[Id] IS NULL) THEN CAST(NULL AS int)
                   ELSE 1
                 END AS [C3],
                 [Extent2].[Id]             AS [Id1],
                 [Extent2].[Title]          AS [Title1],
                 [Extent2].[Text]           AS [Text],
                 [Extent2].[PostedAt]       AS [PostedAt],
                 [Extent2].[BlogId]         AS [BlogId],
                 [Extent2].[UserId]         AS [UserId],
                 CAST(NULL AS int)          AS [C4],
                 CAST(NULL AS varbinary(1)) AS [C5],
                 CAST(NULL AS varchar(1))   AS [C6],
                 CAST(NULL AS varchar(1))   AS [C7],
                 CAST(NULL AS datetime)     AS [C8],
                 CAST(NULL AS varchar(1))   AS [C9]
          FROM   [dbo].[Blogs] AS [Extent1]
                 LEFT OUTER JOIN [dbo].[Posts] AS [Extent2]
                   ON [Extent1].[Id] = [Extent2].[BlogId]
          WHERE  [Extent1].[Id] = 1 /* @p__linq__1 */
          UNION ALL
          SELECT 2                          AS [C1],
                 [Extent3].[Id]             AS [Id],
                 [Extent3].[Title]          AS [Title],
                 [Extent3].[Subtitle]       AS [Subtitle],
                 [Extent3].[AllowsComments] AS [AllowsComments],
                 [Extent3].[CreatedAt]      AS [CreatedAt],
                 1                          AS [C2],
                 CAST(NULL AS int)          AS [C3],
                 CAST(NULL AS int)          AS [C4],
                 CAST(NULL AS varchar(1))   AS [C5],
                 CAST(NULL AS varchar(1))   AS [C6],
                 CAST(NULL AS datetime)     AS [C7],
                 CAST(NULL AS int)          AS [C8],
                 CAST(NULL AS int)          AS [C9],
                 [Join2].[Id]               AS [Id1],
                 [Join2].[Password]         AS [Password],
                 [Join2].[Username]         AS [Username],
                 [Join2].[Email]            AS [Email],
                 [Join2].[CreatedAt]        AS [CreatedAt1],
                 [Join2].[Bio]              AS [Bio]
          FROM   [dbo].[Blogs] AS [Extent3]
                 INNER JOIN (SELECT [Extent4].[UserId]    AS [UserId],
                                    [Extent4].[BlogId]    AS [BlogId],
                                    [Extent5].[Id]        AS [Id],
                                    [Extent5].[Password]  AS [Password],
                                    [Extent5].[Username]  AS [Username],
                                    [Extent5].[Email]     AS [Email],
                                    [Extent5].[CreatedAt] AS [CreatedAt],
                                    [Extent5].[Bio]       AS [Bio]
                             FROM   [dbo].[UsersBlogs] AS [Extent4]
                                    INNER JOIN [dbo].[Users] AS [Extent5]
                                      ON [Extent5].[Id] = [Extent4].[UserId]) AS [Join2]
                   ON [Extent3].[Id] = [Join2].[BlogId]
          WHERE  [Extent3].[Id] = 1 /* @p__linq__1 */) AS [UnionAll1]
ORDER BY [UnionAll1].[Id] ASC,
         [UnionAll1].[C1] ASC

At this point, I am pretty sure, your eyes shut down in self defense. This is one complex query. But, basically, this is a complex query because EF is executing the following two queries and unioning them.

Eager load Blog Posts:

SELECT CASE 
       WHEN ([Extent2].[Id] IS NULL) THEN CAST(NULL AS int)
       ELSE 1
     END AS [C1],
     [Extent1].[Id]             AS [Id],
     [Extent1].[Title]          AS [Title],
     [Extent1].[Subtitle]       AS [Subtitle],
     [Extent1].[AllowsComments] AS [AllowsComments],
     [Extent1].[CreatedAt]      AS [CreatedAt],
     1                          AS [C2],
     CASE 
       WHEN ([Extent2].[Id] IS NULL) THEN CAST(NULL AS int)
       ELSE 1
     END AS [C3],
     [Extent2].[Id]             AS [Id1],
     [Extent2].[Title]          AS [Title1],
     [Extent2].[Text]           AS [Text],
     [Extent2].[PostedAt]       AS [PostedAt],
     [Extent2].[BlogId]         AS [BlogId],
     [Extent2].[UserId]         AS [UserId],
     CAST(NULL AS int)          AS [C4],
     CAST(NULL AS varbinary(1)) AS [C5],
     CAST(NULL AS varchar(1))   AS [C6],
     CAST(NULL AS varchar(1))   AS [C7],
     CAST(NULL AS datetime)     AS [C8],
     CAST(NULL AS varchar(1))   AS [C9]
FROM   [dbo].[Blogs] AS [Extent1]
     LEFT OUTER JOIN [dbo].[Posts] AS [Extent2]
       ON [Extent1].[Id] = [Extent2].[BlogId]
WHERE  [Extent1].[Id] = 1 /* @p__linq__1 */

Eager load Blog Users:

SELECT 2                          AS [C1],
     [Extent3].[Id]             AS [Id],
     [Extent3].[Title]          AS [Title],
     [Extent3].[Subtitle]       AS [Subtitle],
     [Extent3].[AllowsComments] AS [AllowsComments],
     [Extent3].[CreatedAt]      AS [CreatedAt],
     1                          AS [C2],
     CAST(NULL AS int)          AS [C3],
     CAST(NULL AS int)          AS [C4],
     CAST(NULL AS varchar(1))   AS [C5],
     CAST(NULL AS varchar(1))   AS [C6],
     CAST(NULL AS datetime)     AS [C7],
     CAST(NULL AS int)          AS [C8],
     CAST(NULL AS int)          AS [C9],
     [Join2].[Id]               AS [Id1],
     [Join2].[Password]         AS [Password],
     [Join2].[Username]         AS [Username],
     [Join2].[Email]            AS [Email],
     [Join2].[CreatedAt]        AS [CreatedAt1],
     [Join2].[Bio]              AS [Bio]
FROM   [dbo].[Blogs] AS [Extent3]
     INNER JOIN (SELECT [Extent4].[UserId]    AS [UserId],
                        [Extent4].[BlogId]    AS [BlogId],
                        [Extent5].[Id]        AS [Id],
                        [Extent5].[Password]  AS [Password],
                        [Extent5].[Username]  AS [Username],
                        [Extent5].[Email]     AS [Email],
                        [Extent5].[CreatedAt] AS [CreatedAt],
                        [Extent5].[Bio]       AS [Bio]
                 FROM   [dbo].[UsersBlogs] AS [Extent4]
                        INNER JOIN [dbo].[Users] AS [Extent5]
                          ON [Extent5].[Id] = [Extent4].[UserId]) AS [Join2]
       ON [Extent3].[Id] = [Join2].[BlogId]
WHERE  [Extent3].[Id] = 1 /* @p__linq__1 */) 

The query may be complex, but it get the job done and does so without bothering the us much. The question is, can we do the same with NHibernate?

As it run out, we can, pretty easily too. The following code will do just that:

 var blogs = s.CreateQuery("from Blog b left join fetch b.Posts where b.Id = :id")
     .SetParameter("id", 1)
     .Future<Blog>();

 s.CreateQuery("from Blog b left join fetch b.Users where b.Id = :id1")
     .SetParameter("id1", 1)
     .Future<Blog>();

So, what is going on here? We are actually issuing two queries, each of them to eagerly load a single collection. The trick is that we are using future queries to do so. That means that the query that is going to be sent to the database to get those results is:

select blog0_.Id             as Id7_0_,
       posts1_.Id            as Id0_1_,
       blog0_.Title          as Title7_0_,
       blog0_.Subtitle       as Subtitle7_0_,
       blog0_.AllowsComments as AllowsCo4_7_0_,
       blog0_.CreatedAt      as CreatedAt7_0_,
       posts1_.Title         as Title0_1_,
       posts1_.Text          as Text0_1_,
       posts1_.PostedAt      as PostedAt0_1_,
       posts1_.BlogId        as BlogId0_1_,
       posts1_.UserId        as UserId0_1_,
       posts1_.BlogId        as BlogId0__,
       posts1_.Id            as Id0__
from   Blogs blog0_
       left outer join Posts posts1_
         on blog0_.Id = posts1_.BlogId
where  blog0_.Id = 1 /* @p0 */
select blog0_.Id             as Id7_0_,
       user2_.Id             as Id5_1_,
       blog0_.Title          as Title7_0_,
       blog0_.Subtitle       as Subtitle7_0_,
       blog0_.AllowsComments as AllowsCo4_7_0_,
       blog0_.CreatedAt      as CreatedAt7_0_,
       user2_.Password       as Password5_1_,
       user2_.Username       as Username5_1_,
       user2_.Email          as Email5_1_,
       user2_.CreatedAt      as CreatedAt5_1_,
       user2_.Bio            as Bio5_1_,
       users1_.BlogId        as BlogId0__,
       users1_.UserId        as UserId0__
from   Blogs blog0_
       left outer join UsersBlogs users1_
         on blog0_.Id = users1_.BlogId
       left outer join Users user2_
         on users1_.UserId = user2_.Id
where  blog0_.Id = 1 /* @p1 */

Note that we are essentially discarding the results of the second query. The reason for that is that we aren’t actually interested in those results, we execute this query solely to get NHibernate to fill the Users’ collection of the relevant entity.

I generally use this method so I would have the first query to eager load all the many-to-one associations, and then a series of queries (one per required loaded collection) to load one-to-many or many-to-many associations.

I found myself needing to interact with JavaScript API in a WebBrowser hosted in a WPF application. That turned out to be quite a challenge to figure out, but I prevailed :-)

Given the following XAML:

<Window x:Class="WpfApplication1.Window1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Window1" Height="300" Width="300">
    <Grid>
      <WebBrowser x:Name="Browser" />
    </Grid>
</Window>

You can interact with the JavaScript API using this code:

public partial class Window1 : Window
{
    public Window1()
    {
        InitializeComponent();
        Browser.LoadCompleted += BrowserOnLoadCompleted;
        Browser.Navigate(new Uri("http://example.com"));
    }

    private void BrowserOnLoadCompleted(object sender, NavigationEventArgs navigationEventArgs)
    {
        var doc = (HTMLDocument)Browser.Document;
        var head = doc.getElementsByTagName("head").Cast<HTMLHeadElement>().First();
        var script = (IHTMLScriptElement)doc.createElement("script");
        script.text = "alert('hi');";
        head.appendChild((IHTMLDOMNode)script);
        script.text = "alert('bye');";
    }
}

This allow you to inject JavaScript commands to the browser by manipulation the script.text property. It isn’t generally useful, I’ll admit, but for my scenario, it seems to be doing the job.

As I mentioned before, I am actively trying to find what I don’t know, and plug it. As part of that endeavor, I spent the last week learning Java EE. I decided that in the interest of saving time I am going to invest some money and I took a week long course in it. That allowed me to kill several birds in a single stone, I got to experience Java in an environment that had an instructor that I could call on for help, it allowed me to learn Java EE and it allowed to me experience how people using Hibernate Profiler feel like.

Now that it is over, I can say that the course has met my expectations to a T. But that isn’t the point of this post. I wanted to talk about what I learned and my experience.

Nitpicker corner: please note that I am talking only about Java EE and EJB 3.0. I am well aware of the existence of frameworks that build on top of that (and earlier versions), but right now I intend to discuss Naked Java EE.

From language perspective, working with Java was incredibly frustrating. I had not really realized how much I come to rely on such things as object initializers, the var keyword or null coalescing operators.

As a platform, I have a much better understanding for how the Java EE eco system developed. Basically, it works like this (the concepts does not translate exactly, mind you):

One of the things that really drove me crazy is the notion of making everything a singleton. I wonder if there was a time where object creation in Java was very expensive, that might result in this design. The problem in making everything a singleton is that it become shared state, which means that you need to handle concurrency yourself. This is a bad thing. In fact, I would go further and say that any framework that requires users to handle concurrency for standard scenarios is flawed.

When I learned that you had to wire each independent servlet in the web.xml, I was quite shocked. The sheer friction that this is going to add to a project is mind numbing. Imagine the camera unzooming, and the full picture showing up. There are usually very few servlets in an application, often only one. They handle the request by sending the data to an EJB to handle the business end and a JSP to handle the HTML generation end. If it sounds a lot like MVC, it should. The interesting bit is that I don’t think that all the Java MVC frameworks came out because of anything unique in the Java sphere. They came out of sheer self defense out of the amount of toil & trouble that you have to go through using what the bare bones Java EE gave you.

Moving on to what I would call the controller side of things, we have EJBs, Enterprise Java Beans. There are Session Beans and Message Driven Beans. The MDB reminded me quite strongly of a very basic service bus. Considering something like Rhino ServiceBus or NServiceBus, it looks very similar in concept and execution, but without a lot of the things that the those two do for you. Session Beans are supposed to handle requests, they are divided into stateful and stateless beans. In general, I seems that you would generally use the stateless beans.

A stateless bean is a class implementing a specific interface, and that is where a lot of the interesting things are happening, dependency injection, automatic transaction management, session management, etc. It is interesting to note that with .NET we have gotten to the same result, but without having the overreaching presence of the container everywhere. I really like the fact that there is no attempt to disguise how they are doing things, the Java’s language decision to make everything virtual by default has really paid off in here.

Still, I can’t say that I like the default architecture, it seems very inflexible. I do like the ideas of full blown servers that you can just deploy a file into. It is a very nice concept, but it has some sever downsides, the time that it takes to do a modify-deploy-test cycle is significantly larger than what I noticed on the .NET side of things. And keep in mind that I am talking about projects that had about 5 classes in them all told.

During the course, the instructor said something that I found very insightful, “the tooling help you deal with… [wiring the xml, inheriting from the appropriate interfaces, etc]”. I found this very telling, because up until then I was quite puzzled by the behavior of all the standard wizards in Eclipse. They seem to violate the KISS principle, especially after getting used to the Yes, Dear experience when using R# on VS. It was only after I realized just how much work those poor wizards had to do for me that I understood what was going on.

After the course, I took a look at some of the MVC frameworks in the Java market. Just reading the tutorials is fascinating, from a code archaeology perspective. You can clearly see that Struts came early on, since while I am sure it is an improvement over Naked Jave EE, the amount of XML you have to deal with is not funny.

All in all, I find myself unimpressed by the amount of work that was shuffled to the tools, it doesn’t seem right. And it seems like a justification of a bad practice. When I consider my own design principles (Zero Friction!) in light of this, I am much happier that I am mainly working in the .NET world. But I think that having this understanding is going to be very helpful moving forward.

One of the things that I am working on is another commercial extension to EF, a 2nd level cache. At first, I thought to implement something similar to the way NHibernate does this, that is, to create two layers of caching, one for entity data and the second for query results where I would store only scalar information and ids.

That turned out to be quite hard. In fact, it turned out to be hard enough that I almost gave up on that. Sometimes I feel that extending EF is like hitting your head against the wall, eventually you either collapse or the wall fall down, but either way you are left with a headache.

At any rate, I eventually figured out a way to get EF to tell me about entities in queries and now the following works:

// will hit the DB
using (var db = new Entities(conStr))
{
    db.Blogs.Where(x => x.Title.StartsWith("The")).FirstOrDefault();
}

// will NOT hit the DB, will use cached data for that
using(var db = new Entities(conStr))
{
   db.Blogs.Where(x => x.Id == 1).FirstOrDefault();
}

The ability to handle such scenarios is an important part of what makes the 2nd level cache useful, since it means that you aren’t limited to just caching a query, but can perform far more sophisticated caching. It means better cache freshness and a lot less unnecessary cache cleanups.

Next, I need to handle partially cached queries, cached query invalidation and a bunch of other minor details, but the main hurdle seems to be have been dealt with (I am willing to lay odds that I will regret this statement).

This is a new feature available for NHibernate Profiler*, Linq to SQL Profiler and Entity Profiler. Basically, it detects when the same query is executed with different parameter sizes, which generate different query plan in the query cache.

Let us say that we issue two queries, to find users by name. (Note that I am using a syntax that will show you the size of the parameters, to demonstrate the problem).

We can do this using the following queries.

exec sp_executesql 
      N'SELECT * FROM Users WHERE Username = @username',
      N'@username nvarchar(3)',
      @username=N'bob'
exec sp_executesql 
      N'SELECT * FROM Users WHERE Username = @username',
      N'@username nvarchar(4)',
      @username=N'john'

This sort of code result in two query plans stored in the database query cache, because of the different parameter sizes. In fact, if we assume that the Username column has a length of 16, this single query may take up 16 places in the query cache.

Worse, if you have two parameters whose size change, such as username (length 16) and password (length 16), you may take up to 256 places in the query cache. Obviously, if you use more parameters, or if their length is higher, the number of places that a single query can take in the query cache goes up rapidly.

This can cause performance problems as the database need to keep track of more query plans (uses more memory) may need evict query plans from the cache, which would result in having to rebuild the query plan (increase server load and query time).

* Please note that detecting this in NHibernate requires the trunk version of NHibernate. And it is pretty useless there, since on the trunk, NHibernate will never generate this issue.

Krzysztof Koźmic has just announced it, so I will skip posting the details here and just let you read the entire announcement on his blog.

One thing that deserves special attention is that, as you can read from the post title, this Windsor release is for both .NET 3.5 and for Silverlight 3.0.

Have fun…