Okay, you probably noticed by now that I'm talking a lot about the amount of queries that a page makes. Right now I managed to drop it from 91 to 72 (didn't test in the process, and wasted tons of time) from 72 to 63 (tested in the process, took minutes). It was all about putting lazy in the right places (and fixing the tests that assumed that the data is always there).

I then tried to find out exactly where most of the traffic was, the first part was an easy target, just finding out that I made a couple of dozen calls for the exact same data was a big relief. I used the ASP.Net Cache, and that dropped the number of queries to 41.

Then I was stumped. I knew what was going on, but I didn't know how to fix it. I'd the following problem: Blog->Posts->Comments. Where I need to do some processing on blogs that I thought was also involving the comments. The result of that was two statements to load the blog and the posts, and another 38 statements to load each post comments. Looking at the code, I realized that I didn't really need the comments for that particular process. (I likely will never need them for a blog-wide process). Marking the Comments as lazy dropped the number of queries to three. If I was really desperate, I could have gone to two (which is minimal amount of statements that I could produce manually). I don't choose to do that because… well, that would mean that the UI would have to make decisions about how the data would be loaded.

I rather leave that decision outside of the UI layer at the moment. It's not a problem to add this feature, but the difference between two or three queries is not big enough to justify adding responsibilities to the UI that don't belong there.

FireFox 1.5 is nice, but it has a very serious problem for me.

It doesn't support Hebrew well. It can display it, but it doesn't support mixing it with English or with numbers.

Here is how a page looks like in FireFox 1.5:

Here is how it looks like in IE 6: (This is correct).

I once had to implement printing a mixed document of Hebrew, English and numbers. The amount of stuff that you need to do it is staggering. And you are never going to get it truly right.

It's interesting to note that FireFox 1.0 did support it.

Paul Graham has an article about Great Hackers that is worth reading. (I listened to the podcast, and it's much better when you can hear him speaks than just read the text). Joel has a very good article about Leaky Abstractions (although no podcast about that).

Put those things together, and you can get a fairly good idea about why leaky abstractions are so annoying for developers. If you read my blog you're aware of much hair pulling and gnashing of teeth in regards to ASP.Net that has been going on lately. Most of the problem is that I don't really understand what is going on under the hood. On the other side, I've had a similar experience with NHibernate and ActiveRecord.

I had a page that had to display a list of records from the database. Currently the database holds ~5 rows. The page loaded very quickly, but I took a look at the profiler to see what kind of SQL statements were executed and I nearly had a scream fit about it*. 18 statements for loading 5 rows? All of them in the same table? I got into a short argument (with myself, this time) about the usefulness of an ORM that generate such lousy code. I then put a couple of dozens of Lazy = true in various places in my code, and watch the page load again. There was only one SQL statement this time, and while it wouldn't pass the code police (horrible names), it did the work. For a moment, though, I had a very real sense of what would've happened if another developer, who isn't familiar with NHibernate or ActiveRecord would have with the same problem.

The second issue that I had was a matter of cascading. I thought that I had information, because I created it and then saved the root object to the database. But when I tried to get the information, I couldn't find it. The issue is (probably) that I didn't configure the cascades properly. I think that I fixed this by adding the proper cascade (I'm writing this post in the breaks that I have to take because of the long compile times for ASP.Net). Adding cascades broke the tests, which require some fixing, but it was very straightforward, most of the time.

Okay, I usually try to have a point with my posts, so let's try to give this one at least moderate one. The issue here is about control; understanding and how leaky is the abstraction. NHibernate is great, and I think that I cut in more than half the time that I need to spend thinking about data access. In fact, it's so good that I don't think about it most of the time.

Implementing the Session-Pre-Request and Session-Per-Test is very straightforward using ActiveRecord, and it is a joy to work with. I had written some very bad application in the past (and I'm sure that I'll say it again in the futureJ), and I distinctly remember the amount of thought that I had to dedicate to building any non-trivial queries. Keeping an eye on what is going on under the covers is good, but it is so good not having it in your face all the time.

* One SQL statement was so horrendous it was 5KB(!!) and had 11 joins! I couldn't follow it, and I was too afraid to see what the execution plan looked like.

Mario Gutierrez has a post about limiting the configurability of the system. One of the things that he said really caught my eye:

I find that I much rather have a solution where I need to write a little code to make the hard parts work, than have a complex framework that tries to do anything (look at XSD for an example, if you need one.)

I would like to dispute his other claim, that DB portability is not important. It's not important for an in-house project, yes. But the moment you need to integrate a project into an existing infrastructure, you'll soon find that it's not going to be that simple. Some people use SQL Server, some favor MySQL, some like Oracle, etc. You probably wants to sell to all of them. DB portability is an important feature in most applications that are going outside.

The game is called bugs and it's addictive.

I warned you, now check it out. Cute.

Rodrigo just announced that Cecil.FlowAnalysis is available in the Mono Repository.

This has the potential to do some truly fantastic things. I'm really impressed by the capabilities that this tool have, and the options that it open up.

Okay, I'm getting curious about stuff, so I keep the SQL profiler open, and I just saw a single page issue 91(!) SQL statements. Granted, this is a complex page, but I think that I can do it manually with maybe 4 SQL statements (and ~5,000 Line of code to sort it out later).

There are some parts of the code that I really don't know what to do. On the one hand, lazy loading some of the information make sense most of the time, but that may bring me to the horrendous Select N+1 problem.

Right now I'm deferring the decision, I added some lazy attributes, and that reduced the problem somewhat to 72(?!) statements. Still way too high, but this require some thought before acting. I'll let it lie in peace now, while I consider what to do. I also made a huge mistake by not running the test after I changed each relationship to lazy. I got so many errors when I finally did run them that it took me way too long to fix them.

One other thing I discovered, NHibernate assumes that accessing the properties is always safe, and it breaks it pretty badly if you start to mess with uninitialized collections during the initialization process. I reverted back to the old behavior (yes, I'm inconsistent, I know.)  

Okay, I just run into some head scratching bugs when using NHibernate.Generics. The issue was the well documented edge case, which turn to be a bitch when testing. I might have been able to fix this, but the whole thing pissed me too much when I noticed that it would work in the debugger, but not when running outside of it.

This is a crime that I won't tolerate. If I debug a piece of code, it should bloody well work, and not behave differently. I'll remove it from the code tonight, and hopefully I'll also remember to update the documentation for it. This would make NHibernate.Generics behave identically to the usual NHibernate' collections, which is how it should be.

So I was listening to DotNetRocks #115 (with Jackie Goldstein), and part of what they were talking about was the migration wizard that was suppose to migrate VB6 to VB.Net code. One of the thing that they brought up in the talk was that people expected that the wizard will take the code and turn it into a best-practice-using code. One of the examples that they gave was turning VB6 IO to .Net Streams IO (which the migration wizard doesn't do).

I couldn't help thinking about the poor developer who would run this hypothetical wizard and suddenly all his code is transformed from the familiar VB6 code to incomprehensible VB.Net code. I can just feel the anguish of that developer, trying to get the entire framework at once, stumbling along in a suddenly unfamiliar code base.

That is if the wizard was 100% correct, all the time. Occasionally I decide to break away from the base practice in a certain area, usually because I understand what I'm doing, and I know what needs to be done. I would hate to see what a wizard would do to this code.

Okay, the bug I was tracking? The one that I needed to work so hard to track? It was in my own code (not surprising, I found NHibernate to be rock solid.) It has to do with NHibernate.Generics and Cascades. I'm pretty sure that it doesn't interest you, but there will be a fix in the site later tonight.

I really hate it when my own code make me work so hard.

The issue is not that the code isn't good or that it's complex or that I don't understand it. The issue is that I have to manually move all the references to the complied assembly to the project, and when I'm done, I need to reset all of them back. I'm not using NHibernate alone, but several other libraries that depend on it (ActiveRecord and NHibernate.Generics), which I also need to add to the project, and I need to change their references, etc.

I can't even think about keeping them there permanently, I have to track a bug into NHibernate about once a month or so, and the build times are going up when I add all those additional projects. I thought about writing a macro to do this for me, but I don't know enough (or care enough) about VS internals to do it.

Okay, I'm currently uninstalling the 210 build of ReSharper. And for the first time, it's not because it is unusable. The memory it consume is huge, and I really can't afford to dedicate 300 – 400 MB of RAM just to the IDE. Performance improvement should start next week, so I expect to see a severe drop in the memory it needs.

In the meantime, I'm hitting the download page every day or so, just to see if a newer build is out.

The last blog entry was written in Outlook 12, and I took advantage of some of the nicest stuff it has to offer, easy visual selection. I was wondering how it would transfer this to the web. Now I know.

It simply took the text box with the code and generated an image for it. Nice for making sure that things looks the same, I guess. Not so nice when you're sending to a blog. I'm going to keep it as a memento.

Unfortunately, I just run into this "issue", check this out. In the context of an ASP.Net page, put this code;

This code run for a couple of minutes before I noticed something wrong. Suddenly my machine was very slow an unresponsive. At first I suspected the devenv.exe process, which is getting pretty heavy lately (mainly because of ReSharper, to be honest.) When I managed to load the TaskManager, I found the aspnet wp.exe process was consuming over 700Mb of memory, and was swapping like mad. It had over a million page faults, and was advancing fast to oblivion. The machine has 1Gb memory, and it used 1.8Gb, just to give you an idea about the pain this machine had suffered.

This tiny piece of code managed to:

‎  Eat 100% CPU.

‎  Push all other memories from memory.

‎  Push itself out of memory, so each loop iteration caused several page faults.

‎  Clear the pre fetched cache, so any new applications that I load will load much slower.

Pretty impressive, all in all. And all I forgot is a single line of code that would advance the current time.

Apparently I'd a glaring typo in my blog title.

I can usually get away with that, but this time the typo was in my own name*.

Big thanks for Gunnlaugur for finding it out and letting me know.

*well, nickname, you know the story.