Ayende @ Rahien

Hi!
My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

, @ Q c

Posts: 5,972 | Comments: 44,518

filter by tags archive

Combating the evil queries since... the day before yesterday


Okay, you probably noticed by now that I'm talking a lot about the amount of queries that a page makes. Right now I managed to drop it from 91 to 72 (didn't test in the process, and wasted tons of time) from 72 to 63 (tested in the process, took minutes). It was all about putting lazy in the right places (and fixing the tests that assumed that the data is always there).

I then tried to find out exactly where most of the traffic was, the first part was an easy target, just finding out that I made a couple of dozen calls for the exact same data was a big relief. I used the ASP.Net Cache, and that dropped the number of queries to 41.

Then I was stumped. I knew what was going on, but I didn't know how to fix it. I'd the following problem: Blog->Posts->Comments. Where I need to do some processing on blogs that I thought was also involving the comments. The result of that was two statements to load the blog and the posts, and another 38 statements to load each post comments. Looking at the code, I realized that I didn't really need the comments for that particular process. (I likely will never need them for a blog-wide process). Marking the Comments as lazy dropped the number of queries to three. If I was really desperate, I could have gone to two (which is minimal amount of statements that I could produce manually). I don't choose to do that because… well, that would mean that the UI would have to make decisions about how the data would be loaded.

I rather leave that decision outside of the UI layer at the moment. It's not a problem to add this feature, but the difference between two or three queries is not big enough to justify adding responsibilities to the UI that don't belong there.

Firefox 1.5


FireFox 1.5 is nice, but it has a very serious problem for me.

It doesn't support Hebrew well. It can display it, but it doesn't support mixing it with English or with numbers.

Here is how a page looks like in FireFox 1.5:

Here is how it looks like in IE 6: (This is correct).

I once had to implement printing a mixed document of Hebrew, English and numbers. The amount of stuff that you need to do it is staggering. And you are never going to get it truly right.

It's interesting to note that FireFox 1.0 did support it.

On Leaky Abstractions and a developer sanity.


Paul Graham has an article about Great Hackers that is worth reading. (I listened to the podcast, and it's much better when you can hear him speaks than just read the text). Joel has a very good article about Leaky Abstractions (although no podcast about that).

Put those things together, and you can get a fairly good idea about why leaky abstractions are so annoying for developers. If you read my blog you're aware of much hair pulling and gnashing of teeth in regards to ASP.Net that has been going on lately. Most of the problem is that I don't really understand what is going on under the hood. On the other side, I've had a similar experience with NHibernate and ActiveRecord.

 

I had a page that had to display a list of records from the database. Currently the database holds ~5 rows. The page loaded very quickly, but I took a look at the profiler to see what kind of SQL statements were executed and I nearly had a scream fit about it*. 18 statements for loading 5 rows? All of them in the same table? I got into a short argument (with myself, this time) about the usefulness of an ORM that generate such lousy code. I then put a couple of dozens of Lazy = true in various places in my code, and watch the page load again. There was only one SQL statement this time, and while it wouldn't pass the code police (horrible names), it did the work. For a moment, though, I had a very real sense of what would've happened if another developer, who isn't familiar with NHibernate or ActiveRecord would have with the same problem.

 

The second issue that I had was a matter of cascading. I thought that I had information, because I created it and then saved the root object to the database. But when I tried to get the information, I couldn't find it. The issue is (probably) that I didn't configure the cascades properly. I think that I fixed this by adding the proper cascade (I'm writing this post in the breaks that I have to take because of the long compile times for ASP.Net). Adding cascades broke the tests, which require some fixing, but it was very straightforward, most of the time.

 

Okay, I usually try to have a point with my posts, so let's try to give this one at least moderate one. The issue here is about control; understanding and how leaky is the abstraction. NHibernate is great, and I think that I cut in more than half the time that I need to spend thinking about data access. In fact, it's so good that I don't think about it most of the time.

Implementing the Session-Pre-Request and Session-Per-Test is very straightforward using ActiveRecord, and it is a joy to work with. I had written some very bad application in the past (and I'm sure that I'll say it again in the futureJ), and I distinctly remember the amount of thought that I had to dedicate to building any non-trivial queries. Keeping an eye on what is going on under the covers is good, but it is so good not having it in your face all the time.

 

* One SQL statement was so horrendous it was 5KB(!!) and had 11 joins! I couldn't follow it, and I was too afraid to see what the execution plan looked like.

 

KISS and the 80/20


Mario Gutierrez has a post about limiting the configurability of the system. One of the things that he said really caught my eye:

"The other 20% will require the usual manual effort by the developer."

I find that I much rather have a solution where I need to write a little code to make the hard parts work, than have a complex framework that tries to do anything (look at XSD for an example, if you need one.)

I would like to dispute his other claim, that DB portability is not important. It's not important for an in-house project, yes. But the moment you need to integrate a project into an existing infrastructure, you'll soon find that it's not going to be that simple. Some people use SQL Server, some favor MySQL, some like Oracle, etc. You probably wants to sell to all of them. DB portability is an important feature in most applications that are going outside.

Rhino Mocks 2.5.4


Yes, I know that I said that I'm not going to post today. But Jeff fixed a problem with Partial Mocks (couldn't call a method that had parameters). It's 0:45AM, and I'm releasing software... I think that I could be arrested for this. :-)

As usual,  you can download it, or access the source directly.

Thanks Jeff, and it really is turning into a daily build project :-D

Centralizing application logic


Okay, last post for today.

In my current project, I've a class that is the center point for the rest of the system. It's not a Blob, but rather delegate to a lot of other classes and methods to do the work. It's sort of like a facade over the system, but for the system itself. I now find myself in the position where I need to provide some caching support for data that I pull from the database.

The issue is that while the application itself is a web application, I'm trying very hard to make sure that it doesn't depend on the web itself to do its work. But the caching that I need is for something that I need only on certain requests, but then definately need it (currently it makes 21 calls to load the same information). When the request end, I want the cache to go away.

This is very easy on the web, since I can use the current request to store the items, but this cause problems with both non-web usages (which is possible) and while using tests. I'm currently thinking of using some sort of per thread LRU hash table. I'm not very happy about it, though, so any ideas would be welcome.

2Gb Baby!


I have 2Gb on this machine! Whoopee! I already run into issues with MSN Search & Outlook because of it, but I hope to get the patch tomorrow.

Beyond that, life is good, I've so much memory that the OS doesn't know what to do with it, so it leaves most of it alone. That will change when I start doing development again on this machines.

FUTURE POSTS

  1. Reducing parsing costs in RavenDB - 16 hours from now

There are posts all the way to Aug 04, 2015

RECENT SERIES

  1. Production postmortem (5):
    29 Jul 2015 - The evil licensing code
  2. Career planning (6):
    24 Jul 2015 - The immortal choices aren't
  3. API Design (7):
    20 Jul 2015 - We’ll let the users sort it out
  4. What is new in RavenDB 3.5 (3):
    15 Jul 2015 - Exploring data in the dark
  5. The RavenDB Comic Strip (3):
    28 May 2015 - Part III – High availability & sleeping soundly
View all series

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats