Ayende @ Rahien

It's a girl

NHibernate new feature: Lazy Properties

This feature is now available on the NHibernate trunk. Please note that it is currently only available when using the Castle Proxy Factory.

Lazy properties is a very simple feature. Let us go back to my usual blog example, and take a look at the Post entity:

image

As you can see, it is pretty simple example, but we have a problem. The Text property may contain a lot of text, and we don’t want to load that unless we explicitly asks for it.

If we would try to execute this code:

var post = session.CreateQuery("from Post")
    .SetMaxResults(1)
    .UniqueResult<Post>();

You can see from the SQL that NHibernate will load the Text property. In large columns (text, images, etc), the cost of loading a column value is prohibitive, and should be avoided unless absolutely needed.

image

This new feature allows you to mark a specific property as lazy, like this:

<property name="Text" lazy="true"/>

Once that is done, we can try querying for posts:

var post = session.CreateQuery("from Post")
    .SetMaxResults(1)
    .UniqueResult<Post>();

System.Console.WriteLine(post.Text);

And the resulting SQL is going to be:

image

Note that we aren’t loading the Text property when we query for the post, and if we will inspect the stack trace of the second query we can see it being generated from the Console.WriteLine call.

But what if we want to query for posts with their Text property? Doing it this way may very well lead to SELECT N+1 if we need to load all the posts Text properties. NHibernate provide the HQL hint to allow this:

var post = session.CreateQuery("from Post fetch all properties")
    .SetMaxResults(1)
    .UniqueResult<Post>();

System.Console.WriteLine(post.Text);

Which will result in the following SQL:

image

What about multiple lazy properties? NHibernate support them, but you need to keep one thing in mind. NHibernate will load all the entity’s lazy properties, not just the one that was immediately accessed. By that same token, you can’t eagerly load just some of an entity’s lazy properties from HQL.

This feature is mostly meant for unique circumstances, such as Person.Image, Post.Text, etc. As usual, be cautious in over using it.

One last word of caution, this feature is implemented via property interception (and not field interception, like in Hibernate). That was a conscious decision, because we didn’t want to add a bytecode weaving requirement to NHibernate. What this means is that if you mark a property as lazy, it must be a virtual automatic property. If you attempt to access the underlying field value, instead of going through the property, you will circumvent the lazy loading of the property, and may get unexpected results.

Comments

Tobias
01/27/2010 10:56 AM by
Tobias

Neat!

What about "fetch all properties" for Criteria / QueryBy?

tobi
01/27/2010 11:13 AM by
tobi

Very useful yet a simple concept to understand. Thats a good mix.

Frans Bouma
01/27/2010 11:15 AM by
Frans Bouma

Why didn't you implement this using a query operator, like .Exclude("Text") ? Then you can make this flexible at the spot where you need it without having to specify anything in the mapping file.

You then also could have added a method to fetch the excluded fields for a set of entities or an entity. This would then fetch the fields specified for all entities specified (in 1 query) and merge them in an O(n) operation.

I chose that route in llblgen compared to yours as lazy fetching properties is in general a performance killer but specifying that you want to fetch them for a set isn't, and as this is a performance tweak feature, it (IMHO) makes little sense to force a select n+1 onto the user while the user wants to safe performance! :)

Ayende Rahien
01/27/2010 11:24 AM by
Ayende Rahien

Tobias,

There isn't a way to specify this for criteria API right now.

Frans,

I am not sure that I am following you.

Those type of properties tend to be fixed, vs. something that you change per query.

It make sense to make this a global setting, rather than per query.

Rafal
01/27/2010 11:57 AM by
Rafal

Question: you have a GUI for displaying lists where users can choose columns to be displayed. What about 'lazy' column - won't it cause select N+1 if such column is displayed?

Ayende Rahien
01/27/2010 12:15 PM by
Ayende Rahien

Rafal,

Don't bind that column, or eagerly load it.

Thilak nathen
01/27/2010 01:20 PM by
Thilak nathen

Not sure I like the idea of property access only for lazy properties. It's starting to sound like EF with it's property only access to lazy collections.

Ayende Rahien
01/27/2010 01:38 PM by
Ayende Rahien

Thilak,

The reason for that is quite simple, we need to intercept the access.

The other option is bytecode weaving, and we don't want to do this by default

Thilak Nathen
01/27/2010 01:56 PM by
Thilak Nathen

I get the interception dilemma... but what's wrong with

private IMyLazyLoadedProperty <string _text;

instead of

private string _text;

The former could be proxied for lazy loading perhaps? Or perhaps use the latter, CLR string type, but improve on IUserType to support lazy loading? (I'm not sure if we can already do this with a custom NH type).

Btw, how do we do bytecode weaving in NH? And why would that be bad?

David
01/27/2010 02:01 PM by
David

"What this means is that if you mark a property as lazy, it must be a virtual automatic property."

Does that mean this won't work for VB code except in VS2010?

Ayende Rahien
01/27/2010 02:03 PM by
Ayende Rahien

Thilak,

The problem is that it violates one of POCOness, it you need to use NHibernate's types to handle that.

The entity should be aware of its loaded status, either.

Having auto props means that this is basically transparent, and my solution for your dillema would be

protected virtual string TextInternal {get;set;}

public virtual string Text { get { return Text; } set { Text = value; }

You can certainly extend the support for this in NH, of course, if you really care that much about

Bytecode weaving means that you need a post compile step, and that it something we would like to avoid

Ayende Rahien
01/27/2010 02:04 PM by
Ayende Rahien

David,

No, it will work, but you must ensure that you never actually access the field directly.

ivos
01/27/2010 02:20 PM by
ivos

Really nice and expected feature!

Could batch be configured for lazy properties?

Ayende Rahien
01/27/2010 02:33 PM by
Ayende Rahien

Ivos,

No, not at the moment.

Nasty Nitpicker
01/27/2010 07:15 PM by
Nasty Nitpicker

Just a small nitpick as I see this in your posts all the time: it's not "SELECT N+1", that's nonsense. The right expression is "the N+1 SELECTs problem".

Ayende Rahien
01/27/2010 07:16 PM by
Ayende Rahien

Nitpicker,

I call it select n+1.

Edin
01/27/2010 09:56 PM by
Edin

Very nice feature indeed. But, in case of multiple lazy properties, why isn't it possible to load only one lazy property when accessed ? I do however agree on advice of not overusing this feature.

Ayende Rahien
01/27/2010 11:13 PM by
Ayende Rahien

Edin,

Two reasons:

a) it isn't implemented in hibernate

b) having two stage load is usually enough, while you can make arguments that it would be nice to have multiple fetch groups, the problem is that you then start making things so much harder on a lot of levels.

Just deciding what should go where would be hard enough, then you have the complexity of multiple fetch groups in a single entity, the possibility of loading an entity goes from a select or to to M selects, where M is the number of fetch groups.

Thilak Nathen
01/28/2010 09:49 AM by
Thilak Nathen

Something still smells about the whole deal.

configurator
01/28/2010 03:34 PM by
configurator

I like this idea.

Can I use the property setter as a way to know when the lazy property is fetched? Can I assume it will only happen once (except when I set the property myself)?

public string Text {

get { return text; }

set {

    text = value;

    DoLongOperationToPrepareTextForDisplay();

}

}

Ayende Rahien
01/28/2010 03:41 PM by
Ayende Rahien

Configurator,

No, you can't. To be rather more exact, you won't know when NH is doing it and when some other code is.

Your code would work, however.

SS
01/28/2010 04:08 PM by
SS

This is very useful for binary data. However, I'm wondering if there is a way that NHibernate can stream the data? Using something like SqlDataReader for sequential access to varbinaries, rather than pulling all the data at once?

Ayende Rahien
01/28/2010 04:36 PM by
Ayende Rahien

Ssidhu,

No, if you want a stream, you need to do provide a Stream property and a IUserType implementation

lpodolak
01/28/2010 07:52 PM by
lpodolak

"As usual, be cautious in over using it.".

I can only think of this as a way to prevent some blobs and clobs fetched when we don't need them. But doesn't it lead to the point that we should separate blob and clobs in our storage (as keeping them in separate tables)? Keeping the Post.Image together as an business entity would make sense due to consistency with customer's language, However in database, it wouldn't be that necessare to keep image in the Posts table.

Which approach do you recommend and why?

Ayende Rahien
01/28/2010 08:02 PM by
Ayende Rahien

lpodolak,

Separate them to a different table, that is my suggestion

Martin
03/01/2010 02:54 PM by
Martin

Ayende: Is it possible to mark a Component to be lazyload ?

So if any of the properties in the component is accessed all properties in the component is loaded ?

Ayende Rahien
03/01/2010 07:58 PM by
Ayende Rahien

Martin,

I don't think, but I haven't checked.

You can try :-)

Martin
03/01/2010 09:50 PM by
Martin

Thanks for the quick reply.

It is possible to set lazy=true but it isnt working, that is why i am wondering if it actually should work.

:)

InezPage34
03/01/2010 10:26 PM by
InezPage34

I had a dream to start my own business, nevertheless I did not have enough amount of cash to do this. Thank heaven my friend recommended to use the loan. Hence I used the college loan and made real my old dream.

Martin
03/01/2010 11:36 PM by
Martin

Thanks, i have just opened one.

I filed it under Bug, but it might actually be a New Feature depending on if it should be possible.

Rasmus
03/03/2010 07:33 AM by
Rasmus

"This is very useful for binary data. However, I'm wondering if there is a way that NHibernate can stream the data? Using something like SqlDataReader for sequential access to varbinaries, rather than pulling all the data at once?"

Wouldnt IQuery.Enumerable combined with Session.Evict let you do something like this. The manual 14.5 says:

"Whenever you pass an object to Save(), Update() or SaveOrUpdate() and whenever you retrieve an object using Load(), Find(), Enumerable(), or Filter(), that object is added to the internal cache of the ISession. When Flush() is subsequently called, the state of that object will be synchronized with the database. If you do not want this synchronization to occur or if you are processing a huge number of objects and need to manage memory efficiently, the Evict() method may be used to remove the object and its collections from the cache.

IEnumerable cats = sess.Enumerable("from Eg.Cat as cat"); //a huge result set

foreach( Cat cat in cats )

{

DoSomethingWithACat(cat);

sess.Evict(cat);

}

"

Ayende Rahien
03/03/2010 12:42 PM by
Ayende Rahien

Rasmus,

That isn't what Enumerable does, Enumerable is useful if you expect most of your results to reside in the 2nd level cache

Martin
03/27/2010 03:43 PM by
Martin

Hello again,

This is really is a nice feature.

Lazyloading a component actually also works with the latest code.

But there is a really big show stopper.

If i query an object an Castle.Proxies.MyObjectProxy is returned, and it is not possible to call any methods on the MyObject object.

The method is never hit.

It is like it is overriding the methods and never calls the base method.

Is this a know issue ?

Martin

Ayende Rahien
03/27/2010 04:12 PM by
Ayende Rahien

Martin,

No, it is not a known issue, can you create a JIRA issue for this?

Martin
03/27/2010 05:31 PM by
Martin

btw. i just want to add after having tested it some more, that the proxy works fine (i am able to call the methods on the object) if there is no lazy properties set on the object/proxy.

But when a property is set to be lazyloaded, it is not possible to call any methods on the proxy / object being generated.

Comments have been closed on this topic.