http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Can we assume that using Attributes instead of XML mappings remove all of this startup time? I'm a tad surprised no one mentioned attributes at all given the relative extreme counter-ideas that are being thrown around. I appreciate that migrating to attributes from XML would be a great task, but they at least deserve a mention somewhere in the discussion... ? http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment18http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment18Wed, 31 Oct 2007 01:03:39 GMTIn the NH trunk, there is already code that generates C# classes from the xml schema for the mapping document. There is already code that can convert an entire mapping document into an object graph of those classes via XmlSerialization. The mapping document parser is already PARTIALLY taking advantage of this, and will eventually COMPLETELY utilize this as the first step to go from XML to fully realized NH objects. The fully realized objects cannot be guaranteed to be serialized, partially because one can define new types in one's own assemblies outside the control of NH. Plus the order of operations is significant. This is where these schema objects come into play. They are very simple classes that are fully serializable. This would allow them to be cached via binary serialization to disk. Some rudimentary performance tests on this were promising. In theory, one could build a fluent builder interface around these objects to allow definition of the mapping documents in C# as opposed to XML. (And still not be mixed in with your domain objects like the MappingAttributes stuff.) That would actually be pretty awesome I think. But that can't happen until I get the code migration done. (That idea kinda motivates me to get back into it. Damn real life in the way.) FYI, much of the relevant code is under these folders: https://nhibernate.svn.sourceforge.net/svnroot/nhibernate/trunk/nhibernate/src/NHibernate/Cfg/MappingSchema/ https://nhibernate.svn.sourceforge.net/svnroot/nhibernate/trunk/nhibernate/src/NHibernate/Cfg/XmlHbmBinding/ http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment17http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment17Fri, 26 Oct 2007 18:11:33 GMTMike D, The startup times in this case is for the scale far-out scenario. In practice, the startup times are not usually meaningful. If you want to avoid hacking XML, use ActiveRecord, it makes things much eaiser. And I don't agree with the "let us just change this tiny bit" approach, it doesn't work in practice. http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment16http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment16Fri, 26 Oct 2007 15:17:53 GMTI’m with Frans, this is one of those areas where code generation shines. Consider a DSL such as active writer which could store the map in xml and generate c# on the fly. Problem solved without pre-build steps. For others that like manually hacking xml use a pre-build c# code generator. This is one of the reasons I don’t use nHibernate, great product, I just don’t like hacking xml and can’t afford the startup times. Why recompile something each time an app launches, waste of time and energy. Another improvement that could be made is to break the map into two parts, schema and map, the map acts as a bridge between the schema and the objects. So if you convert a relation from 1:N to N:N you only change the schema and all maps update. http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment15http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment15Fri, 26 Oct 2007 15:06:06 GMTAyende, that makes perfect sense. Great ideas! I definitely 2nd the approach for "boobernate" but maybe under a different name... I do have to get clients to approve this stuff, ya know! It would be great to have an NHibernate API for serializing/deserializing whatever gets output from the Configuration, seems like it would be a bunch of hashtables (for settings and the reflection cache) -- and also the mapping format in memory. All of that sounds serializable, so I think your approach could bear fruit. I'm thinking something like this: Configuration cfg = new Configruation(); //add assembly SessionFactory factory = cfg.BuildsessionFactory(); NHibernate.Utility.PersistConfiguration(config, factory, "config-cache.nhconfig" ); slap that into a pre-build Nant/MSBuild task and then you can do this in your application: if( File.Exists("config-cache.nhconfig") ) { SessionFactory factory = NHibernate.Utility.LoadConfigurationCache("config-cache.nhconfig"); } else { // do it the old way } I'm not entirely sure what those methods would look like off the top of my head, but it seems doable. Thoughts? http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment14http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment14Fri, 26 Oct 2007 14:15:48 GMTI've actually started refactoring/recoding the XML mapping parsing bits in the NH trunk. My goal is to be able to use XML deserialization to turn the xml documents into an object graph of simple objects. The effect of this is that this DTO-like object graph could also be binary serialized/deserialized. Currently the mapping parser takes the XML data and directly turns that into more heavyweight NH objects that often cannot be serialized all in one step. Once this process is split into two steps, a schema caching step could be introduced to speed up startup time by more than a few seconds. This is not an insignificant improvement for desktop applications. So far its been a pretty significant effort (most of this processing was done in one large static class) and I haven't had much time to work on it recently with moving and starting a new job. I will get back to it once the dust settles. http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment13http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment13Fri, 26 Oct 2007 12:52:23 GMTPhilip, My first approach would be to slap Serializable on everything, and then see what works and what not. Writing custom binary formats is not something that I wish to do. http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment12http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment12Fri, 26 Oct 2007 10:44:41 GMTFrans, Yes, that is the next step. I don't think that it would be C# code for this. I experimented with Code DOM based persistence, it is nice idea, but awkward to maintain. I think that we will simply allow to save and load a session factory. http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment11http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment11Fri, 26 Oct 2007 10:43:32 GMTAlessandro, Create a second configuration file, and instead of specifying <assembly>, specify just <resource> with the relevant resources. http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment10http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment10Fri, 26 Oct 2007 10:41:54 GMTTomas, Yes, it is doing XSD validation. And that takes most of the time. There is no way to build the configuration without that, am I worry about default values and stuff that are in the configuration that may be missed. From a simple attempt, AddReaderUnvalidate did have a significant performance boost. http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment9http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment9Fri, 26 Oct 2007 10:24:59 GMTI am interested in maybe doing the patch, but how would you go about it? I guess we would have to make some sort of binary file format for NHibernate configurations. Another possibility would be to make a compiler for the mappings, for example to a binary file format and do it as a pre-build step, then it should be possible to load it very fast. What do you think? http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment8http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment8Fri, 26 Oct 2007 08:15:27 GMTHere's a tip from the code-generation Gilde :) Why don't you generate C# code as an option for the XML interpretation, where the C# code simply builds the inner structures build otherwise from the xml data. You can even have a commandline tool which does that at compile time (pre-build actually) where the C# code is embedded into the main program, OR, where the generated code is in a separate assembly and loaded on demand by nhibernate. Sure, you lose the 'flexibility' of changing mappings for production systems. well... who wants to have that option is really not that sane... http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment7http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment7Fri, 26 Oct 2007 08:10:53 GMTRegarding to initialize only a subset of entities using a staring session factory and after replace it with a global session factory, how can I decide which entities include in a session factory and which in another session factory? I only have a assembly with all my entities inside. Thanks http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment6http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment6Fri, 26 Oct 2007 07:18:12 GMTThanks for looking at my problem, I will change it soon and make it faster. http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment5http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment5Fri, 26 Oct 2007 05:57:04 GMTSince I've fallen in like with Binsor I'd like Binsor for NHibernate... Maybe Boobernate. http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment4http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment4Fri, 26 Oct 2007 03:27:33 GMTIs NHibernate actually doing XSD validation on the documents? If so, could it be disabled and see if that improves things somewhat? If not, then what exactly does NHibernate to read all the configuration documents, then? I mean, 1000 files should generate a bit of IO, but not enough to account for such a big difference by itself (though I guess it is building the DOM that results in that). http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment3http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment3Fri, 26 Oct 2007 02:58:18 GMTThis problem was raised at ALT.NET. One of the ideas that was tossed around was persistable configuration which you mention above. Another possible solution that was discussed was to JIT the session factory. I'm sure that if this is a bad idea you'll tell me! Certainly I think that it would be beneficial to do something here because database integration tests are slow anyway and spending 10 secs to build the session factory is too long. Maybe the community can agree upon an approach and then we can take care of implementation? http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment2http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment2Fri, 26 Oct 2007 02:54:53 GMTdeath to XML =o), XML is sloooooww http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment1http://ayende.com/2903/real-world-nhibernate-reducing-startup-times-for-large-amount-of-entities#comment1Fri, 26 Oct 2007 02:50:15 GMT