http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Command-Query separation. I guess Umbraco has a clever db structure from which I learn how to structure CMS data wisely. However the performance side - the query side, this db scheme not working very well. No-SQL db is a better fit in this case. A fused solution if possible I think serves better. Not sure if CQRS pattern could have some contribution in this case.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment40http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment40Tue, 04 Sep 2012 22:47:00 GMTI'm very surprised the Umbraco development team didn't figure these glaring performance issues. I don't think the problem is premature optimization, it is a poor understanding of nanoseconds vs milliseconds. You can optimize a piece of code to the point that it can't run any faster, but if it calls a function that copies a terabyte file over a network, it won't matter. I'm getting the sense that the developers we're seeing in the open source community are young. They were fortunate enough to miss out on the days of assembly language and C as the only way of getting any kind of reasonable performance. With today's compiler technologies and ever increasing storage and speeds, a game like PACMAN can be written in a few days using flash. The same effort would have taken months of painful, handcoded assembly language with no fancy debuggers. Umbraco 5 can be fixed, but it will take some work, but not 9 months as mentioned by Hartvig. The main problem is that they are trying to store dynamic types that aren't geared for storage and retrieval using SQL and certainly not for indexing. They can get a lot of mileage by simply storing user types as xml blobs or value-pairs with built in object fields such as url, id, security, etc. The data can be deserialized into dynamic objects for the sake of creating a view model. Instead, they made objects fields dynamic which is slow to query even with the most optimized SQL statements. This is a result of aiming for maximum flexibility at the expense of performance.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment39http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment39Tue, 19 Jun 2012 21:54:35 GMTSeveral commenters have noted that as much of a CMSs data is not relational, so a document database would be more appropriate. While this may be true, Umbraco is a mass market CMS, so it needs to be installable with minimum fuss on a wide range of real life hosting providers. For .NET hosting, this usually means SQL Server, with occasional MySQL (Umbraco V4 supports both). There simply isn't the critical mass of hosting providers to create a widely used .NET+NoSQL only CMS at this time. You could use an embedded RavenDB installation, but you still need to support the other relational DBs to get market share.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment38http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment38Mon, 18 Jun 2012 08:46:32 GMT@Steve sheldon http://umbraco.com/follow-us/blog-archive/2012/1/4/umbraco-5-on-performance-and-the-perils-of-premature-optimisation.aspxhttp://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment37http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment37Mon, 18 Jun 2012 00:52:47 GMTWhat drugs does it take to create such a horrible mess!http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment36http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment36Sun, 17 Jun 2012 23:38:53 GMTEJB - I'd argue Umbraco did the 'premature' optimization. That is they built a whole bunch of abstraction layers and complexity to solve a problem only to find out it wasn't the right problem to solve. The actual quote is "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%." - Donald Knuth http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment35http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment35Sun, 17 Jun 2012 14:31:42 GMTThis whole debacle is why this quote grates on my nerves every time I hear it: "Premature optimization is the root of all evil" Time and time again programmers use it as an excuse for bad programming practices, while it may have had merit in its original context, it has become the catch-all excuse, for bad programmers to write any kind of cr@ppy code they want, and worry about if it runs fast enough to use later. Well in this case, you can plainly see, Umbraco could have benefited from some 'premature' optimization.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment34http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment34Sun, 17 Jun 2012 13:06:17 GMTThis whole debacle is why this quote grates on my nerves every time I hear it: "Premature optimization is the root of all evil" Time and time again programmers use it as an excuse for bad programming practices, while it may have had merit in its original context, it has become the catch-all excuse, for bad programmers to write any kind of cr@ppy code they want, and worry about if it runs fast enough to use later. Well in this case, you can plainly see, Umbraco could have benefited from some 'premature' optimization.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment33http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment33Sun, 17 Jun 2012 13:04:52 GMTGreat analysis. I apreciate the effort you put into this. I have been programming for 25 years and focused on ultra high performance data systems for 10. I love the new tools we have to work with such as NHibernate and other ORMs, but because of the high performance aspect of what I do their cost becomes very apparent. A programmer who I have a lot of respect for once told me that it is impossible to make a program go faster, you can only make it do less work in the process of getting the results you need. Every time you add an abstraction layer of any type you are making the computer do more work. In addition you are programming yourself further away from the core reason for the program, to get results. The tradeoff is easier programming and possibly better maintainability at the surface level. I am not against this, but too many people use these tools without really understanding them and worse, without really understanding their own programming problem. Just slap another layer on and everything will be cool. When we get down to it a database is an abstraction layer on top of the file system and in memory data. Then we have the database communication layer. Now add an ORM layer. Then a queryable object wrapper. Then expose it as an API. At some point we are so abstracted that we have no clue about what is happening with the core reason for our program, our data. Hopefully there is a lesson in the Umbraco implosion somewere. Mine would be that I shouldn't use Umbraco again.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment32http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment32Sat, 16 Jun 2012 19:24:42 GMT@AndersM: oh, and a wonderful example of a Second System Effect too!http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment30http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment30Sat, 16 Jun 2012 11:43:25 GMT@AndersM: they're called Architecture Astronautshttp://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment29http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment29Sat, 16 Jun 2012 11:32:42 GMTSounds like a copy of NHProf would be well worth their while. http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment31http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment31Sat, 16 Jun 2012 08:46:14 GMT@Michiel: You are right - compared to v4 they added a table for each attribute datatype as well (date, decimal, string, long string and integer). The values are mapped as one-to-many with their attribute. By why on earth they did not consider performance earlier on I cannot understand. They could have benefited a lot from talking to Ayende, or at least buying nhprof.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment28http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment28Sat, 16 Jun 2012 06:43:38 GMT@AndersM That database schema is from Umbraco v4, right? In v5 they opted for an EAV model. See: http://our.umbraco.org/forum/core/umbraco-5-general-discussion/27399-The-database-schema-using-in-v5http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment27http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment27Fri, 15 Jun 2012 21:31:54 GMTNever mind the previous link, look at this: http://blog.hendyracher.co.uk/wp-content/uploads/2009/01/umbracodb1.gifhttp://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment26http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment26Fri, 15 Jun 2012 17:56:41 GMTNo CMS is simply CRUD when you go beyond simple editing. nick: In umbraco' case the requirements are quite well known, so talking about what database to use is highly relevant. The ability to customize the document-types in umbraco requires a lot in sql database: Look at this monster: http://blogbustingbeats.files.wordpress.com/2011/01/screenshot046.jpg This would have been MUCH simpler in a no-sql database.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment25http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment25Fri, 15 Jun 2012 17:47:25 GMTAt the end of the day isn't CMS basically CRUD? I don't understand the need for all this complexity. I've been tasked with replacing our current CMS + shopping cart app. It does a lot of querying from the view and the server just chokes when there are 10 concurrent users. I was hoping to use umbraco for the CMS portion. I really do not want to roll my own but I don't see a good alternative...http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment24http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment24Fri, 15 Jun 2012 17:27:30 GMT@Jeremy, you did notice the "(maybe)" right? ;-) Touche, though, I did word that a bit too absolutely. Nonetheless, those types of tradeoffs I look at very closely.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment23http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment23Fri, 15 Jun 2012 17:12:32 GMTYOU ARE ALL WRONG. THE DATABASE IS AN IMPLEMENTATION DETAIL http://blog.8thlight.com/uncle-bob/2012/05/15/NODB.html AAAAAAAAHHHHHHHHHHHHHHHHHHHHHhttp://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment22http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment22Fri, 15 Jun 2012 16:41:06 GMTRavenDB i guess would be a no-go for Umbraco, considering the licensing costs. Mongodb or any other free nosql with .net support would be a nice fit imho.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment21http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment21Fri, 15 Jun 2012 16:37:54 GMTI contributed a little to Umbraco in the early days, and it has been a architectural mess since forever. But this? They effectively ruined their business, brand and community support for a long long time. No-one will invest time in 5 or 6, until something is released and proven to perform. Ouch. They had a great opportunity to rework/scrap old their architecture, and they missed it completely.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment20http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment20Fri, 15 Jun 2012 16:35:58 GMT@Gene - How's assembly language treating you these days? :)http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment19http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment19Fri, 15 Jun 2012 16:05:53 GMTLooking at their API it seems like they wanted to hide NHibernate behind their API, but the problems bled out while the solutions to those problems were hidden behind their API wall. I bet the developers sat around and said "We don't want developers knowing that we use NHibernate when developing against Umbraco." When the shit hits the fan: "Hey developers, it was all NHibernates fault." @tobi is absolutely right!http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment18http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment18Fri, 15 Jun 2012 15:30:46 GMTMichiel, I was very careful not to inject any RavenDB into this post. Yes, RavenDB is ideal for CMS, and there are quite a few that use us for that. http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment17http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment17Fri, 15 Jun 2012 14:57:59 GMT100% agree with you Ayende...this is not going to get any better with native SQL...sounds like this Dev-team has bought in to the worst-possible definition of "persistence ignorance". Wrapping your persistence logic with so many layers that by the time you get there you have no idea what kind of action we are supposed to be doing is never going to really be helpful. Whatever happened to picking a helpful persistence abstraction for the problem at hand.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment16http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment16Fri, 15 Jun 2012 14:08:57 GMT@Bryan, agreed. Giving up something at runtime to get (maybe) something during the coding has always struck me as a bad bargain.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment15http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment15Fri, 15 Jun 2012 13:30:04 GMTUsing a repository pattern has some benefits, but simplicity is not one of them. It makes it easier isolate business logic in your unit tests because you can use a mock/fake repository. If your repository is in a separate assembly, it's easy to re-use it if multiple applications need the same data. I generally avoid the use of ORMs since in my experience they always seem to perform worse than native calls and impose an additional layer of complexity. I'd rather write a repository with SqlConnection, SqlCommand, and DataReader objects making direct use of stored procedures than fight with configuring NHibernate, LLBLGEN, or any other ORM I've experimented with, including Entity Framework in it's current form. If EF Code First ever gets decent support for stored procedures I might give it another look.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment14http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment14Fri, 15 Jun 2012 13:02:17 GMTHeh, modern .Net gives all the guns and ammo to shoot your foot any way you want. I bet the application would run better if they used Visual Basic 6.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment13http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment13Fri, 15 Jun 2012 12:37:51 GMTHi Ayende, Thanks for the thorough analysis. The core team has said it's going to continue working on Umbraco v4, and I think this will help them focus on 'the good parts' I should add that the Umbraco v5 project folded not only, or maybe even not at all, because of the performance issues. My tweet may have given some of your readers that idea. But the persistent performance issues did lead the core team and the community to re-evaluate the whole project, with the conclusion that it was overall too complex (I mean, even you couldn't find NHibernate). This seems to be built bottom-up, whereas your advice would be to design it form a top-down perspective, i.e. starting with the interfaces we can use inside views. Sounds about right, as that's where most CMS developers will write most of their code. I think a document database like RavenDB is better suited to the needs of a CMS with dynamic content types. I often wonder why we are trying to stuff all this 'dynamic data' into a strictly relational database. Umbraco v5 basically used an EAV model, that can't make querying much faster, can it? Thanks again! -Michiel http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment12http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment12Fri, 15 Jun 2012 12:20:52 GMTMy biggest takeaway from something like this is quite simple: your DATABASE and HOW YOU TALK TO IT matters. I keep seeing these unit of work and repository patterns, but there is a big difference between working with an ISession, a DbContext, an IDocumentSession, an IDbConnection, or plain old in-memory objects. And what about object caches like Redis? Or using Raven as writable transactional system along with an RDBMS for reporting? These are all abstractions anyway. The repository pattern just further abstracts it away, making it all that much more difficult to work with.http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment11http://ayende.com/156577/on-umbraco-s-nhibernate-s-pullout#comment11Fri, 15 Jun 2012 12:18:12 GMT