http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Caching is your friend. :) http://ayende.com/3650/an-optimization-story#comment14http://ayende.com/3650/an-optimization-story#comment14Mon, 20 Oct 2008 20:50:00 GMTNice one, Oren, I think MemCache, Velocity and such can help handle the memory caching for you in a nice way and can help in lookups if you store the data with labels and such. http://ayende.com/3650/an-optimization-story#comment13http://ayende.com/3650/an-optimization-story#comment13Fri, 17 Oct 2008 09:30:19 GMTYes, it can be used outside of MonoRail http://ayende.com/3650/an-optimization-story#comment12http://ayende.com/3650/an-optimization-story#comment12Thu, 16 Oct 2008 06:32:34 GMTHello, calling this dialogue a 'sense of humour' is an abuse, but it's good you're aware of it ;) I have found your blog yesterday and really admire the job you are doing with Boo and DSL-s. This is probably the missing piece I've been looking for and I'll be checking that. Currently I'm implementing my open source workflow engine for .Net - the 'NGinn' project at [http://nginn.googlecode.com](http://nginn.googlecode.com), and I need a script environment with good 'call interception' mechanism - that is, with IQuackFu. I have started with Petro Protsyk's 'Script.Net' language, which is quite useful and elastic, however the Boo seems to have it all and more. I've got a question about the 'Brail' engine - can it be used outside the monorail framework? I would need such functionality for templating in various scenarios. How efficient is Brail? Best regards and thanks for interesting read at your blog Rafal http://ayende.com/3650/an-optimization-story#comment11http://ayende.com/3650/an-optimization-story#comment11Thu, 16 Oct 2008 06:27:04 GMTMemory is pretty easy to come by and the caching (into memory) process is straight forward. I guess disk based caching might be useful in cases where your web application needs to survive restarts or you need to *share* data across domains. http://ayende.com/3650/an-optimization-story#comment10http://ayende.com/3650/an-optimization-story#comment10Wed, 15 Oct 2008 21:35:33 GMTAlbert, It really depends on the data size and what you want to do with it. Memory is cheap, and you can put a _lot_ into memory without feeling any sort of pain. Implemeting disk based caching put you in a pretty bad position in terms of the cost of managing that. http://ayende.com/3650/an-optimization-story#comment9http://ayende.com/3650/an-optimization-story#comment9Wed, 15 Oct 2008 21:04:46 GMTScott, Caching blindly is possible, but it isn't really appropriate for many scenarios. For example, in this case, just the cost of making a single call to calculate the data for the first time is higher than the cost of loading the data and precalculating everything. http://ayende.com/3650/an-optimization-story#comment8http://ayende.com/3650/an-optimization-story#comment8Wed, 15 Oct 2008 21:02:54 GMTSpring let's you use [Cache] attributes I would have used something similar here at my Service layer probably. I'm sure it could be implemented similarly with Castle. http://ayende.com/3650/an-optimization-story#comment7http://ayende.com/3650/an-optimization-story#comment7Wed, 15 Oct 2008 19:28:30 GMTAlbert, I actually have been working recently on implementing a good disk caching pattern for asp.net, not so much the database - but caching resulting html. The caching is not so hard, but its more about designing how dependencies can be wired up, and how they can survive over process restarts.. it lead me down the road of being able to generate what is basically an entity tag or hash, or the current state of the dependent services.. and reinforcing and invalidating this when accessing the cached outputs.. thats where I stopped however, I ended up becoming very busy and didn't find the time to architect a solution. Caching is a really interest area in architecting solutions, as in this blog case, the results can be really rewarding when you make something a ton faster by intelligently handling data. http://ayende.com/3650/an-optimization-story#comment6http://ayende.com/3650/an-optimization-story#comment6Wed, 15 Oct 2008 18:49:43 GMTI had a similar-sounding situation. I had to make many lookups (which couldn't be predicted ahead of time) into a 100,000,000 row table. Fortunately, the nature of this table meant I was able to store it in 100MB of RAM. The performance difference was in the tens of thousands of percent, I think :) http://ayende.com/3650/an-optimization-story#comment5http://ayende.com/3650/an-optimization-story#comment5Wed, 15 Oct 2008 18:42:12 GMTDo you think it is advisable to try and implement a disk-output cache, where you read in data from the database and write it to a temporary files and then have subsequent calls access the data on the disk? I'm assuming the dataset returned is is pretty "large" for the asp.net cache or memory. Just asking out...loud..... http://ayende.com/3650/an-optimization-story#comment4http://ayende.com/3650/an-optimization-story#comment4Wed, 15 Oct 2008 12:02:16 GMTI just saw your day 1 blog poster and i am looking at this now ... you're very much imporved in your writing skills .. especially in vocub. and narration .. keep it up dude .. hard work never fails .. great post http://ayende.com/3650/an-optimization-story#comment3http://ayende.com/3650/an-optimization-story#comment3Wed, 15 Oct 2008 07:35:09 GMTImagine a tree structure that you need to query, now imagine that it is a sparse tree, and you need to do fixups for the tree http://ayende.com/3650/an-optimization-story#comment2http://ayende.com/3650/an-optimization-story#comment2Wed, 15 Oct 2008 07:18:01 GMTCould you show a requirement that can not be expressed in SQL? This seems like an interesting challenge (I can think of several really cumbersome things, but only one inexpressible in a good way ). http://ayende.com/3650/an-optimization-story#comment1http://ayende.com/3650/an-optimization-story#comment1Wed, 15 Oct 2008 07:11:04 GMT