http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Another solution: stop doing pointless micro-optimisations. If this leaves the DBAs with nothing to do, fire them - that's a real saving! http://ayende.com/4671/what-is-the-cost-of-storage-again#comment13http://ayende.com/4671/what-is-the-cost-of-storage-again#comment13Mon, 08 Nov 2010 14:35:00 GMTChris, That is a very good point, I'll put out a new post about that. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment12http://ayende.com/4671/what-is-the-cost-of-storage-again#comment12Mon, 25 Oct 2010 14:56:31 GMT"A much better solution would have been to simply put the database on a compressed directory, which would slow down some IO ..." I don't agree. Compression needs CPU. We got a lot of more IO by switching on compression (it's just less to write and read). Previous our CPU was about 40%, now averaging at 70%. Compression rate safes us about 30% per file. After switching on compression our IO bound application was about 20% faster. We are currently planning switching on compression on all our production servers over christmas, because using cpu-cores for compression is even cheaper than adding hard disks and raid for performance. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment11http://ayende.com/4671/what-is-the-cost-of-storage-again#comment11Mon, 25 Oct 2010 14:47:52 GMTAh, my mistake then. I was under the impression that you were using it to justify repeating identifier names in Raven rows. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment10http://ayende.com/4671/what-is-the-cost-of-storage-again#comment10Sun, 24 Oct 2010 14:53:28 GMTI once was in a ssimilar situation. Because we were running on 10$/month shared hosting I had to turn nvarchars into varchars to save space ;-) I reverted that immediately once we got our own server. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment9http://ayende.com/4671/what-is-the-cost-of-storage-again#comment9Sun, 24 Oct 2010 11:06:22 GMTPeter, We aren't talking about my stuff. We are talking about the scenario shown in the post that I linked to. Where they are using MonogoDB to store customer data. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment8http://ayende.com/4671/what-is-the-cost-of-storage-again#comment8Sun, 24 Oct 2010 11:00:48 GMTOren, but are they 1 of YOUR customers talking about the installation of one of their apps? What I am saying is that as a provider of tools you have to consider how much it costs *all* of your customers combined. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment7http://ayende.com/4671/what-is-the-cost-of-storage-again#comment7Sun, 24 Oct 2010 08:08:33 GMTPeter, a) They are talking about a single installation. They are a service provider. 2) Those numbers that they give are across all customers. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment6http://ayende.com/4671/what-is-the-cost-of-storage-again#comment6Sun, 24 Oct 2010 05:48:42 GMTThe short version. Application developers judge the cost of developing a feature for their single app. Application tool developers need to judge the cost to all their customers combined - you are comparing apples and pears. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment5http://ayende.com/4671/what-is-the-cost-of-storage-again#comment5Sat, 23 Oct 2010 20:56:02 GMTAgain you are considering only a single installation. If Raven is only to be used by you for a single application then these figures are correct. If however Raven is used many people and of all of those there are only 100 of those applications which meet the criteria you mention above then that is a total cost of $48,000 per annum. $480 per annum may not sound like much, but when you are writing a developer tool which is (hopefully) going to be used by hundreds (if not thousands) of people then $48,000 or even $480,000 per annum is a figure that is much more like the *total* cost per annum to your customers for using your tools. Just trying to do what you yourself are doing, adding some "real life" perspective on it. You can't just look at 1 customer's app and say "it's inexpensive", you have to look at the big picture. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment4http://ayende.com/4671/what-is-the-cost-of-storage-again#comment4Sat, 23 Oct 2010 20:51:44 GMTI'm not exactly sure what kind of mapping you are talking about, but would a developer not have to deal with 001 and 002 when updating the data model (and the mapping) to, say, support some new feature? For me personally to go with something like that, it would have to be a pretty sizable saving - maintainability is really gonna suffer. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment3http://ayende.com/4671/what-is-the-cost-of-storage-again#comment3Sat, 23 Oct 2010 20:48:35 GMTYou hit the problem that people raise with the size of column names in your last sentence. MongoDB is in RAM. If you do not have your entire dataset in RAM you will get no where near the performance that MongoDB is touted for and this is exacerbated on EC2 when there are quite a few times per day where disk access can suddenly drop of 400-600ms per request. If you don't have your entire dataset in RAM on EC2 you are not going to get the performance you want. That may not matter for your particular app, but it is an important consideration when trying to understand costs. Its not the disks - its the ram. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment2http://ayende.com/4671/what-is-the-cost-of-storage-again#comment2Sat, 23 Oct 2010 20:08:15 GMTIf you use some kind of mapping the developer never has to see those field names. So what's wrong with it? As far as I'm concerned the names can be "\001" ,"\002", etc. as long as the developer doesn't ever see them. http://ayende.com/4671/what-is-the-cost-of-storage-again#comment1http://ayende.com/4671/what-is-the-cost-of-storage-again#comment1Sat, 23 Oct 2010 18:51:14 GMT