http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660My first question would be exactly the same as Torvin already wrote: In "Path.Combine(basePath, Path.GetFileName(Path.GetTempFileName()));", who is going to delete that empty file created in the TEMP directory by Path.GetTempFileName()? Despite its name, GetTempFileName() doesn't just generate a name, it creates an actual file you are supposed to delete... http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment15http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment15Wed, 17 Nov 2010 12:06:07 GMTDmyrii, Not really. We tested it with millions of items, and it just works http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment14http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment14Wed, 03 Nov 2010 08:10:47 GMTIf the indexes are kept in memory it will take significant amount of time to rebild them every time. Can you comment on it please? http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment13http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment13Wed, 03 Nov 2010 07:35:49 GMTI spent time going back into Ayende's archives starting from the beginning while he was on jail duty and coding. It reminded me a little of a science fiction short story I once read about a child who was a gifted composer whom they kept apart from all other people in order that he should not hear any music composed by others and have his creativity tainted (whether through imitation or by consciously trying to avoid imitation). I think Ayende is the type who should try and rewrite everything from scratch. For example even microsoft re-implemented javascript regex for IE9, as mentioned here: [blogs.msdn.com/.../...-in-internet-explorer-9.aspx](http://blogs.msdn.com/b/ie/archive/2010/03/18/the-new-javascript-engine-in-internet-explorer-9.aspx?PageIndex=2) http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment12http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment12Mon, 01 Nov 2010 14:12:23 GMTJesús, Please note that the codebase there has underwent a lot of changes, I'll have a lot to say about this shortly, but you can look at the current codebase here: [github.com/ayende/ravendb/tree/master/Raven.Munin/](http://github.com/ayende/ravendb/tree/master/Raven.Munin/)http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment11http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment11Sun, 31 Oct 2010 09:26:30 GMTSomething sounds wrong in this code: WriteCommands(cmds, persistentSource.Log); persistentSource.FlushData(); // sync the data to disk before doing anything else WriteCommands(cmds, persistentSource.Log); persistentSource.FlushLog(); // flush all the index changes to disk You are writting commands to log twice. When do you write commands to data? http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment10http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment10Sun, 31 Oct 2010 09:24:22 GMTI don't recall exactly how it works but this link looks as though it explains it quite well. It might give you some additional ideas... [conferences.embarcadero.com/.../32280](http://conferences.embarcadero.com/article/32280#HowDoesVersioningWork)http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment9http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment9Sat, 30 Oct 2010 11:20:02 GMTRickrock, Mostly, because I am not trying to target the same feature set. And by not trying to hit the same feature set, I have drastically simplified my problem set. http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment8http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment8Sat, 30 Oct 2010 11:12:14 GMTTovin, That is done in the ReplaceAtomically function http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment7http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment7Sat, 30 Oct 2010 11:11:17 GMTPeter, I am not aware of the Interbase model, but an append only model make MVCC trivial to work with. http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment6http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment6Sat, 30 Oct 2010 11:10:08 GMTDear Ayende, what exactly makes you think that you as a (unquestionable exceptionally talented) developer and architect will not into the same troubles that all those huge clumsy databases ran after a while? There is a big codebase not only because all other coders suck. Don't get me wrong: Sometimes small projects change the whole way we see things (Ruby is the most prominent example). How about though starting with a more "humble" idea like: "Let's skip some things a see how far we get with it and whether it is usable for some scenarios." Simplifying DBs is good, I support that. Conveying or implying the message that they are complicated without any reason is misleading. Or is a bit of "Not Invented Here" again? http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment5http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment5Fri, 29 Oct 2010 22:55:53 GMTlooks cool! i have a question though public Stream CreateTemporaryStream() { string tempFile = Path.Combine(basePath, Path.GetFileName(Path.GetTempFileName())); return File.Open(tempFile, FileMode.Create, FileAccess.ReadWrite); } who gonna delete that file? http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment4http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment4Fri, 29 Oct 2010 20:07:21 GMTcont. (heh heh..) CouchDB uses an append-only file to store your changes. At the end of the file, your data, the new leaf and all the new branches will be written. 'Snapshotting' doesn't require an immutable structure and an immutable structure doesn't require append-only, but with my feeble mind I wouldn't want to implement them any other way. http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment3http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment3Fri, 29 Oct 2010 12:17:26 GMT@Peter Morris I don't know anything about Interbase, and just a little bit of CouchDB, but this is how I understand CouchDB works. CouchDB uses immutable trees for their tables. When you modify the table, a new tree is created (creating only one new leaf and only the branches from the leaf up to the root). This way you can/could ask CouchDb for the current root and then at your own pace browse through that snapshot, while the rest of the world races by, creating updates by creating new trees. http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment2http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment2Fri, 29 Oct 2010 12:08:44 GMTI remember years ago reading how Interbase does versioned transactions so that people can read existing data with repeatable read whilst someone else is still in a transaction that has updated the data. Is that the approach you meant when you said "an append only model"? http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment1http://ayende.com/4677/the-building-blocks-of-a-database-transactional-persistent-hash-table#comment1Fri, 29 Oct 2010 11:35:08 GMT