Ayende @ Rahien

Ayende Rahien commented on LevelDB & Windows: It ain’t a love story

Sun, 21 Apr 2013 08:44:49 GMT

Matthias, I am sorry, but I didn't know about that. At this point, however, it isn't that relevant.

Matthias Götzke commented on LevelDB & Windows: It ain’t a love story

Sun, 21 Apr 2013 06:30:46 GMT

@ayende ... have you looked at https://github.com/bitcoin/bitcoin/tree/master/src/leveldb btw we use sqlite right now and are fine with it, but are also looking into leveldb (but need cross platform) there is also an article on that: http://www.codeproject.com/Articles/569146/LevelDB-DLL-for-Windows-A-New-Approach-to-Exportin

Ayende Rahien commented on LevelDB & Windows: It ain’t a love story

Tue, 05 Mar 2013 12:26:33 GMT

Giorgi, I lost trust in that, and that is _important_. I prefer concentrating my effort on things that didn't fall off & die the first time I touched them.

Giorgi commented on LevelDB & Windows: It ain’t a love story

Tue, 05 Mar 2013 11:45:42 GMT

The bug which you linked to at http://ayende.com/blog/3411/observations-on-embedded-databases

Ayende Rahien commented on LevelDB & Windows: It ain’t a love story

Tue, 05 Mar 2013 11:07:34 GMT

Giorgi, Which bug are you talking about? And I want to be able to compile & step through the code myself.

Giorgi commented on LevelDB & Windows: It ain’t a love story

Tue, 05 Mar 2013 10:19:01 GMT

Ayende, The bug that you encountered is fix and there is a .Net binding for 5.3 so why not give it a try again?

Ayende Rahien commented on LevelDB & Windows: It ain’t a love story

Mon, 04 Mar 2013 11:50:07 GMT

Giorgi, I don't trust BDB at all. See my previous experiments with it.

Giorgi commented on LevelDB & Windows: It ain’t a love story

Mon, 04 Mar 2013 09:48:12 GMT

There is a Berkeley DB back-end to SQLite which supports multiple writers: http://stackoverflow.com/questions/2824135/how-fast-is-berkeley-db-sql-compared-to-sqlite

Ayende Rahien commented on LevelDB & Windows: It ain’t a love story

Mon, 04 Mar 2013 00:18:32 GMT

j23, Wherever did I said that Google is obligated to do so?

j23 commented on LevelDB & Windows: It ain’t a love story

Sun, 03 Mar 2013 20:30:44 GMT

@ayende Question is why should Google care to support Leveldb on Windows ? I don't think they have any serious server software on non *nix

Yitzchok commented on LevelDB & Windows: It ain’t a love story

Sun, 03 Mar 2013 10:39:43 GMT

Did you check out https://github.com/hsn10/leveldb-mingw it seems like it is active.

Ayende Rahien commented on LevelDB & Windows: It ain’t a love story

Sun, 03 Mar 2013 09:17:02 GMT

Chris, That is a VERY important issue, yes.

Chris Wright commented on LevelDB & Windows: It ain’t a love story

Sun, 03 Mar 2013 08:41:16 GMT

And you would like to have one storage engine at least that can work on all platforms that you want RavenDB to work on. Otherwise you could just use ESENT on Windows and LevelDB on Linux.

Ayende Rahien commented on LevelDB & Windows: It ain’t a love story

Sat, 02 Mar 2013 06:37:20 GMT

Matt, We _tried_ looking at BangDB, I couldn't find the code. It is supposed to be OSS project, and I couldn't find the code (and I looked). That is the point when I gave up.

Ayende Rahien commented on LevelDB & Windows: It ain’t a love story

Sat, 02 Mar 2013 06:36:43 GMT

JDice, As I can't run it yet, I have really no way to tell. Performance is something that we would like to improve, but I have no idea how / whatever that will be the case

Ayende Rahien commented on LevelDB & Windows: It ain’t a love story

Sat, 02 Mar 2013 06:35:30 GMT

Matt, That would NOT be ideal for RavenDB, actually. We really do need to have multiple concurrent writers at the same time. Just to give you some examples, map/reduce indexes, stats, replication, user writes. A lot of those generate concurrent writes. We actually need to make a lot of writes to the same "table" a the same time, and any storage solution we use has to reflect that. And Mongo's decision to do that is... well, let us say that it caused a lot of problems for Mongo's users (search the forums), and Mongo doesn't have nearly as much background stuff as we do.

Ayende Rahien commented on LevelDB & Windows: It ain’t a love story

Sat, 02 Mar 2013 06:32:08 GMT

Justin, I agree, but SQLite is pretty bad at multi threaded access, at that is something that we rely heavily on.

Matt Johnson commented on LevelDB & Windows: It ain’t a love story

Sat, 02 Mar 2013 04:25:12 GMT

Did anything ever come of looking into BangDB?

JDice commented on LevelDB & Windows: It ain’t a love story

Sat, 02 Mar 2013 03:39:54 GMT

Ayende, Is LevelDB going to provide better performance than Esent? If so, how much would you estimate?

Justin commented on LevelDB & Windows: It ain’t a love story

Sat, 02 Mar 2013 00:13:31 GMT

Rob, My point is that is an internal implementation detail to Raven. From the outside Mongo and Raven are very similar and Mongo is able to provide adequate performance, arguably as good or better than Raven even though it has a global lock. If Mongo can do this with a global lock so should Raven, and if so then you now have the option of using a very well tested and performant database engine that's cross platform. If this is not possible because Ravens design is so tied to a storage engine having snapshot isolation then I guess SQLite is not viable choice and my original question is answered. It will be interesting to see what solution is chosen to accomplish this goal in the end.

Rob Ashton commented on LevelDB & Windows: It ain’t a love story

Sat, 02 Mar 2013 00:01:02 GMT

That wasn't what I was saying, I was saying that Mongo gets away with its full-db locks *because* it works that way, and Raven doesn't work that way so it's a bad comparison.

Justin commented on LevelDB & Windows: It ain’t a love story

Fri, 01 Mar 2013 19:21:37 GMT

Rob, Not sure I follow, you can turn off fsync with SQLite and use a single writer thread as well, thats up to Raven? Of course SQLite will let you use multiple writer threads but they will wait on the same table.

Rob Ashton commented on LevelDB & Windows: It ain’t a love story

Fri, 01 Mar 2013 19:15:19 GMT

Hey, I totally mis-read the question above as I was on my iPhone :-) WRT to Mongo doing DB-wide locks, that's not the same thing, it's easy to do DB-wide locks if you have a single writer thread and you don't care for fsync (as an example)

Justin commented on LevelDB & Windows: It ain’t a love story

Fri, 01 Mar 2013 19:13:09 GMT

"Isn't SQLite using BerkeleyDB underneath?" No: It has minimal platform dependencies and a well documented file format: I think someone made a SQLite wrapper for BereklyDB at one point suing virtual tables:

Matt commented on LevelDB & Windows: It ain’t a love story

Fri, 01 Mar 2013 19:02:41 GMT

SQLite has either a database file level or in the case of shared cache mode table level locks. These are shared reade r- single writer locks but you can allow reads while writing with read uncommitted if needed. See: That mode would be ideal for Raven IMO. And as I mentioned you could spread each of the 18 raven tables to separate files if needed and you can still do join across files, which I am not sure if you guy do. But the table level locking should give the same result. This is obviously not as nice as snapshot isolation with MVCC that is given to you for free by more complex DB's like PostgreSQL or ESENT, but it looks like Mongo is doing just fine with a per database global lock: Mongo's was process wide up until 2.2!

Rob commented on LevelDB & Windows: It ain’t a love story

Fri, 01 Mar 2013 18:51:58 GMT

No, ESENT

Rafal commented on LevelDB & Windows: It ain’t a love story

Fri, 01 Mar 2013 18:45:01 GMT

Isn't SQLite using BerkeleyDB underneath?

Matt Warren commented on LevelDB & Windows: It ain’t a love story

Fri, 01 Mar 2013 18:17:32 GMT

@Justin I thought that Sqlite had issues with multi-threaded access, see http://ayende.com/blog/3400/in-search-of-an-embedded-db

Justin commented on LevelDB & Windows: It ain’t a love story

Fri, 01 Mar 2013 17:41:05 GMT

Rob, I would be interested to to hear some more details on what is meant by "already made decisions" on SQLite vs ESENT. Is it that SQLite only does read_uncommitted and serializable vs ESENT only doing snapshot isolation? Control over transactions seem comparable between them. Data types on table columns seems comparable except for no built in multi-value tagged types. SQLite can run in memory, journal in memory, fsync off, etc. to get varying level of performance vs safety. You can spread data across multiple files to increase concurrency among other tricks. Hard to beat how well tested and how stable the file format is, seems like a good fit for an embedded cross platform db engine with good .Net bindings. Of course Postgresql would give you all the concurrency, transactions and datatypes you could ask for including a full JSON type or other multi-value types, but you would need to spin up a separate process for it since it won't run in-process.

Rob Ashton commented on LevelDB & Windows: It ain’t a love story

Fri, 01 Mar 2013 17:18:42 GMT

* and spend 10 hours fiddling with your graphics and network drivers - totally worth it