Ayende @ Rahien

Hi!
My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

, @ Q c

Posts: 10 | Comments: 37

filter by tags archive

ReviewGetting started with LevelDB

time to read 3 min | 401 words

Getting Started with LevelDB

I was asked to review the book (and received a free electronic copy).

As someone that is very into storage engines, I was quite excited about this. After going over the leveldb codebase, I would finally get to read a real book about how it works.

I was disappointed, badly.

This book isn’t really about leveldb. It contains pretty much no background, explanation, history or anything much at all about how leveldb works. Instead, it is pretty much a guide of how to use LevelDB to write iOS application. There is a lot of chapters dealing with Objective-C, NSString and variants, how to do binding, how to handle drag and drop.

However, things that I would expect. Such as explanations of how it works, what does it do, alternative use cases, etc are very rare, if there at all. Only chapter 10 is really worth reading, and even so, I got the feeling that it only made sense to me because I already knew quite a lot leveldb already. I can’t imagine actually starting from scratch and actually being able to understand leveldb from this book.

If you are working on iOS apps / OS X, I guess that this might be a good choice, but only if you want to know about actually implementing leveldb. You’ll need to do your actual leveldb learning elsewhere.

The book does contain some interesting tidbits. Chapter 10 is talking about tuning and key policies, and it did have some interesting things to talk about, but it also contain wrong information* (and if I could spot it, with my relatively little experience with leveldb, I’m pretty sure that there are other things there too that are wrong).

* The book said that is it better to write keys in order, to reduce I/O. But leveldb writes to a skip list in memory, then flush that entire thing in sorted fashion to disk. Your writes have to be bigger than the buffer size of that to actually matter, and that still won’t help you much.

In short, feel free to skip this book, unless you are very focused on writing leveldb apps on iOS. In which case it might be a worth it, but I don’t think so. You are better off reading the docs or any of the tutorials.

More posts in "Review" series:

  1. (03 Dec 2013) Getting started with LevelDB
  2. (20 Jul 2011) Microsoft N Layer App Sample, part X–Architecture for the Space Age
  3. (15 Jul 2011) Microsoft N Layer App Sample, part IX–Not Invented Here is FUN
  4. (13 Jul 2011) Microsoft N Layer App Sample, part VIII–CRUD is so 90s
  5. (11 Jul 2011) Microsoft N Layer App Sample, part VII–Data Access Layer is GOOD for you
  6. (08 Jul 2011) Microsoft N Layer App Sample, part VI–Single responsibility principle is for idiots and morons
  7. (06 Jul 2011) Microsoft N Layer App Sample, Part V–Cross Cutting is a fine line
  8. (04 Jul 2011) Microsoft N Layer App Sample, Part IV-IoC FTW
  9. (01 Jul 2011) Microsoft N Layer App Sample, Part III–Abstraction is as abstraction does
  10. (30 Jun 2011) Microsoft N Layer App Sample, Part II–getting lost in the architecture
  11. (29 Jun 2011) Microsoft N Layer App Sample, Part I
  12. (12 Oct 2009) GoGrid vs.Amazon EC2
  13. (12 May 2009) C# in Depth
  14. (02 Sep 2008) Hibernate Search in Action
  15. (04 Jun 2008) Umbrella project
  16. (04 Jun 2008) Mass Transit Samples
  17. (23 Aug 2005) iRiver H340

Comments

Frank Quednau

To be fair, If I, as a "bog standard" dev buy a book called "Getting started with leveldb", I would expect a book that talks exactly about what you say it contains.

Bruno

Number 1 rule of successful publishing: Don't ask Ayende for a review ;-)

Kidding aside, reading the title i would not have expected that it focuses on iOS app development. There should be a large, obvious indicator of that. Or a change to the title (like: Getting started with employing LevelDB in your iOS Apps).

Duke

Harsh review, get some context dude. This book isn't written for the likes of Howard Chu

zdeslav

iOS part aside (and subtitle reads 'store and retrieve key-value based data quickly on iOS and OS X using Level DB'), it seems that book delivers exactly what title says: how to start using LevelDB. Why would you expect deep analysis of architecture and implementation in a book with such title?

I haven't read it, but from the review, I would say that indeed it is about levelDB, it is just not about levelDB implementation.

Saman

Don't pretend as if you guys don't know what marketing is doing here and everything is just dandy.

If they would wanted to be clear then the word iOS and Leveldb would have to appear within the main title as a MUST.

Andy Dent

I'm the author.

I'm not going to get defensive about the title. I didn't get to change the title after the publisher decided to focus the book on iOS and OS/X. I tried to still keep the book fairly generic - the early chapters are about using C++ and there's a quick overview of three scripting interfaces to give people an idea of how to use it from Python, Ruby or Node.js. So five of the 11 chapters are not about coding in Objective-C and one is pure theory.

I do have to take issue with the claim there's a factual error in chapter 10.

"The book said that is it better to write keys in order, to reduce I/O. But leveldb writes to a skip list in memory, then flush that entire thing in sorted fashion to disk"

For those who don't know anything about leveldb, Ayende's correct in that skip list contents are sorted but they are copied into level0 "young" files which contain overlapping key ranges. Significant sorting occurs in the compaction from level0 to level1.

The specific bit in the book says "When Sample06 moved to using multiple keys, we loaded them with a single loop creating two differently prefixed keys. That causes a lot of key overlapping and consequential sorting in the compaction from level 0 to level 1. If there's such a once-off load of many records, like our 50,000 line sample, consider using two passes through the data being loaded. A separate pass for each prefix means the keys we generate will already be grouped by prefix and reduces sorting at compaction time."

An interesting optimisation that the Basho people have done for their use of LevelDB in Riak is to defer the compactions. They discuss at http://docs.basho.com/riak/latest/ops/advanced/backends/leveldb/ how the compaction occurs between level0 and level1. However, in his RICON East 2013 presentation, Matthew Von-Maszewski from Basho said how they have reduced stalls by deferring the sorting. They allow files down to level2 to contain overlapping ranges.

His presentation at http://www.youtube.com/watch?v=vo88IdglU_8 is also a great overview of how leveldb works, bearing in mind he is talking about tuning for a substantial write load.

Andy Dent

Urk, correction to what I wrote above (perils of writing in small textboxes).

"An interesting optimisation that the Basho people have done for their use of LevelDB in Riak is to defer the compactions. " should read "An interesting optimisation that the Basho people have done for their use of LevelDB in Riak is to defer the sorting in compactions. "

Also, responding to the "marketing" suggestion - I don't pretend to understand how or why the publishing process works but I can assure you that the title was set in stone at the beginning of the process and the outline at that point was a completely different, more generic book. There was certainly no attempt at a bait-and-switch as is implied. I insisted that there be a clear subtitle to make sure anyone looking at the cover knew it was using iOS and OS/X. If you're buying online, the book overview is pretty clear that it's based on iOS and OS/X.

Comment preview

Comments have been closed on this topic.

FUTURE POSTS

  1. Production postmortem: The case of the memory eater and high load - about one day from now
  2. Production postmortem: The case of the lying configuration file - 3 days from now
  3. Production postmortem: The industry at large - 4 days from now
  4. The insidious cost of allocations - 5 days from now
  5. Find the bug: The concurrent memory buster - 6 days from now

And 4 more posts are pending...

There are posts all the way to Sep 10, 2015

RECENT SERIES

  1. Find the bug (5):
    20 Apr 2011 - Why do I get a Null Reference Exception?
  2. Production postmortem (10):
    14 Aug 2015 - The case of the man in the middle
  3. What is new in RavenDB 3.5 (7):
    12 Aug 2015 - Monitoring support
  4. Career planning (6):
    24 Jul 2015 - The immortal choices aren't
View all series

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats