Ayende @ Rahien

Refunds available at head office

Review: Getting started with LevelDB

Getting Started with LevelDB

I was asked to review the book (and received a free electronic copy).

As someone that is very into storage engines, I was quite excited about this. After going over the leveldb codebase, I would finally get to read a real book about how it works.

I was disappointed, badly.

This book isn’t really about leveldb. It contains pretty much no background, explanation, history or anything much at all about how leveldb works. Instead, it is pretty much a guide of how to use LevelDB to write iOS application. There is a lot of chapters dealing with Objective-C, NSString and variants, how to do binding, how to handle drag and drop.

However, things that I would expect. Such as explanations of how it works, what does it do, alternative use cases, etc are very rare, if there at all. Only chapter 10 is really worth reading, and even so, I got the feeling that it only made sense to me because I already knew quite a lot leveldb already. I can’t imagine actually starting from scratch and actually being able to understand leveldb from this book.

If you are working on iOS apps / OS X, I guess that this might be a good choice, but only if you want to know about actually implementing leveldb. You’ll need to do your actual leveldb learning elsewhere.

The book does contain some interesting tidbits. Chapter 10 is talking about tuning and key policies, and it did have some interesting things to talk about, but it also contain wrong information* (and if I could spot it, with my relatively little experience with leveldb, I’m pretty sure that there are other things there too that are wrong).

* The book said that is it better to write keys in order, to reduce I/O. But leveldb writes to a skip list in memory, then flush that entire thing in sorted fashion to disk. Your writes have to be bigger than the buffer size of that to actually matter, and that still won’t help you much.

In short, feel free to skip this book, unless you are very focused on writing leveldb apps on iOS. In which case it might be a worth it, but I don’t think so. You are better off reading the docs or any of the tutorials.

Comments

Frank Quednau
12/03/2013 10:54 AM by
Frank Quednau

To be fair, If I, as a "bog standard" dev buy a book called "Getting started with leveldb", I would expect a book that talks exactly about what you say it contains.

Bruno
12/03/2013 02:35 PM by
Bruno

Number 1 rule of successful publishing: Don't ask Ayende for a review ;-)

Kidding aside, reading the title i would not have expected that it focuses on iOS app development. There should be a large, obvious indicator of that. Or a change to the title (like: Getting started with employing LevelDB in your iOS Apps).

Kijana Woodard
12/03/2013 03:08 PM by
Kijana Woodard

@Bruno - it's in the subtitle.

Duke
12/03/2013 03:12 PM by
Duke

Harsh review, get some context dude. This book isn't written for the likes of Howard Chu

zdeslav
12/03/2013 03:17 PM by
zdeslav

iOS part aside (and subtitle reads 'store and retrieve key-value based data quickly on iOS and OS X using Level DB'), it seems that book delivers exactly what title says: how to start using LevelDB. Why would you expect deep analysis of architecture and implementation in a book with such title?

I haven't read it, but from the review, I would say that indeed it is about levelDB, it is just not about levelDB implementation.

Saman
12/03/2013 06:45 PM by
Saman

Don't pretend as if you guys don't know what marketing is doing here and everything is just dandy.

If they would wanted to be clear then the word iOS and Leveldb would have to appear within the main title as a MUST.

Andy Dent
12/04/2013 06:11 AM by
Andy Dent

I'm the author.

I'm not going to get defensive about the title. I didn't get to change the title after the publisher decided to focus the book on iOS and OS/X. I tried to still keep the book fairly generic - the early chapters are about using C++ and there's a quick overview of three scripting interfaces to give people an idea of how to use it from Python, Ruby or Node.js. So five of the 11 chapters are not about coding in Objective-C and one is pure theory.

I do have to take issue with the claim there's a factual error in chapter 10.

"The book said that is it better to write keys in order, to reduce I/O. But leveldb writes to a skip list in memory, then flush that entire thing in sorted fashion to disk"

For those who don't know anything about leveldb, Ayende's correct in that skip list contents are sorted but they are copied into level0 "young" files which contain overlapping key ranges. Significant sorting occurs in the compaction from level0 to level1.

The specific bit in the book says "When Sample06 moved to using multiple keys, we loaded them with a single loop creating two differently prefixed keys. That causes a lot of key overlapping and consequential sorting in the compaction from level 0 to level 1. If there's such a once-off load of many records, like our 50,000 line sample, consider using two passes through the data being loaded. A separate pass for each prefix means the keys we generate will already be grouped by prefix and reduces sorting at compaction time."

An interesting optimisation that the Basho people have done for their use of LevelDB in Riak is to defer the compactions. They discuss at http://docs.basho.com/riak/latest/ops/advanced/backends/leveldb/ how the compaction occurs between level0 and level1. However, in his RICON East 2013 presentation, Matthew Von-Maszewski from Basho said how they have reduced stalls by deferring the sorting. They allow files down to level2 to contain overlapping ranges.

His presentation at http://www.youtube.com/watch?v=vo88IdglU_8 is also a great overview of how leveldb works, bearing in mind he is talking about tuning for a substantial write load.

Andy Dent
12/04/2013 06:21 AM by
Andy Dent

Urk, correction to what I wrote above (perils of writing in small textboxes).

"An interesting optimisation that the Basho people have done for their use of LevelDB in Riak is to defer the compactions. " should read "An interesting optimisation that the Basho people have done for their use of LevelDB in Riak is to defer the sorting in compactions. "

Also, responding to the "marketing" suggestion - I don't pretend to understand how or why the publishing process works but I can assure you that the title was set in stone at the beginning of the process and the outline at that point was a completely different, more generic book. There was certainly no attempt at a bait-and-switch as is implied. I insisted that there be a clear subtitle to make sure anyone looking at the cover knew it was using iOS and OS/X. If you're buying online, the book overview is pretty clear that it's based on iOS and OS/X.

Comments have been closed on this topic.