Raven’s dynamic queries

time to read 3 min | 534 words

One of the pieces of feedback that we got from people frequently enough to be annoying is that the requirement to define indexes upfront before you can query is annoying. For a while, I thought that yes, it is annoying, but so is the need to sleep. And that there isn’t anything much you can do about it.

Then Rob and asked, why do we require users to define indexes when we can gather enough information to do this ourselves? And since I couldn’t think of a good reason why, he went ahead and implemented this.

var activeUsers = from user in session.Query<User>()
                  where user.IsActive == true
                  select user;

You don’t have to define anything, it just works.

There are a few things to note, though.

First, unindexed queries are notorious for working on small data sets and killing systems in production. That was one of the main reasons that I didn’t want to run

Moreover, since RavenDB philosophy is that it should make it pretty hard to shoot yourself in the foot, we figured out how to make this efficient.

  • RavenDB will look at the query, see that there is no matching index, and create one for you.
    • Currently, it will create an index per each query type. In the future (as in, by the time you read this, most probably) we will have a query optimizer that can deal with selecting the appropriate index.
  • Those indexes are going to be temporary indexes, maintained for a short amount of time.
  • However, if RavenDB notice that you are making a large number of queries to a particular index, it will materialize it and turn it into a permanent index.

In other words, based on your actual system behavior, RavenDB will optimize itself for you :-) !

This feature actually had us stop and think, because it fundamentally changed the way that you work with RavenDB. We actually had to stop and think why you would want to create indexes manually.

As it turned out, there are still a number of reasons why you would want to do that, but they become far more rare:

  • You want to do apply complex filtering logic or do something to the index output. (For example, you may be interested in aggregating several fields into one searchable item)
  • You want to use RavenDB’s spatial support.
  • Aggregations still require defining a map/reduce query.
  • You want to use the Live Projections feature, which I’ll discuss in my next post.