Ayende @ Rahien

Refunds available at head office

Porting MVC Music Store to Raven: StoreManagerController, part 2

There are two methods in StoreManagerController that we haven’t touched yet. Dealing with them is going to just a little bit different:

image

Can you figure out why?

The answer is that when we started, we decided that there really isn’t any reason to store artists as individual documents. They are just reference data, after all. Now, however, we need to reference them.

We could, of course, create a set of artists documents, at which point it would be very easy to port the code:

image

But I still think that artists don’t really exists in this model as an independent entity. So instead of going with this route, we are going to project them.

We define the “Arists” index using the following map/reduce linq queries:

// map 
from album in docs.Albums
select new { album.Artist.Id, album.Artist.Name }

// reduce 
from artist in results
group artist by new { artist.Id, artist.Name } into g
select new { g.Key.Id, g.Key.Name }

If you’ll look carefully, you’ll notice that this is essentially doing a distinct over all the artists across all albums.

And that means that we can now write the code for those two methods like this;

image

There is one very important things to remember here: In Raven’s queries are cheap, because Raven allows you to query indexes only, and those indexes are built in the background, making queries tend to be very fast.

That changes the way that you think about designing you system and data model. You want to move a lot of your processing to indexes and queries upon those indexes, because it tends to be cheaper all around.

Tags:

Published at

Originally posted at

Comments (5)

Raven & Event Sourcing

image I keep trying to work on the replication bundle for Raven, but I keep getting distracted with more interesting stuff.

In this case, I kept coming back to several discussions that I had with people who want to use Raven for storing events, and were thinking about how to go from a stream of events to a complete aggregate. I kept thinking that Raven should already be able to handle that. And indeed it can, quite easily, it turns out.

Raven is already capable of running operations over a stream of document to produce a value, to go from there to event stream producing an aggregate is easy. The only problem was that we needed to support external views. That was easy enough to do, so let me show what we have now.

Let us assume that we have the events shown on the right stored in Raven, as you can see, this is a stream of events for a shopping cart. What we want to have is to go from there to an actual shopping cart.

We define the following view:

image

I am showing the class diagram here to show you all the types that are involved here. Note that ShoppingCart has AddToCart and RemoveFromCart method, which has the typical implementation.

Now, let us look at the actual view code:

    [DisplayName("Aggregates/ShoppingCart")]
    public class ShoppingCartEventsToShopingCart : AbstractViewGenerator
    {
        public ShoppingCartEventsToShopingCart()
        {
            MapDefinition = docs => docs.Where(document => document.For == "ShoppingCart");
            GroupByExtraction = source => source.ShoppingCartId;
            ReduceDefinition = Reduce;

            Indexes.Add("Id", FieldIndexing.NotAnalyzed);
            Indexes.Add("Aggregate", FieldIndexing.No);
        }

        private static IEnumerable<object> Reduce(IEnumerable<dynamic> source)
        {
            foreach (var events in source
                .GroupBy(@event => @event.ShoppingCartId))
            {
                var cart = new ShoppingCart { Id = events.Key };
                foreach (var @event in events.OrderBy(x => x.Timestamp))
                {
                    switch ((string)@event.Type)
                    {
                        case "Create":
                            cart.Customer = new ShoppingCartCustomer
                            {
                                Id = @event.CustomerId,
                                Name = @event.CustomerName
                            };
                            break;
                        case "Add":
                            cart.AddToCart(@event.ProductId, @event.ProductName, (decimal)@event.Price);
                            break;
                        case "Remove":
                            cart.RemoveFromCart(@event.ProductId);
                            break;
                    }
                }
                yield return new
                {
                    cart.Id,
                    Aggregate = JObject.FromObject(cart)
                };
            }
        }
}

We are doing several interesting things happening in the constructor:

  • The display name is the name of the index.
  • In the constructor, we define the map part as filtering for events for the shopping cart.
  • We will create a shopping cart per shopping cart id, so we specify the group by extraction method. Raven will use that to optimize updates.
  • Note the indexes definition, we want to id to be stored as as a primary key, and the aggregate data to be stored, not analyzed for searching.

Now, let us talk about the interesting bits, the Reduce function.

That function should be pretty easy to follow, I think. We are getting a stream of events, grouping them by their shopping cart id. Then, for each shopping cart, we sort the events by date, and proceed to build the aggregate.

Finally, we return the data so Raven will store it in the index.

The result, by the way, is this:

image

I think this is cool.

Using this approach, Raven will automatically keep the aggregate definition up to date with the event streams coming on. Furthermore, that aggregate will only be computed when a change happen, so accessing it is very cheap.

Finally, if we have a storm of events on a particular shopping cart, we can choose whatever to wait and see it in its most version, or get a potentially stale view of it really fast.

Tags:

Published at

Originally posted at

Comments (22)

Porting MVC Music Store to Raven: StoreManagerController

The final part of the port of the MVC Music Store to Raven is the administration section, implemented in StoreManagerController. I am going to show comparisons of all the methods where the port doesn’t offer anything new, and then focus on an interesting conceptual difference between the implementations.

image image

Please note that the main reason that the Raven code is so much shorter is that I threw away the nonsensical error handling (or lack thereof).

  image   image

Again, throwing away the error handling that isn’t made a lot of the difference in the code.

image image

Now we get to an interesting difference. The old code will delete orders if they include the deleted album. Raven’s code does no such thing.

It is important to understand that there is no such thing as referential integrity in Raven (or document databases in general). This can be a plus or a minus, but in this case, we are turning that into a plus, because we can delete an album without losing orders.  I don’t know about you, but I like the idea of keeping the orders around. :-)

A bit more formally, documents in Raven are independent, they aren’t affected by changes to other documents.

There are two more methods to discuss with regards to the StoreManagerController, but I’ll discuss them in my next post.

Tags:

Published at

Originally posted at

Comments (6)

Who stole my transaction?

I just run into an extremely strange bug with the System.Transactions API. It appears that under certain circumstances, you can exit the transaction scope before it has finished committing. Here is the code to reproduce this:

public class EnlistmentTracking : IEnlistmentNotification
{
    public static int EnlistmentCounts;

    public EnlistmentTracking()
    {
        Interlocked.Increment(ref EnlistmentCounts);
    }

    public void Prepare(PreparingEnlistment preparingEnlistment)
    {
        preparingEnlistment.Prepared();
    }

    public void Commit(Enlistment enlistment)
    {
        Interlocked.Decrement(ref EnlistmentCounts);
        enlistment.Done();
    }

    public void Rollback(Enlistment enlistment)
    {
        Interlocked.Decrement(ref EnlistmentCounts);
        enlistment.Done();
    }

    public void InDoubt(Enlistment enlistment)
    {
        Interlocked.Decrement(ref EnlistmentCounts);
        enlistment.Done();
    }
}    

This class simply tracks the number of instances that it has. It does no blocking and operates entirely in memory.

Here is the code to to show the problem:

var newGuid = Guid.NewGuid();
for (int i = 0; i < 100; i++)
{
    using(var tx = new TransactionScope())
    {
        Transaction.Current.EnlistDurable(newGuid, new EnlistmentTracking(), EnlistmentOptions.None);
        Transaction.Current.EnlistDurable(newGuid, new EnlistmentTracking(), EnlistmentOptions.None);

        tx.Complete();
    }

    Console.WriteLine(Thread.VolatileRead(ref EnlistmentTracking.EnlistmentCounts));
}

This just run in a loop, creating two instances of the enlistment (forcing it to be distributed transaction), and commit the transaction. After the transaction is completed, we read how many enlistments are still alive. Surprisingly, I keep getting non zero values here.

The really freaky part is that if I’ll put a small wait there, I’ll get zero value back, which is what I would expect. This is on .NET 4.0, by the way.

Let us look at the documentation for Dispose:

This method is synchronous and blocks until the transaction has been committed or aborted.

Hmm… that is not what I am seeing here.

Any idea what is going on?

From what I see here, I would say that it is only waiting until Prepare is called, not until Commit / Rollback is called. The way I implemented things, prepare does all the actual work, but it is the commit that switch things around so those changes are visible. The result of this behavior is that until Commit has been called, the transaction has not been really committed.

It appears that what I am supposed to do is:

  • On prepare, commit the transaction, but keep around the data required to roll it back.
  • On commit, cleanup everything that is required to do the cleanup.
  • On rollback, use the cleanup data to rollback the transaction.
  • On doubt, dance a merry jig and then throw yourself off the bridge.

But that is based on the behavior of the code, not on what I am seeing on the docs, and it is seems wrong.

Porting MVC Music Store to Raven: Porting the checkout process

The checkout process in the MVC Music Store is composed of two parts, adding address & payment options and completing the order.

The old code for address & payment is on the left, the new on the right.

image image

As you can see, they are quite similar. Raven’s code isn’t complete yet, though.

If you’ll recall, we stated that we are going to store the CountSold property inside the Album document, to allow us to easily sort by that count. We now need to write that logic, I put it directly after the call to CreateOrder:

image

It is important to note that we are loading all the albums document in a single query. And when we save, Raven is going to make a single (batched) call to the server.

And now, merely to completion sake (pun intended) let us look at the Complete method:

image image

I think by now you can tell what is going on in each system. The next post will cover the administration section.

Tags:

Published at

Originally posted at

Porting the MVC Music Store to Raven: ShoppingCartController

The ShoppingCartController is heavily affected by the changes we made to the ShoppingCart. Let us look at those changes, starting from Index():

image image

On the left, we have the original version. You can see that it executes two different queries to process this order, the Raven version, however, is executing only a single query, in the FindShoppingCart method:

image

This just implements the logic of loading the cart from Raven or creating a new cart (with the specified shopping cart id), note that we don’t save the new cart to the database here, merely associate the new cart with the session. There is no need to save, since it contains no meaningful data at this point. When we will call SaveChanges(), the new cart will be send to Raven for storage.

Let us look at the AddToCart action now:

image  image

On the left, you can see the old version, and on the right, you see the Raven version. They are pretty similar, except that in the Raven case, the shopping cart’s AddToCart is concerned solely with adding a new item to the cart or incrementing the quantity of an existing item. There is absolutely no data access in the Raven’s version of ShoppingCart.AddToCart.

One major difference is that the Raven approach is calling the session.SaveChanges() in the action code. The reason for that is simple, it is the proper place for this, as the calling code, it is responsible for the environment, including saving when needed.

image image

Raven’s code is pretty easy to follow here, I think. There is just one thing that you should note, the last line is pretty strange id.Split ? Why do we do that?

The reason for that is that Raven uses id that looks like this: “albums/616” and the DeleteId is used by the calling javascript code to find an element by its id. An element can’t contain a ‘/’ in it, so we strip it away and only send the number part of the id. This is safe to do since we only deal with albums here.

image image

Again, this is about as simple as you can make it, so I’ll note only that Raven’s approach can benefit from the unit of work cache, and the old code approach can’t.

In my next post, we will deal with the order process.

Tags:

Published at

Originally posted at

Comments (15)

Why won’t you take my money, you stupid moronic dummy?

I like reading, specifically, science fiction and fantasy has always had a huge pull on my imagination. That let me to a great series of books by Ilona Andrews, the latest of which has just come out.

I live in Israel, and shipping time & cost from the US is quite prohibitive. At times, I have paid four or five times the cost of the book just to be able to get the bloody thing. So the rise of the Kindle filled me with a great sense of relief. I got a Kindle and started reading on that (I love it). I estimate that I read over 200 books on the thing already.

One thing that I really like is the ability to pre order a book, get it on my Kindle when it is out, and start reading immediately. But recently I have gotten several notifications about books that I really wanted to read. Here is one:

We're writing to let you know that we've canceled your order for Magic Bleeds because it will not be released by the publisher in Kindle format on  Tuesday, May 25, 2071 as previously expected. We don't yet have a date for when this item will be released for Kindle. We will send you an email notifying you when the Kindle edition becomes available.

Okay, annoying. Let us hit audible.com and see if they have the audio book version available. It appears that this is not the case… but I can buy the CD version, which requires physical shipping, from Amazon.

Where does it leaves me?

There seems to be no way for me to get the book in a reasonable timeframe/cost.

Wait, let me rephrase that. There appears to be no legal way. While I have no direct knowledge of that, I am guessing that if I hit a torrent site and try to search for the book, I would not only find the book, but will be able to get the freaking thing faster than going with the legal download route.

It is actually quite simple. I would really like to give you some money, if you make it harder for me to give you money, you won’t get my money.

This decision is stupid, moronic, idiotic, senseless, irritating, annoying and in general lack all sense.

~Ayende the annoyed

Porting the MVC Music Store to Raven: ShoppingCart

image

The ShoppingCart class in the MVC Music Store is my current punching bug, I really don’t like it.

You can see how it looks on the image on the right. The problem is that this code mixes two different responsibilities:

  • Operations about a shopping cart:
    • GetCart
    • GetCartId
    • GetCartItems
    • GetCount
  • Operations of a shopping cart:
    • AddToCart
    • CreateOrder
    • EmptyCart
    • MigrateCart
    • RemoveFromCart

You might have noticed that all the operations about a shopping cart are get operations. All the operations of the cart are things that belong to the cart, it is the cart’s business logic and reason for being. The Get operations don’t belong to the cart, they belong in some other object that manages instances of carts.

In most applications, we would call this object a Repository. I am not sure that we really need one here. Looking at the Get methods, most of them are here because of the decision to only store cart line items, which requires us to issue explicit queries to get the data.

With Raven, we would follow a different model, which means that the only thing we are likely to need is GetCart() and maybe GetCartId().

Here is how a typical cart document will look like as a document:

image

And as an entity in our application:

 image

The GetTotal method was replaced with a Total property.  Until the GetTotal method, with issued a query to the database, this property operates solely in memory. This is another major difference with Raven vs. OR/M solution is that Raven doesn’t do lazy loading. This is by design, since document dbs data models rarely need to traverse data outside their own document. Traversing the document from Raven cannot force lazy loading or result in the dreaded SELECT N+1 issues.

And now let us deal with the operations about a cart. The most important ones are GetCartId and GetCart. I think that those methods has no place there. I created a new class, ShoppingCartFinder, which looks like this:

image

Note that we don’t expose GetCartId anymore, this is an internal detail that shouldn’t be seen by clients of this class. We do need to support setting the cart id, because we also support cart migrations (when an anonymous users logs in). We don’t need any of the other methods, so I removed them.

Let us go over the operations of the cart.

image image

The method on the left is the original code, and on the right you can see Raven’s code. The Raven code operates completely in memory, and in totally persistence ignorance. The old code is deals explicitly with persistence. This isn’t that much of a problem, except that this is the wrong level to deal with persistence issues.

image image

 

Going over RemoveFromCart, you can see that it shrunk significantly in size, and again, it is an in memory operation only. EmptyCart isn’t implemented in the Raven version, since it is just a Lines.Clear()

It is interesting to note that EmptyCart in the old implementation would result in N queries, where N is the number of items in the cart, while with Raven, this will result in 1 query.

imageimage

I don’t think that there is much to say here, except that the old code would execute N*2 queries, while Raven’s code will still execute 1 query :-)

MigrateCart is interesting, because the implementation is drastically different:

image image

With the old code, we update all the items in the cart, one at a time. With Raven, we do something drastically different. The Shopping Cart Id is the document key, so given the shopping cart id (which is the user name or stored in the session), we can load the shopping cart in using a Load (by primary key, to equate to the relational mindset). Migrating a cart is a simple enough operation, all you have to do is change the key. Since Raven doesn’t allow renames, we do it with a Delete/Store, which are executed inside a single transaction.

The calling code for MigrateCart looks like this:

image 

Note that SaveChanges is atomic and transactional, so this has the same effect as issuing a rename.

And that is it for the shopping cart, in my next post, I’ll discuss the ShoppingCartController which uses this class.

Tags:

Published at

Originally posted at

Comments (15)

The down side of porting

I got to this code as a result of a profiling session (shown below):

image

And here is how I got there:

 

image

Well, the fun part is that by the time I found it, it got fixed :-)

TechEd Israel 2010 may only accept speakers from sponsors

This post is copied (with permission) from Roy Osherove. I don’t often do things like that but Roy’s post has pushed a lot of red buttons.

Let me just hand you over to Roy:

A month or so ago, Microsoft Israel started sending out emails to its partners and registered event users to “Save the date!” – Micraoft Teched Israel is coming, and it’s going to be this november!

“Great news” I thought to myself. I’d been to a couple of the MS teched events, as a speaker and as an attendee, and it was lovely and professionally done. Israel is an amazing place for technology and development and TechEd hosted some big names in the world of MS software.

A couple of weeks ago, I was shocked to hear from a couple of people that Microsoft Israel plans to only accept non-MS teched speakers, only from sponsors of the event. That means that according to the amount that you have paid, you get to insert one or more of your own selected speakers as part of teched.

I’ve spent the past couple of weeks trying to gather more evidence of this, and have gotten some input from within MS about this information. It looks like that is indeed the case, though no MS rep. was prepared to answer any email I had publicly. If they approach me now I’d be happy to print their response.

What does this mean?

If this is true, it means that Microsoft Israel is making a grave mistake –

  • They are diluting the quality of the speakers for pure money factors. That means, that as a teched attendee, who paid good money, you might be sitting down to watch nothing more that a bunch of infomercials, or sub-standard speakers – since speakers are no longer selected on quality or interest in their topic.
  • They are turning the conference from a learning event to a commercial driven event
  • They are closing off the stage to the community of speakers who may not be associated with any organization  willing to be a sponsor
  • They are losing speakers (such as myself) who will not want to be part of such an event. (yes – even if my company ends up sponsoring the event, I will not take part in it, Sorry Eli!)
  • They are saying “F&$K you” to the community of MVPs who should be the people to be approached first about technical talks (my guess is many MVPs wouldn’t want to talk at an event driven that way anyway )

I do hope this ends up not being true, but it looks like it is. MS Israel had already done such a thing with the Developer Days event previouly held in Israel – only sponsors were allowed to insert speakers into the event.

If this turns out to be true I would urge the MS community in Israel to NOT TAKE PART AT THIS EVENT in any form (attendee, speaker, sponsor or otherwise). by taking part, you will be telling MS Israel it’s OK to piss all over the community that they are quietly suffocating anyway.

The MVP case

MS Israel has managed to screw the MVP program as well. MS MVPs (I’m one) have had a tough time here in Israel the past couple of years. ever since yosi taguri left the blue badge ranks, there was not real community leader left. Whoever runs things right now has their eyes and minds set elsewhere, with the software MVP community far from mind and heart. No special MVP events (except a couple of small ones this year). No real MVP leadership happens here, with the MVP MEA lead (Ruari) being on a remote line, is not really what’s needed.

“MVP? What’s that?” I’m sure many MS Israel employees would say. Exactly my point.

Last word

I’ve been disappointed by the MS machine for a while now, but their slowness to realize what real community means in the past couple of years really turns me off. Maybe it’s time to move on. Maybe I shouldn’t be chasing people at MS Israel begging for a room to host the Agile Israel user group. Maybe it’s time to say a big bye bye and start looking at a life a bit more disconnected from that giant. I hear the people at Google are pretty Agile!

And now back to me. I had more discussions in the last two years with Microsoft UK than with Microsoft Israel. I think it says it all, and I am an Israeli MVP who spends most of his time in Israel.

Tags:

Published at

Originally posted at

Comments (5)

Porting MVC Music Store to Raven: StoreController

We will start with the Index() method:

image

There are one this in this code that bothers me, and that is that this code is going to perform two DB queries. But that is beside the point, since we are going to modify the whole thing.

And here is my port:

image

As you can see, it is pretty much the same, and not really that interesting. Let us see what else we have:

image

Something that is important to note here is that we are doing a search on the name of a genre. The problem is that the genre name isn’t the primary key, worse, there isn’t even an index on the name column. Now, admittedly, the genre table contains ten rows, but it is the principal of the thing. (If you are smart, you only have to be read the riot act by the DBA about non index queries in production once).

Now, it would be trivial for us to implement this in Raven using the same approach, but I don’t see a reason to do this. The genre that we get in the Browse method is dependant on the data that we return from the Index method, so there is no reason no to pass the id of the genre directly. I modified the Index() action to pass the entire genre, not just the genre name, and to pass the id back to the Browse() action, not the name.

I gotten started implementing this, but I got stuck on the association of Albums from the genre.

image

Document database doesn’t normally have associations, and they don’t have joins. So how can we do this?

By now, you should be pretty familiar with the answer, we need to define an index :-)

// AlbumsByGenre
from album in docs.Albums
where album.Genre != null
select new { Genre = album.Genre.Id }

And this index allows us to write this code:

image

And finally, we have the GenreMenu:

image

Which we can port very easily:

image

And that is all for the StoreController

Tags:

Published at

Originally posted at

Comments (5)

Porting MVC Music Store to Raven: Advanced Migrations

I noticed that I had  typo when I inserted the albums data, the artist data was stored as “Arist”. This give me a chance to show you how we can do a migration that is a bit more advanced.

using (var documentStore = new DocumentStore { Url = "http://localhost:8080" })
{
    documentStore.Initialise();

    var count = 0;

    do
    {
        var queryResult = documentStore.DatabaseCommands.Query("Raven/DocumentsByEntityName", new IndexQuery
        {
            Query = "Tag:`Albums`",
            PageSize = 128,
            Start = count
        });


        if (queryResult.Results.Length == 0)
            break;

        count += queryResult.Results.Length;
        var cmds = new List<ICommandData>();
        foreach (var result in queryResult.Results)
        {
            var arist = result.Value<JObject>("Arist");
            if(arist == null)
                continue;
                        
            result["Artist"] = arist;
            result.Remove("Arist");

            cmds.Add(new PutCommandData
            {
                Document = result,
                Metadata = result.Value<JObject>("@metadata"),
                Key = result.Value<JObject>("@metadata").Value<string>("@id"),
            });
        }

        documentStore.DatabaseCommands.Batch(cmds.ToArray());

    } while (true);
    
}

The code itself should be hard to follow I think, it shows how we can manipulate documents by working with the JSON document directly, instead of having to go through an object layer.

Tags:

Published at

Originally posted at

Comments (2)

A bug that drove me crazy!

I am ashamed to say how much time and effort (and at least one complete re-design) this bug has cost me.

Take a look at the following test, meant to show that Raven can replicate deletes between servers:

image

There is a very subtle bug in this code, which completely tripped me.

Here is a hint, the session interface is the high level client API for Raven.

Can you spot the bug?

Porting MVC Music Store to Raven: Migrations

On my last post, I mention that we need to add a CountSold property to all the albums, in most SQL system, something like that can be pretty painful. The syntax for adding a new column is easy, but actually getting it done, and deployed, and versioned, is pretty hard. With Raven, if you add a new property, it will automatically be added to your document when you next save it. There is no action required on your part. The same, by the way, would happen when you remove a property. Raven will clean it up after you.

The question is what happens when we want to set that value to something, not just to the default value? We need to provide that logic somehow, and here is a simple way of doing so;

using (var documentStore = new DocumentStore { Url = "http://localhost:8080" })
{
    documentStore.Initialise();
    using (var session = documentStore.OpenSession())
    {
        IDictionary<string,int> albumToSoldCount = new Dictionary<string, int>();
        int count = 0;

        do
        {
            var results = session.Query<SoldAlbum>("SoldAlbums")
                .Take(128)
                .Skip(count)
                .ToArray();

            if (results.Length == 0)
                break;
            count += results.Length;
            foreach (var soldAlbum in results)
            {
                albumToSoldCount[soldAlbum.Album] = soldAlbum.Quantity;
            }
        } while (true);

        count = 0;
        do
        {
            var albums = session.Query<Album>()
                .Skip(count)
                .Take(128)
                .ToArray();
            if (albums.Length == 0)
                break;

            foreach (var album in albums)
            {
                int value;
                albumToSoldCount.TryGetValue(album.Id, out value);

                album.CountSold = value;
            }

            count += albums.Length;

            session.SaveChanges();
            session.Clear();
        } while (true);
    }
}

To those of you who haven’t bother to read the code, this is reading the index that we previously created and remembering its value. Then we start reading batches of albums and update their counts. All in all, it is quite simple.

An additional nice property of this script is that you can run it is safe to run it multiple times.

Tags:

Published at

Originally posted at

Comments (17)

Porting MVC Music Store to Raven: Porting the HomeController, the Right Way

As I mentioned, we can solve the GetTopSellingAlbums() problem using map/reduce, but that isn’t really a good way of doing it. The problem with doing that (aside from the scared looks and pained sounds that you get when you mention it) is that it is trying to solve the problem in a relational way. Indeed, the previous solution was an near duplication of how a relational database would process that query. So, what is the doc db approach for solving this issue?

The answer is quit simple, remember that documents are independent, and think about the question. What we are asking is what are the top selling albums. If we add a CountSold property to the album, we would suddenly find it so much easier to handle this problem. This means that we would need to update all the albums that are part of a given order when an order is submitted, but that is acceptable (this exact same operation is commonly done in SQL databases as well).

For now, let us waive how we create the CountSold property and fill it with the right values (I’ll discuss it in my next post), for now, assume that this happened, how can we GetTopSellingAlbums() problem?

Well, that is easy enough. All we need to do is define an index for CountSold.

// AlbumsByCountSold
from album in docs.Albums
select new { album.CountSold };

With that, we can implement GetTopSellingAlbums like this:

image

And now it is done, very simple, very efficient and quit elegant, eve if I say so myself.

Tags:

Published at

Originally posted at

Comments (24)

Find the design flaw

This method has a design flaw, can you see it?

image

Hint, it is using the yield keyword.

Tags:

Published at

Originally posted at

Comments (16)

The RavenDB launch video

A few days ago, RavenDB launched in Skills Matter UK office, a lot of people showed up and there seem to be a lot of interest in Raven.

The session was recorded and can be found here. I know you won’t believe me, but that was planned to be a 60 – 70 minutes talk, it went for twice that.

Tags:

Published at

Porting MVC Music Store to Raven: Porting the HomeController, the map/reduce way

The current HomeController looks like this:

image

I really don’t like the fact that the controller issues queries like that, but we will let it go for now.

This query (thanks to EF Prof) looks like this:

image

And here we run into a very interesting problem, we can’t really replicate this query. The reason is that this query runs over multiple tables which our model says would be in different documents.

There are several ways in which we can fix this. One way of doing this would be to define a map / reduce index on top of orders.

Note: Yes, I am familiar with this comic.

The way that I am about to show you isn’t the way I would recommend going for real, but I want to show it anyway. I’ll discuss the idiomatic Raven way of handling this feature in my next post.

Map/reduce in Raven is just a couple of Linq queries, so it is nothing to be worried about. As a reminder, we have the following order documents in our database:

image_thumb9 image image

We define the index “SoldAlbums” using the following queries.

// map
from order in docs.Orders
from line in order.Lines
select new{ line.Album, line.Quantity }

// reduce
from result in results
group result by result.Album into g
select new{ Album = g.Key, Quantity = g.Sum(x=>x.Quantity) }

As you can see, those are two very simple Linq queries.

The result of which would be:

image

Once we have that, it is trivial to derive the answer to GetTopSellingAlbums. Indeed, the following function implements the exact same logic and has the same output as the previous implementation:

image

The way it work is pretty simple, we get the most sold albums (by sorting on descending quantity), then load them from the database. Because we might have less than count top selling albums, we need to top it off from regular albums.

This mean that this code execute 2 – 3 queries. I don’t really like it, but on my machine, it takes about less than 10 ms to do all three requests, which is livable.

The reason that I am posting this solution is that I want to show this as an approach to a problem, not as the recommended approach for how to solve it, I’ll do that in my next post.

Tags:

Published at

Originally posted at

Comments (15)

C# Coding Challenge: What will this code do?

What is the output of this code?

 IDictionary<object,string> instanceToKey = new Dictionary<object, string>();

 IDictionary<string, int> keyToCost = new Dictionary<string, int>();

 var inst1 = new object();
 var inst2 = new object();

 instanceToKey[inst1] = "one";
 keyToCost["one"] = 1;

 instanceToKey[inst2] = "two";
 keyToCost["two"] = 2;

 string value = null;
 foreach (var key
     in (from inst in new[]{inst1, inst2, new object()}
         where instanceToKey.TryGetValue(inst, out value) 
         select value))
 {
     int cost;
     if(keyToCost.TryGetValue(key, out cost))
         Console.WriteLine("Key: {0}, Cost: {1}", key, cost);
 }
Tags:

Published at

Originally posted at

Comments (18)

Porting MVC Music Store to Raven: Data migration

Here is the code required to take the data in the MVC Music Store database and turn into the appropriate Raven documents:

using (var documentStore = new DocumentStore { Url = "http://localhost:8080" })
{
    documentStore.Initialise();
    using (var session = documentStore.OpenSession())
    {
        foreach (var album in storeDB.Albums.Include("Artist").Include("Genre"))
        {
            session.Store(new
            {
                Id = "albums/" + album.AlbumId,
                album.AlbumArtUrl,
                Arist = new { album.Artist.Name, Id = "artists/" + album.Artist.ArtistId },
                Genre = new { album.Genre.Name, Id = "genres/" + album.Genre.GenreId },
                album.Price,
                album.Title,
            });
        }
        foreach (var genre in storeDB.Genres)
        {
            session.Store(new
            {
                genre.Description,
                genre.Name,
                Id = "genres/" + genre.GenreId
            });
        }
        session.SaveChanges();
    }
}

As you can see, it is pretty simple and, even if I say so myself, pretty slick.

I use anonymous types here because I am only concerned with porting the data, I don’t really care about how to deal types right now.

Tags:

Published at

Originally posted at

Comments (14)

Porting MVC Music Store to Raven: Setting up the application

Just a few words about the way that I setup Raven to be used in the MVC Music Store application before we get to the actual code.

  • The model is (intentionally) very close to the one used by NHibernate. We initialize the document store in the application start.
  • We then open/close the session on request boundary, and create a way to access the current session.
  • If the application supported a container, I would make sure that the controllers got the session instance through that, but it doesn’t, so I just used static gateway.
    • If you don’t like, feel free to submit a patch.
public class MvcApplication : System.Web.HttpApplication
{
    private const string RavenSessionKey = "Raven.Session";
    private static DocumentStore _documentStore;

    protected void Application_Start()
    {
        _documentStore = new DocumentStore { Url = "http://localhost:8080/" };
_documentStore.Initialise(); AreaRegistration.RegisterAllAreas(); RegisterRoutes(RouteTable.Routes); } public MvcApplication() { BeginRequest += (sender, args) => HttpContext.Current.Items[RavenSessionKey] = _documentStore.OpenSession(); EndRequest += (o, eventArgs) => { var disposable = HttpContext.Current.Items[RavenSessionKey] as IDisposable; if (disposable != null) disposable.Dispose(); }; } public static IDocumentSession CurrentSession { get { return (IDocumentSession) HttpContext.Current.Items[RavenSessionKey]; } } }

This is pretty much it, as far as Raven’s initialization is concerned.

Tags:

Published at

Originally posted at

Comments on RavenDB licensing

There appears to be some confusion with regards to RavenDB licensing terms. In part, this is because some people believe that it is priced too highly.

There is currently a debate going on in the mailing list, which I suggest you’ll join. Yes, I am open sourcing the decision process for the pricing model :-)

That said, leaving the pricing aside, we still need to consider the licensing terms, and what they mean. RavenDB is dual licensed, which means that it is available under a standard commercial license and under the AGPLv3 license. The AGPL is one of the most restrictive OSS licenses. Interestingly enough, if you ask the authors of the AGPL they will probably say that it is one of the most free OSS licenses, but that is a discussion for another time.

The AGPL is similar to the GPL in that it is viral, but it has the additional property that it defines distribution as also including web requests and computer networks. (Oversimplifying a bit, I know, but that is why the AGPL FAQ exists).

RavenDB licensing terms are pretty simple:

  • You can use RavenDB with OSS projects (OSI approved licenses), and you don’t have to modify your license to match RavenDB’s licenses. The AGPL doesn’t normally allow it, RavenDB contains a specific exception to allow that.
  • If you want to use RavenDB in commercial closed source software, you need to buy a commercial license.

The last statement is where most people have an issue with, so let me be even clearer:

If you are using RavenDB for an internal application that no external user will access, then the distribution clause never comes into effect.

  • Note, however, that distributing AGPL applications also include making them available to users / systems outside your own organization.
  • Well, the previous point isn’t strictly true, even for internal users, in order to comply with the license, you must allow them a way to get the source.

What if you are using RavenDB as a backend server for your application? RavenDB is never accessed from the outside, so it is not distributed. And even the AGPL doesn’t say that making a web request to an AGPL server forces you to be AGPL’ed. That would be an accurate point, except that RavenDB comes in two pieces, one of which is the server, but the second is the client API. And if you use the client API (which is also licensed under the AGPL), your application now must be AGPL’ed.

That is not a problem, right? RavenDB is a REST based system, you can just write your own client, and license that under the BSD.

Not quite.

Why is that? Because of something called a derived work. If you creating a standard HTTP client (PUT, GET, DELETE, etc) that can work against RavenDB, there is nothing that would stop you from doing that. If, however, your client is built to take advantage of RavenDB’s specific features (replication, transactions, batching, patching, etc), then it stopped being a standard HTTP client and became a derived work of RavenDB. At which point, as you probably guessed, the AGPL kicks in again.

All the legalities aside, my intent is that if you use RavenDB commercially, you pay for a commercial license.

RavenDB goes live!

I have been talking about this for what seems like forever (over two years, if you really care to know), but it is finally out.

image

You can access our site at http://ravendb.net

What you will find there is:

  • Tutorials
  • Sample application
  • Extensive documentation
  • Commercial options & support
  • An active community

Head on there, and please let us know what you think…

But before you go, I would like to send my heartfelt thanks for the following people, for their help & support:

  • Aaron Weiker
  • Andrew Stewart
  • Andrew Theken
  • Andy Stewart
  • Benny Thomas
  • Bjarte Skogoy
  • Bobby Johnson
  • Emil Cardell
  • Luke Hertert
  • Mark Henke
  • Paul B
  • Rob Ashton
  • Steve Strong

Heading to the launch event now. Fly Raven, Fly :-)