Ayende @ Rahien

It's a girl

Awesome RavenDB Feature of the day: Evil Patching

Oh, wait, that is actually Eval Patching.

From the very start, RavenDB supported the ability to patch documents. To send a command to the server with some instructions about how to modify a document or a set of documents. For example, we have this:

documentStore.DatabaseCommands.Patch(
    "blogposts/1234",
     new[]
                    {
                        new PatchRequest
                            {
                                Type = PatchCommandType.Add,
                                Name = "Comments",
                                Value = RavenJObject.FromObject(comment)
                            }
                    });

This approach works, is easy to understand and support, and is quite simple to implement.

Unfortunately, it is limited. Users have all sort of very complex scenarios that they want to run that we aren’t really suitable for. For example, if a user wanted to move from a FirstName, LastName properties to FullName, this won’t give that to you.

Enter Matt Warren, who has contributed some really nice features to RavenDB (like facets), and who contributed the ability to do patching by sending a JavaScript function to the server.

Here is how it works using the new syntax:

store.DatabaseCommands.Patch("blogposts/1234",
    new AdvancedPatchRequest
    {
        Script = "this.Comments.push(newComment)",
        Values = {{"newComment", comment}}
    });

Note that you can send variables to the server and they are exposed to your script.

How about our previous example of moving from FirstName, LastName to FullName? Let us see:

store.DatabaseCommands.UpdateByIndex("Raven/DocumentsByEntityName",
 new IndexQuery{Query = "Tag:Users"},
 new AdvancedPatchRequest
    {
        Script = @"
this.FullName = this.FirstName + ' ' + this.LastName;
delete this.FirstName;
delete this.LastName;
"
    }
);

So we support full computation abilities during the patch Smile. So now you can just modify things pretty much as you feel like.

Here are a few other interesting things you can do.

Remove an item by value from an array:

store.DatabaseCommands.Patch("blogposts/1234",
    new AdvancedPatchRequest
    {
        Script = "this.Tags.Remove(tagToRemove)"
        Values = {{"tagToRemove", "Interesting"}}
    });

Remove an item using a condition:

store.DatabaseCommands.Patch("blogposts/1234",
    new AdvancedPatchRequest
    {
        Script = "this.Comments.RemoveWhere(function(comment) { return comment.Spam; });"
    });

This isn’t all, mind, but I’ll keep the really cool part for my next post.

Tags:

Posted By: Ayende Rahien

Published at

Originally posted at

Comments

Phillip Haydon
07/24/2012 09:15 AM by
Phillip Haydon

New API changes for patching are so hot.

Ender
07/24/2012 09:17 AM by
Ender

What is RavenDB using to interpret JavaScript?

Frank Quednau
07/24/2012 09:18 AM by
Frank Quednau

The really cool part must be the ability to send a *.patch file :)

jhovgaard
07/24/2012 09:18 AM by
jhovgaard

This is very nice! Starting from which version is this possible?

Btw, is there any plans on adding the ability to bulk modify documents directly from the Studio? It's very inconvenient that we need to write code (and compile it) to do "bigger" changes.

Thanks! jhovgaard

Ayende Rahien
07/24/2012 09:25 AM by
Ayende Rahien

jhovgaard , This is in the 203x and up builds, I think Suggest a way to do that from the studio.

Matt Warren
07/24/2012 09:25 AM by
Matt Warren

@Ender, it's using IronJS (https://github.com/fholm/IronJS/) to do this

Matt Warren
07/24/2012 09:35 AM by
Matt Warren

BTW, this means that RavenDB now is part of the NoSQL "cool-gang", it has an embedded JavaScript interpreter!!!!

Rafal
07/24/2012 10:28 AM by
Rafal

Great, I was wondering how long will it be before RavenDB has a javascript interpreter.

Michael Chandler
07/24/2012 10:40 AM by
Michael Chandler

Very cool.

I wonder if you could use something like Deleporter to do it in C# instead of JS?

http://blog.stevensanderson.com/2010/03/09/deleporter-cross-process-code-injection-for-aspnet/

Ayende Rahien
07/24/2012 10:53 AM by
Ayende Rahien

Michael, What happens when you are running on a remote server? What happens if I don't WANT you to execute random unverifable code on my server?

Andreas Kroll
07/24/2012 11:25 AM by
Andreas Kroll

Awesome!

If this is not even the cool stuff as Ayende said, I just cannot wait until the follow-up post...

It's really just like magic to send javascript to a database to patch documents. This makes nearly all of my version migration concerns just ease away... :-)

RavenDB just gets better and better.... and.... to quote Ayende:

   "It just works...." :-)

You Sir, put a big smile on my face today...

Rob
07/24/2012 11:25 AM by
Rob

Very cool. I have some uses I can think of. As you announce new features here, I've been wondering: what's the best way to know when a particular feature is supported on RavenHQ?

Ayende Rahien
07/24/2012 11:30 AM by
Ayende Rahien

Rob, All of those new features are going into the 1.2 version, which we expect will be out in a few months in a stable version. Once that is done, RavenHQ will start doing their own internal testing & rollout.

Michael Chandler
07/24/2012 11:55 AM by
Michael Chandler

Ayende, you would need to serialize/deserialize the delegate (I don't know what the feasibility of doing this is). In terms of the code, presumably you could sandbox or limit it to certain calls. What context does the IronJS script run under?

I wasn't thinking about the technical implementation, but more so what the API could look like... maybe something like: Script = (Doc doc) => { doc.FullName = doc.FirstName + " " + doc.LastName; Remove.Property(() => doc.FirstName); Remove.Property(() => doc.LastName); })

Just thinking out loud...:)

Ayende Rahien
07/24/2012 12:01 PM by
Ayende Rahien

Michael, Between different machines? So the server now depends on the client? And you can't easily limit what they can do.

Andres
07/24/2012 01:26 PM by
Andres

Michael +1

It could be great if you use the same approach / language than index definitions: All Javascript or all .NET or Linq

Matt Warren
07/24/2012 02:02 PM by
Matt Warren

@Michael, @Anders

I've got plans to implement a strongly typed C# api, with LINQ/lambdas, that would then produce the JavaScript for you. But it'll be a while before it's available.

Plus JavaScript is a nice fit in this case, because your're editing Json docs.

Andres
07/24/2012 02:40 PM by
Andres

When RavenDb index also is indexing Json docs.

Ayende Rahien
07/24/2012 02:43 PM by
Ayende Rahien

Andres, Go and compare a CouchDB view to RavenDB index, then tell me you want to write it in JS

Matt Johnson
07/24/2012 03:37 PM by
Matt Johnson

Really cool, but is there a security risk here? I see that input values are parameratized to prevented attacks similar to SQL injection, although I suppose a developer could screw that up with string concatenation when building their script if they wanted to.

How well does the underlying HTTP command encapsulate the script to prevent other types of escaping? I wouldn't want someone to be able to find a magic escape sequence that lets them enter random javascript into a field and have it executed in the patch command.

Matt Warren
07/24/2012 03:55 PM by
Matt Warren

@Matt

What you can do inside the script is pretty constrained. For instance, you only have basic JavaScript functions, plus a few helpers RavenDB injects for you.

Also you are only given access to the Json version of a doc, nothing else.

Damien Guard
07/24/2012 08:15 PM by
Damien Guard

Not to be confused with the Matt Warren that wrote LINQ to SQL and IQToolkit...

Rohland
07/24/2012 08:29 PM by
Rohland

Nice work.

Perhaps the name 'ScriptedPatchRequest' is more suitable? 'AdvancedScriptRequest' is rather limiting given the pace of development ;)

Matt Warren
07/24/2012 09:26 PM by
Matt Warren

@Damien

Unfortunately no, I'm not that Matt Warren. I wish I was though, writing LINQ-to-SQL and working on the C# Compiler team would be pretty cool.

I have got mistaken for him on SO a few times though!! People have thanked me for LINQ-to-SQL!!!!

Michael Chandler
07/25/2012 12:05 AM by
Michael Chandler

Matt, awesome, that was my second train of thought.

Beyers
07/25/2012 12:21 AM by
Beyers

@Matt

Once you implement the strongly typed C# api people will be able to thank you for LINQ-to-JAVASCRIPT :)

Ayende Rahien
07/25/2012 03:49 AM by
Ayende Rahien

Matt, The purpose of this feature is to allow arbitrary code execution on the server. Now, there are safeguards there that make really hard to do BAD THINGS, but the whole point is to allow the user great level of freedom in how they patch the document. We support variables, so that you don't have to do things using string concat. Beyond that, it is the user responsibility to clean up their input.

Ayende Rahien
07/25/2012 03:50 AM by
Ayende Rahien

Rohland, That is an excellent suggestion!

Comments have been closed on this topic.