Ayende @ Rahien

Hi!
My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

, @ Q c

Posts: 5,973 | Comments: 44,525

filter by tags archive

Awesome RavenDB Feature of the dayEvil Patching

time to read 6 min | 1025 words

Oh, wait, that is actually Eval Patching.

From the very start, RavenDB supported the ability to patch documents. To send a command to the server with some instructions about how to modify a document or a set of documents. For example, we have this:

documentStore.DatabaseCommands.Patch(
    "blogposts/1234",
     new[]
                    {
                        new PatchRequest
                            {
                                Type = PatchCommandType.Add,
                                Name = "Comments",
                                Value = RavenJObject.FromObject(comment)
                            }
                    });

This approach works, is easy to understand and support, and is quite simple to implement.

Unfortunately, it is limited. Users have all sort of very complex scenarios that they want to run that we aren’t really suitable for. For example, if a user wanted to move from a FirstName, LastName properties to FullName, this won’t give that to you.

Enter Matt Warren, who has contributed some really nice features to RavenDB (like facets), and who contributed the ability to do patching by sending a JavaScript function to the server.

Here is how it works using the new syntax:

store.DatabaseCommands.Patch("blogposts/1234",
    new AdvancedPatchRequest
    {
        Script = "this.Comments.push(newComment)",
        Values = {{"newComment", comment}}
    });

Note that you can send variables to the server and they are exposed to your script.

How about our previous example of moving from FirstName, LastName to FullName? Let us see:

store.DatabaseCommands.UpdateByIndex("Raven/DocumentsByEntityName",
 new IndexQuery{Query = "Tag:Users"},
 new AdvancedPatchRequest
    {
        Script = @"
this.FullName = this.FirstName + ' ' + this.LastName;
delete this.FirstName;
delete this.LastName;
"
    }
);

So we support full computation abilities during the patch Smile. So now you can just modify things pretty much as you feel like.

Here are a few other interesting things you can do.

Remove an item by value from an array:

store.DatabaseCommands.Patch("blogposts/1234",
    new AdvancedPatchRequest
    {
        Script = "this.Tags.Remove(tagToRemove)"
        Values = {{"tagToRemove", "Interesting"}}
    });

Remove an item using a condition:

store.DatabaseCommands.Patch("blogposts/1234",
    new AdvancedPatchRequest
    {
        Script = "this.Comments.RemoveWhere(function(comment) { return comment.Spam; });"
    });

This isn’t all, mind, but I’ll keep the really cool part for my next post.

More posts in "Awesome RavenDB Feature of the day" series:

  1. (26 Jul 2012) Eval Patching, Part II–Denormalized References
  2. (24 Jul 2012) Evil Patching

Comments

Phillip Haydon

New API changes for patching are so hot.

Ender

What is RavenDB using to interpret JavaScript?

Frank Quednau

The really cool part must be the ability to send a *.patch file :)

jhovgaard

This is very nice! Starting from which version is this possible?

Btw, is there any plans on adding the ability to bulk modify documents directly from the Studio? It's very inconvenient that we need to write code (and compile it) to do "bigger" changes.

Thanks! jhovgaard

Ayende Rahien

jhovgaard , This is in the 203x and up builds, I think Suggest a way to do that from the studio.

Matt Warren

@Ender, it's using IronJS (https://github.com/fholm/IronJS/) to do this

Matt Warren

BTW, this means that RavenDB now is part of the NoSQL "cool-gang", it has an embedded JavaScript interpreter!!!!

Rafal

Great, I was wondering how long will it be before RavenDB has a javascript interpreter.

Michael Chandler

Very cool.

I wonder if you could use something like Deleporter to do it in C# instead of JS?

http://blog.stevensanderson.com/2010/03/09/deleporter-cross-process-code-injection-for-aspnet/

Ayende Rahien

Michael, What happens when you are running on a remote server? What happens if I don't WANT you to execute random unverifable code on my server?

Andreas Kroll

Awesome!

If this is not even the cool stuff as Ayende said, I just cannot wait until the follow-up post...

It's really just like magic to send javascript to a database to patch documents. This makes nearly all of my version migration concerns just ease away... :-)

RavenDB just gets better and better.... and.... to quote Ayende:

   "It just works...." :-)

You Sir, put a big smile on my face today...

Rob
Rob

Very cool. I have some uses I can think of. As you announce new features here, I've been wondering: what's the best way to know when a particular feature is supported on RavenHQ?

Ayende Rahien

Rob, All of those new features are going into the 1.2 version, which we expect will be out in a few months in a stable version. Once that is done, RavenHQ will start doing their own internal testing & rollout.

Michael Chandler

Ayende, you would need to serialize/deserialize the delegate (I don't know what the feasibility of doing this is). In terms of the code, presumably you could sandbox or limit it to certain calls. What context does the IronJS script run under?

I wasn't thinking about the technical implementation, but more so what the API could look like... maybe something like: Script = (Doc doc) => { doc.FullName = doc.FirstName + " " + doc.LastName; Remove.Property(() => doc.FirstName); Remove.Property(() => doc.LastName); })

Just thinking out loud...:)

Ayende Rahien

Michael, Between different machines? So the server now depends on the client? And you can't easily limit what they can do.

Andres

Michael +1

It could be great if you use the same approach / language than index definitions: All Javascript or all .NET or Linq

Matt Warren

@Michael, @Anders

I've got plans to implement a strongly typed C# api, with LINQ/lambdas, that would then produce the JavaScript for you. But it'll be a while before it's available.

Plus JavaScript is a nice fit in this case, because your're editing Json docs.

Andres

When RavenDb index also is indexing Json docs.

Ayende Rahien

Andres, Go and compare a CouchDB view to RavenDB index, then tell me you want to write it in JS

Matt Johnson

Really cool, but is there a security risk here? I see that input values are parameratized to prevented attacks similar to SQL injection, although I suppose a developer could screw that up with string concatenation when building their script if they wanted to.

How well does the underlying HTTP command encapsulate the script to prevent other types of escaping? I wouldn't want someone to be able to find a magic escape sequence that lets them enter random javascript into a field and have it executed in the patch command.

Matt Warren

@Matt

What you can do inside the script is pretty constrained. For instance, you only have basic JavaScript functions, plus a few helpers RavenDB injects for you.

Also you are only given access to the Json version of a doc, nothing else.

Damien Guard

Not to be confused with the Matt Warren that wrote LINQ to SQL and IQToolkit...

Rohland

Nice work.

Perhaps the name 'ScriptedPatchRequest' is more suitable? 'AdvancedScriptRequest' is rather limiting given the pace of development ;)

Matt Warren

@Damien

Unfortunately no, I'm not that Matt Warren. I wish I was though, writing LINQ-to-SQL and working on the C# Compiler team would be pretty cool.

I have got mistaken for him on SO a few times though!! People have thanked me for LINQ-to-SQL!!!!

Michael Chandler

Matt, awesome, that was my second train of thought.

Beyers

@Matt

Once you implement the strongly typed C# api people will be able to thank you for LINQ-to-JAVASCRIPT :)

Ayende Rahien

Matt, The purpose of this feature is to allow arbitrary code execution on the server. Now, there are safeguards there that make really hard to do BAD THINGS, but the whole point is to allow the user great level of freedom in how they patch the document. We support variables, so that you don't have to do things using string concat. Beyond that, it is the user responsibility to clean up their input.

Ayende Rahien

Rohland, That is an excellent suggestion!

Comment preview

Comments have been closed on this topic.

FUTURE POSTS

No future posts left, oh my!

RECENT SERIES

  1. Production postmortem (5):
    29 Jul 2015 - The evil licensing code
  2. Career planning (6):
    24 Jul 2015 - The immortal choices aren't
  3. API Design (7):
    20 Jul 2015 - We’ll let the users sort it out
  4. What is new in RavenDB 3.5 (3):
    15 Jul 2015 - Exploring data in the dark
  5. The RavenDB Comic Strip (3):
    28 May 2015 - Part III – High availability & sleeping soundly
View all series

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats