Ayende @ Rahien

Hi!
My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

, @ Q c

Posts: 5,953 | Comments: 44,408

filter by tags archive

Transitive Replication in RavenDB


TLDR;

Replication topologies make my head hurt.

One of our customers had an interesting requirement, several months ago:

image

Basically, he wanted to write a document at node #1, and have it replicate, through node #2, to node #3. That was an easy enough change, and we did that. But then we got another issue from a different customer, who had the following topology:

image

And that client problem is that when making a write to node #1, it would be replicated to nodes 2 – 4, each of which would then try to update the other two with the new replication information (it would skip node #1 because it is the source). That would cause… issues, because they already had that document in place.

In order to resolve that, I added a configuration option, which controls whatever the node that we replicate to should receive only documents that were modified on the current node, or whatever we need to include documents that were replicated to us from other nodes as well.

It is a relatively small change, code wise. Of course, documenting this, and all of the options that follows is going to be a much bigger task, because now you have to make a distinction between replicating nodes, gateway nodes, etc.


Comments

Rafal

I wonder if these customers REALLY needed bi-directional replication. This is not a very common setup and chances are they could have designed it in a better way - witho only one direction of replication and without cycles.

Maybe this should be handled like a pub/sub message distribution in message bus - each document type has its master/origin database where it can be inserted and updated and all other databases are only allowed to take read-only replicas of that document from the master. And a 'slave' for one document type can be a master of another type.

PS your captchas are barely readable or I'm just a computer programmed to consider itself a human

Dave

@Rafal - It's the google one that helps to read old book texts. One of the words is the real captcha the other is a word that you are helping to decode. But you are right they are a pita.

Valeriu Caraulean

Not related to topic:

You're publishing to RSS only post titles. Is it intentional?

Ayende Rahien

Valeriu, no, it isn't. I just updated the blog software, and it might be related to that, I'll check it next week.

Ayende Rahien

Valeriu, no, it isn't. I just updated the blog software, and it might be related to that, I'll check it next week.

Rude Spell Checker

s/whatever/whether/g

where 'g' means in all your other posts, too...

fschwiet

A little additional context, the transitive replication was needed to support blue/green deployment (in my case, at least). While the replication needed to happen across an arbitrary number of servers, only 2 of those servers would ever actually exist at any one point in time.

Michael L Perry

I showed you Correspondence while you were in Dallas. The way it solves the problem is simple. Every object is immutable. And any two objects with exactly the same values are considered to be the same object. The object can come from multiple directions and it won't be a duplicate.

El Guapo

I agree... I am very surprised this causes "issues"... The document is identical in every respect... Why does it matter if I receive another copy of an identical document? Hmmmmm....

eek
eek

Wow, really brainfucking!

Comment preview

Comments have been closed on this topic.

FUTURE POSTS

No future posts left, oh my!

RECENT SERIES

  1. The RavenDB Comic Strip (3):
    28 May 2015 - Part III – High availability & sleeping soundly
  2. Special Offer (2):
    27 May 2015 - 29% discount for all our products
  3. RavenDB Sharding (3):
    22 May 2015 - Adding a new shard to an existing cluster, splitting the shard
  4. Challenge (45):
    28 Apr 2015 - What is the meaning of this change?
  5. Interview question (2):
    30 Mar 2015 - fix the index
View all series

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats