http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Jack, I could have create a bad impl of a graph db, or created a really awesome doc db.http://ayende.com/157441/self-joins-in-ravendb#comment20http://ayende.com/157441/self-joins-in-ravendb#comment20Wed, 08 Aug 2012 04:01:36 GMTThis sort of stuff is trivially simple with graph databases. If the relationships between records are equally as important as the record contents, I would recommend you check out graph db's and some of the algorithms that they make simple. Oren: I know you were looking at implementing a graph db on top of Raven at some point, but I understand that you dropped it due to market size? I wish you had continued with that.http://ayende.com/157441/self-joins-in-ravendb#comment19http://ayende.com/157441/self-joins-in-ravendb#comment19Wed, 08 Aug 2012 03:21:02 GMTAndreas, The mailing list is the best place for that. We have a BIG documentation push coming up.http://ayende.com/157441/self-joins-in-ravendb#comment18http://ayende.com/157441/self-joins-in-ravendb#comment18Tue, 07 Aug 2012 13:55:55 GMTHi Ayende, man.... I must admit you never cease to amaze with RavenDB. I never thought that recursive actions would be part of the concept. Awesome!!! Is there any source I could read more about those things? Thanks Andreashttp://ayende.com/157441/self-joins-in-ravendb#comment17http://ayende.com/157441/self-joins-in-ravendb#comment17Tue, 07 Aug 2012 13:54:16 GMTThanks Ayende That is super cool! Will try dig into the implementation... Ajaihttp://ayende.com/157441/self-joins-in-ravendb#comment16http://ayende.com/157441/self-joins-in-ravendb#comment16Tue, 07 Aug 2012 04:56:04 GMTAjai, We store some of the intermediate results, exactly for this purpose. It basically works by saying: Delete C1, figure out what part of the map/reduce would be affected, re-run those parts.http://ayende.com/157441/self-joins-in-ravendb#comment15http://ayende.com/157441/self-joins-in-ravendb#comment15Tue, 07 Aug 2012 04:46:02 GMTHi Ayende Not understanding this correctly. Say: C1 cites C2 & C3 C4 cites C2 Run map/reduce: C2 -> TimesCited = 2 C3 -> TimesCited = 1 When C1 is deleted how can indexing work just by reprocessing just the deleted record? Do we not need to rerun map/reduce on whole collection to recompute TimesCited for all then cases? Or are somehow intermediate results also stored? Any pointers on the implementation or link to relevant source is much appreciated... Thanks Ajai http://ayende.com/157441/self-joins-in-ravendb#comment14http://ayende.com/157441/self-joins-in-ravendb#comment14Tue, 07 Aug 2012 04:43:42 GMTAndreas, RavenDB contains builtin support for recursion. Here is an example on one such index: from ex in examples from ov in ex.Overrides let ancestry = Recurse(ov, x => x.Parent) select new { ex.Id, ov.OwnerId, Name = ov.OverriddenValues["Name"] ?? (ancestry.Any(x => x.OverriddenValues["Name"] != null) ? ancestry.First(x => x.OverriddenValues["Name"] != null).OverriddenValues["Name"] : ex.Name), Description = ov.OverriddenValues["Description"] ?? (ancestry.Any(x => x.OverriddenValues["Description"] != null) ? ancestry.First(x => x.OverriddenValues["Description"] != null).OverriddenValues["Description"] : ex.Description) }); You could do this in the transform results and load the entire hierarchy along with the child object.http://ayende.com/157441/self-joins-in-ravendb#comment13http://ayende.com/157441/self-joins-in-ravendb#comment13Tue, 07 Aug 2012 04:08:45 GMTAjai, We would re-run things for the specific case that was updated/deleted.http://ayende.com/157441/self-joins-in-ravendb#comment12http://ayende.com/157441/self-joins-in-ravendb#comment12Tue, 07 Aug 2012 04:06:43 GMTHi Ayende, thanks for the clarification. You are - as most of the time - right that this requirement would indicate a needed change in the model itself. A change in a single node could otherwise trigger a lot of changes in the index and that really would not be performant at all. As Configurator was also interested in some sort of hierarchical information.... how would you tackle such a task so it could be implemented with RavenDB? The classical example in SQL is having an org-chart (person and his/her manager) and the usage of a hierarchical function. You can query for a single node and have all its children or all its parents in the "flat" output. How would that translate to RavenDB? A recursive function in SQL is no "index" of any sort, but rather a function running on the server to "iterate" over the hierarchical layers and aggregate the data. How would you perform that sort of task in RavenDB? Thanks Andreashttp://ayende.com/157441/self-joins-in-ravendb#comment11http://ayende.com/157441/self-joins-in-ravendb#comment11Mon, 06 Aug 2012 22:29:56 GMTJust curious on how re indexing works in RavenDB. Say I delete a Case, or update a case (add/remove cases CitedByThisCase) - do you re-run map/reduce on the whole case document collection? Ajaihttp://ayende.com/157441/self-joins-in-ravendb#comment10http://ayende.com/157441/self-joins-in-ravendb#comment10Mon, 06 Aug 2012 20:08:05 GMTConfigurator, If the Children property on the parents was recursive (So it contains ALL children, not just direct ones), yes, easily. Otherwise, no.http://ayende.com/157441/self-joins-in-ravendb#comment9http://ayende.com/157441/self-joins-in-ravendb#comment9Mon, 06 Aug 2012 16:10:30 GMTThe question is, could you prepare a map/reduce index so that it can be done in a single query, e.g. the results would be { document: "docs/17", parents: [ "docs/4", "docs/9", "docs/13" ] }http://ayende.com/157441/self-joins-in-ravendb#comment8http://ayende.com/157441/self-joins-in-ravendb#comment8Mon, 06 Aug 2012 16:09:20 GMTConfigurator, No need for map/reduce here, you could do that, but you would require multiple queries (O(H), where H is the height).http://ayende.com/157441/self-joins-in-ravendb#comment7http://ayende.com/157441/self-joins-in-ravendb#comment7Mon, 06 Aug 2012 16:02:24 GMTAndreas, You would have to have all of the parents available in the document. So: e-> a, b,c,d d->a,b,c c->a,b b-> a a If you don't have this, then no, you would have to do multiple passes, and we don't allow it.http://ayende.com/157441/self-joins-in-ravendb#comment6http://ayende.com/157441/self-joins-in-ravendb#comment6Mon, 06 Aug 2012 16:01:06 GMTHere's another interesting case. Say I have some kind of hierarchical data, and I want to show its tree. So when accessing a page /a/b/c/d we'd show a tree that shows all the parentage (a, b, and c), and d and its children. Suppose we model our documents so that they only have the Children property and not a Parent (or Ancestry), could you create a Map/Reduce index that returns the ancestry for an object?http://ayende.com/157441/self-joins-in-ravendb#comment5http://ayende.com/157441/self-joins-in-ravendb#comment5Mon, 06 Aug 2012 15:08:35 GMTSorry, the comment engine reformatted the picture of the nodes A has childs B and C C has childs D and E B, D and E have no childs http://ayende.com/157441/self-joins-in-ravendb#comment4http://ayende.com/157441/self-joins-in-ravendb#comment4Mon, 06 Aug 2012 11:22:44 GMTAyende, I was not trying to have a node have multiple parents, but rather leave the structure as you showed it, but instead of counting only immediate childs I'd like to count all childs below a node. A --- B | |---C----D |-----E So B, D and E would have a child count of 0 C has a child count of 2 A has a child count of 4 Would that be possible to achieve with map/reduce? Thanks Andreashttp://ayende.com/157441/self-joins-in-ravendb#comment3http://ayende.com/157441/self-joins-in-ravendb#comment3Mon, 06 Aug 2012 11:21:11 GMTAndreas, If you have all of the parents in each docs, that is easy. Otherwise, it gets complex and probably not worth it.http://ayende.com/157441/self-joins-in-ravendb#comment2http://ayende.com/157441/self-joins-in-ravendb#comment2Mon, 06 Aug 2012 10:47:57 GMTHi Ayende, cool stuff. And published totally at the right time. I just have this "problem" ahead of me. One interesting question would be if it is possible to count not only the immediate childs, but all childs that have the current item as parent. There are a few problems within that approach, because this only works if you have a directed and loop free graph. But if you take an org-chart as example, it is not possible that one person is above and below another person in the hierarchy. Do you also have a solution to count all descendants? Thanks Andreashttp://ayende.com/157441/self-joins-in-ravendb#comment1http://ayende.com/157441/self-joins-in-ravendb#comment1Mon, 06 Aug 2012 10:23:29 GMT