+972 52-548-6969
RavenDB 4.0 on DotNetCore RC2
time to read 1 min | 97 words
We have got pretty much everything working, and all tests are passing on Windows. We’ll test it tomorrow on Linux, but it is looking good.
I’m really excited about this. To start with, just moving to RC2 looks like it dropped about 20% of our test suite runtime. But it also removed a blocking issue that prevented us from getting a core feature up and running (dynamic code generation, which is required for user defined indexes).
Now we can get started on that 
.
Comments
Pleas look at this code: https://github.com/ayende/ravendb/blob/v4.0/src/Voron/Platform/Win32/Win32MemoryMapPager.cs#L123. When you cannot allocate more contiguous pages you remap the file to memory throwing out the part of the file that resides in memory, subsequent access to pages need to be fetched from disk. Are you sure you cannot live without whole file mapped to contiguous memory. I think you could live with less sized contiguous file chunks. Please take a look at this: http://stackoverflow.com/questions/6096485/how-to-dynamically-expand-a-memory-mapped-file/36155244#36155244. In Raven 4.0 there are more chances to fail to allocate more contiguous pages because you have separate Voron storage for indexes.
@Jesús López we actually implementing such a pager but it is more intended for 32 bits systems since it will work slowly compare to the current pager that just needs to do pointer arithmeticians to fetch a page. Also in 4.0 there is actually less chance to fail allocation since indexes with all their related storage (like map-reduce internal storage) will reside in a different file (one per index).
@Tall Weiss. I disagree, there is actually more chance to fail contiguous allocation. Lets have files F1 and F2 with sizes S1 and S2. You map F1 to memory and get A1 address, then you map F2 to memory, the address you get is roughly A2 = A1 + S1. So you cannot contiguously allocate space for F1 because the address A1 + S1 is owned by the F2 mapping. Please experiment with this as I did, memory map F1 and F2 and try to allocate alternatively more contiguous pages you will see that is is consistently failing.
@Tal Weiss the overhead to get a page pointer from the pager is insignificant compared to key comparisons you need to do. Additionally, pages have a constructor that takes the pointer as a parameter so you don't need to get the page pointer from the pager once the page is created.
What really degrades performance is fetching pages from disk again after a file grow.
Jesus, subsequent access to pages after the remapping do NOT need to go to disk. The OS merely map the same physical page to multiple virtual addresses, is all.
Bravo on getting the linux port underway - looking forward to that day.
Could you elaborate a bit more on the dynamic code generation part? I assume you're using Roslyn to compile code to an in memory assembly and load that into the .net core host process?
Hans, Yes, dynamic generation of code via Roslyn and then loading of the assembly
Comment preview