Ayende @ Rahien

Hi!
My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

, @ Q c

Posts: 6,124 | Comments: 45,474

filter by tags archive

Spot the bug: Unmanaged memory traps

time to read 4 min | 769 words

Sometimes I forgot how good life is in the managed lane.  Then I do some unmanaged work and get a good reality check.

Let us look at the following structures:

    [StructLayout(LayoutKind.Explicit, Pack = 1)]
    public struct PageHeader
    {
        [FieldOffset(0)] public long Marker;

        [FieldOffset(8)] public ushort Lower;

        [FieldOffset(10)] public ushort Upper;

        [FieldOffset(12)] public int OverflowSize;

        [FieldOffset(16)] public int ItemCount;
    }


    [StructLayout(LayoutKind.Explicit, Pack = 1)]
    public struct FileHeader
    {
        [FieldOffset(0)] public long Marker;

        [FieldOffset(8)] public LogHeader Active;

        [FieldOffset(44)] public LogHeader Backup;

        [FieldOffset(80)] public TreeRootHeader Root;

        [FieldOffset(142)] public TreeRootHeader FreeSpace;
    }

    [StructLayout(LayoutKind.Explicit, Pack = 1)]
    public struct TreeRootHeader
    {
        [FieldOffset(0)] public long RootPageNumber;
        [FieldOffset(8)] public long BranchPages;
        [FieldOffset(16)] public long LeafPages;
        [FieldOffset(32)] public long OverflowPages;
        [FieldOffset(40)] public long PageCount;
        [FieldOffset(48)] public long EntriesCount;
        [FieldOffset(56)] public int Depth;
        [FieldOffset(60)] public TreeFlags Flags;
    }

    [StructLayout(LayoutKind.Explicit, Pack = 1)]
    public struct LogHeader
    {
        [FieldOffset(0)] public long Marker;

        [FieldOffset(8)] public long LastLog;

        [FieldOffset(16)] public long LastLogPage;

        [FieldOffset(24)] public int ItemCount;

        [FieldOffset(28)] public long Options;
    }

And now we have the following code:

   1:  private static unsafe void Main()
   2:  {
   3:      IntPtr pagePtr = Marshal.AllocHGlobal(4096);
   4:   
   5:      var pageHeader = (PageHeader*) pagePtr.ToPointer();
   6:      pageHeader->ItemCount = 2;
   7:      pageHeader->Marker = 0x128314543423;
   8:      pageHeader->OverflowSize = 32;
   9:   
  10:      FileHeader* fileHeader = (FileHeader*) pageHeader + sizeof (PageHeader);
  11:   
  12:      fileHeader->Root.BranchPages = 0;
  13:   
  14:   
  15:      Marshal.FreeHGlobal(pagePtr);
  16:  }

The fun part about this code is that it would silently corrupt the state of the process.

Here is what happens when you run it under the debugger:

image

Can you figure out why?


Comments

Pawel Kmiec

What is TreeFlags? Looks like it's size is counted as 2 bytes, and those 2 bytes looks invalid to me (just a feeling)

Marc Gravell

Pointer math; adding in pointers means "this many instances of the type".

I'd use:

var fileHeader = (FileHeader*)((byte*)pagePtr.ToPointer() + sizeof(PageHeader));

(actually, I'd probably have the ptr as a byte* at the top, for convenience)

Marc Gravell

as a simple example to my "Pointer math" reply:

    int* p = stackalloc int[100];
    int* p2 = p + 1;

if, say, p is 0x05bdec30 - then p2 is 0x05bdec34; the "+ 1" is "+ the space used by one of the declared type, int - so + 4"

Andre

TreeRootHeader.OverflowPages should have FieldOffset(24)

Mark Heath

Looks like your pointer arithmetic is wrong. By adding sizeof(PageHeader) to pageHeader, you're not moving the point forward 20 bytes, but 20 times the size of the pageHeader structure (which you've cast to a FileHeader). so it's incrementing the pointer 20 * 204 bytes. Something like this is what you want: FileHeader* fileHeader = (FileHeader) ((byte)pageHeader + sizeof (PageHeader));

Mark Heath

sorry, code formatting got screwed up. Mark Gravell beat me to it anyway.

Comment preview

Comments have been closed on this topic.

FUTURE POSTS

  1. RavenDB 3.5 whirl wind tour: You want all the data, you can’t handle all the data - about one day from now
  2. The design of RavenDB 4.0: Making Lucene reliable - 3 days from now
  3. RavenDB 3.5 whirl wind tour: I’ll find who is taking my I/O bandwidth and they SHALL pay - 4 days from now
  4. The design of RavenDB 4.0: Physically segregating collections - 5 days from now
  5. RavenDB 3.5 Whirlwind tour: I need to be free to explore my data - 6 days from now

And 14 more posts are pending...

There are posts all the way to May 30, 2016

RECENT SERIES

  1. RavenDB 3.5 whirl wind tour (14):
    29 Apr 2016 - A large cluster goes into a bar and order N^2 drinks
  2. The design of RavenDB 4.0 (13):
    28 Apr 2016 - The implications of the blittable format
  3. Tasks for the new comer (2):
    15 Apr 2016 - Quartz.NET with RavenDB
  4. Code through the looking glass (5):
    18 Mar 2016 - And a linear search to rule them
  5. Find the bug (8):
    29 Feb 2016 - When you can't rely on your own identity
View all series

RECENT COMMENTS

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats