Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,128 | Comments: 45,550

filter by tags archive

Investigating an OutOfMemoryException

time to read 4 min | 617 words

Originally posted at 1/5/2011

I finally got a reliable reproduction for a repeated error, but I have to say, just based on the initial impression, something very strange is going on.


The stack trace was a bit more interesting:

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
  at System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)
  at System.Text.StringBuilder.GetNewString(String currentString, Int32 requiredLength)
  at System.Text.StringBuilder.Append(String value)
  at System.Text.RegularExpressions.RegexReplacement.ReplacementImpl(StringBuilder sb, Match match)
  at System.Text.RegularExpressions.RegexReplacement.Replace(Regex regex, String input, Int32 count, Int32 startat)
  at System.Text.RegularExpressions.Regex.Replace(String input, String replacement, Int32 count, Int32 startat)
  at System.Text.RegularExpressions.Regex.Replace(String input, String replacement)
  at System.Text.RegularExpressions.Regex.Replace(String input, String pattern, String replacement)
  at HibernatingRhinos.Profiler.BackEnd.SqlStatementProcessor.ReplaceParametersWithValues(String statement, Boolean useComment) 

I put a breakpoint in the appropriate place, and discovered that the error occurred in:

  • A SQL Statement that was 190 kilobytes in size
  • It had 4,060 parameters

Now, let us look at the actual code:

private string ReplaceParametersWithValues(string statement, bool useComment)
  if (sqlStatement.Parameters == null)
    return statement;

  foreach (var parameter in sqlStatement.Parameters
            .Where(x => x != null && x.Name.Empty() == false)
            .OrderByDescending(x => x.Name.Length))
    var patternNameSafeForRegex = Regex.Escape(parameter.Name);
    var pattern = patternNameSafeForRegex + @"(?![\d|_])";
    //static Regex methods will cache the regex, so we don't need to worry about that
      var replacement = useComment ? 
                parameter.Value + " /* " + parameter.Name + " */" : 
      statement = Regex.Replace(statement,
  return statement;

The problem is that:

  • There is no heavy memory pressure.
  • While the string is big, it is not that big.
  • In practice, there is a single replacement for each parameter.

Just for fun, I wasn’t able to reproduce the issue without running the full NH Prof application.

I solved the issue, but I am not entirely pleased with the way I solved it. (That is tomorrow’s post)

Any ideas how to reproduce this?

Any elegant ideas on how to solve this?



My guess is heap fragmentation due to forced usage of the LOH (big strings and maybe some internal regex objects). But that should still not be a problem (maybe the heap is already very fragmented because the NHProf app is running).


The StringBuilder needs a contiguous block of memory for each Append call it makes (if it needs to create a new block of memory - in other worse, when its buffer has been exceeded). It'll throw an OutOfMemoryException if it can't get that.

Perhaps that could be the issue? Is the error occurring on the first Replace call or a later one? if it's having to double its size (i believe that's the pattern it uses) multiple times, i could see it running out of contiguous blocks of memory in its address space... maybe...

Ayende Rahien


That is likely, except that the size isn't that big, 200 KB is small, and even running in 32 bits, the app only uses about 600 MB, so even if it needed to allocate a full 1MB, I find it hard to believe it couldn't.

Erik Juhlin

I don't think it would solve your problem, but looking at the code it seems to do a lot of Regex replaces. Using RegexOptions.Compiled could really improve performance.

Slower the first time, but then a lot faster. The static RegEx.Replace method will handle the caching of it...

Jesse Houwing

First of all, I think there is an error in your expression, at least I guess there is. This error is very unlikely to be the cause of the exception you're seeing but still...

[\d|] is the equivalent of either a number, or '|' or '', I expect you want: [\d] instead or (\d|).

Second, your Regex.Replace call will replace all instances from the beginning on each pass, which makes it pretty expensive I gather. Have you tried constructing one Regex that searches for all parameter names @(param1|param2|param3|... etc) and using a MatchEvaluator to look up the replacement values? That would allow you to replace all the parameters in one pass... Though it would make the actual expression a lot more expensive...

Both things won't explain the error you're seeing as I have no clue why exactly you're getting it :).


Add right before the foreach:

if (statement.Length > 100000)

return "WTF";


Comment preview

Comments have been closed on this topic.


  1. The worker pattern - about one day from now

There are posts all the way to May 30, 2016


  1. The design of RavenDB 4.0 (14):
    26 May 2016 - The client side
  2. RavenDB 3.5 whirl wind tour (14):
    25 May 2016 - Got anything to declare, ya smuggler?
  3. Tasks for the new comer (2):
    15 Apr 2016 - Quartz.NET with RavenDB
  4. Code through the looking glass (5):
    18 Mar 2016 - And a linear search to rule them
  5. Find the bug (8):
    29 Feb 2016 - When you can't rely on your own identity
View all series


Main feed Feed Stats
Comments feed   Comments Feed Stats