Ayende @ Rahien

It's a girl

Challenge: Your own ThreadLocal

I found a bug in Lucene.NET that resulted in a memory leak in RavenDB. Take a look here (I have simplified the code to some extent, but the same spirit remains):

public class CloseableThreadLocal
{
    [ThreadStatic] private static Dictionary<object, object> slots;

    public static Dictionary<object, object> Slots
    {
        get { return slots ?? (slots = new Dictionary<object, object>()); }
    }

    public /*protected internal*/ virtual Object InitialValue()
    {
        return null;
    }

    public virtual Object Get()
    {
        object val;

        if (Slots.TryGetValue(this, out val))
        {
            return val;
        }
        val = InitialValue();
        Set(val);
        return val;
    }

    public virtual void Set(object val)
    {
        Slots[this] = val;
    }

    public virtual void Close()
    {
        if (slots != null)// intentionally using the field here, to avoid creating the instance
            slots.Remove(this);
    }
}

As you can imagine, this is a fairly elegant way of doing this (please note, .NET 4.0 have the ThreadLocal class, which I strongly recommend using). But it has one very serious flaw. It you don’t close the instance, you are going to leak some memory. As you can imagine, that is a pretty bad thing to do.

In general, I consider such designs as pretty bad bugs when writing managed code. We have the GC for a reason, and writing code that forces the user to manually manage memory is BAD for you. Here is an example showing the problem:

class Program
{
    static void Main(string[] args)
    {
        UseThreadLocal();
        GC.Collect(2);
GC.WaitForPendingFinalizers();
Console.WriteLine(CloseableThreadLocal.slots.Count); } private static void UseThreadLocal() { var tl = new CloseableThreadLocal(); tl.Set("hello there"); Console.WriteLine(tl.Get()); } }

This will show that after the UseThreadLocal() run and we force full collection, the value is still there.

Without using the builtin ThreadLocal, can you figure out a way to solve this?

Points goes to whoever does this with the minimum amount of changes to the code.

Comments

tobi
12/17/2010 10:40 AM by
tobi

Implement IDisposable for the common case and Finalize() for the uncommon/bug case. The finalizer could throw if a debugger was attached to assist in finding leaks.

Steve Py
12/17/2010 10:42 AM by
Steve Py

Well, my guess based on the behaviour you describe is to make the class disposable. On dispose, call Close. Anyone implementing the class should be checking for IDisposable and be "using" appropriately.

Other than that it may be a candidate scenario for calling Close() within a finalizer, though that test scenario won't pass with the finalizer implementation likely due to compiler optimization, but I believe it would prevent the memory leak.

I'd opt for best practice though.

Rob Ashton
12/17/2010 10:48 AM by
Rob Ashton

Thinking about this now,

IDisposable doesn't solve anything, it just gives the class a pattern that people recognise so they're more likely to remember to call Close.

Using a finalizer is a great idea, except you've got a ThreadStatic there, and you don't know what thread the finalizer will be called in.

(So calling Close from a finalizer will not work)

Okay, so I've given my opinion on why I think those won't work, I'm still thinking about the best solution =)

Rafal
12/17/2010 10:55 AM by
Rafal

But how does the memory leak occur? [ThreadLocal] objects should be GCd after the thread terminates, so the only possibility is that the thread doesn't terminate and is reused for other purposes, like in a thread pool.

tobi
12/17/2010 11:10 AM by
tobi

Ok, finalization happens on a different thread... thats nasty! Looks like i am still not done with learning.

You could first convert the dictionary into a hashset and store the value in the instance. Then you change the hashset to a weakhashset (not built-in). then you introduce a threadstatic int counter which you use to purge the dead items on every 1000th operation. what a hack...

depending on the perf requirements i would just use a localdatastoreslot which is built-in on 2.0 upwards.

Thomas Eyde
12/17/2010 11:11 AM by
Thomas Eyde

Using WeakReference is the first thing that pops into my mind

Steve Py
12/17/2010 11:13 AM by
Steve Py

Rob's right, Cannot rely on a finalizer. Frankly I'd choose the big stick approach. Implement IDisposable, and a finalizer that throws if Dispose wasn't called.

private bool _isDisposed = false;

void IDisposable.Dispose()

{

Dispose(true);

}

protected virtual void Dispose(bool disposing)

{

if (!_isDisposed)

{

    if (disposing)

        Close();

       _isDisposed = true;

}

}

~CloseableThreadLocal()

{

if (!_isDisposed)

{

if DEBUG

    Debugger.Break();

endif

    throw new InvalidOperationException("Disposable object was not disposed. (CloseableThreadLocal)");

}

}

It is a memory leak because on a long-lived thread repeated use of new ThreadLocal instances will continue to pile up items in the static dictionary. If Close isn't called those items are never removed.

tobi
12/17/2010 11:15 AM by
tobi

And I just peeked under the covers on .net 4. the implementation is really creative and quite funny ;-) I am very keen to find out about how to do this with "minimal changes".

Here is a potential way: Let all access to the dictionary pass through a threadlocal spinlock. When creating the CloseableThreadLocal in the ctor, copy the dict and the lock to instance variables so that they are available on the finalizer. On the finalizer call close. I understand locks are very cheap if they are nearly always uncontended and thread-local (which would be the case here).

Rob Ashton
12/17/2010 12:12 PM by
Rob Ashton

The best solution I can come up with WeakReference (I'm at work so I'm having to do this on idle ;-)

    public class CloseableThreadLocal

    {

        [ThreadStatic]

        private static Dictionary

<object,> slots;

        private WeakReference self;


        public CloseableThreadLocal() { self = new WeakReference(this); }


        public static Dictionary

<object,> Slots

        {

            get { return slots ?? (slots = new Dictionary

<object,> ()); }

        }


        public /*protected internal*/ virtual Object InitialValue()

        {

            return null;

        }


        public virtual Object Get()

        {

            object val;


            if (Slots.TryGetValue(self, out val))

            {

                return val;

            }

            val = InitialValue();

            Set(val);

            return val;

        }


        public virtual void Set(object val)

        {

            Slots[self] = val;

        }


        public virtual void Close()

        {

            if (slots != null)// intentionally using the field here, to avoid creating the instance

                slots.Remove(self);

        }

    }

That gives us the ability to add another method to clear anything in the dictionary in the current thread when an object is created on that thread (It's a bit sweep and cleany). (We can check the dictionary for any weak references that haven't got a valid reference anymore)

Seems a bit overkill though given we're going for as few changes as possible, I suspect there is something I'm missing here.

Ryan Heath
12/17/2010 12:14 PM by
Ryan Heath

There are two problems to overcome:

  • the class is inserting itself into the dictionary so it can never be collected

  • when above problem is fixed, the finalizer is running on a different thread, so we must keep a ref to the slots created in main thread.

public class CloseableThreadLocal

{

[ThreadStatic] private static Dictionary <object,> slots;

object key = new Object();

Dictionary <object,> localSlots = null;

public static Dictionary <object,> Slots

{

get { return slots ?? (slots = new Dictionary

<object,> ()); }

}

Dictionary <object,> LocalSlots

{

get { return slots ?? ( localSlots = Slots); }

}

public /protected internal/ virtual Object InitialValue()

{

return null;

}

public virtual Object Get()

{

object val;


if (LocalSlots.TryGetValue(key, out val))

{

  return val;

}

val = InitialValue();

Set(val);

return val;

}

public virtual void Set(object val)

{

LocalSlots[key] = val;

}

public virtual void Close()

{

if (localSlots != null)// intentionally using the field here, to avoid creating the instance

  localSlots.Remove(key);

}

~CloseableThreadLocal()

{

Close();

}

}

// Ryan

Daniel Grunwald
12/17/2010 12:14 PM by
Daniel Grunwald

Here are some additional problems to consider:

  • The Dictionary is keeping the key object alive, so a finalizer will never be called (unless the thread exits).

  • The value object may have a reference to the CloseableThreadLocal

(key object). Such a cyclic reference shouldn't prevent us from garbage collecting the object.

The first problem alone could be worked around by using a Dictionary <weakreference,> and adding a finalizer. But I don't see any way to work around the second problem using .NET 3.5. There doesn't seem to be any way to distinguish between outstanding references from GC roots and outstanding references from our cycles.

In .NET 4.0, it's easy: just replace the Dictionary <object,object> with a ConditionalWeakTable <object,object> .

Then again, the .NET 4.0 ThreadLocal class also doesn't seem to solve the problem with cyclic references; so I guess a Dictionary <weakreference,> is good enough for most usecases.

But I like how the .NET 4.0 implementation manages to directly use the fast [ThreadStatic] support without having to go through slow hash tables.

Daniel Grunwald
12/17/2010 12:29 PM by
Daniel Grunwald

This comment box seems to be eating generics...

I meant using a Dictionary[of WeakReference, object] and finalizer.

Rob and Ryan posted in the meantime, but I think both their approaches are incorrect:

Rob is missing the finalizer; and Ryan is clearing only one dictionary - but I would expect that dropping the reference to the CloseableThreadLocal gets rid of the values on ALL threads (I would expect the same of the Close() method, so this issue is already present in Ayende's code).

But I like how Ryan avoided the WeakReference by simply using another object as key.

So instead of the "Dictionary localSlots" (race condition!), I would use a "List[of Dictionary] allDicts" to store the dictionaries of all threads on which this CloseableThreadLocal was used. Except that opens up the possibility of referencing the dictionaries of threads that exited, so you might have to use something like a list of weak references to the dictionaries.

But I don't see how anything except ConditionalWeakTable can stop this simple class from leaking:

class Leak {

CloseableThreadLocal local;

public Leak() {

   local = new CloseableThreadLocal();

   local.Set(this);

}

}

Patrick Huizinga
12/17/2010 01:15 PM by
Patrick Huizinga

Daniel, I'm pretty sure WeakReference can deal with cyclic references, since the GC can handle them too.

The GC will determine the only way to reach your Leak instance is through a WeakReference, which it should disregard, and therefore clean up your instance.

Daniel Grunwald
12/17/2010 01:57 PM by
Daniel Grunwald

Patrick Huizinga: But there's the following strong reference chain:

ThreadStatic -> slots -> Values -> Leak -> CloseableThreadLocal

Therefore, the finalizer of CloseableThreadLocal doesn't get invoked, and the entry is never removed from the slots.

Yes, WeakReference on its own can handle cycles; it's the static dictionary that's introducing the trouble here. And you can't solve the problem by introducing more WeakReferences. What you need is a CONDITIONAL weak reference: it's a strong reference as long as the key is alive; but weak otherwise.

In GC literature, this kind of reference is called "Ephemerons" ( http://en.wikipedia.org/wiki/Ephemeron).

Ephemeron support is new in the .NET 4.0 garbage collector, and ConditionalWeakTable is the public API for it.

jonnii
12/17/2010 03:46 PM by
jonnii

This is my answer.

public class CloseableThreadLocal

{

    [ThreadStatic]

    private static Dictionary

<object,> slots;

    private readonly object key = new object();

    private Dictionary

<object,> localref;

    public static Dictionary

<object,> Slots

    {

        get

        {

            return slots ?? (slots = new Dictionary

<object,> ());

        }

    }


    public /*protected internal*/ virtual Object InitialValue()

    {

        return null;

    }


    public Dictionary

<object,> LocalSlots

    {

        get { return localref = Slots; }

    }


    public virtual Object Get()

    {

        object val;


        if (LocalSlots.TryGetValue(key, out val))

        {

            return val;

        }

        val = InitialValue();

        Set(val);

        return val;

    }


    public virtual void Set(object val)

    {

        LocalSlots[key] = val;

    }


    public virtual void Close()

    {

        if (localref != null)

        {

            // intentionally using the field here, to avoid creating the instance

            localref.Remove(key);

        }

    }


    ~CloseableThreadLocal()

    {

        Close();

    }

}

Use an object as the key, so that you don't end up with a reference to yourself and also keep a local reference to the dictionary so that you get a chance to remove yourself.

Patrick Huizinga
12/17/2010 04:08 PM by
Patrick Huizinga

jonnii,

This will break your code:

var ctl = new CloseableThreadLocal()

new Thread(() => ctl.Set(new BigObject())).Start();

new Thread(() => ctl.Set(new BigObject())).Start();

It will not GC one of the BigObjects. Also ctl will never be GC'ed.

Patrick Huizinga
12/17/2010 04:24 PM by
Patrick Huizinga

hmm, made mistake in criticizing jonnii; ctl does get GC'ed.

Anyway, after shooting down jonnii, it's only fair he gets a shot at my attempt :-)

class ClosableThreadLocal

{

[ThreadStatic] static Dictionary slots;

readonly List usedSlots = new List();

public Dictionary Slots

{

get

{

if (slots != null) return slots;

slots = new Dictionary();

lock (usedSlots) usedSlots.Add(slots);

return slots;

}

}

public object Key { get { return usedSlots; } }

public void Set(object value)

{

Slots[Key] = value;

}

public void Close()

{

if (slots == null) return;

slots.Remove(Key);

lock (usedSlots) usedSlots.Remove(slot);

}

void Dispose()

{

foreach(Dictionary slot in usedSlots) slot.Remove(Key);

}

}

As you can see it's mostly the same code as yours (jonnii), except that I keep a list with all used dictionaries, so I can actually enumerate them all and clean them all out.

Ryan Heath
12/17/2010 06:16 PM by
Ryan Heath

I agree that a list of dictionaries is saver, but it is questionable that one is using an instance of this class in multiple threads at once.

Smells like a bug...

// Ryan

Justin Chase
12/17/2010 10:13 PM by
Justin Chase

Don't use ThreadStatic. Create an instance for whatever scope you want this functionality and dispose of it properly when you leave that scope.

Thomas Eyde
12/18/2010 03:13 PM by
Thomas Eyde

I finally got it. And no need for weak references.

Key points, as previously mentioned in this thread:

  1. We need a thread global reference to track each thread local slot so the finalizer can reach it.

  2. We need to remove the strong reference introduced by the dictionary key. I went for a local Guid.

  3. The finalizer must call Close

  4. Handle new potential memory leaks introduced by code changes.

Code and tests here: http://pastebin.com/x26qnpLn

tobi
12/18/2010 07:03 PM by
tobi

Thomas, you need to add locking around the global HashSet. Not sure how performant this is compared to just using LocalDataStoreSlot because the locks will give up many performance gains.

Bernhard Hofmann
12/20/2010 08:05 AM by
Bernhard Hofmann

Why use [ThreadStatic] when you're not using a new thread for each CloseableThreadLocal?

Comments have been closed on this topic.