Ayende @ Rahien

It's a girl

The mysterious life of mutexes

Last night I was talking with Ron Grabowski (of log4net fame) and the subject of mutexes came up. Specifically, the subject of abandoned mutexes across threads, AppDomains, and Processes. I was pretty sure what the behavior should be, but we wrote a set of test cases for that, with... surprising results.

My expectation that after I get a hold on a mutex, I have to explicitly let it go. If I am killed or interrupted somehow, then I would expect the next person to acquire the mutex to get an AbandonedMutexException.

The first code sample does exactly that:

new Thread(delegate()
{
    Mutex m = new Mutex(true, "foo");
    m.WaitOne();
    Console.WriteLine("Got mutex");
    Thread.Sleep(3000);
    Console.WriteLine("Bye..");
}).Start();
 
Thread.Sleep(500);// give the thread some time to get the mutex
Mutex m2 = new Mutex(true, "foo");
m2.WaitOne();

We haven't explicitly release the mutex, but we let the thread die, on acquire, you'll get the expected AME.

Now, let us try it with processes, shall we?

if (args.Length != 0)
{
    Mutex m = new Mutex(true, "foo");
    m.WaitOne();
    Console.WriteLine("Got mutex");
    Thread.Sleep(1000);
    Console.WriteLine("Bye, fast");
    Environment.FailFast("blah");
}
else
{
    Process p = Process.Start(typeof(Program).Assembly.Location, "a");
    Thread.Sleep(2000);
    Mutex m = new Mutex(true, "foo");
    m.WaitOne();
    Console.WriteLine("got it second");
}

We start our process, which spawn a second process, which would acquire a lock on the mutex, and then fail fast, killing the process. The first process, on attempting to get the process, will die horribly, as expected.

But not so fast, because this bit of code doesn't throw anything, and pretend that everything is right.

if (args.Length != 0)
{
    Mutex m = new Mutex(true, "foo");
    m.WaitOne();
    Console.WriteLine("Got mutex");
    Thread.Sleep(1000000);
}
else
{
    Process p = Process.Start(typeof(Program).Assembly.Location, "a");
    Thread.Sleep(2000);
    p.Kill();
    Mutex m = new Mutex(true, "foo");
    m.WaitOne();
    Console.WriteLine("got it second");
}

I am not really sure why this is the case, I would expect Envirionment.FailFast and an external Process.Kill to have the same behavior.

It gets to be even stranger, because we see the exact same problem when we are dealing with AppDomains:

public class Foo : MarshalByRefObject
{
    public void Repeat()
    {
        bool ignored;
        Mutex m = new Mutex(true, "blah", out ignored);
        m.WaitOne();
        while (1 != DateTime.Now.Ticks)
        {
            Console.Write(".");
            Thread.Sleep(1000);
        }
    }
}
 
AppDomain app = AppDomain.CreateDomain("fo");
new Thread(delegate()
{
    Foo f = (Foo)app.CreateInstanceAndUnwrap(
        typeof(Foo).Assembly.FullName,
        typeof(Foo).FullName);
    f.Repeat();
}).Start();
Thread.Sleep(1000);
bool ignored;
Mutex m3 = new Mutex(true, "blah", out ignored);
Console.Write("aquiring");
m3.WaitOne(TimeSpan.FromSeconds(2), false);
AppDomain.Unload(app);
Console.Write("done");

I am not willing to comment on what is going on, I have the feeling that I am missing something.

Nevertheless, this is surprising behavior.

Comments

Jon Skeet
02/28/2008 04:22 PM by
Jon Skeet

I can't say I've looked closely enough for sure, but is it possible that you're running into the issue where the Mutex finalizer releases the mutex for you? That trips up a lot of people, very often. I'm too sleepy to check for sure though :)

Ayende Rahien
02/28/2008 04:24 PM by
Ayende Rahien

Haven't check, so that might be.

I don't think that this is the right behavior, though

Jon Skeet
02/28/2008 04:29 PM by
Jon Skeet

I should have said the potential workarounds to avoid the finalizer:

1) Use a "using" statement with the mutex, so the lifetime is clear

2) Use GC.KeepAlive

3) Use a static variable

Personally I'm a fan of approach 1, not that I use Mutex much at all in practice.

I don't know what effect that would have in your scenarios though - I've no idea whether finally blocks get executed when a process is being killed.

Ron Grabowski
02/28/2008 06:35 PM by
Ron Grabowski

The "Threading in C#" book (http://www.albahari.com/threading/part2.html) says:

"

A good feature of Mutex is that if the application terminates without ReleaseMutex first being called, the CLR will release the Mutex automatically.

"

In case anyone is curious, I'm trying to coordinate resource cleanup between an AppDomain that is being terminated and a new AppDomain that is being bought in to replace it. Specifically when ASP.Net detects that the web.config has changed and brings up a new AppDomain to replace the old AppDomain...if the old AppDomain is taking too long to services the old requests and IIS terminates the process while the old AppDomain has the Mutext will the Mutext be properly released so the new AppDomain can aquire it?

Ayende Rahien
02/28/2008 06:38 PM by
Ayende Rahien

No problem with that.

The issue that I have is that it doesn't throw abandon mutex exception

Jan Van Ryswyck
02/28/2008 08:31 PM by
Jan Van Ryswyck

@ Jon Skeet:

The GC.KeepAlive has some weird side effects when using in Console apps (instead of Windows Forms apps). I used it for a Console app for keeping the Mutex around. I got some weird exceptions from time to time when the app was closing down. It needed some WinDbg debugging to figure it out that the Mutex was causing this behavior (only on production machines). A Using statement around the Mutex solved the issue.

Jon Skeet
02/29/2008 09:17 AM by
Jon Skeet

@Jan: Wow. Weird. What kind of exceptions were you getting? Nice to hear about that though - I'll attempt to remember it for the next time this comes up on the newsgroups!

Eliad
03/04/2008 10:40 AM by
Eliad

I got the same behaviour for Environment.FailFast and Process.Kill, i.e. AbandonedMutexException is thrown...

I did a little refactoring to the given samples, and guess what , only Environment.Exit executes cleanly

using System;

using System.Collections.Generic;

using System.Text;

using System.Diagnostics;

using System.Threading;

using System.Security;

using System.Security.Permissions;

namespace ConsoleApplication5

{

class Program

{

    static void Main(string[] args)

    {

        if (args.Length == 0) return;


        switch (args[0].ToLower())

        {

            case "kill":

                Kill(args);

                break;

            case "exit":

                Exit(args);

                break;

            case "failfast":

                FailFast(args);

                break;

            default:

                break;

        }

    }


    static void Kill(string[] args)

    {

        if (args.Length != 1)

        {

            Mutex m = new Mutex(true, "foo");

            m.WaitOne();

            Console.WriteLine("Got mutex");

            Thread.Sleep(1000000);

        }

        else

        {

            Process p = Process.Start(typeof(Program).Assembly.Location, "kill b");

            Thread.Sleep(2000);

            p.Kill();

            Mutex m = new Mutex(true, "foo");

            m.WaitOne();

            Console.WriteLine("got it second");

        }

    }


    static void Exit(string[] args)

    {

        if (args.Length != 1)

        {

            Mutex m = new Mutex(true, "foo");

            m.WaitOne();

            Console.WriteLine("Got mutex");

            Thread.Sleep(1000);

            Console.WriteLine("Bye, fast");

            Environment.Exit(45);

        }

        else

        {

            Process p = Process.Start(typeof(Program).Assembly.Location, "exit a");

            Thread.Sleep(2000);

            Mutex m = new Mutex(true, "foo");

            m.WaitOne();

            Console.WriteLine("got it second");

        }

    }

    static void FailFast(string[] args)

    {

        if (args.Length != 1)

        {

            Mutex m = new Mutex(true, "foo");

            m.WaitOne();

            Console.WriteLine("Got mutex");

            Thread.Sleep(1000);

            Console.WriteLine("Bye, fast");

            Environment.FailFast("blah");

        }

        else

        {

            Process p = Process.Start(typeof(Program).Assembly.Location, "failfast a");

            Thread.Sleep(2000);

            Mutex m = new Mutex(true, "foo");

            m.WaitOne();

            Console.WriteLine("got it second");

        }

    }


}

}

Comments have been closed on this topic.