Ayende @ Rahien

It's a girl

More information of GC issue

After a lot more study, it looks like there are two separate issues that are causing the problem here.

  1. During AppDomain unload, it is permissible for the GC to collect reachable objects. I am fine with that and I certainly agree that this makes sense.
  2. Application_End occurs concurrently with the AppDomain unload.

Looking at the docs (and there are surprisingly few about this), it seems like 1 is expected, but 2 is a bug. The docs state:

Application_End  - Called once per lifetime of the application before the application is unloaded.

It may be my English, but to me before doesn’t suggest at the same time as.

So I do believe it is a bug in the framework, but not in the GC, it is in the shutdown code for ASP.Net. This is still a very nasty problem.

Comments

Stephen
01/22/2009 04:10 PM by
Stephen

Well, the application_end sub is called when the HttpApplication is unloaded, ASP.NET pools HttpApplication instances and uses them to forefill requests.. when the pool is recycled (which happens after a timeout) all the HttpApplications will end.. but also, .NET can create additional HttpApplication's if non are available in the pool, once they try to return to the pool - if its 'full' they instead get 'disposed' of..

This has always been my understanding, theres quite a lot of 'funky' threading that can happen, like when an application isn't read to forefill the request, its first created and initialized on the request thread before being transfered to the processing threads.. I guess all these things can attribute to a somewhat confusing api.

Bruno Martínez
01/22/2009 07:00 PM by
Bruno Martínez

Finalizers run in a dedicated thread (several threads in the future), so the concurrent execution may be expected.

Ayende Rahien
01/22/2009 07:04 PM by
Ayende Rahien

Bruno,

sigh

That is not the problem.

The problem is concurrent execution of the finalizer and the application_end, which cleans up resources.

Torbjörn Gyllebring
01/22/2009 09:00 PM by
Torbjörn Gyllebring

I've struggled with a diffrent but also quite odd problem relating to how Application_Event's are called. It's quite possible to get an Authenticate event before Start.

Really strange thins seems to be abound when it comes to startup/shutdown is handled.

Michael Morton
01/22/2009 09:23 PM by
Michael Morton

@Ayende: The only guarantee that the statement regarding Application_End makes is that it will be called once. It makes no promises on how it will be called or what else will be going on during that call.

Ayende Rahien
01/22/2009 09:54 PM by
Ayende Rahien

Michael,

This is supposed to be where I am cleaning my resources for the application.

It is not supposed to be a race condition to see who wins, my disposal code or the finalizers

Alois Kraus
01/22/2009 11:02 PM by
Alois Kraus

Hi Ayende,

your call stack of HttApplication does not indicate a concurrent AppDomain unload and application end.

private void ReleaseResourcesAndUnloadAppDomain(object state)

{

...

 try

{

    this.Dispose();

}

catch

{

}

Thread.Sleep(250);

AddAppDomainTraceMessage("before Unload");

AppDomain.Unload(Thread.GetDomain());

...

}

That behaviour does look fine to me. The problem seems to come from the fact that after/during dispose of your objects some object graphs become unrooted and thus eligible for garbage collection.

What do you expect from this program:

class SomeData

{

    public SomeData()

    {

        Console.WriteLine("SomeData created");

    }


    ~SomeData()

    {

        Console.WriteLine("SomeData finalized");

    }

}


class Program

{

    static void DoSomethingWithData()

    {

        var list = new List

<somedata()

        {

            new SomeData(),

            new SomeData()

        };

    }


    static void Main(string[] args)

    {

        DoSomethingWithData();

        GC.Collect();

        GC.WaitForPendingFinalizers();

        Console.ReadLine();

    }

}

Yes the list does reference the SomeData objects but the output is:

SomeData created

SomeData created

SomeData finalized

SomeData finalized

I think your problem is related to the disposal of your root application object which allows GC to shoot down the rest of your objects. You could try to hide the relevant data in a class with statics to force your objects to be rooted.

public class KeepAliveData

{

public static AppObject GlobalAppObject;

}

That should help to force the GC to keep your application object alive. As a side note you should not do too much in your finalizers. The release of unmanaged resources (interop handles) is the only thing that is more or less safe all other things like taking locks will introduce its own set of problems.

Yours,

Alois Kraus

Michael Morton
01/22/2009 11:50 PM by
Michael Morton

I did some digging around with reflector...

Make note of that in HttpRuntime.Init():

this._appDomainUnloadallback = new WaitCallback(this.ReleaseResourcesAndUnloadAppDomain);

Then the shutdown code path:

HttpRuntime.ShutdownAppDomain()

... calls ...

ThreadPool.QueueUserWorkItem(theRuntime.appDomainUnloadallback); <-- new thread here

... calls ...

HttpRuntime.Dispose()

... which eventually fires Application_End.

This call path matches your stack trace and does explain why it can happen concurrently.

Stephen
01/22/2009 11:56 PM by
Stephen

Well, you can store things in the HttpApplicationState, this basically represents the 'top level' single instance of the web app.. although its actually initialized by the first HttpApplication..

As I was saying before, there isn't a single HttpApplication instance in a 'web application', because HttpApplication is what 'runs' requests.. since a server can be running multiple requests at a time, there can be multiple of these running.. this is where the 'Application' property of the HttpApplication (Global / global.asax) comes in - because it returns a single HttpAppliataionState that all HttpApplication's share.. of course you'll need to do locking around this object (but it provides locking for you).

Ayende Rahien
01/23/2009 12:32 AM by
Ayende Rahien

Alois,

The problem happens specifically with ASP.Net, not with console app.

Ayende Rahien
01/23/2009 12:34 AM by
Ayende Rahien

Stephen,

See Michael's post about what is actually happening.

Alois Kraus
01/23/2009 07:02 AM by
Alois Kraus

Hi Ayende,

yes I am aware of that. The reflector code shows the HttRuntime. From your call stack and the HttpRuntime code snipped I can deduce that no AppDomain Unload has happened yet. At least not in this thread. So the question remains in which thread did you find an AppDomain Unload? If none then I suspect that it is a simple Dispose issue

Dispose()

{

myObject.Dispose();

myObject = null;  // GC can kick in and call finalizers on all objects which are contained by myObject.

}

Yours,

Alois Kraus

Ayende Rahien
01/23/2009 07:03 AM by
Ayende Rahien

Alois,

The fact that a finalizer was called while a strong reference is being held to the object was a good indication, I believe.

Michael Morton
01/23/2009 07:38 AM by
Michael Morton

@Alois:

HttpRuntime.ShutdownAppDomain() is the result of a long code path starting in System.Web.HostingEnvironment.InitiateShutdown(). When WebDev server exits it calls Microsoft.VisualStudio.WebHost.Server.Stop() which in turn calls Microsoft.VisualStudio.WebHost.Host.Shutdown(). WebHost.Host.Shutdown() does exactly one thing; call HostingEnvirionment.InitiateShutdown().

I would expect other hosts, including IIS, to behave similarily with regards to the HostingEnvironment.

Another interesting piece of information is that not one, but two threads are spawned to handle the shutdown. HostingEnvironment.InitiateShutdownInternal() queues a work item to the thread pool which in turn calls the rest of it.

Comments have been closed on this topic.