Ayende @ Rahien

Unnatural acts on source code

How do you track that?

I have an interesting problem with SvnBridge.

After around 5000 full revision request (a set of requests that can occur), the application get hung making a web service call to TFS. This comes after making quite a few calls to TFS, and is generally fairly easily reproducible. The actual call being made is not an expensive one (nor is it the same call). TFS is responsive during that time, so it is not its fault.

It looks very much like I am hitting the 2 concurrent HTTP requests, except that all requests are serialized, and there is no multi threaded work involved.

I have been unable to reproduce this under a profiler or debugger...

Thoughts?

Comments

Denis
05/05/2008 01:05 PM by
Denis

Prior to this state, do any TCP connections fail and then succeed when retried? If so, maybe you're hitting the half-open limit (assuming a client OS). Event logs with 4226?

James Curran
05/05/2008 01:53 PM by
James Curran

In my experience helping people diagnose problems through internet message (mostly in C++), when a problem is reproducible outside of a debugger, but not within the debugger, in 100% of the cases, the problem was caused by overrunning a local array.

Ayende Rahien
05/05/2008 02:37 PM by
Ayende Rahien

Denis,

I am testing that on Server 2008, so I don't think so.

Ayende Rahien
05/05/2008 02:40 PM by
Ayende Rahien

James,

Yes, I remember that.

No C++ anywhere...

Yossi
05/05/2008 07:38 PM by
Yossi

Try take a dump when the process is stuck, maybe it would shed some light on what is going on then..

anonymous
05/05/2008 07:40 PM by
anonymous

Are you using TFS OM? If yes, take a look at the connection pools on the proxies. It might have to do something with it.

Alex Simkin
05/05/2008 10:02 PM by
Alex Simkin

@Yossi

"Try take a dump when the process is stuck..."

That's gross, man

Dan
05/06/2008 03:08 AM by
Dan

@Yossi: Always say "Capture a dump". Never "take a dump" :)

Capturing a hang dump with ADPlus when you are experiencing your issue will allow you to determine exactly whats going on in terms of executing threads, objects on the heap, etc. You would analyze the dump using WinDbg (I'm making the assumption that you're not familiar with these tools.)

Ayende Rahien
05/06/2008 08:37 AM by
Ayende Rahien

Dan,

I can actually repro this and break into the debugger.

Comments have been closed on this topic.