WCF works in mysterious ways

Here is the result of about two hours of trying to figure out what WCF is doing:

class Program
    {
        static private readonly Binding binding = new NetTcpBinding
        {
            OpenTimeout = TimeSpan.FromMilliseconds(500),
            CloseTimeout = TimeSpan.FromMilliseconds(250),
            ReaderQuotas =
                {
                    MaxArrayLength = Int32.MaxValue,
                    MaxBytesPerRead = Int32.MaxValue,
                    MaxNameTableCharCount = Int32.MaxValue,
                    MaxDepth = Int32.MaxValue,
                    MaxStringContentLength = Int32.MaxValue,
                },
            MaxReceivedMessageSize = Int32.MaxValue,
        };
        static void Main()
        {
            try
            {
                var uri = new Uri("net.tcp://" + Environment.MachineName + ":2200/master");
                var serviceHost = new ServiceHost(new DistributedHashTableMaster(new NodeEndpoint
                {
                    Async = uri.ToString(),
                    Sync = uri.ToString()
                }));
                serviceHost.AddServiceEndpoint(typeof(IDistributedHashTableMaster),
                                               binding,
                                               uri);

                serviceHost.Open();

                var channel =
                    new ChannelFactory<IDistributedHashTableMaster>(binding, new EndpointAddress(uri))
                        .CreateChannel();
                channel.Join();
            }
            catch (Exception e)
            {
                Console.WriteLine(e);
            }

        }
    }

    [ServiceBehavior(
        InstanceContextMode = InstanceContextMode.Single,
        ConcurrencyMode = ConcurrencyMode.Single,
        MaxItemsInObjectGraph = Int32.MaxValue
        )]
    public class DistributedHashTableMaster : IDistributedHashTableMaster
    {
        private readonly Segment[] segments;

        public DistributedHashTableMaster(NodeEndpoint endpoint)
        {
            segments = Enumerable.Range(0, 8192).Select(i =>
                                                        new Segment
                                                        {
                                                            AssignedEndpoint = endpoint,
                                                            Index = i
                                                        }).ToArray();
        }

        public Segment[] Join()
        {
            return segments;
        }
    }

    [ServiceContract]
    public interface IDistributedHashTableMaster
    {
        [OperationContract]
        Segment[] Join();
    }

    public class NodeEndpoint
    {
        public string Sync { get; set; }
        public string Async { get; set; }
    }

    public class Segment
    {
        public Guid Version { get; set; }

        public int Index { get; set; }
        public NodeEndpoint AssignedEndpoint { get; set; }
        public NodeEndpoint InProcessOfMovingToEndpoint { get; set; }

        public int WcfHatesMeAndMakeMeSad { get; set; }
    }

The problem? On my machine, executing this results in:

Maximum number of items that can be serialized or deserialized in an object graph is '65536'. Change the object graph or increase the MaxItemsInObjectGraph quota.

The freaky part? Do you see the WcfHatesMeAndMakeMeSad property? If I comment that one out, the problem goes away. Since MaxItemsInObjectGraph is set to int.MaxValue, I don’t know what else to do, and frankly, I am getting mighty tired of WCF doing stuff in unpredictable ways.

Protocol Buffers & TcpClient, here I comes.

Print | posted on Thursday, June 04, 2009 8:18 PM

Feedback


Gravatar

# Why is that freaky? 6/4/2009 8:28 PM Paul Betts

That's not freaky at all, you made the Segment object smaller, so there was less objects to serialize, so you manage to get under the 64K limit. Depending on how you count it, there are about 8 objects in total serialized per Segment * 8192 = you're right at the line.

Now I'm not saying WCF is doing the *right* thing, or that an arbitrary unsettable 64K limit is a *good* idea, but it's at least not weird.


Gravatar

# re: WCF works in mysterious ways 6/4/2009 8:28 PM Ayende Rahien

Paul,
Did you notice that I set just about every limit to int.MaxValue?


Gravatar

# re: WCF works in mysterious ways 6/4/2009 8:29 PM Arnon Rotem-Gal-Oz

Add [DataContract] attribute to Segment and [DataMember] attributes to its properties (and you might need a KnownType (for segment) attribute on the IDistributedHashTableMaster

Arnon


Gravatar

# re: WCF works in mysterious ways 6/4/2009 8:31 PM Krzysztof Kozmic

why on earth would he need known type for that?


Gravatar

# re: WCF works in mysterious ways 6/4/2009 8:33 PM Ayende Rahien

Arnon,
Nope, I tried that, it doesn't work.


Gravatar

# re: WCF works in mysterious ways 6/4/2009 8:34 PM Arnon Rotem-Gal-Oz

you probably don't


Gravatar

# re: WCF works in mysterious ways 6/4/2009 8:38 PM Ayende Rahien

With or without it, it doesn't work


Gravatar

# re: WCF works in mysterious ways 6/4/2009 8:39 PM Arnon Rotem-Gal-Oz

Sorry that should be ServiceKnownType not KnownType
NodeEndPoint should also have the [datacontract] and [DataMember] attributes


Gravatar

# re: WCF works in mysterious ways 6/4/2009 8:41 PM Ayende Rahien

Not working either :-(
And NodeEndpoint have those


Gravatar

# re: WCF works in mysterious ways 6/4/2009 8:49 PM Jesse Ezell

The service behavior attribute only applies to the server side. You need to set it on the ClientSide in the endpointbehaviors config or programmatically or you will hit the limit trying to serialize your object graph.


Gravatar

# re: WCF works in mysterious ways 6/4/2009 8:51 PM Krzysztof Kozmic

Try also creating DataContractSerializer and serializing the object graph yourself. If it succeeds then it's a misleading message and the problem lies somewhere else,


Gravatar

# re: WCF works in mysterious ways 6/4/2009 8:55 PM Stephen

I'm pretty sure the data contract serializer has a limit on it as well, you'll need to specify that (probably as a service behavior).


Gravatar

# re: WCF works in mysterious ways 6/4/2009 9:17 PM Arnon Rotem-Gal-Oz

Ok, Now I actually ran your code - so I see the problem :)
You also need to set the MaxItemsOnInfoGraph on the client side
so before you CreateChannel on the factory you need to do something like
var channelFactory =
new ChannelFactory(binding, new EndpointAddress(uri));

foreach (var operationDescription in channelFactory.Endpoint.Contract.Operations)
{

var dataContractBehavior =

operationDescription.Behaviors[typeof(DataContractSerializerOperationBehavior)]

as DataContractSerializerOperationBehavior;

if (dataContractBehavior != null)
{

dataContractBehavior.MaxItemsInObjectGraph = int.MaxValue;

}

}
var channel=channelFactory.CreateChannel();
channel.Join();


Gravatar

# re: WCF works in mysterious ways 6/4/2009 9:28 PM Rafal

Maybe you should set
WCF.IAmSorryForNotHavingAPhD = true
then it will just work


Gravatar

# re: WCF works in mysterious ways 6/4/2009 9:55 PM Huseyin Tufekcilerli

Alternatively, you can attribute your service contract operations with the following NetDataContractFormat attribute and force them to use NetDataContractSerializer instead of DataContractSerializer:

www.pluralsight.com/.../22284.aspx


Gravatar

# re: WCF works in mysterious ways 6/5/2009 12:23 PM Krzysztof Kozmic

Ayende

Did Arnon' suggestion (DataContractSerializerOperationBehavior) fix it for you?


Gravatar

# re: WCF works in mysterious ways 6/5/2009 12:53 PM Juanma

Protobuf + TcpClient is the way to go.

We migrated a project from WCF to Protobuf + TcpClient in less than a day and got a big performance improvement.


Gravatar

# re: WCF works in mysterious ways 6/5/2009 1:59 PM Marc Gravell

Re protocol buffers - it isn't 100% there yet, but there is an in-progress interface-based RPC stack in protobuf-net. I've currently only had time to do the http layer, but this is pluggable, so should support TCP (I just haven't had time).

Or you could just use any PB implementation (I gather you're using dotnet-protobufs) and throw the data along the pipe yourself...


Gravatar

# re: WCF works in mysterious ways 6/5/2009 2:09 PM Marc Gravell

I should also add - if you are talking full .NET to full .NET, there is also a protobuf-net hook *directly* into WCF - by adding a behaviour (attribute), you can swap the serializer to use protobuf-net. If you are using httpBasic this also uses MTOM for maximum throughput.


Gravatar

# re: WCF works in mysterious ways 6/5/2009 2:48 PM Ayende Rahien

Marc,
I gave up on WCF because it got too complex to actually bother using.
Too many things that you have to get just right.
I am currently just throwing data over the wire, and so far, it seems to be pretty cool


Gravatar

# re: WCF works in mysterious ways 6/5/2009 2:50 PM Ayende Rahien

Krzysztof,
Yes, but I already made the decision that I am not going to bother with something that brittle


Gravatar

# re: WCF works in mysterious ways 6/5/2009 5:27 PM Arnon Rotem-Gal-Oz

WCF is not that brittle
I think its main problem is that many (if not all) its defaults are dumb, idiotic, fit only for demos or all of the above
Arnon


Gravatar

# re: WCF works in mysterious ways 6/5/2009 5:32 PM Frank

Sorry to hear that, Ayende. In my personal experience, WCF is not brittle, but one does need a deep understanding about how it is envisioned. Alot like many other frameworks that try to tackle a certain problem but are very flexible.


Gravatar

# re: WCF works in mysterious ways 6/5/2009 5:41 PM Ayende Rahien

The problem is that it is setup to make you fail, too many things that you have to keep in mind at once, and _way_ too much complexity at _way_ too early stage of the gaem


Gravatar

# re: WCF works in mysterious ways 6/5/2009 6:50 PM Frank

I have to disagree with that. Your scenario is not one of the more common scenarios. While developing I don't have to think about much, only configure a default binding and off you go. As soon, as you are doing stuff, like sending over alot of items and/or alot of data, you will be running into 'problems'. Many of which are present to prevent people from doing stuff like blowing up your service or receiving client by bombarding it with a never ending XML. ;)


Gravatar

# re: WCF works in mysterious ways 6/5/2009 8:46 PM Jeremy Gray

The very fact that this thread got so long and filled with so many little details yet still didn't solve the problem largely proves Ayende's point. There is no reason that he should have to fight that hard to get a large graph through. Sure, there should be limits, and they'll have to be set somewhere, but he did exactly what the exception instructed and it didn't work.

WCF is extremely flexible and extremely powerful, and with enough effort can be used to do excellent things. However, the "with enough effort" part seems to have been the real problem any time I've worked with WCF. As soon as you need to move beyond the defaults you are off chasing giant piles of XML (or its programmatic alternatives) and/or trying to figure out what combination of the myriad attributes to apply to what types on which side of the wire.

Given the power that WCF has, these kinds of things will never be dead easy but I should at least be able to spend more of my effort on direct business value and less on using the library. It doesn't seem to take too long to push WCF past a tipping point and end up at "Protocol Buffers & TcpClient, here I comes" or some other alternative.


Gravatar

# re: WCF works in mysterious ways 6/5/2009 9:53 PM Frank

Oh, but the solution is easy. Configure an endpoint behavior for the client that changes the maximum number of items that the data contract serializer allows.









It is OK if you wanna go down another part, that is your choice.


Gravatar

# re: WCF works in mysterious ways 6/5/2009 9:55 PM Frank

Hmmm, for some reason the blog engine doesn't html encode the XML I posted. View the source of this page if you want to view it. ;-)


Gravatar

# re: WCF works in mysterious ways 6/6/2009 3:03 AM John Simons

How is this different from enabling 2nd level caching in NHibernate?
Thank god(Ayende) that there is NHProf to see under the hood.


Gravatar

# re: WCF works in mysterious ways 6/6/2009 3:28 PM James

Interesting, we ran into this exact problem at work on Friday, and were scratching our heads as well.

And the prize for the most misleading exception error message goes to....Team WCF!

In our case, with both endpoints under our control, customers were going WTF, as were we. (Since we send a substantial amount of data back and forth over object graphs, some of them being rather large).


Gravatar

# re: WCF works in mysterious ways 6/6/2009 3:32 PM James

Frank,

Perhaps the solution is easy for you, but pray tell how one is to ascertain that a custom endpoint behavior to set data contract serliazer options is required, when the exception error message states to change the value of another existing option that has no effect?

WCF is archictecture astronauts gone wild.


Gravatar

# re: WCF works in mysterious ways 6/15/2009 11:35 AM KS

We hit various problem with WCF as well, especially the bandwidth it is taking: as both NetData or DataContractSerializer are in XML, they send extra information that we wants.

Looking at ProtoBuffer but I need to define .proto for each entities? Also current Proto.net does not support all C# types right?

Can I just do ISerializable over WCF? Also can WCF take something like 5000 invocation per operation per second kind-of load? We have huge problem with the bandwidth at that rate.

Thank you.


Gravatar

# re: WCF works in mysterious ways 6/20/2009 7:33 PM Yaron Naveh

Setting limits to Int32.MaxValue is not always best and can be used in DOS attacks. See here:

http://webservices20.blogspot.com/2009/06/are-wcf-defaults-considered-harmful.html


Gravatar

# re: WCF works in mysterious ways 6/26/2009 2:22 AM Paul

You know what's rubbish?

DataContractSerializerOperationBehavior is generated for each of your operations. At the time that it gets created it blissfully ignores all options that have been defined, and uses a hardcoded value of 65535.

The ONLY way to change this is with code. Kind of hard when your code is already deployed and the customer refuses to accept a new code change.

WCF is crap. Crap. CRAP.

Never use the pile of garbage, you'll just waste hours of time on plumbing which should Just Work.

Comments have been closed on this topic.