Ayende @ Rahien

It's a girl

Silverlight streaming - the race condition is already included

I am not sure how to call this issue, except maddening. For a simple repro, you can check this github repository.

The story is quite simple, let us assume that you need to send a set of values from the server to the client. For example, they might be tick values, or updates, or anything of this sort.

You can do this by keeping an HTTP connection open and sending data periodically. This is a well known technique and it works quite well. Except in Silverlight, where it works, but only if you put the appropriate Thread.Sleep() in crucial places.

Here is an example of the server behavior:

var listener = new HttpListener
{
    Prefixes = {"http://+:8080/"}
};
listener.Start();
while (true)
{
    var ctx = listener.GetContext();
    using (var writer = new StreamWriter(ctx.Response.OutputStream))
    {
        writer.WriteLine("first\r\nsecond");
        writer.Flush();
    }
    Console.ReadKey();
}

In this case, note that we are explicitly flushing the response, then just wait. If you look at the actual network traffic, you can see that this will actually be sent, the connection will remain open, and we can actually send additional data as well.

But how do you consume such a thing in Silverlight?

var webRequest = (HttpWebRequest)WebRequestCreator.ClientHttp.Create(new Uri("http://localhost:8080/"));
webRequest.AllowReadStreamBuffering = false;
webRequest.Method = "GET";

Task.Factory.FromAsync<WebResponse>(webRequest.BeginGetResponse, webRequest.EndGetResponse, null)
    .ContinueWith(task =>
    {
        var responseStream = task.Result.GetResponseStream();
        ReadAsync(responseStream);
    });

 

We start by making sure that we disable read buffering, then we get the response and start reading from it. The read method is a bit complex, because is has to deal with partial response, but it should still be fairly obvious what is going on:

byte[] buffer = new byte[128];
private int posInBuffer;
private void ReadAsync(Stream responseStream)
{
    Task.Factory.FromAsync<int>(
        (callback, o) => responseStream.BeginRead(buffer, posInBuffer, buffer.Length - posInBuffer, callback, o),
        responseStream.EndRead, null)
        .ContinueWith(task =>
        {
            var read = task.Result;
            if (read == 0) 
                throw new EndOfStreamException();
            // find \r\n in newly read range

            var startPos = 0;
            byte prev = 0;
            bool foundLines = false;
            for (int i = posInBuffer; i < posInBuffer + read; i++)
            {
                if (prev == '\r' && buffer[i] == '\n')
                {
                    foundLines = true;
                    // yeah, we found a line, let us give it to the users
                    var data = Encoding.UTF8.GetString(buffer, startPos, i - 1 - startPos);
                    startPos = i + 1;
                    Dispatcher.BeginInvoke(() =>
                    {
                        ServerResults.Text += data + Environment.NewLine;
                    });
                }
                prev = buffer[i];
            }
            posInBuffer += read;
            if (startPos >= posInBuffer) // read to end
            {
                posInBuffer = 0;
                return;
            }
            if (foundLines == false)
                return;

            // move remaining to the start of buffer, then reset
            Array.Copy(buffer, startPos, buffer, 0, posInBuffer - startPos);
            posInBuffer -= startPos;
        })
        .ContinueWith(task =>
        {
            if (task.IsFaulted)
                return;
            ReadAsync(responseStream);
        });
}

While I am sure that you could find bugs in this code, that isn’t the crucial point.

If we run the server, then run the SL client, we could see that we get just one lousy byte, and that is it. Now, reading about this, it appears that in some versions of some browsers, you need to send 4KB of data to get the connection going. But that isn’t what I have observed. I tried sending 4KB+ of data, and I still saw the exact same behavior, we got called for the first byte, and nothing else.

Eventually, I boiled it down to the following non working example:

writer.WriteLine("first");
writer.Flush();
writer.WriteLine("second");
writer.Flush();

Versus this working example:

writer.WriteLine("first");
writer.Flush();
Thread.Sleep(50);
writer.WriteLine("second");
writer.Flush();

Yes, you got it right, if I put the thread sleep in the server, I’ll get both values in the client. Without the Thread.Sleep, we get only the first byte. It seems like it isn’t an issue of size, but rather of time, and I am at an utter loss to explain what is going on.

Oh, and I am currently awake for 27 hours straight, most of them trying to figure out what the )(&#@!)(DASFPOJDA(FYQ@YREQPOIJFDQ#@R(AHFDS:OKJASPIFHDAPSYUDQ)(RE is going on.

Tags:

Posted By: Ayende Rahien

Published at

Originally posted at

Comments

Andomar
07/25/2012 10:24 AM by
Andomar

Just a question, does the application that sends the lines quit (end its proces) right after it sends the data?

Since the TCP stack is just a library, you can loose data if you ExitProcess right after sending something.

Ayende Rahien
07/25/2012 10:27 AM by
Ayende Rahien

Andomar, No, the application is still there and waiting.

Andomar
07/25/2012 11:26 AM by
Andomar

Ok, only other thing I can think of is that you could check with Wireshark if the server or client is at fault.

Good luck!

grega_g
07/25/2012 01:00 PM by
grega_g
                // Doesn't work
                writer.WriteLine("first");
                writer.Flush();
                writer.WriteLine("second");
                writer.Flush();

works for me: I get "first\nsecond" in my silverlight (firefox)

Ayende Rahien
07/25/2012 01:03 PM by
Ayende Rahien

Grega_g, Try in IE or Chrome, those are the things I tested this with

grega_g
07/25/2012 01:06 PM by
grega_g

works as well

Ayende Rahien
07/25/2012 01:09 PM by
Ayende Rahien

Grega_g, It might be a timing issue, depending on how fast you get the data. What happens when you use just one flush?

grega_g
07/25/2012 01:56 PM by
grega_g

It works even without Flush() calls.

grega_g
07/25/2012 02:03 PM by
grega_g

In firefox request for "/" only executes once, after that is from cache. After FF is restarted than request is made for "/" again.

Looks like this:http://tinypic.com/r/5pd88/6

grega_g
07/25/2012 02:07 PM by
grega_g

Sad to say it took me some time to figure that out...

Ayende Rahien
07/25/2012 03:52 PM by
Ayende Rahien

Grega_g, So, you reproduced the issue without this?

Ken Egozi
07/25/2012 09:33 PM by
Ken Egozi

I'm not really familiar with Silverlight's web requests mechanisms, but this code looks wrong to me.

it takes way too long for you to set up the client request to listen to the next incoming bytes, so they get dropped between the beginning of the handler to when you call read async again. did you try calling ReadAsync first? (after shortcircuiting the EOF signal)

Ken Egozi
07/25/2012 10:57 PM by
Ken Egozi

btw - the server dispose of the writer which probably dispose of the ctx.Response.OutputStream as well so the client is not getting any more data on the stream

Ayende Rahien
07/26/2012 03:37 AM by
Ayende Rahien

Ken, What do you mean, takes too log? We immediately call it again

Ayende Rahien
07/26/2012 03:37 AM by
Ayende Rahien

Ken, We aren't disposing the server, we kep it around in the repro.

Torvin
07/26/2012 05:48 AM by
Torvin

Ken, it's TCP! Packets don't get dropped in TCP.

grega_g
07/26/2012 05:54 AM by
grega_g

Nop, couldn't reproduce. Whenever request is made to "/" I get string back to silverlight and displayed in txtbox

Ken Egozi
07/26/2012 08:59 PM by
Ken Egozi

An interesting thing I noticed: the call to Flush did not actually flush the stream - no network activity was noticed. I remember reading somewhere that http response stream in HttpListener are not flushing when told to. the rest of the bytes are send after the ReadLine, when the writer closes (and closes the outgoing request stream) forcing an actual flush.

I tried with a comparable nodejs server script. The flush behavior there is such that to get proper progressive flushing, you need to write something, wait a little bit, and from then on it will flush every now and then (depending on its internal implementation). When I did that I started seeing the bytes actually being streamed in chunks over the wire to the client and getting there correctly.

Ayende Rahien
07/26/2012 09:01 PM by
Ayende Rahien

Ken, If that was what it was, I would see the same behavior in .NET and SL, but I am seeing different behavior between the two. HttpListener doesn't need to flush, it writes out immediately.

Guillaume
07/29/2012 06:48 AM by
Guillaume

Really weird bug, if you keep the first test and the last one, everything is displayed in silverlight+ chrome ( in a VM if that matter).

Bohdan Trotsenko
08/22/2012 10:52 AM by
Bohdan Trotsenko

That's not faulty Silverlight.

Windows sends tcp data in segments. (And the default waiting delay is 400ms).

Flush() doesn't affect NetworkStream().

There's trickery for working around that.

Ayende Rahien
08/22/2012 10:56 AM by
Ayende Rahien

Bohdan, It works in .NET, but not in SL. Therefor, it is SL fault.

Comments have been closed on this topic.