Silverlight streaming - the race condition is already included
I am not sure how to call this issue, except maddening. For a simple repro, you can check this github repository.
The story is quite simple, let us assume that you need to send a set of values from the server to the client. For example, they might be tick values, or updates, or anything of this sort.
You can do this by keeping an HTTP connection open and sending data periodically. This is a well known technique and it works quite well. Except in Silverlight, where it works, but only if you put the appropriate Thread.Sleep() in crucial places.
Here is an example of the server behavior:
var listener = new HttpListener { Prefixes = {"http://+:8080/"} }; listener.Start(); while (true) { var ctx = listener.GetContext(); using (var writer = new StreamWriter(ctx.Response.OutputStream)) { writer.WriteLine("first\r\nsecond"); writer.Flush(); } Console.ReadKey(); }
In this case, note that we are explicitly flushing the response, then just wait. If you look at the actual network traffic, you can see that this will actually be sent, the connection will remain open, and we can actually send additional data as well.
But how do you consume such a thing in Silverlight?
var webRequest = (HttpWebRequest)WebRequestCreator.ClientHttp.Create(new Uri("http://localhost:8080/")); webRequest.AllowReadStreamBuffering = false; webRequest.Method = "GET"; Task.Factory.FromAsync<WebResponse>(webRequest.BeginGetResponse, webRequest.EndGetResponse, null) .ContinueWith(task => { var responseStream = task.Result.GetResponseStream(); ReadAsync(responseStream); });
We start by making sure that we disable read buffering, then we get the response and start reading from it. The read method is a bit complex, because is has to deal with partial response, but it should still be fairly obvious what is going on:
byte[] buffer = new byte[128]; private int posInBuffer; private void ReadAsync(Stream responseStream) { Task.Factory.FromAsync<int>( (callback, o) => responseStream.BeginRead(buffer, posInBuffer, buffer.Length - posInBuffer, callback, o), responseStream.EndRead, null) .ContinueWith(task => { var read = task.Result; if (read == 0) throw new EndOfStreamException(); // find \r\n in newly read range var startPos = 0; byte prev = 0; bool foundLines = false; for (int i = posInBuffer; i < posInBuffer + read; i++) { if (prev == '\r' && buffer[i] == '\n') { foundLines = true; // yeah, we found a line, let us give it to the users var data = Encoding.UTF8.GetString(buffer, startPos, i - 1 - startPos); startPos = i + 1; Dispatcher.BeginInvoke(() => { ServerResults.Text += data + Environment.NewLine; }); } prev = buffer[i]; } posInBuffer += read; if (startPos >= posInBuffer) // read to end { posInBuffer = 0; return; } if (foundLines == false) return; // move remaining to the start of buffer, then reset Array.Copy(buffer, startPos, buffer, 0, posInBuffer - startPos); posInBuffer -= startPos; }) .ContinueWith(task => { if (task.IsFaulted) return; ReadAsync(responseStream); }); }
While I am sure that you could find bugs in this code, that isn’t the crucial point.
If we run the server, then run the SL client, we could see that we get just one lousy byte, and that is it. Now, reading about this, it appears that in some versions of some browsers, you need to send 4KB of data to get the connection going. But that isn’t what I have observed. I tried sending 4KB+ of data, and I still saw the exact same behavior, we got called for the first byte, and nothing else.
Eventually, I boiled it down to the following non working example:
writer.WriteLine("first"); writer.Flush(); writer.WriteLine("second"); writer.Flush();
Versus this working example:
writer.WriteLine("first"); writer.Flush(); Thread.Sleep(50); writer.WriteLine("second"); writer.Flush();
Yes, you got it right, if I put the thread sleep in the server, I’ll get both values in the client. Without the Thread.Sleep, we get only the first byte. It seems like it isn’t an issue of size, but rather of time, and I am at an utter loss to explain what is going on.
Oh, and I am currently awake for 27 hours straight, most of them trying to figure out what the )(&#@!)(DASFPOJDA(FYQ@YREQPOIJFDQ#@R(AHFDS:OKJASPIFHDAPSYUDQ)(RE is going on.
Comments
Just a question, does the application that sends the lines quit (end its proces) right after it sends the data?
Since the TCP stack is just a library, you can loose data if you ExitProcess right after sending something.
Andomar, No, the application is still there and waiting.
Ok, only other thing I can think of is that you could check with Wireshark if the server or client is at fault.
Good luck!
works for me: I get "first\nsecond" in my silverlight (firefox)
Grega_g, Try in IE or Chrome, those are the things I tested this with
works as well
Grega_g, It might be a timing issue, depending on how fast you get the data. What happens when you use just one flush?
It works even without Flush() calls.
In firefox request for "/" only executes once, after that is from cache. After FF is restarted than request is made for "/" again.
Looks like this:http://tinypic.com/r/5pd88/6
Sad to say it took me some time to figure that out...
Grega_g, So, you reproduced the issue without this?
I'm not really familiar with Silverlight's web requests mechanisms, but this code looks wrong to me.
it takes way too long for you to set up the client request to listen to the next incoming bytes, so they get dropped between the beginning of the handler to when you call read async again. did you try calling ReadAsync first? (after shortcircuiting the EOF signal)
btw - the server dispose of the writer which probably dispose of the ctx.Response.OutputStream as well so the client is not getting any more data on the stream
Ken, What do you mean, takes too log? We immediately call it again
Ken, We aren't disposing the server, we kep it around in the repro.
Ken, it's TCP! Packets don't get dropped in TCP.
Nop, couldn't reproduce. Whenever request is made to "/" I get string back to silverlight and displayed in txtbox
An interesting thing I noticed: the call to Flush did not actually flush the stream - no network activity was noticed. I remember reading somewhere that http response stream in HttpListener are not flushing when told to. the rest of the bytes are send after the ReadLine, when the writer closes (and closes the outgoing request stream) forcing an actual flush.
I tried with a comparable nodejs server script. The flush behavior there is such that to get proper progressive flushing, you need to write something, wait a little bit, and from then on it will flush every now and then (depending on its internal implementation). When I did that I started seeing the bytes actually being streamed in chunks over the wire to the client and getting there correctly.
Ken, If that was what it was, I would see the same behavior in .NET and SL, but I am seeing different behavior between the two. HttpListener doesn't need to flush, it writes out immediately.
Really weird bug, if you keep the first test and the last one, everything is displayed in silverlight+ chrome ( in a VM if that matter).
That's not faulty Silverlight.
Windows sends tcp data in segments. (And the default waiting delay is 400ms).
Flush() doesn't affect NetworkStream().
There's trickery for working around that.
Bohdan, It works in .NET, but not in SL. Therefor, it is SL fault.
Comment preview