Ayende @ Rahien

Ayende Rahien commented on Not all bytes weight exactly 8 bits

Sun, 03 Aug 2008 16:12:18 GMT

tcmaster, var is always initialized, just look at what the value is.

tcmaster commented on Not all bytes weight exactly 8 bits

Sun, 03 Aug 2008 15:15:23 GMT

It seems you really like the "var" thing. I'm really a stupid guy, and I like to be able to figure out meaning of code by reading 1st, then debugging. But "var" does well to prevent this

Davy Landman commented on Not all bytes weight exactly 8 bits

Tue, 29 Jul 2008 13:06:47 GMT

Correction.. > So using the binary writer and bitconverter you have 10000000 copies to a internal buffer and 19532 separate flushes. So using the binary writer or the bitconverter solution you have 10000000 copies to the internal buffer of the filestream and 153 separate flushes to the real file. (Looked over the buffer paramater)

Davy Landman commented on Not all bytes weight exactly 8 bits

Tue, 29 Jul 2008 09:54:47 GMT

Seeing the fact that the buffering of the filestream doesn't slow us down with the single buffer method, maybe it's possible we could convert the int array faster to an byte array... I created a faster variant, but its ugly (Unmanaged code) and I wouldn't use it unless this part was really a bottleneck. stopwatch = Stopwatch.StartNew(); using (var stream = CreateWriter()) { byte[] buffer = new byte[sizeof(int) * count]; int[] data = new int[count]; for (int i = 0; i < count; i++) { data[i] = i; } IntPtr tempBuffer = Marshal.AllocHGlobal(buffer.Length); Marshal.Copy(data, 0, tempBuffer, count); Marshal.Copy(tempBuffer, buffer, 0, buffer.Length); Marshal.FreeHGlobal(tempBuffer); stream.Write(buffer, 0, buffer.Length); stream.Flush(); } stopwatch.Stop(); Console.WriteLine("Single buffer (Marshalling): " + stopwatch.ElapsedMilliseconds);

Alessandro Riolo commented on Not all bytes weight exactly 8 bits

Tue, 29 Jul 2008 09:54:25 GMT

It is not related, but a byte is not ever 8 bits. There are (mostly were, i.e. the PDP-10) many architectures where a byte has a different weight.

Davy Landman commented on Not all bytes weight exactly 8 bits

Tue, 29 Jul 2008 09:20:33 GMT

I find it rather logical that when you create your own buffering system with knowledge of the data size, it will be faster than the default buffering in a framework (which aims for overall average performance) I looked at the source of the FileStream class, and it indeed holds an internal buffer of 4096 bytes. When write is called, the data is copied to the buffer and when the buffer is full, it's flushed tot the actual file handle. So using the binary writer and bitconverter you have 10000000 copies to a internal buffer and 19532 separate flushes. While the single buffer avoids the buffering of the FileStream class and therefore doesn't copy the memory but writes it directly to the file handle. I suspect the memory stream uses a different buffering mechanism, but that's for someone else to look at?

Rik Hemsley commented on Not all bytes weight exactly 8 bits

Tue, 29 Jul 2008 08:49:11 GMT

Similar results here, though MemoryStream and 'Single buffer' seem proportionally faster for some reason (tried many iterations, same results): Binary writer: 1581 BitConverter: 1608 MemoryStream: 1016 Single buffer: 709 With the target stream being a MemoryStream rather than FileStream: Binary writer: 362 BitConverter: 479 MemoryStream: 683 Single buffer: 349

Ayende Rahien commented on Not all bytes weight exactly 8 bits

Tue, 29 Jul 2008 05:32:53 GMT

BIl, Yeah, sense. You found a bug :-) I Updated the post accordingly

Bil Simser commented on Not all bytes weight exactly 8 bits

Tue, 29 Jul 2008 01:31:21 GMT

Is there a line missing in the code when you wrote it to the blog? There's no call to StartNew() in the last chunk.