Riddle me this, why won’t this code work?
The following code will not result in the expected output:
using(var mem = new MemoryStream()) { using(var gzip = new GZipStream(mem, CompressionMode.Compress, leaveOpen:true)) { gzip.WriteByte(1); gzip.WriteByte(2); gzip.WriteByte(1); gzip.Flush(); } using (var gzip = new GZipStream(mem, CompressionMode.Compress, leaveOpen: true)) { gzip.WriteByte(2); gzip.WriteByte(1); gzip.WriteByte(2); gzip.Flush(); } mem.Position = 0; using (var gzip = new GZipStream(mem, CompressionMode.Decompress, leaveOpen: true)) { Console.WriteLine(gzip.ReadByte()); Console.WriteLine(gzip.ReadByte()); Console.WriteLine(gzip.ReadByte()); } using (var gzip = new GZipStream(mem, CompressionMode.Decompress, leaveOpen: true)) { Console.WriteLine(gzip.ReadByte()); Console.WriteLine(gzip.ReadByte()); Console.WriteLine(gzip.ReadByte()); } }
Why? And what can be done to solve this?
Comments
It's because the memory stream is left at the wrong position after the first GZipStream reads.
Either you can set it to the right position manually by assigning it after the first using block to a variable and re-setting it after the first set of reads, or you can do the writes in one using block and the reads in one using block, or use separate memory streams for each read/write pair.
The Flush() does not nothing? You should dispose the stream to flush..
Never mind, they are being disposed :-)
AFAIK, GZipStream on the does its bit:
1) at dispose time for compression 2) at creation for decompression
Considering many compression algorithms do not quite work with writing small buffers (have to work on bigger chunks), this is natural.
Solution is to use stream's CopyTo to copy to another stream and read from that.
The problem is that DeflateStream (which is used by GZipStream) uses a buffer when reading. It tries to read up to 8 kB of the source stream, regardless of the contents, after which the contents of the buffer are parsed. Might be that the easiest solution is to prepend the compressed length of each block, and then feeding only that block to GZipStream.
your Flush are useless http://msdn.microsoft.com/fr-fr/library/system.io.compression.gzipstream.flush.aspx but the dispose should do it. it seems that the first readByte on your GziptStreamm is reading the full memory stream (its position is 250). But I can't figure out where between the Inflater class or the OutputWindow class this is done. Still searching...
You can make the code work by setting the underlying MemoryStreams Position at position 23 between the two decompression blocks. Cfr what Erik says it uses internelly a buffer. So you will need to store the memorystream positionsafter every compress block.
As PJL says, storing the MemoryStream position after each using block seems to be the only solution to make this code works...
The problem is how GzipStream reads it's base stream. kerrubin answer is correct but is not good in real application. I prefer this approch: first create GzipDecorator like this: public class GzipStreamDecorator : Stream {
then change your code like this: using (var mem = new MemoryStream()) { using (var gzip = new GZipStream(mem, CompressionMode.Compress, leaveOpen: true)) { gzip.WriteByte(1); gzip.WriteByte(2); gzip.WriteByte(1); gzip.Flush(); }
Ooh, a puzzle! Intentionally jumping to the bottom to not taint my brain, so apols if this is already answered:
The GZipStream, if I recall correctly, consumes the whole underlying stream in decompress mode...if not that, it definitely won't leave your position where you'd expect it to. A "fix" would be to keep track of where you were after each batch of writes, albeit annoying:
Sorry, correct code is: public class GzipStreamDecorator : Stream {
Hi Ayende, are you hiring and is this the interview question? :)
could you please give us some feedback on the answers and perhaps some more context around this issue. (personally I like PJL's answer)
Waiting...
The disposal of the GZipStreams is disposing of the MemoryStream as well even though we want to append to it later. There is no need for all of the inner using statements.
Comment preview