Ayende @ Rahien

It's a girl

Challenge: write a self extracting program

I had to do this yesterday, and it was fun enough to post about it. Not only that, but the solution that I came up with is enough of a twist that I think would make this hard for many people.

Here is the challenge:

Given a zip file, produce an executable that when run on another machine, will unzip the content of the zip file on that machine. (nitpicker corner: yes, there are many ways to cheat this, but don’t try to cheat). You may use any library and process that you want, but the end result must be a standalone file that can be copied to an separate machine and when execute will extract the content of the zip file.

You can assume that on that separate machine you will have the .Net framework installed, but you cannot depend on the presence of external libraries. I’ll note that the default compression streams in the .Net framework are not applicable here, since they can’t handle zip files (which contains a directory structure), they can only handle zip streams.

I took me roughly 30 minutes, most of which were spent dealing with ICSharpCode.SharpZipLib API, which is not something that I dealt with before.

Comments

configurator
02/18/2009 02:36 AM by
configurator

Why not use a WinZip self extracting zip? Isn't it customizable enough?

What about creating the exe with the zip file as a resource and whatever library you want, then using ILMerge?

Ayende Rahien
02/18/2009 02:44 AM by
Ayende Rahien

WinZip doesn't give me the chance to add my own code to the mix.

David Whitney
02/18/2009 09:05 AM by
David Whitney

After using SharpZipLib (and being slightly annoyed by it) in the past, last week I did a quick google excercise to see if anything else existed.

http://www.codeplex.com/DotNetZip - Seemed far easier to use (and worked on the Compact Framework, which was the goal last week).

But yeah, grab the source of a zip component, embed the zip as a resource, go go go, no?

Rafal
02/18/2009 10:10 AM by
Rafal

Few years ago I did the same thing, but without .Net. The trick was simple: wrote a program, compiled it to an exe and appended zip file to the exe file. Exe size was hardcoded in its source code, so the program knew where the zip contents start. The program read its exe file starting at proper position and extracted the data using some zip library (I don't remember what was the library).

E. Muniz
02/18/2009 01:38 PM by
E. Muniz

I'm not sure what qualifies as cheating but...

What about making the ZIP file an embedded resource? In other words, use the standard plumbing to extract the resource and then CSharpCode.SharpZipLib API to perform the actual unzip.

devdimi
02/18/2009 03:41 PM by
devdimi

I have done this in C++, using the Win32 ressouce APIs last year.

You can save anything in an exe file, even if it is a 40mb MSI or zip file.

On execution the program just extracts the saved ressource and does with it whatever it needs to do - unzip it, install it with msiexec or send it the moon.

I guess with Interop you could use the FindRessouce family of functions and do exactly the same thing in .NET

With C++ I needed a little more than 30 minutes for this though :)

Stuart C
02/18/2009 08:14 PM by
Stuart C

I've used the J# libaries to perform zip compression operations in the past, everything required is in the java.util.zip namespace. As for self extracting, I would create a standard setup exe and include the zip file as an embedded resource which is trivial to do.

Matei
02/19/2009 05:32 PM by
Matei

I would also go with the embeded resource, as above. I'd embed both the ziplib and the zip file and than use something like Activator.CreateInstance to load whatever types are necesary to unzip the file. I should give this a try and see if it works.

Cheers,

Matei

configurator
02/19/2009 11:38 PM by
configurator

@Matei: How would you CreateInstance for the embedded ziplib? Would you embed a dll, unpack it to a temporary location and then run it? Sounds nasty.

Cheeso
02/20/2009 03:22 AM by
Cheeso

DotNetZip takes the "embedded resource" approach for the self-extractors it creates (created using the ZipFile.SaveSelfExtractor() method). Regarding "CreateInstance", there is no CreateInstance since it is .NET, and not COM, but I get your meaning. If the goal is to have a single exe image, then the zip DLL must be embedded as well as the zip archive itself. So how do you call into the library? Well, .NET allows the app to extend the "Assembly Resolution" logic - the way the loader finds libraries. So, the SFX EXE created by DotNetZip uses a custom assembly resolver, and loads the ZIP assembly (DLL) from a byte array that is read from the appropriate embedded resource. Just a few lines of code and no unpacking the DLL to disk. (Yes, .NET can load libraries from in-memory byte streams). Pretty slick. DotNetZip's SFX has 2 embedded resources: one is the zip file and one is the DLL.

WinZip does it slightly differently. A WinZip SFX is native code, does not use .NET. In the Winzip SFX EXE, there is a "Section" with a well-known name, winzip. That contains the zip entry data. The SFX logic just looks for the zip data in that section; which is essentially an "embedded resource" without all the window dressing. But the cool thing is - Because of the way WinZip structures it, the WinZip self-extractor is an EXE, but is also a legal zip archive. if you rename the WinZip sfx exe to .zip, it will (should) be readable by any compliant zip tool, including Windows Explorer. It is both an EXE and a ZIP. A WinZip-created SFX EXE can be loaded into the WinZip tool (or any zip tool) and examined visually.

Info-Zip takes the "known exe length" trick to build its SFX files.

Matei
02/20/2009 08:58 PM by
Matei

@configurator

As Cheeso says we can load an assembly from a byte[]; from there on it becomes and exercise in reflection, invoking the methods/properties on the dynamically loaded types to extract the zip file from the stream we get from the resource. So no hard dependency to the library, the only file that needs to be deployed is the .exe. The ILMerge is also a viable option as far as I'm concerned - bothe you and Simon pointed this out.

Here's something quick and dirty I put toghether. Took me more than 30 mins :) I get no practice with reflection, nor do I know the SharpZipLibApi.

static void Main(string[] args)

    {

        Assembly zipLib =

            GetAssemblyFromStream(

                Assembly.GetExecutingAssembly().GetManifestResourceStream("Engine.lib.ICSharpCode.SharpZipLib.dll"));

        Type inputStreamType = zipLib.GetType("ICSharpCode.SharpZipLib.Zip.ZipInputStream");

        Type entryType = zipLib.GetType("ICSharpCode.SharpZipLib.Zip.ZipEntry");


        Stream inputStream = (Stream)Activator.CreateInstance(inputStreamType,

            new object[] {

                            Assembly.GetExecutingAssembly().

                            GetManifestResourceStream("Engine.assets.Console2.zip") });

        object zipEntry;

        while ((zipEntry = inputStreamType.InvokeMember("GetNextEntry", BindingFlags.InvokeMethod, null, inputStream, null)) != null)

        {

            string path = (string)entryType.InvokeMember("Name", BindingFlags.GetProperty, null, zipEntry, null);


            string directoryName = Path.GetDirectoryName(path);

            if (directoryName.Length > 0)

            {

                Directory.CreateDirectory(directoryName);

            }


            string fileName = Path.GetFileName(path);

            if (fileName != String.Empty)

            {

                using (FileStream streamWriter = File.Create(path))

                {


                    int size = 2048;

                    byte[] data = new byte[2048];

                    while (true)

                    {

                        size = (int)inputStreamType.InvokeMember("Read", BindingFlags.InvokeMethod, null, inputStream, new object[] { data, 0, data.Length }); //zipStream.Read(data, 0, data.Length);

                        if (size > 0)

                        {

                            streamWriter.Write(data, 0, size);

                        }

                        else

                        {

                            break;

                        }

                    }

                }

            }


        }


        Console.ReadKey();

    }
Cheeso
02/22/2009 06:47 PM by
Cheeso

Just one note - You do not need to resort to reflection to invoke the methods and properties on the dynamically-loaded assembly. If you use CreateInstance, I suppose you DO, but if you just load the library "normally" - via a ResolveEventHandler that loads the byte strea - then your code won't have to rely on CreateInstance or reflection.

    static System.Reflection.Assembly Resolver(object sender, ResolveEventArgs args)

    {

        Assembly a1 = Assembly.GetExecutingAssembly();

        Stream s = a1.GetManifestResourceStream("Ionic.Zip.dll");

        byte[] block = new byte[s.Length];

        s.Seek(0, System.IO.SeekOrigin.Begin);

        s.Read(block, 0, block.Length);


        Assembly a2 = Assembly.Load(block);


        return a2;

    }

At compile time you'd reference the dynamically-loaded DLL (DotNetZip or SharpZipLib or whatever) but you would not need to distribute that DLL separately. It'd be embedded as a resource, and it would be loaded via your ResolveEventHandler.

This is the approach DotNetZip takes in its SFX files.

Nair
02/23/2009 10:12 PM by
Nair

I had the same problem. Where i need to create a single exe with all resources in it. First I went with .Net approach with Resource files and since my resources are over 640MB my compiler failed. I dropped that I idea and went to ILMerge route and that also had the same limiation. So I ended up going to inno setup which is more like installshield but open source.

My file size is 1.2GB, do you think I would able to use SharpLib and resolve this problem? I thought SharpLib had a limiation of subdirectories??

Comments have been closed on this topic.