Ayende @ Rahien

Refunds available at head office

Playing with Roslyn

We do a lot of compiler work in RavenDB. Indexes are one such core example, where we take the C# language and beat both it and our heads against the wall until it agrees to do what we want it to.

A lot of that is happening using the excellent NRefactory library as well as the not so excellent CodeDOM API. Basically, we take a source string, convert it into something that can run, then compile it on the fly and execute it.

I decided to check the implications of using this using a very trivial benchmark:

private static void CompileCodeDome(int i)
{
    var src = @"
class Greeter
{
static void Greet()
{
System.Console.WriteLine(""Hello, World"" + " + i + @");
}
}";
    CodeDomProvider codeDomProvider = new CSharpCodeProvider();
    var compilerParameters = new CompilerParameters
    {
        OutputAssembly= "Greeter.dll",
        GenerateExecutable = false,
        GenerateInMemory = true,
        IncludeDebugInformation = false,
        ReferencedAssemblies =
        {
            typeof (object).Assembly.Location,
            typeof (Enumerable).Assembly.Location
        }
    };
    CompilerResults compileAssemblyFromSource = codeDomProvider.CompileAssemblyFromSource(compilerParameters, src);
    Assembly compiledAssembly = compileAssemblyFromSource.CompiledAssembly;
}

private static void CompileRoslyn(int i)
{
    var syntaxTree = CSharpSyntaxTree.ParseText(@"
class Greeter
{
static void Greet()
{
System.Console.WriteLine(""Hello, World"" + " +i +@");
}
}");

    var compilation = CSharpCompilation.Create("Greeter.dll",
        syntaxTrees: new[] {syntaxTree},
        references: new MetadataReference[]
        {
            new MetadataFileReference(typeof (object).Assembly.Location),
            new MetadataFileReference(typeof (Enumerable).Assembly.Location),
        });

    Assembly assembly;
    using (var file = new MemoryStream())
    {
        var result = compilation.Emit(file);
    }
}

 

I run it several times, and I got (run # on the X axis, milliseconds on the Y axis):

image

The very first Roslyn invocation is very costly. The next are pretty much nothing. Granted, this is a trivial example, but the CodeDOM (which invokes csc) is both much more consistent but much more expensive in general.

Comments

Daan Le Duc
09/01/2014 09:16 AM by
Daan Le Duc

Yeah, i've seen David Fowler talk about this issue with new Visual Studio when you build for very first time. Takes lots of time not sure if the roslyn team is going to fix this.

tobi
09/01/2014 10:50 AM by
tobi

.NET startup performance caused the Vista rewrite. Vista was supposed to be based on .NET. Then, they removed the CLR from the startup path entirely.

Kirill Osenkov
09/01/2014 05:16 PM by
Kirill Osenkov

Roslyn is JITting. You can try ngen install Microsoft.CodeAnalysis.CSharp.dll from admin command prompt to see if this improves the cold Roslyn time.

If you do this in production, preheat Roslyn on a background thread before first use by compiling this exact program in memory.

Ayende Rahien
09/01/2014 08:38 PM by
Ayende Rahien

Kirill, Sure, the problem is that I'm going to be using that in order to reduce startup time :-)

James
09/02/2014 09:12 AM by
James

So how about starting with the CodeDOM version, compile via Roslyn on a separate thread. When that compilation has completed, switch out the CodeDOM version for the Roslyn version?

That way you get the best of both worlds!

Ayende Rahien
09/02/2014 09:13 AM by
Ayende Rahien

James, The cost of the CodeDOM isn't high enough to actually justify something like that. This is very costly process.

Pop Catalin
09/02/2014 11:43 AM by
Pop Catalin

The startup time of Roslyn is due to being a preview and not installed in the GAC. Roslyn preview uses a separate process as compilation server that is loaded at first use and the Rolsyn bytecode is jit-ed on load.

http://blog.slaks.net/2014-05-21/exploring-roslyn-part-2-inside-end-user-preview/