@Oren, I assume the big perf hit in the framework implementation is that first it parse the format string, then it create a parser and only then it parse the actual input string. And it will do this for each invocation of DateTime.ParseXYZ. A better approach would be to have a DateTimeParser object or something like that, that should be possible to create only once upfront and then reuse it all the time. In fact the same approach has been taken in Objective-C and their Foundation framework, specifically the NSDateFormatter class (see https://developer.apple.com/reference/foundation/nsdateformatter) which is used to to string->date and date->string conversions.

The typical old-style approach of the BCL of having very quick to use methods is nice to use, but for perf intensive scenarios they should consider this othe approach: they should take into account this parser/formatter based design, and than re-implement the standard methods using this underneath.

I suspect that by just using this approach and instancing only one parser/formatter would speed things up a lot.

What do you think?

That first paragraph is so nice :) I love it when we need to get dirty like that.

For RyuJIT code quality increase was a non-goal (puzzlingly) so it's not surprising to see it not faster. It's faster in certain special cases, and legacy JIT is faster in other special cases. The .NET JITs are garbage basically. I do not understand the severe lack of investment in them.

I say split DateTime and DateTimeOffset into two separate methods.

1. Better chance for locality (especially for large 'boring' jobs of repetitive data).
2. JIT might be happier with smaller methods.
3. No custom enum -- again JIT could be happier with builtins (it might even be clever enough to use CPU flags for returning boolean returns as grownup compilers do).

My understand of the thought behind RyuJIT was that it wanted to focus more on upfront JITing time rather than code execution time.

Or, more accurately, the legacy 64bit JITer was meant for use on servers not desktops, and so time was spent making sure that the code it generated ran the fastest possible but not much time was spent making sure the JITer ran fast. RyuJIT was created, in part, to deal with that problem.

They do seem to be spending more time, now, on improving the generated code, which is great.

Minor code review point - on line 22 of your first code sample, you are checking buffer[16] twice.

Is TryParseNumber getting inlined into the caller? If not, the optimizer cannot really generate good code for this function.

The optimizer must consider the possibility that ptr points to the same memory as the val reference. Thus every change of val must be written to memory, the compiler cannot use a CPU register. You might get slightly better performance by computing the value in a local variable and storing the result in the out-parameter only once at the end of the function.

Also, there's a lot of callers that pass size==2 -- you might want to create a separate function that parses these short integers without using a loop.

Of course if TryParseNumber is getting inlined, the compiler is probably already doing these optimizations. But I'm not sure how likely the JIT is to inline this function. I think the classic JIT never inlines functions containing loops.

@Daniel, probably we could do far better at the micro-instruction level, however, dont forget we are talking about a more than 1 order of magnitude improvement. Right now TryParseDate doesnt even show-up in the top 10 functions to optimize. Usually optimizing at that level involve writting pretty disgusting code, at this point this ugliness is far more than enough for any practical purpose.

RyuJIT does not have good internal architecture. Andy Ayers said in a public talk that the codebase is not fresh, the architecture is baroque and nobody feels quite comfortable with it. Whenever I read the code or read the commit logs it felt like it. It does not even use SSA which is known for a long time to be the correct internal compiler data structure. That's further evidence the code is not fresh.

RyuJIT not being about perf is explicitly stated on the teams blog.

I don't understand why this JIT was created. They should have created an interpreter plus a strong 2nd tier JIT that works unconstrainedly. RyuJIT feels like the result of a project or management failure. If you write a new JIT and all you achieve is 50% faster JIT times plus open source, is that really the point of the project?!

Line 22 looks mildly suspicious:

buffer[16] != ':' || buffer[16] != ':'

Just to underline the point I'll quote Mikedn from the linked issue:

One obvious difference is that RyuJIT loads ptr[i] from TryParseNumber 3 times instead of loading it once and keep it in a register.

RyuJIT does not (here and in general) extract multiple loads from the same location into a register. s.x + s.x loads x twice. RyuJIT does not perform the most basics optimizations. I do not know how it's possible to argue that it is a capable JIT. The JVM guys are laughing at this.

@tobi are you sure you are not talking about LegacyJIT? Because as far as I know RyuJIT uses (at least in some parts of its data flow) a SSA form. Among the optimization are dead code optimization via liveness analysis, loop hoisting, copy propagation and a few others.

Source: https://github.com/dotnet/coreclr/blob/master/Documentation/botr/ryujit-overview.md#ssa-vn

@tobi it does extract multiple loads into a register.

            for (int i = 0; i < size; i++)
00CF0BD5  xor         ebx,ebx  
00CF0BD7  test        edx,edx  
00CF0BD9  jle         00CF0C08  
            {
                val *= 10;
00CF0BDB  mov         eax,dword ptr [edi]  
00CF0BDD  lea         eax,[eax+eax*4]  
00CF0BE0  add         eax,eax  
00CF0BE2  mov         dword ptr [edi],eax  
                if (ptr[i] < '0' || ptr[i] > '9')
00CF0BE4  movzx       esi,byte ptr [ecx+ebx]   // Here the loads are grouped into a register, they are semantically a byte load
00CF0BE8  cmp         esi,30h  
00CF0BEB  jl          00CF0BF2  
00CF0BED  cmp         esi,39h  
00CF0BF0  jle         00CF0BFB  
                    return false;
00CF0BF2  xor         eax,eax  
00CF0BF4  pop         ebx  
00CF0BF5  pop         esi  
00CF0BF6  pop         edi  
00CF0BF7  pop         ebp  
00CF0BF8  ret         4  

                val += ptr[i] - '0';
00CF0BFB  mov         eax,dword ptr [edi]     // Here the load is not grouped with previous load as it it semantically an int
00CF0BFD  lea         eax,[eax+esi-30h]  
00CF0C01  mov         dword ptr [edi],eax  
            for (int i = 0; i < size; i++)
00CF0C03  inc         ebx  
00CF0C04  cmp         ebx,edx  
00CF0C06  jl          00CF0BDB  
            }
            return true;

@Daniel, Tryparse does not get inlined and adding a local variable does indeed improve performance by about 15% on my machine

25 M dates parses: Original: 00:00:03.0097201 Local Var: 00:00:02.5762892

@PopCatlin, this is hardcoded to work in loops. Outside of loops it does not occur. Seems to be hard to make it work in their architecture. For that reason they only addressed the most impactful case which is loops.

@Frederico I did not say the JIT does not perform any optimizations. Clearly it does. It's just quite poor in the level of optimizations that are supported.

@tobi, I'm afraid that's not true, loads are grouped even outside of loops. This basic optimization is done everywhere.

The following code has load optimization without any loops (this is done strictly by the JIT, it is not present in IL):

        private static bool TryParseNumber2(byte* ptr, out int value)
        {
            value = 0;
            if (ptr[0] < '0' || ptr[0] > '9' || ptr[1] < '0' || ptr[1] > '9')
                return false;

            value = (ptr[0]- '0') * 10 + ptr[1] - '0';
            return true;
        }

            if (ptr[0] < '0' || ptr[0] > '9' || ptr[1] < '0' || ptr[1] > '9')
01260C18  movzx       esi,byte ptr [ecx]                          // single load for  ptr[0]
01260C1B  cmp         esi,30h  
01260C1E  jl          01260C33  
01260C20  cmp         esi,39h  
01260C23  jg          01260C33  
01260C25  movzx       eax,byte ptr [ecx+1]  // single load for  ptr[1]
01260C29  cmp         eax,30h  
01260C2C  jl          01260C33  
01260C2E  cmp         eax,39h  
01260C31  jle         01260C38  
                return false;
01260C33  xor         eax,eax  
01260C35  pop         esi  
01260C36  pop         ebp  
01260C37  ret  

            value = (ptr[0]- '0') * 10 + ptr[1] - '0';
01260C38  add         esi,0FFFFFFD0h  
01260C3B  lea         ecx,[esi+esi*4]  
01260C3E  lea         eax,[eax+ecx*2-30h]  
01260C42  mov         dword ptr [edx],eax  
            return true;

Interested article. I love exploring aspects of things within themselves. I recently wrote an article about efforts making it faster to tell time. http://www.circonus.com/time-but-faster/

I found the bar charts of time and stddev a bit awkward. Had you considered using a candlestick graph to visualize the trial runs? https://en.wikipedia.org/wiki/Candlestick_chart

That optimization seems to have been added since I last looked. Testing this on Desktop 4.6.2 x64 Release:

    static void GetValue()
    {
        S s = new S();
        s.X = 1;
        Console.WriteLine(s.X + s.X);
    }
}

struct S
{
    public int X;
}

This was simplified to cw(2) which is good news! I still have the test cases I sent to Microsoft, so I know for sure this was not optimized. But this does not appear to really fold the loads... Rather, it blasts the struct to its components.

    static void GetValue()
    {
        S s = new S();
        s.X = 1;
        Console.WriteLine(s.X + s.X);
    }
}

struct S
{
    public int X;
    public int X2;
    public int X3;
    public int X4;
    public int X5;
    public int X6;
    public int X7;
}

And now it's a desaster:

        S s = new S();

000007FE95504200 push rdi
000007FE95504201 sub rsp,40h
000007FE95504205 lea rdi,[rsp+20h]
000007FE9550420A mov ecx,8
000007FE9550420F xor eax,eax
000007FE95504211 rep stos dword ptr [rdi]
000007FE95504213 xor ecx,ecx
000007FE95504215 lea rax,[rsp+20h]
000007FE9550421A xorpd xmm0,xmm0
000007FE9550421E movdqu xmmword ptr [rax],xmm0
000007FE95504222 mov qword ptr [rax+10h],rcx
000007FE95504226 mov dword ptr [rax+18h],ecx
s.X = 1; 000007FE95504229 mov dword ptr [rsp+20h],1
Console.WriteLine(s.X + s.X); 000007FE95504231 mov ecx,dword ptr [rsp+20h]
000007FE95504235 add ecx,dword ptr [rsp+20h]
000007FE95504239 call 000007FEF423F6B0

Struct on the stack, it's a constant, most of it is dead. It moves 1 into X, then loads X twice and adds it. Can it possibly be worse and less optimized? This is what I mean.

Just wondering: What's driving the need to parse? Would it be possible for whatever your source is to store in a binary format that can be loaded directly?

TryParseNumber seems broken. It is not multiplying the positional numbers by its correct powers of 10.

I know I'm late to the party (just back from a week of downtime), but does your code need to only work in half of the world? Or am I missing something else, where you only need to care about positive offset values?

I.e. I believe that 2016-10-05T21:07:32.2082285-03:00 would fail since there's only an explicit check for +.