Future proofing for crazy performance optimizations
I talked about the new query language and javascript performance quite a bit, but I wanted post about a conversation that we had at the office. It involves the following design decision:
We chose to use an object literal syntax as the complex projection mechanism for several reasons. First, it is easy to implement right now, since we can throw that to the JS engine and let it handle this. Second, it is very readable and given that we are a JSON document database, should be very familiar for users. But the third reason isn’t so obvious. It is using this syntax because we can quite easily optimize most common scenarios very easily.
I can take the object literal and parse that into an AST, and instead of sending that to a JS engine, I can execute that directly. Even if I’m going to implement a trivial AST traverser that can the most basic of operations, that would be enough to handle a lot of scenarios, and I don’t have to be 100%. I can implement an AST walked and run that, and if I find that there is something there that I can’t support I can just fallback to the JS engine.
Since my AST interpreter wouldn’t need any of the JS engine behaviors, it can be implemented much more efficiently and with greater focus. For that matter, I don’t have to limit myself to just interpreting this AST. I can also generate IL code on the fly to evaluate this instead, given me an even higher speed boost.
Basically, this syntax give us at least three levels of optimization opportunities to explore as needed.
Comments
Comment preview