+972 52-548-6969
TalkExtreme Performance Architecture
time to read 1 min | 19 words
I gave this talk a few months ago, and now it is out, enjoy.
More posts in "Talk" series:
- (09 Jun 2023) Scalable Architecture From the Ground Up
- (14 Dec 2021) Scalable architecture from the ground up
- (23 Apr 2020) Advanced indexing with RavenDB
- (22 Apr 2020) Modeling in a non relational world
- (03 Jan 2018) Modeling in a Non Relational World
- (29 Dec 2017) Extreme Performance Architecture
- (28 Nov 2016) System Architecture with NoSQL and RavenDB/DocDB
Comments
You missed a out
Thanks, fixed
Hi,
really nice talk. I know you get this a lot, but after seeing the video and reading your blogs, i get the feeling that choosing a GC language (C#) and doing a lot of direct memory management defeats the purpose of using a GC. I understand the time constraint totally but it might be worth it to use a NON-GC language to do this type of work (a systems language vs a generic programming language). Do not get me wrong i love C# very much but i do not think that it is the correct tool for such kind of work. Have you ever considered Rust? It seems that it is as fast as C, C++ and has a lot of perks of the never languages and has no GC, is safe, which means the time spend to optimize the GC can be used to implement some missing stuff that you get for free in C#. If time was not such an issue (a semi-ideal world) and you could live with taking a little more time to implement would you consider it?
mantzas, We still get a lot of benefits from the GC, make no mistake. In particular, not having to manage anything in the critical path is huge. This basically means that we can write a feature as fast as you can in a high level language with all the benefits that usually come with it, then we can see if it is in the hot path and optimize that.
Rust is several orders of magnitudes more complex than anything else that I have tried using.
If time / resources was not an issue, I might have tried doing that in C++, because modern C++ is just flat amazing, but there are other issues here, not the least of which is the cost of compilation / building that can really hurt development.
Oren,
my point was that if after the optimization the code looks more like C than C# it would be a indicator of having used the wrong tool for the job. But, if you almost have the performance that you get in writing the thing lets say in C++ then i am all for it to make look a little bit like C. Do you have a gut feeling of the performance boost just by using C++ and what it would mean in terms of development cost (ball park numbers)?
BTW I agree on the complexity of Rust and AFAIK Rust suffers from the cost of compilation/build too.
mantzas, Most of our code is actually idiomatic C#. A small core is mostly C looking but that is exposed through an interface that any C# dev can consume. I don't think we pay too high a cost here.
nice, thanks for the info.
Excellent talk! If I understand you, you're reading and writing (via MemMap file) the blittable format. If I have that right, how do you plan to handle changes needed to the blittable format? ie. I assume you may want to rearrange fields commonly used together for best cache locality. If a new feature or profiling beings up the need to rearrange the blittable fields, will that introduce a fork in the formats or need to bring an older format forward to the newer format?
Thinking about it more, if the format were to change, I supposed you'd just have a batch process to rewrite in the new format once and be done with it. I'd assume the file would not be simultaneously consumed by two different versions of the s/w.
Thanks again for sharing such useful low (and high-) level details of your process.
John, Any and all such changes would be breaking changes, as such, we'll have to introduce a migration step. Typically'll recognize the format and convert it to the new one on the fly on first read.
Comment preview