Ayende @ Rahien

Hi!
My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by email or phone:

ayende@ayende.com

+972 52-548-6969

, @ Q c

Posts: 6,517 | Comments: 47,934

filter by tags archive

RavenDB SetupHow the automatic setup works

time to read 8 min | 1456 words

imageOne of the coolest features in the RC2 release for RavenDB is the automatic setup, in particular, how we managed to get a completely automated secured setup with minimal amount of fuss on the user’s end.

You can watch the whole thing from start to finish, it takes about 3 minutes to go through the process (if you aren’t also explaining what you are doing) and you have a fully secured cluster talking to each other over secured TLS 1.2 channels.  This was made harder because we are actually running with trusted certificates. This was a hard requirement, because we use the RavenDB Studio to manage the server, and that is a web application hosted on RavenDB itself. As such, it is subject to all the usual rules of browser based applications, including scary warnings and inability to act if the certificate isn’t valid and trusted.

In many cases, this lead people to chose to use HTTP. Because at least with that model, you don’t have to deal with all the hassle. Consider the problem. Unlike a website, that has (at least conceptually) a single deployment, RavenDB is actually deployed on customer sites and is running on anything from local developer machines to cloud servers. In many cases, it is hidden behind multiple layers of firewalls, routers and internal networks. Users may chose to run it in any number of strange and wonderful configurations, and it is our job to support all of them.

In such a situation, defaulting to HTTP only make things easy. Mostly because things work. Using HTTPS require that we’ll use a certificate. We can obviously use a self signed certificate, and have the following shown to the user on the first access to the website:

image

As you can imagine, this is not going to inspire confidence with users. In fact, I can think of few other ways to ensure the shortest “download to recycle bin” path. Now, we could ask the administrator to generate a certificate an ensure that this certificate is trusted. And that would work, if we could assume that there is an administrator. I think that asking a developer that isn’t well versed in security practices to do that is likely to result in an even shorter “this is waste of my time” reaction than the unsecured warning option.

We considered the option of installing a (locally generated) root certificate and generating a certificate from that. This would work, but only on the local machine, and RavenDB is, by nature, a distributed database. So that would make for a great demo, but it would cause a great deal of hardships down the line. Exactly the kind of feature and behavior that we don’t want. And even if we generate the root certificate locally and throw it away immediately afterward, the idea still bothered me greatly, so that was something that we considered only in times of great depression.

So, to sum it all up, we need a way to generate a valid certificate for a random server, likely running in a protected network, inaccessible from the outside (as  in, pretty much all corporate / home networks these days). We need to do without requiring the user to do things like setup dynamic DNS, port forwarding in router or generating their own certificates. We also need to to be fast enough that we can do that as part of the setup process. Anything that would require a few hours / days is out of the question.

We looked into what it would take to generate our own trusted SSL certificates. This is actually easily possible, but the cost is prohibitive, given that we wanted to allow this for free users as well, and all the options we got always had a per generated certificate cost associated with it.

Let’s Encrypt is the answer for HTTPS certificate generation on the public web, but the vast majority all of our deployments are likely to be inside the firewall, so we can’t verify a certificate using Let’s Encrypt. Furthermore, doing so will require users to define and manage DNS settings as part of the deployment of RavenDB. That is something that we wanted to avoid.

This might require some explanation. The setup process that I’m talking about is not just to setup a production instance. We consider any installation of RavenDB to be worth a production grade setup. This is a lesson from the database ransomware tales. I see no reason why we should learn this lesson again on the backs of our users, so a high priority was given to making sure that the default install mode is also the secure and proper one.

All the options that are ruled out in this post (provide your own certificate, setup DNS, etc) are entirely possible (and quite easily) with RavenDB, if an admin so chose, and we expect that many will want to setup RavenDB in a manner that fits their organization policies. But here we are talkingh about the base line (yes, dear) install and we want to make it as simple and straightforward as we possibly can.


There is another problem with Let’s Encrypt for our situation, we need to generate a lot of certificates, significantly more than the default rate limit that Let’s Encrypt provides. Luckily, they provide a way to request an extension to this rate limit, which is exactly what we did. Once this was granted, we were almost there.

imageThe way RavenDB generates certificates as part of the setup process is a bit involved. We can’t just generate any old hostname, we need to provide proof to Let’s Encrypt that we own the hostname in question. For that matter, who is the we in question? I don’t want to be exposed to all the certificates that are generated for the RavenDB instances out there. That is not a good way to handle security.

The key for the whole operation is the following domain name: dbs.local.ravendb.net

During setup, the user will register a subdomain under that, such as arava.dbs.local.ravendb.net. We ensure that only a single user can claim each domain. Once they have done that, they let RavenDB what IP address they want to run on. This can be a public IP, exposed on the internet, a private one (such as 192.168.0.28) or even a loopback device (127.0.0.1).

The local server, running on the user’s machine then initiates a challenge to Let’s Encrypt for the hostname in question. With the answer to the challenge, the local server then call to api.ravendb.net. This is our own service, running on the cloud. The purpose of this service is to validate that the user “owns” the domain in question and to update the DNS records to match the Let’s Encrypt challenge.

The local server can then go to Let’s Encrypt and ask them to complete the process and generate the certificate for the server. At no point do we need to have the certificate go through our own servers, it is all handled on the client machine. There is another thing that is happening here. Alongside the DNS challenge, we also update the domain the user chose to point to the IP they are going to be hosted at. This means that the global DNS network will point to your database. This is important, because we need the hostname that you’ll use to talk to RavenDB to match the hostname on the certificate.

Obviously, RavenDB will also make sure to refresh the Let’s Encrypt certificate on a timely basis.

The entire process is seamless and quite amazing when you see it. Especially because even developers might not realize just how much goes on under the cover and how much pain was taken away from them.

We run into a few issues along the way and Let’s Encrypt support has been quite wonderful in this regard, including deploying a code fix that allowed us to make the time for RC2 with the full feature in place.

There are still issues if you are running on a completely isolated network, and some DNS configurations can cause issues, but we typically detect and give a good warning about that (allowing you to switch to 8.8.8.8 as a good workaround for most such issues). The important thing is that we achieve the main goal, seamless and easy setup with the highest level of security.

RavenDB Setupa secured cluster in 10 minutes or less

time to read 1 min | 86 words

One of the major features of the RC2 release for RavenDB has been the setup process. In particular, we worked on making sure that the default and easiest manner to install RavenDB will be the one with the highest level of security.

I’m excited enough by this feature that I recorded myself setting up a full blown cluster, including everything you need for production deployment in under 10 minutes, with a lot of my explanations in the middle. Take a look.

The best features are the ones you never knew were thereUnsecured SSL/TLS

time to read 3 min | 598 words

imageI wish I would have been sufficient to use HTTPS for security. With RavenDB 4.0’s move toward TLS as the security mechanism for encryption of data over the wire and authentication using x509 we had to learn way too much about how Transport Layer Security works.

In particular, it can be quite annoying when you realize that just because you use SSL (or more accurately, TLS) that isn’t sufficient. You need to use the proper version, and there are interoperability issues. Many of RavenDB’s users run it in an environments that are subject to strict scrutiny and high level of regulation and oversight. That means that we need to make sure that we are able to operate in such environment. One option would be to use something like a FIPS configuration. We have a “normal”configuration and one that is aimed at people that need stricter standards. For many reasons, this is a really bad idea. Not least of all is the problem that even if you don’t have to meet FIMS mandate, you still want to be secured. Amusingly enough, many FIPS certified stacks are actually less secured (because they can’t get patches to the certified binaries).

So the two options mode was rejected. That meant that we should run in a mode that is can be match the requirements of the most common deployment regulations. In particular interest to us is PCI compliance, since we are often deployed in situations that involve money and payment processing.

That can be a bit of a problem. PCI requires that your communication will use TLS, obviously. But it also requires it to use TLS 1.2. That is great and with .NET it is easily supported. However, not all the tools are aware of this. This put us back in the same state as with HTTP vs. HTTPS. If your client does not support TLS 1.2 and your server require TLS 1.2, you end up in with a with a connection error.

image

Such a thing can be maddening for the user.

Therefor, RavenDB will actually allow Tls and Tls11 connections, but instead of processing the request, it will give you an error that give you something to work with.

image

Updated: I forgot to actually read the message. The reason you are getting the error about no certificate is because there isn’t a certificate here. In order for this to work, we need to actually pass the certificate, in which case we’ll get the appropriate error. I apologize for the error handling, but PowerShell:
image

Armed with this information, you can now do a simple web search and realize that you actually need to do this:

image

And that saves us a lot of TCP level debugging. It took a bit of time to set this (and the other) errors properly, and they are exactly the kind of things that will save you hours or days of frustration, but you’ll never realize that they were there even if you run into them unless you know the amount of effort that went into setting this up.

RavenDB 4.0 Release Candidate 2 is out

time to read 5 min | 867 words

image_thumbIf has been two months since the first release candidate of RavenDB 4.0 and the team has been hard at work. Looking at the issues resolved in that time frame, there are over 500 of them, and I couldn’t be happier about the result.

RavenDB 4.0 RC2 is out: Get it here (Windows, Linux, OSX, Raspberry PI, Docker).

When we were going through the list of issues for this released, I noticed something really encouraging. The vast majority of them were things that would never make it into the release highlights. These are the kind of issues that are all about spit and polish. Anything from giving better error messages to improving the first few minutes of your setup and installation to just getting things done. This is a really good thing at this stage in the release cycle.  We are done with features and big ticket stuff. Now it is time to finishing grinding through all the myriads of details and small fixes that make a product really shine.

That said, there are still a bunch of really cool stuff that were cooking for a long time and that we could only now really call complete. This list includes:

  • Authentication and authorization – the foundation for that was laid a long time ago, with X509 client certificates used for authenticating clients against RavenDB 4.0 servers. The past few months had us building the user interface to manage these certificates, define permissions and access across the cluster.
  • Facet and MoreLikeThis queries – this is a feature that was available in RavenDB for quite some time and is now available as a integral part of the RavenDB Query Language. I’m going to have separate posts to discuss these, but they are pretty cool, albeit specialized ways to look at your data.
  • RQL improvements – we made RQL a lot smarter, allowing more complex queries and projections. Spatial support has been improved and is now much easier to work with and reason about using just raw RQL queries.
  • Server dashboardallows you to see exactly what your servers are doing and is meant to be something that the ops team can just hang on the wall and stare at in amazement realizing how much the database can do.
  • Operations – the operations team generally has a lot of new things to look at in this release. SNMP monitoring is back, and significant amount of work was spent on errors. That is, making sure that an admin will have clear and easy to understand errors and a path to fix them. Traffic monitoring and live tracing of logs is also available directly in the studio now. CSV import / export is also available in the studio, as well as Excel integration for the business people. Automatic backup processes are also available now for scheduled backups for both local and cloud targets and an admin has more options to control the database. This include compaction of databases after large deletes to restore space to the sytem.
  • Patching, querying and expiring UI  – this was mostly exposing existing functionality and improving the amount of details that we provide by default. Allowing users to define auto expiration policy for documents with time to live. On the querying side, we are showing a lot more information. My favorite feature there is that the studio can now show the result of including documents, which allow to easily show how this feature can save you in network roundtrips.  Queries & patching now has much much nicer UI and also support some really cool intellisense.
  • Performance – most of the performance work was already done, but we were able to identify some bottlenecks on the client side and reduce the amount of work it takes to save data to the database significantly. This especially affects bulk inert operations, but the effect is actually wide spread enough to impact most of the client operations.
  • Advanced Linq support – a lot of work has been put into the Linq provider (again) to enable more advanced scenarios and more complex queries.
  • ETL Processes -  are now exposed and allow you to define both RavenDB and SQL databases as target for automatic ETL from a RavenDB instance.
  • Cluster wide atomic operations – dubbed cmpxchng after the similar assembly instruction, this basic building block allow to build very complex distributed behaviors in a distributed environment without any hassle, relying on RavenDB consensus to verify that such operations are truly atomics.
  • Identity support – identities are now fully supported in the client and operate as a cluster wide operation. This means that you can rely on them being unique cluster wide.

Users provided really valuable feedback, finding a lot of pitfalls and stuff that didn’t make sense or flow properly. And that was a lot of help in reducing friction and getting things flowing smoothly.

There is another major feature that we worked on during this time, the setup process. And it may sound silly, but this is probably the one that I’m most excited about in this release. Excited enough that I’ll have a whole separate post for it, coming soon.

The best features are the ones you never knew were thereProtocol fix-ups

time to read 4 min | 755 words

imageRavenDB uses HTTP for most of its communication. It can be used in unsecured mode, using HTTP or in secured mode, using HTTPS. So far, this is pretty standard. Let us look at a couple of URLs:

  • http://github.com
  • https://github.com

If you try to go to github using HTTP, it will redirect you to the HTTPS site. It is very easy to do, because the URLs above are actually:

  • http://github.com:80
  • https://github.com:443

In other words, by default when you are using HTTP, you’ll use port 80, while HTTPS will default to port 443. This means that the server in port 80 can just read the response and redirect you immediately to the HTTPS endpoint.

RavenDB, however, it usually used in environments where you will explicitly specify a port. So the URL would look something like this:

  • http://a.orders.raven.local:8080
  • https://a.orders.raven.local:8080

It is very common for our users to start running with port 8080 in an unsecured mode, then later move to a secure mode with HTTPS but retain the same port. That can lead to some complications. For example, here is what happens in a similar situation if I’m trying to connect to an HTTPS endpoint using HTTP or vice versa.

image

image

This means that a common scenario (running on a non native port and using the wrong protocol) will lead to a nasty error. We call this a nasty error because the user has no real way to figure out what the issue is from the error. In many cases, this will trigger an escalation to the network admin or support ticket. This is the kind of issue that I hate, it is plainly obvious, but it is so hard to figure out and then you feel stupid for not realizing this upfront.

Let us see how we can resolve such an issue. I already gave some hints on how to do it earlier, but the technique in that  post wasn’t suitable for production use in our codebase. In particular, we introduced another Stream wrapping instance and another allocation that would affect all input / output calls over the network. We would really want to avoid that.

So we cheat (but we do that a lot, so this is fine). Kestrel allow us to define connection adapters, which give us a hook very early in the process to how the TCP connection is managed. However, that lead to another problem. We want to sniff the first byte of the raw TCP request, but Stream doesn’t provide a way to Peek at a byte, any such attempt will consume it, which will result in the same problem on an additional indirection that we wanted to avoid.

Therefor, we decided to take advantage of the way Kestrel is handling things. It is buffering data in memory and if you dig a bit you can access that in some very useful ways. Here is how we are able to sniff HTTP vs. HTTPS:

The key here is that we use a bit of reflection emit magic to get the inner IPipeReader instance from Kestrel. We have to do it this way because that value isn’t exposed externally. Once we do have the pipe reader instance, we borrow the already read buffer and inspect it, if the first character is a capital character (G from GET, P from PUT, etc), this is an HTTP connection (SSL connection’s first byte is either 22 or greater than 127, so there is no overlap). We then return the buffer to the stream and carry on, Kestrel will parse the request normally, but another portion in the pipeline will get the wrong protocol message and throw that to the user. And obviously we’ll skip doing the SSL negotiation.

This is important, because the client is speaking HTTP, and we can’t magically upgrade it to HTTPS without causing errors such as the one above. We need to speak the same protocol as the client expect.

With this code, trying to use the wrong protocol give us this error:

image

Now, if you are not reading the error message that might still mean a support call, but it should be resolved as soon as someone actually read the error message.

Random perf results that make me happy

time to read 2 min | 212 words

Michael Yarichuk is one of the core developers of RavenDB. He is going to do a talk and a workshop on Oredev this week. And I just got his latest slides for review.

His talk is about how you can reduce your GC load and improve performance and it includes the following slide:

image

On the left you have RavenDB 4.0 and on the right RavenDB 3.5 running the same load under a profiler. Leaving aside that RavenDB 4.0 is much faster overall, look at the numbers. The 3.5 version spent a lot of time in GC, and a lot of that was blocking GC calls. The 4.0 version barely did any GC, and all of that was in the background.

This scenario wasn’t part of any performance work, it was to show the result of about two years of work and it is amazing to look back and understand that we can see a concrete example of the results so clearly.

Michael will be talking about some of the techniques we use to get there, so I highly recommend you come to his talk. He’ll also be doing a full day workshop on modeling data with documents.

RavenDB 4.0 book update is available

time to read 2 min | 388 words

imageA new update to the Inside RavenDB book is available. I’m up to chapter 9 (although Chapter 8 is just a skeleton). You can read it here.

In particular, the details about running RavenDB in a cluster and the distributed technologies and approaches it uses are now fully covered. I still have to get back to discussing ETL strategies, but there are two full chapters discussing how RavenDB clusters and replication work in detail. I would dearly appreciate any feedback on that part.

This is a complex topic, and I want to get additional eyes on this to make sure sure that it is understandable. Especially if you are new to distributed system design and how they work.

Another major advantage that we now have a professional editor go through chapter 1 – 7, so the usage of the English language probably leveled up at least twice. Errors, awkward phrasing and outright mistakes remains my own, and I would love to hear about any issues you find.

Also new in this drop is a full chapter talking about how to query RavenDB and dive into the new RQL language. There is still a lot to cover about indexes, and this chapter hasn’t been edited yet, but I think that this should give a good insight into how we are actually doing things and what you can do with the new query language.

In addition to that, we are ramping up documentation work as we start closing things down to the actual final release. We are currently aiming that at the end of the year, so it is right around the corner. I also would like to remind people that we are currently giving 30% discount for purchase of RavenDB licenses, for the duration of the Release Candidate. This offer will go away after the RTM release.

Another source of confusion seems to be the community license. I wanted to clarify that you can absolutely use the community license for production usage, including using features such as high availability and running in a cluster.

So grab a license, or just grab the bits and run with them. But most importantly, grab the book (https://github.com/ravendb/book/releases) and let me know what you think.

RavenDB 4.0 Unsung Heroesautomatic conflict resolution

time to read 6 min | 1012 words

RavenDB is a distributed database, it has been a distributed database since pretty much the very start, although over time we have been making the distribution part easier and easier. You might be able to tell that the design of RavenDB was heavily influenced by the Dynamo paper and RavenDB implements a multi master system that allow every node to accept writes and disseminate them across the network.

This is great, because it ensure that we have a high stability in the face of error, but this also opens us up to some interesting failure modes. In particular, if a document is modified in two nodes at the same time, there is no way to immediately detect that. Unlike a single master system, where such a thing would be detected, but requires at least a majority of the nodes to be up. A scenario where we have concurrent modifications on the same document on different server is called a conflict, and is something that RavenDB is quite able to detect and handle.

For a very long time, we had multiple mechanism to handle such conflicts. You could specify that RavenDB would resolve them automatically, in favor of a particular node, or using the latest or specifying a resolution strategy on the server or the client.  But by default, if you did nothing, a conflict would cause an exception and require you to resolve it.

No one ever handled that exception, and very few users set the conflict resolution or did something meaningful with it. We typically heard about it as support calls about “this document is not accessible and the sky has just fallen”. Which is perfectly understandable from the point of view of the user, but incredibly frustrating from ours. Here we are, careful to account for correctness in behavior in a distributed system, properly detecting conflicts and brining them up to the attention of the user and the result is… they just want the error to go away.

In the vast majority of the cases, the user didn’t care about the conflict at all. It wasn’t important and any version would do. And that is after we went to all the trouble of making sure that you have a powerful conflict resolution option and allow you to do some really fun things. The overwhelming response we got was “make this go away”. The problem is that we can’t really make such a thing go away, this is a fundamentally an issue a multi master distributed system must handle. And just throwing one of the conflicted versions under the bus didn’t sit right with us.

RavenDB is an ACID database because I strongly believe that transactions matters, that you data is important and should be respected, not shredded to pieces on a moments notice in fear of someone figuring out that there has been a conflict. I wrote about another aspect of this issue previously what the user expects and the right things are decidedly at odds here. In particular because the right thing (handling conflicts) can be hard for the user, and something that you would typically do only on some parts of your domain model.

Because of this,  with RavenDB 4.0 we moved to automatic conflict resolution. Unless configured outside, whenever RavenDB discover a conflict, it will automatically resolve it (in an arbitrary but consistent manner across the cluster). Here is what this looks like:

image

Notice the flags? This document is the resolve of conflict resolution. In this case, we had both 1st and 2nd as conflicting versions, and we chose one of them.

But didn’t I just finished telling you that RavenDB doesn’t shred your data? The key here is that in addition to the Resolved flag, we also have the HasRevisions flag. In this case, the database doesn’t have revisions defined, but even so, we have revisions for this document. Let us look at them, shall we?

image

We have three of them:

Created on Node A Created on Node B

Resolved

image image image

Pay special attention to the flags. You can see that we have here three revisions. The conflicted versions as well as the resolved document. We’ll be reporting these documents in the studio, so an admin can go and take a look and verify that nothing was missed and this also applies to conflict resolution that wasn’t done by arbitrarily choosing a winner.

Remember, this is the default configuration, so you can set RavenDB to manual mode, in which case you’ll get an error on access a conflict and will need to resolve it manually, or you can define a script that would resolve the conflict. This can be defined on a per collection basis or globally for the database.

Here is an example of how you can handle conflicts using a script:

Regardless of the way you chose to resolve the conflict, you will still have all the conflicting versions available after the resolution, so if your script missed something, no data has been lost.

The idea is that we want to enable you to deploy a distributed system without too much hassle, but without taking undue risks or running in a configuration that is only suitable for demos. I think that this is the kind of features that you would never really notice, until you really notice that it just saved you a bunch of sleepless nights.

And as this is written at 00:12 AM, I think that I’ll listen to my own advice, hit the post button and head to bed.

RavenDB 4.0The Server Dashboard

time to read 1 min | 184 words

One of the things that we have been saving is the hooking together of all the work we have ben doing to expose how RavenDB works into the operations dashboard. This has just landed in the nightly and can give you a lot of insight into exactly what is going on inside your server.

You can see some of the screenshots below. The idea is that in addition to exposing all of these metrics over dedicated endpoints and SNMP, we will also save users the trouble of setting up monitoring and just show them what is going on directly.

Operators can just head to this page and see what is going on, and it is meant to be put as a background for users to observe this during routine operations.

image

image

image

The right thing and what the user expect to happen are completely unrelated

time to read 4 min | 783 words

“In theory, there is no difference between theory and the real world.”

One of the more annoying things to learn was that the kind of things that you worry about from inside the product are almost never the kind of things that your users worry about. Case in point, we spend an amazing amount of time making sure that RavenDB is crash proof, that you will not have data corruption and that transactions are atomic and durable in the face of what is sometimes horribly broken environments.  Users often just assume “this must work” and move along, having no idea how hard some of these things are.

But that much, I get. It make sense that you would just assume that things should work this way. In fact, one of the reason that RavenDB exists is that none of the NoSQL products at the time didn’t provide what I considered to be basic functionality. Since then I learned that what a user consider basic functionality and what a database consider basic functionality are two very distinct things.

But I think that the most shocking thing was that users tend to not care about data consistency anyway near the level you would expect them to. We spend time and effort and a whole lot of coding to ensure that it would be possible to reason about the behavior of a distributed and concurrent system in an fairly predictable manner, that data is never lost or misplaced, and no one notices. What is worse, when you get things right, and another database engine gets it clearly wrong, users will sometimes want to use the other guy (wrong) implementation, because doing the clearly wrong thing is easier for them.

For example, consider the case of two concurrent modifications to the same document. If you do nothing, you’ll get a Last Write Wins scenario. You can also do the proper thing and error when the second write comes, because it is based of a now out of date version of the document. A few weeks ago I got a frantic call from one of the marketing & sales people about “I broke our database” and “found a major issue”. That was quite strange, given that the person talking to me wasn’t a developer, instead, she was using one of our internal systems to update a customer purchase and got an error. She then proceeded to figure out that she could reproduce this error at will. All she had to do was edit the same customer record at the same time as a colleague was also editing it. Whoever saved the record first would work, and the second would get an error.

For the developers among you, that is Optimistic Concurrency in action, absolutely expected and what we want in this scenario. But I had to give a full explanation of how this is not a bug, tell the marketing guys to put down the “Serious Bug Fixed, Upgrade Immediately” email template down and that this is how it is meant to work. The problem, by the way, wasn’t that they couldn’t understand the issue. They just couldn’t figure out why they got an error in the first place, surely the “system” was supposed to figure out what to do there and not given them an error.

I’ll freely admit that we skimp on the UX of our internal systems because… well, they are internal, and it is easier to train a few dozen people on how the systems work than to train the systems how people work at that scale. But this really hit home because even after I explained the issue, asked them what they expected to happen and how this is supposed to work, I couldn’t get through. An error shown to them is obviously something that is wrong in the system. And being able to generate an error by their own actions means that the system is broken.

It took showing the same exact behavior in the accounting software (made by an external company) before they were half convinced that this isn’t actually an issue.

Now, to be fair, our marketing people aren’t technical, so they aren’t expected to understand concurrency and handling thereof, and typically any error by the internal system means that something is broken in the infrastructure level so I can absolutely understand where they are coming from.

The sad thing is, this isn’t isolated to non technical people and we have to be careful to design things in such a manner that they match what the user expect. And user in this case is the developers working with RavenDB and the ops teams responsible for its care and feeding. I’ll talk about one such decision in the next post.

FUTURE POSTS

  1. The best features are the ones you never knew were there: You can’t do everything - 3 days from now
  2. You are doing it REALLY wrong, the shortest code review ever - 4 days from now
  3. Carefully performing invalid operations to get the wrong error and the right result - 5 days from now
  4. If you have a finalizer, watch your ctor - 6 days from now
  5. Production postmortem: The random high CPU - 7 days from now

And 7 more posts are pending...

There are posts all the way to Dec 12, 2017

RECENT SERIES

  1. PR Review (9):
    08 Nov 2017 - Encapsulation stops at the assembly boundary
  2. API Design (9):
    27 Jul 2016 - robust error handling and recovery
  3. Production postmortem (21):
    07 Aug 2017 - 30% boost with a single line change
  4. The best features are the ones you never knew were there (5):
    21 Nov 2017 - Unsecured SSL/TLS
  5. RavenDB Setup (2):
    23 Nov 2017 - How the automatic setup works
View all series

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats