Open Source Success Metrics
I got into a very interesting discussion with Rob Conery recently, talking about, among other things, the CodePlex Foundation. You can hear it TekPub’s Podcast #2, when it is out later this week. Tangential to our discussion, but very important, is how an Open Source project owner defines success for their project.
Usually, when people try to judge if an open source project is successful or not, they look at the number of users, and whatever the general response for the project is positive. But I am involved in a lot of open source projects, so you might say that I have an insider’s view on how the project owner view this.
Now, let us be clear, I am talking about OSS project that are being led by an individual or a group of individuals. There are different semantics for OSS projects that are being led by a company.
There are several reasons for individuals to be involved in OSS projects, those are usually:
- Scratch an itch – I want to do something challenging/fun.
- Need it for myself – This is something that the owner started because they needed it themselves and open sourced it for their own reasons.
- Reputation building – Creating this project would give me some street cred.
- I use it at work – Usually applicable for people joining an existing project, they use it at work and contribute stuff that they require.
We must also make a distinction between OSS adoption and OSS contribution. Typically, maybe a tenth of a percent of the adopters would also be contributors. Now, let us talk about the owner’s success metric again. Unless the owner started the project for getting reputation, the number of people adopting your project is pretty meaningless. I think that DHH , the creator of Ruby on Rails, did a great job describing the owner sentiment:
I'm not in this world to create Rails for you. I'm in this world to create Rails for me and if you happen to like that version of Rails that I'm creating for me, than you are going to have a great time.
Even if you are interested in OSS for reputation building, after a while, it stops being a factor. Either you got your reputation, or you never will. A good example of a project started to gain reputation is Rhino Mocks. And you know what, it worked. But I think that it is safe to say that I am no longer just that Rhino Mocks guy, so the same rules applies as for the other motivations.
So, what does it mean, from an owner perspective, when you ask if an OSS project is successful or not? The answer is simple: it does what I need it to do.
I asked a popular OSS project owner the following question: What would your response be if I told you that I am taking a full page ad in the Times magazine proclaiming your stuff to be the best thing since sliced bread? You’ll get a million users out that that.
His response was: Bleh, no!
Tying it back to the discussion that I had with Rob, I feel that much of the confusion with regards to the CodePlex Foundation role is when you try to talk to projects led by individuals as if they were commercial enterprises. The goals are just completely different, and in many cases, adding more users for the project will actually be bad for the project, because it put a higher support cost for the project ream.
In the .NET ecosystem, most of the projects aren’t being led by a company. They are led by individuals. That is an important distinction, and understanding it would probably make it clear why the most common response for the CodePlex Foundation was: What is in it for me?