Ayende @ Rahien

Refunds available at head office

The paradox of choice: best of breed or cheapest of the bunch

Roy Osherove has a few tweets about commercial tools vs. free ones in the .NET space. I’ll let his tweets serve as the background story for this post:

image

image

The backdrop is that Roy seems to be frustrated with the lack of adoption of what he considers to be better tools if there are free tools that deal with the same problem even if they are inferior to the commercial tools. The example that he uses is Final Builder vs. NAnt/Rake.

As someone who is writing both commercial and free tools, I am obviously very interested in both sides of the argument. I am going to accept, for the purpose of the argument, that the commercial tool X does more than the free tool Y who deals with the same problem. Now, let us see what the motivations are for picking either one of those.

With a free tool, you can (usually) download it and start playing around with it immediately. With commercial products, you need to pay (usually after the trail is over), which means that in most companies, you need to justify yourself to someone, get approval, and generally deal with things that you would rather not do. In other words, the barrier for entry is significantly higher for commercial products. I actually did the math a while ago, and the conclusion was that good commercial products usually pay for themselves in a short amount of time.

But, when you have a free tool in the same space, the question becomes more complex. Roy seems to think that if the commercial product does more than the free one, you should prefer it. My approach is slightly different. I think that if the commercial product solves a pain point or remove friction that you encounter with the free product, you should get it.

Let us go back to Final Builder vs. NAnt. Let us say that it is going to take me 2 hours to setup a build using Final Builder and 8 hours to setup the same build using NAnt. It seems obvious that Final Builder is the better choice, right? But if I have to spend 4 hours to justify buying Final Builder, the numbers are drastically different. And that is a conservative estimate.

Worse, let us say that I am an open minded guy that have used NAnt in the past. I know that it would take ~8 hours to setup the build using NAnt, and I am pretty sure that I can find a better tool to do the work. However, doing a proper evaluation of all the build tools out there is going to take three weeks. Can I really justify that to my client?

As the author of a commercial product, it is my duty to make sure that people are aware that I am going to fix their pain points. If I have a product that is significantly better than a free product, but isn’t significantly better at reducing pain, I am not going to succeed. The target in the product design (and later in the product marketing) is to identify and resolve pain points for the user.

Another point that I want to bring up is the importance of professional networks to bring information to us. No one can really keep track on all the things that are going on in the industry, and I have come to rely more & more on the opinions of the people in my social network to evaluate and consider alternatives in areas that aren’t offering acute pain. That allows me to be on top of things and learn what is going on at an “executive brief” level. That allows me to concentrate on the things that are acute to me, knowing the other people running into other problems will explore other areas and bring their results to my attention.

Comments

Frans Bouma
01/24/2010 11:43 AM by
Frans Bouma

It's not that hard. A person chooses product P (be it free or commercial) because P has USPs the person is looking for. USPs are 'Unique Selling Points'. USPs are the aspects of a product which appeal to groups of people. You name them 'solutions to pain points', but it can be anything.

To be successful with any product, free or commercial, it's key the product has enough USPs to be used by a group of people that's large enough so the product gets momentum, gets more followers (as often people simply pick what others use) and in the end becomes one of the big players. Make no mistake, a USP can be anything, from being first on the market, having great support, have a great shiny UI / website, be free, have pink buttons or a ribbon control...

So the difficulty is in what are for product of type T the USPs to have? That depends on which group(s) you are targeting. What benefits are they looking for? If they are looking for a shiny UI, releasing something that requires text files and a command line isn't going to work and vice versa.

That's also the reason that despite free solid alternatives, there is still room for commercial products of a given type T, simply because they appeal to some groups of people through their USPs.

It's easy to see with this why some commercial products are chosen OVER free alternatives: the commercial product vendors have to research what USPs they have to provide to sell licenses, and are willing to implement these. Free software developers don't do this (why should they), they very often develop what they themselves need for their own, and if someone wants a feature they think is not useful, that feature won't make it. So the free software will appeal to the people in the same group as the free software product developer is in. It's to be seen if that group is big enough.

An ISV always has to think about what groups will be using the software and what USPs these people are looking for. With NHibernate development, you'll likely will measure what's included against what you yourself think is useful. What's included in your commercial products is likely measured against what your customers think is useful, even if YOU think it's a silly stupid feature. Because, those customers are the ones who make your product successful, and if that product is a free toolkit, who cares if it's successful but if it's a commercial toolkit, it's ESSENTIAL it's successful, otherwise the ISV goes belly up.

For ISVs it's harder if the free competition is developed by people who happen to be thinking like the vast majority of the potential customers: that will lead to free software which is appealing to many of the customers out there, leaving a small(er) group of people for the ISVs.

Ayende Rahien
01/24/2010 11:56 AM by
Ayende Rahien

Frans,

re: The part about ISV and silly requests.

Absolutely, LOL

BjartN
01/24/2010 12:20 PM by
BjartN

This is the reason why commercial products should offer the same features as the free ones for free, and take payment for the extra features that makes the commercial product superior.

I think this model will get you more users, and also, if your product is superior, you will get paid as well.

mattmc3
01/24/2010 12:48 PM by
mattmc3

Also, any additional compelling features that the commercial product offers must be challenging enough to implement or orthogonal to either the architecture or the goals of the maintainers of the free product. Otherwise, rev-next of the free product will have implemented those features (assuming of course that the free product devs listen to their userbase and accurately report their product time line - neither of which is a given with free software).

There are a lot of other principals at work here too, including good-is-the-enemy-of-great, and developers-will-spend-more-on-coffee-than-on-tools.

Mikael Henrikson
01/24/2010 02:22 PM by
Mikael Henrikson

I could not agree more about this post. It is commonly known as "The reason developers should not sell their own shit". I have never heard of Final Builder and I've been doing this developing business for a few years now :)

Andrey Shchekin
01/24/2010 02:28 PM by
Andrey Shchekin

It depends on the product niche. For person-level tools (profilers, IDEs, things like TestDriven.NET) or somewhat specialized tools (NCover) I would recommend commercial version over free one if commercial seems better.

However, for basic tools (build/testing/mocking/etc) I would always take a free one. Just because any extensions/patches/skills/blog posts related to it will be useful in much more occasions.

Also, the paid frameworks are closed-source, which means that anything that can not be done, can not be done. Of course, it is possible to request assistance from vendor, but if the limitation is not trivial, fix might not come fast enough/come at all.

For example, we used Telerik web controls, and they had a lot of limitations some of which were advertised and some of which were unexpected. In the end, we ended with almost more customization code than there were code in the controls themselves. And we couldn't even commit it anywhere/share it with anyone since it was so specific to Telerik controls.

Mark
01/24/2010 03:34 PM by
Mark

@ BjartN: That's a good point regarding free versions of commercial products. Sometimes you have to prove that you're right about a particular product, and that, as Ayende said, it will remove pain points.

One of our teams switched our CI to TeamCity and re-did our build process (the old build scripts were an unmaintainable mess). The end result was that the entire project switched over to the new Team City setup, and we've now bought extra licenses.

The only reason the team was able to get the new (vastly better) build system in place was because TeamCity had a free version available. If they'd said "hey, give us some licenses to replace something we already have that is free to do some work that probably appears unimportant to non-programmers", the answer may well have been "no". Fortunately, JetBrains offered us an easy way in.

Michael Valenty
01/24/2010 05:15 PM by
Michael Valenty

I agree with @BjartN in that " commercial products should offer the same features as the free ones for free, and take payment for the extra features." I think this is an excellent model.

TeamCity offers a certain number of projects for free and you have pay to use it in larger teams. This means developers will use it at home on side projects and pitch it at work. We are using TFS and most of us hate it. I have been raving about TeamCity for a while because I've used it before, you can't underestimate how powerful it is to speak from experience rather than a list of bullet points on a website.

I waited to install R# until I could get my employer to pay for it. Then we hired a guy that was using it and the rest of the team installed it within a week. I'm sure NHProf would be the same. A peer speaking from experience is more powerful than reading a list of features on a website.

This is a very interesting topic!

Ayende Rahien
01/24/2010 05:20 PM by
Ayende Rahien

Michael,

How does trials play into this?

Sergey Mirvoda
01/24/2010 05:43 PM by
Sergey Mirvoda

@Ayende

Trial requires TIME even with tools that comes with CountOfProgramRunsLicense. You may run program to check out some features and at the same time some crazy BA file you a new completely stupid requirement.

Michael Valenty
01/24/2010 06:15 PM by
Michael Valenty

I think if a version of NHProf was free maybe just the formatted sql - everybody would use it. People answering questions on nhusers, stackoverflow etc.. would just naturally assume you are using it too.

This is a tough problem and I don't fully understand the psychology - even my own :) I'm sure part of it is just being cheap, and another part of it is investing effort learning something that can't be shared with a large community.

With that said, I have been pushing NHProf to my employer because it does look so compelling from the screenshots and bullet points. They are just insanely slow with purchasing.

Greg Law
01/24/2010 07:16 PM by
Greg Law

I tend to agree, but I think you also need to look at licensing models of commercial vs. free projects. Aside from the obvious non-commercial use licenses, tools or components that require activation or a periodic "phone home" to validate licenses are far less desirable. If the source-code license is fairly reasonable or it's a fairly reputable company that's likely to be around in the next decade, I'll deduct points but still consider it is as a viable option; otherwise, I knock it out of the competition johnny on the spot. This may be harsh, but I've already been bitten by a set of components I can't activate because the company is defunct.

I often switch between my desktop while in the office and my notebook while on the road. If the tools or components are locked to a computer, it's probably going to be knocked out of the competition unless it is significantly superior and justifiable to the alternatives.

If the components require a run-time license, buh-bye. It was nice knowing ya. I'll consider a one-time server license for some stuff, but I don't have the time nor the patience to track, file, report, audit, submit purchase order requests, and justify every month run-time distributions.

Is your trial license sufficiently flexible to allow creating complex testing scenarios? Stuff like NHProf (great product!) is relatively trivial to test and is worth its weight in gold. Stuff like large Win/Web component packages are much more complex to test thoroughly and we won't know if they are really usable without mocking up real apps.

Migrating from a free tool to a paid tool can involve a significant amount of friction because it requires a significant justification in the minds of the developers and (much more important) the holders of the magic paychecks.

Will Hughes
01/24/2010 11:52 PM by
Will Hughes

Where I work - we tend to go with whatever works.

If that's a bit clunky, and takes some finessing -- that's fine.

If someone can show a better, smarter, easier way - and it integrates nicely into our existing toolset, then sure - we'll go out and pick up licences for it.

We picked SVN over TFS mostly because it has a better toolset, and wider compatability.

Our Flex and Java folks like using OSX, so integrating TFS into their project cycle would be difficult. (Yes, TFS-SVN Bridge is out now, thanks, it wasn't when we moved to SVN)

We picked up a bunch of Redgate and Jetbrains tools, because they work better than any free (or cheap) alternatives.

We picked NAnt over MSBuild, Maven, and FinalBuilder because NAnt has wider compatability than MSBuild and Maven, and we've never heard of FinalBuilder.

It also helps that NAnt is relatively simple to understand, even if a clunky XML format.

We picked JIRA over the multitude of opensource bug tracking systems, because although kinda clunky - it just works, and it integrates nicely with the rest of our workflow.

So, yes, there's use of free and non-free tools - usually because they work better in our environment.

It also helps quite significantly to get the word out about your commericial product. There's no use bitching that people don't use your product, if people havn't heard of it.

Josh
01/25/2010 04:26 AM by
Josh

Cost/value of labor is also important - a junior developer in India making $4/hour will have a tough time justifying $200 on a development tool, while many U.S. developers making 10X that might just buy them on their own if their employers nix the idea.

At my company, we supply a standard set of development tools to our developers ... anything else they either have to justify the expense or buy it on their own. Most of the developers have at least one tool that they purchased on their own.

Michael J. Ryan
01/25/2010 04:34 AM by
Michael J. Ryan

I tend to prefer Open-Source over Free over Commercial. It's rather arbitrary, but I'll usually want something to be say 25% better than it's counterpart to justify it personally. Especially with open-source. If there's a source license for a commercial product, it's usually much more than a developer license so that's a consideration as well.

I've come across too many pain points with software that I was able to fix myself with access to the source to not want the source included with what I am working on. I tend to prefer LGPL, BSD or MIT licenses for libraries, depending on the needs of a given project. I try to give back as well.

I'm not as religious as some tend to be about things, more pragmatic. But having the source code has value in developer tools. Often more than a better closed source tool is worth to me.

Ayende Rahien
01/25/2010 07:35 AM by
Ayende Rahien

Michael,

I certainly agree with you here. The problem, from my perspective, is that the # of people buying it would likely drop significantly.

Ayende Rahien
01/25/2010 07:35 AM by
Ayende Rahien

Sergey,

I am not sure that I am following your logic here.

Sergey Mirvoda
01/25/2010 08:09 AM by
Sergey Mirvoda

@Ayende

Here is my testing experience of dotTrace.

I downloaded trial, but when I found a free time for testing our QA also found bug in our system and it takes me two weeks to fix bug and two weeks to catch up project. trial expired.

I bought new PC installed Win7 and downloaded trial of dotTrace and guess what. Our president sold a product that does not exist:) And we must deploy it to customer in 4 weeks.

IMHO trial product should not depends on TIME. It should depends on features or feature completeness.

Frans Bouma
01/25/2010 09:57 AM by
Frans Bouma

"This is the reason why commercial products should offer the same features as the free ones for free, and take payment for the extra features that makes the commercial product superior.

I think this model will get you more users, and also, if your product is superior, you will get paid as well. "

No offence BjartN, but you sound like a consultant. And many in this thread do sound like consultants or people who don't sell their own software tools for a living.

The thing is: why on earth would an ISV give away their work for free? Do you give your work / time away for free? Like, 40 hours a week, 9-5 workdays for free? No you don't. You have bills to pay.

If someone writes some code in his/her spare time and gives that away, no problem. However, every hour spend during workhours is a billable hour and if one spends months writing software that's given away for free, how are you going to justify the lack of income? Like in: 0.0 income at all?

ISVs can't give away software 'just because open source is free too'. ISVs only give away freebees as part of marketing / PR as every license given away is loss of income but at the same time you might get good press, a person who will tell their colleagues the software is great etc.

Hoping that someone will buy the 'extra' features is only going to work if the 'extra' features appeal to a large group of people (see my first post in this thread).

@Sergey: crippled trial ware doesn't work. The reason is simple: a person wants to try out what the product can do. If the product lacks a set of features because it's a demo, it therefore isn't testable as if it is used in production and will not sell. Almost no ISV uses crippleware protection these days anymore.

Frans Bouma
01/25/2010 09:58 AM by
Frans Bouma

(part 2. #$#$@ blogengine)

@Michael: "I think if a version of NHProf was free maybe just the formatted sql - everybody would use it. People answering questions on nhusers, stackoverflow etc.. would just naturally assume you are using it too. This is a tough problem and I don't fully understand the psychology - even my own :) I'm sure part of it is just being cheap, and another part of it is investing effort learning something that can't be shared with a large community."

You 'think' something will be used everywhere, yet you don't know a thing about why that is (you admit that yourself). This is precisely the kind of untrue assumptions why this debate is even necessary. For decades companies sell software and for decades software is also released as free (before 'open source' there was public domain and even shareware. Shareware already proved your assumption is wrong, as most people never payed a fee). You'd think that with so many companies selling software, it would quickly be common to have a free version if that would boost sales drastically, don't you think? Why doesn't every do that?

It might be hard to understand for the people who have a day-job and aren't directly depending for their income on how many licenses they sell each month, and I fully understand that (as I didn't know either when I started a company 13 years ago), but please: try to grasp the simple basic knowledge behind why people buy X and not Y. I tried to explain it above in my first post (which is overly simplified as the subject is way broader than this). 'Buy' is equal to 'use' here as one can't buy open source software.

For code monkeys who like to type for 10 hours straight because they have the illusion that by typing everything in they're more productive, a visual toolkit will never be appealing and a free xml text file based system is. Does that make one product more superior over the other? no not at all, it just appeals to them, but not to others. Other people like visual tools as they have the illusion they have more overview over what they're doing than with textfiles.

What often happens however is that a small group of loudmouths promote one side over the other on the internet, suggesting the other way of doing things is really unprodictive, and the alternative (their favorite) is great.

Which makes this debate rather pointless.

Hadi Hariri
01/25/2010 12:26 PM by
Hadi Hariri

@Ayende,

"But if I have to spend 4 hours to justify buying Final Builder, the numbers are drastically different. And that is a conservative estimate."

This holds true if it were a one-off thing, and if that were the case, yes it makes perfect sense. However, tools such as Finalbuilder, who's primary purpose is to automate tasks is not something you would use only once.

".... ~8 hours to setup the build using NAnt, and I am pretty sure that I can find a better tool to do the work. However, doing a proper evaluation of all the build tools out there is going to take three weeks. Can I really justify that to my client?"

No, you probably wouldn't, but this is all part of the learning process, something that we can't always charge, yet still have to do, to help us improve our skills and ultimately help our customers.

Hadi Hariri
01/25/2010 12:50 PM by
Hadi Hariri

@Sergey

What would you suggest that dotTrace or something like NHProfiler then be limited by?

Both tools are profilers. The main goal for you, as the person evaluating it is to see if it suits your needs. That has to be something that is proven to you during the evaluation period.

If they weren't limited somehow in time, why would you purchase them if their main objective was accomplished?

Krzysztof Kozmic
01/25/2010 02:49 PM by
Krzysztof Kozmic

Ayende, (and Hadi)

The problem here, is that you greatly underestimate the overhead, that a company can impose on a purchase of commercial tool.

Let me give you a real example.

One team at a company wants to purchase a tool for code profiling. Turns out a CEO needs to approve it, and it takes few days to even notify him about the idea.

After that, CEO requests an evaluation of the tool from that team, to justify why it's worth it.

In the meantime it turns out that another team working in different environment (C++ vs .NET) is using another tool from an other vendor, which also provides version for .NET.

So the team is asked to evaluate that other tool since it's "company standard" for C++ development and hence preferred for .NET as well.

After some more time turns out CEO asked some other teams if they would need a purchase of the profiler made for them as well, Turned out they did, and each of them had their own preferred tool.

As a result of that few meetings were held to discuss which tool should be purchased and become "company standard" for .NET. This resulted in some more trials and some more lobbying because every team wanted the tools they liked to be purchased.

To make the long story short - it took 3 months to make the purchase.

Take that and compare to "just download OSS and use it".

Sergey Mirvoda
01/25/2010 05:37 PM by
Sergey Mirvoda

@Frans

What about VS Express Editions, SQL Server etc.

as for ISV, 200$ is not so big price for a tool and I buy it if it prove that it is nice and helpful.

nhProf can perform profiling for free but stack browsing, live reports, CI integration and other "profi" features should be available only for trial time.

Ayende Rahien
01/25/2010 06:17 PM by
Ayende Rahien

Sergey,

One important thing to remember about the Express Editions, they are NOT the profit centers for MS.

From MS perspective, giving developers VS for free would still make sense, because it would mean more people developing on Windows.

Mike Scott
01/25/2010 06:22 PM by
Mike Scott

Krzysztof

And what is the total amount of time that the CEO, programmers and other staff spent over the three months that it took to evaluate and purchase the product?

Then, how much did all this time cost, just using ballpark figures for the hourly rates of each person involved?

What is the ratio of staff cost to product cost?

If the final answer is > 1 (and from the sound of it, it's probably considerably greater than 1!), then the CEO needs to have his arse kicked for wasting company money! That's where the real problem lies.

James Gregory
01/26/2010 02:18 PM by
James Gregory

Mike, Krzystof, I don't think anyone would deny the CEO is at fault, but there are very few people who'd be willing to tell him that (or rather, very few people who'd end up staying in their job after). I've witnessed this exact situation in most companies I've worked for. Even disregarding cost (which can cause a CEO to throw something out even before evaluating it), the time required to get the go-ahead for something can be considerably more than any ramp-up time needed for a free product.

Arne Claassen
01/26/2010 06:01 PM by
Arne Claassen

Totally agree with @sergey regarding trials. There are a lot of programs i've never fully evaluated because i didn't have the time to do it in the time allotted by the trial. And there is other software i've never even installed because i knew i didn't have enough time so i didn't want to start the clock ticking.

Unfortunately, I don't have a good solution for this problem, at least not one that isn't high touch. I would prefer a product that phones home every time i use it during the trial and can report how much i use it and let the vendor determine whether i am evaluating or just using it. And if they thing i'm abusing it, contact me and let me know that the will disable the trial. Still not ideal.

Perforce did this for us when we examined it. We wanted to try it and they provided us a with an open ended license and checked back with us periodically. It did require some trust on their side and willingness to manage a long sales cycle, but I guess they knew that something like perforce you don't use to solve a problem, but either you use it forever or you don't. It worked for them, since we did purchase in the end.

Colin Newell
01/29/2010 10:14 PM by
Colin Newell

I tend to be a price conscious developer and I evaluate it my purchases based on whether I really need the product or not. In the profiling tool field I haven't found anything satisfactory that wasn't commercial so I bought a tool.

I also bought TestDriven.Net because it allows me to run my unit tests from the debugger. The fact that it also happens to overlap/use free tools as well turns out to be a bonus that didn't enter my purchasing decision. It was simply the ease of debugging my tests. I suppose that the 'running unit tests from Visual Studio' might bring in a few developers but honestly I wasn't interested. It turns out to be something I use now.

There are tools that are superior to free ones that I haven't bought, and that's simply because I don't need the unique things that they do. If I find a problem that requires one of those unique solutions I'll probably buy it.

In short I think you need something unique that isn't done by a free tool, or make it a lot easier than the free tools. Then it needs to actually be useful to a lot of people. Some tools do somethings really well but that unique thing is so niche it just isn't going to sell well.

The other thing that makes me use free tools is when I am worried about how many licenses I will need. With a tool like a profiler I can live with just getting 1 to start with and it will be useful. A build tool on the other hand needs at least 1 install per developer. Then we probably want a build server and we might want to run it on laptops too. It quickly develops friction.

Comments have been closed on this topic.