Ayende @ Rahien

It's a girl

Estimates sucks, especially when I pay for them

I recently got an estimate for a feature that I wanted to add to NH Prof. It was for two separate features, actually, but they were closely related.

That estimate was for 32 hours.

And it caused me a great deal of indigestion. The problem was, quite simply, that even granting that there is the usual padding of estimates (which I expect), that timing estimate was off, way off. I knew what would be required for this feature, and it shouldn’t be anywhere near complicated enough to require 4 days of full time work. In fact, I estimated that it would take me a maximum of 6 hours and a probable of 3 hours to get it done.

Now, to be fair, I know the codebase (well, actually that isn’t true, a lot of the code for NH Prof was written by Rob & Christopher, and after a few days of working with them, I stopped looking at the code, there wasn’t any need to do so). And I am well aware that most people consider me to be an above average developer.

I wouldn’t have batted an eye for an estimate of 8 – 14 hours, probably. Part of the reason that I have other people working on the code base is that even though I can do certain things faster, I can only do so many things, after all.

But a time estimate that was 5 – 10 times as large as what I estimated was too annoying. I decided that this feature I am going to do on my own. And I decided that I wanted to do this on the clock.

The result is here:

image

This is actually total time over three separate sittings, but the timing is close enough to what I though it would be.

This includes everything, implementing the feature, unit testing it, wiring it up in the UI, etc.

The only thing remaining is to add the UI works for the other profilers (Entity Framework, Linq to SQL, Hibernate and the upcoming LLBLGen Profiler) . Doing this now…

image

And we are done.

I have more features that I want to implement, but in general, if I pushed those changes out, they would be a new release that customers can use immediately.

Nitpicker corner: No, I am not being ripped off. And no, the people making the estimates aren't incompetent. To be perfectly honest, looking at the work that they did do and the time marked against it, they are good, and they deliver in a reasonable time frame. What I think is going on is that their estimates are highly conservative, because they don't want to get into a bind with "oh, we run into a problem with XYZ and overrun the time for the feature by 150%".

That also lead to a different problem, when you pay by the hour, you really want to have estimates that are more or less reasonably tied to your payments. But estimating with hours has too much granularity to be really effective (a single bug can easily consume hours, totally throwing off estimation, and it doesn't even have to be a complex one.)

Comments

Demis Bellot
09/12/2010 11:39 AM by
Demis Bellot

I don't bother with any work that takes less than 2-3 days myself. The overhead of managing the work, communicating and understanding the requirements, context switching and getting up to speed with the code base etc makes the overhead / coding ratio pretty high and as a techie I'd rather be cutting code.

Bob
09/12/2010 05:07 PM by
Bob

Yeh they will always be just 'estimates' after all but then maybe some other tasks will turn out to take more time than estimated. You can usually mitigate over or under estimating somewhat, by breaking down tasks that appear to be large at first.

Bob
09/12/2010 05:09 PM by
Bob

Also just wondering y you are paying to have estimates done rather than doing them your self? is it a time constraint thing?

Rafal
09/12/2010 05:45 PM by
Rafal

It's normal when you're trying to get someone to do a 3 hour contract for you. Especially when it's a new contractor without a working knowledge of your code. His estimate could be something like 10 hours of coding + several hours for meetings and talking + few hours for acceptance tests and change requests due to specification being incomplete/unclear/misunderstood + some bug fixing = about 20-30 hours of work. For such small projects it's better to batch them in a bigger release and work with someone who already knows the product so the overhead will be smaller.

Richard Dingwall
09/12/2010 10:59 PM by
Richard Dingwall

Ayende: Did they break down the estimate into sub tasks? 32 hours is too big for one task -- that suggests they didn't have much confidence, or the task wasn't clearly defined and weren't sure of the gravity of the changes required. (Where you sound like you had a much better idea what was needed yourself).

Phil Sandler
09/12/2010 11:08 PM by
Phil Sandler

If you can get a reasonable hourly rate with someone you trust and respect, you are much better off doing T&M than fixed bid, as then you don't have to pay for the padding.

However, from my experience in consulting, customers never want T&M, they want to know how much something will cost.

loiveront
09/13/2010 03:33 AM by
loiveront

If you are their customer, they shouldn't share with you any estimations. They should let you know a price/cost and the delivery date, anything else.

Ruben Berenguel
09/13/2010 05:40 AM by
Ruben Berenguel

Maybe they were adding "found two odd bugs, debugging, re-testing" to this time estimate? That would account for the increase in time, but I guess one big bug per feature is enough, and that would be maybe 8-10 hours total estimate.

Ruben

Steve Py
09/13/2010 07:13 AM by
Steve Py

+1 Rafal:

I recently went to get quotes from carpenters to replace an exterior door. (new lock, etc.) Both showed up and said it was a fairly simple job, about 1-2 hours, I even got an hourly rate out of one of them at $45/hr. But when I got the quote they were both around $500 + materials. Obviously their justification was a half-day minimum charge. I'd have been happy enough if they'd said it was a small job, charge me for the 2 hours, but I'd wait until they had an otherwise unfilled opening or finished another job early enough to pop-by. (They were both based within 5 minutes of where I lived.)

I put up the door myself in about 4 hours.

Steve
09/13/2010 03:41 PM by
Steve

Could I be over simplifying things by suggesting that the estimate was 32 hrs because that is what the developers thought you would pay them?

The biggest problem I see in estimating is that people love to work in "day" chunks. Something simple becomes half a day, something a little harder becomes a day, and anything beyond one and a half days only increases in day increments. This is the way most people are wired due to the fact that most companies really only care about when something is due, not how long it took to do the work.

When something takes "a day", I guarantee that a developer doesn't work on it for 8 hrs. In fact, they probably don't even do more than 2 hrs of actual work, the rest is emailing, reading blogs (like this one), going to meetings to talk about something that they already know the answer too, filling out timecards/timeshsets, checking fantasy football, and about another 4 hrs of crap that I can't even put in here since this sentence is already too long.

If something reasonably complex needs to be done by the end of the week and you assign it on a Monday, then miraculously the estimate comes in at 4 days. It's almost a self fulfilling prophesy.

Rafal
09/13/2010 06:12 PM by
Rafal

Steve, a full-time worker gets paid the same amount of money, no matter if there's actual work to be done or not or how effectively he's doing his job. Contractor has no such luxury, he must keep his orders coming in order to get some money. And he must have money also for the time when he's busy running his business and doing all the things that don't count as real work for the customers. Of course he can set his hourly rate higher so that it pays also for the unpaid work, but this would put him instantly out of business because the rate wouldn't be competitive. So there's no choice but to add some overhead to the work estimate, especially when the orders are small and irregular. And I think that's ok and normal (provided that the overhead isn't too excessive).

Steve
09/13/2010 06:32 PM by
Steve

Rafal,

I've been a contractor, so I understand how it works. But It's not Ayende's (or any clients) responsibility to make sure a contractor is bringing in the orders, it's the consultants job.

Apparently this contractor has set 32 hrs as their minimum, which Ayende found too high for his liking. I agree that it's okay for a contractor to add a bit of a premium to their rate (otherwise, why do it?), but in this case it appears to have been excessive for the scope of work. Ayende even said he would have accepted 14 hrs, which is over 400% overhead, so the developer STILL would have made a good ROI, plus they would have gained good will (with the possibility of future contracts as the reward).

Anytime I see an estimate that is divisible by eight, chances are the developer didn't take the act of estimating seriously, that or they are rolling in a heck of a lot of things into their estimate. For a contractor who is billing hourly to estimate in 8 hr blocks, I feel that's even worse.

Steve Py
09/13/2010 11:44 PM by
Steve Py

8-14 hours would have probably been fair, but don't assume it is 400% because you are comparing a subject matter expert (Oren) with someone with argueably less experience with the project.

It may be they are overly pessimistic in their estimating. If that's the case the question would be if they estimated 32 hours, and actually finished it in 14 hours, how much would the bill be? I know there is a lot of temptation to self-reward for a job done quicker than agreed. Why not? If they're happy, you're happy, but it comes down to personal ethics.

I don't recall the name of the billing model, but when I did freelance small I.T. stuff I normally provided 2 rates. A billable rate which I estimated at, and an @cost rate to cover any unexpected surprises. So in the aboved case if I felt comfortable that I could get the feature done in 8 hours it would be an estimate of 8x billable rate, and if anything unexpected came up and it needed to go longer that would be provided @cost. It splits the cost of risk. The client pays for risk, but I don't profit from it. (I'd much rather be using the time for billable work.) The major challenge is to ward off scope creep, but this just meant getting requirements down to distinct features..

Set
09/14/2010 09:35 AM by
Set

It seems that not much people here really worked with or as contractors.

When we are doing estimations at work, we usually ask the dev how long he thinks that it will take him to do a task, he will quote a certain time, on this, we do * 4 because of the overhead, the possibility to get a bug, to get this, or that which will delay a little the work, you can add this and that, unit test, component test, tracing the requirement, writing some documents, if you add a process like CMMI, it will be more even paper work etc. In the end, a 2 hours developing task, can easily 10 h of actual work. Beside this, not much people will do a 1 day task. In each company I have been, they won't bid on projects under the 18 man months ( for ex 3 men * 6 months ) unless it is about improvement over a current contracts.

Peter
09/14/2010 07:58 PM by
Peter

You all may be interested by Dave Laribee's recent piece on estimation:

http://laribee.com/yestimation

His system has 0,1, or 2 point stories. 1 point stories roughly equate to a day's work. This is completely different from everything Agile I've heard. Though I trust his experience, it's certainly not common wisdom.

Steve Py
09/14/2010 11:19 PM by
Steve Py

@Set

That's sub-contracting, and a completely different barrel of fish. It doesn't apply to smaller, agile projects where features are implemented as individual units of work. He certainly isn't about to approach companies like Fujitsu to implement features like sorting.

Case in point, something that does come to mind: Ayende, Given your earlier example of vague requirements, perhaps some of the "fat" added to their estimate was time needed to extract requirements out of the picture in your head.

Set
09/15/2010 08:56 AM by
Set

I will recognize that I don't have a strong knowledge on agile development, but I cannot believe than anyone starting a project and be clueless on things to do or switch providers midway if the previous ones were doing their job, especially on a few hours task.

If this is agile, then it is insanity.

Steve Py
09/15/2010 10:40 AM by
Steve Py

Nah, it's called "still getting stuff done when you don't have or need 18 man-months and $6M". :)

Janki
09/16/2010 10:59 AM by
Janki

I think it can be either of 4 things :

1) They were not interested in doing 3-4 hour job and hence quoted you high as a way of politely turning you down

2) They didn't put time and effort into going deep into the task at hand and gave you just a shallow estimate (based on figure they would like to earn!).

3) They deliberately try to overcharge you, knowing that you are busy and you probably won't have much choice.

4) They were new and didn't understand the code-base at all.

PhilB
09/20/2010 06:02 PM by
PhilB

Single number estimates are far less useful than a range of best to worst case scenarios. Maybe the contractor thought he could do the job in half a day, but it could take as much as 4 days (if any major bugs came up, there were some unclear requirements, he wasn't familiar with the codebase, etc). The single number of 32 hours doesn't indicate if this is the best or worst case scenario and it doesn't indicate the level of confidence that the developer has in the assignment.

Comments have been closed on this topic.