How to make delivery task plan/schedule based on individual task estimates

Question

I am wondering, how "real" work day (7-8 hours) relates to "engineer hours" term used in estimation of the time necessary for completion of some task. I think that estimated effort in EH (if correctly estimated) cannot be simply translated into work days by dividing with 8, and that effective work day of a programmer is shorter than the time he spends in the building in which he works. This can lead to big errors in estimates when estimating small chunks of tasks (i.e. what is scope of one iteration in SCRUM) and when there are no best/worst case estimates, but planning is done based on individual task estimates done by programmers. When programmers need to estimate time necessary to do some individual tasks, they usually estimate the time from the moment they start working on it to the moment they complete it. Needless to say, it is insane to expect that someone will do 4 tasks of 2 hours each in one day.

I would like to know what are best practices in successful companies for relating effective time vs. time spent "at work" and are there some books or researches which estimate what is average time programmer spend working (focused on the tasks at hand), not doing other activities which are not included into estimate, like reading/writing mails (except ones included in estimate, if it is support task or similar), brainstorming, meeting, drinking coffee, estimating tasks and updating status in bug/task tracking tool.

If there are no books or researches, any links to articles about this issue by respected members of community will be of help also as I couldn't find any, these keywords are too generic.

Answer 1

There are two things that we do when we program:

• Things which we've done that are similar to what's gone before, and
• new things that we've never done before.

When we're repeating similar things, estimating - whether in points of relative effort / complexity or in hours - is usually fairly straightforward. Even when estimating in points, these can easily be correlated back to hours after a sprint or two, once the team has got up to speed.

The problem most teams face is that they're often doing new stuff. This is because:

• doing a project where it's all been done before is pointless (see "Waltzing with Bears")
• if it's all been done before, there's probably Open Source or off-the-shelf products which will do it for you, so learning how to use those may be new
• developers hate doing the same thing over and over again, and would rather learn something new by automating it or producing a reusable library.

Thus, a lot of what we do on software projects tends to be new. The usual same-old same-old frequently ends up being given to junior developers, for whom it's new.

The trouble with new things is that you have no idea how long they're going to take. You've never done them before!

I've seen a number of strategies for handling this, depending on how determined someone is to (force devs to) produce reliable estimates, how much trust you have with stakeholders, etc.

1. Pad the estimates horribly and then make the work fit them (this is what developers end up doing if PMs / SMs force them to produce reliable estimates, whether the devs realise it or not).

2. Kanban's "classes of service" allow you to mix some new things with some familiar things. This will tend to allow a less variable flow through the development pipeline, so you can base your estimates on that.

3. Do everything that's risky first (particularly if you don't know how risky it is!). Gradually the flow will stabilise. On a 9 month project, it took 3 months to start producing any kind of useful estimates, so careful with this one. The upside is that your stakeholders will see you addressing the things which keep them awake at night early, so trust can be high. Your PM / SM will go spare during this time. It's OK.

4. Accept that the old metaphor of "time spent = work done" is a myth in projects involving high amounts of learning and knowledge work. It might have been appropriate in the days when developers were treated like a factory - churning out the implementation of others' designs - but in the world of Agile and Scrum, it's completely inappropriate. Even Scrum.org have recently removed the necessity for estimation and velocity measurements from their guide. Find other ways to develop trust, manage risk, win budgets and have conversations around the work - since these are the only reasons you need estimates in the first place.

Answer 2

Besides the mythical man month, which is always worth reading, Agile estimating and planning is one of the best books I found on the topic.

Best quote ever:

“There's no sense in being precise when you don't even know what you're talking about.” (John von Neumann)

Also, experimental software engineering attempts to draw statistically relevant conclusions out of experiments in a controlled environment. Very roughly you get to design experiments applying the scientific method, choosing input variables (ie. experience of the programmers, time of the week, language/framework, etc.), and measuring the output.

From wikipedia:

The scientific method suggests a cycle of observations, laws, and theories to advance science. Experimental software engineering applies this method to software.

A good book I remember from a remarkable university class on the topic is “Experimental Software Engineering”. It assumes you have a grasp over statistical distributions and descriptive statistics, but it's a very nice read.

Answer 3

An easy way to get on top of planned vs actual time spent (which in itself is always fluid to a certain degree) is to use an agile management tool. Target process can run with the heavies, but you can get the full system for free if you use less than 6 licenses. I have used it at 4 different clients over the past 5 years with great success each time.

I do not have the exact answer you want (i.e hours * 1.75 = working time), but TargetProcess' continous adjustment of estimates matched up with time spent will very quickly give you the ratio that applies to YOU.

PS- I do not work for them.