Estimating software projets is one of the most difficult things I've ever done. One guy I worked with recommended this formula:

Take your best guess
Multiply by 2
Add 1
Move to the next higher unit of measure

Therefore: I think it will take 1 week. Submit your estimate as 3 months.

The scary thing is that this facetious formula is sometimes as close to reality as the original estimate. I think the challenge is to make management understand that software estimates are seriously difficult to do correctly and need to be understood as rough guesses. The number of variables and unanticipated problems associated with software estimates makes it a black art at best most of the time.

We find estimation to be vitally important. Without, it becomes impossible to get a stakeholder to admit that his change is really a change requiring more budget or resources, or a whole new project.

Even in smaller projects, lack of project management, especially change management, kills profit. And after a few projects everyone becomes skilled in giving honest estimates, rather than "best spin" estimates.

And to be fair, some pet projects die in estimation. Too bad, so sad.

L

Regarding the comment that, "...all of this does require trust. Without it, a business has much bigger fish to fry than fixing software defects."

While yes, this is true, I wonder about the real-world implications. Many challenged organizations exist. Many people must work in them. Projects are born and need to be fulfilled regardless.

I've worked in organizations where trust didn't exist. The sad part was that the trust had been actively abused and lost. Someone needs to answer the question, "now what?".

In these cases, it's rare that the employees faced with the lack of trust can directly tackle the issue. It's only by acting trustworthy that the trust can be regained. The challenge is that, on the road to a better place, certain "abnormal" CYA measures are necessary.

Bravo, Dr. Bob. I've been in too many places where IT believed that documentation was their primary product, rather than helping the business. Documentation is necessary and valuable, but you do have to strike the right balance and keep priorities straight.

To that end, IT organizations should really be formally reviewed by the entire business to rate their effectiveness. If an IT shop just reports up to a larger IT organization, or to solely the CIO/CTO, its possibly that these are the same entities that promote the "documentation is our product" philosophy. There's shops that may do a wonderful job of documentation and a lousy job being a catalyst for productive change in their organization. Depending on who's yardstick you're rating them by, that same group is either doing a terrific job or an awful one.

It seems to me that somebody has to be accountable for making an estimate. How else can you decide whether the project is worth investing in if you have no way to understand its direct or indirect costs? How can you look forward and understand whether the projects on the board can be accomplished with the resources that you have?

Just because many people make poor estimates is not a reason to avoid having estimates be part of the planning process. If nothing else, these estimates should be used to gauge whether your people are even in the ballpark when it comes to guesstimating what it will take to pull off projects.

If those estimates are consistently overly optimistic, people should understand that so they can ask pointed questions next time - "You say you can do that in a week, are you really sure? Last time it took you double your estimate".

Bob's advice is amlost entirely "agile," an approach which I've found to be wholesome, wise and workable.

As to estimates, if you have to treat them as contracts or not-to-exceed bids, as contrasted to provisional and frequently updated guideline numbers, you're doomed to serious (not to be redeemed by dead chichkens, or even dead goats) disfunction. The only way to improve estimates is to trace historical data (the problem is what to capture) and adapt to the evidence. COCOMO practice and tools can be made to work, but have operational costs that are typically only bearable for high cost/risk/time projects.

Years ago I came up with a really quick and dirty way to estimate the time needed on almost any type of project. It requires that you have a fair quantity of experience with the specific tasks involved so you have a feel for it.

The method is simply sit down and break the project into a bit more bite sized tasks and just go with how much time you would think it would take you to do it if you were on the task...
...Then double it.

Sounds cheesy but it worked out to be astoundingly accurate for me. If your neck is really on the line I wouldn't start out by vocalizing this to anyone. I would do this privately for myself and publicly go with a more conventional estimate method for public use. After a few examples for comparison you should have a feel for how accurate this is for you.

If you do start using this method I wouldn't broadcast it because quite a few people will either rebel against the idea or they may possibly try to push the timeline because 'you are sand bagging' by 50%.

Not sure where I first saw this, but it puts it very succinctly:

The concepts of project management down to six rules-- 28 words total:

1. Define the job in detail.
2. Get the right people involved.
3. Estimate time and costs.
4. Break the job down.
5. Establish a change procedure.
6. Agree on acceptance criteria.

Short change any of these items and you will be in a losing proposition at the end of the process (learned the hard way personally over the last thirty years):
1) If you can not/ do not have time to specify the job out initially, how do you know what is really needed?
2) If you do not commit the correct resources, it will cost more and/or be later than planned.
3) If you can't estimate it, you have no business trying to meet some unrealistic/ imaginary goal. You have to know what the schedule is so you can gage your progress/ lack of progress/ call out delays while there is still time to recover.
4) You have to break up the parts to know what manpower, materials, etc. are needed WHEN.
5) You must control changing the project scope's after initially agreeing to what it is or project creep may cause the project to fail or at least be late. Change is possible but time, cost, and/or resource will need to be changed. “Make the stakeholders feel the pain for their lack of initial planning” is a good motto for this item.
6) Setting acceptance criteria initially lets you estimate what it will take to make the final user happy with the end product and minimizes unnecessary bells and whistle or incorrectly functioning features.

If you want to become better in project management, I suggest contacting a local contact of the Project Management Institute (www.pmi.org) or similar organization and get some training. (disclosure: I'm not a PMI member but I had some of their basic training in the last year and my two project since have ended on time and within the budget – there’s more to PM other than throwing together a rough GANNT chart in 30 minutes.)

I like having daily standup meetings when building stuff. I had a team of 20 people that could do a 15 minute standup meeting every day (assuming I "facilitated" mercilessly - if you back down, some of the team members will eventually demand that facilitation, which is helpful by the way).

Know thy release process (build / test / deploy). Make it as efficient as you can for your project. Execute it early and often.

Provide the team with a smoke test - testing won't waste time looking at something that can't pass the test. The test must be able to be run by the engineers and by the build master (it should be an automated step in the build process - if you can also automate deployment to the test environment you get extra credit).

I'm glad you mention paper testing - testing business processes in a paper environment is a real winner!

Thanks and have fun! - Bob

I just read an excellent blurb about estimating. To sum up: an estimate is not a single figure; it has to be a target that also expresses confidence. For example: so many of us when pressed would respond, "I should be able to finish that in 3 weeks." If we are trying to be 90% accurate, then we should be responding, "I am 90% confident that I can have that completed within 2-4 weeks." In an estimate (a real unknown), we should have very low confidence of hitting a specific date, but a range should be doable.

Or, take technology out of it: Answer with a 90% confidence, how many signers of the Declaration of Independence? Would you rather give a single number or a range to have 90% accuracy?

Ultimately all business stakeholders want to know when a project will be done. How do you provide that without a fairly reliable estimation model to determine level of effort and resource requirements?

Bob is spot-on in this one. Especially that part about trust being the key foundation. I've worked in a company where everyone was either desperate to CYA or trying to do a power-grab or "power-hold" and the result to IT projects was inefficiency, budget bloat, and political infighting, all to extreme degrees.
I've also worked at the other end of the extreme, where the results were, predictably, efficiency, frequent delivery, and a general pleasant interaction between different branches of the org chart.
I don't have to tell you in which environment the morale of the developers and BAs was higher.

Bob,

Trust? TRUST?!?

No one trusts the IT guys in the room. They make more money than the business people, and business people always think IT projects take way too long. The business people want something, but don't have time tell you what it is once, let alone being part of a project every day. I can almost never get a single room dedicated for a project.

I can't start by asking how the business operates, I will be laughed out of the room, or my boss will be asked to get someone who knows the business already.

Trust? Companies will out-source IT in a minute. You have to provide a service, fast and accurate, and then move on to the next thing.

...and I agree that estimating is often useless -- try estimating the requirements, let alone the actual development --- but people want to know what its going to cost, because they have options like buying a package, or cloning something from another part of the mega-corporation. So avoidance is not a solution.

...and unless you get the business people on the project, participating every day providing needs and feedback, then you are going to need those documents, because the development work might be shipped overseas before you know what happenned.

Sorry, but that's the truth.

You avoided the question about documentation.
The stakeholders (as opposed to us pizza holders) will scream if the piece that they sponsored doesn't get done, useful, or useless, or not.
Documentation is one way to at least show what has happened and be able to explain the deep-sixing of the pet of overly egoed power-boy.
I'm always looking for a job as a generalist since my way of considering a problem is not just "let's fix it", but where did this whole thing come from and can we get rid of it? It would seem that the only time anyone will consider such a thing is when there is a problem (If it ain't broke, keep it forever).
Your answer is to me one of those perfect answers that can hardly be made to work. I HATE politics!

I feel really guilty. I live in that ideal company, small (90 people), outrageously fast paced, but ALWAYS looking to IT to find better, more effective ways to accomplish more work (and higher loan volumes) with the same number of people. IT has a staff of five (one dept head who also does web development, a network guy, an analyst who also is an expert in our CRM package, a web designer and a data administrator who both help with floor support). We have two major commercial packages that we use and those databases are used to extract and consolidate data for presentation in web pages, which reduces how much data collection we have to do. We do minimal project planning (high-level only and only for larger projects...a six man-month project might have ten line items) and most of what passes for documentation occurs after the fact. After a quick specification-gathering meeting, we respond with an email that describes what we heard to make sure we said/heard the same things...usually the toughest part of any technical project. After that, we put together a prototype, demonstrate what we have, then go into revise/test/demo iterations until our users are satisfied. Afterwards, we write instructions, do training, and users come back over time with enhancement requests. Because they eventually get almost exactly what they want, they are very forgiving about bugs (our rule is that bugs are OK as long as they don't damage data) and delays in delivery when higher-priority items bump them down. We don't ignore project planning entirely (I'm actually a certified PMP...but only so I can be employable elsewhere...you never know), but we keep most of our projects small enough that we don't need it. Our company president is totally in support of IT (although several of our second-level managers are somewhat suspicious of it), but the majority of the company treats us like rock stars. I agree that much of the desire for in-depth project planning comes from CYA environments (and I've worked at many of those). I've been in IT for 25 years and have found that folks don't give a d@mn about project planning, if you get them something they can use quickly and get it right soon after (and work with everybody to decide priority).

One of the key reasons that estimation has been such a flawed practice is because it's traditionally based on inaccurate proxies and flawed assumptions. Traditionally, organizations create estimates based on the elapsed time to complete a unit of work--based on a memory of the time a similar unit of work took to complete. Elapsed time is the traditional frame of reference for software development. Unit of work begins, effort is expended, and unit of work is completed. And in the absence of another unit of measure, elapsed time becomes a proxy for effort. But the reality is that elapsed time is a very inaccurate unit of measure for estimating the time it will take to build software. Elapsed time estimates typically over or under account for non-development work, parallel projects, and environmental distractions. 6th Sense Analytics measures something called Active Time, which is a very accurate and empirical basis for understanding the actual effort it takes to develop software. Using an Active Time-based proxy, we can truly understand the active and actual effort required to complete a unit of work. We can then divide this value by the actual Active Hours of a developer per week and we can generate a much more accurate estimate. Tracking Active Time by project and development activity (e.g., design, edit, debug, test, etc.) also gives us a basis for truly understanding project progress, without requiring developers to manually report on their efforts. You can learn more about this model here: www.6sa.com.

I strongly disagree with your statement "the whole process of collecting software requirements from various stakeholders and reconciling it is bogus."

Unlike the guy who works in a 90-person company, I work in one that has 66,000 world-wide. IT head count alone is probably close to 1,000. I gather and document requirements for systems that run our core business in four continents. I have a mandate from the global head of IT, based on a mandate from his boss, to build systems that are copy-exact unless there are legal or regulatory reasons that require regional variations. The goal is to maintain one set of systems globally, rather than many regionally. This reduces work for IT and has business benefits as well.

The only way to accomplish my task is to align business processes across the globe. Doing so gives the business a way to compare apples to apples in different regions. Remember Deming? The first step in improving a process is to standardize it. So I conduct global meetings to align business processes in the guise of specifying software requirements. And it works! I've gotten people to come to agreement on sixty major business issues, 56 (91%) without escalation to higher management. (I keep metrics.) The result: Very well-documented requirements that result in high-quality code and high-quality test cases. (In my company requirements drive both, separately.) IMHO you absolutely must have good requirements to build good software.

The proof of the pudding is that we just installed the system I am responsible for as well as a dozen or so more that it interfaces with in a new facility and it went so well that lots of people are going home early. Sure, we've found some code defects, and I found one requirements defect (out of some 3,000), but they are getting fixed rapidly and the systems are humming. Everyone, from people on the floor to way high up the management food chain is impressed with this installation.

Good requirements are not the only success factor -- smart, hardworking developers and testers and project managers, as well as knowledgeable and committed business partners, are indispensable -- but they sure are a crucial one.

OK, time to poke a stick in a hornets nest.

Why is it that the rest of the world has functioned fine on estimating projects of many sizes and scopes, then sticking to them; while IT screams "OH WE can't do that!"?

Funny, folks can build a skyscaper or a dam while saying within the quoted cost and timeline, but IT says it's impossible.

Is it because IT still acts like a group of artists rather than actually employing some professional engineering concepts?

Folks, you can't continue to function like an economic black hole. There's plenty of training, books, etc. for planning projects. Start with a basic Engineering Economics book.

Better yet, read "A Connecticut Yankee in King Arthur's Court". Which are you, the Yankee or Merlin?

Why is it that the rest of the world has functioned fine on estimating projects of many sizes and scopes, then sticking to them; while IT screams "OH WE can't do that!"?

I'll admit a large reason is because lots of people in IT are really bad at estimating. But part of the reason we've managed to stay so bad at estimating, and the main reason all estimates seem to be so far off, is that the business side wants the estimates to be low.

The common complaint is that IT projects are "always over time and over budget". Since the IT budget (for software) is almost entirely time-based, this is the same as saying the projects are always over time.

But most IT projects -- and nearly every one I've worked on -- have a budget set before the detailed design is ever done. Or at least the project sponsors have an idea in mind what they'd like it to cost.

If you don't promise to deliver in the given time frame, they'll go find someone who will. Not that they'll deliver in that time, but they'll promise to deliver in that time.

Compare this to construction. If a cement contractor says it will cost $50k and the project sponsor doesn't like it, they'll go get a second quote. If that quote is also $50k, they'll accept the cost and choose one.

If a software contractor says it will cost $50k, the sponsor can find someone who will promise to deliver for $20k. The sponsor will either accept the lower bid, or use it to negotiate the first vendor down to $25k. When it ends up costing $50k, that project goes in the books as "over time and over budget".

The best way I have found to produce an estimate is the common 3-point estimate (PERT). Get the people who do the work to tell you - What is the worst case estimate (Pessimistic)? If Mr. Murphy jumps in with both feet, how long will this take? or What is the most time it has ever taken you to do this type of task?
Next what is the best case (Optimistic). If everything falls into place perfectly how long would it take? Finally, what is the most likely(M)? How long does it usually take to do? Then use the formula (P+4M+O)/6. If you really want to figure out how good that is, take this information and run it through a monte carlo simulator. This will will take the 3 base numbers, run the variable through thousands of iterations and tell you the likelyhood that you will achieve the date you have set. In addition you then then determine that if you want to be 90% confident you will meet a date, than this is the date you should use.
The problem is everyone wants planning and estimating to be simple and easy. If you want accuracy, it just isn't. The good news is, if you begin doing these things, you begin creating the historical documents you can then use the next time to say well really our average time for this is...
And when they say they can't estimate something, I just say, "Well either you, who has some idea of the work it takes to do this sort of thing, provides the number, or I, who has never done this work, will need to. Either way you will be expected to attempt to achieve the target. Which number do you want?"
If you really want to get your estimates tight, look into Risk Management as that focuses on what things may cause a varience and how can you overcome them?