Bob,

Nice answer. But I think the whole "based on the waterfall method," makes this question somewhat suspect. I agree with Developing Developer that requirements are important. But I feel that making them the "most" important thing is the common trap the waterfall method throws developers into.

So much faith is placed in requirements gathering and that phase is put on so high a pedestal that when someone using waterfall method feels they have the requirements "nailed", that the rest is smooth sailing. Many times this couldn't be further from the truth. Sometimes you can have the best requirements ever created, but your stakeholder later changes their mind. If you don't look up and just fall down the waterfall, you will have a great, smooth running project, but you will not deliver what the stakeholders want.

Again, an Agile process can deal with stakeholders changing their minds, and changing the requirements, a waterfall process can't (at least not gracefully).

The time for waterfall to be the lone methodology to be taught to CS students in America should be fading away. It is my opinion that Software Engineering courses today should begin by outlining the waterfall method, and then move on to teaching agile and iterative methods that are now supplanting it within industry.

Take a look at this post on Tech Dark Side where David Christiansen goes back to "the source" to see what problem Dr. Winston Royce was actually trying to solve when he documented the waterfall methodology back in 1970.

I've known for a while that waterfall shouldn't be the only method. After reading David's post, I'm almost ready to agree with his conclusion that it should never be used.

I think at this time a quote from Dr. W. Edward Deming is in order "You can not inspect quality in, you have to build it in earlier in the process."

In software you can test all you want, the software may even pass the test, but if it isn't what the customer wants, regardless of successful testing, the software is not successful. In the real world the waterfall won't work because the customer will change the requirements continuously so the project will never get out of the requirements stage of the pure waterfall process; and the client will never sit still for having to accept an early version of the requirements being locked in stone or of a project never getting out of the requirement stage. Business information and marketing opportunities change way to fast for waterfall to work.

The idea of 18- 24 month projects and waterfall methodology only lives in the minds of a few ivory tower academics or Luddites, certainly not on the front lines of 21st century software development.

In line with my comment in the "What has the most impact on software quality" topic, I think the build / test ratio depends on the project. Get the web site up and running and start the Alpha testers using it asap, but for the mars rover, spend the time up front. Everything else fits somewhere between the two and IMHO is why this discussion will run and run! Horses for courses. For all types of development, the biggest single leap in quality improvement is to start peer checking.

Hey Drew - thanks for mentioning my post on waterfall here. I think this is an interesting question that "developing developer" poses, because it points out the underlying assumption that is so commonly made in software development - that there is this magical "right" way of building software, that if you just use the right process you'll always get it "right." I never like to see the word "right" used in the context of software development - there is no such thing. Developers and testers shouldn't worry about doing things "right". They should worry about doing things well. When you can, as a developer or tester, contribute to ANY project in a meaningful way, no matter what PROCESS they follow, even if it's no process at all, then you've got talent.

Implementing Agile works better for isolated, new systems than it does for large, integrated, complicated systems; because the cost of every one of those iterations becomes multiplied and knowing when you are "done" is much more difficult.

Everyone needs to invest in better requirements, regardless of whether it's "waterfall" or Agile. Agile can be used as an excuse for no planning or thought at all. In an integrated environment the requirements need to consider all parts of the system lifecycle for a piece of business data, especially if Agile can only be applied to one of the integrating systems.

Dave and Drew are right - initial requirements gathering is about the only place you can start, but prototyping, keeping the customer in the loop, is essential to make sure that the product that evolves is something that actually meets the business needs when it's finally ready for production. I used to develop custom business Apps in Revelation G2b, with a majority of the work done at the customer's office, with their active participation. The 'aha' moment came when they asked me to get out of the way, they had work to do - on the system we'd been developing together. In my current situation, we've brought the customer's most IT-knowledgeable person into our cubicle farm - he's participating with the IT staff in ensuring that the project becomes what his department needs. There have to be loops in the process to ensure that the end product is both useable and useful.

The fundamental problem here is that THE QUESTION IS WRONG!

It's like asking what's more important to human life - water or food? Of course both are essential, though some might argue that water is more important than food since you die from thirst faster than you die from starvation. But if you're scuba diving, the real answer is "air".

Enough with stupid analogies, and back to real comments...

As I said, they are BOTH important, and the failure to do either one well can kill your project.

When you are testing, what exactly are you testing and what is your success criteria? A terribly designed product can "work as designed" perfectly. It can pass every test and fail miserably in implementation.

Likewise, a perfect design can be so full of bugs and and performance issues that again, it will fail in implementation.

Honestly, in my experience it is the lack of "real" requirements and a good, clear design that is more likely to cause expensive problems. If the requirements are good, you can design a reasonable solution that can be constructed and will meet the REAL needs of the users.

Computer code is much more straight forward than working with users, so bug fixes are "a simple matter of programing". If the requirements and design are wrong, you start all over at the beginning and spend countless hours rehashing everything. If a user finds a bug, it is much more specific and easier to deal with.

The challenge, is defining the REAL requirements and documenting them in a way that the stakeholders (that includes end users) can understand them and agree upon them.

All software development projects either follow the waterfall model or they fail. The only difference between "classic" waterfall and the spiral model and the iterative agile model is how they define the beginning and ending of the project. Classic waterfall does one cycle per project, spiral does a few, iterative does a bunch, they even do a bunch before they start to define what the project actually is and then break it down into segments that can be done in one month iterations.
You cannot test _anything_, thus you MUST get some requirements before beginning the test (or what exactly are you testing?)
About all the difference I've seen in methodologies besides the length of the cycle is that agile puts testing 2nd, behind requirements, instead of 3rd behind development. You write tests based on requirements then develop to pass those tests. It still doesn't guarantee user acceptance nor does it validate that the requirements are correct; it merely validates that you have successfully satisfied the requirements given.

Many failed projects of all methodologies have satisfied requirements without satisfying users.

All software development projects either follow the waterfall model or they fail. The only difference between "classic" waterfall and the spiral model and the iterative agile model is how they define the beginning and ending of the project. Classic waterfall does one cycle per project, spiral does a few, iterative does a bunch, they even do a bunch before they start to define what the project actually is and then break it down into segments that can be done in one month iterations.
You cannot test _anything_, thus you MUST get some requirements before beginning the test (or what exactly are you testing?)
About all the difference I've seen in methodologies besides the length of the cycle is that agile puts testing 2nd, behind requirements, instead of 3rd behind development. You write tests based on requirements then develop to pass those tests. It still doesn't guarantee user acceptance nor does it validate that the requirements are correct; it merely validates that you have successfully satisfied the requirements given.

Many failed projects of all methodologies have satisfied requirements without satisfying users.

So many misconceptions; so little time...

Testing cannot find requirements errors and will only rarely find design errors. "Works as designed." doesn't mean the user's needs are met. Likewise, the patches, kludges and lash-ups needed to achieve some semblence of the required functionality of a hastily assembled product generally mean the program will never really be stable.

Agile development methodologies work well to achieve a product only when the project is small. Although there are a few times larger efforts have succeeded with agile methodologies, most start to flail when the team size exceeds about 10 to 15 developers. None have the engineering processes to actually solve large, complex systems let alone execute the non-software tasks that are a major part of large-scal systems development.

The above being said, many large systems have pieces that can be built using agile methods but don't expect the overall system to be built on an agile schedule. That brings me to my thesis and that is that schedule duration is the ultimate arbiter of software quality for large, complex systems.

Cheers,
Dave

My nickel:
I think it all comes down to the quality of the people doing the work. If they are trainable if they have been trained if they are doing what they are trained to do and are motivated to do it well then they will more than likely produce quality work.
If they are untrainable they won't be able to do the work.
If they are not trained they won't be able to do the work well.
If they are trained and not motivated they may or may not do the work well.

PMI teaches quality is planned in, not tested.

The root confusion may be based off who asked the question, and how much/recent their experience in the business world has been. Systems are much more complex, and Ivory tower types think Waterfall or V model works in the business world. My world is different - we always have change requests, requirements are never Fully elaborated/defined, and we always enounter changes during testing. This is due to business model changes, politics (who made the decision) and misunderstandings on the details.

Everyone is more or less right. The problem lies in the definition of testing. In this discussion testing was reduced to the tasks of a QA team. Testing is way more than getting code and running test scenarios against it.
When a new proposal is fed into the process, the testing starts already there. The ideas are tossed around and the reasonably good ones are kept whereas the wonky ones are dropped (hopefully). Then a formal proposal is created, the BAs write functional specs, the developers write technical specs based on that. Each and every time a lot of testing takes place. Others read the documents and provide comments. That IS testing and IMHO it is as important at that stage in the process as it is after code is written.
The professor is right in stating that testing is more important, assuming he/she thinks of testing the similar way as I do. Information gathering is also nothing else than documenting test results. The test is asking stakeholders, observing the result, and documenting the outcome, on which (corrective) action is taken.
Now, does the QA team have to do the testing on proposals? I say yes! Also, at that point support has to be involved as support is the group in the company that knows the end-user best. Needless to say, I also think that QA and support must have veto rights and not blindly follow what others planned out, not saying that these plans are wrong at all times.