Quality Assurance Issue Summary

Problem Summary

One of the high level goals of Chandler is to distingush itself as a high-quality open source end user application, on par or better than the bulk of commercial applications. We believe we need to have a plan for getting to that position, and this summary is more of an attempt to document the steps we need to take than it is a complaint about what steps have been taken so far.

Brief History

0.1 was of low quality, but it wasn't of unknown quality. I content that 0.1 was a QA success, because it performed up to expectations. The deal is to raise the expectations, and to make sure we have a realistic way to achieve those goals.

Issue Summary

• Unit Tests
• End-User Testing
• Who writes the tests
• Who runs the tests
• What defines a "good" test
• What other process improvements are needed to assist us in producing good quality software
• What people are needed to implement these processes
• Is there a non tradtional QA process which could use willing volunteers in a coordianted and useful way
• What is the process for externally developed parcels / modules

It is easy for QA to turn into meaningless number crunching, improving our ratio of bugs filed per candy bar or some such thing. At the same time unmeasured things do not improve. The art is in figuring out how to insert quality as a factor into the marketplace of ideas, and to give it a weight commensurate with our goals, and not to invent a false market which can be gamed.

Risks

Template feels useless, risk of bas quality is obvious, we won't ship Chandler.

Known Answers

• Tenative plan to run a tinderbox of automated tests has been mentioned
• QA is a part of the in-process plan for 0.2 and beyond

Dependencies

-- MichaelToy - 15 May 2003

Discussion

It seems like this QualityAssuranceIssueSummary is related to (and perhaps dependent on) the BuildIssueSummary.

-- BrianDouglasSkinner - 15 May 2003

I believe that the best way to write quality software is to put the burden of testing on the developer who wrote the code. This means developer should be responsible for writing most of the tests, and that the code should be designed to be testable. In my experience, when developers rely too heavily on a QA department for writing test code, quality suffers.

-- John Anderson - 16 May 2003