Quality Policy
Quality Definition
Classification of Defects
Release Policy
Software Quality Tools
Software Quality Processes
The entire delivered Zoltan product will be considered a "class A" program under ASC program guidelines. Modules created before the ASC guidelines had no formal QA program and will lack ASC SQA artifacts (objective evidence).
The Zoltan team shall participate in all ASC reporting processes as directed by ASC management. The Zoltan team will evaluate new practices to reflect ASC Program Office process revisions. FY2004 money has been committed for quality improvement.
Research software will follow the guidelines of QC-1 which allow specific relaxations in the QA rigor for research activities.
The Zoltan project accepts the following definition of quality: "the totality of characteristics of a product or service that bear on its ability to satisfy stated or implied needs." This is known as the "fitness for use" definition of quality (ANSI/ASQC A8402-1994).
Critical: A defect that could lead to loss of life, significant environmental damage, or substantial financial loss.
Major: A non critical defect that significantly impacts Zoltan's fitness for use.
Minor: A (non critical, non major) defect that reasonably impacts Zoltan's fitness for use.
Incidental: Any other defect which does not reasonably reduce Zoltan's fitness for use.
Only the Zoltan team leader may authorize a release. The Zoltan team leader shall not release software with any known critical or major defects. User registration shall allow the Zoltan team to notify all Sandia and ASC users and recall their defective software if a critical or major defect is discovered after release. The Zoltan team leader may determine that it is acceptable to release software with known minor or incidental defects.
Because of the small scale of the Zoltan Project, only a few, simple tools are required for use by developers:
CVS: maintains code, documentation, meeting notes, emails, and QA program artifacts;
Purify, PureCoverage, Quantify (Rational): for code testing, coverage measurements, and performance analysis;
Bugzilla: tracks bugs and requests for changes and enhancements (deployment in FY2004);
Mailman: creates email lists to automatically notify users by area(s) of interest (deployment in FY2004);
Quality Function Deployment (QFD): tracks user requirements through implementation and testing (phased-in deployment in FY2004 in which one pilot project will use QFD);
Makefiles: ensures proper compilation and linking for all supported platforms;
Zoltan Test Script: runs integration, regression, release and acceptance testing; and
Kanban: Team Leader's white board displays all outstanding issues, schedules, and assignments.
The Zoltan's software quality process defines how work may be performed, including process ownership, authorization to perform, activities and their sequence (if sequencing is required), process instructions, metrics, and checklists (with places to identify who performed each activity). The only source for a process is Zoltan's CVS repository. Currently the following processes are defined:
Meeting: defines the process of calling and documenting meetings;
Development: (not currently used) defines the software development process including requirements, design, implementation, testing, reviews, and approvals;
Request: defines the process of capturing user requests for new features;
Requirement: the process of capturing user comments that may become requirements after review and approval;
Review: defines the materials reviewed prior to acceptance for Zoltan release; and
Release: defines the release process including testing requirements and creation of the release product;
A Zoltan developer initiates a process by obtaining the current CVS version of the process, renaming it to reflect its usage, and committing the renamed process back into CVS in an appropriate directory (on the same day to ensure that it is the current version). As a policy, the process may continue under this committed version even if its original process is later superceded. After one or more activities are completed, the renamed process is updated to reflect the results and committed back to CVS (with appropriate comments.) A process is completed when all required activities are completed including reviews and approvals (as necessary), and committed to CVS. The final CVS comment should indicate that the process is complete.