QA process

A. Startup phase
During the startup phase QA effort are targeted mostly to documentation artifacts of the project. The following efforts take place during the phase:
1. Review and analysis of requirements. QA experts together with development experts analyze the requirements and specification documents in order to eliminate any inconsistencies. Software UI and navigation may be reworked greatly to improve the software usability and make it match common UI standards. New functionality may be offered to extend the required features in order to improve the entire application. Security issues that can be found at the early stage of testing the application may be avoided so project resources can be reduced.
2. Definition of testing goals and criteria. Definition of testing goals, criteria of meeting the goals, risk assessment, possible ways of risk mitigation.
3. Definition of testing approaches.Testing approaches and techniques are defined based on software type, project duration, available resources and testing goals. For example, load testing and security analysis are necessary for distributed applications, while automated regression tests are necessary for long-term projects. Tools required to perform the necessary testing are identified.
4. Resource estimation. QA resource estimation is based on the software specification, required QA documents and chosen testing approaches. Automated regression/load testing scripts reduce resource estimation for long-term projects, but may increase it for short projects.
5. Document templates approval. Format of all QA related documents should be approved by both sides, and modified if there are any objections or suggestions to already existing set of templates.
6. Creation of initial testing documents. First versions of test plans and automation plans are created based on the initial versions of software specification and automated testing requirements.
B. Main phase
During the main phase of QA process testing itself takes place, QA artifacts are being created, testing repositories are created, automated tests are developed and executed, found defects are detected and reported, fixes verified, QA documents are completed and maintained:
1. Creation and maintaining of testing documents. The full required set of QA documents, including low-level documents such as test cases, is created. All QA documents are updated correspondingly when requirements are modified. New test cases are created when defects are identified in order to make sure fixed defects do not appear in the software product during further development.
2. Manual testing according to testing documents. Continuous testing of software based on created QA documents guarantees that all features described in software requirements are present in the product, are operable and work as expected.
3. Manual ad hoc testing.Test cases and test plans cannot be created for all possible situations in applications. This is why ad hoc testing (also known as random testing) is also necessary in order to identify defects that cannot be found while performing testing using test plans and test cases.
4. Usability testing. Manual testing of software product to make sure the application has user friendly interface that meets common UI standards, application navigation is task oriented and allows execution of common tasks in minimal required user actions.
5. Documentation testing. All product support documentation such as help system, product information on WEB sites and system requirements are tested for consistence with actual software and product specifications.
6. Security analysis. Security analysis is performed against distributed applications (for instance WEB applications) in order to make sure the applications meet general security requirements. Application should be protected from unauthorized code execution, application data should be protected from unauthorized access, data should be available to authorized users.
7. Automated testing. Automated test scripts are implemented, maintained and executed against the application in order to identify defects that appear in previously operable features of software due to the latest code changes. Automation of such testing helps to save QA resources and to identify such issues at early stage. Integration of automated tests into application build process is common practice.
8. Performance testing. Automated testing of application performance is maid against distributed applications in production-like environments in order to make sure server-side part of application is capable of serving the required number of simultaneous user activities. Also performance testing allows identifying actual number of users the application can serve simultaneously to adjust software system requirements or production configuration.
9. Load/stress testing. Automated load and stress testing scripts are implemented and executed to make sure the application stays stable after continuous load with expected amount of simultaneous users and after short time stress load with more simultaneous users than expected.
10. Monitoring bugs and fixes.Bugs found by QA team, other teams involved in product development as well as customer bugs are monitored by QA team. Bugs that have insufficient information to reproduce them are reproduced by QA team, exact steps and conditions are populated to bug tracking system. Fixed defects are retested and related documents are updated correspondingly.
11. Reporting progress and statistics. At each stage of QA activities reports with the full list of current activities and progress of each activity are created and provided to customer. Detailed reports with currently open defects are provided on demand. Current product quality assessments are provided on demand as well.
C. Release phase
Release phase represents QA activities when software product is about to be deployed to production environment or a new version of software is about to be published for customers.
1. Assessment of testing coverage. Overview of QA activities that have been done, analysis of testing results, assessment of what percentage of the software product has been tested and what additional testing could be performed to guarantee better software quality.
2. Acceptance testing. The final step of testing performed by QA team and representatives of customer team in order to make sure the software product has the desired quality level and matches customer expectations.
3. Goal meeting validation. Project goals are compared to criteria to make sure the goal is met.
4. Review and approval of release notes. Release notes that usually include “What’s new”, “Known issues” and “Bug fixes” sections are made in collaboration with QA team and reviewed by QA team.
5. Transition of QA artifacts. All artifacts created by QA team during testing of the software product are gathered to a single package and delivered to the customer.