QA process

A. Startup phase
During the startup phase QA effort are targeted mostly to documentation artifacts of the project. The following efforts take place during the phase:
1. Review and analysis of requirements. QA experts together with development experts analyze the requirements and specification documents in order to eliminate any inconsistencies. Software UI and navigation may be reworked greatly to improve the software usability and make it match common UI standards. New functionality may be offered to extend the required features in order to improve the entire application. Security issues that can be found at the early stage of testing the application may be avoided so project resources can be reduced.
2. Definition of testing goals and criteria. Definition of testing goals, criteria of meeting the goals, risk assessment, possible ways of risk mitigation.
3. Definition of testing approaches.Testing approaches and techniques are defined based on software type, project duration, available resources and testing goals. For example, load testing and security analysis are necessary for distributed applications, while automated regression tests are necessary for long-term projects. Tools required to perform the necessary testing are identified.
4. Resource estimation. QA resource estimation is based on the software specification, required QA documents and chosen testing approaches. Automated regression/load testing scripts reduce resource estimation for long-term projects, but may increase it for short projects.
5. Document templates approval. Format of all QA related documents should be approved by both sides, and modified if there are any objections or suggestions to already existing set of templates.
6. Creation of initial testing documents. First versions of test plans and automation plans are created based on the initial versions of software specification and automated testing requirements.
B. Main phase
During the main phase of QA process testing itself takes place, QA artifacts are being created, testing repositories are created, automated tests are developed and executed, found defects are detected and reported, fixes verified, QA documents are completed and maintained:
1. Creation and maintaining of testing documents. The full required set of QA documents, including low-level documents such as test cases, is created. All QA documents are updated correspondingly when requirements are modified. New test cases are created when defects are identified in order to make sure fixed defects do not appear in the software product during further development.
2. Manual testing according to testing documents. Continuous testing of software based on created QA documents guarantees that all features described in software requirements are present in the product, are operable and work as expected.
3. Manual ad hoc testing.Test cases and test plans cannot be created for all possible situations in applications. This is why ad hoc testing (also known as random testing) is also necessary in order to identify defects that cannot be found while performing testing using test plans and test cases.
4. Usability testing. Manual testing of software product to make sure the application has user friendly interface that meets common UI standards, application navigation is task oriented and allows execution of common tasks in minimal required user actions.
5. Documentation testing. All product support documentation such as help system, product information on WEB sites and system requirements are tested for consistence with actual software and product specifications.
6. Security analysis. Security analysis is performed against distributed applications (for instance WEB applications) in order to make sure the applications meet general security requirements. Application should be protected from unauthorized code execution, application data should be protected from unauthorized access, data should be available to authorized users.
7. Automated testing. Automated test scripts are implemented, maintained and executed against the application in order to identify defects that appear in previously operable features of software due to the latest code changes. Automation of such testing helps to save QA resources and to identify such issues at early stage. Integration of automated tests into application build process is common practice.
8. Performance testing. Automated testing of application performance is maid against distributed applications in production-like environments in order to make sure server-side part of application is capable of serving the required number of simultaneous user activities. Also performance testing allows identifying actual number of users the application can serve simultaneously to adjust software system requirements or production configuration.
9. Load/stress testing. Automated load and stress testing scripts are implemented and executed to make sure the application stays stable after continuous load with expected amount of simultaneous users and after short time stress load with more simultaneous users than expected.
10. Monitoring bugs and fixes.Bugs found by QA team, other teams involved in product development as well as customer bugs are monitored by QA team. Bugs that have insufficient information to reproduce them are reproduced by QA team, exact steps and conditions are populated to bug tracking system. Fixed defects are retested and related documents are updated correspondingly.
11. Reporting progress and statistics. At each stage of QA activities reports with the full list of current activities and progress of each activity are created and provided to customer. Detailed reports with currently open defects are provided on demand. Current product quality assessments are provided on demand as well.
C. Release phase
Release phase represents QA activities when software product is about to be deployed to production environment or a new version of software is about to be published for customers.
1. Assessment of testing coverage. Overview of QA activities that have been done, analysis of testing results, assessment of what percentage of the software product has been tested and what additional testing could be performed to guarantee better software quality.
2. Acceptance testing. The final step of testing performed by QA team and representatives of customer team in order to make sure the software product has the desired quality level and matches customer expectations.
3. Goal meeting validation. Project goals are compared to criteria to make sure the goal is met.
4. Review and approval of release notes. Release notes that usually include “What’s new”, “Known issues” and “Bug fixes” sections are made in collaboration with QA team and reviewed by QA team.
5. Transition of QA artifacts. All artifacts created by QA team during testing of the software product are gathered to a single package and delivered to the customer.

Accuracy of Quality Assurance

Accuracy of Quality Assurance

Quality Assurance and Control: Why Accuracy is Vital

Competition is intense in today’s business and industrial arenas. When products and services vie heavily for market shares, the ability to deliver quality products that stand up to their promises is often a determining factor when a customer chooses a vendor. The accuracy of that vendor’s quality assurance process is key to gaining or expanding footholds in market shares.

Definition
Quality Assurance (QA) can be defined as “the inspection and testing process a business conducts to verify manufacturing, assembly, or performance standards for a product or service in accordance with business, local, industry, or government guidelines.”

Quality Control (QC) often goes hand-in-hand with Quality Assurance. QC can be defined as “Steps, checks, and balances to detect, repair, and improve upon errors or defects in a manufactured product or a service.”

A QC/QA inspection process can be as simple as checking a document for typographical, grammatical, or punctuation errors. It can be as complex as disassembling a large-capacity airliner and inspecting every part then reassembling it, testing performance as each step is completed.

Typical QC/QA Process Steps
The most popular Quality Assurance process is called the Shewart Cycle, developed by Dr. W. Edwards Deming. The cycle outlines a process untailored to specific businesses or industries.

Commonly abbreviated as PDCA, the Shewart Cycle comprises four steps:
1. Plan. Outline specific and detailed inspection steps and expected results. Explain corrective actions at each step, should a product or service fail to pass inspection. Include the percentage of each product category to undergo Quality Assurance testing.
2. Do. Initiate and complete each action step within the QualityAssurance process.
3. Check. Using prequalified standards, monitor, test, and compare products for known quality points and potential defects or shortcomings.
4. Act. Adjust product generation processes to improve and correct products to meet or exceed expectations or requirements.

Above all, test and retest often.

The Shewart Cycle versus ISO 9000
PDCA is a process to ensure accuracy in quality assurance and is conducted within the business structure. ISO 9000 is an international standard that ensures a company’s Quality Assurance plans and procedures are in place and effective.

Quality Often Lies in the Details
Focusing on details will enable thorough and accurate quality assurance processes. QC/QA in a manufacturing process could easily include not only the final product but the raw materials used in creating a product.
· Is the ordered materiel of the correct chemical composition? Is the steel strong, dense, etc., enough? Is the liquid too acidic, sweet, watery, etc.?
· Is the board precisely the correct length or it is slightly off the required dimension? Is the glass too thick? Too thin, etc.? Is it strong enough or thin enough? Is it made from the right wood?
· Is the color or shade correct? Is it of the correct dye lot?
· Is that bolt under the sealed covering torqued to specifications? Is it too tight or too loose? Is it the right type, length, or width? Is it the color specified or does it matter?
· Is the switch high enough or low enough? Is it rheostat control or levered?

Details Beyond Construction
Accurate Quality Control and Quality Assurance processes offer more to businesses and manufactures than product quality. QC and QA procedures test product generation efficiency and cost analysis data, as well.
· Can efficiency of manufacturing of Widget X increase if the business moves the tooling machinery? How quickly will the move pay for itself?
· Can less costly materials be used and still maintain quality standards? Is materiel the cost cutting tool or is the vendor?
· Can Shipping and/or Receiving Departments’ procedural changes assist in Quality Assurance improvements?
· What faster manufacturing process exists?
· Will better lighting help?
· Will additional personnel be worthwhile?
· Will fewer personnel be worthwhile?
· Would more automation improve product quality?
· Are employees encouraged to forward ideas for QA improvement? Are they rewarded or recognized for it?

Costs of Inaccuracy
Inattentive inspections or overlooked processes can cause product errors and faults, slow downs or stoppage of the manufacturing process, and even cause injury. Quality products are not available; sales decline, and the business finds itself unable to maintain its workforce or its market share.

Product Liability
When a product fails, breaks, or injures someone, a lawyer is second on the call list, right after an ambulance or vendor. When potential injury is catastrophic, civil damages can total in the millions of dollars. The Occupational Safety and Health Administration (OHSA) could site the using organization, which in turn might sue the vendor. No business can afford repeated and repeatedly large cash outlays, simply because they failed to ensure availability or accuracy in QA processes. Costs are considerably lower to develop, maintain, and improve quality than they could be due to the lack of quality.

Long-Term Impact
Slipshod Quality Control and Quality Assurance processes that do not detect degrading quality or performance will heavily impact any business’ bottom line. Customers will cancel orders, possibly return products already sold, because no one likes sub-par quality, regardless of the amount paid for it. When products stay on the shelves, no business can survive.

But the opposite can be true, as well. High quality products or services that meet or exceed customers’ needs and expectations can sell for higher prices and more steadily than products of lesser quality can. Ensuring high benchmarks are the Quality Control and Quality Assurances goals and gold standards.

Summary
A huge market may await a business’ product or service, but unless the business can deliver a product that meets or exceeds customers’ expectations in a safe, timely, and effective manner, others claim that market share, and the business is left behind.

Assuring quality of products and services is paramount, and maintaining accuracy and consistency of those standards is vital to keeping or expanding customer bases.

What makes a good Software QA engineer?

* The same qualities a good tester has are useful for a QA engineer. Additionally, they must be able to understand the entire software development process and how it can fit into the business approach and goals of the organization. Communication skills and the ability to understand various sides of issues are important. In organizations in the early stages of implementing QA processes, patience and diplomacy are especially needed. An ability to find problems as well as to see 'what's missing' is important for inspections and reviews.

Testing:
* An examination of the behavior of a program by executing on sample data sets.
* Testing comprises of set of activities to detect defects in a produced material.
* To unearth & correct defects.
* To detect defects early & to reduce cost of defect fixing.
* To avoid user detecting problems.
* To ensure that product works as users expected it to.

Why Testing?
* To unearth and correct defects.
* To detect defects early and to reduce cost of defect fixing.
* To ensure that product works as user expected it to.
* To avoid user detecting problems.

Test Life Cycle
* Identify Test Candidates
* Test Plan
* Design Test Cases
* Execute Tests
* Evaluate Results
* Document Test Results
* Casual Analysis/ Preparation of Validation Reports
* Regression Testing / Follow up on reported bugs.

Testing Techniques
* Black Box Testing
* White Box Testing
* Regression Testing
* These principles & techniques can be applied to any type of testing.

Black Box Testing
* Testing of a function without knowing internal structure of the program.

White Box Testing
* Testing of a function with knowing internal structure of the program.

Regression Testing
* To ensure that the code changes have not had an adverse affect to the other modules or on existing functions.

Functional Testing
* Study SRS
* Identify Unit Functions
* For each unit function
* - Take each input function
* - Identify Equivalence class
* - Form Test cases
* - Form Test cases for boundary values
* - From Test cases for Error Guessing
* Form Unit function v/s Test cases, Cross Reference Matrix
* Find the coverage

Unit Testing:
* The most 'micro' scale of testing to test particular functions or code modules. Typically done by the programmer and not by testers .
* Unit - smallest testable piece of software.
* A unit can be compiled/ assembled/ linked/ loaded; and put under a test harness.
* Unit testing done to show that the unit does not satisfy the functional specification and/ or its implemented structure does not match the intended design structure.

Integration Testing:
* Integration is a systematic approach to build the complete software structure specified in the design from unit-tested modules. There are two ways integration performed. It is called Pre-test and Pro-test.
* Pre-test: the testing performed in Module development area is called Pre-test. The Pre-test is required only if the development is done in module development area.

Alpha testing:
* Testing of an application when development is nearing completion minor design changes may still be made as a result of such testing. Typically done by end-users or others, not by programmers or testers.

Beta testing:
* Testing when development and testing are essentially completed and final bugs and problems need to be found before final release. Typically done by end-users or others, not by programmers.

System Testing:
* A system is the big component.
* System testing is aimed at revealing bugs that cannot be attributed to a component as such, to inconsistencies between components or planned interactions between components.
* Concern: issues, behaviors that can only be exposed by testing the entire integrated system (e.g., performance, security, recovery).

Volume Testing:
* The purpose of Volume Testing is to find weaknesses in the system with respect to its handling of large amounts of data during short time periods. For example, this kind of testing ensures that the system will process data across physical and logical boundaries such as across servers and across disk partitions on one server.

Stress testing:
* This refers to testing system functionality while the system is under unusually heavy or peak load; it's similar to the validation testing mentioned previously but is carried out in a "high-stress" environment. This requires that you make some predictions about expected load levels of your Web site.

Usability testing:
* Usability means that systems are easy and fast to learn, efficient to use, easy to remember, cause no operating errors and offer a high degree of satisfaction for the user. Usability means bringing the usage perspective into focus, the side towards the user.

Security testing:
* If your site requires firewalls, encryption, user authentication, financial transactions, or access to databases with sensitive data, you may need to test these and also test your site's overall protection against unauthorized internal or external access.

Test Plan:
* A Test Plan is a detailed project plan for testing, covering the scope of testing, the methodology to be used, the tasks to be performed, resources, schedules, risks, and dependencies. A Test Plan is developed prior to the implementation of a project to provide a well defined and understood project roadmap.

Test Specification:
* A Test Specification defines exactly what tests will be performed and what their scope and objectives will be. A Test Specification is produced as the first step in implementing a Test Plan, prior to the onset of manual testing and/or automated test suite development. It provides a repeatable, comprehensive definition of a testing campaign.