I met with my team’s automation experts a few weeks back to get their input on when to automate and when to manually test. The general rule of thumb has always been to use common sense. If you’re only going to run the test one or two times or the test is really expensive to automation, it is most likely a manual test. But then again, what good is saying “use common sense” when you need to come up with deterministic set of guidelines on how and when to automate?
Pros of Automation
• If you have to run a set of tests repeatedly, automation is a huge win for you
• It gives you the ability to run automation against code that frequently changes to catch regressions in a timely manner
• It gives you the ability to run automation in mainstream scenarios to catch regressions in a timely manner (see What is a Nightly)
• Aids in testing a large test matrix (different languages on different OS platforms). Automated tests can be run at the same time on different machines, whereas the manual tests would have to be run sequentially.
Cons of Automation
• It costs more to automate. Writing the test cases and writing or configuring the automate framework you’re using costs more initially than running the test manually.
• Can’t automate visual references, for example, if you can’t tell the font colour via code or the automation tool, it is a manual test.
Pros of Manual
• If the test case only runs twice a coding milestone, it most likely should be a manual test. Less cost than automating it.
• It allows the tester to perform more ad-hoc (random testing). In my experiences, more bugs are found via ad-hoc than via automation. And, the more time a tester spends playing with the feature, the greater the odds of finding real user bugs.
Cons of Manual
• Running tests manually can be very time consuming
• Each time there is a new build, the tester must rerun all required tests - which after a while would become very mundane and tiresome.
Other deciding factors:
• What you automate depends on the tools you use. If the tools have any limitations, those tests are manual.
• Is the return on investment worth automating? Is what you get out of automation worth the cost of setting up and supporting the test cases, the automation framework, and the system that runs the test cases?
Powered by Blogger.
CMM Level 5 companies list List of CMM-5 Certified Software Service Companies in India Listed in no particular order. The purpose of this l...
A formal technical review is a software quality assurance activity performed by software engineers (and others). The objectives of the FT...
The test case design techniques are broadly grouped into two categories: Black box techniques, White box techniques and other techniques tha...
The spiral model, originally proposed by Boehm , is an evolutionary software process model that couples the iterative nature of prototyping ...
The incremental model combines elements of the linear sequential model (applied repetitively) with the iterative philosophy of prototyping. ...
Often, a customer defines a set of general objectives for software but does not identify detailed input, processing, or output requirements...
The spiral model suggests a framework activity that addresses customer communication. The objective of this activity is to elicit pro...
Rapid application development (RAD) is an incremental software development process model that emphasizes an extremely short development cycl...
V-Model: The V-model promotes the idea that the dynamic test stages (on the right hand side of the model) use the documentation identifie...
Severity Value : S1 : Catastrophic Blocking : The use case cannot be completed with any level of workaround. Problem causes data loss, corr...
- A Quick 10-Step Guide (1)
- Black Box Testing (3)
- Bug Life Cycle (2)
- Certifications (3)
- CMM level (2)
- Comparsion (1)
- Configuration Management (3)
- Cookie Testing (1)
- Defect and Failure (1)
- Functional and Non-Functional (1)
- Functional Testing (1)
- Inspection and Walkthrough (1)
- Interview Software Testing (1)
- ISTQB Question Paper Dump (3)
- Load and Stress Testing (1)
- QA and QC (3)
- QA vs QC (1)
- Regression vs Retesting (1)
- RTM (1)
- SDLC (2)
- SDLC Model (5)
- Severity and Priority (1)
- STLC (2)
- Test Cases (4)
- Test Entry and Exit Criteria (1)
- Test Plan (4)
- Types of Testing (2)
- V Model and W Model (1)
- Validation and Verification (2)
- Web Testing (2)
- White Box Testing (1)
- bipin singh
- Different types of testing
- QualityAssurance & QualityControl
- Bug Life Cycle
- Software Testing Life Cycle
- Different Types of SDLC
- Regression vs. Retesting
- Negative Test Cases
- Load Testing & Stress Testing
- When to use Automation testings?
- What is Test Plan?
- What should be done after a bug is found?
- How can it be known when to stop testing?
- What if there isn't enough time for thorough testi...
- Validation and Verification
- Walkthrough & Inspection
- SEI , CMM & CMMI
- Test Cases, Suits, Scripts and Scenario
- What makes a good Software QA engineer?
- Black Box Testing
- White Box Testing
- What's the difference between Black and White box ...
- Requirements Traceability Matrix
- The Certified Software Tester
- Soak Testing and Bucket Testing
- ISTQB Certifications
- Define Defect?
- What is the difference between Defect and Failure?...
- What is the difference between Latent and Masked D...
- In testing can you explain the concept of work ben...
- What’s the difference between Alpha and Beta testi...
- Can you explain the concept of defect cascading?
- Can you explain what is Usability testing?
- What is the difference between Pilot and Beta test...
- How will you do a risk analysis during software te...
- What does entry and exit criteria mean in a projec...
- On what basis is the Acceptance plan prepared?
- what Is coverage and what are th...
- What are different types of verifications?
- What’s the difference between Inspections and Walk...
- What is the difference between Software Testing an...
- What is configuration management?
- Can you explain regression testing and confirmatio...
- Can you explain the concept of baseline in softwar...
- What are the different test plan documents in proj...
- Can you explain calibration?
- Which test cases are first written white boxes or ...
- Can you explain Co-habiting software?
- TEST PLAN OUTLINE
- Version control
- Change control
- CMM Level 5 companies list
- People CMM
- Prototype Model
- Rapid application development (RAD)
- The Incremental Model
- The Spiral Model
- The WINWIN Spiral Model
- THE PROJECT PLAN
- How to test a website by Manual Testing?
- Automation Testing v/s Manual Testing Guidelines:
- Probe Testing
- Defect leakage
- FORMAL TECHNICAL REVIEWS
- Cost of Quality
- Traceability Matrix
- ▼ December (65)
- ► 2011 (31)