What kind of tests should NOT be Automated?

• Usability Testing – at times this can be an impossible task to perform by automation as the computer cannot efficiently judge if the system is of any use to its users
• Tests that only need to be executed once – unless the same test needs to be executed for a large dataset then it makes sense to automate
• Tests without predictable results – test automation should give us confidence in the results of the tests. If there are intermittent failures then the tests cannot be reliable and cannot be dependent on
• Tests that need to be verified visually
• Tests that need to be executed quickly. At first, writing an automated test takes longer. If we want a quick check, we should test manually, however, if that test is a good one which should be run regularly, then it should be automated in time

Does Automation replace manual testing?

Automation is the integration of testing tools into the test environment in such a manner that the test execution, logging, and comparison of results are done with little human intervention. A testing tool is a software application which helps automate the testing process. But the testing tool is not the complete answer for automation. One of the huge mistakes done in testing automation is automating the wrong things during development. Many testers learn the hard way that everything cannot be automated. The best components to automate are repetitive tasks. So some companies first start with manual testing and then see which tests are the most repetitive ones and only those are then automated.

As a rule of thumb do not try to automate:

1. Unstable software: If the software is still under development and undergoing many changes automation testing will not be that effective.
2. Once in a blue moon test scripts: Do not automate test scripts which will be run once in a while.
3. Code and document review: Do not try to automate code and document reviews; they will just cause trouble.

The following figure shows what should not be automated.

All repetitive tasks which are frequently used should be automated. For instance, regression tests are prime candidates for automation because they're typically executed many times. Smoke, load, and performance tests are other examples of repetitive tasks that are suitable for automation. White box testing can also be automated using various unit testing tools. Code coverage can also be a good candidate for automation.