Because of my work as a project recovery consultant, I talk a lot about testing, continuous integration, continuous delivery, continuous deployment, and DevOps. During these conversations, the term automation gets thrown around in abundance.
In general, we all understand what automation means – the use of some technology to complete a task that was done manually before. But when we talk about automation in terms of testing, there are some nuances that we have to make.
Two types of automation
In the world of testing in general, and continuous delivery in particular, there are two types of automation:
1) Automated Tests
2) Test Automation
While it might just seem like two different ways to say the same thing, these terms actually have very different meanings.
Automated tests are tests that can be run automated, often developed in a programming language. In this case, we talk about the individual test cases, either unit-tests, integration/service, performance tests, end-2-end tests or acceptance tests.
Test automation is a broader concept and includes automated tests. From my perspective, it should be about the full automation of test cycles from check-in up-to deployment. Also called continuous testing.
Some linguistic analysis might also help clarify:
Automated Tests – in this case, «Tests» is a noun; the test is a thing. «Automated Tests» are particular types of tests; ones whose execution has been automated via some kind of code so that a person does not have to manually execute the test. You can have 1, 2, 3…. hundreds of automated tests.
Test Automation – this can be a noun as in the subject of «Automation» with «Test» indicating the type of automation. But, it is also referring to an activity; the activity of automating a given test or set of tests. As said previously; «Test Automation» is a broader concept than «Automated Tests».
Essential for continuous delivery
In a traditional environment, testing gets completed at the end of a development cycle. But as more and more companies move toward a DevOps and continuous delivery model in which software is constantly in development and must always be deployment-ready, leaving testing until the end no longer works. That’s where continuous testing comes in – to ensure quality at every stage of development.
While ensuring quality at all times is very important, it’s not all that counts. The speed at which all of the integration and testing occurs is very important as well. That’s because if something in the pipeline stalls or breaks down, it holds up everything else and slows down the release of new developments, which defeats the purpose of continuous delivery.
This «how» and «why» make organization, consistency and speed imperative to support a continuous delivery model, and that’s where test automation can help.
How automated test and test automation help
Managing all of the testing needs in a continuous delivery environment is a massive undertaking – it requires a tremendous communication effort to keep track of which environments have deployed new code, when each piece needs testing and how those requirements integrate back into the moving process of continuously delivering software. A simple continuous delivery pipeline could look like this:
1) Continuous integration server picks-up changes in the source code
2) Starts running the unit-tests
3) Deploys (automated) to an integration environment
4) Runs automated integration tests
5) Deploys (automated) to an acceptance environment
6) Runs automated acceptance tests
7) Deploys (automated or manual) to production
This is a combination of making automated tests by developers or testing engineers and DevOps people automating deployment. In doing so, test automation goes a long way toward helping ensure that teams maintain a high standard of quality at all points along the pipeline.