Six Stages from Manual to Autonomous Testing
Key testing tool features in TestComplete to help you go from manual to advanced functional testing
Start Automating Today
Introduction: AI for Faster, Smarter Testing Tomorrow
A decade ago, artificial intelligence (AI), machine learning (ML), and even automation were barely fathomable concepts in the software testing industry. Automated testing has since become the cornerstone practice for teams looking to implement modern development approaches like Agile and DevOps, or to simply keep up with the increasing market demand for high-quality software.
Teams can now tactically implement automated testing to reap the benefits of reduced testing cycles, faster times to market, improved test coverage and accuracy, and enhanced team efficiency. AI and ML have even begun to weave their way into testing and development, with automated testing tools incorporating intelligent algorithms into product features that can be leveraged by QA teams to enhance testing.
Despite the acknowledged benefits of automation, less than half of QA professionals are automating more than 50% of their tests. Graphical user interface (GUI) tests are notoriously difficult to automate, and as a result, the majority of GUI test cases are still conducted manually. For QA teams responsible for GUI testing, 42% will run their tests via a combination of both manual and automated testing, with 31% running their tests solely via manual execution.
Moving from manual testing to automation can be daunting for many software teams. The transition between the two approaches requires not only a change in regulations and infrastructure, but also mindset – especially when it comes to anything that is process-related. Most importantly, incorporating automation into testing cycles demands a guarantee that it is just as reliable and accurate as manual testing.
Six Stages from Manual to Autonomous Testing
The Society of Automotive Engineers (SAE) describes the transition to fully autonomous vehicles in six stages split between two phases of driving: monitored driving and non-monitored driving.
Monitored vs. Non-monitored Driving
The first three levels focus on the driver being responsible for monitoring the car and making decisions. In the latter three levels, the driver starts to relinquish controls to not being needed at all (i.e., self-driving cars). The biggest transition for the driver is between monitored and non-monitored testing.
Advanced Testing vs Autonomous Testing
Transitioning from driver to driver-less capabilities not only applies to the autorotative industry, but also software development and testing. With each stage, tool capabilities in software testing provide QA engineers and testers with the ability to rely on automation to speed up software development.
The first four stages in the transformation process from manual to autonomous testing focus on building advanced automation capabilities. The tester, or team, moves from manually building, executing, and maintaining tests to leveraging an automation tool to assist with those aspects of their testing cycle.
In the latter two levels, teams will start to accelerate their testing processes and relinquish control to the tool, and in an idealistic near future, reach the final stages where tests can be automatically created and executed without human intervention.
Deep-Dive into the Six Stages
Walking through the six stages will help you understand how to move from manual to autonomous testing. At each stage, we will cover:
- Definitions to describe what each stage entails
- Features in TestComplete to help optimize UI testing during each stage
- Best Practices to help progress to the next stage
Stage 1: Manual Testing
Stage definition: Manual testing is the process in which a tester acts as the end user to validate that a piece of software works and behaves as expected. The goal of the tester is to find defects within the application by physically clicking or interacting with the product.
In manual testing, the tester is in complete control of the test scripts at all times. They are responsible for test creation, execution, maintenance, monitoring, and fallback scenarios for when tests fail. More than half of all testing today is done manually, most of it on the UI layer. While the use of API and automated API testing grows, there seems to be a larger struggle for teams transitioning to test automation on the UI layer.
Stage 1 Best Practices:
- Be an Application Expert: The number one best practice that should occur throughout your transformation and especially when conducting manual tests is to understand the intricacies of your application.
- This will help ensure a quality user experience and instill confidence in your application’s behavior after each build. The top two ways to do this is to read documents about the application beyond just the requirements and to conduct exploratory testing to understand the most critical user scenarios that are important to your customers.
Stage 2: Assisted Automation
Stage Definition: The first step to ensuring a complete and successful transformation is with assisted automation. In this stage of automated testing, a quality assurance (QA) team or individual tester will leverage an automation tool to assist them with test creation and execution. However, at the end of the day, the tester is still required to manage and maintain the test scripts manually to ensure quality.
This is a critical stage for moving from manual to automated testing because it required you to define your automated testing framework. The more organized and structured an automated testing framework is, the more efficient, stable, and scalable your automated testing efforts will become.
The Importance of Object Identification in Automated UI Testing
Most tools on the market today have features that will address your needs at this stage. For UI testing specifically, it’s vital to the success of any automated process that the tool has robust object recognition and object storage capabilities (i.e. an object repository).
Automated UI testing tools, including TestComplete, work by finding objects through the Document Object Model (DOM) layer. The tool will capture elements that exist on the user interface. Each object captured has properties or attributes that can be used to describe what the user is seeing in the application and where it lives, such as number of pixels, colors, handler ID, and text.
These objects are stored in an Object Repository and can then referenced in automated tests via script. Having the ability to accurately locate, identify, and test objects on the screen is central to ensuring your UI tests function properly and provide enough test coverage to instill confidence in your application’s behavior.
Automated Testing Tool Features to Leverage in TestComplete
Popular tool features that can assist with testing at this stage, especially for those new to automation, are those that can accurately identify objects and enable testers to create and run tests without having to write scripts.
Object Spy: It’s essential to your success that your automated UI testing tool is able to accurately identify and store objects on your user interface. In TestComplete, the Object Spy will enable you to explore the properties and methods of any UI element in your application, which can be added to its object repository and hierarchy referred to as Name Mapping.
Record and Replay: Record and replay features allow testers to build UI tests by recording on-screen user actions such as keystrokes, typing, and clicks, and then playing them back with different scenarios to compare results against the expected behavior. The benefit of this technique is that it requires little to no scripting skill, meaning anyone can create and run automated tests. In TestComplete, getting started with test automation is as simple as clicking Record on the Record and Replay toolbar.
Keyword-Driven Testing: In keyword-driven testing, automated UI tests consist of a myriad of words (keywords) that define an action to be performed in that test. These can be actions mentioned previously, such as mouse strokes, clicks, but also dragging and dropping items or opening and closing application windows. For those just getting started with automation, the easiest way to create keyword-driven tests is by recording them with Record and Replay.
In TestComplete, you can combine a series of keywords to create an operation, or the tests. In the image below you can see the operations, or test actions, on the left, and the test case on the right. TestComplete makes it easy for teams or individuals to create complex automated test scenarios with simple point-and-click actions without any expertise in scripting languages.