How to Create a Test Automation Framework from Scratch
As quality initiatives shift left in an agile world, software testing teams realize that they must rely on the fast feedback of automated scenarios for continuous integration and deployment. But even when people understand test automation basics, most automation initiatives fail. There are many reasons why test automation initiatives fall short, but the main reason is a poorly designed architecture for the framework on which the automation was built.
Test automation is living, breathing code, and is developed to provide sanity checks for ever-changing production code. When viewed in that context, it is obvious that test automation code must be reliable, and therefore should be developed with a comparable standard to that which is being shipped.
In addition to being reliable, automation frameworks should also be flexible, and easily extendable. A framework that’s only capable of user interface (UI) automation, for example, places a limit on the reach of a test and can cause for slow and brittle tests.
Are you ready to introduce test automation to your organization? Here’s an overview of the architecture for an automation framework that is stable, robust, flexible, and agile-friendly.
Separate tests from your automation framework
Automation framework consists of the integration of tools, libraries, and utilities needed to interact with the system under test. It should be a neutral entity that gathers information about the application by way of the system’s UI, web services, business logic, databases, and so on.
Automated tests are scripted scenarios that drive the framework to query the application through a series of steps, and then use the information the framework gathered to determine the state of your test.
To promote reusability across tests, the framework, and the tests, should be structurally separate within the automation code base. That means there should be one package that holds all of your framework code, and a separate package that holds the test code. The framework code should include methods that interrogate the application, and return what it finds. The test code should call a sequence of framework methods, and based on the information returned, pass or fail it.
This separation of intentions will keep your code clean, and easier to maintain and extend. The same framework methods can now be used in both positive and negative tests because the framework is a neutral party, and simply provides information about the system. The test can use the information returned from the framework for its own purposes.
Using page object models
For UI applications, you need classes that interact with the pages of the system. These classes should live within the framework layer. The most popular design pattern for creating these classes is the page object model (POM). This model recommends creating a separate class for each page of your application. Within a given class resides the elements of that page (e.g. buttons, text fields, etc), as well as methods for interacting with those elements. You can use a browser automation tool, such as Selenium WebDriver, to handle the actual interaction.
Let’s envision an application’s typical login page. There is a text field for the username, a text field for the password, and a button to submit.
Using the POM, the automation framework would include a class called LoginPage. LoginPage would include handles to the elements:
- usernameField
- passwordField
- submitButton
- errorMessageLabel
and would also include methods such as:
- setUsername
- setPassword
- clickSubmit
- getErrorMessage
It’s good practice to include more compound methods that make calls to smaller methods, such as:
|
1 2 3 4 5 6 |
public HomePage login(String username, String password){ setUsername(username); setPassword(password); clickSubmit(); return new HomePage(); } |
In the test layer, a separate class can be created for tests related to logging in, for example, LoginTests. In this test class, you should create a new method for each test and the methods should be totally independent, meaning they should not rely on anything that might have happened in one of your other test methods. This helps to eliminate brittleness.
Also, because the framework is neutral, the same framework methods can be called for positive and negative tests.
Examples of methods that LoginTests might include:
|
1 2 3 4 5 6 7 |
@Test public void testValidLogin(){ LoginPage loginPage = new LoginPage(); loginPage.login("janedoe", "bad_password"); Assert.assertEquals("Error message", "Invalid username or password", loginPage.getErrorMessage()); } |
The role of inheritance in test automation code
Inheritance, an object-oriented programming principle that enables objects to receive properties from a parent object, also has its place in your automation test code.
In the test layer, every UI test needs to launch the browser. Rather than duplicating your code in every test method, you can use this functionality to extract it out, and place it into a method that runs before each test. To ensure that this method does not get duplicated in every test class, place the method in a base test class from which all test classes inherit. Test runner tools, such as JUnit and TestNG, provide annotations that you can use to denote methods that should run before the tests. They also provide “after” annotations that you can use and can be inherited in the same way to clean up after tests.
You can also use inheritance in the framework layer. For example, if you’re using Selenium WebDriver, every page object class needs a handle to the WebDriver instance in order to interact with the application. One way to accomplish this is by setting up a base page class that has the WebDriver instance as a protected member, and then have the page object classes inherit from the base page.
The base page can also contain objects for other parts of the application that are visible from any page, such as navigation menus, headers, and footers. All of these would be inherited by any page, and therefore accessible without duplicating code.
Look beyond the UI
One sign that an automation framework may need some tweaking is the time it takes for tests to execute. As teams are using the feedback from the tests in continuous integration pipelines, time is of the essence, and fast feedback is a must. One way to cut down the test execution times is by limiting UI testing to only what really needs to be verified on the UI.
However, to accomplish this, you can’t limit the framework to only browser automation tools. Adding a sub-package to your framework layer for web services is a very beneficial approach.
Some automated tests may not need to exercise the UI. It may be sufficient to verify the functionality by way of a web service call, for example. Or perhaps a UI test scenario requires some setup or cleanup that would be more efficiently handled by web service calls. These can be added as service utilities in a WebServicesUtils class, and used by test classes, just like with page object classes. This can help reduce execution time, and eliminate a lot of the brittleness that inherently comes from UI automation testing.
Consider incorporating behavior-driven development
Behavior-Driven Development (BDD) is a process where team members use domain-specific language to express the expected behavior of an application in the form of scenarios. This activity is ideally done between the business analyst, developer, and tester to foster communication to flesh out any missing details or misconceptions about a feature to be developed.
Tools are available that you can integrate with automation code to execute these natural-language scenarios. This is ideal for teams that want to share automation work between developers and testers. By writing scenarios in the domain-specific natural language, testers are essentially writing automated scenarios without needing to code. But of course, this is not magic. A new layer of code is needed to serve as the glue between the scenarios and the framework. This layer essentially translates what’s being written in the natural language, and calls framework methods that correspond to that intention.
Adding a BDD layer to the automation framework should be a decision that is carefully considered. If the team is not actually practicing behavior-driven development for communication purposes and is only using BDD for automation, this extra layer of coding is not as beneficial, and becomes yet another complexity within the framework.
But if your team agrees that this is a good approach, they can easily add a BDD layer to the existing automation code base. Assuming that Cucumber is your BDD automation tool of choice, here are the steps to add the capability.
- Within the test package, add a new sub-package for BDD.
- Under that sub-package, add a folder to hold the scenarios (i.e. feature files) and also add a sub-package where the glue code (i.e. step definitions) resides.
- Also within the BDD sub-package, you need to have a class that’s responsible for executing the scenarios. This class will identify where the feature files and step definition code is.
Using this information, the Cucumber runtime engine can execute the scenarios.
Next steps: Build your own automation framework
Clean code is essential to the success of an automation framework. Neutralizing the framework, and separating it from the tests, is the first step to ensuring a clean design.
Using design patterns such as the page object model allow for easy maintenance and extension. And adding additional layers for service testing and BDD execution will give you the flexibility you need for stronger and quicker testing.
