Why Keyword Driven Test Automation is not way forward for Agile – 2

This post is the second article in a three article series where I discuss about approaching test automation and why conventional methods might not be the way forward, specifically in agile environments.

In my first article I discussed in detail how taking an approach of defining keywords based on application functionality is not an optimal approach in agile environments. This article digs deeper into the second conventional approach to Keyword Driven test automation where the test scripts are created using keywords based on the low level/action level of the automation engine.

Using low level/action level keywords is the quickest way to have the automation test scripts created(and some tools also provide advanced options to record steps, associate data and object maps). Some tools also provide a good documentation when following such an option.

Thus, this approach might seem more suited for agile, since the test scripts can be created quickly based on the UI flows of the application.

Although using this approach has its advantages in an Agile environment, there are few issues with this approach:

• The action level steps depend on an AUT(Application Under Test)
• The documentation provided by these keywords does not evidently provide insight into what behavior of the application is being tested

The first issue, as stated above, is that you need an application under test, with a certain level of stability(in flows and in the UI properties) to create the tests. Since the approach is dependent on the screens of application and the objects in application UI, it still is a Shift right approach where testers can create tests only once application is developed(in most cases).

Any changes in the application flow would mean, the scripts should be changed(or quickly create new scripts supporting the changes). It might result in large maintenance activity(although initial time spent on identifying keywords is very minimal or zero).

Second issue, is that the documentation these test scripts provide are at a action level and it is very difficult to understand what acceptance criteria is being tested looking at the test steps(as the steps are more about action being performed). A documentation at a level above this should then be maintained to represent what specific feature the test script is testing.

The manual testing team, will still have to maintain separate assets(manual test cases and automation test scripts), since the test scripts are not ready till application is available and the test scripts do not provide documentation at the level manual testers would need. This still is a Shift Right approach to testing.

I will in the next post discuss what I envision as an optimal approach to test automation in Agile environments, what existing approaches currently match that criteria and how they can be utilized and customized to suit the Agile test automation approach

I would love to hear from you, What in your experience, have been the disadvantages of using Keyword driven frameworks in an Agile environment?