At last week’s STARWEST conference, I attend a number of sessions focused on testing automation. The topic dominated most of the conference, as more and more dev/test teams are being tasked with building better software faster—so its natural that automation and DevOps are two areas being looking into as potential means to this tackling this challenge.
One session in particular that I was really looking forward to was Ramandeep Singh’s “End-to-End Test Automation with Open Source Technologies.” Skytap has always been a big supporter of open source technologies: from our product’s Jenkins plugin, to our own engineering team’s recent completion of an “un-opinionated, ultra-minimal web framework for Node.js.”
Ramandeep’s presentation was one of the highest attended of all of the track sessions that I sat in on all week, and while obviously a huge supporter of open source technologies, he was quick to point out that due to their inherent independence, they do have their limitations, especially around an often lack of integration.
The primary focus of Ramandeep’s session was to display a framework available from his employer, QAInfoTech, that integrates open source tools like Selenium, Calaba.sh, Sikuli, Spock, Genie, and numerous others, in order to give testers end-to-end coverage across web apps, flash/flex apps, mobile apps, and APIs.
I had the chance to meet Ramandeep and ask him a few questions before his appearance at STARWEST.
Noel: You mention in the abstract for your session that agile methodologies are responsible for getting testers involved earlier in product testing. What other trends or focuses in the software business today might also be responsible for this shift?
Ramandeep: Test teams are looking for testing upstream at APIs, integration and contact points as much as possible. The trend has started because of better understanding of automation processes and the evolution of testers with the right skills. Also, because this gives a better opportunity to deliver quality features per sprint/iteration through early bug detection. And as the testing pyramid suggests, brittle UI tests require more maintenance vs. tests at the services or API level. All of these facts and learning have made test teams adapt to more upstream testing.
Noel: What exactly is “end-to-end automation” and what are some of the misconceptions about automated testing that people should know are not true?
Ramandeep: End-to-end automation is essentially being able to automate all different forms of a software application— i.e. web app, mobile app, web services, content, etc., and also being able to automate across workflows. For example, a user may begin a transaction on his laptop, but ends up completing the workflow on his mobile device. My presentation proposes a solution that it makes it easy and cost effective to build and maintain end-to-end automated tests across different application forms and compatible devices.
Noel: Another point in your abstract that you brought up that I found really interesting is that you say that open source solutions are “available in abundance” but that they often “increase the tester’s work around test automation development.” I would assume that’s the exact opposite of what testers are looking for – more work in setting these up. It seems like one would expect that some of the manual, time-consuming tasks of old would be eliminated by automation.
Ramandeep: Manual tasks are certainly eliminated in huge numbers when you automate, but the way most of the automation tools are designed, testers using them use them often are required to have programming skill sets. And then with different forms of the application, different tools are used to automate them.
So testers building automated tests need to learn and use all of those tools and then maintain the automated test suite as the application under test evolves through its development cycles. All of the tools would require such maintenance on automated tools. But we can limit the amount of maintenance required by carefully planning, designing, and implementing a neat framework around all these tools. My solution is one such framework.
