Test Automation – Getting the most out of open-source

QE is more challenging these days due to the numerous browser, device, and platform combinations that are necessary to test. Furthermore, software release frequency has increased many fold. It used to be once in a month or quarter, but it has come down now to weeks or even every few minutes. To make sure that software quality is not compromised, the QE strategy needs to enable testing in a very short window. Manual testing in such scenarios will not be possible, and the only way to achieve 100% test coverage is to leverage test automation.

To fill this need, many commercial and open-source testing automation tools are available. Commercial tools come with significant cost, and for SMB’s this has been the biggest hindrance in using them. Open-source tools generally are available at no cost and have accordingly gained significant market traction market over the last few years. More and more organizations are leveraging open-source tools due to the many contributors in the open-source communities which provide solutions and help the automation teams. Open-source tools provide automation testers with the flexibility to customize the tool and build hassle-free solutions to their business needs.

To get the most out of any open-source test automation tool, it is important to have a robust test automation framework. The framework is the heart of any test automation project. With the variety of platforms available in the market today, users prefer to access applications per their convenience on web, mobile phone, tablet, and other devices.  It is particularly important that the framework be capable of supporting major web browsers like Chrome, Firefox, Edge, and Safari as well as mobile platforms like Android, iOS, etc. The framework should support scripting in such a way that with minimal changes, a single script can support all browsers and platforms. The framework should also scale to support new technologies or third-party components being integrated into the software being tested.

The effort to maintain an open-source tool can be a bit high. Maintenance efforts can be brought down, however, by using a well-structured framework and following established standards, conventions, and processes. Reviews should also be part of the process to make sure that everyone is following the processes, standards, and conventions and appropriately optimizing code. Through these means, maintenance effort can be significantly minimized.

Open-source tools usually lack quality reports, as most of these tools provide logs rather than a comprehensive report. However, an automation framework can compensate for this shortcoming by creating customizable, comprehensive test execution reports in HTML or PDF format. These reports can show all the details needed to debug the script in case of a failure and can be easily understood by all team members. The reports can even include a dashboard showing the count of test cases executed, passed, and failed together with the platform, browser, and device used for test execution. The reports can also include actual step results, test case data, and the screen shots of every step.

Using an open-source tool provides the following benefits:

• • Cost effective
    Open-source tools are more cost-effective than commercial counterparts. They allow organizations to save costs without compromising on quality. With open-source tools, you don’t need to worry about any recurring license or tool maintenance costs.

• • Evolving continuously
    Continuous improvement is something you can expect from the open-source tools. Developer communities keep upgrading these tools, fixing bugs, and making tweaks. Updates are sent out on a regular basis. Anyone can get the source code and work on it based on their skill set.

• • Flexible (no vendor lock-in)
    With an open-source automation tool, organizations avoid the risk of vendor lock-in. The testing team have the flexibility and freedom to switch to another tool. Moreover, with open-source tools, automation testers can quickly evolve to ever-changing requirements.

• • Documentation and online support
    Today even open-source tools provide very good documentation for tool set up, use, and features. On top of this, many online contributors provide support to users on a regular basis and keep the tool code base up to date.

• • Ability to start small
    Open-source test automation tools enable SMEs to start small and test the open-source software on a smaller scale. This is not only cost-effective but also has other benefits. Companies can assess if the solution is truly the best one for them and later scale it according to their needs.

• • Attracting Talent
    Using open-source software allows organizations to attract better talent. Also, it gives a lot of opportunities for QE personnel to learn and advance in their careers. Many testers enjoy the opportunity to learn test automation and work on it. Since the demand for QE personnel is greater than the supply, recruiters have a hard time finding high-quality talent to work in their organizations. Using open-source tools might be the key to resolving that problem.

• • Scaling and Consolidating
    It’s easier to scale and consolidate using free tools than it is with paid tools for various reasons. Open-source tools can be seamlessly integrated with test management tools, collaboration tools, and CI/CD tools.

• Quality
  Open-source tools are no longer just a cheap alternative to proprietary software. Open-source communities have grown quality conscious over the years. Today’s open-source software is more robust and offers better quality than its proprietary counterparts. Open-source software results from the collaboration of hundreds of developers worldwide with diverse levels of experience and expertise. Good practices followed include TDD, code reviews, code refactoring, multi-testing, and feedback stages. Continuous integration, alerts and monitoring, test days, and bug bounties have helped improve the quality of the open-source tools.

Conclusion
As more open-source tools are gaining traction in the market, it is very important to select the tool that meets your requirements and supports the latest technologies. Apart from the tool, importance should also be given to designing a framework so that the development and maintenance is easy for the users and so that it integrates well with other tools.

ABOUT THE AUTHORS
Vikas Shukla
Director – Quality Engineering
Celsior

Himanshu Gosain
Senior QE Architect
Celsior

Read other blogs of the series to get more insights on Quality Engineering, QE services, and QE Automation:

Moving from Quality Assurance to Quality Engineering
Adding AI capabilities to a Test Automation Framework
Mobile Test Automation – How to get the most bang for your buck
ETL Data Validation – Better decision making through improved data quality
Role of Quality Engineering within DevOps and CI/CD