Software Testing Automation: A Comparative Study on Productivity Rate of Open Source Automated Software Testing Tools For Smart Manufacturing

Abstract

With the growth in emerging technologies, manual software testing possesses a challenge to project team in terms of test planning, test case creation, execution and reporting. Software testing automation with open source tools promises productivity savings for software test engineers. The objective of this study is to compare the level of productivity of open source Smart Manufacturing toolkits for web application testing. Together with literature analysis, subject matter experts in software testing were interviewed and were requested to experiment and compare the productivity rate of two tools in use for the various steps in the software development lifecycle. Katalon Studio 7.0 was found to be a more smart and productive tool than its predecessor, Robot Framework 3.0 in terms of installation, scripting, verification and maintenance for web application testing. By employing a Wilcoxon Matched Pairs Signed-Rank Test, results revealed that the null hypothesis of “no statistically significant differences between SME productivity rate in using RF and KS to performing software test automation in terms of time taken” can be rejected , Z= -6.399, p <; 0.05. Future work could replicate similar studies for selecting mobile manufacturing toolkits that will be suitable and improve productivity rate.