Analysis of Automated Web Testing Frameworks Based on Performance and Usability

Abstract

Advancements in information technology in recent years have accelerated digital transformation across various business sectors. This is reflected in the increasing use of web-based applications that require systems to be responsive, stable, and reliable. However, software testing processes are still often carried out manually, making them less efficient due to the time required, high resource consumption, and the potential for human error, especially when applications undergo continuous feature updates. Therefore, a more effective testing approach is needed through automation testing. This study aims to analyze and compare several automation testing framework to determine the most suitable framework for web application testing. Test scenarios are developed using a Domain Specific Language approach based on Cucumber Gherkin to ensure better readability and structure. The evaluation is conducted based on two aspects main, namely Automated Testing Progress and Tools Usability. The performance aspect includes time complexity, covered test cases, execution start, and element inspection. Meanwhile, the usability aspect covers platform compatibility, supported browsers, scripting languages, and parallel execution. The case study is conducted on a web-based logistics application, muatmuat.com, focusing on the register, login, and reset password features. The data were analyzed using the Multiple Attribute Decision Making (MADM) method based on the Distance to the Ideal Alternative (DIA) approach. The results show that Selenium WebDriver has an Ri value of 0.25799, followed by Cypress (0.31747), as well as Playwright and WebDriverIO (0.33426), making it the most optimal framework.