Evaluation of E-VAKU Application Usability using System Usability Scale and Cognitive Walkthrough Methods

I Made Reyvinno Dirga Narke, Deny Wiria Nugraha

Abstract


The Central Sulawesi Provincial Government, in line with digital transformation and e-government initiatives, has implemented the E-VAKU application (Electronic Financial Accountability Verification). This digital system is designed to address bureaucratic challenges and enhance efficiency and transparency in the verification of regional financial accountability. Given that the success of an information system is strongly influenced by its usability, this study conducts a comprehensive usability evaluation of the E-VAKU application using a combination of two methods. The System Usability Scale (SUS) is employed to quantitatively measure user perceptions, while the Cognitive Walkthrough (CW) method is applied to identify specific issues within workflows and user interface interactions. The SUS evaluation produced an average score of 74.25, which falls into the “Good” category with a grade of B. Meanwhile, the CW analysis recorded a success rate of 90%, an error rate of 12%, and a time-based efficiency of 0.0205 tasks per second. The findings from both methods resulted in specific improvement recommendations, including interface redesigns for the onboarding, login, dashboard, and Payment Order (SPP) pages, as well as the addition of notification and search features. These enhancements aim to make the E-VAKU application more intuitive and user-friendly. This study is expected to serve as a practical reference for developers in refining the application to support more effective and optimal governance practices.

Keywords


cognitive walkthrough (CW); e-government; E-VAKU application; system usability scale (SUS); usability

Full Text:

PDF

References


] Hartanto et al., "Transparency dan Accountability berpengaruh terhadap Persepsi Publik dalam Menilai Kinerja Pemerintah Daerah," Jurnal Ilmiah Rekayasa dan Manajemen Sistem Informasi, Vol. 7, No. 2, pp. 125-132, 2021.

] M.A. Kosim, S.R. Aji, dan M. Darwis, "Pengujian Usability Aplikasi Pedulilindungi dengan Metode System Usability Scale (SUS)," J. Sist. Inf. dan Sains Teknol, Vol. 4, No. 2, pp. 1-7, 2022.

] I. Jaelani, "Uji Usability dengan Metode Cognitive Walkthrough dan System Usability Scale (SUS) pada Situs Web STT Wastukancana," INTECOMS: Journal of Information Technology and Computer Science, Vol. 4, No. 1, pp. 30-39, 2021.

] S. Shintia, M.G. Resmi, dan M. Hafid T, "Pengujian Usability dengan menggunakan Metode Cognitive Walkthrough (CW) dan System Usability Scale (SUS) terhadap Aplikasi MyYOGYA," Jurnal Sains dan Informatika, Vol. 9, No. 1, pp. 1–10, 2023.

] R.J. Musyaffa, A. Priyanto, dan S. Wijayanto, "Evaluasi User Experience Aplikasi KAI Access menggunakan Metode System Usability Scale (SUS) dan Cognitive Walktrough," Jurnal Riset Sistem Informasi dan Teknologi, Vol. 1, No. 2, pp. 1–10, 2023.

] W.F. Hidayat, A. Setiadi, Y. Malau, dan R.A. Purnama, "Usability Testing Aplikasi Informasi Desa Wisata menggunakan Metode Cognitive Walkthrough dan System Usability Scale (SUS)," Jurnal Teknologi Informasi Komputer Dan Ilmu Komputasi, Vol. 5, No. 1, pp. 1–10, 2023.

] K.S.A. Saputra, I.K.P. Suniantara, I.N.D.A. Mahendra, dan A. Chrisniyanti, "Analisis Usability pada Sistem Informasi E-Tourism Ekowisata Hutan Mangrove Ngurah Rai dengan Metode Cognitive Walkthrough (CW) dan System Usability Scale (SUS)," Jurnal Riset Teknologi Informasi dan Komputer, Vol. 1, No. 1, pp. 1–12, 2023.

] M. Defriani, M.G. Resmi, dan I. Jaelani, "Uji Usability dengan Metode Cognitive Walkthrough (CW) dan System Usability Scale (SUS) pada Situs Web STT Wastukancana," Jurnal Inovasi dan Teknologi, Vol. 2, No. 1, pp. 1–12, 2022.

] A. Bangor, P. Kortum, dan J. Miller, "Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale," Journal of Usability Studies, Vol. 4, No. 3, pp. 114–123, 2009.

] M.N.I. Arsana dan A. Ali, "Analisis User Experience (UX) pada Website Layanan dikampus dengan Metode Cognitive Walkthrough (CW)," Jurnal Simbolika, Vol. 10, No. 1, pp. 82–93, 2024.

] J.W. Creswell dan J.D. Creswell, Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage publications, 2018.

] J. Nielsen, "Why You Only Need to Test with 5 Users," 2000. [Online]. Tersedia: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

] J. Nielsen dan R. Budiu, Mobile Usability. New Riders, 2013.

] M. Porta dan P. Palomares, "The Role of User Experience (UX) Metrics in Improving E-commerce Success," Journal of Business Research, Vol. 114, pp. 219–226, 2020.

] J. Sauro dan J.R. Lewis, Quantifying the User Experience: Practical Statistics for User Research, Ed. ke-2. Morgan Kaufmann, 2016.

] D. Mifsud, A. Zammit, dan C. Camilleri, "Enhancing the Cognitive Walkthrough Method for Evaluating Mobile Application Usability," International Journal of Communications, Mobile Computing and Multimedia, Vol. 12, No. 4, pp. 1-18, 2020.




DOI: https://doi.org/10.32520/stmsi.v15i2.5845

Article Metrics

Abstract view : 6 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.