Assessing Academic Information System Performance Through Sentiment Analysis

Zafira Thuraya, Ali Ibrahim, Yadi Utama, Dwi Rosa Indah

Abstract


The Academic Information System (SIMAK) at Sriwijaya University plays a crucial role in facilitating student academic activities; however, it faces several technical issues that affect user satisfaction, including server outages and challenges in data access. This dissatisfaction serves as a vital metric for evaluating the system's effectiveness. This study aims to analyze student sentiment regarding SIMAK utilizing the Naïve Bayes method. A total of 92 tweets were gathered from Twitter through web scraping, which were then categorized into manually labeled training and test datasets for model validation. The data underwent processing that included text cleaning and the application of Term Frequency-Inverse Document Frequency (TF-IDF) to assess the significance of words within a collection of documents. The evaluation results indicated that the model achieved an accuracy of 65%, with a precision of 63% for negative sentiment and a recall of 100%. In contrast, positive sentiment exhibited a low precision of 12.5% and an F1-score of 22.2%, highlighting difficulties in identifying positive sentiment due to data imbalance. The model demonstrated greater effectiveness in identifying user grievances, particularly concerning server disruptions, data delays, and challenges in completing Study Plan Cards and accessing grades. These findings provide valuable insights for SIMAK maintainers to enhance system reliability and user experience. Future research should aim to broaden data coverage and explore alternative analytical methods to yield more representative outcomes.

Keywords


Information System; Sentiment analysis; Naïve Bayes; Social Media; Twitter

Full Text:

PDF

References


D. C. Kurniasih et al., “Optimalisasi Penjualan dan Persewaan berbasis Ketamansiswaan: Niteni, Niroke, Nambahi untuk Peningkatan Kestabilan Usaha di Era Digital,” Jurnal Abdimas Indonesia, no. 4, pp. 2164–2172, 2024, doi: https://doi.org/10.53769/jai.v4i4.1065.

E. Sundari, “Transformasi Pembelajaran di Era Digital: Mengintegrasikan Teknologi dalam Pendidikan Modern,” Sindoro Cendikia Pendidikan, vol. 4, no. 4, pp. 25–35, 2024.

H. Donan, E. S. N. S. Negara, T. Sutabri, and F. Firdaus, “Analysis of Behavioral Use of Academic Information Systems with the Implementation of UTAUT 2 Integration at the Muhammadi-Palembang Institute of Health Science and Technology,” Jurnal Sisfokom (Sistem Informasi dan Komputer), vol. 12, no. 3, pp. 462–470, Nov. 2023, doi: 10.32736/sisfokom.v12i3.1978.

S. Nurhanifah and D. R. Indah, “Evaluasi Kepuasan Pengguna pada Sistem Informasi Akademik (SIMAK) Universitas Sriwijaya dengan menggunakan Metode PIECES Framework,” Jurnal Sistem dan Teknologi Informasi (JustIN), vol. 11, no. 2, p. 339, Jul. 2023, doi: 10.26418/justin.v11i2.56574.

M. Fajaria and K. D. Tania, “Evaluasi User Experience dan Usability Sistem Informasi Akademik menggunakan Metode User Experience Questionnaire dan System Usability Scale,” JOISIE (Journal Of Information System And Informatics Engineering), vol. 7, no. 2, pp. 204–213, Dec. 2023, doi: https://doi.org/10.35145/joisie.v7i2.3812.

M. Fauzan, H. Muslimah Az-Zahra, W. Hayuhardhika, and N. Putra, “Pengukuran Kualitas Layanan Website Akademik Universitas Sriwijaya Kota Palembang menggunakan Metode Webqual 4.0 dan Importance Performance Analysis (IPA),” J-PTIIK, vol. 3, no. 6, pp. 5817–5824, Jul. 2019, doi: https://doi.org/10.33479/jptiik.v3i2.350.

D. W. Ardras and A. Voutama, “Analisis Sentimen Anti LGBT di Indonesia melalui Media Sosial Twitter,” Jurnal Teknika, vol. 15, no. 1, pp. 23–28, Mar. 2023, doi: 10.30736/jt.v15i1.926.

C. Destitus, Wella, and Suryasari, “Support Vector Machine VS Information Gain: Analisis Sentimen Cyberbullying di Twitter Indonesia,” Ultima InfoSys:Jurnal Ilmu Sistem Informasi, vol. XI, no. 2, pp. 107–111, Dec. 2020, doi: https://doi.org/10.31937/si.v11i2.1740.

L. Afuan, M. Khanza, and A. Z. Hasyati, “Enhancing Sentiment Analysis of The 2024 Indonesian Presidential Inauguration on X using Smote-Optimized Naive Bayes Classifier,” Jurnal Teknik Informatika (JUTIF), vol. 6, no. 1, pp. 325–333, Feb. 2025, doi: 10.52436/1.jutif.2025.6.1.4290.

I. Zulfahmi, J. Williem, P. V Medan, S. Tuan, and D. Serdang, “Analisis Sentimen Aplikasi PLN Mobile menggunakan Metode Decission Tree,” Jurnal Penelitian Rumpun Ilmu Teknik (JUPRIT), vol. 3, no. 1, pp. 11–21, 2024, doi: 10.55606/juprit.v3i1.3096.

B. Ramadhani and R. R. Suryono, “Komparasi Algoritma Naïve Bayes dan Logistic Regression untuk Analisis Sentimen Metaverse,” Jurnal Media Informatika Budidarma, vol. 8, no. 2, p. 714, Apr. 2024, doi: 10.30865/mib.v8i2.7458.

D. Prasetia, N. Rahaningsih, R. D. Dana, and C. L. Rohmat, “Analisis Sentimen Pengguna Aplikasi Mybluebird dengan Algoritma Naïve Bayes di Playstore,” Jurnal Informatika dan Teknik Elektro Terapan, vol. 13, no. 1, pp. 602–608, Jan. 2025, doi: http://dx.doi.org/10.23960/jitet.v13i1.5687.

I. Putu Wibina Karsa Gumi et al., “Perbandingan Algoritma Naïve Bayes dan Decision Tree pada Sentimen Analisis,” 2022. [Online]. Available: https://subset.id/index.php/IJCSR

T. V. Meiyanti, M. Hatta, and A. Sevtiana, “Analisis Sentimen Mahasiswa dengan Dosen menggunakan Metode Naïve Bayes Classifier pada Kuesioner Dosen,” Jurnal Manajemen Sistem Informasi, vol. 1, no. 2, pp. 55–59, 2023, doi: https://doi.org/10.51920/jurminsi.v1i2.143.

I. Tri and J. J. Algoritma, “Analisis Sentimen terhadap Sistem Informasi Akademik Mahasiswa Institut Teknologi Garut,” Jurnal Algoritma, vol. 19, no. 1, pp. 458–468, Jul. 2022, doi: https://doi.org/10.33364/algoritma/v.19-1.1112.

D. Lado Kaka, G. Kopong Pati, and K. Wulla Rato, “Analisis Sentimen Komentar SIAKAD menggunakan Metode Naive Bayes Classifier,” Jurnal Kridatama Sains Dan Teknologi, vol. 05, no. 2, pp. 266–277, 2023, doi: https://doi.org/10.53863/kst.v5i02.933.

L. R. Dharmawan, I. Arwani, and D. E. Ratnawati, “Analisis Sentimen pada Sosial Media Twitter terhadap Layanan Sistem Informasi Akademik Mahasiswa Universitas Brawijaya dengan Metode K-Nearest Neighbor,” 2020. Accessed: Mar. 25, 2025. [Online]. Available: https://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/7099

I. Made, Y. Adhitya, D. E. Ratnawati, and I. Arwani, “Analisis Sentimen Feed Back Mahasiswa terhadap Dosen Prodi Teknologi Informasi Departemen Sistem Informasi Fakultas Ilmu Komputer Universitas Brawijaya,” 2023. Accessed: Mar. 25, 2025. [Online]. Available: https://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/13093

R. Fajar, “Implementasi Algoritma Naive Bayes terhadap Analisis Sentimen Opini Film pada Twitter,” Jurnal Inovtek Polbeng , vol. 3, no. 1, pp. 50–59, 2020, doi: https://doi.org/10.35314/isi.v3i1.335.

F. N. Salsabilla and A. Witanti, “Analisis Sentimen Akhir Masa Jabatan Presiden Jokowi pada Media Sosial X menggunakan Naïve Bayes,” SKANIKA: Sistem Komputer dan Teknik Informatika, vol. 8, no. 1, pp. 106–115, Jan. 2025, doi: https://doi.org/10.36080/skanika.v8i1.3331.

N. C. Agustina, D. Herlina Citra, W. Purnama, C. Nisa, and A. Rozi Kurnia, “The Implementation of Naïve Bayes Algorithm for Sentiment Analysis of Shopee Reviews on Google Play Store,” Machine Learning and Computer Science, vol. 2, pp. 47–54, Apr. 2022, doi: https://doi.org/10.57152/malcom.v2i1.195.

R. Asmara, M. Febrian, and M. Anshori, “Analisa Sentiment Masyarakat terhadap Pemilu 2019 berdasarkan Opini di Twitter menggunakan Metode Naive Bayes Classifier,” Jurnal Inovtek Polbeng, vol. 5, no. 2, pp. 193–204, 2020, doi: https://doi.org/10.35314/isi.v5i2.1095.

R. A. Mrg and M. S. Hasibuan, “Best Student Classification using Ensemble Random Forest,” Sistemasi: Jurnal Sistem Informasi , vol. 13, no. 3, pp. 1188–1204, 2024, doi: https://doi.org/10.32520/stmsi.v13i3.4101.

M. Rizqi, A. Rinaldi, and D. Rohman, “Applications Naive Bayes Algorithm to Enhance Sentiment Analysis of Coursera Application Reviews on Google Play Store,” Journal of Artificial Intelligence and Engineering, vol. 4, no. 2, pp. 2808–4519, 2025, doi: https://doi.org/10.59934/jaiea.v4i2.758.

L. Rofiqi and M. Akbar, “Analisis Sentimen Terkait RUU Perampasan Aset dengan Support Vector Machine,” JEKIN (Jurnal Teknik Informatika), vol. 4, no. 3, pp. 529–538, Aug. 2024, doi: 10.58794/jekin.v4i3.824.

S. Abdullah, “Visualisasi Data Analisa Sentimen RUU Omnibus law Kesehatan menggunakan KNN dengan Software RapidMiner,” Jurnal Informatika: Jurnal Pengembangan IT, vol. 8, no. 3, pp. 261–268, 2023, doi: https://doi.org/10.30591/jpit.v8i3.5641.

M. Agym and H. Mayatopani, “Sentiment Analysis Public Perspection From Artemis 2 Mission using Recurrent Neural Network Methods,” Jurnal Teknik Informatika (JUTIF), vol. 5, no. 4, pp. 385–393, 2024, doi: 10.52436/1.jutif.2024.5.4.2365.

Y. Z. Vebrian and Kustiyono, “A Sentiment Analysis of Free Meal Plans on Social Media using Naïve Bayes Algorithms,” Jurnal Inovtek Polbeng, vol. 10, no. 1, pp. 355–366, 2025, doi: https://doi.org/10.35314/3m2fcz69.

I. Habib and N. Cahyono, “Analisis Sentimen Masyarakat terhadap Penggunaan E-Commerce menggunakan Algoritma K-Nearest Neighbor,” Jurnal Pengembangan IT (JPIT), vol. 8, no. 3, Sep. 2023, doi: https://doi.org/10.30591/jpit.v8i3.5734.

A. Triyono, A. Faqih, and G. Dwilestari, “Implementation of the Naive Bayes Method in Sentiment Analysis of Spotify Application Reviews,” Journal of Artificial Intelligence and Engineering Applications , vol. 4, no. 2, pp. 1091–1097, Feb. 2025, doi: https://doi.org/10.59934/jaiea.v4i2.824.

Y. A. Singgalen, “Culture and Heritage Tourism Sentiment Classification Through Cross-Industry Standard Process for Data mining,” International Journal of Basic and Applied Science, vol. 12, no. 3, pp. 110–120, Dec. 2023, doi: https://doi.org/10.35335/ijobas.v12i3.299.

S. Ridwan and Nurfaizah, “Sentimen Analisis Pengguna Produk Ponsel menggunakan Algoritma Naïve BayeS,” Journal of Information System Management (JOISM) e-ISSN, vol. 6, no. 1, pp. 2715–3088, 2024, doi: https://doi.org/10.24076/joism.2024v6i1.1625.

K. Kevin, M. Enjeli, and A. Wijaya, “Analisis Sentimen Pengunaaan Aplikasi Kinemaster menggunakan Metode Naive Bayes,” Jurnal Ilmiah Computer Science, vol. 2, no. 2, pp. 89–98, Jan. 2024, doi: 10.58602/jics.v2i2.24.

M. I. Arif Chandra and R. Yusuf, “Visualisasi Kata Kunci Pemberitaan PEMILU 2024 menggunakan Spacy dan Word Cloud,” TEKNIMEDIA: Teknologi Informasi Dan Multimedia, vol. 5, no. 1, pp. 41–46, 2024, doi: https://doi.org/10.46764/teknimedia.v5i1.187.

I. Syahrohim, S. D. Saputra, R. W. Saputra, V. H. Pranatawijaya, and R. Priskila, “Perbandingan Analisis Sentimen setelah Pilpres 2024 di Twitter menggunakan Algoritma Machine Learning,” Jurnal Informatika dan Teknik Elektro Terapan, vol. 12, no. 2, Apr. 2024, doi: 10.23960/jitet.v12i2.4249.




DOI: https://doi.org/10.32520/stmsi.v14i3.5130

Article Metrics

Abstract view : 136 times
PDF - 34 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.