Application of Artificial Intelligence using K-Means for Programming Question Assessment

Waliyyudin Waliyyudin, Ichsan Ibrahim

Abstract


The manual assessment of programming assignments remains a significant challenge in educational settings due to its time-consuming nature and susceptibility to human error. Observational studies of course instructors reveal that over 40% have made grading mistakes, often due to fatigue or inconsistent evaluation standards. This study aims to develop an automated assessment system using artificial intelligence to enhance both objectivity and efficiency in the evaluation process. The method employed is the K-Means clustering algorithm, chosen for its ability to group answers based on similarities in logic and code structure rather than mere textual similarity. Five assessment categories were used as clustering standards: Logic and Algorithm, Data Structures, Object-Oriented Programming (OOP), Implementation, and Error Handling. The system was developed using an Agile Development approach and evaluated with student responses from programming courses. System performance was validated quantitatively by comparing cluster results against ground truth labels from manual grading. The system achieved 87% clustering accuracy, reduced the average grading time to 4.5 seconds per answer (compared to 13 seconds manually—representing a 65% efficiency gain), and decreased the inter-rater score standard deviation from 7.5 to 2.8 points. The results indicate that the system can deliver accurate real-time feedback. This study focused on programming questions ranging from easy to hard difficulty levels. In the future, the system could be enhanced by integrating advanced syntax analysis and expanding the evaluation criteria to support large-scale deployment.

Keywords


automated assessment; artificial intelligence; k-means clustering; programming problems; agile development

Full Text:

PDF

References


K. Harefa and A. Jabbar, “Aplikasi Sistem Automated Essay Scoring untuk Jawaban Soal Ujian dengan Menerapkan Algoritma Jaro Winkler,” Oct. 2022.

L. R. Cipto and P. Irfan, “Aplikasi Ujian Online dengan Penilaian Otomatis menggunakan Algoritma Cosine Similarity pada SMAN 7 Mataram,” Jurnal BITe, Vol. 2, No. 1, pp. 57–65, Jun. 2020, doi: 10.30812/bite.v2i1.810.

Alfirna Rizqi Lahitani, “Automated Essay Scoring menggunakan Cosine Similarity pada Penilaian Esai Multi Soal,” May 2022. [Online]. Available: http://ejurnal.ubharajaya.ac.id/index.php/JKI

M. N. I. Susanti, A. Ramadhan, and H. L. H. S. Warnars, “Automatic Essay Exam Scoring System: A Systematic Literature Review,” in Procedia Computer Science, Elsevier B.V., 2023, pp. 531–538. doi: 10.1016/j.procs.2022.12.166.

Musdalipah, R. Soekarta, and I. Amri, “Clustering Penilaian Kinerja Dosen menggunakan K-Means di Universitas Muhammadiyah Sorong,” Framework, Vol. 01, No. 02, pp. 136–145, 2023.

W. Satria and M. Riasetiawan, “Essay Answer Classification with Smote Random Forest and AdaBoost in Automated Essay Scoring,” IJCCS (Indonesian Journal of Computing and Cybernetics Systems), Vol. 17, No. 4, pp. 359–370, Oct. 2023, doi: 10.22146/ijccs.82548.

M. O. Farooqui, M. I. Siddiquei, and S. Kathpal, “Framing Assessment Questions in the Age of Artificial Intelligence: Evidence from ChatGPT 3.5,” Emerging Science Journal, Vol. 8, No. 3, pp. 948–956, Jun. 2024, doi: 10.28991/ESJ-2024-08-03-09.

H. Alexander, Y. Umaidah, and M. Jajuli, “Implementasi Clustering untuk menentukan Efektivitas Nilai Siswa sesudah Pandemi Covid-19 menggunakan Algoritma K-Means,” Jun. 2023.

R. Anggara, S. Defit, and B. Hendrik, “Implementasi K-Means Clustering dalam Analisa Soal Ujian CBT Universitas Baiturrahmah,” Apr. 2024.

A. Z. Saputra, N. Suarna, and G. D. Lestari, “Klasterisasi Nilai Ujian Sekolah menggunakan Metode Algoritma K-Means,” Jurnal Janitra Informatika dan Sistem Informasi, Vol. 3, No. 1, pp. 1–9, Apr. 2023, doi: 10.25008/janitra.v3i1.153.

M. ’ Andri, A. Cendekia Siregar, and P. Y. Utami, “Sistem Penilaian Ujian Otomatis untuk Soal Esai menggunakan Metode Vector Space Model,” Dec. 2021.

H. F. Kurniawan, Sukisno, L. Arlianti, and T. Hidayat, “Implementasi Metode Agile untuk Rancang Bangun Sistem Penilaian Kinerja Guru,” Jurnal Informatika dan Teknik Elektro Terapan, Vol. 12, No. 3S1, Oct. 2024, doi: 10.23960/jitet.v12i3S1.5171.

H. Arfandy and I. A. Musdar, “Rancang Bangun Sistem Cerdas Pemberian Nilai Otomatis untuk Ujian Essai menggunakan Algoritma Cosine Similarity,” Dec. 2020.

F. Safnita, S. Defit, and G. W. Nurcahyo, “Penerapan Algoritma K-Means dalam Pengklasteran Hasil Evaluasi Akademik Mahasiswa,” Apr. 2024.

S. T. Boku, R. T. Abineno, and A. Aha Pekuwali, “Pengelompokan Performa Siswa dalam Pelajaran Matematika dengan Algoritma K-means di SMP Negeri 4 Mauliru,” Aug. 2023.

A. Venčkauskas, V. Jusas, and D. Barisas, “Quality Dimensions for Automatic Assessment of Structured Cyber Threat Intelligence Data,” Applied Sciences (Switzerland), Vol. 15, No. 8, Apr. 2025, doi: 10.3390/app15084327.

Bato, B., Pomperada. J. R, “Automated Grading System with Student Performance Analytics,” Technium, Vol. 30, pp. 58-75, 2025.

Sokač, M., Fabijanić, M., Mekterović, I., Mršić, L., “Automated Grading Through Contrastive Learning: A Gradient Analysis and Feature Ablation Approach,” Machine Learning and Knowledge Extraction, Vol. 7, pp. 41, 2025, doi: 10.3390/make7020041.




DOI: https://doi.org/10.32520/stmsi.v14i4.5360

Article Metrics

Abstract view : 105 times
PDF - 23 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.