Combining SVM and Naive Bayes Models using a Soft Voting Approach for Sentiment Analysis of Tong Tji Tea House

Fendi Pradana Saputra, Ozzi Suria

Abstract


In today’s digital technology and social media era, people are increasingly influenced to actively share reviews of restaurant services, expressing a wide range of customer opinions and perceptions. This study aims to analyze sentiment in Indonesian-language review texts using three machine learning models: Support Vector Machine (SVM), Naive Bayes (NB), and a combination of both through an Ensemble Soft Voting Classifier approach. The research focuses on user reviews of the Tong Tji Tea House, collected from the Google Maps platform, with sentiment data distributed as follows: positive (2,676 entries), neutral (670 entries), and negative (251 entries). The class imbalance poses a significant challenge in developing an optimal classification model. To address this, parameter optimization was carried out using the Grid Search method. The SVM model with a linear kernel and C=10 parameter achieved an accuracy of 0.9289 and a positive F1-score of 0.9289. The NB model recorded an accuracy of 0.8340 with an F1-score of 0.9102. Meanwhile, the Ensemble model with a soft voting approach and a 4:1 weight ratio (SVM:NB) demonstrated the best performance, achieving an accuracy of 0.9344 and a positive F1-score of 0.9750. These results indicate that the Ensemble method effectively enhances model accuracy and robustness in handling imbalanced data.

Keywords


Sentiment Analysis; Tong TJi Tea House; Machine Learning; Ensemble Soft Combination; Restaurant Reviews

Full Text:

PDF

References


A. Nikseresht, M. H. Raeisi, and H. A. Mohammadi, ‘Decision Making for Celebrity Branding: An Opinion Mining Approach based on Polarity and Sentiment Analysis using Twitter Consumer-Generated Content (CGC)’, “arxiv, September, 2021. doi: 10.48550/arxiv.2109.12630.

N. Singh and U. C. Jaiswal, ‘Sentiment Analysis using Machine Learning’, ADCAIJ, Vol.12, December 2023. doi: 10.14201/adcaij.26785

F. Rustam, I. Ashraf, A. Mehmood, S. Ullah, and G. S. Choi, ‘Tweets Classification on the Base of Sentiments for US Airline Companies’, 2019 Mdpi. doi: 10.3390/e21111078.

Y. B. Lasotte, E. J. Garba, Y. M. Malgwi, and M. A. Buhari, ‘An Ensemble Machine Learning Approach for Fake News Detection and Classification using a Soft Voting Classifier’, 2022. doi: 10.24018/ejece.2022.6.2.409.

V. Rupapara, F. Rustam, A. Amaar, P. B. Washington, E. Lee, and I. Ashraf, ‘Deepfake Tweets Classification using Stacked Bi-LSTM and Words Embedding’, 2021. doi: 10.7717/peerj-cs.745.

A. Supian, B. T. Revaldo, N. Marhadi, L. Efrizoni, and R. Rahmaddeni, ‘Perbandingan Kinerja Naïve Bayes dan SVM pada Analisis Sentimen Twitter Ibukota Nusantara’, 2024, Jif. doi: 10.33884/jif.v12i01.8721.

A. R. Isnain, N. S. Marga, and D. Alita, ‘Sentiment Analysis of Government Policy on Corona Case using Naive Bayes Algorithm’, IJCCS. 2021, doi: 10.22146/ijccs.60718.

A. Özçift and M. Bozuyla, ‘Evaluation of Ensemble Algorithms and Deep Learning Transformers in Medical Sentiment Prediction’, Dergipark. 2021, doi: 10.31590/ejosat.1010241.

Y. Setiawan, J. Jondri, and W. Astuti, ‘Twitter Sentiment Analysis on Online Transportation in Indonesia using Ensemble Stacking’, ejurnal.stmik-budidarma, 2022, doi: 10.30865/mib.v6i3.4359.

V.R.N. Seileela and N. N. M. Rao, ‘Iterative Ensemble Learning Over High Dimensional Data for Sentiment Analysis’, scpe, 2024, doi: 10.12694/scpe.v25i2.2650.

H. Leidiyana, ‘Ensemble Stacking dalam Analisa Sentimen Reaksi Veteran Militer as terhadap Pengambilalihan Afghanistan Oleh Taliban’, inti, 2023, doi: 10.33480/inti.v18i1.4175.

F. A. Indriyani, A. Fauzi, and S. Faisal, ‘Analisis sentimen aplikasi Tiktok menggunakan Algoritma Naïve Bayes dan Support Vector Machine’, TEKNOSAINS : Jurnal Sains, Teknologi dan Informatika, Vol. 10, no. 2, pp. 176–184, Jul. 2023, doi: 10.37373/tekno.v10i2.419.

N. Agustina dan C. N. ihsan , and N. M. A. Al-Hashedi, ‘Ense2 Tampilan Pendekatan Ensemble untuk Analisis Sentimen Covid19 menggunakan Pengklasifikasi Soft Voting’, Applied Computational Intelligence and Soft Computing , 2023.

D. Triyana, M. M. Al Haromainy, H. Maulana ‘Ensmbl3View of Implementasi Metode Ensemble Majority Vote pada Algoritma Naïve Bayes dan Random Forest untuk Analisis Sentimen Twitter Harga Tiket Pesawat Domestik’, Jurnal Mahasiswa teknik Iformatika, Vol. 8, 2024.

Susi, ‘Analisis Sentimen Cuitan Twitter Pada Proses Seleksi CPNS menggunakan Algoritma Ensembles Vote’ IKRAM: Jurnal Ilmu Komputer Al Muslim, Vol. 1, No. 1, 2022.

A. Onan, ‘View of Ensemble of Classifiers and Term Weighting Schemes for Sentiment Analysis in Turkish’, Scientific Research Communications, Vol. 1, No. 1, 2021.

Y. A. Mustofa, I. Surya, and K. Idris, ‘Pendekatan Ensemble pada Analisis Sentimen Ulasan Aplikasi Google Play Store Ensemble Approach to Sentiment Analysis of Google Play Store App Reviews’, Jambura Juornal of Electrical end Electronics Engineering, Vol. 6, 2024.

B. S. Amalia, Y. Umaidah, R. Mayasari, S. Karawang Jl HSRonggo Waluyo, K. Telukjambe Timur, and K. Karawang, ‘Analisis Sentimen Riview Pelanggan Restoran menggunakan Algoritma Supprt Vector Machine dan K-Nearest Neighbor’, Jurnal Sains Teknologi dan Industri, Vol. 19, No. 1, pp. 28–34, 2021.

O. Fanny and H. Suroyo, ‘Analisis Sentimen Pengguna Media Sosial terhadap Omnibus Law berdasarkan Hashtag di Twitter Analysis of Social Media Users Sentiments Against Omnibus Law Based on Hashtags on Twitter’, SISTEMASI: Jurnal Sistem Informasi 2022. DOI: https://doi.org/10.32520/stmsi.v11i1.1685.

M. Fariz Firdaus, D. E. Ratnawati, N. Y. Setiawan, and P. Korespondensi, ‘Analisis Sentimen berbasis Aspek pada Ulasan Pelanggan Restoran menggunakan Algoritma Support Vector Machine (Studi Kasus: Depot Bamara)’, Jurnal Teknologi informasi dan ilmu komputer, Vol. 11, No. 6, pp. 1265–1272, 2024, doi: 10.25126/jtiik.2024117564.

P. Gracezando Yehova and M. Ihsan Jambak, ‘Perbandingan Metode Naïve Bayes dan SVM pada Ulasan Google Playstore Mobile Legends Bang Bang’ , Jurnal Device,Vol. 14, No. 1, pp. 140–149, 2024.

S. Riyadiiban and S. Riyadi, ‘Analisis Sentimen Opini Masyarakat terhadap Stadion Jakarta Internasional Stadium (JIS) pada Twitter … Analisis Sentimen Opini Masyarakat terhadap Stadion Jakarta Internasional Stadium (JIS) pada Twitter dengan Perbandingan Metode Naive Bayes dan Support Vector Machine’, Jurnal Sains dan Teknologi, Vol. 5, No. 3, p. 2024, 2024, doi: 10.55338/saintek.v5i3.2962.

M. I. Fikri, T. S. Sabrila, Y. Azhar, and U. M. Malang, ‘Perbandingan Metode Naïve Bayes dan Support Vector Machine pada Analisis Sentimen Twitter’, SMATIKA Jurnal, 2024.

S. N. Fadhilah and F. S. Utomo, ‘Algoritma Naïve Bayes untuk Analisis Sentiment Review Blibli.com di Google Play Store Naïve Bayes Algorithm for Sentiment Analysis of Blibli.com Review on Google Play Store’, SISTEMASI: Jurnal Sistem Informasi 2021. DOI: https://doi.org/10.32520/stmsi.v13i2.3887

L. B. Ilmawan and M. A. Mude, ‘Perbandingan Metode Klasifikasi Support Vector Machine dan Naïve Bayes untuk Analisis Sentimen pada Ulasan Tekstual di Google Play Store’, ILKOM Jurnal Ilmiah, Vol. 12, No. 2, pp. 154–161, Aug. 2020, doi: 10.33096/ilkom.v12i2.597.154-161.

N. P. G. Naraswati, D. C. Rosmilda, D. Desinta, F. Khairi, R. Damaiyanti, and R. Nooraeni, ‘ Analisis Sentimen Publik dari Twitter tentang Kebijakan Penanganan Covid-19 di Indonesia dengan Naive Bayes Classification’, SISTEMASI: Jurnal Sistem Informasi, 2021. https://doi.org/10.32520/stmsi.v10i1.1179

A. F. Alkindi and N. Nasution, ‘Analisis Sentimen Ulasan Penggunan pada Game Roblox dengan Metode Support Vector Machine dan Naïve Bayes’, J-Com (Journal of Computer), Vol. 4, No. 2, pp. 164–177, Jul. 2024, doi: 10.33330/j-com.v4i2.3319.

Salahuddin, I. Sabila, ‘Analisis dan Implementasi Sistem Penilaian Ulasan dengan Teknik Sentiment Analysis berbasis Machine Learning untuk Peningkatan Feedback Pemilik Cafe,” Infomedia,Vol.9 No.2,2024. DOI: http://dx.doi.org/10.30811/jim.v9i2.5980

R. A. M. San Ahmed, ‘Hard Voting Approach using SVM, Naïve Bays and Decision Tree for Kurdish Fake News Detection’, Iraqi Journal for Computer Science and Mathematics, Vol. 4, No. 3, pp. 25–33, 2023, doi: 10.52866/ijcsm.2023.02.03.003.




DOI: https://doi.org/10.32520/stmsi.v14i5.5481

Article Metrics

Abstract view : 4 times
PDF - 3 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.