Email this article 
Hide
Show all
Hyperparameter Tuning on Classification Algorithm with Grid Search
Abstract
Currently, machine learning algorithms continue to be developed to perform optimization with various methods to produce the best-performing model. In Supervised learning or classification, most of the algorithms have hyperparameters. Tuning hyperparameter is an architecture of deep learning to improve the performance of predictive models. One of the popular hyperparameter methodologies is Grid Search. Grid Search using Cross Validation provides convenience in testing each model parameter without having to do manual validation one by one. In this study, we will use a method in hyperparameter optimization, namely Grid Search. The purpose of this study is to find out the best optimization of hyperparameters against 7 machine learning classification algorithms. Validation of experimental results using the Mean Cross Validation. The experimental results show that the XGBoost model gets the best value while the Decision tree has the lowest value.
Full Text:
PDFReferences
P. C. Sen, M. Hajra, and M. Ghosh, “Supervised Classification Algorithms in Machine Learning: A Survey and Review,” in Advances in Intelligent Systems and Computing, 2020, vol. 937, pp. 99–111, doi: 10.1007/978-981-13-7403-6_11.
T. M. Mitchell, Machine Learning. Boston: WCB/McGraw-Hill, 1997.
S. Uddin, A. Khan, M. E. Hossain, and M. A. Moni, “Comparing different supervised machine learning algorithms for disease prediction,” BMC Med. Inform. Decis. Mak., vol. 19, no. 1, pp. 1–16, 2019, doi: 10.1186/s12911-019-1004-8.
R. Saravanan and P. Sujatha, “Algorithms : A Perspective of Supervised Learning Approaches in Data Classification,” Proc. Second Int. Conf. Intell. Comput. Control Syst. (ICICCS 2018), pp. 945–949, 2018.
B. Liu, Web data mining: exploring hyperlinks, contents, and usage data, Second. Berlin: Springer, 2011.
F. Hutter, L. Kotthoff, and J. Vanschoren, Automated machine learning: methods, systems, challenges. Springer Nature, 2019.
G. A. Lujan-Moreno, P. R. Howard, O. G. Rojas, and D. C. Montgomery, “Design of experiments and response surface methodology to tune machine learning hyperparameters, with a random forest case-study,” Expert Syst. Appl., vol. 109, pp. 195–205, 2018, doi: 10.1016/j.eswa.2018.05.024.
J. P. Lalor, H. Wu, and H. Yu, “CIFT: Crowd-Informed Fine-Tuning to Improve Machine Learning Ability,” arXiv Comput. Lang., vol. 6, no. February, 2017, [Online]. Available: http://arxiv.org/abs/1702.08563.
R. Bardenet, M. Brendel, B. Kégl, and M. Sebag, “Collaborative hyperparameter tuning,” Proc. 30th Int. Conf. Mach. Learn., vol. 28, no. 2, pp. 199–207, 2013.
J. Bergstra and Y. Bengio, “Random search for hyper-parameter optimization,” J. Mach. Learn. Res., vol. 13, pp. 281–305, 2012.
R. G. Mantovani, A. L. D. Rossi, E. Alcobaça, J. Vanschoren, and A. C. P. L. F. de Carvalho, “A meta-learning recommender system for hyperparameter tuning: Predicting when tuning improves SVM classifiers,” Inf. Sci. (Ny)., vol. 501, pp. 193–221, 2019, doi: 10.1016/j.ins.2019.06.005.
F. Alaa Khaleel and A. M. Al-Bakry, “Diagnosis of diabetes using machine learning algorithms,” Mater. Today Proc., no. xxxx, 2021, doi: 10.1016/j.matpr.2021.07.196.
Y. N. Kunang, S. Nurmaini, D. Stiawan, and B. Y. Suprapto, “Attack classification of an intrusion detection system using deep learning and hyperparameter optimization,” J. Inf. Secur. Appl., vol. 58, no. March, p. 102804, 2021, doi: 10.1016/j.jisa.2021.102804.
Y. Hayashi and S. Yukita, “Rule extraction using Recursive-Rule extraction algorithm with J48graft combined with sampling selection techniques for the diagnosis of type 2 diabetes mellitus in the Pima Indian dataset,” Informatics Med. Unlocked, vol. 2, pp. 92–104, 2016, doi: 10.1016/j.imu.2016.02.001.
B. P. Manoj Kumar, S. R. Perumal, and N. R. K, “Type 2: Diabetes mellitus prediction using Deep Neural Networks classifier,” Int. J. Cogn. Comput. Eng., vol. 1, pp. 55–61, 2020, doi: 10.1016/j.ijcce.2020.10.002.
S. Patel, “Fundamental concepts for Model Selection and Model Evaluation — Part2,” Medium, 2020. https://medium.com/analytics-vidhya/fundamental-concepts-for-model-selection-and-model-evaluation-part2-e72b384f8ab6 (accessed Nov. 26, 2021).
B. Arup, “T103: Filter method-Feature selection techniques in machine learning,” upskillpoint, 2020. https://www.upskillpoint.com/machine learning/2020/03/11/feature-selection-using-filter-method/ (accessed Nov. 27, 2021).
P. Kampstra, “Beanplot: A Boxplot Alternative for Visual Comparison of Distributions,” J. Stat. Softw., vol. 28, no. November, pp. 1–9, 2008, [Online]. Available: papers3://publication/uuid/692988CE-7E10-498E-96EC-E7A0CE3620A3.
B. Roy, “All about Feature Scaling,” towardsdatascience, 2020. https://towardsdatascience.com/all-about-feature-scaling-bcc0ad75cb35 (accessed Nov. 27, 2021).
DOI: https://doi.org/10.32520/stmsi.v11i2.1750
Article Metrics
Abstract view : 1454 timesPDF - 640 times
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.