Data-Centric AI based Indonesian ALPR System using Yolov11 And TrOCR

Zulfikri Rizki Ardi, Andi Sunyoto

Abstract


The implementation of Automatic License Plate Recognition (ALPR) in Indonesia faces challenges arising from visual degradation and anomalies in real-world datasets, including annotation errors, data leakage, and imbalanced distributions of regional license plate codes. This study aims to develop a precise and adaptive end-to-end ALPR system for Indonesian vehicle license plates by optimizing both license plate detection and character recognition architectures. A Data-Centric AI approach was employed to improve dataset quality through bootstrap relabeling and the targeted synthesis of 2,773 images to address regional distribution imbalances. The proposed system adopts a two-stage processing pipeline, consisting of license plate localization using YOLOv11m and text recognition through a comparative evaluation of PaddleOCR, Tesseract, EasyOCR, and TrOCR-base. Specifically, TrOCR was optimized using a two-stage fine-tuning strategy (K4 Two-Stage) that combines synthetic data during pretraining with real-world data during subsequent fine-tuning. The results demonstrate that the proposed framework effectively mitigates dataset anomalies. The YOLOv11m model achieved a license plate detection mAP@50 of 97.05%. Among the recognition models, TrOCR K4 achieved the best performance, with an Exact Match (EM) accuracy of 86.21% and a Character Error Rate (CER) of 3.03%. End-to-end inference evaluation achieved an overall exact-match accuracy of 87.07%, with a low inference speed of 2.84 FPS (approximately 206 ms per license plate) on an NVIDIA L4 GPU. Overall, the integration of Data-Centric AI, YOLOv11, and TrOCR produced a highly accurate and efficient Indonesian license plate recognition system, demonstrating strong potential for real-time vehicle monitoring applications.

Keywords


automatic license plate recognition; image processing; data centric; TrOCR; YOLOv11

Full Text:

PDF

References


D. E. Kalef, M. Zourhri, A. El-Ibrahimi, B. Benhala, C. Bouchaib, and H. Silkan, “Moroccan License Plate Detection for Intelligent Transportation Systems using YOLOv8 and YOLOv11,” in 2026 6th International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET), 2026, pp. 1–10. DOI: 10.1109/IRASET68627.2026.11538797.

N. A. Kurniawan and C. A. Sari, “Automatic License Plate Detection System with YOLOv11 Algorithm,” J. Appl. Informatics Comput., Vol. 9, No. 6, pp. 3097–3109, 2025, DOI: 10.30871/jaic.v9i6.11484.

P. N. Andono and A. Syukur, “Multi-Variant Enhancement and OCR Selection for Robust License Plate Recognition in Hazy Conditions,” Stat. Optim. Inf. Comput., Vol. 16, No. August, pp. 1086–1109, 2026, DOI: 10.19139/soic-2310-5070-3818.

B. Buleu, R. Robu, and I. Filip, “A Deep Learning-based System for Automatic License Plate Recognition using YOLOv12 and PaddleOCR,” MDPI Appl. SCI., pp. 1–23, 2025, DOI: 10.3390/app15147833.

F. J. Ayrin, M. A. Azim, A. N. Chy, and M. K. Islam, “An Optical Character Recognition Technique for Extracting Text from Blurred and Low-Quality Images using TrOCR,” in 2025 International Conference on Electrical, Computer and Communication Engineering (ECCE), 2025, pp. 1–6. DOI: 10.1109/ECCE64574.2025.11013065.

S. Issaoui, S. A. Alzakari, I. Saidi, and R. Ejbali, “Hierarchical Vision – Language Fusion with Structural Constraint Reasoning for Robust Multi-Jurisdiction License Plate Recognition,” Appl. SCI., pp. 1–23, 2026, DOI: 10.3390/app16136792.

N. D. Fazira and A. Fauzan, “Transformer-based Optical Character Recognition Approach for Identifying Motor,” BAREKENG J. Math. Its Appl., Vol. 19, No. 3, pp. 1597–1608, 2025, DOI: 10.30598/barekengvol19iss3pp1597-1608.

F. Sonnara, H. Chihaoui, and F. Filali, “Efficient Real ‑ Time License Plate Recognition using Deep Learning on Edge Devices,” J. Real-Time Image Process., Vol. 22, No. 4, pp. 1–15, 2025, DOI: 10.1007/s11554-025-01738-3.

E. P. Silmina, R. Agil, and Y. Arjun, “Pemanfaatan Model YOLOv8 untuk mendeteksi Plat Nomor Kendaraan Mobil pada Gerbang Masuk Universitas XYZ,” J. Sains dan Inform., Vol. 11, pp. 50–59, 2025, DOI: 10.34128/jsi.v11i1.916.

X. Jin, S. Wang, M. Zhang, G. Xu, and B. Hu, “TrOCR-Driven Seal Instrument Detection and Recognition for Cognitive Robotic Applications ✩,” Cogn. Robot. J., Vol. 5, No. September, pp. 286–298, 2025, DOI: 10.1016/j.cogr.2025.10.001.

H. Moussaoui et al., “Enhancing Automated Vehicle Identification by Integrating YOLO v8 and OCR Techniques for High ‑ Precision License Plate Detection and Recognition,” SCI. Rep., pp. 1–17, 2024, DOI: 10.1038/s41598-024-65272-1.

N. Shabbir, R. Ahmed, M. W. Raza, A. Zeb, H. Elahi, and H. Waheed, “Comparative Performance and Resource Utilization Analysis of OCR Models for Number Plate Recognition on Raspberry Pi 4,” in 2025 International Conference on Communication Technologies (ComTech), 2025, pp. 1–6. DOI: 10.1109/ComTech65062.2025.11034455.

A. Tanapatpiboon, W. Kusakunniran, and W. Limroongreungrat, “Deep Learning-based Detection of Players and Teams in Soccer Videos with Positioning Heatmap Generation,” Appl. Comput. Informatics, 2025.

C. D. C. Pereira, “A Data-Centric Trust Pipeline:An Empirical Framework for Trustworthy AI in Sensitive Domains,” AI Ethics, Vol. 6, No. 4, p. 383, 2026, DOI: 10.1007/s43681-026-01247-4.

D. Asmawati, “Automatic Reading of Analog Gauges using YOLOv9 , Segment Anything Model , and TrOCR,” J. Inf. Syst. Eng. Bus. Intell., Vol. 12, No. 2, pp. 333–347, 2026, DOI: 10.20473/jisebi.12.2.333-347.

X. Du, H. Chen, J. Huan, and Y. Du, “License Plate Recognition System based on Embedded Technology and YOLOv11,” in 2025 5th International Symposium on Computer Technology and Information Science (ISCTIS), 2025, pp. 868–871. DOI: 10.1109/ISCTIS65944.2025.11065377.

A. Sugiharto and R. Kusumaningrum, “Enhanced Automatic License Plate Detection and Recognition using CLAHE and YOLOv11 for Seat Belt Compliance Detection,” Eng. Technol. Appl. SCI. Res., Vol. 15, No. 1, pp. 20271–20278, 2025, DOI: 10.48084/etasr.9629.

E. Kapetanios, “An End-to-End Automated License Plate Recognition System Vehicle Classification,” MDPI Sensors, 2022, DOI: 10.3390/s22239477.

C. J. Islas-yañez and V. Hernández-herrera, “Modular Multi-Attribute Vehicle Analysis by Color , License Plate , Make and Sub-Model using YOLO and OCR : A Benchmark Across YOLO Versions,” Cent. Investig. e Innovación Tecnológica, 2026, DOI: 10.3390/s26092785.

A. Ranjan and M. Ravinder, “ROBDD-TrOCRBERTa: A Novel Robust-Optimized Blurred Document Text Deblurring and Completion with DCGAN-TrOCR and DistilRoBERTa,” Int. J. Inf. Technol., Vol. 16, No. 7, pp. 4611–4619, 2024, DOI: 10.1007/s41870-024-02073-9.

J. N., V. D., F. Jemima, P. P. G, and T. P. M, “YOLOv11-based Real-Time License Plate Detection and Recognition for Smart Transportation Systems,” in 2026 ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS), 2026, pp. 913–921. DOI: 10.1109/ICETSIS68266.2026.11548780.

F. F. Fu, N. F. Oongwidjaja, and K. Jingga, “The Effectiveness of White Vehicle License Plates Compared to Black Vehicle License Plates on ANPR,” in 2025 International Seminar on Intelligent Technology and Its Applications (ISITIA), 2025, pp. 260–265. DOI: 10.1109/ISITIA66279.2025.11137559.

A. Sunyoto, “Automatic License Plate Recognition System in Indonesia using YOLOv8 and EasyOCR Algorithm,” 2023 6th Int. Conf. Inf. Commun. Technol., pp. 384–388, 2023, DOI: 10.1109/ICOIACT59844.2023.10455908.




DOI: https://doi.org/10.32520/stmsi.v15i8.6697

Article Metrics

Abstract view : 1 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.