AI-Powered Real-Time Alcohol Detection Using Fine-Tuned YOLOv8 and OCR-Based Brand Identification

Abstract

This paper presents an AI-powered real-time alcohol detection system built around a YOLOv8n object detector fine-tuned via transfer learning from COCO-pretrained weights, combined with OpenCV-based frame acquisition and an OCR-assisted brand identification layer. The detector was fine-tuned on a public 857-image alcohol/no-alcohol dataset (Roboflow Universe) for 40 epochs on a GPU (NVIDIA Tesla T4, Google Colab), following an initial 15-epoch CPU run that was found to be undertrained. On a genuine held-out test set of 87 images (137 independently verifiable ground-truth boxes), the fully converged fine-tuned model achieves a mAP@0.5 of 0.829 and an F1-score of up to 0.831, compared to 0.364 mAP@0.5 and 0.508 F1 for a stock COCO-pretrained baseline restricted to generic “bottle”, “wine glass”, and “cup” classes — a 128% relative improvement in mAP@0.5 from fine-tuning alone. Three additional components operate on top of detection: an optional EasyOCR label reader coupled with fuzzy keyword matching against a brand database covering major global and Indian alcohol brands; a 180°-rotation fallback inference pass that recovers detections of knocked-over or inverted containers; and a consecutive-frame debounce filter that suppresses single-frame false triggers. Detections are relayed over Socket.IO to a React dashboard and an OS-level popup alert. Deployment inference runs at approximately 22–24 FPS on standard hardware. We report full training convergence curves, a threshold-swept precision–recall analysis, an explicit detection confusion breakdown, and the system's genuine limitations.