🚗 Real-Time Multi-Modal ADAS

An Integrated Framework for Lane Detection, Object Recognition, and Driver Drowsiness Monitoring Using OpenCV and YOLOv11

---

📖 Overview

This project presents a fully integrated Advanced Driver Assistance System (ADAS) that combines:

• 🔍 Real-time object detection using YOLOv11
• 🛣️ Classical lane detection using OpenCV
• 😴 Driver drowsiness detection via facial cues (eyes closed, yawning)

Implemented using Flask, it supports real-time video streaming and inference through a web browser.

---

🧪 Research Publication

📄 Title: Real-Time Multi-Modal ADAS: An Integrated Framework for Lane Detection, Object Recognition, and Driver Drowsiness Monitoring
📍 Published at: NMIT Conference, 2025
📚 Authors: Harshad Jadhav, Priyanshu Wagh, Sahil Chalotra, Diptee Ghusse, Sunita Barve
🔗 [PDF Available Upon Request]

---

🚀 Key Features

• 📦 Unified web-based system using Flask
• 🔍 YOLOv11 object detection with real-time performance
• 🛣️ Lane detection using Canny Edge and Hough Transform
• 😴 Drowsiness detection using eye and yawn classifiers
• 🎛️ User control over detection confidence
• 🖥️ Live stream results: bounding boxes, FPS, resolution, object count

---

🧠 System Architecture

[Webcam / Video Input]
↓
[YOLOv11 Detection]  ← COCO, Roboflow Dataset
↓
[Lane Detection (OpenCV)] + [Drowsiness Detection (YOLOv11)]
↓
[Frame Annotator + Flask Streamer]
↓
[Web Dashboard (Real-time Video + Stats)]

---

🧰 Technologies Used

Component	Tech Stack
Object Detection	YOLOv11, PyTorch
Lane Detection	OpenCV (Canny, Hough Transform)
Drowsiness Detection	YOLOv11 (Closed Eyes, Yawn)
Backend	Flask, Flask-WTF, Jinja2
Frontend	HTML5, CSS3, JavaScript
Video Streaming	Flask + OpenCV
Dataset Tool	Roboflow

---

📁 Directory Structure

project/
│
├── flaskaap.py              # Flask backend
├── lane.py                  # Lane detection module
├── motion.py                # Drowsiness detection
├── hubconfCustom.py         # YOLOv11 inference
├── templates/
│   ├── indexproject.html
│   ├── videoprojectnew.html
│   └── ui.html
├── static/
│   ├── images/
│   ├── uploads/
│   └── voice1.mp3           # Audio alert
└── README.md

---

🔧 Installation Instructions

1. Clone the Repo

git clone https://github.com/yourusername/adas-yolov11.git
cd adas-yolov11

2. Create and Activate Virtual Environment

python -m venv venv
source venv/bin/activate   # Windows: venv\Scripts\activate

3. Install Required Packages

pip install -r requirements.txt

4. Run the Application

python flaskaap.py

Now open your browser and go to:

http://localhost:5000

---

🎥 How It Works

📹 Object Detection (YOLOv11)

• Detects cars, pedestrians, bikes, etc.
• Precision: 0.91, Recall: 0.96
• Latency: ~5–6ms per frame (RTX 3060)

🛣️ Lane Detection

• Grayscale + GaussianBlur + Canny + HoughLinesP
• Accuracy: ~85–95% in ideal lighting
• Lightweight for CPU/GPU

😴 Drowsiness Detection

• YOLOv11 trained on Roboflow data
• Classes: Open Eyes, Closed Eyes, Yawn
• Accuracy: 95%+, Alerts via audio overlay

📊 Real-time Dashboard

• FPS, resolution, and object count via AJAX
• Displays annotated video in browser

---

📸 Sample Output

• Total Objects Detected
• Frame Size
• Frames Per Second (FPS)
• Lane overlays and bounding boxes
• Drowsiness warning audio (voice1.mp3)

---

📊 Performance Summary

Module	Accuracy	Latency	Notes
Object Detection	99% P, 96% R	5–6 ms/frame	YOLOv11 on COCO
Lane Detection	~85–95%	33 ms/frame	Canny + Hough
Drowsiness Detection	>95%	50 ms/frame	Closed eyes & yawns

---

💡 Future Enhancements

• 🌙 Night-mode & fog adaptation
• 📱 Android App Interface
• 🎮 Head pose and tilt detection
• 🧠 Audio command support
• 🔌 Jetson Nano edge deployment

---

📜 License

This project is licensed under the MIT License. See the LICENSE file for details.

---

🧑‍🔬 Authors & Contributions

• Harshad Jadhav
• Priyanshu Wagh
• Sahil Chalotra
• Diptee Ghusse
• Sunita Barve

Special thanks to MIT Academy of Engineering, Pune for guidance and resources.

---

📬 Contact

📧 harshad.jadhav@mitaoe.ac.in
🔗 LinkedIn
🏫 MITAOE, Pune

---

🏁 Acknowledgements

• Ultralytics YOLO
• Roboflow
• OpenCV
• PyTorch
• Flask

---

---