PPE A.I. Vending Machine - Purdue Polytechnic Senior Capstone

Problem Statement

Ensuring factory-wide workplace PPE compliance is a significant challenge. Traditional monitoring through manual checks or signage frequently results in team injuries due to inadequate PPE usage.

Our team has developed an intelligent vending machine system to combat this issue, utilizing AI camera technology to sense if a worker wears correct PPE and a deployment system to properly dispense missing items to them.

Our Solution

Initial Solution Sketch

Completed AI PPE Vending Machine

PySide6User Interface Demo

Testing the PPE Detection

Safety Gate Lock - With Electro-Magnetic Lock

Our result produced a smart vending machine that detects if a user is PPE compliant and will let them through the safety gate. If the user is not compliant, the machine will not let them through and will dispense the missing PPE item.

Utilizing a Yolo computer vision model, the PPE items are detected and their detection is displayed on the graphical user interface through the color indicators on the PPE buttons.

The user can click on the buttons of missing PPE for the machine to dispense the item.

An override button is also provided to let users through the safety gate in case of emergency.

Final Design

Camera

An Onsemi Ar0821 Camera
8MP at 4k resolution
High Dynamic Range for dynamic lighting conditions
Up to 60 frames per second to handle fast moving objects/people
Auto integration for accurate color computation algorithms

Vision Model (You Only Look Once)

YOLO Vision Model — YOLO Model Architecture

A single-shot model looks at the image only once
Splits the image into grids, called cells
For each grid, calculates the class probabilities, the box predictions and the confidence scores
Uses the three vectors calculated to display the output boxes on the image

Graphical User Interface

Application developed with PySide6
Python backend API communication
PPE buttons show if respective PPE is detected, safety gate is unlocked if all PPE is detected
PPE items are dispensed when respective buttons are pressed
Override button overrides all detection states and opens the safety gate lock

Vending Machine Integration

Vending machine is connected with two Avend SmartVend Kits
The Nvidia Jetson communicates with them through the HTTP REST protocol to request items to be dispensed
ESP32 is used to demonstrate a version of safety gate lock implementation

View Detailed Final Design

Team Members

Team 54 Members — From left to right: Adrian Calderon, Christopher Campbell, Aditya Prabhu, Cameron Johnson, (Vending Machine), Max Chen, Ryan Hay

Future Work

Software

Currently the GUI is a very simple script written with PySide6, the application could be further improved with proper Qt application development workflows.
An earlier version of the GUI was written with PyQt5 integrated with ROS 2, it also explores implementing dispensing tracking, inventory management, as well as Amazon AWS integration for robust data analytics.

Model Optimization

This computer vision model is trained using public datasets, but to improve accuracy and reliability, the model should be re-trained on custom datasets with more controlled environments, mainly lighting.
Another approach would be utilizing prompt engineering with other kinds of AI models such as OpenAI's GPT-4.

Hardware

We are at the "proof of concept" stage of the project, so all hardware aside from the main Jetson computer can and should be improved at a later stage.

Safety Gate

Actual safety gate implementations requires strict adherence to related safety codes.

Future Improvements — Completed Vending Machine System

View Detailed Future Work

Customer Background

Packaging Systems of Indiana is a total corporation based in Lafayette, IN. As a focus on sustainability, the corporation's goal is to provide environmentally friendly options across Distribution, Waste, Recycling, and more.

Machine operators are required to a widespread set of various PPE, but video review shows that workers often don't wear PPE such as ear plugs, despite company requirements. The goal of the project is to detect whether or not they are in compliance, and to encourage case of wearing it.