Testing YOLO models on VLC

06-14-2024

After we had trained YOLO models and uploaded them to our GitHub, the next logical step was to make sure they would run on our AUVs, Graey and Onyx. Our script that controls camera streams, camsVersatile.py, allows us to access YOLO models to run on OAK-D cameras, if the models are correctly named and are in the correct directory. We were able to access the correct model, but unfortunately there were issues with latency, as when we opened VLC to view the camera stream with the bounding box, label, and confidence surrounding the object of interest, as is the defining feature of YOLO models, only the first frame opened up, and then the camera stream froze. VLC additionally gave us the error that basically the camera stream was over five seconds late, and suggested that the computer (Jetson) might be too slow. Additionally, even after changing the label settings inside our .json files that denote the settings for the YOLO model, bounding boxes still output “Class_0” or “Class_1”, depending on the type and number of detections the YOLO model was trained to detect. The conundrum is that running models from 2023, which should be the exact same version as the 2024 models, ran fine in real time, and detected well, while ours seem to cause a very large lag time.

Fig. 1: Example of running Gate 2023 YOLO model on OAK-D. The model detects well when shown game elements from last year, demonstrating it is actually the Gate 2023 model running, and runs quickly in real time on VLC. 

Fig. 2: Example of running Buoy model on OAK-D and trying to access camera stream. Note that the model detects correctly but the label remains “Class_0” and stream freezes after one frame.

Fig. 3: Example of running Gate model on OAK-D and trying to access camera stream. Note the label remains “Class_0”, and additionally stream only outputs this frame, then freezes.

Fig. 4: Example error in terminal that is output when trying to open a camera stream that is running a YOLO model (only applicable to OAK-Ds). 

Fig. 5: JSON file for the Buoy (2024) mission. See that the “labels” list contains only one entry, “Buoy”. Even when this is the case however, the model still shows “Class_0”.

Link to folder containing supplementary videos/pictures:

https://drive.google.com/drive/folders/1rZZtk5gPo0LaVnixybPM5pFSdfxXycDx