Deep learning-based automated visual inspection system for production testing

In this thesis, a new test station was created for the production line of mobile devices. This thesis delves into the creation of both hardware and software. This production testing station is a 60 cm*40 cm*40 cm box that has a camera attached to the roof, filming down on a device that is inserted into the box. This box was created with modifiability, speed, and simplicity in mind. The hardware was designed from scratch and two working test station boxes were built during this thesis.

The software can be split into two parts: the test station software and the deep learning models. The test station software, based on a template from other implementations, runs the entire system, including taking a picture of the device, scanning its data matrix sticker, and testing the device with two custom-trained deep learning models. The deep learning models were created using a pretrained YOLOv9 model. This pretrained model was trained further with 93 original pictures with 8 different classes of components, that had been data annotated using polygon annotations. The 93 images were data augmented to 330 images and the models were trained for object detection. The models detect components, the test station software tests these findings against predetermined areas where components should be found on a device. Also, if a confidence score of a prediction is too small, it will be disregarded. Based on these results, it shows the operator of the test station in production if the device is correctly or incorrectly built, and results are uploaded onto a database.

This thesis succeeded in creating a test station for production testing that achieved most of the requirements. The test was extremely consistent in predicting correctly with some components that had plenty of data available. Some bigger components turned out to be difficult for the models to predict. From validation, the best component had an accuracy of 97.8% of correct predictions, whereas all the components with little data during training had unreliable scores in validation testing. The average precision for all classes in testing was 57.7%. The best performing component had over 10000 data points in training data, whereas the worst performers had only 330 total. Validation testing set was an additional set of 99 images after augmentation. In practice, these deep learning models could be used well in production. For the bigger and less reliable components’ prediction confidence limits could be lowered and this resulted in a test station that could confidently confirm if a device was built correctly.