Automatic Meter Reading With YOLOv4
A deep learning model trained to read automatically a meter from a picture
Introduction
Many tasks that require a big workforce are automated. In many areas of the world, the consumption of utilities such as electricity, gas, and water is monitored by meters that need to be read by humans. The reading of such meters requires the presence of an employee or a representative of the utility provider. With progress in the field of artificial intelligence, the task of reading the meters can be partially automated.
In this article, we’ll present a project on a model trained to read a meter from a picture automatically. The notebook used for the training is on Google Colab. The results are displayed on this web app, and the code is available in this GitHub repository.
Above is an image of a meter with its index reading 1357 m³.
Table of contents
1. Presentation of the data set
2. Goal of the project
3. Standard approach for automatic meter reading (AMR)
4. Implementation of a YOLOv4 model
5. Conclusion
6. ResourcesPresentation of the Data Set
The data set, open under the terms of the Etalab Open Licence, comes from SUEZ, a French-based company specialized in water and waste management.
It has:
- A train set of 793 pictures of water meters annotated by humans
- A test set of 416 pictures of water meters
Goal of the Project
The goal of the project is to read the exact consumption of water in cubic meters. As shown in the picture below, the cubic meters are written in white on a black background. In general in our data set, the water meters show two types of information:
- The cubic meters of consumption, with the digits in white on a black background or the reverse. This is the information we need to extract from the pictures.
- The liters of consumption, with the digits in white on a red background or the reverse. This part of the meter will be ignored for the reading of the index, as only the cubic meters are useful when billing the consumers.
The Standard Approach for Automatic Meter Reading (AMR)
There are three main steps for the reading of a meter by a computer:
- The localization of the meter
- The detection of the digits in the meter
- The classification of the digits
There are many challenges with our data set:
- The pictures have been taken from different angles
- All of the pictures have different sizes
Implementation of a YOLOv4 Model
As YOLO is one of the best convolutional neural network (CNN) algorithms for object detection. We decided to implement our model with a PyTorch version of YOLOv4.
Labeling the pictures of the train set
We defined 12 classes of objects we wanted to detect on the pictures: the 10 digits from 0-9, the part of the meter corresponding to the liters, and the whole counter.
We labeled each picture by drawing the bounding boxes of each object found.
The screenshot below shows a picture labeled on Labelbox, which is a platform where you can annotate your data.
From the left to the right, we’ve annotated two objects of the class 0, two objects of the class 8, one object of the class 6, one object of the class liters, and one object of the class counter, which encompasses all of the six previous objects. So in total, we have seven objects in this picture.
Preprocessing of the pictures
As the pictures have different shapes, we decided to resize them to 608x608. We performed an augmentation on Roboflow by randomly rotating the pictures 90° clockwise, 90° counterclockwise, and upside down.
Roboflow is a platform that enables developers to manage their computer-vision projects with your data. It allows you to integrate your images with their annotations done on Labelbox and to apply some transformations on your images.
Training the model
We did the training on Google Colab with a notebook prepared by Roboflow.
Below are the TensorBoard’s plots of the loss of the last 100 epochs.
The metric is a modified zero-one loss, as the readings should be exact:
⌊⋅⌋ is the integer part of a floating-point number.
With 500 epochs for the training split into four phases, with 100 epochs per phase, we reached a score of 0.63 on the test set.
Conclusion
In this article, we’ve presented our approach to reading a water meter with a YOLOv4-based model. The main problem that we faced is the confusion between some digits. Some room for improvements could be more augmentations, such as blur, brightness, and noise.
Resources
[1] R. Laroca, V. Barroso, M. A. Diniz, G. Gonçalves, W. R. Schwartz, D. Menotti, Convolutional Neural Networks for Automatic Meter Reading (2019), Journal of Electronic Imaging
[2] J. Solawetz, J. Nelson, How to Train YOLOv4 on a Custom Dataset (2020), https://blog.roboflow.com/
