Automatic review of reports

Objective

The challenge involves developing an object detection and OCR model to automate the review of documents that a technician collects during each telecommunication antenna installation and that are then manually reviewed when they are delivered to their base, which could lead to human error.

Description

For this challenge we are only asked to determine the location of 3 fields of the format (2 signatures and 1 date) and to obtain the handwritten date separated in day month and year.

Solution

Our solution is “totally free” and not depending of any api’s that might involucrate a cost in the near future. It is divided into 3 main parts:

• First, we label the dates presented in the documents in order to apply supervised object detection on numbers. We used the free available software LabelImg

• Second, we developed an image preprocessing module, which consists in image aligment, standarization of orientation, size, proportion and extraction of sub-images (sign, date)

• Third, we developed a deep learning module that consists in:
  • A fine-tuned CNN (DenseNet) for sign presence classification
  • A FasterRCNN architecture for detecting and extracting the numbers and separators that appear on date

Code

See the solution and code in the github repository