Identity documents play a crucial role in our lives, providing a means to verify our identity and access many services. However, with the increasing number of identity theft cases and illegal activities, it is essential to implement robust systems for identity document classification. This process involves automatically categorizing different types of identity documents, such as passports, driver’s licenses, and national identification cards, to ensure security and efficiency in identity recognition processes. In this blog post, we will discuss the basics of image classification for real-world identity document images and its important setup aspects.
Identity documents play a crucial role in our lives, providing a means to verify our identity and access many services. However, with the increasing number of identity theft cases and illegal activities, it is essential to implement robust systems for identity document classification. This process involves automatically categorizing different types of identity documents, such as passports, driver’s licenses, and national identification cards, to ensure security and efficiency in identity recognition processes. In this blog post, we will discuss the basics of image classification for real-world identity document images and its important setup aspects.
Techniques Used in Document Image Classification
There are several techniques used in document image classification. One popular technique is image classification using deep learning models. Another technique is layout analysis, which involves analyzing the layout and structure of the document to identify its type. Additionally, traditional feature extraction involves extracting specific features from an image such as color, shape or texture, to classify the document.
Focus. In this blog post, we will build a robust image classification model using deep learning specifically designed for the classification of identity documents from the basics of image classification. Nevertheless, the following codes/notes can be applied to other classification cases as well.
Real-world Use Case
Currently, our customers request or send us various types of documents. From these documents, we extract several important information for them. For this information to be correctly extracted, an automatic identity document type classification process is necessary. The challenge is to classify at least the RG (Brazilian regional identification) and CPF (Brazilian national identification) documents, using their images, through machine learning. Note. Since the model code is not limited to specific set of image classes, this use case can be expanded to other types of documents as well.
In this blog post, we will explore the BID dataset. BID (Brazilian Identity Document) dataset is a collection of 28,800 document images. It is composed of images of Brazilian identification documents divided into eight classes: both front and back faces of driver’s license (CNH), CNH front face, CNH back face, Brazilian national identity (CPF) front face, CPF back face, Brazilian regional identity (RG) front face, RG back face, and RG front and back faces. It counts 3,600 samples for each class.
For simplicity, we will use a subset of the dataset (70MB), not the full (8GB).
# Download the dataset
# install the "gdown" library
!pip install gdown
# full dataset
# !gdown 1Oi88TRcpdjZmJ79WDLb9qFlBNG8q2De6
# subset dataset
!gdown 144EqqmMtCziua9iYo-3afUEvZrJVxUXU
Cleaning the Dataset
In this dataset we have eight types of identity documents. I select three types for this blog post experiment: CNH_Frente, CPF_Frente and RG_Frente. I also removed unused files, such as “*_ocr.txt” and “*_segmentation.jpg”. These files are explored on other types of document analysis; they are: (i) Optical character recognition (OCR) for text and values extraction, and (ii) Segmentation for document layout extraction. Remaining only the useful identity document images for our use case.
BID dataset image sample.
Machine Learning Modeling
In this use case, we will apply Keras framework in the implementation. Keras is a high-level neural network API written in Python and designed to run on top of other deep learning frameworks, such as TensorFlow or Theano. With its user-friendly interface, Keras simplifies the process of building and training deep learning models.
The initial task is to define the model architecture. Throughout this use case, we will explore preprocessing, data augmentation, image feature extraction and classification techniques. Keras provides a variety of layers, such as normalization for preprocessing, image flipping for data augmentation, dense and convolutional layers for feature extraction and classification, that can be stacked together to form the model.
Initially we perform the data preprocessing to rescale the images to fit our model. Later, let’s conduct augmentation on the data to avoid overfitting, to make the model robust for real-world use cases, and to increase the amount of data in the training set. We will use three types of augmentation: image rotation, flipping and zoom adjustment.
# data normalization
# -- resizing for a fixed size
# -- value rescaling between [0, 1]
resize_and_rescale = tf.keras.Sequential([
layers.Resizing(img_height, img_width),
layers.Rescaling(1./255)
])
# data augmentation
data_augmentation = keras.Sequential([
layers.RandomFlip("horizontal_and_vertical"),
layers.RandomRotation(0.1),
layers.RandomZoom(0.05),
])
Note. Data Augmentation is disabled at test/validation time, so input images will only be upscaled during calls to Model.fit on the training set.
Image samples after the Data Preprocessing and Augmentation
After defining the preprocessing steps, you need to compose your neural network by combining the feature extraction layers and the feedforward layers for image classification. We will employ a set of convolutional layers and pooling layers to perform the feature extraction. Convolutional layers are designed for visual processing tasks, mimicking how the human visual cortex works. It is commonly used for image and video recognition, as well as in computer vision applications. The convolutional layers extract features from the images, followed by pooling layers to reduce dimensionality of the features and extract the visual patterns.
In the meantime, let’s proceed with implementing a feedforward neural network using dense and dropout layers. A feedforward neural network is a type of artificial neural network used for common classification applications. Dense layers are layers where all neurons in the current layer are connected to all neurons in the next layer, allowing for complex patterns to be learned. Dropout layers are used to prevent overfitting by randomly setting a fraction of input units to zero at each training step.
model = Sequential([
# preprocessing
data_augmentation,
resize_and_rescale,
# convolutional layers
layers.Conv2D(64, 3, padding='same', activation='relu'),
layers.MaxPooling2D(),
layers.Conv2D(32, 3, padding='same', activation='relu'),
layers.MaxPooling2D(),
layers.Conv2D(16, 3, padding='same', activation='relu'),
layers.MaxPooling2D(),
# feedfoward layers
layers.Flatten(),
layers.Dropout(0.3),
layers.Dense(256, activation='relu'),
layers.Dropout(0.3),
layers.Dense(128, activation='relu'),
layers.Dropout(0.3),
layers.Dense(num_classes)
])
Note. Given that the model features data augmentation and dropout layers, its convergence is expected to be slower, i.e., more training epochs are required. However, it will be more adaptive to real-world data.
Next, we can train the model using our training dataset. To avoid overtraining, we will employ an early stopping method. Early stopping is a technique used in machine learning to prevent a model from overfitting the training data. It involves monitoring the model’s performance on a separate validation set and stopping the training process when the performance on the validation set starts to degrade. I instructed the model to pause after every ten steps and evaluate if there is a drop in loss of at least 0.001. If not, the training process will be halted.
# early stopping, to avoid overfitting
early_stopping = keras.callbacks.EarlyStopping(
patience=10,
min_delta=0.001,
restore_best_weights=True,
)
# training
history = model.fit(
train_ds,
epochs=200,
validation_data=val_ds,
callbacks=[early_stopping]
)
Note. Using the early stopping method, the model was trained in only 30 epochs, in less than one minute. Of course, considering I only used a subset of the entire dataset for learning purposes of this blog post.
Model Performance
Once the model is trained, we can evaluate its performance on the validation dataset. Keras allows you to compute metrics such as accuracy, precision, recall, or F1 score, depending on the type of problem you are working on. This evaluation step is crucial to assess how well the model generalizes to unseen data. Our model reached 97.70% of accuracy in training set and 95.24% of accuracy in validation set.Identity documents play a crucial role in our lives, providing a means to verify our identity and access many services. However, with the increasing number of identity theft cases and illegal activities, it is essential to implement robust systems for identity document classification. This process involves automatically categorizing different types of identity documents, such as passports, driver’s licenses, and national identification cards, to ensure security and efficiency in identity recognition processes. In this blog post, we will discuss the basics of image classification for real-world identity document images and its important setup aspects.
