One of the most mundane and repetitive jobs of today is verifying signatures - you are presented with a signature and you compare it to reference signatures, deciding whether they come from the same person or not. The comparison can serve as a proof of identity, authorization of transactions or similar. This is one of the steps in detecting fraud, which is a major and ever-growing problem. Luckily for us, mundane and repetitive is where AI comes in.
BonsAI has designed and developed a custom Artificial Intelligence solution specialized for signature verification capable of human levels of precision.
We call it Cognitive Signature Verification System, based on a Siamese neural network. It is capable of detecting various properties such as edges, curves, specific points and textures. To train it we used Nesterov's Accelerated Adaptive Moment Estimation (Nadam) algorithm, an improved version of the gradient-based RMSprop with momentum, which minimizes the train data bias and ensures good results when generalizing (using unseen data).
We input an image of a document with the desired signature we want to compare and the reference signatures. The system then outputs a likelihood of the signature being valid (from the same person as reference signatures).
To measure how successful our solution is, we use three success metrics:
Accuracy - probability that the system will correctly classify a signature (valid or invalid)
Precision - probability that signatures marked as valid truly are valid
Recall - probability that truly valid signatures are marked as valid
Our system compares two signatures and outputs a level of confidence on the matching rate of the two signatures. To determine how sure we want to be before classifying a signature as valid we can set a threshold. Output below that threshold will mark a signature as invalid, while output above the threshold will mark the signature as valid.
Considering we can arbitrarily set the likelihood threshold, the system is as infallible as we want it to be. Setting the threshold is a compromise between high precision and recall, both affecting accuracy.
On this graph we can see achievable metrics for our model, depending on which threshold we set.
The highest accuracy we can achieve is 95%. This means that 95% of the signatures will be correctly classified. The other 5% could be valid signatures classified as invalid or invalid signatures classified as valid. Both recall and precision would also be 95%.
If 95% is not enough we can tweak the system to be more conservative, thus achieving precision of more than 99.99%, while having the recall of 26%. This means that there is a less than 0.01% chance an invalid signature gets validated! The system would be almost infallible, while potentially reducing the workload over 26%!
The performance can be improved given a larger dataset of signatures which require verification.
While not perfect, the system can take a huge chunk of the workload while maintaining human level of precision. It is also faster and more cost-effective than doing it manually.
If you have any questions or want to talk about the details of implementation, do not hesitate to contact us!