Our client partners with system integrators (SIs) to drive sales of its products and services. The SIs engage with potential business customers to identify upselling and cross-selling opportunities. SIs are paid incentives based on revenue generated through audit engagements.

To receive incentives, SIs must deliver valid proof of engagement (POE) documents (email, Word, or PDFs). Currently, verifying POE documents is a manual, time-consuming process. Approval varies depending on who reads the documents. In some cases, reviewer bias leads to incorrect incentive payments.

We used AI techniques to classify POE documents as valid or invalid and restrict incentive payments for invalid documents.

Key Challenges:

• Not enough people are available to review a large number of documents manually.
• Read data from multiple formats (email, Word, PDFs).
• Extract meaningful phrases (keywords) from documents to create a data dictionary.
• Integrate machine learning (ML) model with POE upload application.

We used Text Classification and Feature Extraction algorithms to classify POE documents as valid or invalid.

Key Highlights:

• Created a utility to parse data from different formats into a consumable form. We staged the parsed data in Azure Blob storage.
• Visualized extracted keywords and phrases in a Word Cloud.
• Identified keywords and their frequency using Feature Extraction algorithms.
• Integrated machine learning model output with POE application.

We started by parsing POE documents into a standard consumable format. We then pre-processed the parsed data to remove redundant and stop words. Next, we applied feature extraction techniques (N-grams TF and Unigrams TF-IDF) to convert variable length text into equal length feature vectors. The vectors represent keywords and their frequency in the text.

We trained the output from the two extraction techniques using Two-Class Logistic regression model. We then finalized the best model, based on the optimum KPI values, and deployed it as a web service that is used by the POE application.

Using the Text Classification and Feature Extraction algorithms, our client was able to identify POE documents as valid or invalid, reducing labor and avoiding invalid incentive payouts.

Outcome Highlights:

• Improved consistency since document approval is no longer subject to POE reviewer bias.
• Saved money by no longer paying incentives for invalid POE documents
• Reduced the time needed to validate and approve POE documents.