Background
On Device Research, ODR is a kickass UK-based tech company that has a world-class platform that allows users to complete online surveys in the exchange for a monetary reward. ODR, and businesses alike, can attract fraudulent users trying to cheat the system for their own gains. It’s essential to protect the business from such users, while at the same time not punishing good users. Furthermore, it’s crucial to have a good user experience, so that the platform can attract and keep new customers, as well as suppliers of surveys.

Solution
We trained, in close collaboration with the talented team at ODR, multiple models per market to streamline ODR’s business operations. Classifiers were trained to predict the likelihood of a survey being finished with bad intent – one using features describing the user, context, and survey. A ranking system was developed to sort the surveys available to a user based on relevance. Finally, model results were aggregated across many tasks and used for user segmentation to enable the personalization of application logic based on segments.

Tools/Tech
Feature engineering was done in relational databases. Gradient boosting models were trained using Dask to handle the large dataset size and the trained models were deployed in AWS using various managed services. Training and validation pipelines were developed in Python with DVC.