The client track everything from the composition of the teams to various events and interactions between players. Our objective was to help them do this at a higher accuracy and speed by building a model which would recognize events and objects in real-time gaming.

A pipeline for synthetic data generation, capable of solving the object detection task was developed. Several state-of-the-art object detection implementations in PyTorch and Tensorflow were tested and tuned. The final solution was based on a Mask R-CNN and improved the existing classical computer vision solution substantially, enabling the client to generalize the solutions to new titles (games) simpler and faster.

The Modulai team worked in close collaboration with the team at client’s side, and handed over the solution for the client to roll out for new titles.

In the absence of large sets of annotated image data, the core of the project was to create a robust pipeline for generating synthetic datasets with images and annotations (location and class of the objects), as well as training a state-of-the-art neural network-based algorithm for detection, localization, and semantic segmentation.