Modulai Blog

Introduction

The esports industry has seen tremendous growth lately. Each tournament is streamed live and reaches several million viewers all around the world, increasing the demand for live updates of games, players, e.g. for live betting and more.
Such information can sometimes be accessed directly through the games’ servers and API integrations, but in most cases, it is limited in volume and accessibility.

To improve the experience of watching these tournaments, Abios Gaming provides an API for live information on games, teams, and players. To strengthen Abios Gaming’s offer, and to enable real-time monitoring of esports games, Modulai joined forces with their tech team to build a deep learning object detection solution, to extract information on-the-fly from real-time video streams of gaming tournaments. The system was required to detect both text and icons, and the extracted information was then used to update the status of live matches (e.g. Counter-Strike: Global Offensive (CS:GO), League of Legends, Dota 2, and Fortnite). Moreover, the framework needed to be general enough to be applied to new games easily, in an AWS production environment.

In this post, we will walk through how we addressed the first selected use-case (CS:GO). We will introduce some essential methodological pieces (Such as Scene text detection [1] [2] to detect and recognize text from game frames, and Mask R-CNN [3] to detect icons) as well as synthetic data generation. We then describe the results and end with some conclusions and lessons learned.