Background
The client is a credit score provider focused on the SME segment. Currently, ML models exist to predict different financial KPIs, such as debt, profits, revenue, etc., of a company, based on their historical values. The values of these fields are related and dependent on each other. However, using separate models to predict these values does not allow us to encode these relationships. In some cases, it can produce erroneous results (for example, predicting a company's profits to be greater than the revenue). A model with good predictive power which simultaneously learns these interdependencies would be beneficial while working with annual reports.

Solution
Keeping this issue in mind, we wanted to build one single model that could learn the relationship between the features and predict them auto-regressively. For this task, we chose to opt for a transformer-based architecture, which is often employed in NLP tasks owing to its ability to learn sequential data. Such a model is able to encode sequential information about the financial features as well as information about the year those reports belong.

Tools/Tech
We gathered annual reports for various companies and used them as data for the model. Following this, we specified different relationships between the financial features and enforced constraints on their values. This processed data was then trained using a JAX-based transformer model. The model was trained on two years worth of data – the previous year and the current year and was then used to predict the financial features of the following year.