Title: Algorithms for identifying generalisation capabilities and degradation in time of machine learning predictive models to improve AI Investments, inbestMe and Holisun current products and services.
Acronym: GenDeg
This project aims to innovate time-series forecasting model assessment by monitoring their generalizability and degradation over time. The resulting methods will work with various AI algorithms across different AI applications. These advancements will be integrated into AI Investments, inbestMe, and Holisun services, offering novel features.
These features are groundbreaking in finance and beyond, utilising recent ML and AI breakthroughs to revolutionise investing. Selecting better future strategies (out-of-sample results) and monitoring model degradation (vanishing alphas), could reshape the investment industry. Demonstrating applicability to other sectors through Holisun, the project enhances services for all three participants.
Identifying and measuring model degradation in time:
The project will focus on tackling the 'vanishing alphas' problem, identifying when a predictive model's effectiveness has declined to the point of losing its predictive advantage. This will lead to significant benefits for clients of AI Investments and inbestMe by preventing capital loss from investing in degraded models and also improving the performance of Holisun predictive maintenance. Algorithms and methods will be studied to determine permanent model degradation. An implementation will be made to identify real-time predictive models with degraded performance, indicating the need for retraining and reevaluation of their prediction accuracy and generalisation capability.
Model degradation refers to the decline in the performance or effectiveness of a predictive model over time. In the context of quantitative finance and investment strategies, model degradation becomes a significant concern because these models are built based on historical data and assumptions. As financial markets are dynamic and constantly changing, models that worked well in the past might lose their predictive power and become less accurate or even irrelevant in the future. This assumption applies to other business areas as well.
Why model degradation can occur:
Changing Time Series Dynamics: Financial time series and other time series are influenced by various factors, including economic conditions, geopolitical events, technological advancements, and investor sentiment. As these factors change, the relationships and patterns in the data that the model was trained on might no longer hold, leading to decreased predictive accuracy.
Data Quality and Availability: Predictive models rely on data to make accurate forecasts. If the quality of the data used deteriorates or if new data becomes available that the model was not trained on, it can impact the model's performance.
Parameter Drift: Some models have tunable parameters that need to be optimised during training. If these parameters are not regularly updated or adjusted to account for changing market conditions, the model's effectiveness may decline.
Concept Drift: Concept drift refers to the situation where the underlying relationships between variables in the data change over time. This is common in financial markets due to shifts in investor behaviour, economic policies, or market structure.
Detecting and addressing model degradation is crucial for maintaining the performance of quantitative investment strategies. When a model's effectiveness declines, it can lead to suboptimal investment decisions and potential financial losses. Therefore, continuous monitoring of the models is essential to identify when retraining or updating the model is necessary.
Assessing Generalisation Performance:
Generalisation is understood as the ability of a model to accurately predict outcome values for previously unseen data.[1] Predictive models are trained during a training period and then, after training, their performance is verified during a test period. While testing the model on a single separate period is a dominant practice it cannot reliably measure the model's ability to generalise on different periods, as they can differ in trend, seasonality and patterns.
Determining the generalisability of predictive models is also a key innovative feature of the improved services of InbestMe and AI Investments, as well as Holisun, as it will allow the selection of only those predictive models to run that will have a high ability to good performance on future periods (and not just show good results on historical periods). This project will research and develop methods and algorithms that will help determine the ability of the model to generalise to periods with different characteristics. It will allow us to estimate the probability of achieving favourable results regardless of future time series run. We will research methods based on advanced, generative machine learning algorithms.
This will be a unique functionality of AI Investments' enhanced services, inbestMe and Holisun, none of the competing solutions analysed have this functionality.
By incorporating these features, we'll offer unmatched cutting-edge services, revolutionising quantitative investing. This will optimise strategies better, minimise model degradation, and select predictive models with superior performance
Funding programme: EUROSTARS 3 Call 5
Project duration:
36 months (01/02/2024 - 31/01/2027)
Budget:
- Total budget: 2.2M€
- Holisun budget: 600k€
Steps and reports:
- Analysis of the design requirements and the implementation data from the partners -
- Development of the GenDeg platform -
- Testing the GenDeg platform 06/12/2025 - 05/12/2026
- Validation of the GenDeg platform 06/12/2026 - 28/02/2027
Partners:
