AI/ML, big data in finance: advantages and effect on business models/activity of financial sector participants
The growing availability of data in financial services and the potential competitive advantage that AI/ML can provide to financial firms are the primary drivers behind the adoption of AI in finance. With the explosion of available data and analytics, coupled with more affordable computing capacity such as cloud computing, machine learning models can analyse data to identify signals and capture underlying relationships in ways that are beyond the capability of humans. The deployment of AI/ML and big data by financial sector firms is expected to improve their efficiency by reducing costs and enhancing the quality of financial services products demanded by customers. According to the US Treasury, this deployment will increasingly drive firms’ competitive advantage.
This section examines the potential impact of AI and big data on financial market activities, such as asset management, trading, lending, and blockchain applications.
2.1. Portfolio allowance in asset administration and the broader investment society(buy-side)
The integration of AI and ML in asset management has the potential to boost the efficiency and accuracy of operational workflows significantly. It can also improve performance, enhance risk management, and elevate the customer experience (Blackrock, 2019) (Deloitte, 2019). NLG, a subset of AI, can help financial advisors simplify data analysis and reporting to clients by making it more understandable and relatable (Gould, 2016).
In addition, ML models can monitor thousands of risk factors daily and test portfolio performance under thousands of market and economic scenarios, thereby enhancing risk management for asset managers and other large institutional investors. AI can also lead to operational benefits such as reducing back-office costs of investment managers, replacing manually intensive reconciliations with automated ones, and potentially cutting down on costs and processing time.
Asset managers can leverage big data to feed Machine Learning models and obtain recommendations that can influence decision-making around portfolio allocation and stock selection, depending on the AI technique. Big data has replaced traditional datasets and is considered a commodity readily available to all investors. Asset managers are using big data to gain insights into their investment process. Details have always been vital for the investment community, and data has been the cornerstone of multiple investment techniques, from fundamental research to systematic trading and quantitative strategies. While traditional approaches were based on structured data, vast amounts of raw or unstructured/semi-structured data can now provide a new informational edge to investors deploying AI in implementing their strategies. AI enables asset managers to quickly digest vast amounts of data from multiple sources and uncover insights that can inform their strategy in very short timeframes.
Using AI/ML and big data may pose a challenge for smaller asset managers or investors who lack the resources to invest in AI technologies. This can make it difficult for them to adopt such techniques and explore new big data sets. Additionally, deploying AI and proprietary models can provide a performance edge against competition, resulting in restricted participation by smaller players. As a result, this trend could reinforce the concentration of more prominent players in the industry as they outpace some of their more nimble rivals. These observations were made by the Financial Times in 2020.
The industry may continue to see limited participation by smaller players until the point where third-party vendors make these tools ubiquitous. However, the accuracy and trustworthiness of third-party datasets may not be held to the same standard across the industry. Therefore, users of third-party tools must build confidence in the integrity of big data to reach a level of comfort sufficient for them to adopt these tools.
It’s important to note that using identical AI models by multiple asset managers may result in herding behaviour and one-way markets. This could pose potential risks to liquidity and the system’s stability, especially in times of stress. Large sales or purchases executed simultaneously could cause market volatility, creating new vulnerabilities (refer to Section 2.2).
The increasing use of AI/ML and big data in investing has the potential to reverse the trend towards passive investing. If these technologies continue to produce consistent alpha-generating results, it could establish a cause-and-effect relationship between the use of AI and superior performance. This presents an opportunity for the active investment community to revive active investing and offer alpha-adding opportunities to their clients, as reported by Blackrock in 2019 and Deloitte in 2019.
2.1.1. Implementation of AI-powered hedge budgets and ETFs
As per Kaal’s (2019) research, hedge funds have been at the forefront of FinTech users and have been utilising big data, AI, and ML algorithms in trade execution and back office functions. Additionally, in recent years, a new category of hedge funds called ‘AI pure play’ has emerged that solely relies on AI and ML, such as Aidiyia Holdings, Cerebellum Capital, Taaffeite Capital Management, and Numerai.
Surprisingly, there has not been an academic or independent review of the performance of AI-powered funds from a non-industry source. This could help in comparing the various funds that claim to be AI-driven. However, fund managers may not be willing to disclose their methodologies to maintain their competitive edge. Additionally, the levels of AI usage and the maturity of deployment of AI may differ significantly among funds that market themselves as ‘AI-powered’, making it difficult to compare their performance.
Private sector-provided indices of AI-powered hedge funds demonstrate superior performance compared to conventional hedge fund indices offered by the same source (as shown in Figure 2.2). However, it is essential to note that indices provided by third parties may be subject to biases, such as survivorship and self-selection bias of constituents to the index or backfilling, and should be used with caution.
Note: The Eurekahedge Hedge Fund Index is Eurekahedge’s primary index consisting of 2195 constituent funds with equal weighting. The index aims to provide a comprehensive measure of the performance of all underlying hedge fund managers, regardless of their regional mandate. The index is base-weighted at 100 as of December 1999, does not include duplicate funds, and is denominated in local currencies.
On the other hand, The Eurekahedge AI Hedge Fund Index is an equally weighted index of 18 constituent funds. It aims to provide a broad measure of the performance of underlying hedge fund managers who utilise AI and ML theory in their trading processes. The index is base-weighted at 100 as of December 2010, does not contain duplicate funds, and is denominated in USD.
Finally, the Credit Suisse Hedge Fund Index is an asset-weighted hedge fund index that includes both open and closed funds.
Currently, the size of exchange-traded funds (ETFs) powered by artificial intelligence (AI) is insignificant. These ETFs rely on models to make and execute investment decisions. According to estimates, the total assets under management (AuM) of such ETFs were around USD 100 million by the end of 2019. Using AI to manage ETFs can help reduce management fees, which averaged around 0.77% per year by the end of 2019. Evidence shows that AI models outperform conventional methods in forecasting macroeconomic indicators like inflation and GDP. This is especially true during economic crises when accurate predictions matter the most. AI-driven techniques have also proven superior in identifying meaningful but previously unknown correlations in financial crisis patterns. Studies show that ML models outperformed logistic regression in out-of-sample predictions and forecasting.
2.2. Algorithmic Trading
AI has a wide range of applications in trading, including providing trading strategy suggestions and powering automated trading systems. These systems can identify and execute trades independently using AI techniques such as evolutionary computation, deep learning, and probabilistic logic. Algo wheels and other AI techniques can help traders systematically strategise upcoming transactions, enabling them to implement an “if/then” thought process as a procedure. With the increasing interconnectedness of asset classes and geographies, AI’s predictive capacity quickly outpaces conventional finance and trading algorithms.
AI-powered trading systems can assist traders in effectively managing their risk and order flow. With the help of AI-based applications, traders can easily track their risk exposure and make necessary adjustments or exit positions as per their requirements without manual reprogramming. These systems are designed to train independently and adapt to the changing market conditions, with little or no human intervention. Additionally, they can aid traders in managing their flows among their brokers and controlling fees or liquidity allocation to different pockets, depending on the regional market preferences, currency determinations or other parameters of order handling. This information was sourced from Bloomberg in 2019.
In highly digitised markets like those for equities and FX products, AI-powered solutions hold the potential to provide competitive pricing, manage liquidity, and optimise and streamline execution. One of the most significant benefits of AI algorithms deployed in trading is their ability to enhance liquidity management and execution of large orders with minimum market impact. This is achieved by optimising size, duration, and order size based on market conditions.
The use of artificial intelligence (AI) and big data in sentiment analysis is a practice that innovative technologies have augmented. This practice is not new, as traders have been mining news reports and management announcements for decades to understand the stock price impact of non-financial information. Today, text mining and analysis of social media posts, tweets, and satellite data through natural language processing (NLP) algorithms have become examples of the application of such technologies. They can inform trading decisions by automating data gathering and analysis and identifying persistent patterns or behaviours on a scale beyond human processing capacity.
I completely agree with the statement. AI-managed trading differs from systematic trading because of the reinforcement learning and adjustment of the AI model to changing market conditions. On the other hand, traditional systematic strategies may take longer to adjust parameters due to the heavy human intervention involved. A conventional backtesting approach based on historical data may not deliver good returns in real-time as previously identified trends break down. However, the use of ML models shifts the analysis towards prediction and analysis of trends in real-time, for example, using ‘walk forward’ tests instead of backtesting. Such tests predict and adapt to trends in real-time to reduce over-fitting (or curve fitting) in rear trials based on historical data and trends.
Box 2.1. AI-based algo revolutions
An algo wheel refers to a system that automates the routing of orders, ranging from fully-automated solutions to mostly trader-directed flow. An AI-based algorithm is a model that utilises AI techniques to assign a broker algorithm to orders from a pre-configured list of algorithmic solutions. It helps to select the best strategy and broker to route the order based on market conditions and trading objectives/requirements. In other words, an AI-based algo wheel automates assigning the optimal design and broker to an order, depending on the specific needs and conditions of the market.
Investment firms use algo wheels for two main reasons: firstly, to improve execution quality and achieve performance gains; secondly, to automate small order flow and standardise broker algorithms into consistent naming conventions, thereby enhancing workflow efficiency. Advocates of algo wheels claim that they mitigate the impact of trader bias when selecting a broker and their algorithm in the market.
Interestingly, algo wheels are gaining acceptance to categorise and measure the best-performing broker a lgos. As per estimates, almost 20% of trading flows go through algo wheels, and those who use it do so for 38% of their algo flow. If algo wheels are widely adopted, it could increase the overall level of electronic trading, potentially benefiting the competitive landscape of electronic brokerage.
The use of AI in trading has indeed evolved. Initially, trading algorithms were simple buy or sell orders with basic parameters. But as time went by, more complex algorithms were developed, allowing for dynamic pricing and better execution strategies. Nowadays, deep neural networks provide the best order placement and execution style that can minimise market impact. These networks are designed to recognise patterns and learn from data input, mimicking the human brain. By using such techniques, market makers can effectively manage their inventory and reduce the cost of their balance sheet. As AI technology advances, these algorithms are expected to become even more automated and less reliant on human intervention.
In trading, advanced AI systems are primarily utilised for identifying signals from “low informational value” incidents, which are harder to identify and extract value from. Rather than improving the speed of execution, AI is deployed to extract signals from data noise and convert this information into decisions about trades. On the other hand, less advanced algorithms are often used for “high informational events”, which consist of financial news that is more obvious for all participants to pick up and where speed of execution is critical.
At present, ML-based models are not focused on front-running trades and profiting from the speed of action, unlike HFT strategies. Instead, they are primarily utilised offline for tasks such as calibrating algorithm parameters and enhancing algorithmic decision-making. However, as AI technology continues to advance and is employed in more use cases, it has the potential to expand the capabilities of traditional algorithmic trading, which could have significant implications for financial markets. This is anticipated to happen when AI techniques are more widely adopted in the execution phase of trades, enabling increased automation of trade execution and the entire process from signal detection to strategy formulation and execution. ML-based algorithms for execution will enable the autonomous and real-time adaptation of their decision-making logic while trading. Under such circumstances, the requirements that are already in place for algorithmic trading, such as pre-trading risk management systems with built-in safeguards and automated control mechanisms to turn off the algorithm if it exceeds the limits embedded in the risk model, should be extended to AI-driven algorithmic trading as well.
2.2.1. Unintended consequences and possible risks
The use of the same or similar trading models by many traders can have unintended consequences for competition and contribute to market stress amplification. If widely used models emerge, traders’ arbitrage opportunities naturally decrease, eventually benefiting consumers by reducing bid-ask spreads. However, this can also lead to convergence, herding behaviour, and one-way markets, potentially threatening market stability and liquidity, particularly in stressful periods. Like any algorithm, the wide use of similar AI algorithms poses the risk of self-reinforcing feedback loops that can trigger sharp price movements.
It’s important to consider that the convergence of AI could potentially increase the risk of cyber-attacks. This is because it becomes easier for cyber-criminals to influence agents acting similarly rather than autonomous agents with distinct behaviours. Furthermore, a significant cyber risk is associated with using AI nefarious. In such cases, AI has the potential to conduct independent attacks on vulnerable systems in trading and financial market systems without any human intervention. This could have severe consequences for the entire financial ecosystem and its participants.
When users of ML techniques are unwilling to reveal their model workings due to fear of losing their competitive edge, it raises issues related to the supervision of algorithms and ML models (as seen in Section 3.4). Using proprietary models that cannot be copied is crucial for traders to maintain any advantage they may have. Still, it can also lead to an intentional lack of transparency, further contributing to the lack of explainability of ML models.
The use of algorithms in trading has raised concerns about the potential for collusive outcomes and illegal practices in digital markets. For instance, the lack of explainability of ML models used to back trading could make it difficult to adjust the trading strategy during poor trading performance. Trading algorithms are no longer model-based linear processes that can be easily traced and interpreted, which makes it challenging for traders to understand the underlying drivers of trading decisions. Additionally, the lack of transparency in AI-driven systems makes it difficult for supervisors to identify illegal practices, such as “spoofing,” if collusion among machines is in place. Even in times of over-performance, users cannot understand why the successful trading decision was made, which makes it challenging to identify whether the performance is due to the model’s superiority or pure luck.
Using AI technologies in trading and HFT may lead to unintended consequences, such as increased market volatility resulting from rough large sales or simultaneous purchases. This could create new sources of vulnerabilities that the market may not be prepared to handle, as seen in some algo-HFT strategies that have contributed to extreme market volatility, reduced liquidity, and flash crashes. Given that HFT are a significant source of liquidity provision under normal market conditions, any disruption in the operation of their models during times of crisis could result in a liquidity crisis, potentially impacting market resilience.
Considering the possible effects of market participants’ massive use of ‘off-the-shelf’ AI models on liquidity and market stability is essential. This could lead to herding behaviour and one-way markets, which in turn could amplify volatility risks and increase the likelihood of unexpected changes in the market, both in terms of scale and direction. Additionally, the absence of ‘shock-absorbers’ or market makers who can take on the opposite side of transactions might result in illiquid markets.
As AI is increasingly deployed in trading, it has the potential to improve the interconnectedness of financial markets and organisations in unpredictable ways. This could potentially lead to increased correlations and dependencies of previously unrelated variables. For instance, the scaling up of algorithms that generate uncorrelated profits or returns may cause correlation in unrelated variables if their use reaches a sufficiently significant scale. It can also boost network effects, such as random differences in the scale and direction of market movements.
To minimise the risks associated with using AI in trading, it may be necessary to implement defences to protect against AI-driven algorithmic trading. Pre-trading risk control systems are developed to prevent and stop possible misuse of such systems. Interestingly, AI is also being used to develop better pre-trade risk management systems, including mandatory testing of every algorithm release and applying equally to AI-based algorithms. Automated control mechanisms that instantly switch off the model are market practitioners’ ultimate lines of defence when the algorithm goes beyond the risk system. These mechanisms involve “pulling the plug” and replacing technology with human handling. However, from a policy standpoint, these agencies may not be optimal as they switch off the operation of the systems when they are most required during times of stress and can create operational vulnerabilities.
Implementing defences at the exchange level where trading occurs may also be necessary. These defences could include automatic order cancellation when the AI system is turned off for any reason and measures that provide resistance against sophisticated manipulation techniques enabled by technology. Circuit breakers, which are currently triggered by significant drops between trades, could be adjusted to identify and be started by large numbers of smaller transactions made by AI-driven systems, which would have the same effect.
