Many financial forecasting models do well in controlled tests but break down when they’re exposed to real market conditions and longer trading horizons. Using supercomputing at the Bristol Centre for Supercomputing, researchers in the UK developed and evaluated more than 600 AI models on their ability to predict financial returns under time-aware conditions. They found that those trained only on financial data produced more reliable return forecasts than much larger general-purpose AI models developed by major technology companies, giving investors and institutions clearer proof of which AI models can be trusted in real markets.
Getting predictions right
Financial forecasting is difficult to get right and easy to get wrong. Markets are typically noisy and forecasting models that look convincing in tests often fail when applied across global markets and over longer periods. These models guide when to invest and what financial assets to hold. For example, banks use forecasts to set their risk limits and determine capital requirements, while investment managers rely on them to build portfolios and meet target returns.
When forecasts fail, organizations incur trading losses and underestimate downside risk during market volatility, with losses often flowing through pension funds and other investors. “Finance is a very good challenge for AI forecasting models because the signal is extremely weak,” explains Dr. Eghbal Rahimikia, assistant professor in financial technology at the University of Manchester’s Alliance Manchester Business School. “It’s very difficult to find predictive information. Even with conventional models, it’s very hard to build a profitable trading model. Otherwise, everyone would be making money.”
As large language models have become prominent, similar architectures have appeared in time series forecasting. These models are applied to time series data and often trained on large and mixed datasets, based on the assumption that scale alone can improve performance. They are known as time series foundation models (TSFMs) and are being developed with backing from major technology companies.
Rahimikia and his collaborators wanted to know if this assumption holds when forecasts are tested under conditions that mirror actual financial decision-making. Using supercomputing at the Bristol Centre for Supercomputing at the University of Bristol, the team’s FinText.ai project pretrained hundreds of TSFMs using finance-specific data taken from global markets. In the study titled Re(Visiting) Time Series Foundation Models in Finance, they found that TSFMs trained on financial data consistently outperformed substantially larger off-the-shelf TSFMs in predicting returns.
Stress-testing forecasting models
FinText.ai grew out of practical questions about how modern AI models behave when applied to financial settings. Early work done by Rahimikia and his collaborators, called Re(Visiting) Large Language Models in Finance, also tested large language models for trading. They found that while general-purpose large language models perform well in other areas, their behavior in finance was inconsistent and hard to interpret, raising doubts about whether scale alone was enough.
“Finance lets you test these models under genuinely difficult conditions,” says Rahimikia. “It allows you to compare them and see which ones actually hold up.”
FinText.ai was structured to answer this question by training models using only financial data and comparing hundreds of models side by side, instead of working with small selective samples.
One reason finance is useful for testing forecasting models is that weak evaluation is more difficult to disguise when predictions are tied to money and risk. In many AI benchmarks, models are scored on predefined tasks that might overlap with their training data.
This generates impressive results that don’t normally carry over into real markets. As Rahimikia points out, “Those benchmarks are often very different from actual use.”
Return prediction in finance can be tested through trading simulations and portfolio construction as well as key risk indicators, which reveal when a model’s signals are too weak to be useful. FinText.ai evaluated models using statistical metrics and portfolio performance over extended simulations. This helped distinguish models that only reproduced historical patterns from those that generated signals with economic value.
“Statistical accuracy is an important first step,” says Rahimikia. “But it’s the financial performance that really counts.”
Testing models without data leakage
Look-ahead bias is a major problem in financial AI. Models often learn from information that wouldn’t have been available at the time a forecast was made, which makes their reported performance unreliable once they’re deployed.
To avoid this risk, the FinText.ai project structured the data year by year, starting with 1990 records, when global market data became deep and consistent enough to support reliable modelling.
This year-by-year data used for pretraining time-aware models reduced the risk that apparent improvements were due to hidden data leakage. It also ensured that AI models were trained and tested under conditions closer to how financial forecasts are actually used. The dataset covers daily excess returns and financial factors across 94 countries, making it possible to compare models consistently.
Separating robust models from weak ones
Using Isambard-AI at the Bristol Centre for Supercomputing, the project trained and evaluated more than 600 AI models. All models followed the same training and evaluation process so differences in performance could be traced back to data and modelling choices, and not to testing variation.
This level of testing revealed limits that smaller studies usually miss. Traditional linear models, for example, struggle to process terabytes of datasets efficiently. In contrast, transformer-based models evaluated in the project can be trained in parallel across many GPUs, which enabled the team to run the same experiments consistently across hundreds of models.
The project used more than 50,000 GPU hours on Isambard-AI for pretraining and fine-tuning models, with an additional 25,000 CPU hours for benchmark evaluation on separate systems. In addition to the HPE Cray EX Supercomputer, Isambard-AI is configured with HPE Slingshot 11 Interconnect, Cray ClusterStor E1000 Storage System, and NVIDIA® GH200 Grace Hopper Superchips, all integrated within a dedicated modular AI data center, the AI Mod POD from HPE.
“Isambard-AI made scaling possible,” says Rahimikia. “Without it, we would have had to test on a few stocks, over a few months.”
Finance-specific training performs better
One of FinText.ai’s standout findings is that model size alone doesn’t explain forecasting quality. Models trained specifically on financial data, even when much smaller than leading general-purpose AI models, consistently produced stronger predictions of financial returns.
Using these smaller models, investment managers and banks running daily equity signals can train models directly on current market data and review their behavior before they go live. This kind of scrutiny is important when predictions feed into trading decisions. For example, it affects how much capital is committed to a trade or how losses are managed during volatile periods.
“Our results show that you don’t need one model that does everything,” says Rahimikia. “You need the right data and you must benchmark properly. You need models that are built for the task.”
Attracting interest from industry
FinText.ai has attracted strong interest from financial institutions looking for more reliable ways to develop and evaluate AI models, according to Rahimikia.
“We built the models mostly for academics, but the response from industry has been much, much bigger.”
He has presented the work to investment companies and financial market intelligence providers. And in just a few months, the study has been downloaded more than 8000 times.
“People want to understand how these models were tested and how AI models can be correctly evaluated before being used in live markets,” adds Rahimikia.
Opening the work to scrutiny
The team has openly released all the models produced through FinText.ai, making the project one of the largest controlled comparisons of financial forecasting models published so far.
“We’re not saying these models are perfect,” says Rahimikia. “We’re saying, this is the code, this is what we did. Test it yourself.”
Releasing the models allows other researchers to check the results and build on the study without starting from scratch. Rahimikia sees this transparency as important to encourage independent verification of forecasting claims in finance.
“Using untested models in real markets is like releasing a medicine without clinical trials. It’s dangerous. Our work shows a direction for how this new generation of AI models can be tested properly for financial forecasting.”