An invention by scientists at the École Polytechnique Fédérale de Lausanne (EPFL), Switzerland, could save stock market players and people who like to invest their money in shares from unpleasant surprises and financial losses in the future. Based on methods from topological data analysis, the scientists have developed a model that can be used to predict changes or crashes in the stock market.
Topological data analysis (TDA) takes information from clouds of data points and uses that information to identify patterns, classify data and predict trends, for example. Scientists at EPFL’s Laboratory of Topology and Neuroscience are currently working with researchers at EPFL spin-off L2F and HEIG-VD on a model that uses TDA to predict when a system is about to undergo a major change. This model, giotto-tda, is available as an open-source library. It can help analysts tell when a stock market crash is coming, for example. But it should also work for events such as earthquakes, traffic jams, coups d’état or locomotive malfunctions.
Predicting the unpredictable
Disasters and other unexpected events are difficult or impossible to predict using conventional methods. That’s why scientists used TDA methods to develop a novel approach. This is based on the fact that “when a system reaches a critical state, such as when water is about to solidify into ice, the data points representing the system begin to form shapes that change its overall structure,” they explain. So by keeping a close eye on a system’s data point clouds, they can identify the system’s normal state and thereby also detect when an abrupt change is about to occur. Another advantage of TDA is that it is insensitive to noise, meaning that signals are not distorted by irrelevant information, the scientists point out.
Until now, TDA has been used primarily in fields such as medical imaging, fluid mechanics, materials science and 3D modeling. But with giotto-tda, the method also works for modeling many different types of data sets, such as gravitational waves. “The data contained in these datasets feeds the model’s machine learning algorithm, improving the accuracy of its predictions and providing warnings.”
Noise and muddled signals
As test events for giotto-tda, the researchers used data from the stock market crashes of 2000 and 2008 by looking at daily price data for the S&P 500 from 1980 to the present and comparing it to the predictions generated by their model. The S&P 500 is usually considered a benchmark for the state of the financial market. Here, the price-based chart showed numerous spikes that exceeded warning levels in the run-up to the two crashes. “Conventional forecasting models contain so much noise and give so many signals that something is about to go awry that you don’t really know which signals to follow,” said Matteo Caorsi, project team leader at L2F. “If you listen to them all you’ll end up never investing because there are very few times when the signals are truly clear.”
With giotto-tda, however, the signals were very clear. The peaks were well above the warning level, so they clearly indicated that crashes were coming. This would show that TDA can be used to detect volatile movements that may indicate an impending crash, the researchers said. But these results only cover a specific market and a short period of time, which is why the scientists now plan to conduct more research. “The next step will be to apply TDA to deep-learning methods. That will give us valuable information about our model, how interpretable its results are and how robust it is,” Caorsi says.
