Traditionally, risk management methodologies have applied the laws of mathematics and physics to our financial markets, with the aim of quantifying all sorts of objects in this artificial economy. Correlations are an intrinsic assumption in almost all approaches to financial instrument pricing and the quantification of risks. But most correlations are derived from historic time series – and will only fulfil their own prophecy until periods of stress or crisis occur.

Risk managers therefore need to look beyond correlations to understand the knock-on effects and the causal structure of the market. But are they seeking to explain the inexplicable: the human behaviour that shapes the markets’ state? And could big data help shine a new light on this great unknown?

There is no doubt that, in recent years, the role of risk management has changed. With trade lifetime costs analysis and all associated simulation-based methodologies becoming a day-to-day part of business for financial institutions, managing such “X value” adjustments (XVA) at inception is vital for both risk-based pricing and all-round profitability. That said, the fundamental function hasn’t altered or expanded beyond all recognition. At the heart of the risk manager’s role is the need to steer the organization through either business as usual or, more problematically, periods of stress.

In the first scenario – those preferable times of normality – it is the objective of risk managers to prevent their organization from defaulting or failing due to careless errors. In the second, the emphasis shifts: making survival the immediate priority. But the main difference between the two scenarios lies in the varying behaviour of the financial markets themselves. And here lies a defining challenge for risk management. One may argue that any significant differences in behaviour can be captured by the financial models and explained as being “beyond the confidence interval”: accepted, in other words, as predicted outliers. Alternatively you could say that in a financial markets crisis, any applied theory is likely to fail.

Science versus nature
From Merton to Markowitz, risk managers have a whole arsenal of Nobel Prize-winning methodologies and pricing models at their disposal when it comes to gaining insight into the financial markets. Intrinsic to these theories is the concept of volatilities and correlation: the statistical measure of how market factors like foreign exchange rates, interest rates and stock prices move in relation to one another. Volatilities and correlations provide important input parameters for the pricing of derivatives. In other words, correlation is usually key to creating a diversified portfolio and, in turn, to mitigating risk.

So far so logical. But in truth, correlation only works properly when the markets are functioning normally. The plain fact is that risk management methodologies assume an efficient, rational and complete market with no arbitrage, reasoned decisions, and the ability to buy and sell any stock. And in a market crisis, grounds for this assumption all but disappear. In periods of stress, theories begin to falter and causal patterns of movement tend to dominate.

Clearly, when markets are under stress, a multitude of causes and effects will still be at work; in fact, the relationships between them may be stronger than ever. But with so many participants involved, these links become more complex, less predictable and harder to explain – especially when using the static and inflexible models of physics and mathematics. The plain reason for this is that financial industry participants apply simple models to explain the financial markets – i.e., something entirely created by human beings. And since when has the human mind conformed strictly to scientific principles?

Art versus science
In their struggle to explain the financial markets, risk managers and regulators have already identified that correlations may change more quickly than some models recognize – and, as a result, have increased the focus on stress tests and introduced the concept of stressed value at risk.

Stress testing has become an important tool, enabling risk managers to prepare for the adverse market conditions that will most affect their business. In many ways, however, stress tests can be seen more as an art than a science, crucial as they are to making potential crisis scenarios real and plausible to senior management. Through the credible stories that stress tests can tell, risk managers are in a stronger position to explain their strategies and encourage board-level buy-in.

Prediction versus explanation
Stress tests, however, only go some way to preparing organizations for periods of stress. They can help to clarify a risk management strategy, but they can’t fully explain the complex workings of the financial markets – or the multitude of causes and effects that create and compound a crisis. Again, this comes down to the fact that at the root of these causalities are human beings, whose behaviour, thought patterns and decisions science has sought to explain for centuries – rarely with total success.

Applied financial theory assumes that that the parameters for pricing and risk assessment coincide with those that define the structure of the financial markets. But maybe, for now, risk management needs to take a slightly different tack. Based as they are on current and historic time series of observable market prices, correlations implicitly make precise assumptions about something that, quite simply, we cannot explain: the financial markets.

Perhaps, rather than trying – and failing – to explain the markets’ underlying structure, the aim should be first to predict how they will look in the future – by using what we know already to discover what we don’t. This may, at first, sound impossible: how, after all, can we predict the state of the world without upfront knowledge of its parameters? The answer lies in big data, and our ability to capture and reuse all the information at our disposal.

Other industries are already pointing the way. In retail, for example, the US chain store Target has developed an algorithm to predict the probability of pregnancy in a female customer, based on her past and current buying behaviour. Crime-fighting, meanwhile, is fast catching up with science fiction. In Chicago, police are now using big data to identify which individuals are most likely to commit future crimes. In Los Angeles, a wealth of statistics is helping determine where crimes are likely to happen. And in Europe, research projects are underway to gather data for predicting both crime and terror.

Correlations won’t end
Risk management, of course, cannot change overnight, embedded as much of it is in both tradition and regulation. But, as in other industries, use of big data can start to complement existing tools, theories and methodologies. So, while correlations will remain, gathering more data and storing a greater number of attributes could improve their flexibility and give risk managers greater opportunities for structured analysis.

In terms of analytics, there is still some way to go before risk managers can reap the full benefits of big data. Nevertheless, technology providers are taking a step in the right direction by improving organizations’ ability to collect risk data for analysis. Here the enabler is the in-memory database which, when incorporated into a risk management solution, makes it easier to constantly add new data attributes, and store and rapidly query huge amounts of real-time data. This in
turn provides levels of flexibility and performance that simply aren’t achievable through historical data technology.

With such a mine of continually updated information at their fingertips, risk managers can finally edge a little closer to understanding all the complexities of the financial markets. It may only be the start of the journey, but the possibilities are endless – and certainly worth exploring.

Dr Sven Ludwig is senior vice president for risk management and analytics for EMEA at SunGard and regional director at PRMIA, an international risk management association with more than 80,000 members globally. Before joining SunGard, Ludwig headed up the trading, risk, wealth management and custodian business IT at a major German bank. Ludwig started his professional career as a financial consultant for trading and risk management.