Wall Street & Technology is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Trading Technology

10:42 AM

Algo Trading, HFT Wreaking Havoc on Commodities

Recent research showing a big jump in correlation between commodity and equity markets and among commodities themselves, over short periods from a second to an hour, has unleashed fresh bad tempered debate about whether "speculators" - especially high-frequency traders - are distorting commodity prices.

By John Kemp, market analyst for Reuters.

LONDON - Recent research showing a big jump in correlation between commodity and equity markets and among commodities themselves, over short periods from a second to an hour, has unleashed fresh bad tempered debate about whether "speculators" - especially high-frequency traders - are distorting commodity prices.

The full paper on "The synchronized and long-lasting structural change on commodity markets: evidence from high frequency data" is available here.

Predictably, supporters and opponents of regulation (including position limits and restrictions on trading speed) have rushed to praise or damn the report from David Bicchetti and Nicolas Maystre at the United Nations Conference on Trade and Development (UNCTAD) as evidence or not that HFT is pushing commodity prices away from fundamentals determined by real supply and demand.

In a stinging rebuke posted on his website, Houston University Professor Craig Pirrong has accused the authors of over-interpreting their results.

But the real over-interpretation is taking a report on how HFT may be affecting market microstructure over seconds and hours, maybe days and weeks, and project onto it larger questions about whether speculation can move prices away from fundamentals over periods ranging from months to years, and whether that justifies a response from governments.

The report is sensibly silent on these bigger issues. It is a mistake to read it as a manifesto for more regulation on the size, number or speed of positions.

Instead it provides a thoughtful examination of how commodity and other financial markets are becoming more entwined over short timescales, as well as some hypotheses and suggestions for future research about what is driving the shift and what it portends for the way markets operate.


The increasingly close correlation among different commodity prices, and between commodities and other financial assets such as equities, since just before the middle of the last decade has been extensively studied by researchers and is immediately obvious to anyone equipped with a spreadsheet and a correlation function.

But previous research focused on price moves at relatively long (daily) intervals and speculated about the possible role of passive index funds. Bicchetti and Maystre extend the analysis to look at movements over shorter time frames (1 second, 10 seconds, 5 minutes and 1 hour) and whether active market participants such as HFTs have been responsible for enforcing greater convergence.

Using tick-by-tick data from actual trades, sourced from the Thomson Reuters Tick History (TRTH) data set, Bicchetti and Maystre present convincing evidence about how short-term correlations have changed.

At daily frequency, correlations among commodities and with other asset classes have been rising since around 2003-2004. But at shorter frequencies, correlations remained insignificantly different from zero until there was a big increase starting in the second quarter of 2008 and persisting almost continuously since then.

Pirrong points out the structural break coincides with the intensification of the financial crisis and the Lehman Moment, which represented an enormous synchronised demand shock. It is well known that in a crisis all correlations go to one.

"When demand shocks predominate ... positive correlations are to be expected," Pirrong wrote. "In this environment, commodity specific, supply-related information was swamped by systematic demand-related information."

Pirrong's argument is correct, as far as it goes. The authors mention Lehman five times in their paper. But they also point out that while commodity correlations with equities have been rising since mid-2008, around the Lehman insolvency, correlations with the EUR/USD pair started rising earlier, in the summer of 2007.

Crucially, correlations have remained at an unusually high level for most of the four years since 2008 (with a temporary exception in oil linked to the Libyan civil war). It is a big stretch to suggest the markets are still stuck in a four-year long global demand shock (global output, not the U.S. and European economies, are relevant here).


Bicchetti and Maystre examine a variety of possible explanations for increased intra-day correlations (which are especially pronounced during U.S. trading hours from 1300 GMT to around 2000 GMT). None of them provides a sufficient explanation.

Instead, they examine the possible influence of the rising number of HFT-related trades across equity and more recently commodity markets as a possible cause.

The big increase in HFT-related trades has been well documented (though exchanges have been reluctant to break out the rising share of HFT trades for fear of inflaming demands for a clampdown). It has contributed to a surge in the number of trades in NYMEX crude from under 1 million in 2005 to almost 42 million in 2011, and in CBOT corn from 133,000 to 10.7 million, as well as a reduction in average trade size.

It is a truism to say that correlation is not the same as causation. But nothing worthwhile in social science has ever been proved conclusively to this standard. Bicchetti and Maystre do present some convincing arguments for a possible link, though, including the presence of statistical arbitrage programmes operating across multiple markets, and invite further work to establish its extent and the mechanisms.

The authors touch on an even more sensitive area when they speculate about whether HFT programmes import non-fundamental influences onto specific commodity prices.

The paper points out "(conventional explanations) fail to explain how economic fundamentals or the risk appetite of financial investors changed quickly. Indeed, news frequencies or human investors reaction is certainly not as high as 1-second."

Later the authors go on to argue "the strong correlations between different commodities and the S&P 500 at very high frequency are really unlikely to reflect economic fundamentals since these indicators do not vary at such speed."

In other words, HFT traders translate volatility from one commodity or financial market to another at timescales from seconds to minutes by arbitraging them very rapidly.

In a more serious charge, the authors suggest a lot of HFT programmes have feedback characteristics that tend to be self-reinforcing. "Although individually rational, the overall effect of trend following strategies may destabilise markets". Bicchetti and Maystre cite the "flash crashes" in equities (May 2010) and oil (May 2011) as examples of destabilisation.

The potential combination of HFT enforcing tighter correlations, and perhaps also propagating volatility within individual markets through herding behaviour, leads the authors to conclude that "as commodity markets become financialised, they are more prone to external destabilising effects".

That perhaps overstates the case. But critics of the paper must contend with the extraordinary plunge in oil prices on May 5, 2011, which had no apparent trigger other than the market's internal microstructure.


The biggest problem with social science is the tendency of researchers to work backwards from conclusions (based on deep-rooted personal preferences) to find evidence that confirms them. Unfortunately it is rarely possible to conduct empirical research from behind philosopher John Rawls' "veil of ignorance" about how it will be

As a market analyst for Reuters, the views expressed in this Jack Kemp column are his own.

More Commentary
A Wild Ride Comes to an End
Covering the financial services technology space for the past 15 years has been a thrilling ride with many ups as downs.
The End of an Era: Farewell to an Icon
After more than two decades of writing for Wall Street & Technology, I am leaving the media brand. It's time to reflect on our mutual history and the road ahead.
Beyond Bitcoin: Why Counterparty Has Won Support From Overstock's Chairman
The combined excitement over the currency and the Blockchain has kept the market capitalization above $4 billion for more than a year. This has attracted both imitators and innovators.
Asset Managers Set Sights on Defragmenting Back-Office Data
Defragmenting back-office data and technology will be a top focus for asset managers in 2015.
4 Mobile Security Predictions for 2015
As we look ahead, mobility is the perfect breeding ground for attacks in 2015.
Register for Wall Street & Technology Newsletters
Exclusive: Inside the GETCO Execution Services Trading Floor
Exclusive: Inside the GETCO Execution Services Trading Floor
Advanced Trading takes you on an exclusive tour of the New York trading floor of GETCO Execution Services, the solutions arm of GETCO.