High-Frequency Traders Prowl The Web For Even More Data

The amount of market data high frequency traders are using as trading indicators continues to grow more rapidly than Facebook can add users or change its privacy policies, and vendors are scrambling to offer new technology that can help firms - and algos - find the data, sort it and leverage it.

One of the latest vendors to tap into hedge funds' increasing use of micro-blogging data as a trading indicator is Gnip, a provider of public social media data for enterprise applications, which now delivers more than 1 billion social media activities every day to its customers.

At the start of 2011, Gnip was delivering 300 million social activities per month. By May, that number was up to 3 billion activities. And in October, it delivered 30 billion activities. At this rate, the vendor estimates it will be providing its clients with 300 billion activities per month by March of 2012.

In a bid to incorporate the most "relevant" social media data streams into a single solution for hedge funds and high-frequency traders, the vendor has launched a product called Gnip MarketStream, which aggregates high-volume, low latency new media for trend analysis.

The service provides hedge fund managers, traders and signal providers with real-time streams of public data from Twitter and StockTwits, a financial platform that provides real time trading research data and allows investors to share market insights, ideas, charts and news. Ultimately, market data both on and off the web is skyrocketing and as hedge funds try to digest it as quickly as possible and to make intelligent use of it, they need to make sure they have an efficient storage and retrieval platform, can integrate various data sets and can provide tools to help their traders understand the data and patterns. [See 5 Steps To Turning The Market Data Challenge Into An Opportunity in our latest digital issue.]

"In terms of storing data and helping firms deal with data as efficiently as possible, the biggest step is to try to take robust information and consistent information across the attributes of a product transaction or a portfolio," says Peter Sanchez, CEO of Northern Trust Hedge Fund Services, which currently handles one million trades a day, with peaks of 4 million transactions.

The need for more robust attributes around data is more pressing for hedge funds than for traditional asset managers, since they need to get more accurate pricing on a real time basis, he adds.

With the aim of helping its clients sift through market data more efficiently, Northern Trust Hedge Fund Services provides its clients with the ability to access, configure and manage live data across current and historical transactions, as well as to generate heat maps, dot plots, pie charts and bar graphs through its hedge Fund Passport Application Suite.

In the meantime, as they continue to lap up more data, hedge funds are rapidly regaining lost ground. According to McKinsey analysis for Northern Trust, hedge fund assets will have reached an estimated $1.8 trillion in AUM by the end of 2011, and are expected to grow to $2.4 trillion in 2013.

Melanie Rodier has worked as a print and broadcast journalist for over 10 years, covering business and finance, general news, and film trade news. Prior to joining Wall Street & Technology in April 2007, Melanie lived in Paris, where she worked for the International Herald ... View Full Bio