Test Finds Infrastructure Eliminates Latency Spikes

With market data message rates skyrocketing, Wall Street firms are worried about data latency affecting the behavior of their algorithms. One cool technology that is getting some attention comes from the networking side where market data platforms have traditionally used Ethernet. On March 20, Securities Technology Analysis Center (STAC) published a report examining the performance of the latest software from Wombat Financial Software running

The test was conducted on servers using Linux on AMD Operteron Processors. Although the test was financed by Cisco, "Paying for a test does not equate with influencing the test and results or getting good results," says a Cisco spokesperson. "STAC ensures that the tests they do are neutral and fair in every way," adds the spokesperson.

STAC discovered that the combination of these technologies eliminated nearly two-thirds of the mean latency experienced by the Ethernet network, which is the network medium on which most clients run their market data systems."When trading firms use the InfiniBand SFS instead of the Ethernet fabric, it is much faster and produces lower latency," says Jack Gidding, associate director of STAC, which began the test last December and finished up in mid-January.

Using message update sizes of 214 bytes - which is the default size that Wombat uses in its benchmarks - the STAC test found that latency was reduced by 63 percent, from 220 microseconds to 80 microseconds. When larger packet sizes with 1 kilobyte (KB) and 6 KB were used, latency reductions of more than 50 percent were demonstrated. In addition, the test simulated a real market data feed, "with intense activity and periods of disruption," says Peter Lankford, director of STAC, a consulting firm that conducts research into the latest technology stacks for the capital markets industry.

In this case, the goal of the test was to scrutinize the behavior of Wombat's market data solution on Cisco InfiniBand, as compared to an identical system using 1-gigabit Ethernet, according to the report. Though InfiniBand technology is available from other vendors, this test focused on Cisco InfiniBand. "The tests were not conducted against any competitors - they were just comparing two different data transport protocols. Both are offered by Cisco," says the Cisco spokesperson.

"While the scope of the test was limited, STAC believes that the results show great promise for the Cisco InfiniBand solution to reduce the latency and increase the predictability of market data delivery for algorithmic trading, market-making, arbitrage and other time-sensitive functions in a number of markets," states the report.

The results come at a time when the market data messaging rates are growing exponentially. In January the Options Price Reporting Authority(OPRA) notified the industry that data rates are going to exceed 700,000 messages per second by January of 2008.

The results are significant for automated trading environments with algorithms where applications run on servers and there's communication that happens between the servers, says Pramod Srivatsa, senior product manager, Cisco Server Virtualization Business unit. If the messages take a long time to transmit between one server and another or between layers of servers, this can result in slippage, he notes.

Hedge funds are interested in this kind of infrastructure because it could improve the "hit rates" of algorithms executing orders in liquidity pools, says Srivatsa. Hedge funds that track slippage find that only four out of five trades are executed because "when they go back to the liquidity pool, the market has moved and they'd do the trade," says Srivatsa. Only 20 percent of the trades actually get done. "Their big challenge to us is to increase this number even from 20 to 30 percent. That in itself would be a big increase in our profits," says Srivatsa.

Another benefit of the InfiniBand infrastructure is that it eliminated latency spikes of 30 milliseconds. "In the tests, they have a period of disruption when there is a lot of intense activity going on and the applications are making lots of requests for data when there are extreme amounts of background traffic," says Lankford.

"The spikey period was at a time in the tests when the consuming applications - the trading applications - were blasting the systems asking for messages," explains Gidding. The application is requesting 100,000 instruments back from the publishing application. "This is where we see the disruption,' says Gidding.

On the Ethernet system "there were spikes, brief periods when latency jumped up, but on the InfiniBand infrastructure they couldn't see any difference between the disruptive period and the steady-state period," says Lankford. This is important because algorithms prefer predictability of latency, Lankford explains. "Once you get down in sort of the sub-millisecond range, a lot of the algorithms have difficulty dealing with uncertainty surrounding the latency itself," adds Lankford. "The predictability or consistency of latency, as long as it's below a certain threshold, is more important than the average latency," he says.

Srivatsa says InfiniBand is faster than Ethernet due to remote direct memory access (RMDA) technology, which is the transfer of information from the memory of one computer to that of another without involving the operating systems of either. "The fast messages get through extremely quickly between servers and you can have a lot of capacity," says Srivatsa. Lankford says the benefit is not just in the networking switch, but the rest of the components that offload processing to the server. "A lot of the networking work that used to be done in the CPU gets offloaded to the network cards. That definitely is part of the speed. "This will give more CPU processing power to the algorithm that is actually doing the trading," says Gidding.

Ken Barnes, Wombat's VP of business development and planning, says that clients of Wombat, including hedge funds and investment banks, are gearing up for deployment over the next quarter or two. Though Wombat has not been involved in any exchange deployments at this point, "I am aware of some of the exchanges investing in this as well," says Barnes. Ivy is Editor-at-Large for Advanced Trading and Wall Street & Technology. Ivy is responsible for writing in-depth feature articles, daily blogs and news articles with a focus on automated trading in the capital markets. As an industry expert, Ivy has reported on a myriad ... View Full Bio