Wall Street & Technology is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Infrastructure

01:35 PM
Gong Ke Gouldstone
Gong Ke Gouldstone
Commentary
50%
50%

Maximizing CEP Through Content Aware Messaging Middleware

Maximizing CEP Through Content Aware Messaging Middleware


In the financial services industry, people and applications need to act on events in real-time. Knowledge of what the event was and how, why and where it happened needs to be acted on in milliseconds or less. Today’s automated trading applications can make over 30 trades per second by analyzing stock trends, market movements, news and other events that may only be relevant for a fraction of a second. For example, a trading algorithm may infer a positive stock trend line for the next tenth of a second and may purchase the stock for that duration of time. Such applications must incorporate extremely low-latency solutions to stay competitive.

With dramatic increases in market data volume and market volatility, especially with options data (OPRA) and ARCA, trading firms need to revamp and update their technology so that their algorithms can remain competitive. This requires distributed applications that can interact with each other while maintaining sub-millisecond latency from end-to-end.

Latency in trading applications can be broken down into two main categories: application logic and non-application logic. Application logic is the part that makes the decision to buy, sell or hold. It is the “secret sauce” for any trading firm. Non-application logic is the support that helps the application logic make and execute the decision. Traditionally the majority of the latency occurs in the distribution of the data in the messaging layer. A successful trading application needs to minimize latency in both parts of the system.

CEP engines are utilized to optimize the execution of application logic. These engines have the ability to detect patterns of activity from multiple data streams and infer events in a continuous manner. This typically includes the preparation and filtering of the raw data stream so that patterns can be detected and trading logic applied only to data that is relevant. For example, a CEP engine’s input may be the entire Consolidated Quotation System (CQS) market data feed but its algorithm only applies to Apple and Microsoft stock. The data needs to be filtered so that just Apple and Microsoft updates are given to the algorithm. As the data volumes increase, the required filtering of data may take up a large proportion of the CEP engine’s cycles. This results in increased CPU power and bandwidth usage.

To further optimize a trading application's performance and thus its effectiveness, latency incurred by non-application logic can be minimized through the use of intelligent middleware. Traditional middleware, such as Tibco and IBM, uses a broker-based architecture where daemons or servers act as software routers for messages. This design served the data rates of years past, but has reached its limit in both throughput and latency. Next generation middleware uses a peer-to-peer design that avoids the pitfalls of traditional architectures (See diagram below). It also represents a new crop of bleeding-edge technologies that perform additional tasks at the middleware layer so that application logic can perform its best. For example, traditional middleware may send the data to its destination, but intelligent next-generation middleware not only delivers the data to the right destination, but can do so using preferred methods and with the desired level of reliability.

Peer to Peer vs. Broker Based

Leading intelligent middleware vendors offer a key capability that provides profound advantages when combined with CEP engines: content awareness. Content awareness means that the middleware has the capability to process the data in its raw form and be aware of data structures and types. Being content aware has additional benefits such as allowing easy detection of incompatibilities between applications in a trading system, often resulting from improperly staged deployments or updates of trading applications. Last but not least, a content aware middleware makes use of serialization and deserialization code that is generated and optimized in advance, thus decreasing the run-time latency of sending and receiving data. Applied to the previous example, content-aware middleware can filter market data so that a CEP engine only receives updates for Apple and Microsoft. With optimized serialization and deserialization done by the middleware, the CEP engine does not need to decode the data. Reducing the incoming traffic to the CEP engine provides significant savings in CPU utilization. When the filtering is applied on the send side, content-aware middleware significantly reduces network bandwidth usage.

Tests comparing the performance of a leading CEP engine running a simple algorithm on one symbol in simulated market data and the same engine running the same algorithm with middleware filtered market data show savings of over 60% in CPU usage and over 80% in network bandwidth usage. The test was conducted using typical off-the-shelf hardware running Linux over a one Gigabit Ethernet network. Results and performance savings may vary depending on the algorithm and how much data can be filtered through the intelligent middleware. Such CPU performance savings are possible because most CEP engines must convert the input data into an XML format for processing. Consequently, the CEP engine no longer needs to convert irrelevant data into XML if the middleware can filter the data.

As we work our way through these challenging times, firms are looking for ways to do more with less resources and less time. Using CEP engines to perform trading analysis provides the quickest time-to-market and can also ease on-going maintenance. Intelligent next-generation middleware offers significant latency and resource savings. The two technologies combine naturally and trading firms that can take advantage of this integration will continue to succeed in a competitive market place.

About the Author
Gong Ke Gouldstone has been working with real-time middleware and data distribution with a focus on performance optimization. Currently, Ms. Gouldstone is the Technical Account Manager for Financial Services in the Eastern Region for Real Time Innovations, Inc. (RTI), and specializes in helping customers solve problems by building solid and flexible applications that scale with time and demand. Gong Ke earned both her BS and MS Degrees in Computer Science from Massachusetts Institute of Technology

More Commentary
A Wild Ride Comes to an End
Covering the financial services technology space for the past 15 years has been a thrilling ride with many ups as downs.
The End of an Era: Farewell to an Icon
After more than two decades of writing for Wall Street & Technology, I am leaving the media brand. It's time to reflect on our mutual history and the road ahead.
Beyond Bitcoin: Why Counterparty Has Won Support From Overstock's Chairman
The combined excitement over the currency and the Blockchain has kept the market capitalization above $4 billion for more than a year. This has attracted both imitators and innovators.
Asset Managers Set Sights on Defragmenting Back-Office Data
Defragmenting back-office data and technology will be a top focus for asset managers in 2015.
4 Mobile Security Predictions for 2015
As we look ahead, mobility is the perfect breeding ground for attacks in 2015.
Register for Wall Street & Technology Newsletters
Video
7 Unusual Behaviors That Indicate Security Breaches
7 Unusual Behaviors That Indicate Security Breaches
Breaches create outliers. Identifying anomalous activity can help keep firms in compliance and out of the headlines.