Wall Street & Technology is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management

02:55 PM
By Steve Fingerhut, LSI Corporation
By Steve Fingerhut, LSI Corporation
News
Connect Directly
RSS
E-Mail
50%
50%

High-Performance Financial Analysis Goes Flashy

Accelerating real-time financial analytics and reporting with flash-based storage solutions

In today’s global economy the concept of “after-hours” no longer exists. The days of running reports after business hours to analyze data representing market pressures and opportunities is no longer feasible. Capital market and financial services firms must be able to perform real-time analysis of market data in order to anticipate and respond to market swings and gain a competitive edge.

Steve Fingerhut
Steve Fingerhut, LSI Corporation

These businesses are demanding the latest technology solutions in order to perform data analysis and modeling at high speeds with low latency, and against all relevant market variables. Technology solutions must also drive operational efficiencies to new levels while helping to meet risk and compliance requirements.

Financial institutions are turning to the performance and low-latency benefits of NAND flash memory to accelerate access to data and mitigate the delays associated with traditional hard drive-based storage. Flash storage solutions deliver the performance required to quickly traverse massive data sets and provide real-time analytics and reporting.

Flash in Enterprise Storage Architectures

The use of flash technology in enterprise server and storage architectures is experiencing tremendous growth, and is forecasted to continue to grow rapidly throughout the next decade. Initially, flash technology was packaged as solid state drives (SSDs) since they utilize the same packaging and interface as hard disk drives (HDDs). This allowed SSDs to be easily integrated into existing storage solutions as a HDD replacement. Without the delay of a spinning disk, SSDs support lower latencies and higher transaction rates. However, before organizations move to SSDs, several cautions should be observed.

First, check with your storage array or RAID card provider to verify that the specific SSD you select will be supported. Some storage arrays simply will not use the SSD unless you purchase them from the array manufacturer.

Second, SSD robustness is a concern. Extensive error correction and limited write life mitigation strategies are required for SSDs to provide enterprise reliability. Even major suppliers will limit their implementations, so read available reviews and check with other colleagues about their experiences.

One symptom frequently mentioned is “write cliff,” which occurs when write performance to flash suddenly slows. Writing data to areas on the flash that were previously written is slower than reading data because of the many maintenance tasks. To mitigate this delay, vendors are reserving some areas on the flash for sustained writes. In general, the larger a reserve size, the better the performance. When a vendor’s specification sheet shows a difference between the “raw” and “usable” capacity, that difference usually indicates the size of the reserve area. To save costs, some SSDs will minimize the reserve area, but this negatively impacts write performance.

Third, the interface used by the SSD can limit performance. When accessing SSDs installed in an external storage array, the type of SAN connection used may also limit SSD performance. Typically, Fibre Channel, iSCSI or SAS are commonly used.

A recent study conducted by Demartek found that when iSCSI is used, the SSD performance can be bottlenecked. As iSCSI must encapsulate transactions within a LAN protocol, an appreciable delay in sending the transaction will occur. Compared to a Fibre Channel or SAS connection to the storage array, iSCSI is much slower. While the iSCSI overhead is largely insignificant when using HDDs, it becomes significant when accessing SSDs.

Technologies such as LSI SandForce flash storage processors are designed to address most of the shortcomings of flash in the enterprise. Extensive error correction increases flash reliability to the levels required by enterprise datacenter customers, and techniques that ensure data is uniformly written on the chip extend the lifetime of flash drives and devices. Offloading computation-intensive flash management tasks such as these onto dedicated chips also helps to ensure the performance of financial applications by not requiring the server to perform these lower-level maintenance tasks.

Previous
1 of 2
Next
Register for Wall Street & Technology Newsletters
Video
5 Things to Look For Before Accepting Terms & Conditions
5 Things to Look For Before Accepting Terms & Conditions
Is your corporate data at risk? Before uploading sensitive information to cloud services be sure to review these terms.