Wall Street & Technology is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management

06:34 AM
Sean O'Dowd
Sean O'Dowd
Connect Directly

Building A Big Diverse Data Platform For Wall Street

Although there is no one panacea for increasing the analytical quotient in capital markets and there is no big data analytics in a box, there are, however, components or architectural ingredients that are essential in getting value throughout the information supply chain.

The entire capital markets industry is buying into the big data hype, or what I'm calling business intelligence optimization (BIO). I touched on this change in wording in an earlier piece that wasn't focused on ‘big data', but the process to harvest and exploit new knowledge -- regardless of where the data resides.

In reviewing this definition, there is no panacea for raising the analytical quotient in capital markets. There is no big data analytics in a box. There are, however, components or architectural ingredients that are essential in getting value throughout the information supply chain. And, how are those insights operationalized to best impact the bottom line? There are straightforward and relatively simple approaches that offer different ways to build a platform, without the usual complexity associated with data analytics projects.

To build a successful BIO platform, start by understanding:

Data Skills: Talent is scarce in the data science/quant market to leverage and utilize these data and tools.

Standards: Standards, data quality and governance are changing rapidly and not fully formed, so deployment and management must be taken in measured steps.

Culture: Some companies may not be internally built for BIO. A data driven culture that is willing to experiment and understand there are some activities that can generate immediate ROI and others that will take some time is required. It's called continuous improvement and failing fast, and some firms will not be able to access fast insights.

Complexity: It's not easy. Firms have difficulty mapping and stepping through the fundamental factors for any project -- people, process and a clear vision for a "future state" or "next generation" platform -- and BIO is more complex than most projects.

Dedication: Let's not mince words -- it's hard. This will take dedication. The "edge" firms can gain takes hard work, time and patience. Gartner suggests that only 20% of the global 1,000 organization having a strategic focus on "information infrastructures" by 2015.

[For more on how Wall Street organizations are approaching big data challenges, read: Big Data Could Transform Global Financial Markets.]

Is there a path of least resistance? At this time and for the near-term future, a unified data architecture (UDA) is one of the best ways to build a platform that not only enables and operationalizes analytics, big data, BIO or information -- whatever term is used -- but also leverage existing IT assets. The goal is to take current systems, introduce new technology only when needed, and maintain flexibility so the company can rapidly develop applications and analytics that result in better performance. This is a hybrid ecosystem.

The components of a UDA include open source Hadoop, a discovery platform and an integrated data warehouse. Hadoop is an excellent solution for data staging, preprocessing and simple analytics among other applications. The data discovery platform should enable rapid exploration of data using a variety of analytic embedded techniques accessible by business users. And to operationalize findings and insights, a warehouse puts factors, rules or other outputs from Hadoop and the discovery platform into an integrated environment for production to be used across the organization or for a specific business project or unit.

The current combination of components are not just adding value, but also enabling speed to market. A UDA is a sound approach that injects new complimentary components, but does not compete with the fundamental architecture. Rather it enables firms to fill voids in their systems and create the capabilities to rapidly develop analytics and applications. Most importantly, it delivers business intelligence that is not just interesting, but actionable.

All the speed, analytics, insights and connectivity are nice, yet without an actionable outcome to improve the business, the former is just a piecemeal approach that is creating an expensive big data science experiment.

Sean O'Dowd leads the Global Capital Markets program at Teradata for Industry and Marketing Solutions. In this role Sean focuses on industry strategy, marketing and field enablement. Areas of focus span financial market structure, regulations and technologies that impact the ... View Full Bio
More Commentary
A Wild Ride Comes to an End
Covering the financial services technology space for the past 15 years has been a thrilling ride with many ups as downs.
The End of an Era: Farewell to an Icon
After more than two decades of writing for Wall Street & Technology, I am leaving the media brand. It's time to reflect on our mutual history and the road ahead.
Beyond Bitcoin: Why Counterparty Has Won Support From Overstock's Chairman
The combined excitement over the currency and the Blockchain has kept the market capitalization above $4 billion for more than a year. This has attracted both imitators and innovators.
Asset Managers Set Sights on Defragmenting Back-Office Data
Defragmenting back-office data and technology will be a top focus for asset managers in 2015.
4 Mobile Security Predictions for 2015
As we look ahead, mobility is the perfect breeding ground for attacks in 2015.
Register for Wall Street & Technology Newsletters
5 Things to Look For Before Accepting Terms & Conditions
5 Things to Look For Before Accepting Terms & Conditions
Is your corporate data at risk? Before uploading sensitive information to cloud services be sure to review these terms.