The Road to Big Data

As big data becomes inescapable in today's financial services industry, institutions can prepare by surveying the analytics landscape, targeting a specific problem to solve, studying relevant cases and expanding capabilities with the right technology solution for their environment.

Organizations today are inundated with data from transactions, sensors, and a mix of day-to-day communications as companies track what we do, what we say, and how we feel. The financial services industry is far from immune to the big data megatrend, and some may even contend that it is helping to lead the charge. There is no question that big data stands to forever change how financial services institutions (FSIs) do business – presenting new business opportunities, unexpected challenges and even competitive threats.

Big data is inescapable in today’s financial services industry – from new types and levels of customer information flowing into the organization to record volumes of transactional data. FSIs not only want to make productive use of this potential goldmine of information, but are being required to leverage it to comply with increasingly stringent regulatory requirements for stress testing and reporting.

While FSIs may have a conflicted view of big data – understanding its potential but uncertain how to fully realize it, there is good reason to move forward boldly. Given that the financial services sector is one of the most data-driven industries, the opportunities afforded by this phenomenon are tremendous.

Failure to act can have significant financial consequences and financial services executives are reporting that their inability to effectively manage big data is having an impact on their bottom line. C-level financial services executives, in a recent Oracle-sponsored research study, said that their companies are unable to realize, on average, 12 percent of additional revenue per year by not being able to fully leverage the information they collect .

The question most often asked when contemplating a big data initiative is “Where do we start?” The simple answer is “Assess and then start small.” Financial institutions can prepare for the big data opportunity by surveying the analytics landscape, targeting a specific problem to solve, studying relevant cases and expanding capabilities with the right technology solution for their environment.

Survey the Analytics Landscape

Get familiar with emerging trends in analytics, strategies for analytical transformation and the variety of solutions that are available to help financial services institutions deal with big data. Technology has advanced rapidly in the last few years, meaning that organizations no longer need to transform unstructured data to structured data for analysis. Industry tools, such as Hadoop and Hive, have helped decrease computing time, even for large data sets. Because data manipulation does not need to be consolidated in order to optimize performance, FSIs have a number of new analytical advantages they can exploit, such as:

• Unstructured data analysis – Distributed data grids with MapReduce-style processing enable organizations to avoid transforming unstructured data to a structured format for analysis.

• Dynamic information discovery versus static analytic outputs – New techniques allow data discovery or ad-hoc data analysis with better performance results as opposed to pre-defined analytical requirements.

• Real-time vs. batch – New computing advantages and the availability of all critical data, as well as infinite history data, allow for real-time computing, with the most potential in fraud/compliance areas.

FSIs should analyze applications currently running in their environments to fully understand the opportunity for these applications to work in big data. How do the applications of one department work with other departments, and how do they need to work together? By understanding what solutions FSIs can leverage and which they need to acquire, organizations will obtain a solid understanding of their business needs.

Target a Specific Problem to Solve

Fully grasping big data is a big undertaking, best achieved by tackling problems one at a time. Isolate the most important area to tackle first, such as the detection of rogue trading based on transaction and accounting records. A capital markets firm utilizing an analytics platform that correlates accounting data with position tracking and order management systems can provide insights that are not available using traditional data management tools.

Depending on your organization’s management goals, the following areas are strong candidates for utilizing big data:

• Sentiment Analysis ‒ Sentiment analysis is considered straightforward, as the data resides outside the institution and is, therefore, not confined by organizational boundaries. In fact, sentiment analysis is becoming so popular that some hedge funds are basing their entire strategies on trading signals generated by Twitter analytics.

• Predictive Analytics ‒ Encompassing correlations, back-testing strategies and probability calculations using Monte Carlo simulations, these analytics are the bread and butter of all capital market firms, and are relevant both for strategy development and risk management. The large amounts of historical market data and the speed at which FSIs sometimes need to evaluate new data (e.g. complex derivatives valuations) certainly make this a big data problem.

• Risk Management ‒ As we move closer to continuous risk management, broader calculations, such as the aggregation of counter-party exposure or value at risk (VaR) also fall within the realm of big data, if only due to the mounting pressure to rapidly analyze risk scenarios well beyond the capacity of current systems, while dealing with ever-growing volumes of data.

• Rogue Trading ‒ A less common use case ‒ but one that is frequently discussed as we're faced with increasing implications ‒ is rogue trading. Deep analytics that correlate various systems can provide valuable insights that are not available using traditional data management tools. Here, too, a lot of data needs to be crunched from multiple, inconsistent sources in a very dynamic way, requiring some of the technologies and patterns discussed in earlier posts.

• Fraud ‒ Correlating data from multiple, unrelated sources has the potential to catch fraudulent activities earlier than current methods. Consider, for instance, the potential of correlating point of sale (POS) data (available to a credit card issuer) with web behavior analysis (either on the FSI's site or externally), and cross-examining it with other financial institutions or service providers, such as First Data or SWIFT. This would not only improve fraud detection but could also decrease the number of false positives (which are part and parcel of many travelers' experience today).