Wall Street & Technology is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management

04:50 PM
Greg MacSweeney
Greg MacSweeney
Connect Directly

Wall Street Firms Are Not Prepared for Big Data

New data sources, including social media and mobile devices, can provide valuable information and insights into customer tendencies and trends. But firms do not have the infrastructure or skills to handle the deluge of unstructured data.

With the technology and information economy in full swing, financial firms should be in a good (if not a great) position to capitalize on the growth of digital information. After all, data has been the name of the game for financial services firms for decades — who better to benefit from the leap in availability and usage of data?

But Wall Street firms, while proficient at handling market data and certain financial information, are not well prepared for the recent explosion in unstructured data. Data from a variety of non-traditional sources -- including the web, social media, mobile devices and more — is flooding data warehouses.

These new data sources churn out data at a phenomenal rate, easily dwarfing the volumes of reference data created daily in the global markets. And they can provide valuable information and insights into customer tendencies and trends. Firms, though, do not have the infrastructure or skills to handle the data deluge.

Dealing with big data requires new technologies as well as new skill sets. Traditional database technologies, for example, fall short of handling massive amounts of data (think: petabytes). And the talent needed to analyze large datasets is just starting to be nurtured and developed in many universities.

But financial firms can't wait for the next crop of graduates to arrive with these new skills in three or four years. Big data solutions and methodologies are so new and evolving so quickly that financial firms need to develop these skills on the fly. Firms need to invest more in training and workforce development to bridge the knowledge gap if they want to take advantage of what big data can offer.

Meanwhile, one major U.S. school apparently doesn't think the technology economy is for real. With the need for new skill sets becoming more apparent every day on Wall Street, it's appalling that the University of Florida is closing its computer science department. That's right — at a time when companies in all industries are clamoring for more tech talent, the Gators apparently think that computer science is an outdated major. Talk about short-sighted. Maybe they simply didn't see all of the statistics about the growth of big data.

[Facebook's Acqusition of Instagram Is All About Big Data.]

Greg MacSweeney is editorial director of InformationWeek Financial Services, whose brands include Wall Street & Technology, Bank Systems & Technology, Advanced Trading, and Insurance & Technology. View Full Bio
More Commentary
A Wild Ride Comes to an End
Covering the financial services technology space for the past 15 years has been a thrilling ride with many ups as downs.
The End of an Era: Farewell to an Icon
After more than two decades of writing for Wall Street & Technology, I am leaving the media brand. It's time to reflect on our mutual history and the road ahead.
Beyond Bitcoin: Why Counterparty Has Won Support From Overstock's Chairman
The combined excitement over the currency and the Blockchain has kept the market capitalization above $4 billion for more than a year. This has attracted both imitators and innovators.
Asset Managers Set Sights on Defragmenting Back-Office Data
Defragmenting back-office data and technology will be a top focus for asset managers in 2015.
4 Mobile Security Predictions for 2015
As we look ahead, mobility is the perfect breeding ground for attacks in 2015.
Register for Wall Street & Technology Newsletters
5 Things to Look For Before Accepting Terms & Conditions
5 Things to Look For Before Accepting Terms & Conditions
Is your corporate data at risk? Before uploading sensitive information to cloud services be sure to review these terms.