When big data gets too big

The promise of big data now is generally accepted as a way to increase personalization of services to customers, make work flows more efficient, and differentiate offerings in otherwise commoditized industries, such as financial services.

The premise of big data, however—he analysis and effective use of such knowledge rather than simply its existence—seems either to have been overlooked or found to be technically too hard to achieve. Based on several recent studies and observations, the solution combines both technological and cultural solutions.

“Most organizations cannot utilize the vast majority of the big data they collect,” says VoltDB, summarizing its survey findings. The data processing and analytics vendor polled 368 database managers, analysts, administrators and other IT professionals in a variety of industries. Specifically, it found that 72% say that they cannot access and/or use most of the data coming into their organizations.

Nevertheless, says Bruce Reading, CEO, “Organizations must have the ability to not only ingest massive amounts of data, but also immediately analyze and act on that data in a meaningful way to realize the big payoff—improving their bottom lines.”

Experian Data Quality, which is part of Experian Marketing Services, looked at how companies use big data specifically to improve marketing efforts. It found that while almost half of companies personalize messages across more than one channel, organizations struggle to gain insight quickly and maintain an accurate data source.”

“Without a foundation in data quality, organizations simply will be unable to fully achieve personalization goals,” says Thomas Schutz, senior vice president. “Organizations need to continue to invest in analytics, but they also should leverage resources to consolidate information and ensure its accuracy.”

Looking more closely at the financial services industry, IDC Financial Insights issued a report outlining best practices for big data analytics. Michael Versace, research director, says: “In-market adoption of big data and analytics has reached the point where the capabilities and applications these technologies enable are becoming main stream for a growing number of financial services firms. Yet many do not yet have a set of completely mature BDA [big data analytics] competencies across the five critical dimensions that are necessary to effectively reduce execution risks and compete with strong business, technology, and operational value propositions.”

IDC describes these five dimensions as intent, people, process, technology, and data.

Interestingly, it observes that the effective use of big data analytics will provide a competitive advantage. “Business models in this industry have been data-oriented for decades. But with the adoption of BDA technologies, a new platform for competition has emerged, confronting firms with the complexities of new technologies, skills requirements, and the seemingly endless opportunities to use data in ways not previously possible,” says Versace.

Mercator Advisory Group, in its own report, observes that “the U.S. retail banking industry is in the early stages of big data usage, which is expected to grow quickly in the years ahead.” Still, it says, the conversation about big data now needs to go from “what is it?” stage to the “so what?” stage of development.

“Big data and debit are a natural combination. Extraordinarily large volumes of data are created and available related to the debit card and its core demand deposit account. Whether this data is profile, behavioral, or miscellaneous other data, it is accumulating at ever-increasing rates (velocity), and the variety of data is increasing. However the significance of big data derives from more than just these Vs. To make use of it effectively, it is most important to understand the value of the data—and the actionable outcomes accomplished by using predictive analytics,” says Ron Mazursky, director, Debit Advisory Service at Mercator.

Volt DB—which specializes in high-speed data processing and real-time, in-memory analytics—not surprisingly advocates the technological solution of using in-memory databases. Briefly these—called IMDBs—rely on using a computer’s main memory for data storage, as opposed to relational database management systems—called RDBMs—which, simplistically defined, present a view of data as a collection of rows and columns.

It backs up this claim with results from its survey: 89% of respondents say IMDB architecture delivers better performance characteristics, and 62% say in-memory will become mainstream within the next five years.

Whether in-memory truly is the answer is yet to be seen, but it’s not too great a leap to think that such a technology-centered issue as big data analytics could have a technology-related solution.

At the same time, there are indications that corporate culture regarding the use of big data, and urgency of that use, can play a factor in making big data analytics more useful. This comes from MIT Sloan Management Review and the technology company SAS, which collaborated on new research.

“We found that in companies with a strong analytics culture, decision-making norms include the use of analytics, even if the results challenge views held by senior management,” says David Kiron, executive editor for MIT Sloan Management Review. “This differentiates those companies from others, where often management experience overrides insights from data.”

Their research indicates that companies with a top-down mandate for fact-driven decision making are experiencing gains with analytics to a far greater extent than other organizations.

“For example, at a large Boston-based bank’s exchange division, every level of the team works to field, analyze, and act on the results of quarterly surveys they send to various groups. Executive support and evangelizing a lot of little wins helps to both push and pull employees towards an analytics culture,” they say.

The MIT study recommends that companies seeking a competitive edge with analytics ask, and answer, these questions about their culture:

• Is my organization open to new ideas that challenge current practice?

• Does my organization view data as a core asset?

• Is senior management driving the organization to become more data driven and analytical?

• Is my organization using analytical insights to guide strategy?

• Are we willing to let analytics help change the way we do business?

Sources used for this article include:

Companies struggle to access information quickly for marketing personalization

New IDC Maturity Model Examines Big Data and Analytics in Financial Services

More is Better, Except with Big Data

New research from MIT Sloan Management Review and SAS finds that culture is the most important factor in achieving success with analytics

VoltDB Survey Finds That Big Data Goes to Waste at Most Organizations