By Betsy Scherzer, Project Manager, Center for Economic Transformation, and Michael Jacobs, Intern, Center for Economic Transformation

The volume, velocity, variety, and value of private sector, government, and individual data have expanded with a big bang, opening up a whole new universe of Big Data: datasets whose sizes are beyond the ability of typical databases to capture, store, manage, and analyze.

Tech research firm IDC estimates that digital data is more than doubling every two years, much of which are stored in "Big Data" bundles. And according to a recent McKinsey report, in 2009, nearly all U.S. companies with more than 1,000 employees stored, on average, at least 200 terabytes of data—twice the amount Walmart stored in 1999.

As a result of this growth, business strategists are now paying increased attention to the potential economic value of Big Data as it helps generate significant performance efficiencies, realize product/service model innovations, and identify customer service improvements. In fact, the Big Data technology and services industry is forecasted to grow from $3.2 billion in 2010 to $16.9 billion in 2015.

This spring, as part of our ongoing Media.NYC.2020 initiative, NYCEDC President Seth Pinsky and Hearst Magazine President David Carey convened about 50 media industry thought leaders at Hearst HQ to discuss the strategic implications of Big Data for New York City’s media industry. The discussion was facilitated by our knowledge partner, Oliver Wyman.

Here's what several participants had to say about strategic usages of Big Data:

“[Big Data’s] purpose ranges from ‘more effective ad businesses’ to ‘better customer experience’ and ‘understanding the shape of the audience and the movements of content.’”

“It is about closing the loop on transactions. Bad use of data is when it becomes targeted advertising. Good use is when it benefits and empowers consumers."

“Big Data is not new; Financial Services have been doing this for a decade. This [new phrase] just drives popularity for an idea…to get attention and investments.”

Regardless, all event participants agreed that over time, Big Data analytics platforms will become widely available, shifting competitive advantage to companies who understand and creatively solve business problems. New York City’s diverse talent base positions it to lead in the creative application, use, and promotion of Big Data tools, particularly given the strength of the City’s sales and marketing sectors. However, Big Data challenges remain, including:

• Existing data silos within and between companies prevent a comprehensive view of customers 
• Better enterprise agility is needed to increase data forecasting quality, shorten test and learn cycles, and reduce inefficiencies
• Privacy issues need to be considered, especially given evolving shopping and sharing habits

Participants underscored that new types of teams integrating data scientists, business analysts, and user interface specialists are required to fully capture the value of Big Data, presenting challenges to New York City’s technical labor pool:

• Engineering talent is scarce, requiring better recruiting from other cities and industries 
• Companies must cultivate data scientists who also think creatively about business applications
• Engineering, business, and design staff may find it difficult to effectively work together, given different languages and cultures

As one participant put it:

“Innovation is coming from engineers and established media groups are struggling to attract the best of them. Young engineers don’t know yet what’s impossible! They alone can stretch technologies beyond their current limits.”

NYCEDC initiatives aim to address these realities. For example, the Tech Talent Draft brings together NYC start-up executives and top computer science/engineering students, while the NYC Media Lab fosters effective collaboration between companies and researchers advancing digital media technologies. The new Applied Sciences NYC campuses on Roosevelt Island and in Brooklyn will also help supply future data scientists. Media.NYC.2020 will continue to facilitate interaction and collaboration between industry and research leaders with an eye towards emerging Big Data opportunities in the Big Apple.