Discussing “Big Data” with Thomas Friedman

The New York Times “Global Forum” held in San Francisco last week by New York Times columnist Thomas Friedman gave me a chance to briefly put aside the technical discussion about “Big Data” that we in the technology community so often obsess about, and to instead contemplate the far-reaching and high-level changes big data is bringing about.

In a nutshell, what I discussed was that “Big Data” is something of a misnomer; that what is important about it isn’t the volume of data that businesses are becoming able to access (as that’s been growing at incredible rates since the dawn of the Internet), but instead, the speed at which businesses should be able to access it. They want to see important data in near real-time, quickly adjust how they respond and speed the time to action. Big data, treated as a ‘continuous feedback loop’ created as a result of this dynamic, will radically change industries. To put it simply, data as a continuous feedback loop means accessing it fast, all the time from internal and external sources, making insights intuitive and then taking the numbers and feeding it right back into the process for continuous business improvement.

Most Friedman readers know that he is interested in the big picture and macro trends of how technology and demographic trends are changing our world. Last week’s Global Forum, called “The Next New World,” and the first of its kind to be held in San Francisco had a series of panel discussions, with Silicon Valley entrepreneurs heavily represented. Also speaking that day, for example, was Reid Hoffman, the founder of LinkedIn, along with media figure Barry Diller and Beth Comstock, Chief Marketing Officer at General Electric.

Friedman is clearly interested in the big data revolution, which he sees as part of a larger global trend to what he calls the “hyper-connected” world. Appearing with me on the big data panel was Slava Rubin, CEO of Indiegogo, a San Francisco startup enabling crowdsourced funding, a new way to raise capital for anyone with a great idea or dream.

Friedman asked the sorts of questions about big data that any diverse audience would want to hear. My favorite was “How is Big Data going to impact me five years from now?” This gave me the chance to talk about what its real impact on business will be and the way big data will allow companies to quickly see information, in ways never before imaginable. A whole generation of businesses will change as a result.

One interesting example involves what’s now being done by soft drink companies. Several have installed special soda-mixing machines in restaurants and fast-food outlets that allow customers to blend their own drinks, mixing different combinations of flavors according to their taste. The data from these devices is incredibly rich and valuable. And the result is a new kind of ‘crowdsourced product development’ process to identify the next new flavor from country to country that opens up dramatic new business opportunities. Crowdsourced product development is an opportunity that is possible only because of the new volumes of data available combined with the new speed of processing it all. And leading edge, data-driven companies are doing it today.

In the past, developing a new flavor involved millions of dollars and years of effort, with the project starting in a special lab and working its way through endless focus groups before the new drink hit the shelves. But now, beverage companies can watch and see what their customers like, and how they live. That gives them remarkably granular insights into the next new opportunity. They can see how tastes differ around the world, with, for example, people in San Diego preferring a sweeter drink than maybe people in Singapore.

But it’s not just happening with everyday consumer products. Car companies will be able to watch the driving patterns of new owners, and then tailor their leasing or insurance options in response. Media companies will completely change decade-old TV advertising models as the adoption of the ‘second-screen’ (mobile devices and tablets) opens a far more interesting ad targeting platform over TV. Financial services companies will use new sources of data to assess small business risk to determine a new model for extending loans to small businesses looking to expand.

Perhaps the biggest change, though, will come courtesy of our smart phones – which essentially are personal tracking devices. Though privacy issues will obviously be addressed, companies will be able to know where their customers are, their location at all times, and will be able to send the sort of real-time messages and ads that can influence them at precisely the moment they are about to make a purchase decision based on their location. McKinsey & Company has estimated that this new kind of “personal location-based marketing” could be worth hundreds of billions of dollars.

All these changes are sweeping, so much so that those of us involved in the daily work bringing them about sometimes lose sight of them. Which is why Tom Friedman’s panel on big data, and the perspective brought by him, was so interesting and helpful. Members of the audience seemed deeply engaged with what they heard. They certainly appreciate a world-changing technology development but more so they appreciate understanding the application of ‘Big Data’ and the impact it will have as it revolutionizes industries and impacts everyday life.