Big Data: It’s revered, feared — and a moving target.
Friday’s news alone revealed new uses, ranging from reading emotion to improving our health to picking the Super Bowl champs. It’s the reason Starbucks knows the hottest neighborhoods before we do and it’s giving the government concerns over how to produce “official” statistics.
It’s everywhere, and it can be a valuable, if not critical tool. But before we declare Big Data as the end-all-be-all, maybe we ought to look at what Big Data cannot do.
Which brings us to this New York Times post on the eight (no, nine!) problems with big data.
A few takeaways from this post:
- Big Data helps us find correlations, but it won’t tell you which correlations are meaningful.
- Big Data can be gamed by spammers and the lot.
- The data isn’t always as robust as it first seems. When data is collected in different ways for different reasons, it can be risky to draw conclusions.
Another thing we should know about Big Data? Some of the best in the biz still rely on good old human judgment. We all know Netflix uses our viewing habits to make big decisions. But recently its own chief content officer admitted that it’s important to know what data to ignore, indicating their decisions are 70% data, 30% judgment (with the 30% being placed “on top, if that makes sense”).
Big Data is of course here to stay. But if we’re using it, we need to be careful. As Gary Marcus and Ernest Davis said, “it’s not a silver bullet.”