Okay, first I just have to get this out of my system. If you collect tons of crap over a long period of time what you generally end up with is many tons of crap in many nasty piles. Okay, substitute the word “junk” and you generally end up with stacks like a hoarder with the rare potential to end up clapping your hands in glee on Antiques Roadshow (but more likely it will be one of your annoying nieces or nephews who pawed through your hoarder stash.) This is my analogy to the Big Data craze:
If you just collect a ton of random data without any idea what you are collecting or why – you end up with piles of data that do little more than increase your risk of fines for privacy breaches.
There are three key problems as I see it:
- “Big Data” is a term being pushed by those who want to profit from it, IT consultants and vendors of software and hardware;
- The term is being confused with data analytics (which represents a good use of time and investment);
- There is a lot of useful data that is not “big” that is going untapped in search of magic bigness.
Christopher Mims wrote a great post a couple of weeks ago that pointed out that even Google and Facebook make little use of big data. He also pointed out some statistical facts about predictive modeling and the impact of data set size on false positives and the identification of misleading correlations. In general, relatively big data sets are useful for understanding macro trends in consumer behavior and are not particularly effective for predicting the behavior of an individual consumer during a transaction. Small, carefully curated data sets about customers are far more informative in supporting analytics on the micro (audience of one) scale.
So, before the big data magicians hit your procurement department and you’re the one expected to pull a rabbit out of your hat, here are a couple of things to think about:
- Quality of data is far more important than quantity
- The bigger the data set, the harder (and more expensive) it is to maintain it
- Do you really need another multi-year initiative right now?
Before you worry about big data, consider initiating programs to take care of your “small data” and use it effectively. Welcome new customers with personalized information on how to use their specific account features. Recognize the loyalty of existing customers and the frequency with which they use your services and tailor loyalty offers to the customer’s specific region. The wealth of information found in the transaction history of a typical customer bill or invoice can drive some pretty magical marketing without the need for any magicians.
Elizabeth Gooding is the editor of the Insight Forums blog and president of Gooding Communications Group www.GoodComm.net
