Big Data is such an ubiquitous term now, I hear it everywhere: at the gym, in the supermarket and even in the pub. The merest mention of working with data and the question “Oh you mean Big Data?” seems inevitable.
Adding yet another V to the Vees of Big Data also seems an ubiquitous trend. We started with three: Volume, Velocity and Variety. This set implied that big data was data too large for mere mortal computing that changed faster than Clark Kent to Superman in a phone booth and could be more amorphous than a Space:1999 alien. Big Data would lead to Big Money. And so, the consultants geared up and Big Data became a phenomenon.
It was soon clear that three wouldn’t cut it. Gillette, “the best a man can get”, also agree with the need to increment for improved performance: They gave us the Mach 3 and Fusion. Big Data followed, perhaps unknowingly in their footsteps. And now there were four: meet Veracity. It was no longer enough to be big, fast and a mixed bag, Big Data now had to be credible too. After all, garbage in = garbage out.
Like a well fuelled hype cycle, Big Data was peaking, but was it fulfilling the promise of Big Money? It was big, fast, credible and a mixed bag. Having a Big Data project showed your organisation had its finger on the pulse of technology.
Question: What was Big Data actually doing for you?
Like all tools, and Big Data is a tool, just having it lying around, proudly on display doesn’t make it useful. So a fifth V joined the crew: Value (Your mileage may vary, viability, viscosity and virality are contenders). Big Data is now big, fast, credible, a mixed bag that gets you to where you’re going. A bit like the Gillette Fusion, more Vees = closer shave.
So, what have we learned from this correlation between razors and Big Data? Firstly, new terms are rarely defined cleanly to start with. Big Data caught on but to keep being useful, it had to evolve. Part of that evolution was users having their lightbulb moments. When you’re dealing with cutting edge, the odd knick is perhaps a small price to pay.
Secondly, for some, Big Data may have been a “one ring to rule to rule them all”. A knight in shiny SAN that would slay the data dragons. However seductive data is, there is one non-technology motto I embrace: A solution without a problem is a white elephant. To put it another way: Let the punishment fit the crime.
Perhaps even, it starts with your business problem. Firstly, define your problem. Really understand what it is you’re trying to solve, not just what you think you’re trying to solve. Make sure everyone (who needs to be involved, your stakeholders) understands the same problem – “sing from the same hymn sheet” if you must. Then find the best fit solution for the problem at hand – there may be constraints and barriers that make “the best” unusable. Plan your attack and regularly review your progress. Has the problem changed? Is it different now you’ve set off on your data expedition? No problem! Back to the drawing board, plan your next iteration. And if you’re fixing the wrong problem, short iterations give you a rapid exit point before you invest too much feeding a white elephant.
Sometimes, you may even need to think small.