Are You Storing Data Unnecessarily in the Name of Big Data?

By Mastufa Ahmed@Mastufa

Added 27th March 2014

Even with the cheaper storage and cheaper processing options now available, you still need to prioritize what data you want to capture


While the concept of big data excites many as it offers new business avenues for them, there is a danger as well associated with it if don’t prioritize what to store and what not. A report by ThoughtWorks identifies a dangerous trend of businesses storing humongous amount of personal data unnecessarily.  The report contrasts between exciting insights made possible by big data with the dangers of capturing every bit of data businesses get their hands on.

We spoke with John Haddad, Senior Director of Big Data Product Marketing, Informatica, on how to maintain the equilibrium while leveraging big data.

john-haddad-senior-director-of-big-data-product-marketing-informaticaJohn Haddad, Senior Director of Big Data Product Marketing, Informatica

Should businesses capture every bit of data they can get their hands on?
Take a look at your business goals and map your prioritized business goals to the data that you need, to answer certain questions. If your goal is to optimize marketing spending, for example, given a variety of different channels (digital, social, traditional), what data do you need? Which demographic sets of your customers are producing the most revenues? This will help you determine what data you need to capture. Then, once you successfully complete one or two of these objectives, you can expand to other business goals.

Do think it’s really possible to harness big data without storing in-depth information?
With respect to the question, “As you make data anonymous, does it still give you enough information to infer some insights,” the answer is “absolutely yes.” There is a variety of information that you can glean from “anonymized” data. It’s not enough to just fence off access to data; you need to build security into the data itself through data masking and data encryption. Once the data is anonymized, you can gather numerous insights, at the aggregated level – such as patterns and trends that you see with certain sets of demographics, for example. As businesses move to one-to-one marketing, Big Data can tell you a lot about the demographics and segments in which you want to classify customers. You can do all of this, but sill protect privacy.


comments powered by Disqus

Arbor Secures Three New Patents for DDoS...

Arbor now has 25 patents focused on DDoS defense.

OpenDaylight Announces Third Release of Open...

Broadens Programmability of Intelligent Networks, Boosts Stability and Performance...

AWS to Set Up Infrastructure in India

Move will enable users to run workloads in India, benefit from lower latency