How to set the Big Data Strategy

Senior IT managers have started harnessing the big data trend and are working out ways to make a big deal out of it

Senior IT managers firmly believe that big data solutions are in pilot mode, and most companies have developed prototype and used cases to extend big data visualisation. Senior IT managers definitely are making a big deal of big data, making all efforts to capitalise on the big data wave. Besides possessing a deeper understanding of big data and its benefits, they are leveraging analytics and a well-defined architecture to make complex things simpler. Here is what they think of big datas perils, promises, and ways to fetch greater value out of it.

Balasubramaniam Vedagiri, VP & Head - Enterprise Technology Solutions, Mphasis emphasises on making sense of Big Data

According to Vedagiri, Big Data driven solutions are in the pilot stage and in some cases, we have developed a prototype and used Proof Of Concepts for extending big data into data visualisation space. Web crawlers are also used as web analytics/text analytics tools to aid in unstructured data analytics in retrieving context specific information from social media. The web scraping technique is used to extract data from web logs, Clickstream and other sources in analysing customer sentiments and behaviour patterns. In short, wherever large volumes are processed in streams it requires a differentiated approach to come up with a big data driven solution.

How to Measure 4 Vs

In an era of information explosion, over a billion bytes of data is generated daily--and this number continues to increase with each passing day. While big data has existed in different contours for quite a while, the dimensions of data have undergone a drastic change in recent times, thus making the entire process daunting.

The volume of data is increasing every day and measured in Kilobyte, megabyte, gigabyte, terabyte, petabyte, Exabyte, zettabyte and yottabyte (10 to the power of 24 bytes). With the advent of smarter devices like RFID and Smart meters, there is increase in the speed, number and frequency of transactions.

Velocity is measured at xTB/second or yyy microseconds as patterns are detected from a few thousand transactions occurring per second.

Information Convergence is the need of the hour since there exists a Variety of data sources. More and more types of data are being integrated and analyzed every day--both structured and unstructured. Variety measures include text, sensor data, audio, video, click streams, log files and more.

Value is the new differentiator for big data. With new data integrated into the ecosystem for analysis, the Value of data has really grown to be a critical element as it directly impacts the insights, benefits and business processes within organisations. How big data helps in creating new businesses or drive more sales is the value quantification to measure impact.

Setting Right Strategies

The trend towards big data is growing so rapidly that we cannot afford to dismiss it as hype. However, the need is to understand why traditional business intelligence/data warehousing cannot solve a given problem, as big data is not necessarily the only answer. Develop a minimal set of big data governance directives upfront. Big data governance is a chicken-and-egg problem--you can't govern or secure what you haven't explored. However, exploring vast datasets without governance and security introduces risk, and firms must address this. When launching big data initiatives, avoid getting too complicated too fast, and be prepared to scale once a solution catches on: keeping big data as small as is reasonable, reducing scope to the simplest, most valuable objectives.

Big data, big deals

Big data is now being bundled with other technology domains like mobility, Customer Relationship Management, Cloud, Enterprise Applications, etc. Thus, any combination of one or more of these to create data sets drives the deal size to be larger than a big data requirement alone.

Recently, we received an inquiry to provide some of the outcome from big data solutions to be made available on the mobile platform. With the Data Visualization layer on top of the Analytics engine, customers show more and more interest to have m-enabled versions. In similar vein, implementation criteria for large data sets interaction using Cloud (with the help of products like Cloudera) are also gaining a lot of traction.

Sync with Vendors

The logical sync among IT managers/developers and big data vendors is definitely in a phase which cannot be termed as common ground. With big data analytic discovery vendors like Datameer and Hadapt still in the process of being invented and have competing goals among them, IT managers are not sure when that would evolve. While Hadoop developers are trying to bring the big data platform to a broader user base, IT leaders are exploring options as they are unsure about how to proceed. Skepticism abounds based on past data warehousing projects that never provided the promised business outcome. In some cases, IT managers are relatively leery of making additional investments in big data and advanced analytics due to the complexity of their current analytics programme.

Key Benefits

We can derive benefits from improved performance with predictive analytics, if the mechanism to source data from multiple locations and channels are properly planned by any organisation planning to launch big data initiatives. The infrastructure and IT architecture need to be upgraded for easy merging of data. Models built with an equal balance of complexity and ease of use would succeed. Finally, the processes and tools need to be simple and updated so as to enable proper use by the entire organisation. It is also very beneficial if the learnings and findings are amplified across the organisation for future rollouts.

Some issues that restrict the full potential of big data need to be addressed. Policies related to privacy, security, intellectual property, and even liability would need to be addressed in a big data world. Once reliability, manageability and performance improve, companies will increasingly need to integrate information from multiple data sources, often from third parties. Enabling access to data is critical for success.

Subhamoy Chakraborti, GM-IT, Magma Fincorp Limited, strongly states that Its a universal truth that data is getting bigger and more complex with every passing day. Financial Institutions (FI) have been handling huge data since long, and utilising their customer data to make decisions on customer profiling and take many more important strategic decisions, even if not termed big data.

New 4 Vs (volume, velocity, variety etc)

Chakraborti says that Volume, Variety and Velocity have been considered the three parameters of big data for some time. Apart from these, there are many other keywords which are contenders for the 4th V. However, Value largely remains the winner amongst them and it surely needs to be considered as the 4th V.

While Volume, Variety and Velocity would remain as hygiene factors, the additional V would become the key differentiating factor, as businesses would consider tthe value of the analytics.

Reduction in Complexity

Lets look at the scenario in financial institutions. Prospects traditionally are used to walking in to banks or financial institutions to seek loans. So far, it has been assumed that the borrower would explicitly state his/her need, based on which the decision to grant the loan or not is taken. Once the customer reaches out to the FI, the institution would initiate the next steps of collecting the KYC (Know Your Customer) documents. Based on the customers previous track records, institutions take a call as to whether to approve the required loan. Once the loan is disbursed, the recovery team would make timely calls or send field executives to collect the necessary EMIs for the pre-defined period till the loan is closed.

However, with the change, the process has become simpler as the number ofchannels has increased. The customer can access the institution through its online portal or through its Facebook page, raise a request through the toll-free helpline for a loan, or even send out a Direct Message at Twitter.

A customer may not be able to convey his/her needs explicitly in terms of loan amount, rate, tenure etc.. Indeed, its expected that the financial institution would help in giving options based on his/her income and lifestyle patterns. In other words, the loan offer needs to be contextual. Based on the insights into customer background, the institution also needs to predict the most feasible payment term (which, for example, may depend on the crop cycle for a farmer).

To enable the predictive behaviour about the customer, to know him/her even before he/she has approached you, it would definitely require more data, and importantly, more information out of that data.

Fortunately, most FIs have not been lagging behind in opening up multiple channels. However, the challenge would be to extract useful information out of that data.

As Social and Mobile movements pick up speed among FIs, pretty soon they would find themselves submerged under a huge data pile, some of which not so useful. To make sense of the unstructured data, new strategies like big data would be necessitated as the relational database would not be able to create decisions from such a volume of data.

Is Big Data a Form of Business Analytics?

From a business benefit perspective, there has hardly been much change from BI to Business Analytics to big data. However, under the hood, things have changed. Now, we talk about disparate sources of data including social, mobile, audio, video etc. and then (heres the similarity) make a decision out of them at lesser cost in lesser time. BI tools had made a basic assumption that all the analytical questions are known beforehand. However, it will not be so in future. Also the traditional way of separating Operational and Analytical tasks may not exist in future.

Take for example the future Mobile Office strategy: The Mobile Tablet trotting sales executives would need the analytical edge even as they talk to a prospect. It requires faster, more accurate, more predictive information and decisions about the lead, which needs dynamic information discovery over data from all sources within the FI.

Big Data, Big Deals

Big Data would get a big push along with Social network and Mobile adoption. Big Data and Cloud would provide the computing power to handle Social and Mobile data. The Social presence would soon be a differentiating factor for service providers who have an urban/semi-urban client base with expendable incomes.

Being Strategic

On the systems design side, a 360-degree view of key entities of your business, like a customer or a loan account in financial services, would be the basic considerations. This would help in defining products appropriately, and finding the risk profile of a customer from different knowledge bases. Also, keep in mind that the source of data would keep increasing very fast in future. Further, most importantly, it requires a complete solution instead of half-hearted effort to tackle big data. Big data discussions so far have been focusing only on the data at rest. The attention has been on the analysis process once we capture the data. Considering the diverse data sources including unstructured data and its volume, network and infrastructure plans also need to be designed in detail.


Add new comment