In For the past 40 years, the IT industry has been focussed on automating business processes by using relational databases to process transaction data. The issue now is finding a way for IT managers to derive best value from big data growth, given the humongous challenges of tracking big data.
According to John Haddard, Director of Enterprise Data Integration, Informatica, data has become fragmented and locked within operational and analytical systems, both on premise and in the cloud. Data integration technology integrates these transactional data silos. Over time the volume of this transaction data has grown to outpace the capabilities of IT to effectively manage and process what has become big transaction data.
This big transaction data is throwing up challenges at IT managers, while creating new opportunities for growth.
Big Data, Bigger Challenges
Haddard says that while access to this data is critical for the empowerment of the enterprise, IT organisations are not adequately prepared to access, process, integrate and deliver this data. Combining big interaction data with big transaction data will unleash great new opportunities for data-centric enterprise and drive competitive advantage.
Syed Masroor, Head-Technology and Solutions, Netapp India, finds the IT heads facing big data challenges at every stage of their business process. Every source that generates data is a big data, for instance an organisation outsourcing its credit card transaction to the banka huge and particular type of data is generated. A retailer too would have a huge data generated in real time, with a major impact on the business, which through a process will be accessed by the IT manager. This is transaction data, says Masroor.
The major challenge for the IT mangers is how to analyse data which is sourced from multiple channels and systems, adds Masroor.
According Dr B Muthukumaran, IT professional and consultant, the major challenge for the IT managers at this point of time related to big data is the training, which calls for huge investment. Putting up a migration plan with the existing infrastructure and data to churn out big data from the associated groups is a challenge, says Muthukumaran.
The key challenge for IT managers and CIOs that Arun Ramachandran, Country Manager, Data Computing Division, EMC India and SAARC sees is to realise the significance of big data and how the evolving economic environment facilitates IT to become a business transformation function.
Referring to cost related challenges of big data, Mitesh Agarwal, CTO and Director, Systems Solution Consulting, Oracle explains, For example in ERP, they have to store all the data for a certain number of years while in the case of big data, not all the data will need to be permanently stored. Most of the data may not have relevance but CIOs cant know in advance what to keep and what to discard, says Agarwal.
Harsha E, Head-IT and Consultant, HK Group, agrees that the main impediment to big data usability is that its very hard to get all the relevant bits of data related to a clients portfolio into one place. Even the most powerful portfolio analytics systems, including those that are based in the cloud and can draw on massive stores of performance and pricing data, dont always provide a view that draws on a comprehensive data set, points Harsha.
Grace Kumar, Associate Director-IT Infrastructure Services, Cognizant Business Consulting, finds that big data could be a disruptive force if not controlled.
IT managers are bogged down by the challenges of big data, but using right methods and tools will enable them to derive better value out of big data.
Most agree that with big data in the right hands and handled strategically; the massive amounts of information companies collect today, it can become a valuable new asset. EMCs Ramachandran admits that players seeking additional organic revenue streams should consider tapping their data trove to power a new information services growth engine.
Best Practices for Best Value
Analysts agree that cloud has set the stage to allow big data to be leveraged.
Cloud computing makes big data possible by providing an elastic pool of resources to handle the massive scale of big data, says Ramachandran. With cloud computing, IT resources are more efficient and IT teams more productive, freeing up resources to invest in big data. He further explains, Rising IT costs, exploding data volumes, and ever-evolving competitive challenges have spurred new ways of thinking about effective systems for data analytics. All these developments have led to radical changes in database technology and a new approach to exploiting data.
The best practices to measure the value of big data that Dr MVK Sarma, VP-RDW Product Development, Ramco Systems, suggests, is to draw a maturity map information flow and optimise the same. Change is constant and IT managers need to look at technology to streamline the processed data and create a single version of truth by building an analytical system to present it in a structured form to derive value, says Sarma.
Some of the best practices and solutions that Vivekanand Venugopal, VP and GM, India, Hitachi Data Systems, recommends are to go in for well-managed software and hardware to store and search both structured and unstructured content. Information cloud is an ideal tool which has advanced engines to analyse, visualise and repurpose both structured and unstructured content and also manage them well via advanced systems management capabilities, points Venugopal.
The key focus for IT managers as Netapps Masroor observes is to spot big data and analyse how beneficial it is in providing a tangible solution to the business problem.
According to Oracles Agarwal, to derive real business value from big data, IT heads will require the right tools which will help their organisation capture and organise a wide variety of data types from different sources.
Agarwal says that IT execs should: a) Align big data initiative with specific business goals b) Ensure centralised IT strategy for standards and governance c) Ensure security for big data d) Correlate big data with structured data, and e) Look at big data as an extension of existing information architecture.
IT managers need to assemble a suitable set of hardware and software components to create big data architecture and leverage commercial quality support with the entire system being supported by a single vendor, he says.
HK Groups Harsh points out that in reality a comprehensive view is very difficult to achieve. Say, for example, that an asset manager uses a portfolio management interface that lets him slice and dice data to show risk hot spots, assess historical performance and stress-test client portfolios. Without confidence in the comprehensiveness of the data, portfolio managers cant be sure they are getting a single, clear and accurate view of how their clients portfolios are performing, says Harsha.
Some of the best methods to derive value from big data, according to Informaticas John Haddard, are to use core data integration to integrate all the data silos managed within the enterprise. Cloud data integration to retain control over off-premise data managed by cloud computing vendors is one method to address big data, says Haddard.
Other critical tools would include information lifecycle management to cost-effectively and securely manage the growing proliferation and volumes of data in all these applications Finally, IT managers must master data management tools to gain a 360 degree view of authoritative master data assets.
Grace Kumar opines that the same simple formula applies, wherein big data techniques complement business intelligence (BI) tools to unlock value from enterprise information. Berjes Eric Shroff, Senior Manager-IT, Tata Services Ltd, observes that on software front, database management systems such as Oracle, Microsofts Database SQL Server, MySQL, etc, have been able to cope with such increases in data. On the hardware front, the option of either increasing a servers CPU, RAM, disk space, etc, or opting for NAS or SAN storage devices, has addressed the needs of the relative quantum increase in the data size. So far, a combination of these hardware and software (database management system) technologies, has contributed towards successful decision-making for the organisation, says Shroff.
Add new comment