In 
With big data and analytics playing an influential role helping organizations achieve a competitive advantage, IT managers are advised not to deploy big data in silos but instead to take a holistic approach toward it and define a base reference architecture even before contemplating positioning the necessary tools.
What is Reference Architecture and why should one opt for it?
Like any other base architectures associated with any technology, the ‘Reference Architecture’ in case of big data will describe the ins and the outs of the platform.
In a nutshell, reference architecture will provide:
- A common lexicon and taxonomy
- A common (architectural) vision for the big data project
- Modularization and the complementary context of the project
With user needs and technological dynamics changing constantly, it is essential to standardize the key components of a reference architecture for scalability purposes
With user needs and technological dynamics changing constantly, it is essential to standardize the key components of a reference architecture for scalability purposes.
According to Sunil S Ranka, a BI professional, “Our experience on working with various clients compelled us to have a reference architecture in place which would define the very objective of the big data project and evolve a mechanism to support a unified vision for deploying information management and analytics.”
Reference architecture is described in terms of technological components that achieve the capabilities and drive the vision of the project.
Big data technologies are mapped to the architecture in order to illustrate how the architecture can be implemented and deployed. Organizations can use this reference architecture as a preparatory point for outlining their own distinctive and custom-tailored architecture.
“With heavy investment in current BI systems, customers want to enhance the current capability of their analytics by bringing in big data solutions to their existing enterprise system’s landscape, but the million dollar question on how this is to be done,” says Ranka.
How can one fit big data in the existing landscape of the enterprise?According to a study conducted at the Embedded Systems Institute and Stevens Institute of Technology, there are processes to create and maintain a reference architecture.
This suggests that a reference architecture captures a previous experience by mining, or by generalizing existing architectures. To be of value for future architectures, a reference architecture is based on proven concepts.
The validation of the concepts in a reference architecture is often derived from their predecessors. This is especially true in cases where disruptive technologies or innovative applications are introduced and it is challenging to have sufficient proof to support reference architecture.
In these cases, Reference Implementations, prototyping and an incremental approach might be an appropriate alternative for validation and proof.
More often than not, preceding architectures are mined for these proven concepts. Architecture renovation, innovation validation and proof can be based on reference implementations and prototyping.
According to an architecture, the big data solution should comprise of traditional BI reports, charts, information access application, data warehouses for information integration, and information delivery and decision making, which drives unstructured data transformation through ETL systems.
The enterprise systems and content stores absorb content management systems, data bases, unstructured data, files systems, ERP, CRM, SCM, SOA, ESB, Web service and so on. These are integrated with the solutions to analyze them.
Using this process, big data components are designed to refer to four focus areas in terms of how the data is analyzed.
How can a reference architecture help?
According to the study, reference architecture captures knowledge from existing architecture. Then, based on an elaboration of the mission, vision and strategy, and on future customer needs the reference architecture is transformed into architecture that provides guidance to multiple organizations that evolve or create new architectures.
Reference architectures should address the technical, business architectures and the context.
One of the main challenges here is to make this inherently abstract concept of a reference architecture more concrete and understandable by providing sufficient specific information and guidelines.
The value of a reference architecture is evident in environments with high multiplicity factors that are creating social, organizational, business, application and technical complexities.
This is a young, relatively under-developed area which has more questions than answers. These questions range from the proven value to the lifecycle of reference architectures.
Add new comment