Volume alone is not Big Data: Gartner

According to Gartner's latest Magic Quadrant report for the data warehouse DBMS market, introduction of big data is causing transformational changes for both vendors and enterprise users.

The supplier side of the data warehouse database management system (DBMS) market consists of those vendors supplying DBMS products for the database infrastructure of a data warehouse and the required operational management controls, says Gartner.

For the purposes of its Magic Quadrant analysis, Gartner defines a DBMS as a complete software system that supports and manages a logical database or databases in storage. Data warehouse DBMSs are systems that, in addition to supporting the relational data model (extended to support new structures and data types such as materialized views, XML and metadata-enabled access to content), support data availability to independent front-end application software and include mechanisms to isolate workload requirements and control various parameters of end-user access within a single instance of the data.

This market is specific to DBMSs used as a platform for a data warehouse. It is important to note that a DBMS cannot be used as a data warehouse rather, a data warehouse (solution/data architecture) is deployed on a DBMS platform, says Gartner. A data warehouse solution architecture can and often does, use many different data constructs and repositories. Importantly, the definition of this market is changing and a DBMS will become only part of the overall market definition as the logical data warehouse (LDW) continues to grow in acceptance and deployment.

A data warehouse is a database in which two or more disparate data sources can be brought together in an integrated, time-variant information management strategy. Its logical design includes the flexibility to introduce additional disparate data without significant modification of any existing entity design. A data warehouse DBMS is now expected to coordinate virtualization strategies, as well as distributed and/or processing approaches such as MapReduce, to handle one aspect of big or extreme data situations.

A data warehouse can be of any size. The sizing definitions of traditional warehouses remain as:

- Small data warehouses are less than 5 TB.
- Midsize data warehouses are 5 TB to 20 TB.
- Large data warehouse are greater than 20 TB

Importantly, none of these categories qualify a warehouse as a "big data" warehouse. Volume alone is not "Big data." For the purpose of measuring the size of a data warehouse database, Gartner defines data as source-system-extracted data (SSED), excluding all data warehouse design-specific structures (such as indexes, cubes, stars and summary tables). SSED is the actual row/byte count of data extracted from all sources.

From 2012 onwards, defining the size of a warehouse will become less important and information asset access will become more important. Within SSED it is important to separate the actual data size in a data warehouse from the database total size. Gartner clients report that many 100-terabyte warehouses often hold less than 30 terabytes of actual data. Throughout 2012 and 2013, the size of a warehouse will evolve toward a combined metric, relative to the repositories under direct management of the warehouse and complemented by the volume of available information accessed by the warehouse, as well as its performance in doing so.

In addition, for the purposes of this analysis, Gartner treats all of a vendor's products as a set. If a vendor markets more than one DBMS that can be used as a data warehouse DBMS, it notes this fact in the section related to the specific vendor, but evaluate its products together as a single entity. Further, a DBMS product must be part of a vendor's product set for the majority of the calendar year in question. If a product or vendor is acquired mid-year, it will be labeled appropriately but placed separately on the Magic Quadrant until the following year, clarifies Gartner.

There are many different delivery models, such as stand-alone DBMS software, certified configurations, data warehouse appliances and cloud (public and private) offerings. These are also evaluated together within the analysis of each vendor.

Figure 1. Magic Quadrant for Data Warehouse Database Management Systems

Source: Gartner (February 2012)

To read Gartner's comments on and analysis of various vendors and their offerings in the Magic Quadrant, >click here.

odziez