Much bigger than the human population explosion is the data explosion that is continuing to take place. There are many factors that are responsible for this sudden spurt in enterprises’ appetite for data; from the shift to a hyper-connected world to an almost freefall in the cost of storage space and solutions. Other related factors like video and audio streaming, digital content creation, and statutory requirements are also driving the demand for storage.

Historically, the move from mainframes to distributed client/server environments has also added to the complexity of storage requirements. When almost all data resided on a mainframe and was supervised at a data centre, the focus was on conserving file space and storing data efficiently.

But in distributed processing, users have more control over creating and saving information. Given this scenario, most of the companies are shifting to a centralised working model as this helps in better data management and increased security.

Besides, IT managers also need more than ad hoc approaches and need to understand the complete dataflow within the company. Here are a few simple approaches that can help you deal with the data monster in your organisation.

Design, buy as per the need: A little homework always goes a long way, so diligently work out what you need before actually jotting it down in the RFP document. It is better to invest some time at analysing the storage requirements so as to identify a suitable solution.

The important factors to bear in mind are—capacity, throughput and performance, IOPS, disk type, future requirement, scalability, application compatibility, backup and DR, management tools, and reports and alerts. Since all solutions come with their own sets of pros and cons, this due diligence will help in deciding on the right solution.

Simple optimised design: Simpler designs generally offer good performance and more flexibility. Outline the growth strategy upfront. As the data size grows, how will you manage growth of data files, logic unite numbers (LUNs) and RAID groups? It is much better to design for all this at an early stage than to do a rebalancing act later.

Consolidate storage silos: As companies grow, they inevitably create islands of storage that are both costly and inefficient. Consolidating these islands of storage or silos into a single storage pool provides tangible benefits, such as lower costs and improved operations.

There are other advantages as well—centralised troubleshooting, remote maintenance and centralised administration.

Data management: The IT manager should know how the information flows within the organisation and how it is stored.
This helps minimise or eliminate the data duplication and de-duplication. To do so, one must put in tools that provide relevant information on such issues to be able to delete or move the duplicate information to less expensive disks.

The tools should also provide additional information on any unwanted files, especially videos and music audios. Armed with information like this, managers can often defer the purchase of more storage while boosting the utilisation of existing infrastructure.

Validate configurations: Before going live, perform basic throughput testing of the IO subsystem with some benchmark tools like IOzone failover testing and policy testing.

Storage policies: It’s very important to create storage policies for efficient management. A few examples of such policies are user or group quotas, file access permissions, file type policies, backup and DR, file aging policies by which the unused files are automatically moved to virtual tape libraries.

Data backup and DR: Many companies store data in multiple locations as a safeguard, but this increases the storage requirement, complexity, maintenance and power utilisation. These hassles can be eliminated by designing reliable online storage, nearline storage and a tape backup.

Storage maintenance: To avoid or minimise storage issues, one needs to carry out pre-emptive maintenance like updating the host bus adapter certified drivers, storage array firmware, and multipath IO. Always remember that a better managed rack is not only more efficient but also easier to manage.

---

The author is Head, IT at Crest Animation Studios