ILM has its roots in hierarchical storage management (HSM), which was popularized with mainframe storage management strategies in the early 1980s. Driven by the high cost of mainframe disk subsystems and exploding compound annual storage growth of 50% or more, mainframe managers sought to use HSM to reduce storage costs by migrating data, as it no longer was being accessed regularly (aged) from expensive online hard drives, to near-line optical jukebox storage (at the time about 1/4 the cost of disk storage) and eventually to offline tape library storage (about 1/10 the cost of disks). By establishing migration rules correctly, the organization would see little to no delay in information access (keeping frequently accessed data or data requiring instant access regardless of age near-line), but save significantly by not having to waste precious disk subsystem space and avoid having to purchase disk subsystems to support growth.
The savings could be significant not only by savings in storage costs. As research by Strategic Research and other luminaries showed, over $7 was being spent for storage management, administration and utilities costs for every $1 of disk capital cost. Because over 90% of data on the data stored on hard disks was not actively accessed by users or applications, the data on hard disks was ripe for more intelligent management and migration to less expensive storage.
Although HSM remained popular as a mainframe process, early attempts at HSM for distributed computing in the late 1980s/early 1990s never gained mainstream adoption. HSM solution and heterogeneous network complexity, distributed storage arrays and falling hard disk prices were to blame. With storage consolidation to storage area networks and advances in storage management solutions, the concept of storage hierarchies and intelligent management is again at the forefront of storage architecture planning.
Fast forward to 2003 and the concept of managing storage for lower costs is all the rage with storage vendors seeking to re-energize the storage marketplace. CIOs desire to get a handle on storage growth (still over 50% annually), escalating capital and operating costs (the 7:1 administration to storage cost ratio remains), increased security and disaster risks, and growing compliance regulations and mandates. According to Gartner research in late 2004, less than 6% of organizations have purchased and implemented ILM solutions to date, but over 50% of organizations are currently planning on purchasing ILM solutions in 2005 and 2006.
ILM as a concept is not new, introduced by records and information librarians in the early 1990s as a way to track the lifecycle of data from creation or acquisition, editing and refinement, publication and evolution, through retention and data disposal. Similar to HSM, ILM seeks to minimize storage costs by providing various storage mediums such as disk, optical and tape, and various rules to manage migration, archiving and disaster recovery. As with mainframe HSM, the economics are similar in that most data older than 90 days is seldom if ever accessed and online storage remains expensive to procure and manage, making migration to increasingly cost-efficient technologies attractive. All data is not created equal, and taking advantage of this fact helps save costs.
ILM outdoes HSM by taking a more holistic view by incorporating the business value of data into the equation. Implementing ILM identifies not only the right storage medium to minimize storage costs while meeting access speed, but the right process and path to manage the data by meeting business requirements such as information security, disaster recovery, data retention rules, compliance regulations and retirement. The right storage at the right time and at the right price -- ILM solutions and strategies help IT manage larger amounts of information, while at the same time lowering costs and improving the efficiency of storage operations.
Some of the business value of ILM include:
For all storage:
- Reduce the need to add hard disk storage for growth.
- Reduce storage administration tasks such as moves, adds and changes to support growth,
performance optimization, and backup and restore tasks.
- Reduce backup window overruns and resultant performance degradations.
- Reduce disaster recovery time by minimizing the necessary online storage restoration.
- Reduce compliance costs and reduce risks of compliance issues.
- Improve application and database performance by archiving older information and minimizing the
size of production databases.
- Reduce database upgrade or maintenance downtime.
- Reducing risk of batch job window overruns.
- Reduce development / test workloads and storage requirements.
ILM ROI success is at risk should the solution and processes overwhelm the capability and maturity of the IT team, leading to incorrect migration rules, delay in information access for users, lost data, compliance issues or application performance or availability issues. Prior to implementing ILM technology, the team should thoroughly understand and document the processes by which data is to be managed: analyzing and categorizing all data by age and usage characteristics, value, retention and compliance rules and other important management criteria. Profiling all management rules and lifecycle paths comes first and drives the selection of systems and solutions to meet the needs, rather than selecting an ILM solution first.
It is through a combination of proper analysis and planning, thorough documentation and valuation, and then technology selection, integration and implementation by which ILM can minimize the total costs and maximize the business value of storage.
This was first published in December 2005