When time was of the essence, a federal agency turned to Censeo for help in identifying and preparing for an upcoming increase in big data needs. By undertaking an enterprise-wide diagnostic, Censeo was able to identify more than $10 million in potential savings.
estimated annual growth in data volumes
For the first time ever, we know the size of the problem we are facing and how fast the train is coming. Thank you for opening our eyes!
A federal regulatory agency was anticipating an enormous growth in the volume of data to be ingested over the next three to five years, driven by growth of the regulated industry, changes in technology, and expansion of the agency’s regulatory mandate. However, there was virtually no enterprise-wide visibility into current data ingestion and historical growth upon which to base projections and preparations. The agency was flying blind, and the IT organization lacked the information necessary to effectively support the needs of its business users, who were already beginning to experience the pain of rapidly increasing data volumes (e.g., slow processing, downtime). Ultimately, this was affecting the agency’s ability to effectively deliver on their mission.
Censeo conducted a 10-week diagnostic to size the agency’s current data footprint, estimate anticipated data growth, pinpoint capacity constraints, and identify management implications and strategic options. To assemble this baseline, the Censeo team developed collaborative relationships with over 30 business and IT stakeholders across the agency, and overcame an overwhelming culture of organizational silos that had hindered past attempts at enterprise-wide coordination and visibility.
Through these collaborative relationships, Censeo identified and sized more than 200 data sources being ingested or generated by the agency, amounting to over 3PB of data. Censeo then developed analytically rigorous projections of data volume growth, which revealed that the agency should expect growth of 30-40 percent annually over the next three to five years. In addition, Censeo found that the agency had been significantly underestimating the growth of their data consumption needs.
The Censeo team also worked with the IT organization to understand the agency’s capacity to handle the anticipated growth in data. The team identified multiple hard constraints (e.g., storage capacity, end-of-support for mission-critical enterprise storage environments) that would be triggered within two years if management did not act—and act quickly.
Thorough analysis demonstrated a need to act quickly to prepare for the imminent influx of data. Censeo identified a key deadline for action based on data growth projections – the agency’s storage capacity would be breached within two years – and estimated that immediate action (e.g., replacing aging infrastructure) would save the agency more than $10 million in avoided costs if the agency acted quickly.
The Censeo team also identified a number of critical decisions that management needed to make in order to help the agency navigate the incoming big data wave, including shifting the capacity demand-supply balance, adjusting governance practices, and making IT investments.