Establishing Total Cost of Ownership for a Big Data Platform
In addition to confirming a potential big data platform has the performance and features needed, IT teams should also consider other factors if they are to gain a true understanding of the capital and operational expenses required to keep a big data platform operational and scalable.
The list below captures twelve criteria to consider. There may be additional factors to consider based on the specific needs of the enterprise looking to adopt a big data platform.
Fees for seat licenses – Proprietary big data platforms require enterprises to purchase licenses, often sold on a per seat basis. As the platform grows and matures, additional IT staff and seat licenses may be necessary, so license fees could increase over the long term.
Technical support – If the platform isn’t performing as required, enterprises need access to technical support resources that can quickly and effectively solve the problem. But identifying which vendor is responsible for providing a technical fix in mixed-vendor datacenter environments is difficult, and it’s common for customers to be stuck without a fix for their problem because their vendors can’t agree which component of the data lake is at fault. Technical support is particularly important for security. If a platform is orphaned by its vendor, security patch development ceases, leaving the platform’s users and data at risk.
4. Hardware – The amount of processing and storage hardware a big data platform needs will vary over time. Many businesses have seasonal data workloads that rise and fall throughout the year. This can lead to overprovisioning: purchasing extra hardware to manage spikes in datacenter capacity only to watch that same hardware go unused during periods of low activity. Furthermore, more power efficient processor architectures are now available to help enterprises keep datacenter electricity consumption low but using them is only possible if their datacenter software supports these new architectures.
5. Cloud support – Cloud-based data centers allow enterprises to scale their compute and storage capacity up or down in real time to keep overprovisioning under control. That said, cloud computing has security risks that may require specific security capabilities to meet regulations and SLAs around data privacy, sovereignty, and security, and IT teams will need to make sure any cloud-based big data solutions comply with those requirements.
6. Staffing – IT managers need to determine if their choice of big data platform will require adding additional staff to address any skill gaps or increase code output. What skills will those new hires need? As big data grows in popularity, potential hires with expertise in big data and cloud computing will be in high demand and their salaries will reflect this. IT managers must keep staff payrolls in mind when considering a big data platform’s TCO.
7. Implementation time – After a platform is selected, how long will it take to get the platform up and running? Sourcing software and hardware from different sources can cause compatibility issues that must be addressed before the platform goes live, potentially delaying the platform’s launch date.
8. Ongoing maintenance – Once a big data platform is operational, how much ongoing opex will it cost to keep it running? How much electricity does it consume? Will the datacenter require more square footage to accommodate additional hardware? If the platform’s processing and storage capacity need to expand in the future, how long will that expansion take and how cost-effective will it be?
9. Flexibility – If an IT team requires its big data platform to support specific features, what resources are available to provide that feature if the platform’s vendor is unable or unwilling to build it?
10. Developer ecosystem – Is there a robust, global network of developers working on value-added projects for the platform? Does an enterprise need their big data platform to support a specific vertical industry? Or a particular application? The larger a big data platform’s developer community, the more likely software for specific industries or use cases is already available.
11. Reliability/maturity – Is the platform’s technology new and without extensive real-world testing? Is the vendor a startup who may not be around to support their technology, or currently not able to scale to meet demand for support? Do they have good technology AND good customer service? Can they provide localized support resources for different regions?
12. Data support – Does the platform process data in any format? Does data in different formats work well together? Data in different formats often end up siloed in separate databases that don’t communicate with one another, which can lead to inaccurate or incomplete data analysis.