A TCO Analysis of HPCC Systems

Now let’s revisit the TCO criteria list and see how HPCC Systems addresses each one.1. Fees for seat licenses – As an open source software platform (released under v2.0 of the Apache License), HPCC Systems is available to use by any organization without a licensing fee.2. Technical support – As HPCC Systems is a complete data lake platform, its software supports all stages of the data processing pipeline. Accordingly, support is available for the entire platform from HPCC Systems global network of developers and system consultants (Infosys, for example). HPCC Systems developers also maintain multiple active community forums and a JIRA database for cataloguing bugs/fixes and new features added to the platform.3. Third-party software – HPCC Systems is a complete big data solution, but if an enterprise needs additional software not currently available in the HPCC Systems stack, it can source the needed software from HPCC Systems global developer network. This open source model has been used by other technologies with great success. For example, Google developed the Kubernetes code base and made it available to a global network of developers so they could continue to expand and improve upon it, and Google would be able to leverage the developers’ innovations on its own cloud platform.4. Hardware – There is interest among enterprises in leveraging alternative processor technologies to help keep power consumption and hardware costs low. Fortunately, HPCC Systems is compatible with alternative processor architectures like ARM and RISC-V. In fact, HPCC Systems was originally designed to run on commodity hardware solutions to help keep implementations of the platform more cost effective. And as available memory is a key aspect of cluster configuration, HPCC Systems focuses on available memory to provide a more accurate estimate of necessary compute resources than other platforms, which means there is less risk of hardware overprovisioning.5. Cloud support – HPCC Systems has offered cloud support since 2020. Cloud-based instances of HPCC Systems provide hardened security, end-to-end encryption, authentication, authorization, and other important security measures. The goal is to deliver state-of-the-art security features that protect customer data in the cloud, while still providing the excellent data management and analysis performance seen in on-premises HPCC Systems deployments.6. Staffing – Because HPCC Systems uses the ECL language across the entire data workflow, it’s possible to maintain an HPCC Systems data lake using a smaller IT team. Furthermore, since ECL is more efficient for coding data-related tasks, a smaller development team can generate more data analysis algorithms and reports than a larger team using a less efficient programming language could in a similar time frame.

7. Implementation time – The completeness of the HPCC Systems data workflow means users can deploy a complete data lake platform in just a few hours. Creating a data lake with capabilities like HPCC Systems requires a combination of individual solutions from different vendors that could require weeks of compatibility testing before it is fully operational.

8. Ongoing Maintenance – Because HPCC Systems can run on commodity hardware and requires a smaller cluster footprint in comparison to competing platforms, if an HPCC Systems data lake needs additional processing or data storage capacity, that capacity can be added at lower cost. Additionally, the platform has native support features like user authentication, authorization, reliability monitoring, performance monitoring, and job management that make managing the data lake less burdensome in terms of time and number of IT staff needed.

9. Flexibility – The HPCC Systems roadmap uses a monthly feature release plan that consists of release candidates and gold releases. As the platform is open source, outside developers can contribute directly to the feature release plan. This balance creates a flexible development model to quickly bring new features to market.

10. Developer ecosystem – HPCC Systems has a global developer network actively working to customize the platform for use in a range of vertical markets, including agriculture, finance, healthcare, insurance, IoT, and others.

11. Reliability/maturity – The HPCC Systems data lake platform was created in 1999. Since then, there have been numerous successful deployments of the platform that serve as proof of its reliability and efficacy. LexisNexis Risk Solutions and their customers use HPCC Systems to execute production workloads daily using a platform that is battle tested and production ready. Several of these deployments are detailed in case studies available on the HPCC Systems website.

12. Data support – HPCC Systems is very adept at handling and storing data, no matter the format (CSV, XML, JSON, plain text, and binary files are all supported). Raw data files are maintained in standard Linux filesystems, and data storage is implemented via a modernized version of Indexed Sequential Access Method (ISAM) files to quickly create search indexes that make it easier to find individual records within that dataset.