Today’s May 28, 2019, Wall Street Journal reports that data challenges are halting AI projects. They quoted IBM executive Arvind Krishna as saying, “ Data-related challenges are a top reason IBM clients have halted or canceled artificial-intelligence projects”.” About 80% of the work with an AI project is collecting and preparing data. Some companies aren’t prepared for the cost and work associated with that going in”, he added.
This is not a criticism of IBM’s AI tools. Our AI tools would have the same problems if the data was not collected and curated properly. This is supported by a report this month by Forrester Research Inc. which found that data quality is among the biggest AI project challenges. This report said that companies pursuing such projects generally lack an expert understanding of what data is needed for machine-learning models and struggle with preparing data in a way that’s beneficial to those systems.
At Hitachi Vantara, we appreciate the importance of preparing data for analytics, and we include that in our DataOps initiatives. DataOps is a framework of tools and collaborative techniques that enable data engineering organizations to deliver rapid, comprehensive and curated data to their users. It is the intersection of data engineering, data integration, data governance and data security that attempts to unify all the roles and responsibilities in the data engineering domain by applying collaborative techniques to a team that includes the data scientists and the business analysts. We have a number of tools like Pentaho Data Integration, PDI, and Hitachi Content Platform, HCP, but we also include other best of breed tools to fit different analytic and reporting requirements.
Its time to press your DataOps Advantage