Paul Lewis

Digital Transformation - Data Analytics and the Dissolution of Baggage Carts

Blog Post created by Paul Lewis Employee on Jun 24, 2016

Modern EIM.png


I play the flute. Well, if you’ve got it, flout it.


How long is this flight?  6 hours ?!?  I don’t want to complain for that long.


Show me a normal person, and I’ll show you a person you don’t know very well.


If there was a list of things that would make me feel more comfortable, lists would be at the top.


“You’ve always been an intense vomiter”, uttered the raspy voiced salamander on my shoulder.


It’s a toss-up on which opening line I should use for this blog.  I like them all for very different reasons but none seems to fit perfectly with the theme of my commentary today.   I’m just going to use the very last observation I wrote down as I headed to my car after landing at the airport yesterday. I walked by several “free carts” and wondered to myself:


“Do we still need baggage carts at airports when all luggage has wheels”?


Of course the answer is “Yes”, or I wouldn’t be running a giant slalom race around dozens of abandoned 4 wheeled steel contraptions constantly.  The reason why is seemingly simple:  Not all bags have wheels.


In this day and age?  Are there not ample products on the market that include wheels, of all sizes, colours and styles?  Of course.  But availability isn’t the problem.  It’s brittleness of change.  The inability to use new luggage designs since the current travel packaging method is tried and true, and new changes might impact your perfect packing technique throwing off decades of perfect vacationing experiences.  The potential negative impact is boundless!


Let’s consider a second example, something closer to the hearts of my intended audience: The CIO, the Chief Data Officer, the VP of Business Intelligence and the Director of Analytics.   Really, anyone that considers data to be an Asset, and Insight as requirement to create business growth.


In the world of a classic Enterprise Information Management program, you’ve spent $10 million dollars and 10 years perfecting an architecture that produces literally 1000's of reports created and distributed daily, weekly, monthly, quarterly and annually to busy operations folks that ACTUALLY RUN the business day to day.  You measure the value to the organization with both timeliness and accuracy as CRITICAL business decisions are made based on the absolute information sourced from pages of calculations and exception lists. Your time and money was spent on carefully selecting data from source systems, hand coding transformations to comply with an Enterprise Data Model, perfecting the Star Schema within the Data Warehouse, creating data marts with views related to product, geography and line of business, and carefully curating and testing hundreds or thousands of MIS documents, scheduled and delivered with precision.  Every minute that information is not available to the operations team, a well-documented financial impact rests solely on your shoulders.


It is a perfect world, and you won’t let anything or anyone interfere with the purity of the input, process or output of your Insights.    I’m certainly not going to try. 


I will suggest however, that as the CEO redefines the business strategy to include a strong push toward Digital Transformation, including the key initiatives of 1) Rethinking Operations and Process to dramatically affect Time to Market, 2) Changing the Customer Experience to distinctively grow loyalty, and Evaluating new Business Models to unlock new revenue streams and markets, the CEO will stress the capabilities of the current Enterprise Information Management architecture.   Specifically:


  • As the detailed hours, minutes, and seconds are evaluated within the full operations of the business from “design to shelf” to determine how logistics can be reinvented to change the time to market from a year to a month, deeper data collection with non-traditional analysis will be required.


  • As new customer experiences are created, across a variety of mobile and social platforms, including devices that could be used by end consumers to collect rich customer habit, the “mix” of data sources will change dramatically for you to consume.


  • Evolving the business model, by concurrently analysing the future impact of economic change, relative customer acceptance, and competitive reaction requires complex analytics models, is beyond current analytical skillsets.


To derive business value and insight within a Digital Transformation initiatives highlighted, the Enterprise Information Management architecture, will need to accommodate some change.  Specifically:

  • Data will be need to delivered faster, in greater quantity and with more variation
  • Insights will be requested in “real time” and requests for analysis will be requested more frequently
  • The credibility and integrity of external data will be in question, and requests for clarity will be tougher to solve


Even though I’m still in the middle of writing this entry, I can already hear the responses now: “Okay I hear you, and those changes have an “eerie” resemblance to Big Data problems. But I’ve told you time and time again, we don’t have a Big Data problem”.


To that I only have one word response: YET.


You will eventually have those problems, and there is value in preparing for the complex requests from all your various bosses.  You remember them:  The CMO asking for detailed customer segmentation, the COO requesting a real-time dashboard of operational processes and the CFO wanting to crunch the numbers on 15 hypothetical changes to your pricing, today.   I’m reasonably sure they will expect you to be prepared.


But let’s just say, even though the likelihood is VERY low, that you don’t have near term data insight problems that fall into these categories.  You do have one problem that is coincidentally shared with our luggage carrying wheel-less family members:  Brittleness of change: The inability to use new data in a daily report, that is ONLY available in the source databases without SUBSTANTIAL potential negative impacts. 


To make that data available, you now need to augment the various ETL jobs to SELECT and INSERT the new information, modify the EDW schema to store it, modify to Data Marts to index it, and change the various BI reports to use it.  Then of course, the most expensive part of modifying the perfect and pure architecture is to regression test all of the 1000 reports and transformations to ENSURE the integrity is not compromised.  That, my friends, is the infamous source of the $1 million dollar and 6 month problem; the cost of any change, however minor to this perfected environment. It's that cost and agility problem that will "bring the pain", to which you should look to avoid.


So how do you solve both problems, provide for new insight for Digital Transformation and alleviate the brittleness of change?  Create a PARALLEL but integrated architecture that SPECIFICALLY address those business problems:


  • Use integration tools in addition to ETL tools, for the purpose of connecting to hundreds of varied internal and external sources of data, in real time and batch
  • Implement an Enterprise Data Lake, to store and manage your enterprise unstructured, semi structured and IoT data, absent of a pre-defined Data Model
  • Use Data Refinery mechanisms to apply mathematical and statistical modelling techniques to discover and reduce data into a series of analytic databases
  • Present information to business operations using Visualization tools, with access to hundreds of potential illustrations
  • Allow for blending of information within the visualizations to combine in real time, information from data lakes, EDWs, internal source data and even external real time data
  • Implement these tools/techniques with experimentation and innovation in mind.  The goal is to drilldown on insight and provide a platform for research, not provide hardened accuracy


Now you know why the picture in the beginning is relevant….


Allowing for a parallel architecture will keep the purity and perfection of the structured MIS, but allow for the business experimentation and growth needed for the Digital Transformation initiatives.


That being said, maybe I’m just saying…buy a wheeled suitcase for your carry-on…and see if it helps.


(originally published in IT Media Group, FOR CIO's Section: Digital Transformation - Data Analytics and the Dissolution of Baggage Carts)