Paul Lewis

Release the Kraken

Blog Post created by Paul Lewis Employee on Dec 3, 2013

Exactly like having children, I enjoyed both the creation, early adoption, and ultimate exploitation of Big Data, however (and also like my children) I also fear the Kraken that controls my every life decision.


Like a peacock I walk around proudly all day purposely maximizing the data creation and collection by random and faceless public and private instructions.  Take this week for example.


  • I set my GPS, “Find my Phone”, and (to ensure location accuracy) my WIFI to ON at all times.  I need to make sure that not only all the telecommunications companies know where I am at all times, but also for their ability to trace my individual steps to the square inch
  • As I walk through office buildings, transit systems, and even public streets, I often pause at each CCTV camera posing at different angles to ensure future positive identification
  • For my purchases, I never use cash.  Not only is it untraceable, I’m not a huge fan of the germs.  No, I must use easily traceable electronic payment methods, specifically ones that track the individual items in my basket.  I really need the retail chains to have a much better appreciation for my preference of bathroom tissue
  • When I surf the web I allow the website to collect my personal search habits, since they requested it.  Who am I to deny them that privilege?
  • When I drive I use toll booths.  When I travel I fly on the same carriers, because loyalty matters.
  • Completing the picture is important.  I use all my social technology to not only unify my contacts, email, and instant messaging, but also as a forum to add context to all the information being collected by so many different companies, because trust matters.  Posts like: “By the way, I was at the Stationary Store buying a card for my brother Stephen Lewis’s birthday” and “If I were to list my favorite brands in descending order they would be…” and the most importantly “I like vanilla ice cream…VANILLA…understand?  Not chocolate”


The value derived from all my personal and private data being collected is significant. Security is enhanced by identifying criminals before mischief ensues (think stadiums with facial recognition technology).  Product pricing becomes fluid as the supply and demand curve can be applied in real time with streams of localized data (think hydro pricing with smart meters). Combining the farmer’s almanac with my car’s mechanical degradation history, I can predictably pay more than intended on maintenance for the lemon I bought three years ago….no more surprises. Retail stores can even tell me what sales for what products I usually buy are available at the store I happen to be walking by at that very moment.  Powerful stuff.


I only see good things happening….a happy happy life…until I fall asleep in the airport lounge and the dream sequence begins… 


<<insert fog here>>


Could there possibly be some “potential” pitfalls with the collection of my personal and private information?  Fortunately I dream in list form, so here are some “watch list” items and “redeeming feature” items that might provide some insight (Note to reader: the word “insight” in this sentence was used purposely to draw your attention and correlate to the concept of Big Data) :


Watch list: My data is being collected all the time, by many and different organizations, sometimes with my explicit permission, but mostly with my implicit permission (ie If I choose to walk on this street, and that street happens to have cameras, I have given permission to record myself walking aimlessly for hours).  Along with billions of other people like me,  my data is collected mostly in secret, using methods and technologies that are mostly unknown, and will only grow over time. Who are these people, and what do they want!


Redeeming feature: The good news is that the more data collected, and the faster it is collected, the less capable any one organization will be able to act upon it. Companies will become the stewards of massive data “lakes” which will impossibly be able to manage each individual drop of rain added constantly in an endless  water cycle of un-manageability.  Individual activity recorded  will be constantly more difficult to action.


Watch list: If a goal of collecting massive amounts of personal data seeks to reveal and promote value to individuals, however the amount of data is unmanageable at the individual level, it’s much more likely to produce content related to “groupings” of individuals.  Organizations will work on collections of people, and regardless of individual preference, the characteristics of the group will overwhelm the desires of the individual.  The net effect would actually be a movement away from personalization of preference. It won’t matter that I like VANILLA. It will only matter that people like me, like CHOCOLATE.


Redeeming feature:  Anonymity and therefore personal information security actually increases in a big data setting.  Arguably, if the analytics derived from the massive amount, quality, and form of data results in ideological groupings of people, the value of any one individual person’s data significantly decreases to the point of being valueless.    


Watch list: The aggregation and correlation of personal information across previously non-correlated data sources, may significantly effect how organizations make decisions on our behalf. A simple example:  When I apply for a mortgage, my credit history is evaluated to calculate an underwriting score to determine my eligibility and propensity to pay back the principle, with interest.  The only source of the data is my credit report, that details my payment history with a variety of financial institutions over time.  Now imagine a situation where, using other data collected including where I shop, the bloggers that I follow, even the sentiment of my tweets using analytics can more precisely predict the likelihood that I would be gainfully employed over time…and that knowledge is now used in the calculation of my score during the mortgage adjudication.  Great for organizations to reduce portfolio risk, but as individuals, it completely changes the mindset to the potential negative impact of every thought and my every physical step on any street and into any store…whether it may or may not harm me in the future.


Redeeming feature: Here is where we need to rely on legislation and regulation on the collection of data.  We, as the People, need to ensure collection of data is transparent, individuality is maintained, and the data can only be used for the purpose it was collected. 


<<flog dissipates…brow sweat wiped>>


Exactly like having children, I presume Big Data will have a significant and positive impact on my life, and society in general, however it will also make way too many messes that need to be cleaned up before bedtime…and hopefully learn to turn off the damn lights when they leave the room!