Chris Moberly

Why I Think Big Data Is Cool

Blog Post created by Chris Moberly on Sep 24, 2013

My (Very) Fuzzy Logic

I do my best work when I am passionate about my material. I remember when I was taught that continuity testing on a fibre-channel cable could be done with a flashlight. Pretty slick. Or the first time I saw Linux boot up inside a virtual machine. How cool was that!? I am just now being introduced to the world of data analytics and spent this morning hiking through the woods, daydreaming about what it might entail.


I like to think of human beings as self-aware learning machines. We have our hardware, which is being well documented using current genome sequencing technology. Our BIOS comes tacked on to this hardware, and our operating system is developed at a young age by the first programmers we encounter - our parents. Updates are provided for the life of the hardware and come in many forms - schooling, media, nutrition, and any other experience processed through our I/O stack, traversed via the cache-like hippocampus, and committed to long-term storage.


Let's temporarily leave the theological debate of free will aside. Given a known platform and a known input, the output should be predictable 100% of the time. Right? If, at any given time, we are the sum of our base hardware, base software, and all input received, there is a logical and calculable reaction to every decision we encounter. That covers the human component, and I believe a similar philosophy can be applied to the forces of nature - given enough data, that is.


What does this mean? With enough information, we can predict the future. Now how cool is THAT!?



Further Pondering

Of course, this would assume that one maintains a database of every stimulus a person has ever received for their entire life. Pretty far fetched. Or is it? DNA analysis companies are advertising to the public and compiling massive databases. I have an app on my phone where I record ever calorie I consume as well as all of the exercise I perform. I rate the movies I watch and the books I read. My GPS tracks the coordinates of every route I travel. All of my conversation via text, email, and potentially phone are recorded and archived. Social media reports the important little details in life, like what I think of the new Star Wars rumors or which Hollywood celebrity I think I most resemble. To top it all off, we now have technology like Google Glass which has the potential to continuously soak these updates in and upload them to the cloud.


But what about the data that are missing up until now - don't we need a baseline? Perhaps our existing thought process could be reverse engineered to build a database of our current patch levels. A piece of software could crawl though all of my publicly available opinions (such as this) and cross reference with a massive database of those sharing or directly opposing these thoughts. Based on commonalities, a profile could be established to define all of my past updates.


If my personal past can be extrapolated, what about that of all the players in a major world event? Could something like a school shooting be whittled down to a hash table of physical infrastructure and I/O? If so, could we apply this logic to future events with a deduplication algorithm in place as a preventative measure?



Batter Up!

Where do we go from here? Persons greater than myself have most definitely been through this thought process far before I had even heard the term "Big Data". The Googles and Facebooks of the world are already all over this. Having said that, I'd like to throw out a few random ideas to wrap this up.


- Where is the motivation to use this data for the greater good, as opposed to just selling me a new vacuum cleaner when Facebook sees my living room is a bit dusty?

- Whether we like it or not, the data is being collected. How will we adapt to live in a world where the term "privacy" is being completely redefined?

- My first idea for a mobile app in the new world: "WhatIf?" - queries publicly available information and references your own patch level to tell you the outcome of any major decisions facing you. Example: "If I eat one more of these cookies, will I nail the interview next Friday?"

- Does this mean that Hitachi's HAF modules are basically the modern flux-capacitor? Sorry, had to throw a plug in somewhere!