In the early days of my career as a Teradata programmer I didn’t care much about metadata. I was handling customer transactions and aggregating them by day-of-week and product type. While I knew there was data about the data there somewhere in the system, basic transactional data it is not nearly that complex.
What about now? How often are we looking at customer 1234 buying product ABC on date April 11th? As we incorporate unstructured data, and expand analysis to the internet of things (IoT), the metadata can mean as much or more to the business as the data itself.
Let’s look at IoT, which is really just the source of a more complex form of analytics. An IoT-enabled fridge will have numerous sensors sending data back to a central repository. Likely it will have multiple temperature sensors, but keep in mind that where those sensors are located will have a material impact on the accuracy of the information they generate. If you don’t know if a sensor is in the cold box or attached to the compressor, how will you know if the data, when analyzed, is predicting the fridge is about to fail, or working perfectly fine?
Extending the analogy, let’s look at a refrigerated warehouse where millions of dollars of food is stored, or the large unit where vaccines are kept. If the lab is not collecting the data from the refrigerator, how do they know when it needs maintenance? Can they really tell if they are they spending too much time raising or lowering temperatures by accessing the fridge too frequently, or optimizing cooling by placing product at the correct shelf or position?
In these examples, it’s the metadata that are critical to making the right decisions.
This is part 2 in a 4 part series, stay tuned for the next blog in the series