Given all the talk and hype around big data are there any standard set of security controls being used?
My general observation has been that big data tools tend to lack even basic security functionality. This is a bit ironic because security (audit logging, IPS/IDS, etc.) is considered a big data problem in and of itself.
The Cloud Security Alliance (CSA) has formed a Big Data Working Group (see Big Data : Cloud Security Alliance), which has conducted some of the earliest work in this space so far. As an initial effort, the CSA Big Data Working Group interviewed Cloud Security Alliance members and surveyed security practitioner-oriented trade journals to determine the initial list of high-priority security and privacy problems, studied published research, and arrived at the following top ten challenges:
With the fate of Edward Snowden and the PRISM program getting such high profile coverage, people are beginning to waking up to the security issues and implications for big data.
Eric Hibbard points out some solid requirements, and like Eric I fear that now this is exactly what many of these are when considering Big Data technologies. However as we've experienced with some of our emerging PoC activities, in Big Data, when adding in humans and processes some (not many and certainly not all) of the above are accounted for. For example, during sensitive PoC activities we are being required to run our testing and early development within the "four walls" of the customer site to account for privacy as well as contractual restrictions for data sets.
Moreover, I agree with Eric about the irony that we're solving Big Data Security problems with insecure technologies. In this case purism/religion isn't helping the cause. A classic example of this is HDFS vs. HDFS shims to feature rich systems with ACLs, POSIX permissions, AD integration, etc. The notion that Hadoop style systems must run on 2U nodes with 12-24 drives with standard HDFS is ignoring the security (and usability) tradeoff. I believe we need some security professionals to enter the scene and start working with business units and IT departments to help everyone understand the tradeoff being made when purism enters the scene.
Retrieving data ...