Ken Wood

Securing IoT Data with Pentaho's MQTT Plugin

Blog Post created by Ken Wood Employee on Jul 11, 2017

Opening Logo.png


Pentaho Labs’ MQTT Blog 2 of 3

Methods for Securing Your IoT Data

by Ken Wood



Digital security is on everyone’s minds. With the growth of the Internet of Things (IoT), the security concerns are now taking even more mindshare. Machine data and personal data privacy has been under constant attack it seems, forever! The primary problem is that because the idea and implementation of interconnecting appliances and other “things” is a relatively modern concept. Security for sensors and edge devices has not always been considered in product design like it has recently in the enterprise. As the IoT starts to converge with enterprise applications and enterprise data, and the raise of the Industrial Internet of Things (IIoT), security and privacy is again taking the spotlight. There are several popular ways to improve IoT security at the device, include placing an IoT device that needs to be directly accessible over the Internet, to be segmented into its own network and have network access restricted. Locking down and restricting any and all access to the embedded operating systems which usually is a skinnied-down version of a Linux flavor. There are other infrastructure approaches to securing an IoT device and the applications that use this data, but what about the data? With Pentaho and the new Pentaho Lab’s MQTT Plugin, we will look at a couple of methods for securing the communication links and the messages being transported as a way of protecting your IoT data.


In our first blog, our simple example was straight forward in building a simple Pentaho application with a transformation publishing data with a topic to a broker and another transformation subscribing to that topic to receive the message stream. Figure-1 recalls the MQTT architecture and the Pentaho MQTT implementation of this architecture and Figure-2 illustrates our application.



Figure 1: From the 1st blog, the basic MQTT Architecture and the Pentaho MQTT Plugin steps.




Figure 2: Previous application with a device publishing messages and a subscriber receive

the message, all through a broker service.


While we touched on the use of using the Secure Sockets Layer (SSL) transport for protecting the communications link, we opted to use the easier to implement TCP (Transmission Control Protocol) transport for illustration purposes. Here, we will now use SSL to protect our data while it is on the Internet. To show this, we use Amazon’s Web Service (AWS) IoT Platform. You can Visit this page to learn how to setup AWS IoT Platform for your IoT needs and applications. After signing up and accessing your AWS account and setting up your AWS IoT Platform, you will need to download the “device connection kit”. This package basically contains the certificate files needed to setup the SSL form in the PDI MQTT publisher and subscriber steps. Figure-3 shows the device connection kit download page from the AWS IoT Platform console and how this might look on your system. Figure-4 shows the SSL tabs in the PDI MQTT steps to setup the secure connection and the URL provided by the AWS IoT Platform interface. What you get in the “connect_device_package” is,


  • Your ”thing”.cert.pem
  • Your ”thing”.private.key
  • Your ”thing”.public.key
  • A shell script,, for your platform to test your connectivity with these certificates and keys


Lastly, to complete the device connection kit, you will see another page after clicking the “Next” button, used to download your “root CA” that you will need to save in a file.



Figure 3: AWS IoT Platform download pages for obtaining you device connection kit.



Figure 4: PDI MQTT dialect forms for setting up SSL certificates and the URL for your application.
This is all provided by AWS IoT Platform for use by your application.


Once all four of these certificates are saved, you can fill out the form for the SSL tabs in the MQTT publisher and subscriber steps. Also, the AWS IoT Platform console will provide you with some other important information to fill in the General tab form such as,


  • what URL to use – might look something like this -


  • the port number - 8883 is used for MQTT with SSL
  • A client ID to start using (all client (subscribers and publishers) IDs must be unique)
  • your topic to use



NOTE: the “Key file password” in the form is not implemented by AWS IoT Platform, but needs to be filled in, so entering anything will work.


Our network links for the MQTT Publisher and Subscriber steps should now be set up for SSL communication to the AWS IoT Platform. However, in Figure-4, you may have noticed an added step before the Publisher and Subscriber steps in our transformations. We inserted the PGP Encrypt Stream step for the publishing transformation and PGP Decrypt Stream step for the subscribing step. PGP (Pretty Good Privacy) is an encryption program that provides cryptographic privacy and authentication for data communication. Here we’re going combine our SSL protected communication links with encrypting our message stream to provide data privacy and protection.


To illustrate what this does, refer to Figure-5.  We use a tool called “MQTT Spy” to monitor, test and diagnose our MQTT based IoT applications and test the brokers. You can download and use MQTT Spy, and is located here. To connect to the AWS IoT Plaform with MQTT Spy, you will use the same certificates that you used to setup SSL in the MQTT steps in our PDI transformations.



Figure 5: MQTT Spy used to test and monitor a MQTT Broker and your applications. Here is essentially the same message
unprotected clear text and encrypted, even though the communication links are secured with SSL.


We’ll use this tool to illustrate that your messages are in the broker in clear text, or when your message stream is encrypted before leaving the device, the only message in the broker is an encrypted data blob. The message remains encrypted until decrypted at our subscriber transformation. In this transformation, we’re using a 4096-bit encryption key that we generated at the Raspberry Pi publisher device, then shared the key with the transformation running our subscriber transformation on my server. The broker does not know the message is encrypted, in fact, the broker doesn’t know anything, it just passes messages. From the time the message leaves the PGP Encrypt Stream step until the message is received and decrypted by the PGP Decrypt Stream step, the message is unreadable.


In Figure-6, the PGP Encrypt Stream step’s dialect form is opened to show how to set up encrypting the message. Since PDI uses the local host’s encryption programs, the location may vary from the device and the server that are being used. You will have to create your certificates on your device our server with this program as well and share them between the subscriber and publisher transformations. In the case of the Raspian OS, it is at /usr/bin/gpg.



Figure 6: This is the same MQTT Publisher transformation from the 1st blog, but this time we're sending  an encrypted message,
so we set up and use the PGP Encrypt Stream step be for we pass the message to the MQTT Publisher Step.


On the MQTT Subscriber side, the PGP Decrypt Step dialect is similar to set up using the same certificates as the PGP Encryption Step dialect. This is shown in Figure-4. Notice that this is now on the subscription side or the server side which means the host’s location of its encryption/decryption program is different, and the passphrase is needed.



Figure 7: This is out same MQTT Subscriber transformation from the 1st blog, but this time we're receiving an
encrypted message, so we use the PGP Decrypt Step be for we can use it.


So there you have it! We’ve secured our IoT application by using a MQTT Broker that supports the SSL capabilities of MQTT. Then we added another layer of data protection to our Pentaho MQTT IoT application by encrypting the messages as they are published to a public internet MQTT Broker that can only be decrypted once our message is received by our subscriber. This means we have a secure link between the MQTT Publisher and the MQTT Broker, and between the MQTT Broker and the MQTT Subscriber, and in the interim, while the message is in the MQTT Broker, the message is encrypted.


Of course, this is not the perfect security solution. There are several other techniques for securing your applications, devices, servers, networks and data. Building, configuring and controlling your own MQTT Broker might be another good option. However, these two methods might be good enough for development and experimentation, and getting you started building a secure Pentaho IoT application.


With the rapid expansion of IoT, more and more security issues are coming to light, especially at the edge and in the device, threatening to slowdown this exciting emerging trend. We hope this blog helps you move forward with your Pentaho based IoT application projects.