Hitachi Content Platform​

 How do we setup SSL authentication in HCI Kafka queue connector

  • Object Storage
  • Hitachi Content Intelligence HCI
Miguel Gaspar's profile image
Miguel Gaspar posted 06-08-2020 17:47

How do we setup SSL authentication in HCI Kafka queue connector? I guess that enabling SSL option is not enough as we must also set the keystore and keytruststore. 

Where would we set the options? 

Thanks in advanced

 


#HitachiContentIntelligenceHCI
Jonathan Chinitz's profile image
Jonathan Chinitz

@Miguel Gaspar​  Sounds like you are using HCI's Kafka as broker itself (read: the server). That is not a supported action. The connector config allows HCI to act as a consumer or a producer of events and it expects to communicate either over PLAINTEXT or SSL with an *external* Kafka broker.

Miguel Gaspar's profile image
Miguel Gaspar

Hi @Jonathan Chinitz​, in reality it's an *external* Kafka broker. The thing is that I have enabled authentication trough SSL on the *external* Kafka broker and not sure where I can set the certificates on the client side that's HCI in this case.

 

Someone told me I could see the options when clicking "test", after changing communication to be performed over SSL, I would get a popup to set it up however not being able. And if I would be able to do it, is that a configuration it will be persisted in HCI connection configuration to be able to use it in a workflow?

Thanks in advanced.

Jonathan Chinitz's profile image
Jonathan Chinitz

Good to hear that it is an external Kafka broker :-). I am not sure how the Kafka connector code works, but for HCP as an example, the first time you TEST the connector and it is configured to use SSL the code will pop up a window with the certificate information and ask if you (the client) wants to accept the certificate and store it inside HCI's certificate cache (assuming the certificate is NOT trusted). This is the same function as a browser accepting a connection from a server with a self-signed (or untrusted) cert. If you tried it and TEST doesn't pop up the window then that extra code is missing unfortunately from the Kafka connector.

 

Another option would be to add the Kafka broker certificate to HCI directly. You can do that through the Admin App >> Configuration >> Certificates >> Client >> Upload. Your certificate will need to be a PEM formatted file (e.g. using openssl x509).

Jonathan Chinitz's profile image
Jonathan Chinitz

@Miguel Gaspar​ I successfully created a Kafka connector that uses SSL -- there was no problem. I created a Kafka cluster in AWS (MSK) that had brokers that were listening on PLAINTEXT (port 9092) as well as SSL (9094) connections. I sent a few messages in and then read them thru a simple pipeline. Works like a charm. 😁