Roguen Keller

URL Connector Example

Discussion created by Roguen Keller Employee on Mar 3, 2017

I've created a sample HCI Connector which crawls a web page looking for links. 

It will use two regular expressions (configurable) to determine if the URL is something we want to capture and will follow the link down to a configurable depth.

 

The connect with then make a simple document which contains the most basic metadata to pass testing, and the URL.  This list can then be fed into a workflow stage which could do additional evaluation of the link and determine the appropriate metadata.

 

HCI-Java-Examples/HCI-Connector-Examples at master · roguen/HCI-Java-Examples · GitHub

Outcomes