Nitesh Taneja

HCI v1.1 Email Processing Performance

Blog Post created by Nitesh Taneja Employee on Apr 5, 2017

The HCI characterization team has characterized email processing performance of HCI v1.1 and come up with a performance white paper documenting the results. The primary objectives of this white paper are to:

  • Summarize HCI performance when processing the Enron data set in terms of objects/sec for various HCI configurations.
  • Demonstrate that the performance scales as CPU cores, memory, and number of HCI instances are scaled.
  • Document the methodology used to modify the workflow settings for performance optimizations.
  • Provide Index and Workflow-Agent service settings and recommendations for an eight-instance configuration.
  • Recommend when to scale-up versus scale-out HCI configuration based on performance needs.
  • Compare performance of the regular HCP and HCP MQE data connections.


Some of the key findings from the performance results:

  • Performance generally scales linearly as instances are scaled up from minimum hardware to recommended hardware or HCI is scaled out to add more instances.
  • Scaling out the instances results in better performance as compared to scaling up the hardware configuration.
  • HCP MQE data connection performance is 60% better as compared to the regular HCP data connection.
  • Removing unwanted pipeline stages and optimizing the required ones can result in huge performance gains.
  • Modifying the workflow performance task settings can influence the memory requirements for the workflow and should be adjusted accordingly.
  • Ideally, for optimal index performance, an index should have one shard for each HCI instance running the Index service.
  • Workflow-Agent service should be scaled up to run on all available instances for optimal workflow performance.


For more details, please look at the attached white paper.


If you have any questions regarding the content in the white paper, please free to ask in the comments section below.