HCP Coding Best Practice

Document created by Steven Looby Employee on Mar 27, 2018
Version 1Show Document
  • View in full screen mode

This topic discusses HCP best practices for for optimizing REST I/O performance. 

 

Small Object I/O - importance of directory distribution

HCP has similar recommendations to Amazon:

https://docs.aws.amazon.com/AmazonS3/latest/dev/request-rate-perf-considerations.html  

 

Specifically, they and HCP recommend that key names contain a bit of randomness. 

In our case the translates to using different directory names.

 

Putting all objects into the top directory "/" will pin database operations to a single region on a single node.

Data itself will get distributed, but the database updates will all converge on the same node.

 

One popular solution is to hash the object name, and create a directory prefix.

Imagine a simple hash that defined two directories called  /odd and /even.

For HCP, they might consider a hash prefix , using the last two characters of MD5 of the file name.

This would create 256 directories.

 

/00

/01

...

/FF

1 person found this helpful

Attachments

    Outcomes