0.00.0100005050NY50000YN1000100clusterNdynamicY1master${framework.pentaho.server.host}${framework.pentaho.server.port}${framework.pentaho.server.webapp}${framework.pentaho.server.username}${framework.pentaho.server.password}NNacave2019/09/18 09:15:30.253admin2022/06/03 10:38:41.608N#Story
Moves all the zipped files from the passed in folder to
the AWS S3 bucket s3://
s3 long-term storage is about 1% of the cost of the
EFS (Elastic File System) used for /mnt/staging
After 8 days it is moved by AWS to S3 Glacier storage
After 2556 days (7 years) it is deleted by AWS.
Logs into localhost using a key file so it
can run AWSCLI commands00358197Ubuntu11NN000255205112100100100Y#Date: 2019/09/18
#Author: Andrew Cave
384013172Segoe UI9NN000255205112100100100Y#HowToTest
P_AUDIT_ID 100 Process ID of parent process
P_ARCHIVE_FOLDER_NAME this will be the sub-bucket name . eg "test_folder"
P_ARCHIVE_FOLDER_PATH this is the folder to upload . eg "/Users/<yourlocalusername>/Downloads/test_folder
aws.s3bucket.name
44896605104Segoe UI9NN000128255128100100100Y#ParametersIn
P_ARCHIVE_FOLDER_NAME this will be the sub-bucket name
P_ARCHIVE_FOLDER_PATH this is the folder to upload
P_AUDIT_ID
aws.s3bucket.name
-- this is from kettle.properties
5120394132Segoe UI9NN000255205112100100100Y#ParametersOut
none321929742Segoe UI9NN000255205112100100100Y#VariablesIn1921927426Segoe UI9NN000255205112100100100Y#VariablesOut3201928526Segoe UI9NN000255205112100100100YUse keyfile for passwordless login3534318926Segoe UI9NN000255205112100100100Yfilter std_errWrite std_err from archive files to S3YWrite std_err from archive files to S3AbortYSSH Archive to S3filter std_errYfilter std_errarchive files to S3 log 2YAbortAbortY1none0YABORT240480YSSH Archive to S3SSHY1noneNaws s3 mv ${P_ARCHIVE_FOLDER_PATH} s3://${aws.s3bucket.name}/${P_ARCHIVE_FOLDER_NAME}/ --recursive;22localhostbizcubedEncrypted Y/home/bizcubed/.ssh/id_rsaEncrypted 2be98d6a71fd58285b80abe78cc93bcdfstdOutstdErr0Encrypted 96288YWrite std_err from archive files to S3WriteToLogY1nonelog_level_basicYN0Command that caused the error
----------------------------
aws s3 mv ${P_ARCHIVE_FOLDER_PATH} s3://${aws.s3bucket.name}/${P_ARCHIVE_FOLDER_NAME}/ --recursive;stdErr240384Yarchive files to S3 log 2WriteToLogY1nonelog_level_basicYN0Moving files to S3
----------------------------
moving ${P_ARCHIVE_FOLDER_PATH} to s3://${aws.s3bucket.name}/${P_ARCHIVE_FOLDER_NAME}/368288Yfilter std_errFilterRowsY1nonearchive files to S3 log 2Write std_err from archive files to S3NstdErr<>constantBooleanY-1-1N240288YNexplorerState