Pentaho

 View Only
  • 1.  Permission issue with tmp location - Pentaho 9.3 version

    Posted 04-19-2023 16:03

    Hi,

    When running a job on Pentaho 9.3 version, we are facing a permission issue with tmp location.

    Below is the full log trace:
    15:44:18,753 INFO  [DriverManager] Installing driver kars.
    15:44:18,753 INFO  [DriverManager] Drivers installation directory not found: //drivers
    2023/04/19 15:44:19 - J_MCO_EXCEPTION_REPORT - Start of job execution
    2023/04/19 15:44:19 - J_MCO_EXCEPTION_REPORT - Starting entry [Archive Old Exception Files in SFTP]
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP - Running on platform : Linux
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP - Executing command : /tmp/kettle_933857c0-deea-11ed-bdfc-0356986798a7shell SIT
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP - ERROR (version 9.3.0.0-428, build 9.3.0.0-428 from 2022-04-12 04.56.25 by buildguy) : Error running shell [/opt/SIT/Reporting/Scripts/MCO_EXCEPTION_FILE_SFTP_ARCHIVE.sh] : java.io.IOException: Cannot run program "/tmp/kettle_933857c0-deea-11ed-bdfc-0356986798a7shell": error=13, Permission denied
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP - ERROR (version 9.3.0.0-428, build 9.3.0.0-428 from 2022-04-12 04.56.25 by buildguy) : java.io.IOException: Cannot run program "/tmp/kettle_933857c0-deea-11ed-bdfc-0356986798a7shell": error=13, Permission denied
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at org.pentaho.di.job.entries.shell.JobEntryShell.executeShell(JobEntryShell.java:592)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at org.pentaho.di.job.entries.shell.JobEntryShell.execute(JobEntryShell.java:428)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at org.pentaho.di.job.Job.execute(Job.java:703)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at org.pentaho.di.job.Job.execute(Job.java:844)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at org.pentaho.di.job.Job.execute(Job.java:513)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at org.pentaho.di.job.Job.run(Job.java:393)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP - Caused by: java.io.IOException: error=13, Permission denied
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:340)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:271)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
    2023/04/19 15:44:19 - Archive Old Exception Files in SFTP -     ... 7 more
    2023/04/19 15:44:19 - J_MCO_EXCEPTION_REPORT - Finished job entry [Archive Old Exception Files in SFTP] (result=[false])
    2023/04/19 15:44:19 - J_MCO_EXCEPTION_REPORT - Job execution finished
    2023/04/19 15:44:19 - Carte - Installing timer to purge stale objects after 1440 minutes.
    2023/04/19 15:44:19 - Kitchen - Finished!
    2023/04/19 15:44:19 - Kitchen - ERROR (version 9.3.0.0-428, build 9.3.0.0-428 from 2022-04-12 04.56.25 by buildguy) : Finished with errors
    2023/04/19 15:44:19 - Kitchen - Start=2023/04/19 15:44:19.172, Stop=2023/04/19 15:44:19.251
    2023/04/19 15:44:19 - Kitchen - Processing ended after 0 seconds.

    Instead of modifying the permissions on /tmp mount point, can we change the temporary directory location ? like can we pass it as a parameter ?

    I tried running the command as "sh -x /opt/SIT/data-integration/kitchen.sh -file:/opt/SIT/Reporting/Transformations/J_MCO_EXCEPTION_REPORT.kjb -param:tmpDir=/opt/SIT/temp" but ignores the parameter. I might be using a wrong a parameter field for passing the temporary location.

    Can you please help me with it ?



    ------------------------------
    Thanks & Regards

    Saikrishna Vadlakonda
    Deloitte Consulting LLP.
    saikrishna5@deloitte.com | www.deloitte.com

    Please consider the environment before printing.
    ------------------------------


  • 2.  RE: Permission issue with tmp location - Pentaho 9.3 version

    Posted 04-20-2023 15:20

    HI Saikrishna,
    as you say, this is permission problem of system.
    If you need change temp directory, set JVM system property java.io.tmpdir, for ex. define env variable PENTAHO_DI_JAVA_OPTIONS=-Djava.io.tmpdir=/path/to/own/tmp or change in spoon.sh|bat.



    ------------------------------
    Petr Prochazka
    Systems Engineer
    P.V.A. systems s.r.o.
    ------------------------------



  • 3.  RE: Permission issue with tmp location - Pentaho 9.3 version

    Posted 04-25-2023 16:12

    Thank you for your response.

    The changes worked like a charm and the job ran successfully.



    ------------------------------
    Thanks & Regards

    Saikrishna Vadlakonda
    Deloitte Consulting LLP.
    saikrishna5@deloitte.com | www.deloitte.com

    Please consider the environment before printing.
    ------------------------------



  • 4.  RE: Permission issue with tmp location - Pentaho 9.3 version

    Posted 04-21-2023 14:50
    Expanding on Retr's response, 
     
    The -Djava.io.tmpdir=/path/to/own/tmp system property sets the location of the Java Virtual Machine (JVM) temporary directory to the specified directory path.
     
    In Pentaho, this system property is used to set the temporary directory location for various operations such as sorting data or writing temporary files. By default, the JVM uses the operating system's default temporary directory. However, in some cases, it may be necessary to change this location to a different directory to avoid permission issues or to use a faster or larger storage location.
     
    By specifying -Djava.io.tmpdir=/path/to/own/tmp, the JVM will use /path/to/own/tmp as the temporary directory location instead of the default location. You should replace /path/to/own/tmp with the desired directory path.
     
    You could export the PENTAHO_DI_JAVA_OPTIONS and run your script as follows:
     
    export PENTAHO_DI_JAVA_OPTIONS="-Djava.io.tmpdir=/path/to/own/tmp" && sh -x /opt/SIT/data-integration/kitchen.sh -file:/opt/SIT/Reporting/Transformations/J_MCO_EXCEPTION_REPORT.kjb
     
    Or if you have the PENTAHO_DI_JAVA_OPTIONS set in your environment, consider something along these lines to append the java.io.tmpdir
     
    export PENTAHO_DI_JAVA_OPTIONS="${PENTAHO_DI_JAVA_OPTIONS} -Djava.io.tmpdir=/path/to/own/tmp" && sh -x /opt/SIT/data-integration/kitchen.sh -file:/opt/SIT/Reporting/Transformations/J_MCO_EXCEPTION_REPORT.kjb
     
    And in this example, the PENTAHO_DI_JAVA_OPTIONS variable only exists for the duration of the kitchen.sh script.
     
    (PENTAHO_DI_JAVA_OPTIONS="${PENTAHO_DI_JAVA_OPTIONS} -Djava.io.tmpdir=/path/to/own/tmp" && sh -x /opt/SIT/data-integration/kitchen.sh -file:/opt/SIT/Reporting/Transformations/J_MCO_EXCEPTION_REPORT.kjb)


    Thanks for reaching out! Let us know if that resolves your issue. 



    ------------------------------
    Charles Chatt
    Global Director, Lumada Solutions Engineering
    Hitachi Vantara
    ------------------------------



  • 5.  RE: Permission issue with tmp location - Pentaho 9.3 version

    Posted 04-25-2023 16:13

    Thank you for your response.

    These in-detail explanation is very helpful to my team in successfully running the jobs.



    ------------------------------
    Thanks & Regards

    Saikrishna Vadlakonda
    Deloitte Consulting LLP.
    saikrishna5@deloitte.com | www.deloitte.com

    Please consider the environment before printing.
    ------------------------------