Pentaho

 View Only
Back to discussions
Expand all | Collapse all

Troubleshooting 'No such file or directory' Error for Network Paths in Pentaho Server Job Execution

This thread has been viewed 47 times

  • 1.  Troubleshooting 'No such file or directory' Error for Network Paths in Pentaho Server Job Execution

    Posted 02-22-2024 14:33

    I'm facing an issue with a Pentaho Server job that executes a Python script (Automacao_latromi_relatoriobi_inbound_V4.py). The job fails with a "No such file or directory" error when attempting to access an Excel file located on a network drive. This problem occurs only when the job is executed on the Pentaho Server; the script runs successfully on my local machine and directly within PDI Spoon.

    I've tried various ways to specify the directory paths to ensure they're correct and accessible, but the problem persists exclusively in the Pentaho Server environment.

    Error Log:

    

    2024-02-22 11:33:34,339 INFO  [org.pentaho.di] 2024/02/22 11:33:34 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout) importando dados
2024-02-22 11:33:34,339 ERROR [org.pentaho.di] 2024/02/22 11:33:34 - Automacao_latromi_relatoriobi_inbound_V4.py - ERROR : (stderr)   File "D:\KN_BI_MASTER\Pentaho_pipelines_e_scripts\1. Latromi\1. Check in Inbound\1. Scripts\Automacao_latromi_relatoriobi_inbound_V4.py", line 150, in <module>
2024-02-22 11:33:34,339 INFO  [org.pentaho.di] 2024/02/22 11:33:34 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout) Falha na tentativa 1: [Errno 2] No such file or directory: 'H:\\POWER_BI.sao\\Bases_Shein\\Latromi/Relatório Janelas e Apontamentos - YMS Shein.xlsx'

    Path Variations Tried:

    • r'H:\POWER_BI.sao\Bases_Shein\Latromi\Relatório Janelas e Apontamentos - YMS Shein.xlsx'
    • 'H:\POWER_BI.sao\Bases_Shein\Latromi\Relatório Janelas e Apontamentos - YMS Shein.xlsx'
    • 'H:/POWER_BI.sao/Bases_Shein/Latromi/Relatório Janelas e Apontamentos - YMS Shein.xlsx'
    • 'H://POWER_BI.sao//Bases_Shein//Latromi//Relatório Janelas e Apontamentos - YMS Shein.xlsx'
    • r'H://POWER_BI.sao//Bases_Shein//Latromi//Relatório Janelas e Apontamentos - YMS Shein.xlsx'

    Despite these variations, the error persists when the job is run on the Pentaho Server.

    Troubleshooting Steps Taken:

    1. Confirmed the network path and file accessibility; the file is accessible from my machine, and the path is correct.
    2. Successfully ran the script on my local machine and through PDI Spoon.
    3. Checked network permissions; the Pentaho Server should have access to the network drive.

    I suspect the issue might relate to how Pentaho Server handles network paths or permissions, but I'm unsure how to proceed to resolve this. Has anyone encountered a similar issue or can provide guidance on troubleshooting this further in the Pentaho Server environment?

    Any insights or suggestions would be greatly appreciated.

    Thank you.



    ------------------------------
    Lucas Abrantes
    Others
    Kuehne Nagel
    ------------------------------



  • 2.  RE: Troubleshooting 'No such file or directory' Error for Network Paths in Pentaho Server Job Execution

    Posted 02-23-2024 05:19

    Hi Lucas,

    how is start server, as windows service or standalone via bat file?

    IMHO it's account permission to remote disk drive.



    ------------------------------
    Petr Prochazka
    Systems Engineer
    P.V.A. systems s.r.o.
    ------------------------------

    Original Message


  • 3.  RE: Troubleshooting 'No such file or directory' Error for Network Paths in Pentaho Server Job Execution

    Posted 02-23-2024 06:43

    @Petr Prochazka 

    Hello, and thank you for your response!

    I am starting the Pentaho Server by running the startup.bat file located in the Pentaho folder, and I am doing this as an administrator on Windows Server 2016.



    ------------------------------
    Lucas Abrantes
    Others
    Kuehne Nagel
    ------------------------------

    Original Message


  • 4.  RE: Troubleshooting 'No such file or directory' Error for Network Paths in Pentaho Server Job Execution

    Posted 02-23-2024 06:59

    OK, could you provide full stack trace of exception?

    Excel file is loaded from Python script, right?



    ------------------------------
    Petr Prochazka
    Systems Engineer
    P.V.A. systems s.r.o.
    ------------------------------

    Original Message


  • 5.  RE: Troubleshooting 'No such file or directory' Error for Network Paths in Pentaho Server Job Execution

    Posted 02-23-2024 08:27

    @Petr Prochazka  Here is the full stack trace of the exception encountered:

    

    
2024-02-23 09:44:40,250 ERROR [org.pentaho.platform.util.logging.Logger] misc-org.pentaho.platform.web.http.context.PentahoSystemReadyListener: Cannot obtain JCR repository. Exiting
    2024-02-23 09:50:19,703 INFO  [org.pentaho.platform.engine.core.system.status.PeriodicStatusLogger] Caution, the system is initializing. Do not shut down or restart the system at this time.
    2024-02-23 09:50:20,061 INFO  [org.pentaho.platform.osgi.OSGIBoot] Checking to see if org.pentaho.clean.karaf.cache is enabled
    2024-02-23 09:50:23,224 INFO  [org.pentaho.platform.osgi.KarafInstance]
    *******************************************************************************
    *** Karaf Instance Number: 1 at C:\Program Files\Pentaho Server\pentaho-ser ***
    ***   ver\pentaho-solutions\system\karaf\caches\default\data-1              ***
    *** Karaf Port:8802                                                         ***
    *** OSGI Service Port:9051                                                  ***
    *******************************************************************************
    2024-02-23 09:50:24,875 DEBUG [org.pentaho.di.core.vfs.ConcurrentFileSystemManager] Using "C:\Program Files\Pentaho Server\pentaho-server\tomcat\temp\vfs_cache" as temporary files store.
    2024-02-23 09:50:24,973 DEBUG [org.pentaho.di.core.vfs.ConcurrentFileSystemManager] Skipping provider "org.apache.commons.vfs2.provider.webdav.WebdavFileProvider" because required class "org.apache.jackrabbit.webdav.client.methods.DavMethod" is not available.
    2024-02-23 09:50:31,595 INFO  [org.pentaho.hadoop.shim.DriverManager] Installing driver kars.
    2024-02-23 09:50:31,598 INFO  [org.pentaho.hadoop.shim.DriverManager] 0 drivers will be installed.
    2024-02-23 09:50:31,599 INFO  [org.pentaho.hadoop.shim.DriverManager] Finished installing drivers kars.
    2024-02-23 09:50:43,243 INFO  [org.pentaho.platform.engine.core.system.status.PeriodicStatusLogger] The system has finished initializing.
    2024-02-23 10:05:24,316 DEBUG [org.pentaho.di.repository.RepositoryConnectionUtils] Creating Meta-repository
    2024-02-23 10:05:24,320 DEBUG [org.pentaho.di.repository.RepositoryConnectionUtils]  Populating Meta repository
    2024-02-23 10:05:24,320 DEBUG [org.pentaho.di.repository.RepositoryConnectionUtils] singleDiServerInstance=true, loading default repository
    2024-02-23 10:05:24,325 INFO  [org.pentaho.di] 2024/02/23 10:05:24 - RepositoriesMeta - Reading repositories XML file: FromInputStream
    2024-02-23 10:05:24,337 DEBUG [org.pentaho.di.repository.RepositoryConnectionUtils]  Finding repository metadata
    2024-02-23 10:05:24,337 DEBUG [org.pentaho.di.repository.RepositoryConnectionUtils]  Getting repository instance
    2024-02-23 10:05:24,363 DEBUG [org.pentaho.di.repository.RepositoryConnectionUtils] Connecting to repository
    2024-02-23 10:05:24,390 DEBUG [org.pentaho.di.repository.RepositoryConnectionUtils]  Connected to repository
    2024-02-23 10:05:24,685 INFO  [org.pentaho.di] 2024/02/23 10:05:24 - Carte - Installing timer to purge stale objects after 240 minutes.
    2024-02-23 10:05:24,707 INFO  [org.pentaho.di] 2024/02/23 10:05:24 - check_in_inbound_dashboard2 - Start of job execution
    2024-02-23 10:05:24,760 INFO  [org.pentaho.di] 2024/02/23 10:05:24 - check_in_inbound_dashboard2 - Starting entry [Automacao_latromi_relatoriobi_inbound_V4.py]
    2024-02-23 10:05:24,765 INFO  [org.pentaho.di] 2024/02/23 10:05:24 - Automacao_latromi_relatoriobi_inbound_V4.py - Running on platform : Windows Server 2016
    2024-02-23 10:05:24,773 INFO  [org.pentaho.di] 2024/02/23 10:05:24 - Automacao_latromi_relatoriobi_inbound_V4.py - Executing command : cmd.exe /C ""C:\Program Files\Pentaho Server\pentaho-server\tomcat\temp\kettle_353e0bd5-d24c-11ee-b563-7704f62af544shell.bat""
    2024-02-23 10:05:24,838 INFO  [org.pentaho.di] 2024/02/23 10:05:24 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout)
    2024-02-23 10:05:24,841 INFO  [org.pentaho.di] 2024/02/23 10:05:24 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout) C:\Program Files\Pentaho Server\pentaho-server\tomcat\bin>python "D:\KN_BI_MASTER\Pentaho_pipelines_e_scripts\1. Latromi\1. Check in Inbound\1. Scripts\Automacao_latromi_relatoriobi_inbound_V4.py"
    2024-02-23 10:05:27,232 ERROR [org.pentaho.di] 2024/02/23 10:05:27 - Automacao_latromi_relatoriobi_inbound_V4.py - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) Exception managing chrome: error sending request for url (https://googlechromelabs.github.io/chrome-for-testing/known-good-versions-with-downloads.json): error trying to connect: An existing connection was forcibly closed by the remote host. (os error 10054)
    2024-02-23 10:07:01,334 ERROR [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) Traceback (most recent call last):
    2024-02-23 10:07:01,335 INFO  [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout) importando dados
    2024-02-23 10:07:01,338 ERROR [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)   File "D:\KN_BI_MASTER\Pentaho_pipelines_e_scripts\1. Latromi\1. Check in Inbound\1. Scripts\Automacao_latromi_relatoriobi_inbound_V4.py", line 156, in <module>
    2024-02-23 10:07:01,339 INFO  [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout) importando dados
    2024-02-23 10:07:01,340 ERROR [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)     data_relatorio  = data_relatorio .rename(columns={'Solicitado por': 'Solicitante'})
    2024-02-23 10:07:01,340 INFO  [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout) Falha na tentativa 1: [Errno 2] No such file or directory: 'H:\\\\POWER_BI.sao\\\\Bases_Shein\\\\Latromi\\\\Relat�rio Janelas e Apontamentos - YMS Shein.xlsx'
    2024-02-23 10:07:01,342 INFO  [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout) importando dados
    2024-02-23 10:07:01,342 ERROR [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)                       ^^^^^^^^^^^^^^^^^^^^^^
    2024-02-23 10:07:01,343 INFO  [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout) Falha na tentativa 2: [Errno 2] No such file or directory: 'H:\\\\POWER_BI.sao\\\\Bases_Shein\\\\Latromi\\\\Relat�rio Janelas e Apontamentos - YMS Shein.xlsx'
    2024-02-23 10:07:01,344 ERROR [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) AttributeError: 'NoneType' object has no attribute 'rename'
    2024-02-23 10:07:01,345 INFO  [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout) importando dados
    2024-02-23 10:07:01,349 INFO  [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout) Falha na tentativa 3: [Errno 2] No such file or directory: 'H:\\\\POWER_BI.sao\\\\Bases_Shein\\\\Latromi\\\\Relat�rio Janelas e Apontamentos - YMS Shein.xlsx'
    2024-02-23 10:07:01,350 INFO  [org.pentaho.di] 2024/02/23 10:07:01 - Automacao_latromi_relatoriobi_inbound_V4.py - (stdout) erro
    2024-02-23 10:07:01,560 INFO  [org.pentaho.di] 2024/02/23 10:07:01 - check_in_inbound_dashboard2 - Finished job entry [Automacao_latromi_relatoriobi_inbound_V4.py] (result=[false])
    2024-02-23 10:07:01,562 INFO  [org.pentaho.di] 2024/02/23 10:07:01 - check_in_inbound_dashboard2 - Job execution finished
    2024-02-23 10:07:01,654 ERROR [org.pentaho.platform.plugin.kettle.PdiAction] PdiAction.ERROR_0008 - Job or Job Result had errors - Job Errors: 1, Job Result Errors: 1

    Regarding the Excel file, it is indeed loaded from a network location via a Python script, and it functions correctly when not run on the Pentaho server.

    and here is the exception in python code, as you can see thath even the local files cannot can be read (local in the code bellow is D: directory no in network) i forgot to mentioned but i also tried the UNC path type but the same issue and i could not execute in local so in pentaho server i also get a erros SAO-RESTRICT$ (\\aaagkptp1brz.CORP.INT.KN)

    usuarios_data = pd.read_excel(r'D:\KN_BI_MASTER\Pentaho_pipelines_e_scripts\1. Latromi\1. Check in Inbound\3. Planilhas\relacao_usuarios_shein.xlsx', sheet_name='relacao_sheinXfornecedores')
    data_relatorio = pd.read_excel(r'H:\POWER_BI.sao\Bases_Shein\Latromi\Relatório Janelas e Apontamentos - YMS Shein.xlsx')
     
    usuarios_data = pd.read_excel(r'D:\KN_BI_MASTER\Pentaho_pipelines_e_scripts\1. Latromi\1. Check in Inbound\3. Planilhas\relacao_usuarios_shein.xlsx', sheet_name='relacao_sheinXfornecedores')
     
    usuarios_data = pd.read_excel(r'D:\KN_BI_MASTER\Pentaho_pipelines_e_scripts\1. Latromi\1. Check in Inbound\3. Planilhas\relacao_usuarios_shein.xlsx', sheet_name='relacao_sheinXfornecedores')
     
    usuarios_data = pd.read_excel(r'D:\KN_BI_MASTER\Pentaho_pipelines_e_scripts\1. Latromi\1. Check in Inbound\3. Planilhas\relacao_usuarios_shein.xlsx', sheet_name='relacao_sheinXfornecedores')
     
    arquivo_para_deletar = r'H:\POWER_BI.sao\Bases_Shein\Latromi\Relatório Janelas e Apontamentos - YMS Shein.xlsx'
     
    print("importando dados")
    tentativas_maximas = 3
    tentativa_atual = 0
    data_relatorio = None
    while tentativa_atual < tentativas_maximas and data_relatorio is None:
        try:
            print("importando dados")
            # data_relatorio = pd.read_excel(DestinoRelatorio + r'/Relatório Janelas e Apontamentos - YMS Shein.xlsx')
            # data_relatorio = pd.read_excel(r'H:\POWER_BI.sao\Bases_Shein\Latromi\Relatório Janelas e Apontamentos - YMS Shein.xlsx')
            data_relatorio = pd.read_excel(r'H:\\POWER_BI.sao\\Bases_Shein\\Latromi\\Relatório Janelas e Apontamentos - YMS Shein.xlsx')
        except (FileNotFoundError, PermissionError) as e:
            tentativa_atual += 1
            print(f"Falha na tentativa {tentativa_atual}: {e}")
            time.sleep(1)
    if data_relatorio is not None:
        print("sucesso")
    else:
        print("erro")

     



    ------------------------------
    Lucas Abrantes
    Others
    Kuehne Nagel
    ------------------------------

    Original Message


  • 6.  RE: Troubleshooting 'No such file or directory' Error for Network Paths in Pentaho Server Job Execution

    Posted 02-23-2024 08:48

    OK, you run external Python via Shell job entry. I don't known Python too deep, so I can not help you with this problem.

    IMHO script doesn't work correctly. You can try run from command line this: C:\Program Files\Pentaho Server\pentaho-server\tomcat\bin>python "D:\KN_BI_MASTER\Pentaho_pipelines_e_scripts\1. Latromi\1. Check in Inbound\1. Scripts\Automacao_latromi_relatoriobi_inbound_V4.py". This is called in generated script via Shell entry.



    ------------------------------
    Petr Prochazka
    Systems Engineer
    P.V.A. systems s.r.o.
    ------------------------------

    Original Message


  • 7.  RE: Troubleshooting 'No such file or directory' Error for Network Paths in Pentaho Server Job Execution

    Posted 02-23-2024 15:17

    @Petr Prochazka

    I attempted to run the script from the Tomcat directory in the Pentaho Server via the CMD terminal, but I encountered the same "No such file or directory" error.



    ------------------------------
    Lucas Abrantes
    Others
    Kuehne Nagel
    ------------------------------

    Original Message


  • 8.  RE: Troubleshooting 'No such file or directory' Error for Network Paths in Pentaho Server Job Execution

    Posted 02-27-2024 07:38

    bump



    ------------------------------
    Lucas Abrantes
    Others
    Kuehne Nagel
    ------------------------------

    Original Message


  • 9.  RE: Troubleshooting 'No such file or directory' Error for Network Paths in Pentaho Server Job Execution

    Posted 03-15-2024 08:44

    I know this is possibly a workaround but does your python script work if the file is local on the pc?  perhaps you can write a job that copies the file locally for your python script to process.

    I have a similar issue where (only excel files) i cannot read in pdi from my server - i need to copy them to a temp drive to process (i can write to a server however) 



    ------------------------------
    Taryn-Vee Burnett
    Others
    Marketaxess
    ------------------------------

    Original Message


  • 10.  RE: Troubleshooting 'No such file or directory' Error for Network Paths in Pentaho Server Job Execution

    Posted 03-26-2024 11:55

    Hi @Lucas Abrantes

    I tough it might be problem with accent chars in name of file. Can you try remove accent chars from directories and files?



    ------------------------------
    Petr Prochazka
    Systems Engineer
    P.V.A. systems s.r.o.
    ------------------------------

    Original Message


Related Content