Pentaho

 View Only
Expand all | Collapse all

Python Shell Scripts ModuleNotFoundError

This thread has been viewed 4 times
  • 1.  Python Shell Scripts ModuleNotFoundError

    Posted 15 days ago

    Hello comunity again!

    Today i have the following issue...

    I have some Python Shell Scripts where they do some activities with my PDI ETL. Those scripts contain the pandas and bs4 dependencies but when i run those scripts at console level it doesn't trow an error in the running procedure (because we have installed pandas and bs4 inside our server where it is installed Pentaho Server) but when i schedule it in the Pentaho Web Service it trows out the following error:

    Job - Start of job execution
    Job - Starting entry [Shell 3]
    Shell 3 - Running on platform : Linux
    Shell 3 - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87037a5e-0e27-11ef-9789-f761d4300bd8shell
    Job - Starting entry [Shell]
    Shell - Running on platform : Linux
    Shell - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87857b9f-0e27-11ef-9789-f761d4300bd8shell
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) Traceback (most recent call last):
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)   File "/home/pentaho/convertir_excel_a_csv.py", line 1, in <module>
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)     import pandas as pd
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) ModuleNotFoundError: No
    Job - Start of job execution
    Job - Starting entry [Shell 3]
    Shell 3 - Running on platform : Linux
    Shell 3 - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87037a5e-0e27-11ef-9789-f761d4300bd8shell
    Job - Starting entry [Shell]
    Shell - Running on platform : Linux
    Shell - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87857b9f-0e27-11ef-9789-f761d4300bd8shell
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) Traceback (most recent call last):
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)   File "/home/pentaho/convertir_excel_a_csv.py", line 1, in <module>
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)     import pandas as pd
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) ModuleNotFoundError: No module named 'pandas'
    Job - Finished job entry [Shell] (result=[false])
    Job - Start of job execution
    Job - Starting entry [Shell 3]
    Shell 3 - Running on platform : Linux
    Shell 3 - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87037a5e-0e27-11ef-9789-f761d4300bd8shell
    Job - Starting entry [Shell]
    Shell - Running on platform : Linux
    Shell - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87857b9f-0e27-11ef-9789-f761d4300bd8shell
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) Traceback (most recent call last):
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)   File "/home/pentaho/convertir_excel_a_csv.py", line 1, in <module>
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)     import pandas as pd
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) ModuleNotFoundError: No module named 'pandas'
    Job - Finished job entry [Shell] (result=[false])
    Job - Finished job entry [Shell 3] (result=[false

    Job - Starting entry [Shell 3]
    Shell 3 - Running on platform : Linux
    Shell 3 - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87037a5e-0e27-11ef-9789-f761d4300bd8shell
    Job - Starting entry [Shell]
    Shell - Running on platform : Linux
    Shell - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87857b9f-0e27-11ef-9789-f761d4300bd8shell
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) Traceback (most recent call last):
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)   File "/home/pentaho/convertir_excel_a_csv.py", line 1, in <module>
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)     import pandas as pd
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) ModuleNotFoundError: No module named 'pandas'
    Job - Finished job entry [Shell] (result=[false])
    Job - Finished job entry [Shell 3] (result=[false])
    Job - Job execution finished

    As i can said Kettle at Web Service instance doesn't find the location where its installed the library and they said that the module isn't it installed so my question is...

    Where i need to install all those dependencies so the Kettle in the WebServer can find it?

    Thanks in advance!

     



    ------------------------------
    Mariano Dabove
    Others
    ItPatagonia
    ------------------------------


  • 2.  RE: Python Shell Scripts ModuleNotFoundError

    Posted 4 days ago

    Hi is there any solution for this?

    If i can have an specific situation is:

    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)     import pandas as pd
    Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) ModuleNotFoundError: No module named 'pandas'

    I tried to create a virtual environment so i can have installed all of those libraries so pentaho can read all of those modules but when i want to run the virtual environment on pentaho pdi i have this statement...

    2024/05/14 18:36:00 - Job - Start of job execution
    2024/05/14 18:36:00 - Job - Starting entry [Shell 3]
    2024/05/14 18:36:00 - Shell 3 - Running on platform : Linux
    2024/05/14 18:36:00 - Shell 3 - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_f4cdd26a-1239-11ef-9789-f761d4300bd8shell
    2024/05/14 18:36:00 - Job - Starting entry [Shell]
    2024/05/14 18:36:00 - Shell - Running on platform : Linux
    2024/05/14 18:36:00 - Shell - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_f51cdccb-1239-11ef-9789-f761d4300bd8shell
    2024/05/14 18:36:00 - Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) /home/pentaho/pentaho-server/tomcat/temp/kettle_f51cdccb-1239-11ef-9789-f761d4300bd8shell: 1: source: not found
    2024/05/14 18:36:00 - Job - Finished job entry [Shell] (result=[false])
    2024/05/14 18:36:00 - Job - Finished job entry [Shell 3] (result=[false])
    2024/05/14 18:36:00 - Job - Job execution finished

    Please i need to find a solution of this because its the only thing i need to run my proyect with schedules from pentaho server...

    Thanks in advance!



    ------------------------------
    Mariano Dabove
    Others
    ItPatagonia
    ------------------------------