Pentaho

Hello comunity again!

Today i have the following issue...

I have some Python Shell Scripts where they do some activities with my PDI ETL. Those scripts contain the pandas and bs4 dependencies but when i run those scripts at console level it doesn't trow an error in the running procedure (because we have installed pandas and bs4 inside our server where it is installed Pentaho Server) but when i schedule it in the Pentaho Web Service it trows out the following error:

Job - Start of job execution
Job - Starting entry [Shell 3]
Shell 3 - Running on platform : Linux
Shell 3 - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87037a5e-0e27-11ef-9789-f761d4300bd8shell
Job - Starting entry [Shell]
Shell - Running on platform : Linux
Shell - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87857b9f-0e27-11ef-9789-f761d4300bd8shell
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) Traceback (most recent call last):
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)   File "/home/pentaho/convertir_excel_a_csv.py", line 1, in <module>
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)     import pandas as pd
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) ModuleNotFoundError: No
Job - Start of job execution
Job - Starting entry [Shell 3]
Shell 3 - Running on platform : Linux
Shell 3 - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87037a5e-0e27-11ef-9789-f761d4300bd8shell
Job - Starting entry [Shell]
Shell - Running on platform : Linux
Shell - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87857b9f-0e27-11ef-9789-f761d4300bd8shell
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) Traceback (most recent call last):
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)   File "/home/pentaho/convertir_excel_a_csv.py", line 1, in <module>
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)     import pandas as pd
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) ModuleNotFoundError: No module named 'pandas'
Job - Finished job entry [Shell] (result=[false])
Job - Start of job execution
Job - Starting entry [Shell 3]
Shell 3 - Running on platform : Linux
Shell 3 - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87037a5e-0e27-11ef-9789-f761d4300bd8shell
Job - Starting entry [Shell]
Shell - Running on platform : Linux
Shell - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87857b9f-0e27-11ef-9789-f761d4300bd8shell
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) Traceback (most recent call last):
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)   File "/home/pentaho/convertir_excel_a_csv.py", line 1, in <module>
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)     import pandas as pd
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) ModuleNotFoundError: No module named 'pandas'
Job - Finished job entry [Shell] (result=[false])
Job - Finished job entry [Shell 3] (result=[false

Job - Starting entry [Shell 3]
Shell 3 - Running on platform : Linux
Shell 3 - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87037a5e-0e27-11ef-9789-f761d4300bd8shell
Job - Starting entry [Shell]
Shell - Running on platform : Linux
Shell - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_87857b9f-0e27-11ef-9789-f761d4300bd8shell
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) Traceback (most recent call last):
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)   File "/home/pentaho/convertir_excel_a_csv.py", line 1, in <module>
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)     import pandas as pd
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) ModuleNotFoundError: No module named 'pandas'
Job - Finished job entry [Shell] (result=[false])
Job - Finished job entry [Shell 3] (result=[false])
Job - Job execution finished

As i can said Kettle at Web Service instance doesn't find the location where its installed the library and they said that the module isn't it installed so my question is...

Where i need to install all those dependencies so the Kettle in the WebServer can find it?

Thanks in advance!

Hi is there any solution for this?

If i can have an specific situation is:

Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr)     import pandas as pd
Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) ModuleNotFoundError: No module named 'pandas'

I tried to create a virtual environment so i can have installed all of those libraries so pentaho can read all of those modules but when i want to run the virtual environment on pentaho pdi i have this statement...

2024/05/14 18:36:00 - Job - Start of job execution
2024/05/14 18:36:00 - Job - Starting entry [Shell 3]
2024/05/14 18:36:00 - Shell 3 - Running on platform : Linux
2024/05/14 18:36:00 - Shell 3 - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_f4cdd26a-1239-11ef-9789-f761d4300bd8shell
2024/05/14 18:36:00 - Job - Starting entry [Shell]
2024/05/14 18:36:00 - Shell - Running on platform : Linux
2024/05/14 18:36:00 - Shell - Executing command : /home/pentaho/pentaho-server/tomcat/temp/kettle_f51cdccb-1239-11ef-9789-f761d4300bd8shell
2024/05/14 18:36:00 - Shell - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : (stderr) /home/pentaho/pentaho-server/tomcat/temp/kettle_f51cdccb-1239-11ef-9789-f761d4300bd8shell: 1: source: not found
2024/05/14 18:36:00 - Job - Finished job entry [Shell] (result=[false])
2024/05/14 18:36:00 - Job - Finished job entry [Shell 3] (result=[false])
2024/05/14 18:36:00 - Job - Job execution finished

Please i need to find a solution of this because its the only thing i need to run my proyect with schedules from pentaho server...

Thanks in advance!