Pentaho

 View Only

 Unable to execute pan.bat from JSP

  • Pentaho
  • Kettle
  • Pentaho
  • Pentaho Data Integration PDI
saket Maheshwary's profile image
saket Maheshwary posted 08-06-2018 17:52

Currently in production using pentaho 6.1.0.1. Among other things we use it as a quick way to download data for our users. We have select queries to get this data and generate a CSV file.

To launch these queries we have JSP pages. Using JSP pages users can give input data. then we use

Process proc = Runtime.getRuntime().exec(etlCommand);

statement to execute a batch file. This batch file internally calls pan.bat/kitchen.bat to execute a PDI transformation/job

Once PDI job/transformation is done executing it generates a file at a given location. Once the batch file is done it returns the control back to JSP which provides a link for users to download the file.

I am currently in the process to upgrade to PDI 8.1.0.0. Now when we execute the batch file from within JSP, it generates the data file but even after batch file is done executing it does not returns the control back to the JSP. Because of which JSP eventually times out and does not provides a link to download.

When I execute the batch file directly from command prompt it shows me a bunch of info statements but eventually it finishes.

Can someone help me with I control is not returning? Let me know if any other information is required.

Thanks


#PentahoDataIntegrationPDI
#Kettle
#Pentaho
John Craig's profile image
John Craig

I am not sure this will be any help, but in some cases (such as in a PowerShell script), we've found it easier to run the Pentaho classes directly by calling java.exe rather than through the standard batch files. We just pulled the pieces out of the batch file that we needed and built the full command line. That turns the call into java.exe with the appropriate classes and parameters. I don't know if that will affect returning control to the JSP as you need to happen or not--may be worth a try because it bypasses the command shell.

saket Maheshwary's profile image
saket Maheshwary

Thanks for replying John, Can you give me an example of what exactly you did?

That might work for me because I can execute it as a Java command from inside our JSP and then once the command is done redirect to next page for download.

John Craig's profile image
John Craig

Sorry to be so slow getting back to you. I assume you've figured something out by now. But here's a fragment from a PowerShell script that we use as part of upgrading our ETL. The first part just builds the long set of arguments to run the launcher.jar which invokes org.pentaho.di.imp.Import (note that the launcher.jar must be used to run the Import class or you won't be able to connect to the repository. Only the last 2 lines contain parameters to the PDI Import class itself.

$BasicArgs =              "/k $PentahoDir\java\bin\java.exe "

$BasicArgs = $BasicArgs + '"-Dpentaho.installed.licenses.file=C:\Pentaho\.installedLicenses.xml" "-Xms4096m" "-Xmx8192m" "-Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2" '

$BasicArgs = $BasicArgs + '"-Djava.library.path=libswt\win64" "-DKETTLE_HOME=" "-DKETTLE_REPOSITORY=Local" "-DKETTLE_USER=admin" "-DKETTLE_PASSWORD=password" "-DKETTLE_PLUGIN_PACKAGES=" '

$BasicArgs = $BasicArgs + '"-DKETTLE_LOG_SIZE_LIMIT=" "-DKETTLE_JNDI_ROOT=" "-Dpentaho.installed.licenses.file=C:\Pentaho\.installedLicenses.xml" '

$BasicArgs = $BasicArgs + '-jar launcher\launcher.jar -lib ..\libswt\win64 -main org.pentaho.di.imp.Import '

$BasicArgs = $BasicArgs + '/rep:Local /user:admin /pass:password /dir:/ /norules:Y /replace:Y /comment:' + """$NewRelease"" "

$BasicArgs = $BasicArgs + '/filedir:' + """$ReleaseFolder\ETL"""

There's some additional logic to build a list of files to  load (saved in the variable $LoadFiles), then we run it like this:

# Append filenames to load to arguments

$BasicArgs = $BasicArgs + "$LoadFiles"

Start-Process -FilePath cmd.exe -ArgumentList $BasicArgs

(Presumably, you could skip running via Cmd.exe and just run java.exe as the process)

saket Maheshwary's profile image
saket Maheshwary

Hi John,

Sorry for the late reply, I got side tracked to a different project.

I have some more information, when I run the job from JSP, I get this extra line in log file:

18:28:09,390 ERROR [KarafLifecycleListener] The Kettle Karaf Lifecycle Listener failed to execute properly after waiting for 100 seconds. Releasing lifecycle hold, but some services may be unavailable.

If I execute the same command from command prompt, I do not.