Pentaho

 View Only

 TextFileOutput does not add filenames to result

  • Pentaho
  • Kettle
  • Pentaho
  • Pentaho Data Integration PDI
Jordan Sztanyo's profile image
Jordan Sztanyo posted 03-14-2019 16:52

Hello,

I am seeing an issue, starting in 8.1, where the TextFileOutput step does not add filenames to the result if the "Do not create file at start" and "Add filenames to result" options are checked.  The transformation runs successfully in Soap (creates the file) but when I execute it in my Maven-based Java project, the result does not contain any added filenames from the step (even though the files are created).

If I uncheck the "Do not create file at start" option for the step, then the filenames will be added to the result in 8.1 but will not be added in 8.2 and higher. In other words, in 8.2 and higher, the "Add filenames to result" option does not appear to be working correctly.

There are several Jira tickets that have been created around the performance of the TextFileOutput step so I do not know if what I am seeing is a known issue or a bug. I have not found anything in Jira related to what I am seeing but I do not know if creating a Jira ticket is the right next step (so I am reaching out to the community first).

What is the preferred avenue for triaging and reporting these kinds of issues?

I have attached a Maven project with some test cases that demonstrate the issue. It has profiles for different versions of Kettle which can be run using the following commands:

  • 8.0 - mvn -P8.0 clean test
  • 8.1 - mvn -P8.1 clean test
  • 8.2 - mvn -P8.2 clean test
  • 8.3 - mvn -P8.3 clean test

TextFileOutput related tickets:

https://jira.pentaho.com/browse/PDI-17952 (on backlog)

https://jira.pentaho.com/browse/PDI-17577 (fixed in 8.2)


#PentahoDataIntegrationPDI
#Kettle
#Pentaho
Jordan Sztanyo's profile image
Jordan Sztanyo

Just to follow up, there was a commit to fix the issue I was seeing. I updated to the 8.3.0.0-RC and it appears to have addressed the problem.

Here is a link to the commit:

[BACKLOG-28007] "Get files from result" Step Is Not Returning The Files · pentaho/pentaho-kettle@b202a79 · GitHub

Data Conversion's profile image
Data Conversion
Attachment  View in library