Pentaho

 View Only

 Copy rows to result behavior

  • Pentaho
  • Kettle
  • Pentaho
  • Pentaho Data Integration PDI
Daniel Clark's profile image
Daniel Clark posted 02-19-2019 15:46

I'm following this guide located at https://www.clearpeaks.com/copy-rows-to-result-in-kettle/. I'm using the REST client in one transformation to retrieve a resultset and then using the copy rows to result to pass the resultset to another transformation, where I access the individual fields as parameters. This is mostly working, however I'm finding that certain parameters that should contain a null value instead contain a value for previous field in the resultset. Is this expected behavior? Any ideas on how to overcome this? I'm using pdi 8.2.0.0-342. I have 'Execute every input row' checked.

masterjob_completed_courses_repthave a null value, get_limos_users_parametersget_limos_users_execget_limos_users_transformation_get_variables


#Pentaho
#Kettle
#PentahoDataIntegrationPDI
Sparkles Sparkles's profile image
Sparkles Sparkles posted 02-20-2019 10:25

I have never found any satisfying documentation for the dirty details of result rows. It seems like nobody knows exactly how this works. Only way to find out is test yourself. That's what I did. What I found was similar to what you describe here.

For example if you in a transformation get the result rows (from previous job/trans), then filter the rows (true/false), then only copy the true rows to the result set for subsequent jobs/trans, this will only work IF there is at least 1 true row! If there are no true rows, all the input rows (true and false) are sent to the subsequent jobs/trans.

Which means that if you wanted to have this kind of rows filter for result rows, you would need to implement some sort of dummy null "empty" row so that you could ALWAYS send 1 row to "copy rows to result" even if you filtered away all of them.

This is messy and I have yet to find info on this level in any books or articles. Since result rows operate partially on job level also, it's difficult to debug and see what's actually going on.

Related Content