AnsweredAssumed Answered

Appending new data to existing table based on date

Question asked by Alina Lot on Apr 3, 2018
Latest reply on Sep 26, 2018 by Alina Lot

Hi,

I am pretty new to Kettle so apologies if my question is too simple -

I want to create an ETL process that will allow me to test what rows currently exist in the output table (Max date for example) and then them use the import step to only import new data from the DB to the output table? i.e. only append  new data to the existing table without having to re pull all the data again from one DB to another (millions of records) Is there a combination of steps that can help me to achieve this?

 

Thanks!

Outcomes