This might be a simple question so apologies in advance.
I have a job I only want to run if the data in the source table has been updated I thought about checking the latest update date in the source file or comparing counts but I am not sure which combination of actions to us to get this to work.
My ideal flow will be:
1. select max('update_date) from table (This i can achieve by a sql query with Table Input)
2. Check id now() == max('update_date) -
if Yes run the Job, if not Do nothing
So i guess my main question is how do I perform this data matching condition?
If I can then only add the new data to the existing table instead of pulling all the available data that will be great as well.
Any advice here will be greatly appreciated!