I'm using a lookup table to map some values to each other. During the ETL development, I realized a new column needed to be added to the lookup table.
After dropping and recreating the table, the lookup step only seems to recognize the old columns - the new column will not display in the dropdown. I even tried deleting the lookup step and recreating it - same thing.
If I create a Table Input step and let it auto-generate the SQL, it picks up the new column just fine. Any idea what might be going on here and how to correct it? If I create the mapping table with a new name, it picks up the new column just fine, so I'm thinking the table definition might be getting cached somewhere and not refreshed?