Pentaho

 View Only

 How to temporarily save data in the process of conversion

Jump to Best Answer
Syuu Hasegawa's profile image
Syuu Hasegawa posted 02-12-2023 20:29

Hi,

There is an issue with the data migration I am working on. It is a way to temporarily save data in the process of conversion in Pentaho.

Pentaho has the ability to export converted data to tables and text files, but I cannot find a way to save the data in the process of conversion to memory, etc. I could be missing it.

If it exists, I would be happy to be informed.

Thanks in advance.

Thomas Stutz's profile image
Thomas Stutz Best Answer

You might want to use the "Serialize to file", and "De-serialize from file" steps.

Derek Wilson's profile image
Derek Wilson

Various ways to debug your transformations while in flight locally or through server. In both cases I simply create a new hop (copy) out to either a file, db or dummy step. 

John Craig's profile image
John Craig

You can use the somewhat oddly-named "Blocking step"; this allows you can cache up data. Check the box to pass all rows; set how many rows are contained in memory. That may be what you need. We use this step when we have multiple paths through a transformation that need to be resynced at some point.

Syuu Hasegawa's profile image
Syuu Hasegawa

Hello Thomas,

The answers were close to what I need right now.
Everyone else's answers were also very helpful.

Thank you.