Pentaho

 View Only

 Process rows into subsets

  • Pentaho
  • Kettle
  • Pentaho
  • Pentaho Data Integration PDI
Raj Karan's profile image
Raj Karan posted 02-26-2018 20:55

I am not sure is it possible or not.

I will just go ahead and explain my scenario

I have an HBase table in which I am recording HL7 messages. All the consecutive messages for a patient is one record in Hbase table.

Now I have find out all the patients who were in a given department during given time period, this part is simple.

After this I have to determine how long patient was in the department, was he transferred in or admitted, was patient transferred or discharged. This requires me to break the table into subset of patient with their consecutive messages. I am not sure with PDI I can achieve this.

Any suggestions please?


#Pentaho
#Kettle
#PentahoDataIntegrationPDI
Paulo Pires's profile image
Paulo Pires posted 03-01-2018 11:02

You could use regular expressions

Diego Mainou's profile image
Diego Mainou posted 03-01-2018 14:41

Well, it kind of sounds like you need to call your data preform your calculations and add back to either the patient's own message queue or to a summary queue.

Raj Karan's profile image
Raj Karan posted 03-01-2018 21:01

How would regular expression will help.

I have list of patient who were in department during given time period. I will have to break down this list into subsets containing consecutive messages for a patient. This doesn't seems to be possible with PDI.

I will have to process every subset in parallel and put all results back together or process every subset linearly.

Not sure how would I do that.

Paulo Pires's profile image
Paulo Pires posted 03-02-2018 08:29

regexp is great for patterns, and it looked to me you have a pattern in your messages, but I don't have any example so I could be wrong

Related Content