We are upgrading Pentaho from 7.1 to 8.3.
v7.1 connects to Hadoop 2.6
whereas v8.3 connects to Hadoop 3.1
and we are finding that the map-reduce step in 7.1 writes to Hadoop 2.6 in readable format. Code is exactly the same in both cases.
whereas the same step in 8.3 writes to Hadoop 3.1 in non-readable format.
As an example, below is what we see in 7.1 when we read the data in HDFS
hdfs dfs -text msgs_hr21_d221025_gz/part-00000 |more
1``214348`T FF: 4DDX300 :
whereas the same in 8.3 yields
hdfs dfs -text msgs_hr21_d221025_gz/part-00000 |more
6`��`��5`�`�r�+��)P��H���7��`h�`�`=`w� ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` `
Has anyone else faced this issue before? If you have, please can you provide any pointers or solutions?
thank you.
------------------------------
Satyan Iyer
Director of Services
Travelport, LP
------------------------------