Profile

Fisher Hao

This individual is no longer active. Application functionality related to this individual is limited.

Contact Details

My Content

1 to 13 of 13 total

Parquet input error

Posted By Fisher Hao 06-04-2019 04:06
Found In Community: Pentaho \ view thread
Dear expert I have tried to use Parquet input step to read a parquet file, but it encounter some error. I have already set hadoop distribution to CDH5.14 and reboot spoon, but it dose not work.the ktr and parquet file and error log can be found in attchment.Any comments and feedback is very much appreciated. ...

Parquet input error

Posted By Fisher Hao 06-04-2019 04:06
Found In Library: Pentaho

RE: how to launch more than 2 executors when using AEL running KTR on spark ?

Posted By Fisher Hao 05-11-2018 00:22
Found In Community: Pentaho \ view thread
Hi Christopher Got it, thanks!

RE: where can I find pentaho-workernodes-8.0.0.0-XXX-dist.zip?

Posted By Fisher Hao 03-28-2018 00:50
Found In Community: Pentaho \ view thread
Brandon Thanks for your kindly support.

where can I find pentaho-workernodes-8.0.0.0-XXX-dist.zip?

Posted By Fisher Hao 03-23-2018 00:53
Found In Community: Pentaho \ view thread
Hi experts I want to learn how to config pentaho workernodes, but I cant find the installation zip file. As Pentaho documents said : Access to this ZIP file and the Pentaho Worker Nodes product is available to select customers. I wonder if its open to download for hitachi vantara internal employees or ...

RE: how to launch more than 2 executors when using AEL running KTR on spark ?

Posted By Fisher Hao 03-20-2018 04:41
Found In Community: Pentaho \ view thread
Christopher Thats great, Im looking forward to it, thanks!

RE: how to launch more than 2 executors when using AEL running KTR on spark ?

Posted By Fisher Hao 03-20-2018 00:55
Found In Community: Pentaho \ view thread
Hey Diethard Thanks for your comment! Looking forward to be solved in PDI8.1

RE: "Kafka Consumer" step run abnormally when using AEL-spark

Posted By Fisher Hao 02-14-2018 11:05
Found In Community: Pentaho \ view thread
It have been solved, the field list in Get Records from Stream step was incorrect. I only define one field in this step. According to this doc [PDI-16829] The Get Records From Stream step is retrieving topic metadata in an order unchangeable by the user in Spark … The field list must equal to the following ...

RE: "Kafka Consumer" step run abnormally when using AEL-spark

Posted By Fisher Hao 02-14-2018 08:11
Found In Community: Pentaho \ view thread
attached spark executor log for reference.

"Kafka Consumer" step run abnormally when using AEL-spark

Posted By Fisher Hao 02-14-2018 03:26
Found In Community: Pentaho \ view thread
Dear experts      Im trying to use PDI to process streaming data from kafka, I have created two transform.      In the parent transform, there is only one step Kafka Consumer, its used to receive the stream data from topic1 on kafka cluster.      In the sub-transform, there are two steps: Get records ...

"Kafka Consumer" step run abnormally when using AEL-spark

Posted By Fisher Hao 02-14-2018 03:26
Found In Library: Pentaho

RE: how to launch more than 2 executors when using AEL running KTR on spark ?

Posted By Fisher Hao 02-13-2018 02:41
Found In Community: Pentaho \ view thread
Hey Christopher Thanks for your reply, I have tried again by following your instruction and it works (PDI8+hadoop2.8.3+jdk8+spark2.1). The AEL daemon starts 4 executors and each with 4 cores. It run faster than before. But there are still a problem remain, acctually Im trying to do a simple wordcount ...

how to launch more than 2 executors when using AEL running KTR on spark ?

Posted By Fisher Hao 02-11-2018 02:42
Found In Community: Pentaho \ view thread
Dear Im trying to use PDI AEL to run transform with spark engin on yarn. But I found that the transform job was running very slow and there were only two executors launched, I have tried adding two lines in AEL application.properties as the following:sparkNumExecutors=8   sparkExecutorCores=4   it seems ...