Pentaho

 View Only

 Need help with building pentaho-kettle from github

  • Pentaho
  • Pentaho
Nikola Garafolic's profile image
Nikola Garafolic posted 05-09-2019 08:26

This content was either too long or contained formatting that did not work with our migration. A PDF document is attached that contains the original representation

 

I have problems building pentaho-kettle from github, either by cloning or using latest release, build always fails:ERROR] Failures: [ERROR] PurRepositoryStressTest.runLoadTest:578[INFO] [ERROR] Tests run: 262, Failures: 1, Errors: 0, Skipped: 0[INFO] [INFO] ------------------------------------------------------------------------[INFO] Reactor Summary for Pentaho Data Integration 8.3.0.0-SNAPSHOT:[INFO] [INFO] Pentaho Data Integration ........................... SUCCESS [ 2.497 s][INFO] PDI Engine Extensions .............................. SUCCESS [ 0.120 s][INFO] PDI Engine API ..................................... SUCCESS [ 6.411 s][INFO] PDI Core ........................................... SUCCESS [ 39.454 s][INFO] PDI Engine ......................................... SUCCESS [07:13 min][INFO] PDI DB Dialog ...................................... SUCCESS [ 13.388 s][INFO] PDI User Interface ................................. SUCCESS [ 36.657 s][INFO] PDI Plugins ........................................ SUCCESS [ 0.160 s][INFO] PDI Log4j Plugin ................................... SUCCESS [ 0.119 s][INFO] PDI Log4j Plugin Core .............................. SUCCESS [ 2.121 s][INFO] PDI Log4j Plugin Assemblies ........................ SUCCESS [ 0.071 s][INFO] PDI Log4j Plugin Distribution ...................... SUCCESS [ 0.854 s][INFO] PDI Core Plugins ................................... SUCCESS [ 0.074 s][INFO] PDI Core Plugins Impl .............................. SUCCESS [ 8.602 s][INFO] PDI Core Plugins UI ................................ SUCCESS [ 0.912 s][INFO] PDI Core Plugins Assemblies ........................ SUCCESS [ 0.096 s][INFO] PDI Core Plugins Distribution ...................... SUCCESS [ 0.455 s][INFO] Core UI Plugin ..................................... SUCCESS [ 10.956 s][INFO] PDI File Open and Save Plugin ...................... SUCCESS [ 0.139 s][INFO] PDI File Open and Save Plugin Core ................. SUCCESS [ 10.073 s][INFO] PDI File Open and Save Plugin Assemblies ........... SUCCESS [ 0.113 s][INFO] PDI File Open and Save Plugin Distribution ......... SUCCESS [ 3.434 s][INFO] PDI File Open and Save New Plugin .................. SUCCESS [ 0.155 s][INFO] PDI File Open and Save New Plugin API .............. SUCCESS [ 0.843 s][INFO] PDI File Open and Save New Plugin Core ............. SUCCESS [ 3.507 s][INFO] PDI File Open and Save New Plugin Assemblies ....... SUCCESS [ 0.180 s][INFO] PDI File Open and Save New Plugin Distribution ..... SUCCESS [ 1.892 s][INFO] Connections Plugin ................................. SUCCESS [ 0.118 s][INFO] Connections Plugin UI .............................. SUCCESS [ 8.957 s][INFO] Connections Plugin Assemblies ...................... SUCCESS [ 0.148 s][INFO] Connections Plugin Distribution .................... SUCCESS [ 1.265 s][INFO] Get Fields Plugin .................................. SUCCESS [ 0.072 s][INFO] PDI Get Fields Plugin Core ......................... SUCCESS [ 3.393 s][INFO] PDI Get Fields Plugin Assemblies ................... SUCCESS [ 0.133 s][INFO] PDI Get Fields Plugin Distribution ................. SUCCESS [ 1.639 s][INFO] PDI Repositories Plugin ............................ SUCCESS [ 0.089 s][INFO] PDI Repositories Plugin Core ....................... SUCCESS [ 8.983 s][INFO] PDI Repositories Plugin Assemblies ................. SUCCESS [ 0.071 s][INFO] PDI Repositories Plugin Feature .................... SUCCESS [ 1.663 s][INFO] PDI Metadata Injection Plugin ...................... SUCCESS [ 27.664 s][INFO] PDI Engine Configuration Plugin .................... SUCCESS [ 0.063 s][INFO] PDI Engine Configuration Plugin API ................ SUCCESS [ 0.938 s][INFO] PDI Engine Configuration Plugin Implementation ..... SUCCESS [ 4.521 s][INFO] PDI Engine Configuration Plugin UI ................. SUCCESS [ 17.828 s][INFO] PDI Engine Configuration Plugin Assemblies ......... SUCCESS [ 0.027 s][INFO] PDI Engine Configuration Plugin Feature ............ SUCCESS [ 1.312 s][INFO] PDI Get Previous Row Field Plugin .................. SUCCESS [ 0.074 s][INFO] PDI Get Previous Row Field Plugin Core ............. SUCCESS [ 0.254 s][INFO] PDI Get Previous Row Field Assemblies .............. SUCCESS [ 0.022 s][INFO] PDI Get Previous Row Field Plugin .................. SUCCESS [ 0.088 s][INFO] PDI Aggregate Rows Plugin .......................... SUCCESS [ 0.028 s][INFO] PDI Aggregate Rows Plugin Core ..................... SUCCESS [ 1.956 s][INFO] PDI Aggregate Rows Plugin Assemblies ............... SUCCESS [ 0.024 s][INFO] PDI Aggregate Rows Plugin Distribution ............. SUCCESS [ 0.256 s][INFO] PDI Elasticsearch Bulk Insert Plugin ............... SUCCESS [ 0.022 s][INFO] PDI Elasticsearch Bulk Insert Plugin Core .......... SUCCESS [ 4.426 s][INFO] PDI Elasticsearch Bulk Insert Plugin Assemblies .... SUCCESS [ 0.052 s][INFO] PDI Elasticsearch Bulk Insert Plugin Distribution .. SUCCESS [ 5.302 s][INFO] PDI Google Analytics Plugin ........................ SUCCESS [ 0.019 s][INFO] PDI Google Analytics Plugin Core ................... SUCCESS [ 4.267 s][INFO] PDI Google Analytics Plugin Assemblies ............. SUCCESS [ 0.032 s][INFO] PDI Google Analytics Plugin Distribution ........... SUCCESS [ 3.034 s][INFO] PDI GP Bulk Loader Plugin .......................... SUCCESS [ 0.025 s][INFO] PDI GP Bulk Loader Plugin Core ..................... SUCCESS [ 2.110 s][INFO] PDI GP Bulk Loader Plugin Assemblies ............... SUCCESS [ 0.021 s][INFO] PDI GP Bulk Loader Plugin Distribution ............. SUCCESS [ 0.154 s][INFO] Pentaho Google Drive VFS ........................... SUCCESS [ 0.019 s][INFO] Pentaho Google Drive VFS Plugin Core ............... SUCCESS [ 3.310 s][INFO] Pentaho Google Drive VFS Assemblies ................ SUCCESS [ 0.027 s][INFO] Pentaho Google Drive VFS Plugin Distribution ....... SUCCESS [ 1.554 s][INFO] PDI HL7 Plugin ..................................... SUCCESS [ 0.358 s][INFO] PDI HL7 Plugin Core ................................ SUCCESS [ 0.460 s][INFO] PDI HL7 Plugin Assemblies .......................... SUCCESS [ 0.019 s][INFO] PDI HL7 Plugin Distribution ........................ SUCCESS [ 1.417 s][INFO] PDI Drools Plugin .................................. SUCCESS [ 0.019 s][INFO] PDI Drools Plugin Core ............................. SUCCESS [ 1.685 s][INFO] PDI Drools Plugin Assemblies ....................... SUCCESS [ 0.025 s][INFO] PDI Drools Plugin Distribution ..................... SUCCESS [ 3.724 s][INFO] PDI Dummy Plugin ................................... SUCCESS [ 0.020 s][INFO] PDI Dummy Plugin Core .............................. SUCCESS [ 0.982 s][INFO] PDI Dummy Plugin Assemblies ........................ SUCCESS [ 0.019 s][INFO] PDI Dummy Plugin Distribution ...................... SUCCESS [ 2.046 s][INFO] PDI Json Plugin .................................... SUCCESS [ 0.017 s][INFO] PDI Json Plugin Core ............................... SUCCESS [ 3.609 s][INFO] PDI Json Plugin Assemblies ......................... SUCCESS [ 0.018 s][INFO] PDI Json Plugin Distribution ....................... SUCCESS [ 0.090 s][INFO] PDI S3 CSV Input Plugin ............................ SUCCESS [ 0.020 s][INFO] PDI S3 CSV Input Plugin Core ....................... SUCCESS [ 4.211 s][INFO] PDI S3 CSV Input Plugin Assemblies ................. SUCCESS [ 0.018 s][INFO] PDI S3 CSV Input Plugin Distribution ............... SUCCESS [ 0.143 s][INFO] PDI SAP Plugin ..................................... SUCCESS [ 0.019 s][INFO] PDI SAP Plugin Core ................................ SUCCESS [ 1.333 s][INFO] PDI SAP Plugin Assemblies .......................... SUCCESS [ 0.026 s][INFO] PDI SAP Plugin Distribution ........................ SUCCESS [ 0.100 s][INFO] PDI Shape File Reader Plugin ....................... SUCCESS [ 0.019 s][INFO] PDI Shape File Reader Plugin Core .................. SUCCESS [ 0.324 s][INFO] PDI Shape File Reader Plugin Assemblies ............ SUCCESS [ 0.021 s][INFO] PDI Shape File Reader Plugin Distribution .......... SUCCESS [ 0.066 s][INFO] PDI GPLoad Plugin .................................. SUCCESS [ 0.163 s][INFO] PDI GPLoad Plugin Core ............................. SUCCESS [ 2.568 s][INFO] PDI GPLoad Plugin Assemblies ....................... SUCCESS [ 0.026 s][INFO] PDI GPLoad Plugin Distribution ..................... SUCCESS [ 0.069 s][INFO] PDI Openerp Plugin ................................. SUCCESS [ 0.036 s][INFO] PDI Openerp Plugin Core ............................ SUCCESS [ 0.552 s][INFO] PDI Openerp Plugin Assemblies ...................... SUCCESS [ 0.017 s][INFO] PDI Openerp Plugin Distribution .................... SUCCESS [ 0.109 s][INFO] PDI Palo Plugin .................................... SUCCESS [ 0.032 s][INFO] PDI Palo Plugin Core ............................... SUCCESS [ 3.438 s][INFO] PDI Palo Plugin Assemblies ......................... SUCCESS [ 0.035 s][INFO] PDI Palo Plugin Distribution ....................... SUCCESS [ 0.091 s][INFO] PDI Version Checker Plugin ......................... SUCCESS [ 0.017 s][INFO] PDI Version Checker Plugin Core .................... SUCCESS [ 0.141 s][INFO] PDI Version Checker Plugin Assemblies .............. SUCCESS [ 0.025 s][INFO] PDI Version Checker Plugin Distribution ............ SUCCESS [ 0.163 s][INFO] PDI Lucid DB Streaming Loader Plugin ............... SUCCESS [ 0.041 s][INFO] PDI Lucid DB Streaming Loader Plugin Core .......... SUCCESS [ 0.355 s][INFO] PDI Lucid DB Streaming Loader Plugin Assemblies .... SUCCESS [ 0.041 s][INFO] PDI Lucid DB Streaming Loader Plugin Distribution .. SUCCESS [ 0.103 s][INFO] PDI Salesforce Plugin .............................. SUCCESS ...
#Pentaho
Attachment  View in library
Need help with building pentaho-kettle from github 38 KB
Sergio Ribeiro's profile image
Sergio Ribeiro posted 05-09-2019 09:16

Hello Nikola Garafolic,

The error seems to be on the test phase so if you just want to compile, try using "-Dmaven.test.skip=true" (it'll skip testing).

Having said that, you should try to identify the test that is failing and find out why it's failing:

Please refer to /home/xx/Downloads/pentaho-kettle/plugins/pur/core/target/surefire-reports for the individual test results.Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.

It may be something on your system configuration or (less probable) some error on the code/test...

Regards,

Sérgio Ribeiro

Porto - Portugal.

Nikola Garafolic's profile image
Nikola Garafolic posted 05-09-2019 09:20

Hi, thanks for response. Will try but on github they mention skipping tests is not recommended, thats why I didnt skip them.

Sergio Ribeiro's profile image
Sergio Ribeiro posted 05-09-2019 12:26

As a rule of thumb, if there're tests, they shouldn't be ignored: they help prevent changes that will introduce errors (or behaviour differences).

 

If you're using code from the master branch, it's very difficult to have a test failing because of a code change (but not impossible). My suggestion to skip the tests was only so that you could have a functional version...

 

There're a few tests that are "very sensible" on how much resources they can use - It's not uncommon having tests (momentarily) failing from time outs, memory exhaustion or other sporadic problem...

 

As I said on my former reply, you can look into the result files to check why the test failed. If you find an error, you may report it!

 

Oh, do you use the "-U" (or "update-snapshots") option? If not, you may be using some wrong/old dependency...

 

Regards,

 

Sérgio Ribeiro

Porto - Portugal.

Larry Grill's profile image
Larry Grill posted 05-10-2019 18:01

All Pentaho maven project builds fail by default when there is a failing test. This behavior is controlled by the 'maven-surefire-plugin.testFailureIgnore' property

in the global parent pom here: https://github.com/pentaho/maven-parent-poms/blob/master/pentaho-ce-parent-pom/pentaho-ce-jar-parent-pom/pom.xml#L39

So instead of merely not running the unit tests with skipTests, you can override this behavior for all your local builds by adding this property to your settings.xml and setting it to 'true.'

Here is an example using a settings.xml profile:

<profiles>

   <profile>       <id>default</id>       <activation>          <activeByDefault>true</activeByDefault>       </activation>       <properties>          <maven-surefire-plugin.testFailureIgnore>true</maven-surefire-plugin.testFailureIgnore>       </properties>    </profile> </profiles> You can also control it on any individual command line build like this: 

`mvn clean install -Dmaven-surefire-plugin.testFailureIgnore=true`

This will run all the tests, but not fail your build if any tests fail. This can be helpful, but because maven won't aggregate all the test results at the end of your multi-project build log, it is easy to miss those failing tests. This method is recommended when you have a CI platform like Jenkins doing your build and are using a plugin to aggregate the test results.

Data Conversion's profile image
Data Conversion posted 08-15-2019 00:14
Attachment  View in library
70682.pdf 38 KB

Related Content