Pentaho

 View Only
  • 1.  In Pentaho ,table output throwing an error when checked the return auto generated key and using the connection as snowflake.

    Posted 03-22-2023 02:56
      |   view attached

    Hi Everyone,

     

    Pentaho ETL (Community edition) + Snowflake :

     

    I am facing an issue while using the return auto generated key (checked) which exist in table output step in Pentaho ETL when connection made to snowflake db.

     

    Please find the screenshot and log.

     

    Log:

    2023/03/22 01:58:08 - Table output 3.0 - ERROR (version 9.3.0.0-428, build 9.3.0.0-428 from 2022-04-12 04.56.25 by buildguy) : Because of an error, this step can't continue:

    2023/03/22 01:58:08 - Table output 3.0 - ERROR (version 9.3.0.0-428, build 9.3.0.0-428 from 2022-04-12 04.56.25 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseException: 

    2023/03/22 01:58:08 - Table output 3.0 - Couldn't prepare statement:

    2023/03/22 01:58:08 - Table output 3.0 - INSERT INTO PUBSDATA_LOAD.ADDR_CLONE (ADDR_LINE1_ADDR, ADDR_LINE2_ADDR, ADDR_LINE3_ADDR, ADDR_LINE4_ADDR, ADDR_DESC, ADDR_TYP_CD, CITY_KEY, TERRITORY_KEY, COUNTRY_KEY, POSTAL_CD_KEY, SOURCE_CITY_TXT, SOURCE_TERRITORY_TXT, SOURCE_COUNTRY_TXT, SOURCE_POSTAL_CD_TXT, SOURCE_SYS_CD, DELETED_IND, CREATED_DTM, CREATED_BY_USER_ID, UPDATED_DTM, UPDATED_BY_USER_ID, SOURCE_SYS_ID, SOURCE_ORG_NM, SOURCE_DEPARTMENT_NM) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

    2023/03/22 01:58:08 - Table output 3.0 - at java.lang.Thread.run (null:-1)

    2023/03/22 01:58:08 - Table output 3.0 - at org.pentaho.di.trans.step.RunThread.run (RunThread.java:62)

    2023/03/22 01:58:08 - Table output 3.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.processRow (TableOutput.java:125)

    2023/03/22 01:58:08 - Table output 3.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.writeToTable (TableOutput.java:250)

    2023/03/22 01:58:08 - Table output 3.0 - at org.pentaho.di.core.database.Database.prepareSQL (Database.java:988)

    2023/03/22 01:58:08 - Table output 3.0 - at net.snowflake.client.jdbc.SnowflakeConnectionV1.prepareStatement (SnowflakeConnectionV1.java:463)

    2023/03/22 01:58:08 - Table output 3.0 - 

    2023/03/22 01:58:08 - Table output 3.0 - at org.pentaho.di.core.database.Database.prepareSQL(Database.java:993)

    2023/03/22 01:58:08 - Table output 3.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.writeToTable(TableOutput.java:250)

    2023/03/22 01:58:08 - Table output 3.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.processRow(TableOutput.java:125)

    2023/03/22 01:58:08 - Table output 3.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)

    2023/03/22 01:58:08 - Table output 3.0 - at java.lang.Thread.run(Unknown Source)

    2023/03/22 01:58:08 - Table output 3.0 - Caused by: net.snowflake.client.jdbc.SnowflakeLoggedFeatureNotSupportedException

    2023/03/22 01:58:08 - Table output 3.0 - at net.snowflake.client.jdbc.SnowflakeConnectionV1.prepareStatement(SnowflakeConnectionV1.java:463)

    2023/03/22 01:58:08 - Table output 3.0 - at org.pentaho.di.core.database.Database.prepareSQL(Database.java:988)

    2023/03/22 01:58:08 - Table output 3.0 - ... 4 more

    2023/03/22 01:58:09 - load_snowflake - ERROR (version 9.3.0.0-428, build 9.3.0.0-428 from 2022-04-12 04.56.25 by buildguy) : Errors detected!

    2023/03/22 01:58:09 - load_snowflake - ERROR (version 9.3.0.0-428, build 9.3.0.0-428 from 2022-04-12 04.56.25 by buildguy) : Errors detected!

    2023/03/22 01:58:09 - load_snowflake - ERROR (version 9.3.0.0-428, build 9.3.0.0-428 from 2022-04-12 04.56.25 by buildguy) : Errors detected!

    Any input highly appreciated !!!

    Regards

    Ashutosh



    ------------------------------
    Ashutosh Singh
    IT Support Manager
    Newgen
    ------------------------------


  • 2.  RE: In Pentaho ,table output throwing an error when checked the return auto generated key and using the connection as snowflake.

    Posted 03-22-2023 06:51

    Hi Ashutosh,

    2023/03/22 01:58:08 - Table output 3.0 - Caused by: net.snowflake.client.jdbc.SnowflakeLoggedFeatureNotSupportedException
    2023/03/22 01:58:08 - Table output 3.0 - at net.snowflake.client.jdbc.SnowflakeConnectionV1.prepareStatement(SnowflakeConnectionV1.java:463)
    2023/03/22 01:58:08 - Table output 3.0 - at org.pentaho.di.core.database.Database.prepareSQL(Database.java:988)

    IMHO this exception saying that auto-generated key is not supported by this JDBC driver. You can try newer version (replace snowflake-jdbc.jar in lib directory)



    ------------------------------
    Petr Prochazka
    Systems Engineer
    P.V.A. systems s.r.o.
    ------------------------------



  • 3.  RE: In Pentaho ,table output throwing an error when checked the return auto generated key and using the connection as snowflake.

    Posted 03-23-2023 04:11

    Thank you Petr for the suggestion .

    Currently I am using the version snowflake-jdbc-3.13.10 . And tried set of multiple newer snowflake jar version but it not worked out .

    Any clue which version I have to prefer will be great ?



    ------------------------------
    Ashutosh Singh
    IT Support Manager
    Newgen
    ------------------------------



  • 4.  RE: In Pentaho ,table output throwing an error when checked the return auto generated key and using the connection as snowflake.

    Posted 03-23-2023 05:28

    Sorry, I don't use snowflake. In PDI CE package is delivered v3.13.10 (I checked v9.4).

    I checked lastest sources (v3.13.29) and generated keys is not supported, see GitHub.



    ------------------------------
    Petr Prochazka
    Systems Engineer
    P.V.A. systems s.r.o.
    ------------------------------



  • 5.  RE: In Pentaho ,table output throwing an error when checked the return auto generated key and using the connection as snowflake.

    Posted 03-23-2023 07:16

    Thank you Petr .

    I appreciate the effort you made it.

    Regards

    Ashutosh



    ------------------------------
    Ashutosh Singh
    IT Support Manager
    Newgen
    ------------------------------