Profile

Stephen Donovan

Hitachi Vantara

Contact Details

Hitachi Vantara

My Content

1 to 20 of 50+ total

RE: Pentaho 9.4 BigQuery input/output?

Posted By Stephen Donovan 06-09-2023 12:02
Found In Egroup: Pentaho \ view thread
More details: We use a Simba BigQuery driver for TableInput from BIgQuery.  The Simba driver will not work well with TableOutput because there are quotas on Insert statements. It is why we have a  job entry plugin in the Enterprise Endition to bulk load BIgQuery. https://help.hitachivantara.com/ ...

RE: Pentaho 9.4 BigQuery input/output?

Posted By Stephen Donovan 06-09-2023 10:49
Found In Egroup: Pentaho \ view thread
Did you follow these steps? https://help.hitachivantara.com/Documentation/Pentaho/9.4/Setup/Google_BigQuery Can you include the error for "it fails"? ------------------------------ Stephen Donovan General IT Consulting & Implementation Pre-eminent Professional Hitachi Vantara ------------------- ...

RE: Pentaho metadata editor tool standalone - exported XMI files hold connection password in plain text

Posted By Stephen Donovan 06-06-2023 11:17
Found In Community: Pentaho \ view thread
The majority of Penthao connections can handle Penthao encrypted passwords.  https://help.hitachivantara.com/Documentation/Pentaho/9.4/Setup/Use_password_encryption_with_Pentaho Just take the output that command and use that full string (including the 'Encrpypted') as your password.  Docs don't mention ...

RE: Table Input takes too long

Posted By Stephen Donovan 06-06-2023 11:15
Found In Egroup: Pentaho \ view thread
Need a lot more detail as asked above.  What are the "Get Variables"?  Base on youyr question it seems like actually the row data from the prior transform.  Then you replace variables in the stream.  Why are you blocking and what are those settings? Are you blocking for all input rows 100k+?  That will ...

RE: Is there a way for Pentaho job to run to only get new records and not rerun the old records?

Posted By Stephen Donovan 05-04-2023 15:42
Found In Community: Pentaho \ view thread
The simple answer is, "But of course".  There are a few options.  As John stated "None of the following will work if you do not a unique key column or columns that is in common  to source and target. " Insert/Update step versus Table Output is the easiest.  However, this step does single row requests ...

RE: Access to Pentaho JIRA instance

Posted By Stephen Donovan 03-17-2023 11:01
Found In Egroup: Pentaho \ view thread
I was unaware that this site is not fully opened up yet.  After the move between technologies, the process and the site are undergoing an internal privacy review.  Sorry these things take time.  Hopefully, it will be "public" as it states very soon. ------------------------------ Stephen Donovan ...

RE: Access to Pentaho JIRA instance

Posted By Stephen Donovan 03-08-2023 10:09
Found In Egroup: Pentaho \ view thread
You are correct that you will have to reregsiter with the new JIRA.  Tickets were migrated but we did not bring over all of the users. Are you saying you cannot see anything here? https://pentaho-public.atlassian.net/jira/software/c/projects/PDI/issues ------------------------------ Stephen Donovan ...

RE: Pentaho 9.3: Truncate option in Table Output doesn't reset the Auto Increment Field

Posted By Stephen Donovan 03-01-2023 16:07
Found In Community: Pentaho \ view thread
Note the change being reverted was here. https://github.com/pentaho/pentaho-kettle/blob/9.4.0.0/core/src/main/java/org/pentaho/di/core/database/Database.java#L3303 I am not CE user or able to help you compile a fix that that code in 9.4.  But there may be members that can get you down that road if d ...

RE: Pentaho 9.3: Truncate option in Table Output doesn't reset the Auto Increment Field

Posted By Stephen Donovan 03-01-2023 16:06
Found In Community: Pentaho \ view thread
I am not seeing that.  I am on MySQL 8.0.30 with the same driver.  Make sure you don't have a driver mismatch. If you are using shared DB connections and consitent naming, the toggle of the driver and even the Type from MySQL to Generic and back should be transparent.

RE: Pentaho 9.3: Truncate option in Table Output doesn't reset the Auto Increment Field

Posted By Stephen Donovan 02-23-2023 17:41
Found In Community: Pentaho \ view thread
My initial tests were invalid.  I had the select in the same Transform as the TableOutput.  This creates a potential shared connection group which forced the DELETE due to parallel thread use. https://github.com/pentaho/pentaho-kettle/blob/8.3/core/src/main/java/org/pentaho/di/core/database/Database.java#L3325-L3336 ...

RE: Pentaho 9.3: Truncate option in Table Output doesn't reset the Auto Increment Field

Posted By Stephen Donovan 02-23-2023 14:45
Found In Community: Pentaho \ view thread
I don't have an answer, but I think I have a workaround.  In your connection, change from the MySQL dialect to the Generic Database dialect.  Specifiy the URL and driver class (likely com.mysql.cj.jdbc.Driver).  Then select MySQL in the dialect drop down at the top.  If you forget this last step, you ...

RE: Pentaho 9.3: Truncate option in Table Output doesn't reset the Auto Increment Field

Posted By Stephen Donovan 02-23-2023 14:17
Found In Community: Pentaho \ view thread
Duplicate details removed.

RE: Pentaho 9.3: Truncate option in Table Output doesn't reset the Auto Increment Field

Posted By Stephen Donovan 02-23-2023 13:00
Found In Community: Pentaho \ view thread
Not an answer yet.  I have confirmed this behavior in 7.1, 8.3.0.10 and 9.4.  With the current Mysql 8 Driver and the older version 5 org.gjt.mm.mysql.Driver.

RE: IO Error when PDI 9.4 connect AWS RDS Oracle 19C

Posted By Stephen Donovan 02-23-2023 11:33
Found In Egroup: Pentaho \ view thread
Is there a restriction that requires the use of the private key?  Most often we use local token (temporary) credentials for testing and then run within the same subnet on an ec2 instance with associated IAM Roles for other assets (like S3, Snowflake etc). Trying to bettr understand the use case beyond ...

RE: Pentaho Server CE 4.0 and SALM/Shibboleth

Posted By Stephen Donovan 02-09-2023 10:06
Found In Egroup: Pentaho \ view thread
SAML Auth is supported in the Enterprise Edition.  We do not release this with CE.  As such, I am unaware of any other community support for SAML. ------------------------------ Stephen Donovan Digital Solutions Architect Hitachi Vantara ------------------------------

RE: Parameters are not passed to sub tranformation in pentaho 9.3

Posted By Stephen Donovan 02-07-2023 13:13
Found In Community: Pentaho \ view thread
If you are running in parrallel, you may have hit a caching issue with the parameter. "MetaFileLoaderCache causes Incorrect Behavior/Results with Parallel Execution" This is fixed in 9.4 (EE and CE) and in 9.3.0.1+ (EE).

RE: How can i filter null values in an certain column so only the data that is null shows-up

Posted By Stephen Donovan 02-03-2023 12:05
Found In Egroup: Pentaho \ view thread
Column Name reflection with Javascript. https://help.hitachivantara.com/Documentation/Pentaho/8.2/Products/Data_Integration/Transformation_Step_Reference/Modified_Java_Script_Value#:~:text=Examples-,Check%20for%20the%20Existence%20of%20Fields%20in%20a%20Row,-The%20following%20code One similarly easy ...

RE: How can i filter null values in an certain column so only the data that is null shows-up

Posted By Stephen Donovan 02-03-2023 12:05
Found In Library: Pentaho

RE: How can i filter null values in an certain column so only the data that is null shows-up

Posted By Stephen Donovan 02-03-2023 11:10
Found In Egroup: Pentaho \ view thread
Can you explain why/how the Filter step does not work?  Are you trying to inject the column name(s) dynamically? ------------------------------ Stephen Donovan Digital Solutions Architect Hitachi Vantara ------------------------------

RE: PDICE 9.4, transformation Miscellaneous tab, Shared objects file field, doesn't support variable

Posted By Stephen Donovan 02-02-2023 16:28
Found In Community: Pentaho \ view thread
For CE, there are no patches.  This issue could not even be considered until 9.6.    Based on changes to metadata connections and variable context that I know are being considered, I am unsure if variables in transform metadata and toggling shared.xml would even be possible.  Toggling the dialect with ...