AnsweredAssumed Answered

Kettle Beam on Dialogflow

Question asked by Virgilio Pierini on Jun 12, 2019

Hi,

has anybody experienced this error when trying to run a transformation on beam/dialogflow?

 

java.lang.RuntimeException: Failed to create a workflow job: (b7b4bb91eba42106): The workflow could not be created. Causes: (b7b4bb91eba42677): SDK pipeline options or staging file list exceeds size limit. Please keep their length under 256K Bytes each and 512K Bytes in total. For example, shorten pipeline options or staging file paths, or reduce the number of staging files by building a Java uber JAR.
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:869)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:178)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
at org.kettle.beam.pipeline.KettleBeamPipelineExecutor.execute(KettleBeamPipelineExecutor.java:76)
at org.kettle.beam.xp.RunBeamTransExecutionPoint$1.run(RunBeamTransExecutionPoint.java:149)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "(b7b4bb91eba42106): The workflow could not be created. Causes: (b7b4bb91eba42677): SDK pipeline options or staging file list exceeds size limit. Please keep their length under 256K Bytes each and 512K Bytes in total. For example, shorten pipeline options or staging file paths, or reduce the number of staging files by building a Java uber JAR.",
"reason" : "badRequest"
} ],
"message" : "(b7b4bb91eba42106): The workflow could not be created. Causes: (b7b4bb91eba42677): SDK pipeline options or staging file list exceeds size limit. Please keep their length under 256K Bytes each and 512K Bytes in total. For example, shorten pipeline options or staging file paths, or reduce the number of staging files by building a Java uber JAR.",
"status" : "INVALID_ARGUMENT"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1067)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.runners.dataflow.DataflowClient.createJob(DataflowClient.java:61)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:855)
... 6 more

 

Regards

Virgilio

Outcomes