Last week I got to try the Pentaho 8 Data Integration with Spark. I did a fresh install of Hadoop/Yarn and Spark with Ubuntu running in a VM and started trying to run in Local mode. Everything went fine, I ran a simple transformation over some CSV data and a math operation and got the results. Next step: Run it in Yarn mode, on the pseudo-distributed cluster.
I configured the entire environment, pdi-spark-executor inside the hdfs, got the daemon running properly and proceeded to run the same transformation using this new Run Configuration I set up. The daemon receives the request but throws a Unexpected Error.
Attached here are two images: My Spoon execution log and the daemon log from the console.
Following the stack trace I opened the SparkRunner class file to check this "execute" method, and there's a bunch of code inside a try/catch block and any exceptions are rethrowed as this "UnexpectedError" exception, so I have no clue about what really happened.
Could someone shed me a light on this matter, please?