AnsweredAssumed Answered

DI - MongoDB Aggregation broken with MongoDB 3.6

Question asked by Daniel Polistchuck on Feb 8, 2018
Latest reply on May 18, 2018 by Roguen Keller

Any MongoDB aggregation Query using the MongoDB Input node is broken if the query runs against the (latest) 3.6 version of MongoDB.

 

The usage of cursors is now a requirement in aggregation pipelines and this is implemented in the latest 3.6 MongoDB Java drivers.

 

Here's the current stacktrace if someone tries to use the included (old, 3.2.2) Java driver:

 

2018/02/08 09:28:33 - MongoDB Input.0 - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : org.pentaho.di.core.exception.KettleException:

2018/02/08 09:28:33 - MongoDB Input.0 - com.mongodb.MongoCommandException: Command failed with error 9: 'The 'cursor' option is required, except for aggregate with the explain argument' on server localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "The 'cursor' option is required, except for aggregate with the explain argument", "code" : 9, "codeName" : "FailedToParse" }

2018/02/08 09:28:33 - MongoDB Input.0 - Command failed with error 9: 'The 'cursor' option is required, except for aggregate with the explain argument' on server localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "The 'cursor' option is required, except for aggregate with the explain argument", "code" : 9, "codeName" : "FailedToParse" }

2018/02/08 09:28:33 - MongoDB Input.0 -

2018/02/08 09:28:33 - MongoDB Input.0 - at org.pentaho.di.trans.steps.mongodbinput.MongoDbInput.processRow(MongoDbInput.java:137)

2018/02/08 09:28:33 - MongoDB Input.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)

2018/02/08 09:28:33 - MongoDB Input.0 - at java.lang.Thread.run(Thread.java:748)

2018/02/08 09:28:33 - MongoDB Input.0 - Caused by: com.mongodb.MongoCommandException: Command failed with error 9: 'The 'cursor' option is required, except for aggregate with the explain argument' on server localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "The 'cursor' option is required, except for aggregate with the explain argument", "code" : 9, "codeName" : "FailedToParse" }

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:159)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:286)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:173)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:215)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:206)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:112)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.operation.AggregateOperation$1.call(AggregateOperation.java:227)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.operation.AggregateOperation$1.call(AggregateOperation.java:223)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:239)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:212)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.operation.AggregateOperation.execute(AggregateOperation.java:223)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.operation.AggregateOperation.execute(AggregateOperation.java:65)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.Mongo.execute(Mongo.java:772)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.Mongo$2.execute(Mongo.java:759)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.DBCollection.aggregate(DBCollection.java:1377)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.DBCollection.aggregate(DBCollection.java:1308)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.DBCollection.aggregate(DBCollection.java:1294)

2018/02/08 09:28:33 - MongoDB Input.0 - at com.mongodb.DBCollection.aggregate(DBCollection.java:1282)

2018/02/08 09:28:33 - MongoDB Input.0 - at org.pentaho.mongo.wrapper.collection.DefaultMongoCollectionWrapper.aggregate(DefaultMongoCollectionWrapper.java:47)

2018/02/08 09:28:33 - MongoDB Input.0 - at org.pentaho.di.trans.steps.mongodbinput.MongoDbInput.initQuery(MongoDbInput.java:191)

2018/02/08 09:28:33 - MongoDB Input.0 - at org.pentaho.di.trans.steps.mongodbinput.MongoDbInput.processRow(MongoDbInput.java:79)

Outcomes