Apache Beam - Exception caught and throwed even though program excuting continuosly. How to stop that process or handle in pipeline

Question

i have a pipeline which fetch data mysql and used to transfer data to mongo db after running this pipeline with the below code, data fetched from mysql but unable to load to mongodb

noSqlresult.apply(MongoDbIO.write().withUri(mongoUri)
                .withDatabase(mongoDatabase)
                .withCollection(resultCollectionName));

i have found the following exception and some logs that continuously trying to communicate with mongo db

com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=null, userName='mongoUser', source='db1', password=<hidden>, mechanismProperties={}}
at com.mongodb.connection.SaslAuthenticator.wrapException(SaslAuthenticator.java:162)
at com.mongodb.connection.SaslAuthenticator.access$200(SaslAuthenticator.java:39)
at com.mongodb.connection.SaslAuthenticator$1.run(SaslAuthenticator.java:68)
at com.mongodb.connection.SaslAuthenticator$1.run(SaslAuthenticator.java:46)
at com.mongodb.connection.SaslAuthenticator.doAsSubject(SaslAuthenticator.java:168)
at com.mongodb.connection.SaslAuthenticator.authenticate(SaslAuthenticator.java:46)
at com.mongodb.connection.DefaultAuthenticator.authenticate(DefaultAuthenticator.java:32)
at com.mongodb.connection.InternalStreamConnectionInitializer.authenticateAll(InternalStreamConnectionInitializer.java:122)
at com.mongodb.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:52)
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:127)
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:114)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.mongodb.MongoCommandException: Command failed with error 18: 'Authentication failed.' on server severip:27017. The full response is { "ok" : 0.0, "errmsg" : "Authentication failed.", "code" : 18, "codeName" : "AuthenticationFailed" }
    at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:164)
    at com.mongodb.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:295)
    at com.mongodb.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:255)
    at com.mongodb.connection.CommandHelper.sendAndReceive(CommandHelper.java:84)
    at com.mongodb.connection.CommandHelper.executeCommand(CommandHelper.java:34)
    at com.mongodb.connection.SaslAuthenticator.sendSaslStart(SaslAuthenticator.java:119)
    at com.mongodb.connection.SaslAuthenticator.access$000(SaslAuthenticator.java:39)
    at com.mongodb.connection.SaslAuthenticator$1.run(SaslAuthenticator.java:52)
    ... 9 more
18/11/09 12:49:29 DEBUG org.mongodb.driver.cluster: Updating cluster description to  {type=UNKNOWN, servers=[{address=severip:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=null, userName='mongoUser', source='db1', password=<hidden>, mechanismProperties={}}}, caused by {com.mongodb.MongoCommandException: Command failed with error 18: 'Authentication failed.' on server severip:27017. The full response is { "ok" : 0.0, "errmsg" : "Authentication failed.", "code" : 18, "codeName" : "AuthenticationFailed" }}}]
18/11/09 12:49:29 DEBUG org.mongodb.driver.connection: Closing connection connectionId{localValue:17}
18/11/09 12:49:29 DEBUG org.mongodb.driver.cluster: Updating cluster description to  {type=UNKNOWN, servers=[{address=severip:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=null, userName='mongoUser', source='db1', password=<hidden>, mechanismProperties={}}}, caused by {com.mongodb.MongoCommandException: Command failed with error 18: 'Authentication failed.' on server severip:27017. The full response is { "ok" : 0.0, "errmsg" : "Authentication failed.", "code" : 18, "codeName" : "AuthenticationFailed" }}}]
18/11/09 12:49:29 DEBUG org.mongodb.driver.connection: Closing connection connectionId{localValue:18}

how to handle such cases mongo has ability to create db if not exists but not creating internally mongoIO using mongo client to connect. Is there any possiblity to handle

The error showing it is due to attentication issue but exception is not getting for existing databases only getting new databases what is the exact reason and how to handle those errors

there's not enough information in the question. For example it is unclear how authentication is configured in your MongoDB instance and what auth information are you passing to MongoIO — Anton, Nov 09 '18 at 21:08
there is no authenication issue but getting mongo auth exception actually i'm trying to write my output in new db which is not exists. i have tried for existing db it accepted there is no issue. @Anton — jithu, Nov 12 '18 at 05:00
ok, now i got your question @Anton, mongoUri contains username and password mongoUri = mongodb://mongoUser:*******@severip:27017/db1 and it is a valid uri but it continously throwing mongo exception with error code:18 — jithu, Nov 12 '18 at 05:07

score 0 · Accepted Answer · answered Nov 26 '18 at 09:00

0

There is no object to store the result of ptransform so need to wait for connection timed-out exception, after that you can perform operations which you declare in catch block if declared other connections automatically closed.

if you are new to handle exception, i think you will get answer from the following link

java_exception_handler_StackOverFlow

answered Nov 26 '18 at 09:00

Talasila Anusha

31
1

yes, got it. Since mongo driver class was implemented like this so that it retry many times to establish connection. fault was _no permission to create a new db_ – jithu Nov 26 '18 at 09:06

Apache Beam - Exception caught and throwed even though program excuting continuosly. How to stop that process or handle in pipeline

1 Answers1