0

I have been experimenting with Flume lately. I am currently using the JMS Source (http://flume.apache.org/FlumeUserGuide.html#jms-source)

However, when I try to run the flume agent, it succeeds in creating the channel and sink, but fails while creating the Source. I am using TIBCO EMS and I need to authenticate using the EMS username/password combination.

The documentation asks me to define a passwordFile, which contains the password for the destination/provider. Is there a proper way to create this file (I currently have the password stored in a text file).

Here is the error stack (I am pretty sure user/password is correct)

13/11/17 11:49:08 INFO source.DefaultSourceFactory: Creating instance of source WeatherData, type jms
13/11/17 11:49:08 ERROR node.AbstractConfigurationProvider: Source WeatherData has been removed due to an error during configuration
org.apache.flume.FlumeException: Could not lookup ConnectionFactory
        at org.apache.flume.source.jms.JMSSource.doConfigure(JMSSource.java:222)
        at org.apache.flume.source.BasicSourceSemantics.configure(BasicSourceSemantics.java:65)
        at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
        at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:331)
        at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
        at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: javax.naming.AuthenticationException: Not permitted: invalid name or password [Root exception is javax.jms.JMSSecurityException: invalid name or password]
        at com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:668)
        at com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:489)
        at javax.naming.InitialContext.lookup(InitialContext.java:392)
        at org.apache.flume.source.jms.JMSSource.doConfigure(JMSSource.java:219)
        ... 14 more
Caused by: javax.jms.JMSSecurityException: invalid name or password
        at com.tibco.tibjms.Tibjmsx.buildException(Tibjmsx.java:543)
        at com.tibco.tibjms.TibjmsConnection._create(TibjmsConnection.java:1044)
        at com.tibco.tibjms.TibjmsConnection.<init>(TibjmsConnection.java:2707)
        at com.tibco.tibjms.TibjmsQueueConnection.<init>(TibjmsQueueConnection.java:36)
        at com.tibco.tibjms.TibjmsxCFImpl._createImpl(TibjmsxCFImpl.java:186)
        at com.tibco.tibjms.TibjmsxCFImpl._createConnection(TibjmsxCFImpl.java:239)
        at com.tibco.tibjms.TibjmsQueueConnectionFactory.createQueueConnection(TibjmsQueueConnectionFactory.java:87)
        at com.tibco.tibjms.naming.TibjmsContext$Messenger.request(TibjmsContext.java:325)
        at com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:655)
        ... 17 more
user1771840
  • 35
  • 1
  • 7

3 Answers3

0

About the passwordFile not sure, that seems to be Flume specific. TIBCO EMS authenticates on 2 levels:

  • lookup of connectionFactory
  • lookup of destination

Some "prebuild" configurations like spring etc (and might be flume) struggle with this and need some rework.

Check the code below where you can see user/pw are applied on twice levels to obtain the Queue queue.sample from TIBCO EMS: 

/**
 * @param args
 */
public static void main(String[] args) {

    String userName = "user";
    String password = "password";
    try {
        // Obtain a JNDI connection
        Properties env = new Properties();
        env.put(Context.INITIAL_CONTEXT_FACTORY,
                "com.tibco.tibjms.naming.TibjmsInitialContextFactory");
        env.put(Context.PROVIDER_URL, "tibjmsnaming://localhost:7222");
        env.put(Context.SECURITY_PRINCIPAL, userName);
        env.put(Context.SECURITY_CREDENTIALS, password);
        // ... specify the JNDI properties specific to the vendor
        InitialContext jndi = new InitialContext(env);

        ConnectionFactory factory = (ConnectionFactory) jndi
                .lookup("ConnectionFactory");

        try {
            Connection connection = factory.createConnection(userName,
                    password);
            Session session = connection.createSession();

            Queue sampleTopic = (Queue) jndi.lookup("queue.sample");

        } catch (JMSException je) {
            je.printStackTrace();

        }

    } catch (NamingException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

}

Seb
  • 684
  • 4
  • 11
0

Also make sure that your password file is encoded in UTF-8 if it contains special characters like accents. Flume reads the file using this charset.

Nicolas Heitz
  • 664
  • 5
  • 10
0

There is a defect in the Flume source that does not pass credentials properly to the JNDI server. FLUME-2095 addresses this issue and a patch has been proposed. There is likely nothing wrong with they way you are specifying the username/password, since without this fix it will never work.

The only workaround I am aware of is to turn off access control at the JMS level, which removes the need for credentials entirely, but leaves your queue/topic wide open.

mcasbon
  • 1
  • 1
  • 1