3

I am trying to launch Accumulo, and I am running into a nasty exception that seems to be tied to some network problem with my Virtual Machine. I thought the command

sudo -u accumulo /usr/lib/accumulo/bin/accumulo init --instance-name lumify --password password

would initialize accumulo. However, I get a warning that an address change has been detected, and then a fatal exception. I noticed that I don't have $ACCUMULO_HOME defined in the accumulo-env.sh file. I also noticed that the only "masters" and "slaves" in the masters and slaves files are a single line that says localhost, and nothing with my IP address there. I changed localhost to my IP address in those files, and the same issue persisted.

I'm fairly sure that it's an IP address issue, but I followed the instructions closely. Perhaps it's a problem because I'm on a virtual machine (even though I'm using the bridged adapter and got a real IP address for my virtual box)?

I check my eth0 connection using ifconfig and it still shows 192.168.1.232, while I still get the messages

Hadoop Filesystem is hdfs://192.168.1.232:8020

Accumulo data dir is /accumulo

Zookeeper server is localhost:2181

and then the warning

Address change detected. Old: localhost.localdomain/192.168.1.232:8020 New: localhost.localdomain/127.0.0.1:8020

and the exception

Failed to check if filesystem is already initialized.

It seems like even though there is no different network, my VM is the same IP address that it was when I ran the setup, I get this brick in the middle of initializing accumulo.

Any suggestions? The stack trace continues with a

ConnectException: Call from localhost.localdomain/127.0.0.1 to localhost.localdomain:8020 failed on connection exception: Connection refused

Here's the stack trace if it helps:

[root@localhost conf]# sudo -u accumulo /usr/lib/accumulo/bin/accumulo init --instance-name lumify --password password
2014-08-20 17:49:35,488 [util.Initialize] INFO : Hadoop Filesystem is hdfs://192.168.1.232:8020
2014-08-20 17:49:35,490 [util.Initialize] INFO : Accumulo data dir is /accumulo
2014-08-20 17:49:35,490 [util.Initialize] INFO : Zookeeper server is localhost:2181
2014-08-20 17:49:35,490 [util.Initialize] INFO : Checking if Zookeeper is available. If this hangs, then you need to make sure zookeeper is running


Warning!!! Your instance secret is still set to the default, this is not secure. We highly recommend you change it.


You can change the instance secret in accumulo by using:
   bin/accumulo org.apache.accumulo.server.util.ChangeSecret oldPassword newPassword.
You will also need to edit your secret in your configuration file by adding the property instance.secret to your conf/accumulo-site.xml. Without this accumulo will not operate correctly
2014-08-20 17:49:35,915 [ipc.Client] WARN : Address change detected. Old: localhost.localdomain/192.168.1.232:8020 New: localhost.localdomain/127.0.0.1:8020
2014-08-20 17:49:35,923 [util.Initialize] FATAL: java.io.IOException: Failed to check if filesystem already initialized
java.io.IOException: Failed to check if filesystem already initialized
    at org.apache.accumulo.server.util.Initialize.checkInit(Initialize.java:178)
    at org.apache.accumulo.server.util.Initialize.doInit(Initialize.java:185)
    at org.apache.accumulo.server.util.Initialize.main(Initialize.java:545)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.accumulo.start.Main$1.run(Main.java:103)
    at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.ConnectException: Call From localhost.localdomain/127.0.0.1 to localhost.localdomain:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:782)
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:729)
    at org.apache.hadoop.ipc.Client.call(Client.java:1242)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
    at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
    at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:629)
    at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1545)
    at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:820)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1380)
    at org.apache.accumulo.server.util.Initialize.isInitialized(Initialize.java:512)
    at org.apache.accumulo.server.util.Initialize.checkInit(Initialize.java:163)
    ... 8 more
Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:207)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:528)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:492)
    at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:510)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:604)
    at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:252)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1291)
    at org.apache.hadoop.ipc.Client.call(Client.java:1209)
    ... 23 more
Thread "init" died java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.accumulo.start.Main$1.run(Main.java:103)
    at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: java.io.IOException: Failed to check if filesystem already initialized
    at org.apache.accumulo.server.util.Initialize.main(Initialize.java:549)
    ... 6 more
Caused by: java.io.IOException: Failed to check if filesystem already initialized
    at org.apache.accumulo.server.util.Initialize.checkInit(Initialize.java:178)
    at org.apache.accumulo.server.util.Initialize.doInit(Initialize.java:185)
    at org.apache.accumulo.server.util.Initialize.main(Initialize.java:545)
    ... 6 more
Caused by: java.net.ConnectException: Call From localhost.localdomain/127.0.0.1 to localhost.localdomain:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:782)
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:729)
    at org.apache.hadoop.ipc.Client.call(Client.java:1242)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
    at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
    at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:629)
    at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1545)
    at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:820)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1380)
    at org.apache.accumulo.server.util.Initialize.isInitialized(Initialize.java:512)
    at org.apache.accumulo.server.util.Initialize.checkInit(Initialize.java:163)
    ... 8 more
Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:207)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:528)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:492)
    at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:510)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:604)
    at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:252)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1291)
    at org.apache.hadoop.ipc.Client.call(Client.java:1209)
    ... 23 more
Community
  • 1
  • 1
Ben Friedman
  • 55
  • 4
  • Are you able to execute hadoop fs commands from your VM as the user that your accumulo processes will be running as? – MikeD Aug 20 '14 at 22:23
  • Have you tried using the DNS name for your HDFS instance instead of the IP? Perhaps adding a line to /etc/hosts for that IP address? – Christopher Aug 21 '14 at 02:19
  • The DNS name for my HDFS instance? How would I find that? I believe Hadoop is running on the same VM, not sure that I understand what to add to /etc/hosts. And the user that accumulo processes will be running as, I thought it would all be under the same user, but I do remember a part of the instructions I'm following, linked here: [link](https://github.com/lumifyio/lumify/blob/master/docs/setup-centos-6.4.md) making a new user called accumulo. I'll try running the accumulo init command as that user? – Ben Friedman Aug 21 '14 at 16:44
  • Looks like HDFS isn't actually running. Did you start it, visit the HDFS monitor page (typically http://localhost:50070), and validate that you can interact with it (like MikeD said)? – elserj Aug 21 '14 at 21:04
  • I was able to go to localhost:50070 and see that the namenode is up and running. It says NameNode 'localhost:8020' (active) and gives statistics about the node, so it's definitely there. What commands can I throw out there to see that I can properly interact with it? – Ben Friedman Aug 22 '14 at 14:57

1 Answers1

1

You should verify that your DNS and /etc/hosts settings are configured correctly and are consistent with your masters and slaves files.

Boo Radley
  • 684
  • 1
  • 13
  • 25