2

Yesterday, I tried to install opscenter and initiated import of existing DSE 5.0 cluster. It failed with certain errors which pointed out that installed version was not opscenter 6.0. root cause - opscenter installation was done from datastax community repo and not enterprise one. I corrected repo information to pointed to enterprose repo, removed existing opscenter 5.2.4, installed new opscenter 6.0 - All looks good. finally when i started the service - startup failed, opscenter fails to start and nothing comes in log files.

I did following to remove opscenter 5.x and to install opscenter 6.0

  1. Removed Old Version of ospcenter - Sudo yum remove opscenter
  2. Modified repor to point to enterprise repo e.g sudo vi /etc/yum.repos.d/datastax.repo
  3. Installed 6.0-1 version of opscenter . Sudo yum install opscenter
  4. Started Opscenter - sudo service opscenterd start

Opscenter process is not running . it starts and get killed after few seconds - i was able to verify it using top command.I went to check log files and can find only two log files startup.log & gc.log.0.current. nothing in log file opscenterd.log

startup.log
-----------
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
2016-07-06 15:00:22,763 [opscenterd] INFO: Loading OpsCenter...
2016-07-06 15:00:22,779 [opscenterd] INFO: Updating system path
2016-07-06 15:00:22,780 [opscenterd] INFO: Importing twisted logging
2016-07-06 15:00:25,710 [opscenterd] INFO: Finished importing twisted logging
2016-07-06 15:00:25,710 [opscenterd] INFO: Opscenterd starting up...

Python version on installation is 2.7.5

When I Start Opscenter in foreground mode - I can see some trace not sure if relevant

sudo ./opscenter -f 
--------------------
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
2016-07-06 15:00:22,763 [opscenterd] INFO: Loading OpsCenter...
2016-07-06 15:00:22,779 [opscenterd] INFO: Updating system path
2016-07-06 15:00:22,780 [opscenterd] INFO: Importing twisted logging
2016-07-06 15:00:25,710 [opscenterd] INFO: Finished importing twisted logging
2016-07-06 15:00:25,710 [opscenterd] INFO: Opscenterd starting up...
Traceback (most recent call last):
 File "./bin/twistd", line 63, in <module>
    from twisted.scripts.twistd import run
  File "/usr/share/opscenter/lib/py/twisted/scripts/twistd.py", line 13, in <module>
    from twisted.application import app
  File "/usr/share/opscenter/lib/py/twisted/application/app.py", line 17, in <module>
    from twisted.application import service, reactors
  File "/usr/share/opscenter/lib/py/twisted/application/service.py", line 24, in <module>
    from twisted.internet import defer
  File "/usr/share/opscenter/lib/py/twisted/internet/defer.py", line 29, in <module>
    from twisted.python import lockfile, failure
  File "/usr/share/opscenter/lib/py/twisted/python/lockfile.py", line 23, in <module>
    from os import kill
ImportError: cannot import name kill

I tried to Verify if Kill is available in installed version of Python - Looks Ok

$ python -c "import os; from os import kill;print kill"
<built-in function kill>

opscenter is not running and i cant really find anything in opscenter.log.

can this problem be there because i have removed older version and should have taken upgrade path ?

Edit - If i revert back opscenter 6.0 to 5.2.4 then it starts to work. Looks like i messed up something and need clean way to remove 5.2.4 first in order to get opscenter 6.0 work. Any help or direction is highly appreciated.

SumitSharma
  • 63
  • 6

2 Answers2

3

I have found the root cause and resolution to opscenter 6.0 startup problem.

Root Cause - /tmp folder of My RHEL machine is mounted witn noexec permissions.
----------

Logs were not providing any clue around this. I installed opscenter 6.0 on a virtual machine and its all working so it encouraged me to find the difference between two env. I zeroed down on /tmp folder is noexec and permission issues causing opscenter terminating at startup.

Solution - Setting explicitly a tmp directory for opscenter.

Set explicitly java temp directory for opscenter in JVM arguments e.g. Set -Djava.io.tmpdir=/usr/share/opscenter/tmp.

For RPM based installation
1. sudo vi /usr/share/opscenter/bin/opscenter
2. Add java temp directory parameter to JVM options
OPSC_JVM_OPTS="-server -Xmx1024m -Xms1024m -XX:MaxPermSize=128m -Dpython.cachedir.skip=false -XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled -XX:+ScavengeBeforeFullGC -XX:+CMSScavengeBeforeRemark -verbose:gc
-XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCCause
-XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5
-XX:GCLogFileSize=1M -Xloggc:$OPSC_GC_LOG_PATH/gc.log
-Djava.io.tmpdir=/usr/share/opscenter/tmp
$OPSC_JVM_OPTS"

SumitSharma
  • 63
  • 6
  • Glad you were able to get this working! Java uses the /tmp directory by default to store things like temporary garbage collection files. That directory is also used by JNA to load files, which Jython uses internally. DataStax does have documentation on this, but it was written for DSE. http://docs.datastax.com/en/datastax_enterprise/4.8//datastax_enterprise/sec/secMakingTmpNonexecutable.html. – tom.alexander Jul 07 '16 at 15:33
0

I also met the same problem. But opscenter 6.0 cannot start using the solution above.

Checking out the support platfrom list, I think it is because my os is CentOS 6.2, which is not in the support list.

So I just use version 5.2 and it works fine.

Ritchie Cai
  • 1