Nexus keep creating deleted file descriptors in /tmp directory makes the system run out of memory

Question

I've installed Nexus repository 3 on Ubuntu 18.04 LTS and it's working find excepted it keep spawning processes with Huge file descriptors which are jar_cache packages. Almost the memories are consuming by Nexus user.

lsof | grep deleted

java      25189                   nexus  274r      REG                8,1    168119    2586774 /data/nexus/tmp/jar_cache7208625739217484334.tmp (deleted)
java      25189                   nexus  275r      REG                8,1    212164    2586775 /data/nexus/tmp/jar_cache462451415006799037.tmp (deleted)
java      25189                   nexus  382r      REG                8,1   1061044    2587366 /data/nexus/tmp/jar_cache6165149551453976210.tmp (deleted)
java      25189                   nexus  383r      REG                8,1    148176    2587367 /data/nexus/tmp/jar_cache751007135540811314.tmp (deleted)
quartz-7- 25189   686             nexus  274r      REG                8,1    168119    2586774 /data/nexus/tmp/jar_cache7208625739217484334.tmp (deleted)
quartz-7- 25189   686             nexus  275r      REG                8,1    212164    2586775 /data/nexus/tmp/jar_cache462451415006799037.tmp (deleted)
quartz-7- 25189   686             nexus  382r      REG                8,1   1061044    2587366 /data/nexus/tmp/jar_cache6165149551453976210.tmp (deleted)
elasticse 25189 25360             nexus  383r      REG                8,1    148176    2587367 /data/nexus/tmp/jar_cache751007135540811314.tmp (deleted)
 elasticse 25189 25360             nexus  384r      REG                8,1   1012761    2587368 /data/nexus/tmp/jar_cache3471292813592489580.tmp (deleted)
elasticse 25189 25360             nexus  385r      REG                8,1    592237    2587369 /data/nexus/tmp/jar_cache601982945780052716.tmp (deleted)
elasticse 25189 25360             nexus  386r      REG                8,1    565410    2587370 /data/nexus/tmp/jar_cache2853120646898296069.tmp (deleted)
elasticse 25189 25360             nexus  387r      REG                8,1     27585    2587371 /data/nexus/tmp/jar_cache6330732713641440746.tmp (deleted)
elasticse 25189 25362             nexus  386r      REG                8,1    565410    2587370 /data/nexus/tmp/jar_cache2853120646898296069.tmp (deleted)
blobstore 25189 25373             nexus  274r      REG                8,1    168119    2586774 /data/nexus/tmp/jar_cache7208625739217484334.tmp (deleted)
blobstore 25189 25373             nexus  275r      REG                8,1    212164    2586775 /data/nexus/tmp/jar_cache462451415006799037.tmp (deleted)
blobstore 25189 25373             nexus  382r      REG                8,1   1061044    2587366 /data/nexus/tmp/jar_cache6165149551453976210.tmp (deleted)
blobstore 25189 25373             nexus  383r      REG                8,1    148176    2587367 /data/nexus/tmp/jar_cache751007135540811314.tmp (deleted)
blobstore 25189 25373             nexus  384r      REG                8,1   1012761    2587368 /data/nexus/tmp/jar_cache3471292813592489580.tmp (deleted)
blobstore 25189 25373             nexus  385r      REG                8,1    592237    2587369 /data/nexus/tmp/jar_cache601982945780052716.tmp (deleted)
blobstore 25189 25373             nexus  386r      REG                8,1    565410    2587370 /data/nexus/tmp/jar_cache2853120646898296069.tmp (deleted)
nexus_Qua 25189 25375             nexus  274r      REG                8,1    168119    2586774 /data/nexus/tmp/jar_cache7208625739217484334.tmp (deleted)
nexus_Qua 25189 25375             nexus  275r      REG                8,1    212164    2586775 /data/nexus/tmp/jar_cache462451415006799037.tmp (deleted)
nexus_Qua 25189 25375             nexus  382r      REG                8,1   1061044    2587366 /data/nexus/tmp/jar_cache6165149551453976210.tmp (deleted)
nexus_Qua 25189 25375             nexus  383r      REG                8,1    148176    2587367 /data/nexus/tmp/jar_cache751007135540811314.tmp (deleted)
nexus_Qua 25189 25375             nexus  384r      REG                8,1   1012761    2587368 /data/nexus/tmp/jar_cache3471292813592489580.tmp (deleted)
nexus_Qua 25189 25375             nexus  385r      REG                8,1    592237    2587369 /data/nexus/tmp/jar_cache601982945780052716.tmp (deleted)
status-de 25189 25376             nexus  383r      REG                8,1    148176    2587367 /data/nexus/tmp/jar_cache751007135540811314.tmp (deleted)
status-de 25189 25376             nexus  384r      REG                8,1   1012761    2587368 /data/nexus/tmp/jar_cache3471292813592489580.tmp (deleted)
status-de 25189 25376             nexus  385r      REG                8,1    592237    2587369 /data/nexus/tmp/jar_cache601982945780052716.tmp (deleted)
quartz-7- 25189 25384             nexus  274r      REG                8,1    168119    2586774 /data/nexus/tmp/jar_cache7208625739217484334.tmp (deleted)
quartz-7- 25189 25384             nexus  275r      REG                8,1    212164    2586775 /data/nexus/tmp/jar_cache462451415006799037.tmp (deleted)
quartz-7- 25189 25384             nexus  382r      REG                8,1   1061044    2587366 /data/nexus/tmp/jar_cache6165149551453976210.tmp (deleted)
task-logg 25189 25385             nexus  274r      REG                8,1    168119    2586774 /data/nexus/tmp/jar_cache7208625739217484334.tmp (deleted)
task-logg 25189 25385             nexus  275r      REG                8,1    212164    2586775 /data/nexus/tmp/jar_cache462451415006799037.tmp (deleted)
task-logg 25189 25385             nexus  382r      REG                8,1   1061044    2587366 /data/nexus/tmp/jar_cache6165149551453976210.tmp (deleted)
search-se 25189 25386             nexus  382r      REG                8,1   1061044    2587366 /data/nexus/tmp/jar_cache6165149551453976210.tmp (deleted)
search-se 25189 25386             nexus  383r      REG                8,1    148176    2587367 /data/nexus/tmp/jar_cache751007135540811314.tmp (deleted)
search-se 25189 25386             nexus  384r      REG                8,1   1012761    2587368 /data/nexus/tmp/jar_cache3471292813592489580.tmp (deleted)

The configuration in nexus.vmoptions

-Xms2703m
-Xmx2703m
-XX:MaxDirectMemorySize=2703m
-XX:+UnlockDiagnosticVMOptions
-XX:+LogVMOutput
#-XX:LogFile=../sonatype-work/nexus3/log/jvm.log
-XX:-OmitStackTraceInFastThrow
-Djava.net.preferIPv4Stack=true
-Dkaraf.home=.
-Dkaraf.base=.
-Dkaraf.etc=etc/karaf
-Djava.util.logging.config.file=etc/karaf/java.util.logging.properties
#-Dkaraf.data=../sonatype-work/nexus3
#-Dkaraf.log=../sonatype-work/nexus3/log
#-Djava.io.tmpdir=../sonatype-work/nexus3/tmp
-Dkaraf.startLocalConsole=false

-Dkaraf.data=/data/nexus/nexus-data
-Djava.io.tmpdir=/data/nexus/tmp
-XX:LogFile=/var/log/nexus/jvm.log
-Dkaraf.log=/var/log/nexus/log

#
# additional vmoptions needed for Java9+
#
# --add-reads=java.xml=java.logging
# --add-exports=java.base/org.apache.karaf.specs.locator=java.xml,ALL-UNNAMED
# --patch-module=java.base=lib/endorsed/org.apache.karaf.specs.locator-4.2.9.jar
# --patch-module=java.xml=lib/endorsed/org.apache.karaf.specs.java.xml-4.2.9.jar
# --add-opens=java.base/java.security=ALL-UNNAMED
# --add-opens=java.base/java.net=ALL-UNNAMED
# --add-opens=java.base/java.lang=ALL-UNNAMED
# --add-opens=java.base/java.util=ALL-UNNAMED
# --add-opens=java.naming/javax.naming.spi=ALL-UNNAMED
# --add-opens=java.rmi/sun.rmi.transport.tcp=ALL-UNNAMED
# --add-exports=java.base/sun.net.www.protocol.http=ALL-UNNAMED
# --add-exports=java.base/sun.net.www.protocol.https=ALL-UNNAMED
# --add-exports=java.base/sun.net.www.protocol.jar=ALL-UNNAMED
# --add-exports=jdk.xml.dom/org.w3c.dom.html=ALL-UNNAMED
# --add-exports=jdk.naming.rmi/com.sun.jndi.url.rmi=ALL-UNNAMED
#
# comment out this vmoption when using Java9+
#
-Djava.endorsed.dirs=lib/endorsed

I have no idea why the Nexus produces that kind of processes. I've tried to kill -9 25189 but it always re-creates itself.

Also tried with delete /tmp directory, change /tmp location gave the same result.

Are you on the latest NXRM3? This sounds familiar but I think it may have been fixed. — joedragons, Sep 09 '20 at 22:11

Nexus keep creating deleted file descriptors in /tmp directory makes the system run out of memory

0 Answers0