0

Issues with solr settings while migrating from solr 4.0 to solr6.0.

Issue Faced : My cpu consumption goes to unacceptable levels. ie. load on solr4.0 is between 6 to 10 while load on solr 6 reaches 100 and since its the production i rolled back quickly.

My Solr4 setting

 - Running on tomcat
 - JVM Memory : 16GB
 - 24 core cpu
 - JVM settings :
   - JVM Runtime Java HotSpot(TM) 64-Bit Server VM (24.45-b08) 
   - Processors   24 
   - Args : Paths mentioned here


**My Solr6 setting**

 - Running on jetty
 - JVM Memory : 20GB
 - 32 core cpu
 - JVM settings :
   - Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 1.8.0_45 25.45-b02
   - Processors   32
   - Args
      - DSTOP.KEY=solrrocks
      - DSTOP.PORT=7983
      - Djetty.home=/usr/local/solr-6.4.1/server-Djetty.port=8983
      - Dlog4j.configuration=file:/usr/local/solr-6.4.1/example/resources/log4j.properties
      - Dsolr.install.dir=/usr/local/solr-6.4.1-Dsolr.log.dir=/usr/local/solr-6.4.1/example/techproducts/solr/../logs
      - Dsolr.log.muteconsole
      - Dsolr.solr.home=/usr/local/solr-6.4.1/example/techproducts/solr-Duser.timezone=US/Eastern
      - XX:+AggressiveOpts
      - XX:+CMSParallelRemarkEnabled
      - XX:+CMSScavengeBeforeRemark
      - XX:+ParallelRefProcEnabled
      - XX:+PrintGCApplicationStoppedTime
      - XX:+PrintGCDateStamps
      - XX:+PrintGCDetails
      - XX:+PrintGCTimeStamps
      - XX:+PrintHeapAtGC
      - XX:+PrintTenuringDistribution
      - XX:+UseCMSInitiatingOccupancyOnly
      - XX:+UseConcMarkSweepGC
      - XX:+UseGCLogFileRotation
      - XX:-UseSuperWord
      - XX:CMSFullGCsBeforeCompaction=1
      - XX:CMSInitiatingOccupancyFraction=70
      - XX:CMSMaxAbortablePrecleanTime=6000
      - XX:CMSTriggerPermRatio=80
      - XX:GCLogFileSize=20M
      - XX:MaxTenuringThreshold=8
      - XX:NewRatio=2
      - XX:NumberOfGCLogFiles=9
      - XX:OnOutOfMemoryError=/usr/local/solr-6.4.1/bin/oom_solr.sh 8983 /usr/local/solr-6.4.1/example/techproducts/solr/../logs
      - XX:PretenureSizeThreshold=64m
      - XX:SurvivorRatio=15
      - XX:TargetSurvivorRatio=90-Xloggc:/usr/local/solr-6.4.1/example/techproducts/solr/../logs/solr_gc.log-Xms21g-Xmx21g-Xss256k-verbose:gc

What i looking for

  1. My guess its related to gc setting of jetty as i am not expert in jetty(java8).please help how to tune these settings. Also how should i chosoe these values or how to to debug these issue ?

1 Answers1

0

if you are using Solr's default jvm settings, I doubt it is due to what you mention. You are probably hitting this regression that was introduced in 6.4.1 by new metrics functionality.

It affects queries that match many terms, so stuff like prefix queries etc. You can:

  1. wait for a 6.4.2 that should be out soon
  2. apply the patch in the jira issue, that has solved the issue for people who tired it. You can do this right now

EDIT after your comment: if solr6.1.0 is equally bad, then the issue above is not the culprit. You should look through all your schema/solrconfig settings and making sure they make sense. 6.4 should be at the very least on par with Solr4.0, being more performant in most cases. We have no info on what are your queries like, how does your index look like etc, to give you more detailed insights.

Persimmonium
  • 15,593
  • 11
  • 47
  • 78