Sunspot-solr is eating up too much memory? What's a convenient way to reduce memory usage?

Question

So it seems Sunspot-Solr is eating a lot of memory up. And its probably due to my malpractice as a programmer allocating proper usage of it.

First of all, I do not allow sunspot to reindex by itself.

searchable :auto_index => false do

This alone I believe is preventing it from reindexing. Instead I run the reindex process on a cron-tab that runs once a day in the early morning.

The reason I started doing this, was because as the application scaled, the data seemed to sitting so heavily on the machine that it would take quite awhile just to simply load the homepage.

What am I doing wrong to cause the error below? And what can I do better?

Error in question:

ActionView::TemplateError (Solr Response: Lock_obtain_timed_out_NativeFSLocktmpindexlucenede61b2c77401967646cf8916982a09a0writelock__orgapachelucenestoreLockObtainFailedException_Lock_obtain_timed_out_NativeFSLocktmpindexlucenede61b2c77401967646cf8916982a09a0writelock__at_orgapachelucenestoreLockobtainLockjava85__at_orgapacheluceneindexIndexWriterinitIndexWriterjava1545__at_orgapacheluceneindexIndexWriterinitIndexWriterjava1402__at_orgapachesolrupdateSolrIndexWriterinitSolrIndexWriterjava190__at_orgapachesolrupdateUpdateHandlercreateMainIndexWriterUpdateHandlerjava98__at_orgapachesolrupdateDirectUpdateHandler2openWriterDirectUpdateHandler2java173__at_orgapachesolrupdateDirectUpdateHandler2addDocDirectUpdateHandler2java220__at_orgapachesolrupdateprocessorRunUpdateProcessorprocessAddRunUpdateProcessorFactoryjava61__at_orgapachesolrhandlerXMLLoaderprocessUpdateXMLLoaderjava139__at_orgapachesolrhandlerXMLLoaderloadXMLLoaderjava69__at_orgapachesolrhandlerContentStreamHandlerBasehandleRequestBodyContentStreamHandlerBasejava54__at_orgapachesolrhandlerRequestHandlerBasehandleRequestRequestHandlerBasejava131__at_orgapachesolrcoreSolrCoreexecuteSolrCorejava1316__at_orgapachesolrservletSolrDispatchFilterexecuteSolrDispatchFilterjava338__at_orgapachesolrservletSolrDispatchFilterdoFilterSolrDispatchFilterjava241__at_orgmortbayjettyservletServletHandler$CachedChaindoFilterServletHandlerjava1089__at_orgmortbayjettyservletServletHandlerhandleServletHandlerjava365__at_orgmortbayjettysecuritySecurityHandlerhandleSecurityHandlerjava216__at_orgmortbayjettyservletSessionHandlerhandleSessionHandlerjava181__at_orgmortbayjettyhandlerContextHandlerhandleContextHandlerjava712__at_orgmortbayjettywebappWebAppContexthandleWebAppContextjava405__at_orgmortbayjettyhandlerContextHandlerCollectionhandleContextHandlerCollectionjava211__at_orgmortbayjettyhandlerHand) on line #24 of app/views/main/_main_nav.html.haml:
21:         %br
22:         community calendars
23: 
24:   - if (current_user.blank? || current_user.card_signup.blank?)
25:     %li
26:       - if current_user.blank?
27:         = link_to 'Get Your HQcard', signup_path, :title => "Signup for your free HQcard and redeem local deals and promotions."

    rsolr (0.12.1) [v] lib/rsolr/connection/requestable.rb:39:in `request'
    rsolr (0.12.1) [v] lib/rsolr/client.rb:34:in `request_without_rails_logging'
    /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/solr_logging.rb:25:in `request'
    /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/solr_logging.rb:24:in `request'
    rsolr (0.12.1) [v] lib/rsolr/client.rb:22:in `update'
    rsolr (0.12.1) [v] lib/rsolr/client.rb:46:in `add'
    sunspot (1.2.1) lib/sunspot/indexer.rb:101:in `add_documents'
    sunspot (1.2.1) lib/sunspot/indexer.rb:26:in `add'
    sunspot (1.2.1) lib/sunspot/session.rb:91:in `index'
    sunspot (1.2.1) lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `index'
    sunspot (1.2.1) lib/sunspot.rb:175:in `index'
    /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/searchable.rb:349:in `solr_index'
    /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/searchable.rb:405:in `maybe_auto_index'
    vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/acts_as_authentic/session_maintenance.rb:73:in `save_without_session_maintenance'
    vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/callbacks.rb:83:in `save_record'
    vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/priority_record.rb:30:in `save_record'
    vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/persistence.rb:60:in `persisting?'
    vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/persistence.rb:39:in `find'
    app/controllers/application_controller.rb:23:in `current_user_session'
    app/controllers/application_controller.rb:28:in `current_user'
    (eval):2:in `send'
    (eval):2:in `current_user'
    app/views/main/_main_nav.html.haml:24:in `_run_haml_app47views47main47_main_nav46html46haml_locals_main_nav_object'
    haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:11:in `render'
    haml (2.2.2) [v] lib/haml/helpers.rb:96:in `non_haml'
    haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:11:in `render'
    app/views/main/index.html.haml:2:in `_run_haml_app47views47main47index46html46haml'
    haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:13:in `render'
    haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:13:in `render'
    haml (2.2.2) [v] rails/./lib/sass/plugin/rails.rb:19:in `process'
    lib/flash_session_cookie_middleware.rb:14:in `call'
    vendor/gems/hoptoad_notifier-2.2.2/lib/hoptoad_notifier/rack.rb:27:in `call'

Are you seeing this in development or production? It could just be a lack of server resources. — jamesc, Aug 10 '11 at 17:46
What's the size of your index? How many documents do you have? Lock timeouts on solr are usually related to a machine that's doing much more than it should. — Maurício Linhares, Aug 10 '11 at 18:32
This is on production. And it's EngineYard's High CPU Medium 32-bit with 1.7 GB RAM, 5 ECU. @Mauricio, how do you tell the size of your index, or the quantity of documents? They are fairly sophisticated and heavily associated objects, but there's no more than 5,000 of any of the 30 models I have. So I can't see that being a problem. — Trip, Aug 10 '11 at 19:42
I have just looked at your code again and from the looks of it you're reindexing the user model on **every** request, which would surely cause a **lot** of performance issues on solr and it would lock for index write at many moments. There is probably a model that's updating your user model and possibly causing the reindex for this model on Solr. — Maurício Linhares, Aug 10 '11 at 22:45
Wow great eye! What would cause something to reindex like that on default? — Trip, Aug 11 '11 at 03:27

Chris · Accepted Answer · 2011-10-30T13:15:43.757

12

You can specify minimum and maximum memory allocation as per the current environment in your config/sunspot.yml file, e.g.

development:
  solr:
    min_memory: 512M
    max_memory: 1G
    solr_jar: /path/to/start.jar
    log_level: DEBUG

Documentation: http://outoftime.github.com/sunspot/rails/docs/classes/Sunspot/Rails/Configuration.html

edited Oct 30 '11 at 13:15

answered Oct 21 '11 at 15:19

Chris

1,501
17
32

What would happen if you went over your max_memory? – Trip Oct 24 '11 at 11:19
1

You wouldn't - that's the point. It's a trade off in memory- and time-complexity. – Chris Oct 30 '11 at 13:14

score 1 · Answer 2 · answered Sep 06 '19 at 07:40

max_memory and min_memory was changed to memory in https://github.com/sunspot/sunspot/commit/74ea3881872398bd14cf6e96481826141ee82c93

So the current answers should be:

development:
  solr:
    memory: 1G
    solr_jar: /path/to/start.jar
    log_level: DEBUG

score 0 · Answer 3 · edited Jan 17 '14 at 23:59

I reversed the Lucene engine execution and forced it to use 64 bit JVM which allowed it full access to all the memory available on my machine ...

Why would rake use 32bit JVM on 64 bit windows is beyond me ...

I'm now using sunspot:solr run script to execute manually:

cd \RailsInstaller\Ruby1.9.3\lib\ruby\gems\1.9.1\gems\sunspot_solr-2.1.0\solr\

java -d64 -Djetty.port=8982 -Dsolr.data.dir=C:/[your-project-dir]/solr/data/development -Dsolr.solr.home=solr -Djava.util.logging.config.file=C:/Users/[user-name]/AppData/Local/Temp/logging.properties20140117 -jar start.jar

Sunspot-solr is eating up too much memory? What's a convenient way to reduce memory usage?

3 Answers3