Mechanize memory leak despite setting history size to 0 and clearing history

Question

Here is an example script that reproduces the issue

require 'mechanize'

agent = Mechanize.new
agent.history.max_size = 0 

5000.times do |i| 
  agent.get('http://www.yahoo.com')
  agent.history.clear

  p `ps -o rss -p #{$$}`.strip.split.last.to_i * 1024 # Prints out memory usage of the ruby process
end

I'm doing both agent.history.max_size and agent.history.clear but it seems that memory usage is increasing with each loop.

Here is the output showing increasing memory usage (starts at 48MB and increases by 1-2MB with every loop).

How do I get Mechanize to stop leaking memory?

Does the behaviour change if you create a new agent in each iteration instead of reusing the same agent multiple times? — spickermann, Nov 05 '14 at 10:02
No change, still see the same increase in memory size as above. — Cheenu Madan, Nov 05 '14 at 10:13

score 3 · Accepted Answer · answered Nov 06 '14 at 07:27

3

That's not a memory leak, some things just haven't been gc'ed yet. Put:

GC.start

in the loop if you feel like you need it, otherwise it's probably safe to ignore.

answered Nov 06 '14 at 07:27

pguardiario

53,827
19
119
159

Thanks. Putting `GC.start` at the end of the loop fixed things. – Cheenu Madan Nov 06 '14 at 08:45

Mechanize memory leak despite setting history size to 0 and clearing history

1 Answers1