2

My application seems to be having issues with memcached not stopping.

I'm using php, and have specific keys expiring after every hour. However those keys & values are not repopulating anymore.

When I run:

/etc/init.d/memcached restart

I get the following:

Stopping memcached:                                        [FAILED]
Starting memcached:                                        [  OK  ]

I have to run a killall memcached for memcached to stop. I then run a restart and everything is fine.

I'm not exactly sure what is causing this, but I need memcached to be restarting every hour. Where should I be looking to find out what is causing this?

Jhouse153
  • 21
  • 1
  • 2
  • 3
    You need memcached to restart once an hour? Sounds like you've got some other problem you're trying to inelegantly solve, too. You can set an expiration time for memcached keys when you set them... – ceejayoz Apr 04 '13 at 15:03
  • Perhaps I was unclear, but the expirations are set once an hour in the php code. But the keys aren't being reset is the issue I'm trying to solve. Memcached doesn't need to be reset, just the keys/values. – Jhouse153 Apr 04 '13 at 15:08
  • 4
    Inelegance is not the question there... but expiring all of a cache at one point in time is a good way to stampede the application under heavy load. – rackandboneman Apr 04 '13 at 15:16
  • 2
    I won't comment on the why, but have a look at the PID recorded in /var/run/memcached (or similar) then compare it to the running PID of memcached. – NickW Apr 04 '13 at 15:45
  • Thank you for responding to the question Nick. That is one thing I've noticed. The memcached.pid is empty, I'm assuming it shouldn't be. – Jhouse153 Apr 04 '13 at 15:54
  • No, it shouldn't be. Is there some other process monitoring, or restarting memcached? – NickW Apr 04 '13 at 16:02
  • No, there isn't. – Jhouse153 Apr 04 '13 at 16:08

2 Answers2

1

For me it was because I initially ran memcached -d manually.

Then later when I tried to run service memcached restart/stop, the stop command would fail. After killing the process manually service memcached restart/stop worked as expected.

KennyCason
  • 111
  • 3
1

On face value, I would suggest that something is making memcached crash. And if memcached isn't running when you try to stop it, it will fail (because it's stopped already). So that would explain why /etc/init.d/memcached stop (or the first part of /etc/init.d/memcached restart) is failing.

But then you go on to say that you need to run killall memcached to get it to stop. That would suggest to me an issue with your init system losing track of the process. As I don't know what init system you are using, it's hard to be sure.

Most distros these days use SystemD and I've had some edge case issues with Debian because of problems caused by the backwards compatibility with the old SysVinit scripts.

Looking at the commands you are using to control your service, you are directly calling the SysVinit script, so I'll assume that either you're using SysVinit, or SystemD with SysVinit compatability.

If you don't have any SystemD, then it's almost certainly something wrong with your initscript and/or memcached config.

If you do have SystemD, then it's likely being caused by some communication issues between your initscript and SystemD (and there could still be memcached config issues too). IMO, the best path forward there, would be to write a proper SystemD service file and use SystemD commands directly; i.e. systemctl COMMAND SERVICE_NAME.service. They're super easy to write, google will help you out (try something like "how to write systemd service file")

Jeremy Davis
  • 406
  • 4
  • 15