python-memcached can't get, or set 'JAC_SF'

Question

After checking a whole load of things, I've found that for some keys, python-memcached just won't get, or set them on my machine (Mac OSX 10.9, homebrew memcache 1.4.15, pip installed python-memcached 1.53). My first issue was a shot in the dark, not having a clear idea as to what was going on, but after more digging I now definitely now.

It all hinges around def _get_server(self, key):

If we add two debug printout lines,

def _get_server(self, key):
    if isinstance(key, tuple):
        serverhash, key = key
    else:
        serverhash = serverHashFunction(key)

    for i in range(Client._SERVER_RETRIES):
        server = self.buckets[serverhash % len(self.buckets)]
        if server.connect():
            #print "(using server %s)" % server,
            print 'got server {} for {}'.format(serverhash % len(self.buckets),
                                                key)
            return server, key
        print 'server {} failed for {}'.format(serverhash % len(self.buckets),
                                            key)
        serverhash = serverHashFunction(str(serverhash) + str(i))
    return None, None

Trying to get or set the key 'NFL::CAR_TB',

import memcache
mc_cl = memcache.Client('127.0.0.1')
mc_cl.set('JAC_SF', 1)
mc_cl.get('JAC_SF')
mc_cl.set('UAC_SF', 1)
mc_cl.get('UAC_SF')

results, on my machine with,

server 3 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 7 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 3 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 2 (inet:7:11211 (dead until 1384821475)) failed for JAC_SF
server 5 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 7 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 3 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 7 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 7 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 2 (inet:7:11211 (dead until 1384821475)) failed for JAC_SF
server 3 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 7 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 3 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 2 (inet:7:11211 (dead until 1384821475)) failed for JAC_SF
server 5 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 7 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 3 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 7 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 7 (inet:.:11211 (dead until 1384821475)) failed for JAC_SF
server 2 (inet:7:11211 (dead until 1384821475)) failed for JAC_SF
server 7 (inet:.:11211 (dead until 1384821475)) failed for UAC_SF
got server 6 for UAC_SF
server 7 (inet:.:11211 (dead until 1384821475)) failed for UAC_SF
got server 6 for UAC_SF

If I alter _get_server to

def _get_server(self, key):
    choices = range(len(self.buckets) - 1)
    random.shuffle(choices)

    if isinstance(key, tuple):
        choice, key = key
    else:
        choice = choices.pop()

    for _ in range(Client._SERVER_RETRIES):
        server = self.buckets[choice]
        if server.connect():
            #print "(using server %s)" % server,
            print 'got server {} for {}'.format(choice,
                                                key)
            return server, key
        print 'server {} ({}) failed for {}'.format(choice, server, key)
        choice = choices.pop()
    return None, None

Then everything works much better,

got server 4 for JAC_SF
server 0 (inet:1:11211 (dead until 1384821552)) failed for JAC_SF
server 3 (inet:.:11211 (dead until 1384821552)) failed for JAC_SF
got server 4 for JAC_SF
server 7 (inet:.:11211 (dead until 1384821552)) failed for UAC_SF
server 0 (inet:1:11211 (dead until 1384821552)) failed for UAC_SF
got server 6 for UAC_SF
server 7 (inet:.:11211 (dead until 1384821552)) failed for UAC_SF
server 2 (inet:7:11211 (dead until 1384821552)) failed for UAC_SF
server 5 (inet:.:11211 (dead until 1384821552)) failed for UAC_SF
server 1 (inet:2:11211 (dead until 1384821552)) failed for UAC_SF
got server 6 for UAC_SF

I was getting very inconsistent results when trying to set multiple keys in my python program, import memcache TWO_HOURS = 2 * 60 * 60 mc_cl = memcache.Client('127.0.0.1') mapping = {...} mc_cl.flush_all() ret = mc_cl.set_multi(mapping=mapping, time=TWO_HOURS) getret = mc_cl.get_multi(mapping.keys()) if len(mapping) != len(getret): print 'not set\n\t{}'.format('\n\t'.join([str((k, mapping[k])) for k in [a for a in mapping.keys() if a not in getret.keys()]])) And after analysing the raw memcache output, it seems like not all the keys are being set, and not all the keys are being requested thereafter. All things done and up to date on homebrew and pip, on Mac OSX 10.9. Memcache output below and reading it states that only 101 keys were attempted to be written and read, whereas there were 228 items. import re gamesSet = [] gamesGet = [] with open('memout.log') as f: for line in f.read().split('\n'): match = re.match('^21 OK 22 OK 21 STORED

score 1 · Answer 1 · answered Dec 05 '13 at 17:54

This is an odd one that I'm currently debating with the writer/maintainer of the python-memcached library, https://github.com/linsomniac/python-memcached/issues/21.

Randomly, what was wrong here was the very first line,

mc_cl = memcache.Client('127.0.0.1')

Client expects an iterable, but doesn't ensure to check it doesn't get a string, so in a very pythonic way, it treats this string as an iterable and iterates through it, setting each one of '1', '2', '7' '.' and '0' to be the IP address of a memcached server, but where the real confusion started was that '0' is actually a valid IP address, and evaluates to 0.0.0.0 and is served by your local memcached server, if that's how you're set up (that was how my dev environment is set up).

If you simply change it to,

mc_cl = memcache.Client(('127.0.0.1',))

Everything works exactly as expected, but if you pass in the '127.0.0.1' string, you don't get any error messaging, nor any reports of keys not being set, nor being able to be 'got', just very erratic and non explainable behaviour like above.

python-memcached can't get, or set 'JAC_SF'

1 Answers1