0

I have a weird problem interfacing C# and Python through Memcached. I have Memcached installed on a server. A C# application would write some data on Memcached using the Enyim.Caching library, then a Python application would read it using python-memcached (I also tried pymemcache and pylibmc). I also would like to reverse the process and having C# reading from Python.

My findings are:

  1. Both C# and Python can write and read variables they wrote themselves on the server.
  2. When C# writes a string, Python can retrieve it just fine.
  3. Other configurations fail and rise the following errors:
    • C# int to Python raises File "C:\Users\...\Anaconda2\envs\py37\lib\site-packages\memcache.py", line 1257, in _recv_value buf = self.decompressor(buf) error: Error -3 while decompressing data: incorrect header check"
    • C# byte to Python raises ValueError: invalid literal for int() with base 10: b' '
    • Python string to C# raises System.ArgumentException: 'Destination array is not long enough to copy all the items in the collection. Check array index and length.'
    • Python int to C# raises An unhandled exception of type 'System.NullReferenceException' occurred in testDatabaseC.dll Object reference not set to an instance of an object.
    • Python byte to C# raises System.InvalidCastException: 'Specified cast is not valid.'

The fact that a string can go one way from C# to Python let me think that there is nothing wrong with my code. The first exception in this list might suggests that the problem is in the compression format used. In know that this Python library uses zlib, but I haven't found any working alternatives.

Any ideas? Thanks!

Some code used bellow:

To write with Python and read with C# (in that case an int, but could also be a byte or a string):

import memcache
memcachedServerIP = "192.168.1.101"
client = memcache.Client([(memcachedServerIP, 11211)])
dataToWrite = 5 # just a random integer to try
print(client.set("key", dataToWrite, time=10))


using System;
using MySql.Data.MySqlClient;
using Enyim.Caching;
using Enyim.Caching.Configuration;
using RGiesecke.DllExport;
using System.Runtime.InteropServices;

namespace testDatabase
{
    public class Test
    {
        public static int readFromCache()
        {
            MemcachedClientConfiguration mcc = new MemcachedClientConfiguration();
            MemcachedClient client;
            mcc.AddServer("192.168.1.101:11211");
            mcc.SocketPool.ReceiveTimeout = new TimeSpan(0, 0, 10);
            mcc.SocketPool.ConnectionTimeout = new TimeSpan(0, 0, 10);
            mcc.SocketPool.DeadTimeout = new TimeSpan(0, 0, 20);
            client = new MemcachedClient(mcc);
            int dataToRead;
            dataToRead = (int)client.Get("key");
            return dataToRead;
        }
    }
}

To write with C# and read with Python:


using System;
using MySql.Data.MySqlClient;
using Enyim.Caching;
using Enyim.Caching.Configuration;
using RGiesecke.DllExport;
using System.Runtime.InteropServices;

namespace testDatabase
{
    public class Test
    {
        public static int writeToCache()
        {
            MemcachedClientConfiguration mcc = new MemcachedClientConfiguration();
            MemcachedClient client;
            mcc.AddServer("192.168.1.101:11211");
            mcc.SocketPool.ReceiveTimeout = new TimeSpan(0, 0, 10);
            mcc.SocketPool.ConnectionTimeout = new TimeSpan(0, 0, 10);
            mcc.SocketPool.DeadTimeout = new TimeSpan(0, 0, 20);
            client = new MemcachedClient(mcc);
            int dataToWrite = 5;
            client.Store(Enyim.Caching.Memcached.StoreMode.Set, "key", dataToWrite);
            return dataToWrite;
        }
    }
}
import memcache
memcachedServerIP = "192.168.1.101"
client = memcache.Client([(memcachedServerIP, 11211)])
print(client.get("key"))


Sirkous
  • 1
  • 1
  • Show us the code you used to identify the problems in "finding 3." – Robert Harvey Oct 20 '20 at 16:02
  • Hi, I just added a minimal code to reproduce the errors at the end of my post – Sirkous Oct 20 '20 at 16:23
  • I would say it's a bad idea to do what you want to do. Memcache is a really dumb server and is heavily dependant on a Client. You would nead a Clint that have the same hashing algorithm and serializing algo. Any object that is sent via the client is serialized in a binary format which is usually not readable by other langs. In c# there is an option to store objects using a text protocol. Try that out. Or also try storing everything as a string (json for obj). This is if there is no other way. This may still cause issues if you use more than 1 Memcache servers. As hashing algo will be diff – pratikvasa Oct 20 '20 at 23:51
  • All that didn't really worked, but I found a solution by calling a C# dll library with python, and all the functions handling memcached being written in C# it eventually worked out... – Sirkous Nov 12 '20 at 23:35

0 Answers0