I'm trying to migrate from Couchbase 2.5.12 into 3.2.2 (or 3.2.3) but I'm getting a mem leak. I faced this same Problem with 2.5.12 and I was able to sort that problem by disabling tracing in the connection string like this: couchbase://localhost:8091?operation_timeout=15&bootstrap_on=http&detailed_errcodes=1&enable_tracing=false.
When moved to Python SDK 3.2.2 I noticed that I have the same (mem leak) problem, so I tried to disable tracing by:
- Using same conn string (with enable_tracing=false) without luck, mem leak still persists
- Pass a ClusterTracingOptions instance to Cluster init with all fields set to None:
tracing_opts = ClusterTracingOptions(
**dict(
tracing_threshold_kv=None,
tracing_threshold_view=None,
tracing_threshold_query=None,
tracing_threshold_search=None,
tracing_threshold_analytics=None,
tracing_threshold_queue_size=None,
tracing_threshold_queue_flush_interval=None,
tracing_orphaned_queue_size=None,
tracing_orphaned_queue_flush_interval=None,
)
)
clu = Cluster(
COUCH_HOST,
ClusterOptions(
PasswordAuthenticator(COUCH_USER, COUCH_PASS),
),
tracing_options=tracing_opts
)
- Pass a Tracer (dummy obj) to Cluster init as:
from couchbase.otel_tracing import CouchbaseOtelTracer
from opentelemetry import trace
...
tracer = trace.get_tracer(__name__)
clu = Cluster(
COUCH_HOST,
ClusterOptions(
PasswordAuthenticator(COUCH_USER, COUCH_PASS),
),
tracer=CouchbaseOtelTracer(otel_tracer=tracer)
)
- Used
tracing_optsandtracer^ toguether on Cluster init.
Any ideas on how can I disable tracing? or avoid the memory leak?
Here is a complete example of the fn that I'm executing:
def multi_proc_insert(a_id, f_path):
tracing_opts = ClusterTracingOptions(
**dict(
tracing_threshold_kv=None,
tracing_threshold_view=None,
tracing_threshold_query=None,
tracing_threshold_search=None,
tracing_threshold_analytics=None,
tracing_threshold_queue_size=None,
tracing_threshold_queue_flush_interval=None,
tracing_orphaned_queue_size=None,
tracing_orphaned_queue_flush_interval=None,
)
)
tracer = trace.get_tracer(__name__)
clu = Cluster(
COUCH_HOST,
ClusterOptions(
PasswordAuthenticator(COUCH_USER, COUCH_PASS),
),
tracing_options=tracing_opts,
tracer=CouchbaseOtelTracer(otel_tracer=tracer),
)
cbu = clu.bucket(BUCKET_NAME)
collection = cbu.default_collection()
lines = mapcount(f_path)
logger.info(f"Start to process {f_path}\t lines: {lines}")
inserted = 0
t1 = time.time()
with open(f_path, "r") as fp:
while True:
e_hash = fp.readline()
if not e_hash:
logger.info(f"e_hash not found (EOF)")
break
key = DOC_TYPE + e_hash.strip()
try:
collection.mutate_in(key, [SD.array_addunique("", a_id)])
except PathExistsException:
logger.debug(f"a_id {a_id} already exists on doc {key}")
except DocumentNotFoundException:
collection.insert(key, [a_id])
except Exception:
logger.exception(
f"Error processing {f_path}, inserted rows: {inserted} remaining: {lines - inserted}"
)
inserted += 1
t2 = time.time()
logger.info(
f"Finished with {f_path}\t lines: {lines} inserted: {inserted} remaining: {lines - inserted} in seconds {t2 - t1}"
)
Thanks in advance,
