Currently the logs of rq is something like this:-
15:15:03
15:15:03 *** Listening on ingest...
15:17:41 ingest: tasks.ingest_job(aws_key='SDLKASFJHJKAHAJ', aws_secret='LDFKASJKDFJKSAHKJHkjsadhfkjaKJDSAHK')
So, It logs the arguments of the job too, which is a security issue, as the argument contains aws secret and access key. Is there any way to fix this in python rq? such that it won't log the arguments, or any other way
You can setup your own worker script (described in the documentation) instead of running rq worker. This allows you to set the log_job_description flag of the Worker object.
For example:
#!/usr/bin/env python
import sys
from rq import Connection, Worker
# Preload libraries
import library_that_you_want_preloaded
# Provide queue names to listen to as arguments to this script,
# similar to rq worker
with Connection():
qs = sys.argv[1:] or ['default']
w = Worker(qs, log_job_description=False)
w.work()
This will log each job as it's created, but won't display the function parameters in the log.
You can silence the logging:
rq worker ingest --quiet
I don't think there is a straightforward way for the user to modify the format of the logging.
Just run worker with --log-format https://python-rq.org/docs/workers/#worker-arguments
