1

The logs file generated by scrapyd occupies too much memory. I want to find it and delete it. How can I find it?

Denuka
  • 1,142
  • 3
  • 13
  • 21
liubindetuimao
  • 31
  • 1
  • 5

2 Answers2

1

as per scrapyd documentation, the logs should be located in /var/log/scrapyd/
the main log file /var/log/scrapyd/scrapyd.log

there are other two logs which log console output and error

/var/log/scrapyd/scrapyd.out
/var/log/scrapyd/scrapyd.err

there will be individual logs for each project /var/log/scrapyd/PROJECT/SPIDER/ID.log

you can delete the logs using rm command

but I suggest you implement log rotate link instead of deleting the logs manually

Nakul Narayanan
  • 1,412
  • 13
  • 17
  • I am wondering if this is still valid in 2021. The folder exists on my ubuntu 20.04 machine but is empty. – merlin Dec 16 '21 at 17:33
1

They are in the directory which you have specified in logs_dir

Or get the name of that log file and run

find / -name "name_of_file.log"
Umair Ayub
  • 19,358
  • 14
  • 72
  • 146