I was wondering if anybody knew how to correctly use snakemake in a sun grid engine computing context - I'm using a shared HPC cluster and am running many jobs concurrently, but it's a large strain on the scheduler to queue up thousands of nearly identical jobs that rely on each other.
I've used array jobs in the past to get around this bottleneck and the server admin's emails, but haven't found a good resource to learn how to schedule array jobs efficiently with snakemake.
Has anyone done this in the past?
Currently starting with command :
snakemake --jobs 500 \
--cluster "qsub -pe smp 12 -cwd -j y -o snake.log"
