1

I have a slurm cluster of 8 separate rhel9 server nodes to serve as master, compute, and database nodes. The master and compute nodes are up and talking, but I have not activated the database node yet. I know that the slurmdbd node can be configured easily for Job Accounting Storage and Job Completion Logs via the slurm.conf file parameters. However, I cannot find any resources or documentation for how I would store the slurmd computational output files themselves, nor do any of the LLM at my disposal have any idea.

Is there built-in functionality within slurm to connect to a database server and automatically send the output files to that server? If not, any tips or pointers for automatically storing the output files from each node in one database that I can access remotely?

Not important, but if possible I was hoping I would be able to use postgreql and pgadmin4 in particular on this project, due to already having a psql server running on my database node.

paul runner
  • 23
  • 4

1 Answers1

3

The Slurm database configuration is only meant for Slurm's internal use (accounting, reporting, limits and priority, etc.). It offers no functionality to push the results into a database.

If you want to store the results of your jobs in a database, you can add the functionality at the end of your submission script, but that is something you will have to do explicitly, either using the mysql/pgsql command line utility, or some other way if the software you ran has that functionality enabled, and you will have to configure a dedicated database and manage the access rights, etc.

Some workflow management systems can help you build workflows where the ingress/outgress of the data from/to a database are managed automatically.

damienfrancois
  • 52,978
  • 9
  • 96
  • 110