1

Docker Help Request: I am trying to connect a container within our Swarm to a device in /dev. The connection works correctly via device mapping when running in a docker container, however we are unable to connect to the device within our Swarm node.

The device happens to be graphics hardware to use vaapi drivers with our Gstreamer pipeline, and is located here: /dev/dri/renderD128, but this is a generic question for connecting to any device from a container in a swarm.

I am looking for advice on what it would look like to map to our device through Swarmkit Generic Resources.

There is discussion available regarding Swarmkit and graphics hardware including:

But, I have been unable to distill these into a solution.

One docker run command that successfully connects to this device when NOT running in the swarm looks like this:

docker run -ti \
-e XDG_RUNTIME_DIR=/tmp \
--device=/dev/dri/card0:/dev/dri/card0 \
--device=/dev/dri/renderD128:/dev/dri/renderD128 \
-v $XDG_RUNTIME_DIR/$WAYLAND_DISPLAY:/tmp/$WAYLAND_DISPLAY \
<docker-image> bash

But again, this is a generic question for containers, swarms, and device mapping in general, and is not related to this particular device.

We are using docker compose file version 3, and have tried the "device" option which fails as expected: https://docs.docker.com/compose/compose-file/#devices

"This option is ignored when deploying a stack in swarm mode with a (version 3) Compose file."

I am grateful for your help.

David Manpearl
  • 12,362
  • 8
  • 55
  • 72

1 Answers1

3

Have you seen https://github.com/NVIDIA/nvidia-docker/issues/141#issuecomment-356458450?

with 17.12.0-ce you can configure the docker daemon to expose your GPUs to swarm.

Using 'docker service' now supports it however there is no --generic-resource equivalent in docker-compose yet as far as I can tell.

EDIT: actually there is support in compose 3.5 :

       generic_resources:
           - discrete_resource_spec:
                kind: 'gpu'
                value: 2

[ https://github.com/docker/cli/commit/1ff73f867df382cb5a19df4579da3570f4daaff5 ]

galp
  • 31
  • 5