-1

i am running Production Docker Swarm Cluster with 3 manager nodes and many workers.

Every node (managers, worker) running same Docker and BTRFS Version:

Server Version: 17.12.1-ce
Storage Driver: btrfs
Build Version: Btrfs v4.9.1
  Library Version: 102

I got deployed service with 1 replication. This service using secret.

"Secrets": [
{
    "File": {
        "Name": "/var/secret",
        "UID": "0",
        "GID": "0",
        "Mode": 400
    },
    "SecretID": "vb8485hcixfhnqrp29m8lrfm2",
    "SecretName": "supersecret"
}

This secret exists on Docker Swarm manager Leader:

{
    "ID": "vb8485hcixfhnqrp29m8lrfm2",
    "Version": {
        "Index": 124153
    },
    "CreatedAt": "2020-08-17T12:22:29.656205519Z",
    "UpdatedAt": "2020-08-17T12:22:29.656205519Z",
    "Spec": {
        "Name": "supersecret",
        "Labels": {}
    }
}

But cannot start container from this service. When i'm trying to update this service with "docker sevice update --force ${service_name}" always got exited container with Error:

Error response from daemon: unable to get secret from secret store: secret vb8485hcixfhnqrp29m8lrfm2 not found.

Container is creater without "secrets" folder in /var/lib/docker/container_ID/

drwx------. 1 root root    0 Aug 19 11:06 checkpoints
-rw-------. 1 root root 9305 Aug 19 11:41 config.v2.json
-rw-r--r--. 1 root root 1599 Aug 19 11:41 hostconfig.json
-rw-r--r--. 1 root root   13 Aug 19 11:41 hostname
-rw-r--r--. 1 root root  150 Aug 19 11:41 hosts
-rw-r--r--. 1 root root   48 Aug 19 11:41 resolv.conf
-rw-r--r--. 1 root root   71 Aug 19 11:41 resolv.conf.hash
drwx------. 1 root root    0 Aug 19 11:06 shm

I dont know what to do or what's wrong. Your help will be much appreciated.

Neo Anderson
  • 5,957
  • 2
  • 12
  • 29
xHybridC
  • 1
  • Use the name `supersecret` instead. Side note: changes on existing secrets won't be available inside a container, until the service is undeployed/redployed. If you add a postfix to your secret name and update the service to use the new secret, it will immediatly use the new secret. At least this is how it works with a compose.yml based stack deployment. – Metin Aug 19 '20 at 18:23
  • Thank you for answer. No changes are made on created secret. Only thing that is changing on service is Docker image (new version(tag)). Do you think there is a problem with secret ID and Name? Everything is still the same (secret ID and Name) :-( There is only 1 solution that works for me. Its removing service and created it again, but thats not best workaround for production deploy. – xHybridC Aug 19 '20 at 20:06
  • Seems I did add more confusion with my side note.. Did you try to reference the secret by its name instead of its id? Honestly, I never even though about referencing secrets by its randomly generated id, when there is a stable self declared name. – Metin Aug 20 '20 at 07:22

1 Answers1

0

As a response to

Did you try to reference the secret by its name instead of its id? Honestly, I never even though about referencing secrets by its randomly generated id, when there is a stable self declared name. – Metin

as per Docker API dopcumentation:

SecretName is the name of the secret that this references, but this is just provided for lookup/display purposes. The secret in the reference will be identified by its ID.

Which means that adressing the secret under it's name likely won't solve the problem. Im starting to suspect, that this error message doesn't describe what is actually happening (or at least not fully).

OscarKilo
  • 1