-2

I want to attach persistent storage to my docker containers. I have tried NFS using TrueNAS and seem to have no luck there.

Surely, I am approaching this in the wrong way. What are the common conventions for attaching persistent storage OUTSIDE of the container so that if the container fails or is remade, the data is re-attached or at the very least remains existent.

EDIT: Apologies for the confusion. I am not talking about bind mounts or volumes, I am well aware of those technologies. I am referring to the underlying filesystems i.e. NFS etc.. are there standard toolings or management systems e.g. Longhorn for block storage on kubernetes. is it better to use a file, object, or block storage solution etc...

EDIT 2: I have ready everything in the Docker docs. I know about volumes and drivers. What I don't know is how to use them properly with respective options as they are not documented properly, or how these things are commonly installed into a solution. Yes I can just "use the driver" but how exactly? Is NFS a common theme? What about block storage? Object storage? How is this architected on a system level.

EDIT: I'm adding this edit because some weasal closed it for not being specific enough. As if asking about this problem isn't relevant. I'm the only one with had an actual answer. Not even the common users of docker have a clue as to how this all works under the hood. You don't think we should have a post about that question? Oh no of course not.. just have ppl repeat the same shit over and over again.

BitShift
  • 95
  • 1
  • 6
  • Have you read about [bind mounts](https://docs.docker.com/storage/bind-mounts/) and [volumes](https://docs.docker.com/storage/volumes/)? One or both of those is what you need. – Moshe Katz Dec 06 '21 at 23:54
  • That's not what I'm referring to... let me clarify the question – BitShift Dec 07 '21 at 00:48
  • Docker only knows volumes. It doesn't care how you store them. – Gerald Schneider Dec 07 '21 at 08:22
  • You should read the section about volumes more carefully, especially the part about volume drivers. – Gerald Schneider Dec 07 '21 at 08:26
  • @GeraldSchneider I've read everything there is to read about storage from the docker docs, what remains unclear is how these drivers are useful. They are sparse in documenting use cases, options, and general architectural design. The last of which I have come here to clarify. – BitShift Dec 08 '21 at 06:44

2 Answers2

2

What are the common conventions for attaching persistent storage OUTSIDE of the container

There aren't.

The whole Docker paradigm is based on the assumption that you can create and destroy containers as you wish. They are not persistent, and they are not supposed to be.

If your containers expect something to be there, You're Doing It Wrong™.

Massimo
  • 70,200
  • 57
  • 200
  • 323
  • Well sure, it needs to be as stateless as possible and lack of dependency is my aim. Things like Wikijs content, Organizr/Heimdall dashboard config files, Ansible cookbooks that I can save outside the container. This way, in the event I need to rebuild the container, I can simply reload my saved content. Surely that sits within the ethos and philosophy of containers no? – BitShift Dec 08 '21 at 07:01
  • Yes, of course. But for all those cases the solution is to store your persistent data *outside* the containers, in some place they can access. *Where*, exactly, is up to your implementation. – Massimo Dec 08 '21 at 17:48
  • Yes. That is the purpose of the question. I'm trying to learn more about the underlying technologies used to facilitate that and I've since learnt about different storage technologies. I'll do some kind of write up here when I'm finished. For now it looks as though NFS for non-performant persistent storage outside the container is preferable. Though for the way I want to configure these containers, I think that block-level with iSCSI will be better. – BitShift Dec 08 '21 at 20:31
0

The convention in terms of underlying file system technology appears to be using block storage solutions such as Cinder, BGFS etc... (which can use Ceph, iSCSI, and other block file types) or object storage like Minio

These kinds of solutions help to manage the underlying disk space easily and also take into account an implicit need of distributed file systems.

It does depend on use cases and NFS is perfectly suitable for use cases where performance is not a concern, like writing files. In this way, a wikijs container would be a perfect candidate for NFS so that writing the user-generated content can remain separate from the container.

Where block storage solutions also shine for containers is providing ease of deployment in rapid create-destroy cycles. As is the typical use case.

Then again, if you are running lots and lots of containers that have indefinite life spans (like I am with Nextcloud, Heimdall, pihole, etc...) then block storage solutions may also provide you with a more flexible arrangement. I like the idea of using block storage in this scenario as the disk and file system are handled by the application/container, rather than it depending on external factors. That may or may not be a real world concern, but I could imagine there being potential issues with NFS because of those reasons or more specifically: less compromise with a block storage solution.

Finally, and I need to confirm this, tracking the volumes associated with each container becomes a task in and of itself. These block storage solutions may provide a neat and manageable way to track that across create-destroy cycles for backup and restore.

BitShift
  • 95
  • 1
  • 6