43

From Google's documentation:

It is possible to attach a persistent disk to more than one instance. However, if you attach a persistent disk to multiple instances, all instances must attach the persistent disk in read-only mode. It is not possible to attach the persistent disk to multiple instances in read-write mode.

If you attach a persistent disk in read-write mode and then try to attach the disk to subsequent instances, Google Compute Engine returns an error.

So, I need to have a share persistent-disk as frontend for all my compute engine, good, how can you write on this shared disk?

My guess (I hope) is a read/write persistent-disk can be attached only with 1 compute engine but this same disk can be share in read only to others VMs, is thats right?

Lets say I have 2 Compute Engine VMs and 2 persistent disks, is this flow is possible?

  • compute1 read/write disk1 and read only disk2
  • compute2 read/write disk2 and read only disk1
Misha Brukman
  • 12,938
  • 4
  • 61
  • 78
flaubert
  • 593
  • 1
  • 5
  • 10
  • You may be interested in our recently-launched [Google Cloud Filestore](https://cloud.google.com/filestore/) — it's perhaps closest to what you're looking for, as it provides an NFSv3 file system. I've updated my answer with this link as well. – Misha Brukman Aug 01 '18 at 15:36

2 Answers2

59

Update: this is available as of 2020-06-16

As per another answer by Matthew Lenz, the functionality for creating multi-writer persistent disks is available, but it's still in alpha status (even though it's documented as being in the beta track) and requires special per-project enablement.

Note: This GitHub issue notes that the functionality is still in alpha, even though it's labelled as beta. You can submit feedback via Cloud Console to request it for your project if you'd like to get early access to this functionality, but it's not guaranteed to be enabled.

Assuming your project has the permissions to use this feature (or the feature becomes public-access), note that it comes with some caveats:

--multi-writer

Create the disk in multi-writer mode so that it can be attached with read-write access to multiple VMs. Can only be used with zonal SSD persistent disks. Disks in multi-writer mode do not support resize and snapshot operations.

You can use this via:

$ gcloud beta compute disks create DISK_NAME --multi-writer [...]

Note the caveats:

  • zonal SSD persistent disks only
  • no disk resizing
  • no snapshots

If these trade-offs are not acceptable to you, see the original answer (below) which has a long list of recommended storage alternatives for sharing data between multiple GCE VMs.

Original answer (valid prior to 2020-06-16)

No, this is not possible, as the documentation that you cited at the time of writing said (since updated):

However, if you attach a persistent disk to multiple instances, all instances must attach the persistent disk in read-only mode.

The documentation has been re-arranged since then; the new docs are at a different URL but with the same content:

You can attach a non-root persistent disk to more than one virtual machine instance in read-only mode, which allows you to share static data between multiple instances. Sharing static data between multiple instances from one persistent disk is cheaper than replicating your data to unique disks for individual instances.

If you attach a persistent disk to multiple instances, all of those instances must attach the persistent disk in read-only mode. It is not possible to attach the persistent disk to multiple instances in read-write mode. If you need to share dynamic storage space between multiple instances, connect your instances to Cloud Storage or create a network file server.

If you have a persistent disk with data that you want to share between multiple instances, detach it from any read-write instances and attach it to one or more instances in read-only mode.

which means you cannot have one instance have write access while another has read-only access.

If you want to share data between them, you need to use something other than Persistent Disk. Below are some possible solutions.

You can use any of the following hosted/managed services:

Alternatively, you can run your own:

  • self-managed or third-party managed file servers solutions, including NetApp and Panzura
  • self-managed Elastifile storage deployment (for fully-managed, see previous section for the link)
  • database (whether SQL or NoSQL)
  • distributed filesystem such as Ceph, GlusterFS, OrangeFS, ZFS, etc.
  • file server such as NFS or SAMBA
  • single VM as a data storage node, and use sshfs to create a FUSE mount from other VMs that want to access that data
Misha Brukman
  • 12,938
  • 4
  • 61
  • 78
  • thanks Misha , it seems very suprising for me,so if i want to share a simple disk with all my pics on it , i cannot thats right ? i don't want to use cloud storage, an my project needs lots of photo storage – flaubert Nov 13 '14 at 17:36
  • However,they said you can share disk if its not attached to a read/write compute engine ( all instances must attach the persistent disk in read-only mode) , so how do you write on this share disk ?? – flaubert Nov 13 '14 at 17:38
  • 3
    @flaubert since the disk must be mounted read-only to all instances, one possible use case is to mount it once to a single instance in read-write mode, detach it, and then mount it to several instances in read-only mode. If that doesn't work for your use case, consider creating a file share from a single instance to other instances, e.g., using NFS or SAMBA (see updated answer for links). – Misha Brukman Nov 15 '14 at 18:54
  • How can one write to this shared disk then? Is there any way to write to it from outside any instance? – Daniel Möller Sep 14 '17 at 19:59
  • 1
    @DanielMöller — you can only write to a disk if it's attached to exactly one VM. So you can (a) attach to a single VM in read/write mode, (b) write data to it, (c) detach it from that VM, (d) attach it to N VMs in read-only mode. This is good for sharing read-only data; for shared read-write data, you need to use a distributed file system or database such as those listed in the answer; a raw disk does not make a good shared file system because it won't be able to prevent corruption, since every OS expects exclusive read/write access to a mounted disk. – Misha Brukman Sep 15 '17 at 03:05
  • @DanielMöller — please consider also [Google Cloud Filestore](https://cloud.google.com/filestore/), a newly-released product, which is probably the closest to what you're looking for for sharing files between multiple VM instances. – Misha Brukman Aug 26 '18 at 23:30
  • I'm in the same situation. I'm looking at Google Cloud Storage, but I find it very expensive. Which solution is the cheapest? – Apyc Jun 09 '20 at 10:25
  • 1
    --multi-writer sounds promising, but got an error `ERROR: (gcloud.beta.compute.disks.create) Could not fetch resource: - Invalid resource usage: 'Multi Writer Disk feature is not available for this project.'.` – Jonathan Lin Nov 23 '20 at 08:58
  • 1
    @JonathanLin — according to [GitHub issue](https://github.com/GoogleCloudPlatform/magic-modules/pull/3911), apparently this feature is actually still in alpha and limited-access, even though it was labeled as beta (probably unintentionally). You can subscribe to that GitHub issue to follow the updates, and/or feel free to submit feedback in the Cloud Console to see if you can get early access to it before it's widely available. Sorry about that! I'll update the answer. – Misha Brukman Dec 06 '20 at 17:57
7

GCP has alpha functionality for 'multi-write' persistent disks. It's been in alpha for quite a long time so who knows if it'll make it to beta or ga any time soon. Here is a link to the documentation. https://cloud.google.com/sdk/gcloud/reference/beta/compute/disks/create#--multi-writer EDIT: 2020-06-16. This has been promoted to beta.

Matthew Lenz
  • 332
  • 4
  • 8
  • 1
    This has now moved to Preview https://cloud.google.com/compute/docs/disks/sharing-disks-between-vms However, there are a few limitations to the sharing e.g. cannot be shared with more than 2 N2 instances etc. – mkumar118 Apr 22 '21 at 11:58
  • Oh, bummer about 2 instances! Glad I saw this comment before spending many hours debugging issues. – Gajus Dec 06 '21 at 21:22