2

I have Replicated Gluster Storage which is used by Auto scaling Servers. Both, Auto scaling and Storage are allocated in two Availability zones.

Gluster:

Number of Bricks: 4 x 2 = 8
Transport-type: tcp
Bricks:
Brick1: gluster01:/storage/1a # Zone A
Brick2: gluster02:/storage/1b # Zone B
Brick3: gluster03:/storage/2a # Zone A
Brick4: gluster04:/storage/2b # Zone B
Brick5: gluster01:/storage/3a # Zone A
Brick6: gluster02:/storage/3b # Zone B
Brick7: gluster03:/storage/4a # Zone A
Brick8: gluster04:/storage/4b # Zone B

I used Round Robin DNS for Gluster entry point, so DNS name resolves to all of the storage server addresses which are returned in different order all the time:

# host storage.domain.com
storage.domain.com has address xx.xx.xx.x1
storage.domain.com has address xx.xx.xx.x2
storage.domain.com has address xx.xx.xx.x3
storage.domain.com has address xx.xx.xx.x4

The Storage is mounted with Native Gluster Client:

# grep storage /etc/fstab
storage.domain.com:/storage /storage glusterfs defaults,log-level=WARNING,log-file=/var/log/gluster.log 0 0

I have heard Gluster might be mounted with the first Server IP and after that it will fetch its configuration with the rest of Servers. Personally, I never tested single Server mount setup and I don't know how Gluster handles this.

On EC2, traffic among single Availability zone is free and between different zones is not. When Client in zone A writes to storage and IP of Storage in zone B is returned, it will cost me twice more for data transfer: Client (Zone A) -> Storage Server (Zone B) -> Replication to Storage Server (Zone A).

Question: Would it be better to mount Storage Server of the same zone, so that data transfer charges apply only for replication (A -> A -> B)?

Roman Newaza
  • 632
  • 4
  • 13
  • 23

1 Answers1

2

Doing the mount to a server in the same zone will save the initial gluster session setup traffic from traversing the paid links. However, it won't do anything to force data traffic to the local AZ. Gluster distributed-replicated volumes such as yours consider all bricks to be equal. This means that it can put files in unpredictable places; it is entirely possible that the two replicas are in the same AZ, at which point all access to that file from the other AZ will traverse the metered link. Since both replicas are in the other AZ, you'll get charged twice the network transfer due to how Gluster handles replication.

The only place that Gluster has any sense of site-awareness is in their Geo-Replication module. Which is sadly asynchronous.

sysadmin1138
  • 133,124
  • 18
  • 176
  • 300