4

I currently have 2 web servers (CentOS 5.5) and am using GlusterFS as client/server on both using Automatic File Replication.

For both webservers:

The GlusterFS mount point is:

 /mnt/glusterfs/site1

The local GlusterFS brick is

 /export/sdb1/glusterfs/site1

I am using a bind mount to point to the GlusterFS mount in Apache:

/var/www/site -> /mnt/glusterfs/site1

The dynamic pages (where caching the page itself isn't an option) that require many small files to be read before the page is rendered, the performance with GlusterFS is at least 5 times slower than using the local filesystem. i.e. the local GlusterFS brick.

Note that I am using the option read-subvolume option so that the files are read from the local GlusterFS brick but I understand that the metadata is checked on the other brick over the network which is what I assume is slowing things down.

Unfortunately I can't just get Apache to use the local GlusterFS brick as Apache itself does some writes.

Since /mnt/glusterfs/site1 and /export/sdb1/glusterfs/site1 are effectively the same filesystem is there any way for Apache to read directly off the local brick but write to the GlusterFS mount?

This would significantly increase performance.

I am aware of NFS, OCFS, GFS2, Lustre and DRBD but if it's possible to do the above, this would be the simplest solution.

Perhaps an Apache directive, Apache module (custom module even) or even at the system level?

I would appreciate any suggestions/comments/direction.

crb
  • 7,998
  • 1
  • 38
  • 53
Vipul Limbachiya
  • 81
  • 1
  • 3

1 Answers1

1

If you write directly, it will not be replicated through glusterfs. If you read directly, you run the risk of your repair not working correctly if your drive fails. When you read a file, it checks the metadata, and if the file doesn't exist locally, replicates it rather than replicating the filesystem when the drive is brought online.

If those files are created regularly, can you write them to a non-distributed mount and use that for the files?

You can do what you want, but, it'll probably break in some very odd ways. One scenario would be if you modify a file on one node, the other node wouldn't know that the metadata changed and would read the old file. You might stop by the GlusterFS mailing list, they are extremely helpful.

karmawhore
  • 3,865
  • 18
  • 9
  • "The written files need to be replicated. Ideally, I'm wanting Apache to read from the local GlusterFS brick and to write to the GlusterFS mount so that writes will be replicated. i.e. Apache reads from reads from /export/sdb1/glusterfs/site1 but writes to /mnt/glusterfs/site1. Whenever a brick is disconnected and re-connected I do a full repair to bring it in sync again. I do take your point about hard drive failure. I'm running RAID 6 so this risk is relatively low and I'm willing to take that risk for an increase in performance. Thanks, I'll check out the GlusterFS mailing list." – Vipul Limbachiya Jul 02 '10 at 17:49