0

I need to implement Linux-HA configuration in two servers. I've decided to use DRBD for block level replication on both hosts, mainly for MySQL data replication.

As I understand, in DRBD configuration there's always a primary server, others are slaves (which can have slave of their own). Replication is only passed to from masters to slaves, not the other way around.

So what happens, if I have MySQL processes performing writes on both servers at the same time, one of which is master, the other is slave?

The slave can perform writes, but no data is written?

Provided that this configuration will go in conjunction with Heartbeat, it would be Heartbeats' job to ensure that MySQL runs only on the master, but lets assume for the moment, that Heartbeat failed for some reason.

Karolis T.
  • 2,719
  • 7
  • 33
  • 45

5 Answers5

3

As far as I know, DRBD does partition level replication. The slave does not mount the partition so it cannot write to it. When there's a problem in the master, the slave will mount the partition and start working.

So you can't have both servers writing over the same data.

chmeee
  • 7,370
  • 3
  • 30
  • 43
  • Can the slave mount the block device as read only? – Karolis T. Jun 17 '09 at 08:56
  • I have not tried it myself but I guess it could be tricky. Because that partition is being updated constantly with information from the master by drbd, I don't know how well will behave mounted, even read-only. – chmeee Jun 17 '09 at 09:06
  • Unless the underlying filesystem "knows" that the block device is being updated, the block cache will get out of date and the FS will look massively corrupted. – womble Jun 17 '09 at 10:22
1

The commonly used configuration, and one that I've used myself in the past, is to have two nodes with some resources between them. These resources are a DRBD storage block device, a MySQL daemon and on IP address.

The resources can only ever be "up" on on the primary node. Heartbeat takes care of selecting which node is primary and starting the resources in the correct order - tell DRBD to become primary, mount the BD, start MySQL and bring up the IP. They occur in that order to ensure data consistency. You can failover between the primary and secondary nodes manually at your choosing either by demoting the primary or promoting the secondary.

Whilst the primary is up the only actions that the secondary performs is to replicate DRBD data and participate in Heartbeat communications to say that it's alive. During it's time as a secondary, MySQL is never running and you are unable to use the storage block device. The resources can only be used on one machine at a time.

This setup differs to MySQL's conventional "lazy replication" where a secondary machine runs MySQL and store it's own copy of data. Both have their pros and cons. In my experience, if HB is setup correctly and you have a good backup policy in addition, then the LinuxHA approach can provide much better high availability.

Dan Carley
  • 25,617
  • 5
  • 53
  • 70
0

A better solution for MySQL HA is to use circular MySQL replication. This article explains it further, but the basic premise is to make each server the slave of the other. You will need to set up some sort of load balancer in front.

One thing to worry about is that database connections may be long-lived and you will need to configure applications to reconnect if they die.

David Pashley
  • 23,497
  • 2
  • 46
  • 73
  • Best to put large scare quotes around "better". Circular MySQL replication is great in theory, but sucks rocks in practice. BTGTGTTS. – womble Jun 17 '09 at 09:02
  • womble> That sounds like the haggard voice of experience :) – Dan Carley Jun 17 '09 at 09:22
  • Far too long spent fighting MySQL replication. I'm a DRBD replication booster all the way. – womble Jun 17 '09 at 10:23
  • We use MySQL replication in a master/slave configuration without problems on some very large databases. We have a couple of smaller ones using a circular replication scheme, but they don't have much use right now, so I can't say how well it works. – David Pashley Jun 17 '09 at 10:44
0

"Can't happen". DRBD will ensure that only one server has the block device active at once. If you get a "split brain", where both DRBD hosts think the other is dead (so they're both active), they won't reconnect and you need to resolve the inconsistency by hand (usually discard the changes on one master or the other).

womble
  • 96,255
  • 29
  • 175
  • 230
  • Can DRBD be integrated with Heartbeat to rely on it's mechanism of setting the master/slave combo, instead of using it's own? I don't want multiple mechanisms checking for the same thing. Also, can the slave mount partition as read only and would that be enough to run MySQL in "read only" mode on the slave? Thanks! – Karolis T. Jun 17 '09 at 09:07
  • For this reason it's best to use a range of alternative keepalive modes. IP to each other, IP to a common host, serial to each other, etc. Then for completeness a STONITH method over the top which ensures that a node really is dead if it claims that it is. – Dan Carley Jun 17 '09 at 09:08
  • Karolis> *1* DRBD doesn't have it's own mechanism to promote or demote. It cares only about the consistency of data between two nodes. Whereas Heartbeat does much more. *2* Generally the answer is no you can't mount from the secondary. Not because of DRBD, but because your filesystem would dislike it. – Dan Carley Jun 17 '09 at 09:15
  • @Kariolis: With LVM snapshotting, you can take a snapshot of the DRBD volume on the slave and mount that, but updates won't go through. Apart from that, listen to Dan C, for he knows of what he speaks. – womble Jun 17 '09 at 10:25
-1

If you need DRBD "mainly" for MySQL replication, maybe it is easier to use the replication features built into MySQL?

Sven
  • 98,649
  • 14
  • 180
  • 226
  • The usual method of doing MySQL replication, with a master and a slave, is not a HA solution. You would need to reconfigure all your clients to use the slave in the event of the master failing. – David Pashley Jun 17 '09 at 08:52
  • Mainly, but not only. I'm using MySQL 5.0, and a lot of improvements regarding replication came with 5.1 as far as I know. – Karolis T. Jun 17 '09 at 09:00