Exchange Server 2013 Content Indexes Failed in a DAG that spans a WAN

Question

We recently were the victim of database corruption in one of our mailbox servers. In our environment we have two mailbox servers and two CAS servers. One Mailbox Server and one CAS Server is in our Corporate Headquarters, the other Mailbox Server and CAS server is a Co-Lo over two hours away. The WAN between them is limited in bandwidth so re-seeding the mailbox databases and the content indexes across the WAN was not an option.

In our first attempt we followed this technet article on how to Manually copy and offline database which worked for the mailbox databases themselves but not for the content indexes, they still wanted to seed over the WAN.

Answer 1

In the end we wound up contacting Microsoft. They tried a couple different things but what wound up working was:

On the active server with the healthy indexes, stop the Microsoft Exchange Search and Microsoft Exchange Search Host Controller services

1. Go into the folder where the mailbox databases are stored and copy out the content index folder. This folder looks like a guid, in my case one of them was named "32CBE741-FBB2-4116-9E85-12D59725D6E912.43.Single".

2. Start the services back up on the active server.

3. Transport the folders to the failed site.

4. Stop the same services on the server with the failed content indexes.

5. Copy the content index folder from the healthy server to the appropriate folder in the non-healthy server. You should now have two content index folders, one that was already there that was probably "seeding" and the one you copied from the healthy site.

6. With the search services still stopped (carefull they may restart on their own) rename the "bad" content index folder to add something like .old on the end. Then rename the "good" content index folder to match what the "bad" one was named.

7. Start the search services back up.

In our case, within about 10 minutes the content indexes went from Unknown to Healthy. Hope this helps someone else