5

I am running a website with 2 servers for website code (in PHP), and 1 server as load-balancer. All 3 are also running couchbase instances, as part of one single cluster.

enter image description here

In PHP code, I using couchbase buckets as follows:

$cluster = new \CouchbaseCluster('http://127.0.0.1:8091');
$greyloftWebbucket = $cluster->openBucket('some_bucket');
$query = \CouchbaseViewQuery::from('abcd', 'pqrs');

This arrangement works fine when all couchbase instances are running. When any one of them is closed and I try to access buckets, I get following error randomly:

[2015-07-17 13:46:08] production.ERROR: exception 'CouchbaseException' with message 'Generic network failure. Enable detailed error codes (via LCB_CNTL_DETAILED_ERRCODES, or via `detailed_errcodes` in the connection string) and/or enable logging to get more information' in [CouchbaseNative]/CouchbaseBucket.class.php:282
Stack trace:
#0 [CouchbaseNative]/CouchbaseBucket.class.php(282): _CouchbaseBucket->http_request(1, 1, '/_design/abcd...', NULL, 1)
#1 [CouchbaseNative]/CouchbaseBucket.class.php(341): CouchbaseBucket->_view(Object(_CouchbaseDefaultViewQuery))
#2 /var/www/greyloft-laravel/app/couchbasemodel.php(25): CouchbaseBucket->query(Object(_CouchbaseDefaultViewQuery))
#3 /var/www/greyloft-laravel/app/Http/Controllers/Listing.php(42): App\couchbasemodel::listings()
#4 [internal function]: App\Http\Controllers\Listing->index()

That is, one time the page will load correctly and show bucket content, and then one time will show me above error. It doesn't matter if I access load balancer or any of the server directly.

Also, autofailover in enabled with replication set to 1 in couchbase cluster. In all 3 servers, I have set LCB_LOGLEVEL=5

What is happening? Is it problem in Couchbase PHP SDK or anything else? I would appreciate any help at all.

Update:

As per mnunberg's suggestion, I'm using new connection string

$cluster = new \CouchbaseCluster('http://127.0.0.1:8091?detailed_errcodes=1');

With this, the error message is now. It still pops over randomly (around half the times):

CouchbaseException in CouchbaseBucket.class.php line 74: The remote host refused the connection. Is the service up?

The autofailover is taking place. In Couchbase console log:

Failed over 'ns_1@<ip_address>': ok
Node ('ns_1@<ip_address>') was automatically failovered.

It seems to me that SDK is still trying to read the failed over node. Why is this happening? Any possible solution?

  • 2
    Set `detailed_errcodes=1` in the connection string to give you a more detailed error message – Mark Nunberg Jul 21 '15 at 16:31
  • Also please verify that autofailover has indeed taken effect. The default AutoFailover timeout is 120 seconds (two minutes), so expect to see failures (if you've brought a node down) for two minutes until auto failover actually happens – Mark Nunberg Jul 21 '15 at 16:46
  • @mnunberg thanks for the suggestion, I have updated the connection string and included the detailed message in question update. Also, the autofailover is working. Can you give any hint why is error message is popping up? –  Jul 22 '15 at 06:46
  • Can you also explain what the load balancer is for? is the load balancer for your site, or is it for Couchbase? – Mark Nunberg Jul 22 '15 at 15:36
  • Client-side logs on the server in this case would also be extreeeemely helpful. See http://docs.couchbase.com/developer/c-2.4/logging.html. You can also use the (currently undocumented) `console_log_level` and `console_log_file` connection string options as well – Mark Nunberg Jul 22 '15 at 15:45

0 Answers0