Hbase timeout errors keep occuring

Question

I'm accessing Hbase using happybase with python.

I've a very simple function:

def connect():

    connection = happybase.Connection('myhost',myport)
    table = connection.table('MY-TABLE')
    try:
        return str(table.row('my-row'))
    except Exception as ioe:
        return str(ioe)
    finally:
        connection.close()

When I run this function it works fine for a few minutes, then I start getting timeout errors.

The fix is to go into Hbase console and open a new thrift port, and point to that.

This works again for a few minutes and then I get the timeout errors again.

Not a good fix, any idea why this might be happening?

EDIT

Here is error:

`Traceback (most recent call last):
  File "appi.py", line 38, in <module>
    print connectx()
  File "appi.py", line 26, in connectx
    print str(table.row('1464242429566-2531079631688429'))
  File "C:\Users\me\Downloads\happybase-master\happybase-master\happybase\table.py", line 116, in row
    self.name, row, columns, {})
  File "C:\Python27\lib\site-packages\thriftpy-0.3.8-py2.7.egg\thriftpy\thrift.py", line 160, in _req
    return self._recv(_api)
  File "C:\Python27\lib\site-packages\thriftpy-0.3.8-py2.7.egg\thriftpy\thrift.py", line 172, in _recv
    fname, mtype, rseqid = self._iprot.read_message_begin()
  File "C:\Python27\lib\site-packages\thriftpy-0.3.8-py2.7.egg\thriftpy\protocol\binary.py", line 372, in read_message_begin
    self.trans, strict=self.strict_read)
  File "C:\Python27\lib\site-packages\thriftpy-0.3.8-py2.7.egg\thriftpy\protocol\binary.py", line 164, in read_message_begin
    sz = unpack_i32(inbuf.read(4))
  File "C:\Python27\lib\site-packages\thriftpy-0.3.8-py2.7.egg\thriftpy\transport\__init__.py", line 32, in read
    return readall(self._read, sz)
  File "C:\Python27\lib\site-packages\thriftpy-0.3.8-py2.7.egg\thriftpy\transport\__init__.py", line 14, in readall
    chunk = read_fn(sz - have)
  File "C:\Python27\lib\site-packages\thriftpy-0.3.8-py2.7.egg\thriftpy\transport\buffered\__init__.py", line 39, in _read
    self._rbuf = BytesIO(self._trans.read(max(sz, self._buf_size)))
  File "C:\Python27\lib\site-packages\thriftpy-0.3.8-py2.7.egg\thriftpy\transport\socket.py", line 108, in read
    buff = self.sock.recv(sz)
socket.timeout: timed out`

Ram Ghadiyaram · Accepted Answer · 2016-06-01T15:54:01.663

1

Increase the time-out by setting the property in hbase-site.xml which will be available in hbase/conf

<property>
    <name>hbase.regionserver.lease.period</name>
    <value>900000</value> <!-- 900 000, 15 minutes -->
  </property>
  <property>
    <name>hbase.rpc.timeout</name>
    <value>900000</value> <!-- 15 minutes -->
  </property>
<property>
    <name>zookeeper.session.timeout</name>
    <value>20000</value>
</property>

edited Jun 01 '16 at 15:54

answered Jun 01 '16 at 15:47

Ram Ghadiyaram

28,239
13
95
121

I'll try this. But why is it that it seems to work fine at the beginning, and then it times out 100% of the time. It must be the case that something "fixable" is causing this? – Greg Peckory Jun 01 '16 at 15:49
Can you pls. post complete stack trace in question it self. – Ram Ghadiyaram Jun 01 '16 at 15:56
after seeing stack trace it doesn't seem like hbase time out errors. please see this http://happybase.readthedocs.io/en/happybase-0.6/api.html. BTW what is the version of Happybase you are using? see the connection setting in above mentioned link. – Ram Ghadiyaram Jun 01 '16 at 17:00
I'm using Happybase 0.9 and have read all the docs, but still can't figure it out – Greg Peckory Jun 02 '16 at 08:35
Hhave you changed above mentioned settings at hbase level ? – Ram Ghadiyaram Jun 02 '16 at 11:24
I m good in hbase & not aware of Happybase but I have done some research on this to find out timeout setting was there in earlier versions of HappyBase for e.g.0.6 . class happybase.Connection(host='localhost', port=9090, timeout=None, autoconnect=True, table_prefix=None, table_prefix_separator='_', compat='0.92', transport='buffered') But 0.9 version some how timeout setting was missing in connection. there is no configurable setting in latest version of Happybase. I feel that you have to revisit thrift bindings setting some where which Im not aware... Thx – Ram Ghadiyaram Jun 03 '16 at 03:46
No I've moved on toJava API – Greg Peckory Jun 03 '16 at 12:07

score 1 · Answer 2 · answered Mar 09 '17 at 15:09

1

You can also increase the timeout in the happybase.Connection instantiation:

conn = happybase.Connection(server_url, 9090, timeout=100000)

There is an accepted (but not released) pull request for honoring None as "never timeout"

answered Mar 09 '17 at 15:09

Steen

6,573
3
39
56

Hbase timeout errors keep occuring

2 Answers2