Closing connection to hbase database using happybase in python

Question

def hbasePopulate(self,table="abc",MachineIP="xx.xx.xx.xx"):

    connection=happybase.Connection(MachineIP,autoconnect=True)
    tablename=Reptype.lower()+'rep'
    print "Connecting to table "
    print tablename
    try:
        table=connection.table(tablename)
        for key,data in table.scan():
            print key,data
        print table
    #except IOError as e:
    except:
        print "Table does not exists,creating"
        self.createTable(table=table,machineIP=machineIP)

    with table.batch() as b:
        with open('xxx.csv','r') as queryFile:

            for lines in queryFile:

                lines=lines.strip("\n")
                splitRecord=lines.split(",")
                key=splitRecord[0]
                key=key.replace("'","")
                val=",".join(splitRecord[1:])
                val=ast.literal_eval(val)
                table.put(splitRecord[0],val)

    for key,data in table.scan():
        print key,data

def createTable(self,table="abc",MachineIP=""):
    connection=happybase.Connection(MachineIP,autoconnect=True)
    print "Connection Handle",connection
    tname=table.lower()
    tablename=str(tname)
    print "Creating table : "+table+", On Hbase machine : "+MachineIP
    families={"cf":{} ,}   #using default column family
    connection.create_table(table,families=families)
    print "Creating table done "

Every time I run this script it populated data to hbase table but it leaves a connection open. When I check using netstat -an I see the connection count has increased which persists even after the script completes.

Am I missing something? Do we need to explicitly close connection?

Thanks for helping.

facing the similar issue it seems there is bug in Happybase if we use in REST model. — Ritesh Srivastava, Sep 06 '17 at 11:46

score 1 · Answer 1 · answered Feb 21 '15 at 18:46

1

Got the solution .Turns out to be this

    try:
        connection.close()
    except Exception as e:
            print "Unable to close connection to hbase "
            print e

answered Feb 21 '15 at 18:46

MultipleCrashes

63
1
9

When a program quits, all open sockets are automatically closed. – wouter bolsterlee May 03 '15 at 12:55
But it is a problem if you open a new connection inside a loop – Leticia Santos Aug 19 '16 at 15:38

score 0 · Answer 2 · answered Apr 12 '15 at 09:55

0

If the program exited, any network connections are automatically closed. What you're likely seeing is the TIME_WAIT state for already closed connections.

answered Apr 12 '15 at 09:55

wouter bolsterlee

3,879
22
30

Closing connection to hbase database using happybase in python

2 Answers2