How to insert async into MySQL using Python

Question

solved,every greenlet should have one connection rather than share the same connection.

I want to insert a lot data into a MySQL database. I use gevent to download data from the internet, then insert the data into MySQL. I found umysqldb to insert into MySQL async. However I am getting the following error: Mysql Error 0: Concurrent access in query method.

My code is following:

def insert_into_mysql(conn,cur,pid,user,time,content):
    try: 
        value=[pid,user,time,content] 
        #print 'value is', value
        print 'hi'
        cur.execute('insert into post(id,user,time,content) values(%s,%s,%s,%s)',value) 
        print 'after execute'
        conn.commit() 
    # except MySQLdb.Error,e: 
    except umysqldb.Error,e: 
        print "Mysql Error %d: %s" % (e.args[0], e.args[1])

insert_into_mysql is included in download_content:

while len(ids_set) is not 0:
    id = ids_set.pop()
    print 'now id is', id
    pool.spawn(download_content,conn,cur,int(id))
    r.sadd('visited_ids',id)
    pool.join()

you can monkey patch standard mysqldb driver and it is should be async also see https://github.com/esnme/ultramysql — Denis, Nov 09 '12 at 13:18
`is not 0`? So when integer interning changes it's going to break. — doug65536, Jul 01 '22 at 16:27

score 0 · Answer 1 · answered May 24 '13 at 00:09

ultramysql doesn't allow you to make multiple queries on the same mysql connection, it just makes it async friendly. So you will either need a new mysql connection for each greenlet or use locking primitives to makes sure only one greenlet is using the connection at a time.

How to insert async into MySQL using Python

solved,every greenlet should have one connection rather than share the same connection.

1 Answers1