How to use cur.executemany() to store data from Twitter

Question

I am trying to download tweets from a list of three different accounts and then store all the informations in a SQL3 database.

I have tried with the code below, but it seems to run forever. Am I missing something? Is this because I used .executemany() instead of .execute()?

step=0
a_list=["A","B","C"]
for s in a_list:
    cursor = tweepy.Cursor(api1.user_timeline, id = s, tweet_mode='extended').items(3189)

    for tweet in cursor: 
        tw_text.append(tweet.full_text)
        created_at.append(tweet.created_at)
        rtws.append(tweet.retweet_count)
        favs.append(tweet.favorite_count)
        for h in tweet.entities['hashtags']:
            hashlist.append(h['text'])
        for u in tweet.entities['urls']:
            linklist.append(u['expanded_url'])
        try:
            medialist.append(media['media_url'] for media in tweet.entities['media'])
        except:
            pass 
        step+=1
        print('step {} completed'.format(step))

#preparing all the data for .executemany()               
    g = [(s,tw,crea,rt,fv,ha,li,me) for s in ['GameOfThrones'] for tw in tw_text for crea in created_at for rt in rtws for fv in favs for ha in hashlist for li in linklist for me in medialist] 
    cur.executemany("INSERT INTO series_data VALUES (?,?,?,?,?,?,?,?)", (g))
    con.commit()
    print('db updated')

I expect the program to write table in SQL3 but I never receive the message 'db updated' (i.e. the very last print() line)

score 0 · Answer 1 · answered Apr 10 '19 at 13:02

cur.executemany() takes a list of tuples. Each tuple will have as many elements as number of columns you want to insert value for.

For example, if you have a table with following structure

create table tbl_test(firstname varchar(20), lastname varchar(20));

and you want to insert 3 records in it using executemany(), your object and the call should be like following

list = [('Hans', 'Muster'), ('John', 'Doe'), ('Jane', 'Doe')]
cur.executemany('insert into tbl_test values(?, ?)', list)

How to use cur.executemany() to store data from Twitter

1 Answers1