Twython load 3200 tweets and encode it (Cyrillic text)

Question

I wrote small program on Python 3.4.2 with Twython module (3.1.2). It works, but not exactly right. Program loads tweets and display them (you can see code below). It loaded 200 tweets, but I read that Twitter API can give access to 3200 tweets. Can I do this with Twython ? And the main problem - tweets load in unreadable form :( I noticed that this is due to the fact that tweets were written on the Cyrillic.

import sys
from twython import Twython

APP_KEY = 'MY_APP_KEY'
APP_SECRET = 'MY_APP_SECRET'
OAUTH_TOKEN = 'MY_OAUTH_TOKEN'
OAUTH_TOKEN_SECRET = 'MY_OAUTH_TOKEN_SECRET'

client_args = {
  "headers": {
    "accept-charset": "utf-8"
  }
}

twitter = Twython(APP_KEY, APP_SECRET,
                  OAUTH_TOKEN, OAUTH_TOKEN_SECRET)

user_tweets = twitter.get_user_timeline(screen_name='poroshenko',
                                        count=200)

for tweets in user_tweets:
   print (tweets['text'].encode('utf-8'))

score 0 · Answer 1 · answered Oct 26 '14 at 07:22

0

Change count= to 3200

For printing tweets see https://twython.readthedocs.org/en/latest/usage/special_functions.html

Use:

for tweet in user_tweets:
    tweet['text'] = Twython.html_for_tweet(tweet)
    print(tweet['text'])

answered Oct 26 '14 at 07:22

sunshinekitty

2,307
6
19
31

This does not help. If change count= to 3200, still get only 200 values. In Twitter API documentation says that need provide multiple request, but how it makes in Twython? – Daniel Petrov Oct 26 '14 at 11:55
About .html_for_tweet(tweet) - this is only html format for printing. And if I don't use encode('utf-8'), I get error message. `Traceback (most recent call last): File "C:\Python\Python3\projects\twitter.py", line 25, in print (tweets['text']) File "C:\Python\Python3\lib\encodings\cp1251.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f631' in position 89: character maps to ` – Daniel Petrov Oct 26 '14 at 12:00
The encoding error is likely relative to your terminal. If you're on Windows run `chcp 65001` – sunshinekitty Oct 26 '14 at 23:14

score 0 · Answer 2 · answered Aug 06 '15 at 03:39

You have to play with the max_id parameter.

Once you get your first 200 tweets, you get the id of the last tweet (the oldest one) and you call the twitter API again. Let's assume this tweet id is lastId

user_tweets = twitter.get_user_timeline(screen_name='poroshenko',count=200,max_id=str(long(lastId)-1))

Make sure that you also take care of handling when you exceed the rate limit.

Twython load 3200 tweets and encode it (Cyrillic text)

2 Answers2