For each trend in list, pull the 1000 most recent tweets using python-twitter (And remove/exclude retweets)

Question

Using python-twitter library (https://github.com/bear/python-twitter) I am trying to retrieve the 1000 most recent tweets for the top 10 trends in a location using the yahoo Where on earth ID but can't seem to figure out how to retrieve the tweets from the trends in the list.

I don't know if I'm missing something in the documentation and can't find any examples online of how to get trend specific tweets from the list returned and how to remove the retweets.

import twitter
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer


ACCESS_TOKEN = 'XXXX'
ACCESS_TOKEN_SECRET = 'XXXX'
CONSUMER_KEY = 'XXXX'
CONSUMER_SECRET = 'XXXX'

# Define the location id for the UK
WOEID = 23424975
# Define language of tweets
LANG = "en"
# Define type of tweets we are after
TWEETS_TYPE = "recent"
# Define max number of tweets per trend
MAX_STATUSES = 1000

# API config
# API with request rate limited
api = twitter.Api(CONSUMER_KEY,
                  CONSUMER_SECRET,
                  ACCESS_TOKEN,
                  ACCESS_TOKEN_SECRET,
                  sleep_on_rate_limit=True)

print(api.VerifyCredentials())

# Query the Twitter API for the current top 10 trends in the UK.
uk_trends = api.GetTrendsWoeid(WOEID)
print(uk_trends)


# Return the 1000 most recent tweets for each trend
# This is where I want to retrieve the tweets per trend
for trend in uk_trends:
    count = MAX_STATUSES
    trend = trend
    search_results = api.GetSearch(term=trend, count=count)
    print(search_results)

Should I be using the twitter API itself instead?

To remove the retweets, you can exclude them for the search with the q parameter and "-filter:retweets" — JeffProd, May 20 '18 at 07:25

score 0 · Accepted Answer · answered May 28 '18 at 13:02

I was able to solve the problem by accessing the name attribute of the Trend() object, parsing it to a string and iterating through the results returned in my code as follows:

```

import os
import json
import csv

import twitter
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
import pandas as pd
import matplotlib.pyplot as plt

ACCESS_TOKEN = 'XXXX'
ACCESS_TOKEN_SECRET = 'XXXX'
CONSUMER_KEY = 'XXXX'
CONSUMER_SECRET = 'XXXX'

# Define the location id for the UK
# With Yahoo Where on Earth ID
WOEID = 23424975
# Define language of tweets
LANG = "en"
# Define type of tweets we are after
TWEETS_TYPE = "recent"
# Define max number of tweets per trend
MAX_STATUSES = 10

# API configuration
# API with request rate limited
api = twitter.Api(CONSUMER_KEY,
                  CONSUMER_SECRET,
                  ACCESS_TOKEN,
                  ACCESS_TOKEN_SECRET,
                  sleep_on_rate_limit=True)

# Check twitter account api details are correct
# print(api.VerifyCredentials())

# Query the Twitter API for the current top 10 trends in the UK.
uk_trends = api.GetTrendsWoeid(WOEID)

print(uk_trends)


# Return the 1000 most recent tweets for each trend
for trend in uk_trends:
    '''
    Extract name of each trend returned by Trend model
    and search required count value of recent tweets per trend
    '''
    trend = str(trend.name)
    count = MAX_STATUSES
    search_results = api.GetSearch(term=trend, count=count)
    print(search_results)

```

For each trend in list, pull the 1000 most recent tweets using python-twitter (And remove/exclude retweets)

1 Answers1