Trying update Twitter status with scraper data using Twython. Unsure on what to do

Question

So I have these two scripts:

redditScraper.py

# libraries
import urllib2
import json

# get remote string
url = 'http://www.reddit.com/new.json?sort=new'
response=urllib2.urlopen(url)

# interpret as json
data = json.load(response)
#print(data)
response.close()


print data['data']['children'][3]['data']['title']
print data['data']['children'][3]['data']['permalink']
print data['data']['children'][3]['data']['subreddit']

and minerTweets.py

#!/usr/bin/env python
import sys
from twython import Twython
CONSUMER_KEY = 'XXXXXXXXXXXXXXXX'
CONSUMER_SECRET = 'XXXXXXXXXXXXXXXX'
ACCESS_KEY = 'XXXXXXXXXXXXXXXX'
ACCESS_SECRET = 'XXXXXXXXXXXXXXXX'

api = Twython(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_KEY,ACCESS_SECRET) 

api.update_status(status=sys.argv[1])

This is a Raspberry Pi that will update a twitter account (it's for academic purposes). Being new to python I took each part of the script I'm trying to write one at a time. I have one script that successfully scraps the title, link, and subreddit of the reddit "new" page and prints it. And then I have another that successfully hits the Twython API to update a status taking the sys.argv currently for testing. What I want the finished script to do is take the printed data from the redditScraper.py and update a twitter account's status with my minerTweets.py script. I've looked all over the place and since I'm just learning python my knowledge for the best way to accomplish this is limited.

I appreciate any advice in advance. Thank you!

aIKid · Accepted Answer · 2013-12-01T23:21:18.237

0

You can store the results of redditScrapper.py to a file, and let minerTweets.py take the data from there:

with open('test.txt', 'w') as fp:
    data = json.load(response)
    json.dump(data, fp)

With test.txt stored in the same directory. Now, the only thing left is to read it:

with open('test.txt') as fp:
    api = Twython(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_KEY,ACCESS_SECRET) 
    data = json.load(fp)
    api.update_status(data['data']['children'][3]['data']['title'])

Edit: If you want to merge the script, it's not so hard.

import urllib2
import json
from twython import Twython

CONSUMER_KEY = 'XXXXXXXXXXXXXXXX'
CONSUMER_SECRET = 'XXXXXXXXXXXXXXXX'
ACCESS_KEY = 'XXXXXXXXXXXXXXXX'
ACCESS_SECRET = 'XXXXXXXXXXXXXXXX'

# get remote string
url = 'http://www.reddit.com/new.json?sort=new'
response=urllib2.urlopen(url)

# interpret as json
data = json.load(response)
#print(data)
response.close()

api = Twython(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_KEY,ACCESS_SECRET) 

d = data['data']['children'][3]['data']
title = d['title']
permalink = d['permalink']
subreddit = d['subreddit']

api.update_status(status=title+permalink+subreddit) #or you can post this as different 
#status, depends on how you'd like to format the tweet.

edited Dec 01 '13 at 23:21

answered Dec 01 '13 at 22:52

aIKid

26,968
4
39
65

Is there a way to have it all run within the same script maybe? – oddnonymous Dec 01 '13 at 23:11
Ah.. that's what you wanted.. Yeah, it's even easier. – aIKid Dec 01 '13 at 23:17
Yes it is and that makes sense now. However, I get this message when I run the script in the terminal: File "/usr/lib/python2.7/urllib2.py", line 528, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 429: Unknown – oddnonymous Dec 01 '13 at 23:33
There is more too it, but it won't let me post the rest :/ – oddnonymous Dec 01 '13 at 23:35
Oh wait! I figured it out. The post is too long, so I'll have to post less than what I have here. Thanks so much for the help! – oddnonymous Dec 01 '13 at 23:38

Trying update Twitter status with scraper data using Twython. Unsure on what to do

1 Answers1