3

I'm trying to write a chat bot and I want to feed it data from specific subreddits, e.g. https://www.reddit.com/r/leagueoflegends/

I'm already able to crawl all comments while recursively looping 

reddit = praw.Reddit()
submission = reddit.submission(submissionId) #id: https://redd.it/7jjtoq -> "7jjtoq"

I have already tried different approaches to receive (all) submission ids from the subreddit:

subreddits = reddit.subreddits.search_by_name('leagueoflegends', include_nsfw=True, exact=False)

or 

for submission in reddit.get_new(limit=300):
...

Is there any way to receive all of the id's from a subreddit?

alxwrd
  • 2,320
  • 16
  • 28
hi im vinzent
  • 163
  • 2
  • 15
  • 1
    From the docs (https://praw.readthedocs.io/en/latest/code_overview/other/listinggenerator.html): limit – The number of content entries to fetch. If limit is None, then fetch as many entries as possible. Most of reddit’s listings contain a maximum of 1000 items, and are returned 100 at a time. This class will automatically issue all necessary requests (default: 100). – audiodude Dec 13 '17 at 21:45
  • that leads me to my approach of subreddits = reddit.subreddits.search_by_name('leagueoflegends' ...) which returns me a so to say ListingGenerator, how can I utilize the generater properly? type says its a list (is that always when i got a generator?), for item in subreddits: ... is only looping two times, which seems to be wrong taking a look in the leagueoflegends subreddit itself – hi im vinzent Dec 13 '17 at 21:56
  • 1
    What do you mean by "all of the ids"? – audiodude Dec 14 '17 at 01:10
  • e.g.: https://www.reddit.com/r/leagueoflegends/ has a vast amount of submissions, i would like to gather all submission id's of this (or another subreddit), if you follow the link and go to an undfined comment section of a submission you will find e.g.: https://redd.it/7jnfw5, as described i'm already able to crawl all comments recursively there, but to fully automate the crawler i need to find all these submission id's of the subreddit .../r/leagueoflegends – hi im vinzent Dec 14 '17 at 08:27
  • it's more of a one time crawl, but thinking into the future to feed the chat bot current content, being able to make a difference between old and new submissions would make sense (but is not the priority yet) – hi im vinzent Dec 14 '17 at 08:29

1 Answers1

0

It seems praw does not want to crawl subreddits like questioned. praw provides several "get subreddit" methods to receive specific ones or get "popular" or "gold"-categorized ones. I'm now working with scrapy to receive the needed data from specific subreddits.

hi im vinzent
  • 163
  • 2
  • 15