3

Python 3.7.2 PyCharm

I'm fairly new to Python, and API interaction; I'm trying to loop through the API for Rocket Chat, specifically pulling user email address's out. Unlike nearly every example I can find, Rocket Chat doesn't use any kind of construct like "Next" - it uses count and offset, which I had actually though might make this easier. I have managed to get the first part of this working, looping over the JSON and getting the emails. What I need to do, is loop through the API endpoints - which is what I have ran into some issue with.
I have looked at this answer Unable to loop through paged API responses with Python as it seemed to be pretty close to what I want, but I couldn't get it to work correctly.

The code below, is what I have right now; obviously this isn't doing any looping through the API endpoint just yet, its just looping over the returned json.

import os
import csv
import requests
import json

url = "https://rocketchat.internal.net"
login = "/api/v1/login"
rocketchatusers = "/api/v1/users.list"
#offset = "?count=500&offset=0"


class API:

    def userlist(self, userid, token):
        headers = {'X-Auth-Token': token, 'X-User-Id': userid}
        rocketusers = requests.get(url + rocketchatusers, headers=headers, verify=False)
        print('Status Code:' + str(rocketusers.status_code))
        print('Content Type:' + rocketusers.headers['content-type'])
        userlist = json.loads(rocketusers.text)
        x = 0
        y = 0
        emails = open('emails', 'w')
        while y == 0:
            try:
                for i in userlist:
                    print(userlist['users'][x]['emails'][0]['address'], file=emails)
                    # print(userlist['users'][x]['emails'][0]['address'])
                    x += 1
            except KeyError:
                print("This user has no email address", file=emails)
                x += 1
            except IndexError:
                print("End of List")
                emails.close()
                y += 1

What I have tried and what I would like to do, is something along the lines of an easy FOR loop. There are realistically probably a lot of ways to do what I'm after, I just don't know them.

Something like this:

import os
import csv
import requests
import json

url = "https://rocketchat.internal.net"
login = "/api/v1/login"
rocketchatusers = "/api/v1/users.list"
offset = "?count=500&offset="+p
p = 0

class API:

    def userlist(self, userid, token):
        headers = {'X-Auth-Token': token, 'X-User-Id': userid}
        rocketusers = requests.get(url + rocketchatusers+offset, headers=headers, verify=False)

        for r in rocketusers:
            print('Status Code:' + str(rocketusers.status_code))
            print('Content Type:' + rocketusers.headers['content-type'])
            userlist = json.loads(rocketusers.text)
            x = 0
            y = 0
            emails = open('emails', 'w')
            while y == 0:
                try:
                    for i in userlist:
                        print(userlist['users'][x]['emails'][0]['address'], file=emails)
                        # print(userlist['users'][x]['emails'][0]['address'])
                        x += 1
                except KeyError:
                    print("This user has no email address", file=emails)
                    x += 1
                except IndexError:
                    print("End of List")
                    emails.close()
                    y += 1
        p += 500

Now, obviously this doesn't work, or I'd not be posting, but the why it doesn't work is the issue. The error that get report is that I can't concatenate an INT, when a STR is expected. Ok, fine. When I attempt something like:

str(p = 0) I get a type error. I have tried a lot of other things as well, many of them simply silly, such as p = [], p = {} and other more radical idea's as well.

The URL, if not all variables and concatenated would look something like this:

https://rocketchat.internal.net/api/v1/users.list?count=500&offset=0
https://rocketchat.internal.net/api/v1/users.list?count=500&offset=500
https://rocketchat.internal.net/api/v1/users.list?count=500&offset=1000
https://rocketchat.internal.net/api/v1/users.list?count=500&offset=1500

I feel like there is something really simple that I'm missing. I'm reasonably sure that the answer is in the response to the post I listed, but I couldn't get it to work.

2rojan
  • 51
  • 4

1 Answers1

2

So, after asking around, I found out that I had been on the right path to figuring this issue out, I had just tried in the wrong place. Here's what I ended up with:

def userlist(self, userid, token):
    p = 0
    while p <= 7500:
        if not os.path.exists('./emails'):
            headers = {'X-Auth-Token': token, 'X-User-Id': userid}
            rocketusers = requests.get(url + rocketchatusers + offset + str(p), headers=headers, verify=False)
            print('Status Code:' + str(rocketusers.status_code))
            print('Content Type:' + rocketusers.headers['content-type'])
            print('Creating the file "emails" to use to compare against list of regulated users.')
            print(url + rocketchatusers + offset + str(p))
            userlist = json.loads(rocketusers.text)
            x = 0
            y = 0
            emails = open('emails', 'a+')
            while y == 0:
                try:
                    for i in userlist:
                        #print(userlist['users'][x]['emails'][0]['address'], file=emails)
                        print(userlist['users'][x]['ldap'], file=emails)
                        print(userlist['users'][x]['username'], file=emails)
                        x += 1
                except KeyError:
                    x += 1
                except IndexError:
                    print("End of List")
                    emails.close()
                    p += 50
                    y += 1
        else:
            headers = {'X-Auth-Token': token, 'X-User-Id': userid}
            rocketusers = requests.get(url + rocketchatusers + offset + str(p), headers=headers, verify=False)
            print('Status Code:' + str(rocketusers.status_code))
            print('Content Type:' + rocketusers.headers['content-type'])
            print('Populating file "emails" - this takes a few moments, please be patient.')
            print(url + rocketchatusers + offset + str(p))
            userlist = json.loads(rocketusers.text)
            x = 0
            z = 0
            emails = open('emails', 'a+')
            while z == 0:
                try:
                    for i in userlist:
                        #print(userlist['users'][x]['emails'][0]['address'], file=emails)
                        print(userlist['users'][x]['ldap'], file=emails)
                        print(userlist['users'][x]['username'], file=emails)
                        x += 1
                except KeyError:
                    x += 1
                except IndexError:
                    print("End of List")
                    emails.close()
                    p += 50
                    z += 1

This is still a work in progress, unfortunately, this isn't an avenue for collaboration, later I may post this to GitHub so that others can see it.

2rojan
  • 51
  • 4