0

I have written some code with python to obtain the list of my followers and following users in Twitter. Once I have this information, I create nodes and relationships in neo4j with py2neo by looping over the list of followers and following users that I obtained.

The code seems to work fine, however not all nodes and relationships are created. I am trying to generate about 170 nodes, however only around 25 are created.

I am wondering if there is any kind of connection limit, of uploading threshold or any other thing that might be creating the problem.

I am using Python 3.6, py2neo 3.1.2 and neo4j Community distribution 3.1.3.

I am not a python expert, so please forgive my code:

import py2neo
from py2neo import Graph
from py2neo import Node, Relationship
from py2neo import authenticate
import tweepy
import time

auth = tweepy.OAuthHandler('...', '...')
auth.set_access_token('...', '...')

api = tweepy.API(auth, wait_on_rate_limit=True, wait_on_rate_limit_notify=True)

user = api.get_user(myUser)

def getFollowers(user_screen_name):
    follower_ids=[]
    for page in tweepy.Cursor(api.followers, screen_name=user_screen_name).pages():
        time.sleep(60)
        follower_ids.extend(page)
    return follower_ids

def getFollowing(user_screen_name):
    following_ids=[]
    for page in tweepy.Cursor(api.friends, screen_name=user_screen_name).pages():
        time.sleep(60)
        following_ids.extend(page)
    return following_ids

def createNode(screen_name):   
    node=Node("User", screen_name=screen_name)
    gf.merge(node)
    return

def createRelationship(nodeA, nodeB, relationship):
    #creates relationship and nodes (if not existant)
    nodeA=Node("User", screen_name=nodeA)
    nodeB=Node("User", screen_name=nodeB)
    gf.merge(nodeA)
    gf.merge(nodeB)

    gf.merge(Relationship(nodeA, relationship, nodeB))
    return

authenticate("localhost:7474",myID,myPass)
gf = Graph()    
#First time graph is created
gf.run("CREATE CONSTRAINT ON (u:User) ASSERT u.screen_name IS UNIQUE")

createNode(myUser)

user_followers=getFollowers(myUser)
user_following=getFollowing(myUser)

for followers in user_followers:
    createRelationship(followers.screen_name, sc, "FOLLOWS")

i=1
for following in user_following:
    createRelationship(sc, following.screen_name, "FOLLOWS")

I cannot think of any reason why did would not work, but I believe it is more a problem related to neo4j than the code itself.

Any help would be very much appreciated,

Thanks in advance

cristinik
  • 133
  • 1
  • 10
  • So you have 170 screen names in `user_followers` and `user_following` but only 25 nodes are created? You could try to first create all nodes to see if 170 are written to the database. It's difficult to debug without more information/data. – Martin Preusse Apr 20 '17 at 13:30
  • Hi, I just found the issue. My bad! I was using the web browser to check the nodes and relationships and didn't realize that by default, the cypher query returns LIMIT 25. Thanks a lot anyway! – cristinik Apr 20 '17 at 18:24

0 Answers0