0

I have a set of websites and their links in this format:

{
"thisite.com" : ["test.com", "example.com"],
"test.com": ["examples.com"]
...
}

How could I turn this into a directed graph easily? I know there are many different libraries, such as NetworkX, but I don't know a way to do this efficiently. I would be turning this graph into an adjacency matrix, so if possible, the library should have a way to do this,

My only solution is this:

def loadgraph(fname):
        G=pg.AGraph(directed=True)
        for line in open(fname):
            j=json.loads(line)
            url=j["url"]
            G.add_node(url)
            for linked_url in j["linkedurls"]:
                G.add_edge(url,linked_url)
        return G

This is not efficient at the scale I would be trying to run this program at. Does anyone know a more efficient way to do this, or would this be the best solution?

R3FL3CT
  • 551
  • 3
  • 14
  • Your adjacency matrix for this is going to be large and sparse. Best to go with an adjacency list for each node. – ravenspoint Jun 30 '23 at 00:51
  • I need an adjacency matrix to run the PageRank algorithm, – R3FL3CT Jun 30 '23 at 00:52
  • I am not familiar with that algorithm. However, any algorithm can be adapted to act on adjacency list instead of a matrix. You just need a method to get the nodes adjacent to any specified node. Whether a matrix or a list is used will be hidden inside the method implementation. – ravenspoint Jun 30 '23 at 00:54
  • The matrix makes the code a lot easier, as I can just do a series of matrix multiplications. – R3FL3CT Jun 30 '23 at 01:00
  • 1
    Yes adjacency matrix are easier to code. They are great for toy problems. For large, serious problems you need adjacency lists. Matrices are not scalable. – ravenspoint Jun 30 '23 at 01:02
  • Have you considered a [sparse matrix representation](https://stackoverflow.com/questions/36969886/using-a-sparse-matrix-versus-numpy-array)? – Stef Jun 30 '23 at 08:44
  • Also could you please include in your post: 1) How many websites are in your list 2) How many links total? – Stef Jun 30 '23 at 08:45

0 Answers0