Parse/split URLs in a pandas dataframe using urllib

Question

I'm trying to split URLs and put the fragments in a dataframe. I found this thread pythonic way to parse/split URLs in a pandas dataframe and try to apply it, but for some reason it gives me an error.

I am under Python 3.x so I used the following:

import pandas
import urllib

urls = ['https://www.google.com/something','https://mail.google.com/anohtersomething', 'https://www.amazon.com/yetanotherthing']
df['protocol'],df['domain'],df['path'],df['query'],df['fragment'] = zip(*df['url'].map(urllib.parse.urlsplit))

I get an error saying KeyError: 'urls', not sure what it means.

If someone could help would be great. Thanks.

score 1 · Accepted Answer · answered Feb 22 '18 at 13:33

The example you used assumes that the links are in a dataframe. Here's the correct solution:

import urllib
import pandas as pd

df = pd.DataFrame()
urls = ['https://www.google.com/something','https://mail.google.com/anohtersomething', 'https://www.amazon.com/yetanotherthing']
df['protocol'],df['domain'],df['path'],df['query'],df['fragment'] = zip(*[urllib.parse.urlsplit(x) for x in urls])

Result

  protocol           domain               path query fragment
0    https   www.google.com         /something
1    https  mail.google.com  /anohtersomething
2    https   www.amazon.com   /yetanotherthing

Parse/split URLs in a pandas dataframe using urllib

1 Answers1