Stack Overflow
Stack Overflow

Questions tagged [urlparse]

urlparse is used for parsing a URL into components like (addressing scheme, network location, path etc.)

urlparse is module in Python2.7 and renamed to urllib.parse in Python 3

Links:

urlparse

urllib.parse

196 questions
1
vote
1 answer

Url Parse is missing fragment - Python

I need to save a file with the name of the given acquisition path's file. Given an URL I would like to parse it and extract the name of the file, here's my code... I read a JSON parameter and give it to the Parse Url function. The acquisition path…
Giulio
  • 150
  • 2
  • 11
1
vote
1 answer

What is "G-d intended"?

I see G-d intended below. I don't understand what it means does anybody know? Thanks. Help on function parse_qsl in module urlparse: parse_qsl(qs, keep_blank_values=0, strict_parsing=0) Parse a query given as a string argument. …
user1424739
  • 11,937
  • 17
  • 63
  • 152
1
vote
1 answer

How to scrape two URLs and and put the elements of each url in one single table?

I have two URLs of the same page that I want to scrape to get room prices in NYC. I used BeautifulSoup to get the address, price, and availability of each room. After that, I make a dictionary so that I can create a DataFrame. I am getting two…
Diego
  • 386
  • 4
  • 19
1
vote
1 answer

Pass file path as parameter in Flask app.route()

Here is my code. I'm trying to take the file path in as a parameter but the "/" in the path between directories is making the method treat the directories as separate parameters. from flask import Flask, abort, redirect, url_for,…
smokscreen_145
  • 13
  • 1
  • 5
1
vote
0 answers

Why does python present the url (instead of base combined with url) when a url starts with mailto?

Why does python present the url (instead of base combined with url) when a url starts with mailto? This is what happened: from urllib.parse import urljoin >>> urljoin('http://www.w3.org/Consortium/mission.html', 'mailto:site-comments@w3.org') …
Hibisceae
  • 33
  • 2
  • 8
1
vote
2 answers

Website Name extract in Python

I want to extract website names from the url. For e.g. https://plus.google.com/in/test.html should give the output as - "plus google" Some more testcases are - WWW.OH.MADISON.STORES.ADVANCEAUTOPARTS.COM/AUTO_PARTS_MADISON_OH_7402.HTML Output:-…
Anubhav Sarangi
  • 179
  • 1
  • 5
  • 15
1
vote
2 answers

Parse URL in Azure functions

Am learning how to use Azure functions and so new to it. I have a httptrigger Azure function written in NodeJs. I am thinking of a logic on how to parse data from httptrigger function URL and use it in my code. Would like some suggestions on…
Ross
  • 99
  • 2
  • 12
1
vote
1 answer

Python 3 urlib urlparse URI parsing

I'm a little bit puzzled. I hope somebody would help me =) Python urlparse function result depends on a scheme that was specified in a URI. For example, this call returns '/path;' urllib.parse.urlparse('some://foo.bar/path;').path But this call…
Ildar Gafurov
  • 13
  • 1
  • 4
1
vote
2 answers

How can I append null in list when particular parameter is absent for some lines while iterating through log file using urlparser

I want my file to get parsed by url but some of the url have missing parameters and when I am iterating through lines of log I am getting error of missing parameter. I need to append blank or null value into in parse list so that I can transform it…
ni3
  • 43
  • 2
  • 6
1
vote
1 answer

Scrapy - Does urlparse.urljoin behave in the same way as str.join?

I am trying to use urlparse.urljoin within a Scrapy spider to compile a list of urls to scrape. Currently, my spider is returning nothing, but not throwing any errors. So I am trying to check that I am compiling the urls corectly. My attempt was to…
Maverick
  • 789
  • 4
  • 24
  • 45
1
vote
2 answers

Replacing ONLY domain in Python

I have been using a regex that searches a document for all URLS and replaces them but now I want to only replace the hostname, not the subdomain or any other part of the URL. For example I want https://ftp.website.com > https://ftp.mything.com This…
zek
  • 11
  • 3
1
vote
1 answer

urlparse: ModuleNotFoundError, presumably in Python2.7 and under conda

I am attempting to run my own scrapy project. The code is based off a well written book and the author provides a great VM playground to run scripts exampled in the book. In the VM the code works fine. However, in an attempt to practice on my…
R.Zane
  • 350
  • 1
  • 6
  • 16
1
vote
1 answer

Python urlparser Gives wrong result

I'm trying to separate the different parts of a url with python s urlparse, but I'm seeming to get the wrong values in the results. baseline = runSql(conn,"Select url from malware_traffic where tag = 'baseline';") for i in baseline: print i[0] …
MikeSchem
  • 950
  • 2
  • 16
  • 29
1
vote
1 answer

Can I convert a dict to a url?

For instance I have the following url pattern: url_pattern = { "scheme": "http", "netloc": "for.example.com", "path": "/ex1", "params": "", "query": a=b&c=d, "fragment": "" } It's just like the inverse of the output of…
Lerner Zhang
  • 6,184
  • 2
  • 49
  • 66
1
vote
3 answers

Connecting ElephantSQL with Python

I'd like to connect ElephantSQL with Python. import os import psycopg2 import urlparse urlparse.uses_netloc.append("postgres") url = urlparse.urlparse(os.environ["MY_DATABASE_URL"]) conn = psycopg2.connect(database=url.path[1:], …
Krasnal
  • 85
  • 2
  • 9
1 2 3
…
13 14