Passing a variable in url?

Question

So I'm new in python and I desperately need help.

I have a file which has a bunch of ids (integer values) written in 'em. Its a text file.

Now I need to pass each id inside the file into a url.

For example "https://example.com/[id]"

It will be done in this way

A = json.load(urllib.urlopen("https://example.com/(the first id present in the text file)"))
print A

What this will essentially do is that it will read certain information about the id present in the above url and display it. I want this to work in a loop format where in it will read all the ids inside the text file and pass it to the url mentioned in 'A' and display the values continuously..is there a way to do this?

I'd be very grateful if someone could help me out!

pyfunc · Accepted Answer · 2012-06-21T07:03:40.607

30

Old style string concatenation can be used

>>> id = "3333333"
>>> url = "https://example.com/%s" % id
>>> print url
https://example.com/3333333
>>>

The new style string formatting:

>>> url = "https://example.com/{0}".format(id)
>>> print url
https://example.com/3333333
>>>

The reading for file as mentioned by avasal with a small change:

f = open('file.txt', 'r')
for line in f.readlines():
    id = line.strip('\n')
    url = "https://example.com/{0}".format(id)
    urlobj = urllib.urlopen(url)
    try:
        json_data = json.loads(urlobj)
        print json_data
    except:
        print urlobj.readlines()

edited Jun 21 '12 at 07:03

answered Jun 21 '12 at 05:17

pyfunc

65,343
15
148
136

Thank you very much for helping me out. I'm facing a new problem now. Suppose I try to read a text file with say, just a handful of ids, it has no problem in executing. But my text file from where I'm reading the ids has a LOT of them. And the code fails to execute. What is the reason for this? Is there a way to get around this? – user1452759 Jun 21 '12 at 06:47
@user1452759: What is the error that you are getting? It would be nice to start from there. The solution should work for even if you have many ids. Chances are that not all the url fetch is returning data that can be loaded into json – pyfunc Jun 21 '12 at 06:55
@user1452759: Try the solution as I have mentioned in edited example. It will check if data can be loaded as json or else will print the output of what was fetched. – pyfunc Jun 21 '12 at 07:04

xvatar · Answer 2 · 2012-06-21T05:29:00.360

lazy style:

url = "https://example.com/" + first_id

A = json.load(urllib.urlopen(url))
print A

old style:

url = "https://example.com/%s" % first_id

A = json.load(urllib.urlopen(url))
print A

new style 2.6+:

url = "https://example.com/{0}".format( first_id )

A = json.load(urllib.urlopen(url))
print A

new style 2.7+:

url = "https://example.com/{}".format( first_id )

A = json.load(urllib.urlopen(url))
print A

Jainik · Answer 3 · 2021-02-08T16:15:35.203

Python 3+

New String formatting is supported in Python 3 which is a more readable and better way to format a string. Here's the good article to read about the same: Python 3's f-Strings

In this case, it can be formatted as

url = f"https://example.com/{id}"

Detailed example

When you want to pass multiple params to the URL it can be done as below.

name = "test_api_4"
owner = "jainik@test.com"

url = f"http://localhost:5001/files/create" \
f"?name={name}" \
f"&owner={owner}" \

We are using multiple f-string here and they can be appended by ''. This will keep them in the same line without inserting any new line character between them.

For values which have space

For such values you should import from urllib.parse import quote in your python file and then quote the string like: quote("firstname lastname")

This will replace space character with %20.

And any clue for value with `/` in it? Like dataset ids in Freenas? — Kevin Lemaire, Nov 18 '20 at 16:49

score 2 · Answer 4 · answered Jun 21 '12 at 05:21

The first thing you need to do is know how to read each line from a file. First, you have to open the file; you can do this with a with statement:

with open('my-file-name.txt') as intfile:

This opens a file and stores a reference to that file in intfile, and it will automatically close the file at the end of your with block. You then need to read each line from the file; you can do that with a regular old for loop:

  for line in intfile:

This will loop through each line in the file, reading them one at a time. In your loop, you can access each line as line. All that's left is to make the request to your website using the code you gave. The one bit your missing is what's called "string interpolation", which allows you to format a string with other strings, numbers, or anything else. In your case, you'd like to put a string (the line from your file) inside another string (the URL). To do that, you use the %s flag along with the string interpolation operator, %:

url = 'http://example.com/?id=%s' % line
A = json.load(urllib.urlopen(url))
print A

Putting it all together, you get:

with open('my-file-name.txt') as intfile:
  for line in intfile:
    url = 'http://example.com/?id=%s' % line
    A = json.load(urllib.urlopen(url))
    print A

Thank you very much for helping me out. I'm facing a new problem now. Suppose I try to read a text file with say, just a handful of ids, it has no problem in executing. But my text file from where I'm reading the ids has a LOT of them. And the code fails to execute. What is the reason for this? Is there a way to get around this? — user1452759, Jun 21 '12 at 06:44
Could you post the error message? It's hard to say without knowing what it says the problem is. — Haldean Brown, Jun 21 '12 at 13:12

Passing a variable in url?

4 Answers4