django search field redering a request from another site url

Question

How can i use django to create a search field that will be able to perform a request from another website and render back some information from the site. for example if i submit this form number (PGA1512194112) on the search field, the app will be able to go to https://mis.unijos.edu.ng/pg/admissions/, search for PGA1512194112 and return back PGA1512194112's fullname, form number, faculty, department, programme and session.

Example(this is not a demo) of the view

    def form(request):
        query = request.GET.get['q']
        if query is valid
            return HttpResponseRedirect(form2)
        else
            search page
        context = {'query': query}
        return render(request,'.html', context)

Note: I dont even have idea of how the format of the view will look like.

' def form(request):

        if request.GET:
            url = 'URL'
            query = request.GET.get('q')
            resp = requests.post(url, data=query)
            print ('resp.content')
        else:
            query=''
        context = {'query': query, 'resp': resp}
        return render(request, 'library/terms.html', context)'

I might be wrong, but I think you need this site permission to do that or the site's api. — Gabriel Belini, May 31 '17 at 14:32
Usually that is what API is used for, so that you have accessible endpoint that you can simple formated query and retrieve simple formated response. If you do not have this option you will probably have to send request using requests library and then parse response html with soup for particular value. Will be even more horrible if request is resolved using javascript (which it looks like on this site) — Tomasz Plaskota, May 31 '17 at 14:33
you should be able to do using the web scraping technique. There are lot of scraping techniques in python. try with `selenium` or `phantomjs` etc... You also see if there is any publicly published rest-api for the website. — Hara, May 31 '17 at 14:33

score 0 · Answer 1 · answered May 31 '17 at 14:38

One way to do this, which I do not know if this is "legal" or not is:

Use selenium package to be able to input your text into the website search box, then selenium will be able to access the resulting webpage and finally you will be able to perform a request to the website using url package or whatever you want to retrieve the whole html from the website.

The information of the resulting webpage should always be in the same format so it shouldn't be too hard to parse it and retrieve only what you need. You can also use beautifulsoup to have more tools to parse it.

score 0 · Answer 2 · answered May 31 '17 at 15:21

0

If you have permission to use their API, the following will work:

import requests

url = 'https://mis.unijos.edu.ng/pg/admissions/ajax/ajax_get_admission.php'
data = {
    'q': 'PGA1512194112'
}
resp = requests.post(url, data=data) 

print resp.content

This returns html, but it shouldn't be too hard to parse the resulting html to get the information you're looking for.

answered May 31 '17 at 15:21

Zachary Blackwood

558
2
9

thank you for your comment. but am i to add the code to my view or where – LORD MILA Jun 04 '17 at 17:16
Even if i can get something that will help me to determine if the form number exist in the php url. something like if number exist in https://mis.unijos.edu.ng/pg/admissions/ return to a form page else return to search page. – LORD MILA Jun 04 '17 at 17:32
@LORDMILA This may be helpful: https://stackoverflow.com/questions/16904173/writing-a-very-basic-search-form-in-django My code could go in a view, something like the `search` view in that example – Zachary Blackwood Jun 06 '17 at 20:04
the search link above is something similar to what i want but i dont know how to capture the url on the request.Get. An example of how to do that will be better. thank you – LORD MILA Jun 06 '17 at 23:20
@LORDMILA Do you mean the url containing the results of the query? You don't need to capture the url to get the results. In my code above, resp.content contains the results of the query. – Zachary Blackwood Jun 07 '17 at 13:47
Am still confused of how to design the view. can you please show me how with a full example of how it should look like. thank you – LORD MILA Jun 08 '17 at 20:42
I tried you sample above and its giving me the error below: ValueError at /forms/ The view peruse.views.form didn't return an HttpResponse object. It returned None instead. – LORD MILA Jun 13 '17 at 12:14
' def form(request): if request.GET: url = 'https://mis.unijos.edu.ng/pg/admissions/ajax/ajax_get_admission.php' query = request.GET.get('q') resp = requests.post(url, data=query) print ('resp.content') else: query='' context = {'query': query, 'resp': resp} return render(request, 'library/terms.html', context)' – LORD MILA Jun 14 '17 at 08:39
I was wondering if the code will be in this format. but the code ontop is useless because is not functioning – LORD MILA Jun 14 '17 at 08:43

django search field redering a request from another site url

2 Answers2