No output displays when execute python file

Question

Getting a blank screen on executing the python program.

Please help. It may be a duplicate question, but I don't know Python very much, because I am an Android developer.

Here is my code:

import sys
import requests
from bs4 import BeautifulSoup, SoupStrainer

home_url = 'https://parivahan.gov.in/rcdlstatus/'
post_url = 'https://parivahan.gov.in/rcdlstatus/vahan/rcDlHome.xhtml'
# Everything before the last four digits: GJ03KA
first = sys.argv[1]
# The last four digits: 0803
second = sys.argv[2]

r = requests.get(url=home_url)
cookies = r.cookies
soup = BeautifulSoup(r.text, 'html.parser')
viewstate = soup.select('input[name="javax.faces.ViewState"]')[0]['value']

data = {
    'javax.faces.partial.ajax':'true',
    'javax.faces.source': 'form_rcdl:j_idt32',
    'javax.faces.partial.execute':'@all',
    'javax.faces.partial.render': 'form_rcdl:pnl_show form_rcdl:pg_show form_rcdl:rcdl_pnl',
    'form_rcdl:j_idt32':'form_rcdl:j_idt32',
    'form_rcdl':'form_rcdl',
    'form_rcdl:tf_reg_no1': first,
    'form_rcdl:tf_reg_no2': second,
    'javax.faces.ViewState': viewstate,
}

r = requests.post(url=post_url, data=data, cookies=cookies)
soup = BeautifulSoup(r.text, 'html.parser')
table = SoupStrainer('tr')
soup = BeautifulSoup(soup.get_text(), 'html.parser', parse_only=table)
print(soup.get_text())

Answer 1

If you print out the result from the requests post (r), you're getting a 500 error which is a generic http response for a server error. My guess is the url resource is bad or the data being posted to it isn't formatted correctly

Answer 2

Let me open a new answer in response to the renewed question.

After trying some methods with just requests and urllib, I think it is better to use the selenium webdriver controller.

The following code will grab the table rows as you want.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from bs4 import BeautifulSoup

url = 'https://parivahan.gov.in/rcdlstatus/'

# Optional: Getting "Headless" browser, ie suppressing the browser window from showing
chrome_options = Options()  
chrome_options.add_argument("--headless")  

# Let the driver open, fill and submit the form
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get(url)
driver.delete_all_cookies()
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.NAME, 'form_rcdl:j_idt34')))
input1 = driver.find_element_by_name('form_rcdl:tf_reg_no1')
input1.send_keys('GJ03KA')
input2 = driver.find_element_by_name('form_rcdl:tf_reg_no2')
input2.send_keys('0803')
driver.find_element_by_name('form_rcdl:j_idt34').click()
wait = WebDriverWait(driver, 10)

# Get the result table
try:
    element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.ID, "form_rcdl:j_idt63"))
    )
    result_html = driver.page_source
    #print(result_html)
    soup = BeautifulSoup(result_html, 'lxml')
    print(soup.findAll('tr'))
except TimeoutException:
    driver.quit()
    print('Time out.')

Below demonstrates the result of printing out the table html tags in soup.

I hope the government does not find out and block this way before you try out lol

Hope this helps! You may refer to the following references if interested:

• Selenium manual: http://selenium-python.readthedocs.io/
• Driving headless Chrome with Python: https://duo.com/decipher/driving-headless-chrome-with-python
• Submitting form with selenium: Selenium/Python - Extract dynamically generated HTML after submitting form

Answer 3

The URL that actually returns a valid form webpage is 'https://parivahan.gov.in/rcdlstatus/'.

By inputting the example ID (Reg No.) in browser, error message "Registration No. does not exist!!! Please check the number." pops up. (which makes total sense. I do hope you didn't put a real ID in public lol)

Since I don't have a valid ID to test. Please see if this solves your problem.

Another thing noticed is that the fields for inputting the registration number should be "form_rcdl:tf_reg_no1" and "form_rcdl:tf_reg_no2". You can view the HTML source of the webpage (e.g. Ctrl+C in Chrome) to verify.

Answer 4

You have hardcoded jdt32 as button id... please note that button id in this website is dynamic.... your program should dynamically pickup the right button id. here is the solution

import sys
import re
import requests
from bs4 import BeautifulSoup, SoupStrainer

home_url = 'https://parivahan.gov.in/rcdlstatus/?pur_cd=102'
post_url = 'https://parivahan.gov.in/rcdlstatus/vahan/rcDlHome.xhtml'
# Everything before the last four digits: MH02CL
first = sys.argv[1]
# The last four digits: 0555
second = sys.argv[2]

r = requests.get(url=home_url)
cookies = r.cookies
soup = BeautifulSoup(r.text, 'html.parser')
viewstate = soup.select('input[name="javax.faces.ViewState"]')[0]['value']
#print soup.findAll('button', id=re.compile('form_rcdl^'))
#print soup.findAll('button', id=lambda x: x and x.startswith('form_rcdl'))
i = 0
for match in soup.find_all('button', id=re.compile("form_rcdl")):
  if i ==  0:
    button_id= match.get('id')
  i = 1

data = {
    'javax.faces.partial.ajax':'true',
    'javax.faces.source':button_id,
    'javax.faces.partial.execute':'@all',
    'javax.faces.partial.render': 'form_rcdl:pnl_show form_rcdl:pg_show form_rcdl:rcdl_pnl',
    button_id:button_id,
    'form_rcdl':'form_rcdl',
    'form_rcdl:tf_reg_no1': first,
    'form_rcdl:tf_reg_no2': second,
    'javax.faces.ViewState': viewstate,
}

r = requests.post(url=post_url, data=data, cookies=cookies)
#print (r.text)
soup = BeautifulSoup(r.text, 'html.parser')
table = SoupStrainer('tr')
soup = BeautifulSoup(soup.get_text(), 'html.parser', parse_only=table)
print(soup.get_text())