hook POST parameters in asp.net

Question

I'm recently mining a website to build some database. I already built a python script parsing retrieved information but the problem is that it requrires a query word to retrieve web pages which contain information I want to see. And this page is in POST method so I cannot see how this page retrieves a page list.

To describe an outline for your clear understanding:

1. on inputKeyword.aspx : This contains a form to input a query(let's say ID)
                        When I input an ID and press search, it retrievs a 
                        relevant list
2. Press Search
3. on inputKeyword.aspx : A relevant list is showed on the same aspx page
                        (which means POST method), so I cannot see how this query
                        works on inputKeyword.aspx page.

It would be so much easier if this webpage is in GET method, since I can simply hook a url with queries, but it's not possible in POST method.

Is there any way that I can open step #3 skipping step #1 and #2?

The webpage is built in asp.net but there's no restriction on languages as long as there's way to do this.

Answer 1

If I understand correctly you want to be able to accept a an ID as part of your query string. eg

http://your.domain.com/inputKeyword.aspx?ID=555

So in the pages load event you can check the request object for query params, ie Request.QueryString[param] as the following example shows

protected void Page_Load(object sender, EventArgs e)
{
    string id = Request.QueryString["ID"];
    if (!string.IsEmptyOrNull(id))
    {
        //do something with the requested identifier
    }       
}

Note: you can use Page.IsPostBack() to determine if the page is being hit for the first time or is posting back as a result of a button click.

To get your Search button to behave correctly you have a couple of options. For example; you can use javascript to capture the buttons onclick event and redirect the page to itself with the url amended to include the identifier from the id textbox.

But perhaps the following is the easiest, keeping the code all server-side:

private _identifer string;

protected void Page_Load(object sender, EventArgs e)
{
    string id = Request.QueryString["ID"];
    if (!string.IsEmptyOrNull(id))
    {
        _identifer = id;
    }
}

protected void SearchButton_Click(object sender, EventArgs e)
{
    _identifer = IdentiferTextbox.Text;
}

protected void Page_PreRender(object sender, EventArgs e)
{
    if (!string.IsEmptyOrNull(_identifer))
    {
        PopulateListForidentifer(_identifer);
    }
}

Basically the example shows that you can cope with scenarios. ASP.Net's page life cycle means that events are processed in the following order Page_Load -> Control Events (eg button click) -> Page PreRender.

• If the page is hit for the first time without an identifier in the url, PopulateListForidentifer method isn't called since _identifer is never set.
• But if the url contains an identifier then _identifer is set in the page load event, when the page pre-render is called PopulateListForidentifer will be called.
• Finally if the page is posting back to itself because the search button has been hit then the click handler is called and _identifer is set to the content of the IdentiferTextbox; the pages prerender is called and also PopulateListForidentifer. Note this would override the point about ie when the identifer was passing as part of the url.

Answer 2

From what I understand, it seems you want to simulate the HTTP Post operation in your Search form, where by without entering the ID and clicking search, you directely want to have access to the search results.

Here is a Blog Post by Scott Hanselman, where he discusses a similar topic using WebClient.

You may also want to check this thread