0

First of all I have to say, I do not want anybody to do my work for me.

I want to make an java application that will search for data on a particular website. It will look like this:

  1. I will type a keyword into my jTextField and than i will press my OK button
  2. after pressing Ok button my application will send request to that site and perform search on it. By searching I mean regular search on websites with textfield and search button. Not ctrl+f search.
  3. After searching the website my app will read the result and store it for example in an array.

How can I start? search on that website looks like this:

 <input id='searchText' type='text' name='text' value='zboží nebo kód' title='title'/>
 <input id='searchButton' type='submit' value='Hledej' />
Andrew Thompson
  • 168,117
  • 40
  • 217
  • 433
user2179427
  • 331
  • 5
  • 16

1 Answers1

0

You will need to check the entire html form or just check the request with the "Network" tab of Firebug or the equivalent built-in tool in Firefox or Chrome to find out if there are other fields, what the URL is and if the site expect an HTTP GET or HTTP POST request.

Then you will need to make such a request using a HTTP client class (Apache HttpClient comes to my mind).

You will get a web site as a result, which you will need to parse, either using a DOM parser or regular expressions.

Jan Schejbal
  • 4,000
  • 19
  • 40
  • I do not really get that part with that "Firebug" plugin for FireFox. Do I have to search if there are other input fields on that website even if I know the "id" of that one I need? – user2179427 May 05 '13 at 11:45
  • You need to make a request that the search function will accept. If it is happy with a request that only contains the "text" field, you are done (just need the form method which you can find in the HTML). Should there be other fields and should the site actually require them, you can easily find them (and acceptable values for them) using firebug. – Jan Schejbal May 05 '13 at 11:47
  • Ok, thank you very much :). Now I can perform search on that site and with bufferReader I am getting response. The thing is, that I am getting whole html. Now I need to search for a particular word in that html source and read the value next to that word. Is it possible? – user2179427 May 05 '13 at 21:25
  • Yes. As I said, you will need to use either a HTML parser or Regular Expressions. – Jan Schejbal May 05 '13 at 21:46
  • Thx. One more question. What i need to do is to check on that website for goods that thay have and compare it with my database. This means I need to perform several requests in a short period of time. Is there any limit of request from one IP per minute? – user2179427 May 05 '13 at 22:10
  • Depends on the site. Do not run more than one request in parallel, and obey their robots.txt, found at the root of their web site (i.e. for example.com, the robots.txt is at http://www.example.com/robots.txt). – Jan Schejbal May 05 '13 at 22:46