1

What is the difference between the way a browser calls the URL and doing it via Rest Debugger or HTTP Components? I have a 3rd party Web REST API that work every time in a browser (IE it returns JSON as expected), but when I use (GET) the same URL in the Delphi REST Debugger it returns error code 429 Too Many Requests. I am not allowed to post the exact URL here (I'm sorry, boss has the last say but it is like this https://xxxx.yyyy.com.au/search/resources/store/zzzzz/productview/123456).

For additional information the result is consistent giving the 429 error when I use NetHTTPClient and NetHTTPRequest components as well as using the Delphi REST Components. I thought that setting the user agent to be the same as my browsers might help, but alas it didn't. I use Delphi 10.3.3 Rio

I'm a bit new to REST and haven't found an answer by googling for a couple of days now. Any help will be most appreciated. Thanks, John

The answer is cookies. When I rejected all cookies I could see the behavior as stated by @RemyLebeau where the page is in a continuous loop. The browser sends a cookie with the request header. I'm new to all of this, so I'll try to replicate what the browser is doing and see what happens. If I get really stuck I'll post another question specifically about cookies. Many thanks to all who offered advice. Most appreciated. I put this here because someone deleted this as an answer.

JohnT
  • 31
  • 5
  • My guess is probably an HTTP redirect that is getting stuck in an endless loop – Remy Lebeau Sep 08 '20 at 04:43
  • @RemyLebeau Thanks for that, but why would that loop not happen when using a browser? I have Max Redirects set to 5 in the NetHTTPClient component as well? Is there anyway that I can emulate the browser so that I get my JSON? – JohnT Sep 08 '20 at 05:13
  • 1
    You can send the exact same request as the browser. For that, you need to know all the involved request which is not very easy. One possibility is to install WireShark (or similar) to grab all requests from the browser and from your program, compare and update your program to generate the same request. – fpiette Sep 08 '20 at 06:13
  • @fpiette Thank you. Do you mean that I should send the same HTTP Request header? If that's the case what is the correct way to construct the header and send it with the request? ( I can install wire shark or fiddler or something but don't know the correct way to construct the header or send the header with the request – JohnT Sep 08 '20 at 06:20
  • This depends on the component you use. I use ICS components, not the one you mention. I suggest you first analyse the differences between the browser request and the component you use and then ask another question related to that exact difference (Start a new question on Stack Overflow). – fpiette Sep 08 '20 at 06:29
  • @fpiette "*For that, you need to know all the involved request which is not very easy*" - actually, it is fairly easy, given that modern browsers have built in debuggers that show all of the requests they make in detail. – Remy Lebeau Sep 08 '20 at 07:14
  • Does the API require authentication? When you try in the browser, are you logged in on the site? – Olivier Sep 08 '20 at 07:22
  • @Olivier Thanks. No Authentication required at all. – JohnT Sep 08 '20 at 07:28
  • @RemyLebeau, I will probably start with using the Firefox debugger network tab. I guess I'm really struggling with the concept that when I just copy and paste the URL into a browser I get the expected JSON returned, but the standard HTTP and REST components do not. That;s why I thought that the only difference could be the user agent. But when I copied the user agent and pasted into the NetHttpClient component it had no effect. That's why I was wondering if there is something different with the structure of the request is sent to the server – JohnT Sep 08 '20 at 07:40
  • 1
    @JohnT that is why you need to capture a browser's actual requests, and compare them to the actual requests being made by your app. Only then can you see the differences and account for them in your code. And HOW you can account for them will depend on what they actually are. – Remy Lebeau Sep 08 '20 at 07:43
  • @RemyLebeau Okey Dokey. I'm on it. I'll post back here when I have some more info in case it helps out someone else as well. Cheers. – JohnT Sep 08 '20 at 07:48
  • @JohnT make sure you post it as an [edit] to your question, not in comments. – Remy Lebeau Sep 08 '20 at 07:49
  • 1
    try fiddler as well it has the ability to show you the exact request. – Nasreddine Galfout Sep 08 '20 at 16:27

0 Answers0