Google Flights CFHTTP not working

Question

I am trying to get the following URL with CFHTTP but I am not getting the page. Any ideas?

http://www.google.com/flights/#search;f=JNB;t=MRU;d=2016-12-19;sel=JNBMRU0MK854;s=0

<cfhttp url="https://www.google.com/flights/##search;f=JNB;t=MRU;d=2016-12-19;sel=JNBMRU0MK852;s=0" method="GET" resolveurl="true" useragent="Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.87 Safari/537.36">
    <cfhttpparam type="header" name="HTTP_REFERER" value="http://example.com/feed/" >
    <cfhttpparam type="header" name="Accept-Encoding" value="gzip,deflate,sdch" >
    <cfhttpparam type="header" name="Proxy-Connection" value="keep-alive" >
    <cfhttpparam type="header" name="Accept" value="application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5">
    <cfhttpparam type="header" name="Accept-Language" value="en-US,en;q=0.8">
    <cfhttpparam type="header" name="Accept-Charset" value="ISO-8859-1,utf-8;q=0.7,*;q=0.3">
    <cfhttpparam type="cookie" name="some-cookie" value="1">
</cfhttp>
<cfoutput>#cfhttp.filecontent#</cfoutput>

Why would my question be down voted? – Marthinus Strydom Nov 11 '16 at 07:18 — Marthinus Strydom, Nov 11 '16 at 07:18

score 1 · Answer 1 · answered Nov 11 '16 at 06:43

1

Using http instead of https worked for me. If you want to test it, use the method suggested by @bkbk.

<cfdump var="#cfhttp.filecontent#">

But do note that, outputing the filecontent will not work similar to the original site. This is because the site uses some security header like

x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-xss-protection:1; mode=block

Which will not allow loading of few of the original site scripts and css.

answered Nov 11 '16 at 06:43

Pankaj

1,731
1
13
15

Thank you. I can display the result using `` but I need to get to the html. I need to scrape the flight information from the result. Looking through the dump, there is no way that I can use it effectively that I can see, unless I am missing something? [http://www.google.com/flights/#search;f=JNB;t=MRU;d=2016-12-19;sel=JNBMRU0MK854;s=0] shows what I need. – Marthinus Strydom Nov 11 '16 at 07:14
Done. Now open link Flight Schedule – BKBK Nov 12 '16 at 16:35

score 0 · Answer 2 · answered Nov 11 '16 at 06:02

It is very likely that the response is coming in as HTML, but your browser isn't displaying the tags. To display a result you can see, use

<cfoutput>#xmlformat(cfhttp.filecontent)#</cfoutput>

or

<cfdump var="#cfhttp#">

As an aside, what about just running

<cfhttp url="https://www.google.com/flights/##search;f=JNB;t=MRU;d=2016-12-19;sel=JNBMRU0MK852;s=0" method="GET" resolveurl="true" useragent="Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.87 Safari/537.36" />

Google Flights CFHTTP not working

2 Answers2