Change non ascii character in an HTTP request

Question

i have to make an http get call to an external service. By sending an address, the latitude and longitude coordinates are returned. The problem is that if the address is Russian or French I have errors like:

URI must be ascii only "http://nominatim.openstreetmap.org/?format=json&addressdetails=1&q={Barrage de G\u00E9nissiat, Rue Marcel Paul, Injoux, Franclens, Nantua, Ain, Auvergne-Rh\u00F4ne-Alpes, Francia metropolitana, 74910, Francia}&format=json"

My code is:

url = "http://nominatim.openstreetmap.org/?format=json&addressdetails=1&q={"+ address_search +"}&format=json";
response = Faraday.get url

the variable address_search is:

 Barrage de Génissiat, Rue Marcel Paul, Injoux, Franclens, Nantua, Ain, Auvergne-Rhône-Alpes, Francia metropolitana, 74910, Francia

Faraday is just a grm in rails to make an HTTP request, nothing strange. I have to manipulate the URL. Do you have any suggestion?

EDIT:

  print Rails.logger.info "body " + response.body
    print Rails.logger.info response.status
    print Rails.logger.info response.env.url

This is the output

body []
200
https://nominatim.openstreetmap.org/?addressdetails=1&format=json&q=Barrage+de+G%C3%A9nissiat%2C+Rue+Marcel+Paul%2C+Injoux%2C+Franclens%2C+Nantua%2C+Ain%2C+Auvergne-Rh%C3%B4ne-Alpes%2C+Francia+metropolitana%2C+74910%2C+Francia
  Rendering homes/index.html.erb within layouts/application

Yakov · Answer 1 · 2020-10-15T19:46:09.653

The query should be encoded, you can do it manually:

query = 'Barrage de Génissiat, Rue Marcel Paul, Injoux, Franclens, Nantua, Ain, Auvergne-Rhône-Alpes, Francia metropolitana, 74910, Francia'
address_search = URI.escape(query)
url = "https://nominatim.openstreetmap.org/?format=json&addressdetails=1&q=#{address_search}&format=json"
response = Faraday.get(url)

Or leave the job for Faraday:

query = 'Barrage de Génissiat, Rue Marcel Paul, Injoux, Franclens, Nantua, Ain, Auvergne-Rhône-Alpes, Francia metropolitana, 74910, Francia'
connection = Faraday.new('https://nominatim.openstreetmap.org')
response = connection.get do |request|
  request.params = { format: 'json', addressdetails: 1, q: query }
end

Here is the response:

=> #<Faraday::Response:0x0000556afc452060
 @env=
  #<struct Faraday::Env
   method=:get,
   body=
    "[{\"place_id\":104161692,\"licence\":\"Data \xC2\xA9 OpenStreetMap contributors, ODbL 1.0. https://osm.org/copyright\",\"osm_type\":\"way\",\"osm_id\":80667335,\"boundingbox\":[\"46.0516289\",\"46.0531641\",\"5.8112371\",\"5.81
4139\"],\"lat\":\"46.0523765\",\"lon\":\"5.812744291298651\",\"display_name\":\"Barrage de G\xC3\xA9nissiat, Rue Marcel Paul, Injoux, Franclens, Nantua, Ain, Auvergne-Rh\xC3\xB4ne-Alpes, France m\xC3\xA9tropolitaine, 74910, France\"
,\"class\":\"tourism\",\"type\":\"attraction\",\"importance\":1.7983764706069638,\"icon\":\"https://nominatim.openstreetmap.org/images/mapicons/poi_point_of_interest.p.20.png\",\"address\":{\"tourism\":\"Barrage de G\xC3\xA9nissiat\
",\"road\":\"Rue Marcel Paul\",\"suburb\":\"Injoux\",\"village\":\"Franclens\",\"municipality\":\"Nantua\",\"county\":\"Ain\",\"state\":\"Auvergne-Rh\xC3\xB4ne-Alpes\",\"country\":\"France\",\"postcode\":\"74910\",\"country_code\":\
"fr\"}}]",
   url=
    #<URI::HTTPS https://nominatim.openstreetmap.org/?addressdetails=1&format=json&q=Barrage+de+G%C3%A9nissiat%2C+Rue+Marcel+Paul%2C+Injoux%2C+Franclens%2C+Nantua%2C+Ain%2C+Auvergne-Rh%C3%B4ne-Alpes%2C+Francia+metropolitana%2C+74910%2C+Francia>,
   request=#<struct Faraday::RequestOptions params_encoder=nil, proxy=nil, bind=nil, timeout=nil, open_timeout=nil, write_timeout=nil, boundary=nil, oauth=nil, context=nil>,
   request_headers={"User-Agent"=>"Faraday v0.17.3"},
   ssl=
    #<struct Faraday::SSLOptions
     verify=true,
     ca_file=nil,
     ca_path=nil,
     verify_mode=nil,
     cert_store=nil,
     client_cert=nil,
     client_key=nil,
     certificate=nil,
     private_key=nil,
     verify_depth=nil,
     version=nil,
     min_version=nil,
     max_version=nil>,
   parallel_manager=nil,
   params=nil,
   response=#<Faraday::Response:0x0000556afc452060 ...>,
   response_headers=
    {"server"=>"nginx",
     "date"=>"Thu, 15 Oct 2020 19:42:53 GMT",
     "content-type"=>"application/json; charset=UTF-8",
     "transfer-encoding"=>"chunked",
     "connection"=>"close",
     "access-control-allow-origin"=>"*",
     "access-control-allow-methods"=>"OPTIONS,GET"},
   status=200,
   reason_phrase="OK">,
 @on_complete_callbacks=[]>

Make sure that you use https schema, otherwise, you'll get a response with a redirect.

Do you find something with that address? Cause I have returned an empty json — Vito, Oct 15 '20 at 16:29
query = "Barrage de Génissiat, Rue Marcel Paul, Injoux, Franclens, Nantua, Ain, Auvergne-Rhône-Alpes, Francia metropolitana, 74910, Francia" connection = Faraday.new('https://nominatim.openstreetmap.org') response = connection.get do |request| request.params = { format: 'json', addressdetails: 1, q: query } end This code return me nothing but in the website https://nominatim.openstreetmap.org there is a result — Vito, Oct 15 '20 at 16:54
Could you try to request the same URL by `curl` in the bash console? `curl 'https://nominatim.openstreetmap.org/?addressdetails=1&format=json&q=Barrage+de+G%C3%A9nissiat%2C+Rue+Marcel+Paul%2C+Injoux%2C+Franclens%2C+Nantua%2C+Ain%2C+Auvergne-Rh%C3%B4ne-Alpes%2C+Francia+metropolitana%2C+74910%2C+Francia'` — Yakov, Oct 16 '20 at 10:33
Also, I'd check a different URL. For example `http://ip.jsontest.com/`, `response.body` should be `{"ip": "8.8.8.8"}` but instead of `8.8.8.8`, there should be your IP. — Yakov, Oct 16 '20 at 10:42
ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-linux], faraday 1.0.1 — Vito, Oct 16 '20 at 13:20
For me, it works under the same versions. Please check what the result `curl` will return. — Yakov, Oct 16 '20 at 13:29
Empty array even in curl but if i use www.google.it, something it's returned to me — Vito, Oct 16 '20 at 13:40
If `curl` returns the empty result, I can only guess that there are some network or firewall issues. The solution is workable. Try to use another machine. I have no more advice at this point. — Yakov, Oct 16 '20 at 13:54
The problem is that if I search only the first part of the address: Barrage de Génissiat I obtain a result....This is really really strange. No, I don't think firewall issue cause the autocompletion is working and it uses same api — Vito, Oct 16 '20 at 14:52

score 0 · Answer 2 · answered Oct 15 '20 at 20:31

0

I use this code:

  query = "Barrage de Génissiat, Rue Marcel Paul, Injoux, Franclens, Nantua, Ain, Auvergne-Rhône-Alpes, Francia metropolitana, 74910, Francia"
  connection = Faraday.new('https://nominatim.openstreetmap.org')


    response = connection.get do |request|
      request.params = { format: 'json', addressdetails: 1, q: query }
    end


    print Rails.logger.info response.body

And I receive []..

answered Oct 15 '20 at 20:31

Vito

746
2
9
25

It works perfectly in my Rails console and in a pure IRB session. – Yakov Oct 15 '20 at 21:34
I'm getting crazy. What kind of debug can I do? – Vito Oct 15 '20 at 21:37
Try to analyze the whole `response` object, `pp response`, `response.status`, `response.env.url` – Yakov Oct 15 '20 at 21:51

Change non ascii character in an HTTP request

2 Answers2