0

I am trying to automate downloading our weather data for our school. I'm not a huge tech guy but I'm the best at the school I suppose. My problem is trying to insert the time variables into the web address. Here is what I have done so far.

Currently This works:

curl -o /Library/Server/Web/Data/Sites/wupmooksgmol.ca/weather/"$(date +%Y)"/"$(date +%Y-%m-%d)".weather.csv 'http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IBRITISH322&day=16&month=1&year=2015&graphspan=day&format=1'

But, in the web address, it is only downloading the Jan. 16th 2015 weather data. I want to put in the current day, month and year into the web address itself. Thus, at 23:57 each day it downloads the weather data for that day. I have tried many variation on the following but no luck:

curl -o /Library/Server/Web/Data/Sites/wupmooksgmol.ca/weather/"$(date +%Y)"/"$(date +%Y-%m-%d)".weather.csv 'http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IBRITISH322&day=“$(date +%d)”&month=“$(date +%m)”&year=“$(date +%Y)”&graphspan=day&format=1'

I have also attempted a numerous variations on this shell script:

#!/bin/bash

day=$(date '+%d')
month=$(date '+%m')
year=$(date '+%Y')
ymd=$(date '+%Y-%m-%d')

curl -o /Library/Server/Web/Data/Sites/wupmooksgmol.ca/weather/“$year"/"$ymd".weather.csv 'http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IBRITISH322&day=“$day”&month=“$month”&year=“$year”&graphspan=day&format=1'

Thanks for any help you can provide.

iQuestion
  • 3
  • 4

2 Answers2

0

I think you might just be running into a problem with how you're wrapping the string — specifically, the sorts of quotes you're using. I haven't used OSX in a while, but it's basically the same (if not the same) shell that's used in most Linux distros. In general, anything within double quotes will allow variable substitution, but anything within single quotes will be taken absolutely literally (without substitutions).

Try wrapping the URL in double-quotes rather than single quotes, and then don't wrap the variables in quotes within that string. So:

curl "http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IBRITISH322&day=$(date +%d)&month=$(date +%m)&year=$(date +%Y)&graphspan=day&format=1"

Testing the above myself seems to get current results, but then I've never used this API, so I'm not sure what I'm looking at. ^_^ I can read a date, though, and the results are from today.

In the future, rather than spamming a site with possibly bogus requests, try testing your URL syntax with echo. That will just output your string to the terminal, so you can debug it without sending a request. Once you get the output looking right by testing with echo, then try using it as the destination for curl. This technique is useful for debugging any input substitution that you are trying to assemble in a shell script.

zerobandwidth
  • 1,213
  • 11
  • 18
  • Excellent. This worked well. However, the shell script is more useful so I will be going with that. – iQuestion Jan 20 '15 at 05:08
  • @iQuestion Happy to help. For reference in future shell scripts, try reading the excellent manual page for `bash` by entering **`man bash`** on the command line. In addition to variable substitutions, it discusses other useful features of the scripting language that allow you to build pretty sophisticated control logic. – zerobandwidth Jan 21 '15 at 13:22
0

I think you need this:

#!/bin/bash

day=$(date '+%d')
month=$(date '+%m')
year=$(date '+%Y')
ymd=$(date '+%Y-%m-%d')

curl -o weather.csv "http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IBRITISH322&day=${day}&month=${month}&year=${year}&graphspan=day&format=1"

The way I think about quotes is like this. If you surround stuff with single quotes, nothing is going to get expanded and there is no point putting variables inside them. If you use double quotes, variables will get expanded and stuff with spaces in will get "held together" and treated as a single parameter. Not very technical, but it works.

Mark Setchell
  • 191,897
  • 31
  • 273
  • 432