creating a bash function to print csv to screen

Question

I'm trying to create a bash function that will, once the user inputs an item code, print a csv file to the screen.

For example, when the item code is 526, I want the following to occur.

wget -O stats.zip "http://data.un.org/Handlers/DownloadHandler.ashx?DataFilter=itemCode:526&DataMartId=FAO&Format=csv&c=2,3,4,5,6,7&s=countryName:asc,elementCode:asc,year:desc"

zcat stats.zip

Is there a way to encapsulate all of this into one general function? How would one go about this using a bash script? For example, if I pass in the value XYZ, I want my script to do

wget -O stats.zip "http://data.un.org/Handlers/DownloadHandler.ashx?DataFilter=itemCode:XYZ&DataMartId=FAO&Format=csv&c=2,3,4,5,6,7&s=countryName:asc,elementCode:asc,year:desc"

zcat stats.zip

Answer 1

function dl {
 wget -O stats.zip "http://...?DataFilter=itemCode:$1"; 
 unzip -c stats.zip | columns -s, -t; 
}

1. You can use unzip's "-c" command to print the content of files
2. Bash (and other shells) allow you to wrap statements into functions

Additionally I've used the column command in order to format the CSV output.

You can call the function with the ItemCode as parameter: dl 526

Answer 2

At first sight, using unzip --help, I can't see an option to unzip that would assist in this (but I was looking at an old version, UnZip 5.52 of 28 February 2005, by Info-ZIP as shipped with Mac OS X 10.10.5). The -M option pipes through more, but that is just the list of file names as they're extracted, not the contents of the files. So, if that was the whole story, you'd need to unzip to extract the file, and then cat the extracted file, and then remove the extracted file and probably the zip file.

Yes, you can write a script (or even a function — though to my antiquated mind, a script is more appropriate) to do this.

OTOH, if you go to the Info-Zip home page, you can find that the current version is Unzip 6.0 released in 2009, and the documentation says it supports the -c option to

extract files to stdout/screen (''CRT''). This option is similar to the -p option except that the name of each file is printed as it is extracted, the -a option is allowed, and ASCII-EBCDIC conversion is automatically performed if appropriate. This option is not listed in the unzip usage screen.

This is consistent with other compressor and decompressor commands, such as gzip, bzip2, xz, and so on. So, there might be a -c option in the older version, and a quick experiment shows that there is in version 5.52.

So, if you have a recent enough version of unzip (for an ill-defined value of 'recent enough'), you can use:

unzip -c stats.zip

to send the contents of stats.zip to standard output.

How would I script this though?

Same as you would any other script: write the commands you want executed in the script. If you want to supply the item code as an argument, you can write code to require one or more arguments and pass those to the programs. One of the nice things about shell scripts is that the basics of a shell script are exactly what you'd type at the terminal to do the same job. This is in contrast to a C program, for example.

For example:

trap "rm -f stats.zip; exit 1" 0 1 2 3 13 15

for item in "$@"
do
    wget -O stats.zip "http://data.un.org/Handlers/DownloadHandler.ashx?DataFilter=itemCode:$item&DataMartId=FAO&Format=csv&c=2,3,4,5,6,7&s=countryName:asc,elementCode:asc,year:desc"
    zcat -c stats.zip
done

rm -f stats.zip
trap 0

If given no arguments, this does nothing. For each argument it is given, it gets the information from the web site, unzips the file content to standard output, and goes back for the next item. At the end, it removes the stats.zip file. It has protection so that if it is interrupted, the file is removed. The trap 0 at the end cancels the exit trap, so the script can exit successfully.

You should probably make the intermediate file into a better (unique) name, probably using mktemp command:

tmp=$(mktemp "${TMPDIR:-/tmp}/items.XXXXXX")

and then use $tmp where the code uses stats.zip.