0

I am trying to retrieve data from a website for which the parameters that you need to define does not show up in the url ie. http://www.vmm.be/webrap/ibmcognos/cgi-bin/cognosisapi.dll?b_action=cognosViewer&ui.action=run&ui.object=%2fcontent%2ffolder[%40name%3d%27Water%27]%2ffolder[%40name%3d%27Afvalwater%27]%2freport[%40name%3d%27Individuele%20analyseresultaten%20per%20RWZI%27]&ui.name=Individuele%20analyseresultaten%20per%20RWZI&run.outputFormat=HTML&run.prompt=false&ui.backURL=%2fwebrap%2fibmcognos%2fcgi-bin%2fcognosisapi.dll%3fb_action%3dxts.run%26m%3dportal%2fcc.xts%26m_folder%3di5DDA04E5A00C4B6AB6DF44BB4FAD7CEC&p_RwziNr=51&run.prompt=false

how can I extract the data for different years and parameters in a programmatic way?

I am using matlab's urlread but since the data I want to import does not show up in the html-code (I have checked this with the Web Developer Toolbar in FireFox), nothing is being read in. I have no experience with websites, only matlab and c-programming, so I have no idea how the data can be shown in the browser if it is not showing up in the html-source code so could someone point me to the right direction on how to get this job done? Is it at all possible? I hope so because I will have to repeat this for around 500 measuring stations, each 10 years so I am not planning on copying the required data manually as I did before when I just needed one station.

Ingrid
  • 15
  • 8
  • Does one need some login to access the data in the link? I only get an error message when I try to access the data. I saw this server runs on IBM Cognos Software - I don't know this software, but they might have an API you can use to get the data. – hbaderts Dec 12 '14 at 14:53
  • No there is no login required. The data is freely availble to everyone, just not in a convenient data format unfortunatly: Where could I find such an API? – Ingrid Dec 13 '14 at 19:39
  • If there's nothing obvious on the website, you contact the owners of the data and ask. – nkjt Dec 16 '14 at 11:51

1 Answers1

0

it turns out it is not possible to do what I want in Matlab. I however did manage to get the required data programmatically by using a combination of Selenium with C# and chrome driver. It's slow, but it's working and I can do other stuff in the meanwhile so I can recommend it to everyone who is downloading data from servers in a tedious way.

Ingrid
  • 15
  • 8