0

I'm trying to install the harvester plugin (https://github.com/ckan/ckanext-harvest) on CentOS 6.7 / Python 2.6.6 / CKAN 2.5.0a and everything's fine until I execute the following:

paster --plugin=ckanext-harvest harvester initdb --config=/etc/ckan/default/production.ini

Then I get the following error message:

Traceback (most recent call last):
  File "/usr/lib/ckan/default/bin/paster", line 9, in <module>
    load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
  File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/script/command.py", line 104, in run
    invoke(command, command_name, options, args[1:])
  File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/script/command.py", line 143, in invoke
    exit_code = runner.run(args)
  File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/script/command.py", line 238, in run
    result = self.command()
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 129, in command
    self._load_config()
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 199, in _load_config
    super(Harvester, self)._load_config()
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 147, in _load_config
    load_environment(conf.global_conf, conf.local_conf)
  File "/usr/lib/ckan/default/src/ckan/ckan/config/environment.py", line 232, in load_environment
    p.load_all(config)
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 124, in load_all
    unload_all()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 181, in unload_all
    unload(*reversed(_PLUGINS))
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 209, in unload
    plugins_update()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 116, in plugins_update
    environment.update_config()
  File "/usr/lib/ckan/default/src/ckan/ckan/config/environment.py", line 316, in update_config
    search.check_solr_schema_version()
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/__init__.py", line 291, in check_solr_schema_version
    res = urllib2.urlopen(req)
  File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib64/python2.6/urllib2.py", line 397, in open
    response = meth(req, response)
  File "/usr/lib64/python2.6/urllib2.py", line 510, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib64/python2.6/urllib2.py", line 429, in error
    result = self._call_chain(*args)
  File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.6/urllib2.py", line 616, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib64/python2.6/urllib2.py", line 397, in open
    response = meth(req, response)
  File "/usr/lib64/python2.6/urllib2.py", line 510, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib64/python2.6/urllib2.py", line 435, in error
    return self._call_chain(*args)
  File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.6/urllib2.py", line 518, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 401: Unauthorized

Any help appreciated (I could even use the SQL text for table creation as workaround, if available).

Regards,

Fr3dY
  • 1
  • 3
  • I imagine most other ckan paster commands also fail then on the check_solr_schema_version step. Why don't you check the SOLR url it is trying manually? – D Read Mar 16 '16 at 12:18
  • I added some debug lines to the urllib2.py file and now I can see the URL which gives the error: `http://www.cfauth.com/?cfru=aHR0cDovLzEyNy4wLjAuMTo4MDgwL3NvbHIvYWRtaW4vZmlsZS8/ZmlsZT1zY2hlbWEueG1s` Do you know what's this? – Fr3dY Mar 16 '16 at 12:29
  • Am afraid not. Do you have the normal SOLR setup as per the install instructions? – D Read Mar 16 '16 at 13:21
  • Fixed it!! I did set the http_proxy and https_proxy env variables in order to execute this step: `pip install -e git+https://github.com/ckan/ckanext-harvest.git#egg=ckanext-harvest` However, with the proxy already enabled, when the paster script looked for 127.0.0.1 (SOLR), it was using the proxy too... I just unset the http_proxy and https_proxy variables and then run the paster command without problems. – Fr3dY Mar 16 '16 at 13:41

1 Answers1

0

I did set the http_proxy and https_proxy env variables in order to execute this step:

pip install -e git+https://github.com/ckan/ckanext-harvest.git#egg=ckanext-harvest

However, with the proxy already enabled, when the paster script looked for 127.0.0.1 (SOLR), it was using the proxy too... I just had to unset the http_proxy and https_proxy variables and then run the paster command without problems:

    unset http_proxy
    unset https_proxy

    paster --plugin=ckanext-harvest harvester initdb --config=/etc/ckan/default/production.ini

    2016-03-16 14:33:45,918 DEBUG [ckanext.harvest.model] Harvest tables defined in memory
    2016-03-16 14:33:46,177 DEBUG [ckanext.harvest.model] Harvest tables created
    DB tables created
Fr3dY
  • 1
  • 3