Using a proxy that requires authentication with pybliometrics

Question

I am using pybliometrics, a Python interface to the Scopus API, to download the abstracts of some papers.

Unfortunately Scopus only works inside the network of the university that subscribed to it. I am currently at home and whenever I try to download something using pybliometrics it gives me the following error:

pybliometrics.scopus.exception.Scopus401Error: The requestor is not authorized to access the requested view or fields of the resource

I need to use my university's proxy in order to enter the internet with the IP address of my university. The proxy has a WPAD configuration file available, but I fail to realize how to use it with pybliometrics. The pybliometrics documentation says to add a block in the configuration file like this:

[Proxy]
ftp = socks5://127.0.0.1:1234
http = socks5://127.0.0.1:1234
https = socks5://127.0.0.1:1234

But this proxy requires authentication. How can I specify the proxy username and password?

EDIT: I have tried setting up the block in config.ini like:

[Proxy]
ftp = http://username:password@proxy.thing.it:8080
http = http://username:password@proxy.thing.it:8080
https = http://username:password@proxy.thing.it:8080

but it still fails with the following error message:

requests.exceptions.ProxyError: HTTPSConnectionPool(host='api.elsevier.com', port=443): Max retries exceeded with url: /content/abstract/scopus_id/84983158344?view=META_ABS (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required')))

score 0 · Answer 1 · answered Jun 18 '20 at 11:09

0

From our perspective the API will work via a proxy as long as the proxy is configured correctly. I would suggest you speak to the provider of the proxy to see if they can help.

We don't have specific instructions on how to use APIs with a proxy (as there are many potential different versions and potential configurations); however, the general instructions are here:

https://service.elsevier.com/app/answers/detail/a_id/29026/supporthub/elsevieraccess/

answered Jun 18 '20 at 11:09

Elsevier Developers

86
4

Yes, the problem was that my proxy required DigestAuth rather than BasicAuth. Thanks! – robertspierre Dec 31 '21 at 00:46

score 0 · Answer 2 · answered Aug 27 '20 at 07:07

0

To me your new proxy block looks suspicious. It funnels ftp and https requests through the http as well. Maybe try ftp and https as protocols in the corresponding sections.

The other solution is to ask Scopus Integration Support for an InstToken, which you use instead of a proxy. You then specify the InstToken in the configuration file as well.

answered Aug 27 '20 at 07:07

MERose

4,048
7
53
79

The problem was that my proxy required DigestAuth rather than BasicAuth. Thanks! – robertspierre Dec 31 '21 at 00:46

score 0 · Accepted Answer · answered Dec 31 '21 at 00:46

0

The problem was that my proxy requires DigestAuth rather than BasicAuth.

answered Dec 31 '21 at 00:46

robertspierre

3,218
2
31
46

Using a proxy that requires authentication with pybliometrics

3 Answers3