0

I am currently using the R packages RISmed and bibliometrix to analyze publication data.

Here is my code: 

library(bibliometrix) 
library(RISmed)

search_topic <-  "(((Dental Health Services[MESH]) OR (Dentistry[MESH] OR Dentists[MESH)) AND Research[MESH]) AND Research[MESH] AND (Computers[MESH] OR Medical Informatics[MESH] OR Information Technology[MESH]) "
search_query <- EUtilsSummary(search_topic, retmax=200, mindate=2014, maxdate=2020)
summary(search_query)
D <- EUtilsGet(search_query)
M <- pubmed2df(D)

Everything works great, but when I run the last line, I get the error: 

In readLines(the_url) :
  cannot open URL 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?retmode=xml&dbfrom=pubmed&id=31173232&cmd=neighbor': HTTP status was '429 Unknown Error'

It says in the documentation for the EUtilsGet function that users can post no more than 3 times/second:

In order not to overload the E-utility servers, NCBI recommends that users post no more than three URL requests per second and limit large jobs to either weekends or between 9:00 PM and 5:00 AM Eastern time during weekdays. Failure to comply with this policy may result in an IP address being blocked from accessing NCBI.

After consulting this post too, it became clear that I might be able to fix the solution by either restarting R and updating the package (both of which had no effect), or by modifying the code to increase the time between requests.

How might one increase the time between requests, as this seems like the only option for me in this case? If there are any other alternative solutions, those would also be appreciated as well.

Mousa
  • 23
  • 5
  • 1
    We can use `Sys.sleep` to suspend the execution of the code for a moment i.e. `Sys.sleep(1)` to wait one second. – dario Mar 15 '20 at 07:35
  • @dario Would I have to insert this within the pubmed2df function itself? – Mousa Mar 15 '20 at 07:37
  • No, just between calls to functions that access a website. Also it's **not** the `pubmed2df` function that makes the webquers?! We can use `?pubmed2df` to learn what a function does. I.e. you should try `?EUtilsGet` – dario Mar 15 '20 at 07:42
  • @dario You know, that's what I thought too, but I get the error after running 'pubmed2df' function. Also, whatever function that is making the webqueries is doing them all within the same call :( That's why I feel like need to insert that line within the function itself. – Mousa Mar 15 '20 at 08:16
  • You are, of course, free to feel as you like ;). But I think it's going to be difficult to insert the sleep statement into a function that is defined in an external package. But the source is available, so you could change and compile it. But I'm quite sure that's not how this should be done ;) You could show the output of `search_query` and `D` if you need more help. Without that I'm not sure what else to tell you ;) Best of luck – dario Mar 15 '20 at 08:48
  • The `RISmed` package hasn't been updated in a while, as it seems. I succesfully got some data from pubmed using the `easyPubMed` package sometime ago. Maybe that'll work for you. – Till Mar 15 '20 at 09:15

0 Answers0