3

I'm trying to get some date from and endpoint and then send it to another endoint (with some modifications in the middle)

My goal is to use Streams in both cases (in and out)

This is kind of what i have:

library(jsonlite)
library(curl)

hPOST <- new_handle()
hGET <- new_handle()
handle_setheaders(hGET, "Authorization"=....)
handle_setheaders(hPOST, "Authorization"=...., "Content-Type"="application/octet-stream")

# to mark it as a POST
handle_setform(hPOST)

con_out <- curl(POST_URL, handle=hPOST)
#con_out <- stdout() // this works
con_in <- curl(GET_URL, handle=hGET)

inputs <- stream_in(con_in, handler = function(df){
    # some changes to df... and then save it
    stream_out(df, con_out, pagesize = 1)
}, pagesize = 1)

close(con_out)

The thing is that i can get the IN stream (in fact I can show it in the console by defining the con_out as stdout()), but when trying to save it to the OUT stream, it says "Error in open.connection(con, "wb"): can only open URLs for reading..."

For what I've read, not all modes are applicable to all connections: for example URLs can only be opened for reading

So my question: is there a way i can stream out to a POST endpoint in R language ?

Here I'm using jsonlite but I have no problems on using httr, etc.

oriuken
  • 653
  • 3
  • 7
  • 19
  • Why not just read it with `httr::GET` and send the result back with `httr::POST`? – Allan Cameron Jul 15 '20 at 16:54
  • Because i did not find a way to call POST with a stream. I will be manipulating big data so i really need to use streams – oriuken Jul 15 '20 at 17:06
  • Have you looked at the [streams package](https://www.r-bloggers.com/massive-online-data-stream-mining-with-r/) ? – Allan Cameron Jul 15 '20 at 18:16
  • I already checked that library but doesn't seems to help with what i need. Even if i could use the write_stream function it provides, the connection cannot be open as "Write".. I get the same error as before "Error in open.connection(con, open = mode): can only open URLs for reading" – oriuken Jul 15 '20 at 19:31

0 Answers0