Twitter API v2 Streaming with Retrofit or OkHttp

Question

I am trying integrate the new Twitter API specifically the streaming tweets part in my android app, I am using Retrofit for my http calls.

When I try to make the call to get the streaming tweets it just hangs and does not return anything.

this is my retrofit call

@Streaming
@GET("tweets/search/stream")
suspend fun getFilteredStream(@Header("Authorization") token:String)

I then tried making a call with just OkHttp as shown in the documentation I get a successful response but I dont know how to stream the data.

I can make the call successfully via a curl call and see the data no problem.

How do I stream the data via retrofit or OkHttp

Update:

With OkHttp I was able to get data by doing this

val client: OkHttpClient = OkHttpClient().newBuilder()
            .build()
        val request: Request = Request.Builder()
            .url("https://api.twitter.com/2/tweets/search/stream")
            .method("GET", null)
            .build()
        val response: Response = client.newCall(request).execute()
        val source = response.body?.source()
        val buffer = Buffer()
        while(!source!!.exhausted()){
            response.body?.source()?.read(buffer, 8192)
            val data = buffer.readString(Charset.defaultCharset())
        }

data holds the string data representation of multiple tweet objects but how do I read one tweet at a time, or parse the response like this?

score 0 · Accepted Answer · answered Aug 31 '20 at 15:45

0

From the docs, I think you'll need to combine the two examples you have.

https://github.com/square/retrofit/blob/108fe23964b986107aed352ba467cd2007d15208/retrofit/src/main/java/retrofit2/http/Streaming.java

Treat the response body on methods returning {@link ResponseBody ResponseBody} as is, i.e. without converting the body to {@code byte[]}.

And example calling code

https://github.com/square/retrofit/blob/108fe23964b986107aed352ba467cd2007d15208/retrofit/src/test/java/retrofit2/CallTest.java#L599

I suspect, dependending on the JSON API you use you may be able to use a Streaming API from the response body. But if not you could split on either just newline or newline followed by { on next line and parse individually. Sorry I can't help here.

See https://en.wikipedia.org/wiki/JSON_streaming#Concatenated_JSON

answered Aug 31 '20 at 15:45

Yuri Schimke

12,435
3
35
69

Using `ResponseBody` for the retrofit call worked, I then split the string on `\r\n` and it seems to work, I just worry about how reliable its going to be parsing it like that – tyczj Sep 01 '20 at 01:19
It should be reasonably, you can change to using a real JSON parser for this, if you see examples from wikipedia it should be clear that as long as it's line split it will keep working. – Yuri Schimke Sep 01 '20 at 05:46
1

@tyczj can you please post your solution? I am also facing the same issue. – r4jiv007 Aug 12 '21 at 12:25
@tyczj does your solution with `ResponseBody` give you every message when it arrives, or does it give you one chunk? – rwst May 22 '22 at 18:08

Twitter API v2 Streaming with Retrofit or OkHttp

1 Answers1