-2

I am trying to push the mobile apps data into kafka broker. I have read many posts and blogs, got to know there's Rest Proxy which can push the data on HTTP to Kafak broker. I followed https://docs.confluent.io/current/kafka-rest/docs/quickstart.html# to gets the flavour of Rest Proxy. it's okay for practice but i want to get some real world examples on it. had followed Sending data from android/iOS app to Kafka or Hadoop. I just wanted to know how mobile => Rest Proxy => Kafka broker.

How to configure it please share steps that needs to be taken.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
DP0808
  • 33
  • 1
  • 6
  • 1
    not sure what you're looking for. why would mobile be any different than any other client running against the REST proxy? – Lior Chaga Dec 05 '18 at 12:30
  • how to configure it with mobile apps to rest proxy. – DP0808 Dec 05 '18 at 12:32
  • 1
    mobile app should just use the REST api of the proxy, like any other rest api... – Lior Chaga Dec 05 '18 at 12:34
  • can you please share some steps or elaborate this. i didn't understand completely. haven't used rest proxy before. so kind of unaware about functionality. – DP0808 Dec 05 '18 at 12:38
  • well, I have to say that I'm not a mobile app developer, so I can't direct you to specific SDK for that, but rest api just means you're using http requests methods (GET, POST etc.) – Lior Chaga Dec 05 '18 at 12:42
  • okay how app will hit post request to the rest proxy and rest proxy to kafka cluster. – DP0808 Dec 05 '18 at 12:50
  • 1
    that's exactly the role of the proxy. It serves as a proxy... once any client calls the rest api, the proxy acts on it's behalf and delegate the request to kafka brokers. – Lior Chaga Dec 05 '18 at 12:54
  • never mind as i mentioned haven't got a chance to work on Rest Proxy if you have any example, share it or share some steps. my worry about proxy configuration. how to do that and app send that so do i need to configure my IP with port. – DP0808 Dec 05 '18 at 13:12

1 Answers1

0

I would suggest rather creating your own REST API, mostly to learn how it works, and how your app would communicate with it, period. And within that server side code, you can embed input validation, custom logic specific for your use case(s). On any valid data, you can send it along to a Kafka producer.

That way, you will be able to understand all the pieces of the architecture from a high level, and customize the Kafka events outside of the client payload, if needed.

If you're not satisfied with that, you'll need to be more specific about the issues you're having following the installation instructions for the REST Proxy, because I'm sure if you have problems, it might be good feedback for Confluent to try and accommodate and fix the documentation.

Leaning how to use OkHttp, Volley, or other Java HTTP libraries on Android is completely at the other end of this pipeline, and maybe will take a few more days/weeks of learning how to use them

Another option that is presented is to use MQTT/AMQP/JMS rather than REST, and a Kafka connector, but that's assuming you already have that infrastructure in place, I suppose

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • thank you so much. i have given them my feedback and asked for examples/elaborate Rest Proxy functionality in more details. meanwhile i am stuck and keep to test this. does it really works. looking forwards for configuration steps. – DP0808 Dec 05 '18 at 13:49
  • I'm not sure I understand what more detail you require than what's already there... You can produce, consume, and describe topics all via HTTP – OneCricketeer Dec 05 '18 at 13:54
  • Thanks for the info it really helped but i am looking on HTTP only since i have one device for testing purpose only. since confluent provide rest proxy i believe it play vital role in messaging world but here lack of info derive this way. – DP0808 Dec 05 '18 at 14:02
  • Again, lack of **what** info? Yours personally, or what's in the website? – OneCricketeer Dec 05 '18 at 14:05
  • I am talking about myself i have lack of info about Rest Proxy that derives me many wrong concepts which i don't even need to touch for this particular issues. anyways you have always been helpful that gives me fresh leap. – DP0808 Dec 06 '18 at 06:51