-3

I can not use the

MQTTUtils.createPairedStream()

in Scala?

How to specify list of topics as parameters? I tried all the ways like dictionary, list,tuples, but it did not worked. And then i tried in python, at that time it shows an error like

Java gateway process exited before sending the driver its port number

Ashok v
  • 77
  • 1
  • 8

1 Answers1

1

Scala is a statically-typed language where the arguments you provide to a method must have specific types in order for the method call to compile. Sometimes methods are overloaded, in which case different combinations of arguments can be provided, and in Scala sometimes some arguments may have default values, in which case they don't need to be provided at all.

There are several ways to determine what arguments are possible for a given method. Your IDE might show a tool-tip when you type MQTTUtils.createPairedStream(. Sometimes library developers publish API documentation that shows method signatures (which include the argument types and return value) in an easy-to-read format. If neither of these are options for you and the project is open source, you can just look up the source code. In this case you'll find something the following (note that you may need to change the tag in the GitHub interface if you're using a different version):

  /**
   * Create an input stream that receives messages pushed by a MQTT publisher.
   * @param ssc           StreamingContext object
   * @param brokerUrl     Url of remote MQTT publisher
   * @param topics        Array of topic names to subscribe to
   * @param storageLevel  RDD storage level. Defaults to StorageLevel.MEMORY_AND_DISK_SER_2.
   */
  def createPairedStream(
      ssc: StreamingContext,
      brokerUrl: String,
      topics: Array[String],
      storageLevel: StorageLevel = StorageLevel.MEMORY_AND_DISK_SER_2
    ): ReceiverInputDStream[(String, String)] = {
    new MQTTPairedInputDStream(ssc, brokerUrl, topics, storageLevel)
}

This is the simplest of eight createPairedStream method overloads (all the rest take additional arguments). This shows that you'll need to provide (in order) a StreamingContext, a broker URL, and an array of topics.

Travis Brown
  • 138,631
  • 12
  • 375
  • 680
  • Can you give an example of this createPairedstream()? I have little bit confusion of specifying topics. Here can we specify a list of topics? – Ashok v Mar 08 '19 at 04:52
  • @Ashokv Yes, the third argument can be an array like `Array("foo", "bar", "baz")`. – Travis Brown Mar 08 '19 at 09:59
  • java.lang.NoSuchMethodError: org.apache.spark.streaming.mqtt.MQTTUtils$.createPairedStream This is what i got when i run the program. As you said earlier the IDE show a tool-tip when i typed MQTTUtils. But got error when I executed – Ashok v Mar 13 '19 at 10:32
  • That looks like a version mismatch somewhere. – Travis Brown Mar 13 '19 at 10:44
  • I build in one system and executed in other system. Need to check only JDK,Scala and Spark versions? – Ashok v Mar 13 '19 at 11:18
  • The Spark ecosystem is incredibly bad about dependency hygiene, so this is just a guess, but it's probably that you have differing Bahir versions. That seems totally unrelated to the original question, though, so you should ask a new question. – Travis Brown Mar 13 '19 at 11:22
  • When i used createstream() there is no problem. Apache spark-bahir need to install seperatly? – Ashok v Mar 13 '19 at 11:40