I have a flink project that is connecting to nifi to pull data. The setup to pull get the datastream works just fine when running locally.
.url("http://1.2.3.4:8080/nifi")
.portName("MyPortName")
.requestBatchCount(5)
.buildConfig();
But when I add the .jar to the remote cluster and run the job it throws this:
java.net.UnknownHostException
at sun.nio.ch.Net.translateException(Net.java:177)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:127)
at org.apache.nifi.remote.client.socket.EndpointConnectionPool.establishSiteToSiteConnection(EndpointConnectionPool.java:712)
at org.apache.nifi.remote.client.socket.EndpointConnectionPool.establishSiteToSiteConnection(EndpointConnectionPool.java:685)
at org.apache.nifi.remote.client.socket.EndpointConnectionPool.getEndpointConnection(EndpointConnectionPool.java:301)
at org.apache.nifi.remote.client.socket.SocketClient.createTransaction(SocketClient.java:129)
at org.apache.flink.streaming.connectors.nifi.NiFiSource.run(NiFiSource.java:90)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:55)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)
at org.apache.flink.streaming.runtime.tasks.StoppableSourceStreamTask.run(StoppableSourceStreamTask.java:39)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:272)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:655)
at java.lang.Thread.run(Thread.java:745)
The only reason I can find for an UnknownHostException is that it is because the IP of the host name can't be resolved, but I am giving the IP already. There was an issue earlier with it being unable to connect to nifi because I have to set what IP is allowed to access the nifi instance. So I added the AWS server as allowed and it fixed that, but obviously I have this now.
Any help is greatly appreciated!
