0

I want to transfer the files from local system to flink using nifi. I have configured pipeline in nifi with GetFile processor and output port with name "Data For Flink". On Flink end i am using the flink-connector-nifi_2.11 with flink. Below is my nifi client code in Flink.

         SiteToSiteClientConfig clientConfig = new SiteToSiteClient.Builder()
             //   .url("https://localhost:9443/nifi")
                .url("http://localhost:8080/nifi")
                .portName("Data For Flink")
                .transportProtocol(SiteToSiteTransportProtocol.HTTP)
                .requestBatchCount(5)
                .buildConfig();

        SourceFunction<NiFiDataPacket> nifiSource = new NiFiSource(clientConfig);
        DataStream<NiFiDataPacket> streamSource = streamExecEnv.addSource(nifiSource);
        DataStream<String> data = streamSource.map(new MapFunction<NiFiDataPacket, String>() {
            @Override
            public String map(NiFiDataPacket line) throws Exception {
                String row = new String(line.getContent(), Charset.defaultCharset());

                return row;
            }
        });
        data.print();
        streamExecEnv.execute();

When i ran the above code, i dont get any output as well as any error. Is my Nifi pipeline correct? Am i missing anything here?

Also Do i need to configure the secure nifi instance or any site to site properties to transfer the data from Nifi to Flink ?

I have tried that as well, but i am getting the issue with ca certificate. I have generated PKCS12 certificate with tls-toolkit. but when i try to upload that to java truststore, i am getting error that - x509 certificate required.

Edit 1: I am running the above code on IntelliJ IDEA with Flink Dependencies added on classpath.

I am new to both NiFi and Flink. I appreciate any help!

vishal
  • 1
  • 1

0 Answers0