java.net.ConnectException: Connection refused (Connection refused) in deepLearning4j

Question

I am working on Deeplearning4j models on Spark. I want to enable Deeplearning4j-UI to follow up some graphs (training/testing..). I search about it and in some links they said that I should install the dependency maven-shade-plugin and the dependency deeplearning4j-ui-model instead of deeplearning4j-ui. Here is my code :

    public static void createNetwork(JavaSparkContext sparkContext, boolean isSaved) throws IOException {
        DefaultStatsUpdateConfiguration statsConfig = new DefaultStatsUpdateConfiguration.Builder().reportingFrequency(1).build();
        StatsListener statsListener = new StatsListener(null, null, statsConfig, sessionId, null);
        JavaRDD<DataSet> trainingData = createRDD(sparkContext, true);
        JavaRDD<DataSet> testData = createRDD(sparkContext, false);

        MultiLayerConfiguration networkConfig = NeuralNetworkConfig.getCnnNetwork(seed);
        TrainingMaster<?, ?> trainingConfig = configureTraining();

        SparkDl4jMultiLayer sparkNetwork = new SparkDl4jMultiLayer(sparkContext, networkConfig, trainingConfig);
        StatsStorageRouter remoteUIRouter = new RemoteUIStatsStorageRouter("https://localhost:9000");
        sparkNetwork.setListeners(remoteUIRouter, Collections.singletonList(statsListener));
        log.info("Execute training");
        
        executeTraining(trainingData, sparkNetwork);
        log.info("evaluation beginning");
        Evaluation evaluation = performEvaluation(testData, sparkNetwork);
        log.info("saving begin");
        if (isSaved==false) {
            log.info("I am saving");
            saveResult(evaluation, sparkNetwork.getNetwork());
        }
        log.info("I am cleaning");
        doCleaning(sparkContext, trainingConfig);
    }

I am using the Maven Dependency :

        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>deeplearning4j-ui-model</artifactId>
            <version>1.0.0-beta4</version>
        </dependency>

I got this error while submitting my spark application :

 22/06/19 10:22:00 WARN RemoteUIStatsStorageRouter: Error posting to remote UI at https://localhost:9000/remoteReceive, consecutive failure count = 9. Waiting 256000 ms before retrying
java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
        at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)

My pom.xml is :

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>deepLearningSimpleOne</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <java.version>1.8</java.version>
        <dl4j.spark.version>1.0.0-spark_2.12</dl4j.spark.version>
        <dl4j.version>1.0.0-beta7</dl4j.version>
        <nd4j.version>1.0.0-beta7</nd4j.version>
        <nd4j.backend>nd4j-native-platform</nd4j.backend>
        <scala.binary.version>2.12</scala.binary.version>
        <exec-maven-plugin.version>1.4.0</exec-maven-plugin.version>
        <maven-shade-plugin.version>2.4.3</maven-shade-plugin.version>
        <shadedClassifier>shaded</shadedClassifier>
        <jackson.version>2.5.1</jackson.version>
        <jcommander.version>1.81</jcommander.version>
        <logback.version>1.2.3</logback.version>




    </properties>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.5.1</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>

            </plugin>
            <!--      Added to enable jar creation using mvn command-->
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.4.1</version>
                <configuration>
                    <archive>
                        <manifest>
                            <mainClass>fully.qualified.MainClass</mainClass>
                        </manifest>
                    </archive>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <!-- bind to the packaging phase -->
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <version>1.4.0</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>exec</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <executable>java</executable>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>${maven-shade-plugin.version}</version>
                <configuration>
                    <shadedArtifactAttached>true</shadedArtifactAttached>
                    <shadedClassifierName>bin</shadedClassifierName>
                    <createDependencyReducedPom>true</createDependencyReducedPom>
                    <filters>
                        <filter>
                            <artifact>*:*</artifact>
                            <excludes>
                                <exclude>org/datanucleus/**</exclude>
                                <exclude>META-INF/*.SF</exclude>
                                <exclude>META-INF/*.DSA</exclude>
                                <exclude>META-INF/*.RSA</exclude>
                            </excludes>
                        </filter>
                    </filters>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <transformers>
                                <transformer
                                        implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                                    <resource>reference.conf</resource>
                                </transformer>
                                <transformer
                                        implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
                                <transformer
                                        implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                </transformer>
                            </transformers>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.projectlombok</groupId>
                <artifactId>lombok</artifactId>
                <version>1.18.8</version>
            </dependency>

        </dependencies>
    </dependencyManagement>


    <dependencies>

        <dependency>
            <groupId>org.nd4j</groupId>
            <artifactId>nd4j-native-platform</artifactId>
            <version>1.0.0-beta7</version>
        </dependency>

        <dependency>
            <groupId>org.nd4j</groupId>
            <artifactId>nd4j-native</artifactId>
            <version>1.0.0-beta7</version>
        </dependency>

        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>${logback.version}</version>
        </dependency>

        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>deeplearning4j-ui-model</artifactId>
            <version>1.0.0-beta7</version>
        </dependency>


        <dependency>
            <groupId>org.nd4j</groupId>
            <artifactId>nd4j-x86</artifactId>
            <version>0.4-rc3.8</version>
        </dependency>




        <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core -->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.13</artifactId>
            <version>3.2.1</version>
        </dependency>

        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.11</artifactId>
            <version>2.4.0</version>
        </dependency>
        <dependency>
            <groupId>com.beust</groupId>
            <artifactId>jcommander</artifactId>
            <version>${jcommander.version}</version>
        </dependency>
        <dependency>
            <groupId>com.intel.analytics.zoo</groupId>
            <artifactId>zoo-core-parent</artifactId>
            <version>0.6.0</version>
            <type>pom</type>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.8</version>
        </dependency>


        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>2.11.12</version>
        </dependency>



        <!--dependency>
            <groupId>org.nd4j</groupId>
            <artifactId>nd4j-onnxruntime</artifactId>
            <version>1.0.0-beta7</version>
        </dependency-->

        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>dl4j-spark_${scala.binary.version}</artifactId>
            <version>1.0.0-beta7</version>
        </dependency>
        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>deeplearning4j-core</artifactId>
            <version>1.0.0-beta7</version>
        </dependency>
        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>deeplearning4j-zoo</artifactId>
            <version>${dl4j.version}</version>
        </dependency>

        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>dl4j-spark-parameterserver_${scala.binary.version}</artifactId>
            <version>1.0.0-beta7</version>
        </dependency>


        <!-- Used for patent classification example -->
        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>deeplearning4j-nlp</artifactId>
            <version>${dl4j.version}</version>
        </dependency>

        <dependency>
            <groupId>com.intel.analytics.bigdl</groupId>
            <artifactId>bigdl</artifactId>
            <version>0.2.0</version>
        </dependency>

        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacpp</artifactId>
            <version>1.5.5</version>
        </dependency>



    </dependencies>


</project>

Anyone can please help me ?

score 0 · Answer 1 · answered Jun 19 '22 at 08:56

0

First of all the version you're using is a few years old. Please make sure you upgrade to M1.1 or M2 then we can help you.

I'm not sure where you got that specific version from but please check the examples: https://github.com/eclipse/deeplearning4j-examples

or search more recent versions in maven central: https://search.maven.org - any artifactId should do.

Beyond that, Connection Refused is a standard java exception not specific to the library. Consider that you might have a network issue or the UI didn't start. Try both spark and the UI locally to make sure it's not something like a firewall problem. From there, just get more familiar with how working with remote computers works (eg: they need to be able to connect to each other)

answered Jun 19 '22 at 08:56

Adam Gibson

3,055
1
10
12

Thank you for your response. Actually I am running my application on a HPC and when I change the version of nd4j to M1.1 I got the following error : Caused by: java.lang.UnsatisfiedLinkError: no jnind4jcpu in java.library.path Caused by: java.lang.RuntimeException: ND4J is probably missing dependencies. – nour rekik Jun 19 '22 at 11:16
Could you file an issue with more information please? That might be related to your OS. If it is, you might be able to try our compat classifier: https://repo1.maven.org/maven2/org/nd4j/nd4j-native/1.0.0-M1.1/ Use linux-x86_64-compat and nd4j-native with no classifier for your maven dependencies. If that doesn't work, please post the full stack trace. – Adam Gibson Jun 20 '22 at 13:08
I am running my application on a server remotely which the OS is linux-x86_64. every time I allocate the needed resources such as CPUs or GPUs and load Maven and java from this server too. Then submit my spark application on this server. At the beginning I started with beta4 and it was fine then I tried with M1.1 and M2 but I got this error : https://blog.csdn.net/wang_xiaoxin/article/details/123580718. I searched about this and they said that I have to install GLIBC but I didn't succeed. the most recent version that he accepted is beta7. but still got the same error regarding dl4j-ui. thanks – nour rekik Jun 20 '22 at 15:04
I added my pom.xml above and the command I use : spark-submit --class com.examples.DeepLearningOnSpark.mnist_image.multi_layer_perceptron_mnist.MnistMLPMain target/deepLearningSimpleOne-1.0-SNAPSHOT-jar-with-dependencies.jar – nour rekik Jun 20 '22 at 17:18
You have a bigger problem: you're pulling in random dependencies from different versions of the library including one that's years old at this point. Please don't just randomly copy/paste stuff from the internet (especially years old random articles) Follow the examples: https://github.com/eclipse/deeplearning4j-examples). All dl4j related libraries should always use the same version. Please only use nd4j-native for whatever cpu workloads you have. Anything else is old/invalid. My recommendation around compat should help you for more recent versions then. Try that first. – Adam Gibson Jun 21 '22 at 05:32
Regarding the GLIBC (sorry can't read chinese) but this person probably didn't follow the documentation or doesn't follow the community forums/github issues. It's specifically documented in the release notes that older glibc compatibilities are preserved using the compat classifier. – Adam Gibson Jun 21 '22 at 05:35

java.net.ConnectException: Connection refused (Connection refused) in deepLearning4j

1 Answers1