0

I've created one program for snappy data in Java. I'm not able to get the table name in cluster. Also I'm can't understand the log file. Any hints?

public static void main( String[] args )
{
    SparkSession spark = SparkSession
       .builder()
       .appName("SparkApp")
       .master("local[*]")
       .getOrCreate();
    JavaSparkContext jsc = new JavaSparkContext(spark.sparkContext());
    SnappySession snappy = new SnappySession(spark.sparkContext());
    snappy.dropTable("CUSTOMER",true);
    snappy.sql("CREATE TABLE CUSTOMER ( " +
            "C_CUSTKEY     INTEGER NOT NULL," +
            "C_NAME        VARCHAR(25) NOT NULL," +
            "C_ADDRESS     VARCHAR(40) NOT NULL," +
            "C_NATIONKEY   INTEGER NOT NULL," +
            "C_PHONE       VARCHAR(15) NOT NULL," +
            "C_ACCTBAL     DECIMAL(15,2)   NOT NULL," +
            "C_MKTSEGMENT  VARCHAR(10) NOT NULL," +
            "C_COMMENT     VARCHAR(117) NOT NULL)" +
            "USING COLUMN OPTIONS (PARTITION_BY 'C_CUSTKEY')");
}
Neil
  • 14,063
  • 3
  • 30
  • 51
  • Firstly i created the jar then in my master ,through this command i run "$SNAPPY_HOME bin/run-example jarName.jar"... –  Apr 07 '17 at 11:02
  • Can you please elaborate where you are checking. $SNAPPY_HOME bin/run-example will run Spark in a local mode. – Rishitesh Mishra Apr 07 '17 at 11:38
  • yes in a local machine i run that command , so many logs are came ,in this some processing in that sql create command but not geeting table in the cluster –  Apr 07 '17 at 12:21
  • Where are you checking for the table name? This is local[\*] mode so only that program can see the table (or in Spark GUI on port 4040). If you have a query like: snappy.sql("select count(*) from CUSTOMER").collect(), in that code snippet above does it work or not? – Sumedh Apr 07 '17 at 14:15
  • i am checking in my snappy cluster, snappy.sql("select count(*) from CUSTOMER").collect() this line not working –  Apr 07 '17 at 15:01
  • This is local[\*] mode so where exactly is the cluster? My question was whether that line is working or not in the above program snippet i.e. if you add it immediately after the CREATE TABLE above then does it work or not. Also when you say "not working" then does it fail with some exception? – Sumedh Apr 08 '17 at 06:43
  • that line not worked, and it doesn't show any exception . And my cluster in local machine only –  Apr 08 '17 at 11:32
  • I think the reason you are not getting any output from this line is that you have not inserted any rows in the table snappy.sql("select count(*) from CUSTOMER").collect(). And if you change that line to snappy.sql("select count(*) from CUSTOMER").show() it will show you actual count of the table, which is 0 in your example. – Yogesh Mahajan Apr 12 '17 at 23:22

1 Answers1

0

I think the reason you are not getting any output from this line is that you have not inserted any rows in the table snappy.sql("select count() from CUSTOMER").collect(). And if you change that line to snappy.sql("select count() from CUSTOMER").show() it will show you actual count of the table, which is 0 in your example.

Yogesh Mahajan
  • 241
  • 1
  • 4
  • but table name also not showing.. if table is created...so it will also show there. –  Apr 14 '17 at 08:23