VORA Tables in Zeppelin and Spark shell

Question

We have created test table from spark shell as well as from Zepellin. But when we do show tables on single table is visible in respective environment. Table created via spark shell is not displayed in Zepellin show table command. What is the difference between these two tables ? can anybody please explain.

Answer 1

The show tables command only shows the tables defined in the current session.

A table is created in a current session and also in a (persistent) catalog in Zookeeper. You can show all tables that Vora saved in Zookeeper via this command:

SHOW DATASOURCETABLES 
    USING com.sap.spark.vora 
    OPTIONS(zkurls "<zookeeper_server>:2181")  

You can also register all or single tables in the current session via this command:

REGISTER ALL TABLES 
    USING com.sap.spark.vora 
    OPTIONS(zkurls "<zookeeper_server>:2181")  
REGISTER TABLE <tablename> 
    USING com.sap.spark.vora 
    OPTIONS(zkurls "<zookeeper_server>:2181")

So if you want to access the table that you created in the Spark Shell from Zookeeper and vice versa you need to register it first.

You can use those commands if you need to clear the Zookeeper Catalog. Be aware that tables then need to be recreated:

import com.sap.spark.vora.client._
ClusterUtils.clearZooKeeperCatalog("<zookeeper_server>:2181")

This (and more) information can be found in the Vora Installation and Developer Guide