Spark Thrift maintain caches among sessions

Question

In Spark Thrift, when using the beeline client, is it possible to:

1. User connects to the server using beeline and creates a cached table.
2. User connects again to the server using beeline (different session), and can use the cached table created in the previous session.

I tested this workflow, and in 1. I can see in the Spark UI that the cached table is there. When I finish the session in 1., the table is still there. But when I reconnect with the same user, I can't use it.

Answer 1

As far as I know, you cannot do this. Sharing RDDs, DataFrames, and Datasets across applications (Spark Contexts or Spark Sessions) is where Alluxio comes in. https://www.alluxio.org/ Including Spark SQL tables in the Spark Catalog. However, you can always write it to a hive table.