DSX: Insert to code link is missing

Question

After uploading some files to my project and creating a catalog, I can see the list of files in the Find and Add Data section. However, there is no link Insert to code. This is true for files of type csv, json, tar.gz as well as for a data set from a catalog. What am I doing wrong?

Answer 1

Insert to Code Option is only available for data that you upload in Object Storage service.

I see that you are using Catalog for storage in DSX.

Catalog is still in beta state and currently insert to code is not added or supported for Catalog data assets.

Feel free to add enhancement request here:- https://datascix.uservoice.com/forums/387207-general

If you create a project with Object storage as storage , you will see the insert to code for csv files.

For reading from catalog , you will need to use projectUtil.

Catalog data asset is considered as a resource of project so to access it you would need access token.

So first step, generate the token to access the catalog resource. Go to Project Settings and create access token and then clear next cell and click insert project token from those 3 dots above in notebook and you will see code generated as below

The generated code just creates project context.

import com.ibm.analytics.projectNotebookIntegration._

val pc = ProjectUtil.newProjectContext(sc, "994b03fa-XXXXXX", "p-XXXXXXXXXX")

Lets make list of available files.

val fileList = ProjectUtil.listAvailableFilesData(pc)

fileList.indices.foreach( i => println(i + ": " + fileList(i)))

So the fileList contains your filenames. You can directly use the name of the file as second argument.

val df = ProjectUtil.loadDataFrameFromFile(pc, fileList(1))

or

val df1 = ProjectUtil.loadDataFrameFromFile(pc, "co2.csv")

You will see below:- "Creating DataFrame, this will take a few moments... DataFrame created."

df.show() and you will see content.

Full Notebook:- https://github.com/charles2588/bluemixsparknotebooks/blob/master/scala/Read_Write_Catalog_Scala.ipynb

The below doc also has python and R examples.  Ref for projectUtil:- https://datascience.ibm.com/docs/content/local/notebookfunctionsload.html

Thanks, Charles.