I am writing a data frame to elasticsearch using pyspark. I am able to get the fields in Kibana. The problem here is in kibana whenever i refer a string column that has space between the content, the string is displayed as two different fields in the graph. After some analysis i found that the field has to be set as "not analyzed" in ES so that the split will not happen.
dfcols = sqlContext.createDataFrame(cols)
dfcols.write.format("org.elasticsearch.spark.sql")\
.option("es.resource","index_name/sample")\
.option("es.nodes","http://server.qa.com")\
.option("es.port","9200")\
.save(mode="append")
This is how i am writing my dataframe to elasticsearch. How to specify the analyzed or not analyzed property of a field here?
