Questions tagged [bluedata]
16 questions
1
vote
1 answer
How can I read data from DataTap using cpython?
I would like to read data from DataTap using cpython.
In spark, I can do something like:
df = spark.read.csv("dtap://MaprClus2/tmp/airline-safety.csv")
How can I do the same if I am using cpython, for example when I don't have a pyspark Jupyter…
Chris Snow
- 23,813
- 35
- 144
- 309
1
vote
1 answer
bluedata pyspark hdfs write acces problem: hdfs_access_control_exception: permission denied
We have BlueData 3.7 running and I started the Cloudera 5.14 cluster with Spark and YARN.
I get a csv file from Qumulo per NFS per DTAP into the Spark container and just do a small filter and save the outcome as parquet file per DTAP into our…
Aileen Toleikis
- 31
- 3
1
vote
2 answers
bdcsv.py returns "The datetime zone id '00:00' is not recognised"
I'm trying to run bdcsv.py:
$ sudo python /opt/bluedata/bundles/bluedata-epic-entdoc-minimal-release-3.7-2207/scripts/monitoring/bdcsv.py \
-c localhost \
-f cred.json \
-s 2018/02/07-00:00:00 \
-e 2018/02/07-23:59:59
I received…
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
BlueData - Kibana server is not ready yet
I've followed the instructions to add Kibana to my BlueData 4.0 controller:
Login to the Controller.
Execute the command docker exec -it monitoring- bash, where is the IP address of the Controller.
Change the permissions of…
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
catalog entry json has api specification error
I have created an application image for BlueData and deployed it to the app store.
I noticed in tooltips on the app icon (App Store) the following error:
catalog entry json has api specification error
What can be causing this issue?
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
BlueData MLOPS - PermissionError: [Errno 13] Permission denied
I've created a MLOPS project on BlueData 4.0 and mounted the Project Repo (NFS) folder. I created the NFS service on Centos 7x as below:
sudo yum -y install nfs-utils
sudo mkdir /nfsroot
echo '/nfsroot *(rw,no_root_squash,no_subtree_check)' |…
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
bluedata mlops prediction - 'NoneType' object does not support item assignment
I'm calling the BlueData MLOPS prediction API to get a score for my model, but I'm getting the following response:
============================response begin==========================================
Status code : {}200
Status text : {}OK
Headers …
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
How to read and write from datatap using Tensorflow on BlueData?
I want to be able to use BlueData's datatap directly from TensorFlow.
With pyspark, I can do something like this:
df.write.parquet('dtap://OtherDataTap/airline-safety_zero_incidents.parquet')
Note that I don't need to set up any libraries - it's…
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
How to debug "ERROR: Could not reach the worker node."?
I am trying to set up a gateway and a worker node on a BlueData 3.7 controller using SSH credentials.
I believe that I have met all the pre-requisites in the docs, however, I get an error trying to install the gateway and the worker in the…
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
How to add the lustre file system client to a BlueData container?
I'm trying to set up a lustre client (docs) inside a docker container running on BlueData.
As per this post, I've modified the BlueData config on each worker and the controller node:
$ vi…
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
How to work with sessions with the BlueData REST API?
I would like to use the BlueData API to programmatically manage by BlueData installation.
I have seen some documentation here - it appears that I need to start off with Fetching a session but I would expect to create a session before I can fetch…
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
How to read data from DataTap using pyspark on Cloudera 5.x?
I have created a Cloudera 5.x cluster with the Spark option set:
I would like to run a simple test using PySpark to read data from one Datatap and write it to another Datatap.
What are the steps for doing this with PySpark?
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
Starting Kibana - "No such container: monitoring-x.x.x.x"
I'm working through the BlueData docs to enable Kibana.
To start Kibana:
Login to the Controller.
Execute the command docker exec -it monitoring- bash, where is the IP address of the Controller.
...
I've ssh'd into…
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
How do I pass docker parameters such as `--cap-add=XXX` to my docker instances running in BlueData?
I would like to run a container with --cap-add=IPC_LOCK.
According to the BlueData 3.7 release notes, IPC_LOCK is supported:
HAATHI-13547: Docker configuration now includes default IPC_LOCK capability for all deployed containers. IPC_LOCK is the…
Chris Snow
- 23,813
- 35
- 144
- 309
0
votes
1 answer
How to create credentials for Jupyterhub?
I have provisioned a Spark 2.3.1 notebooks & Jupyterhub with SSL & centos7x cluster on BlueData.
When I access the Jupyerhub service, I am prompted for a username and password:
My BlueData Tenant:
is NOT configured for LDAP/AD
has Cluster…
Chris Snow
- 23,813
- 35
- 144
- 309