IBM InfoSphere BigInsights brings the power of Hadoop to the enterprise. Apache Hadoop is the open source software framework, used to reliably manage large volumes of structured and unstructured data.
Questions tagged [biginsights]
103 questions
2
votes
2 answers
Copy files from Remote Unix and Windows servers into HDFS without intermediate staging
How can I copy files from remote Unix and Windows servers into HDFS without intermediate staging from the command line?
buckeyeosu
- 45
- 8
2
votes
1 answer
Inconsistent runtime kernel parameters in DOCKER container and on host
my host is on Ubuntu 14.04.2 LTS and I'm using the latest centos base image in order to create a DOCKER image of IBM InfoSphere BigInsights in order to push it to the Bluemix Container Cloud.
I've solved nearly everything but I'm stuck with setting…
Romeo Kienzler
- 3,373
- 3
- 36
- 58
2
votes
1 answer
gpfs file access through Java API/Python
I am new to IBM-GPFS.
As we can access HDFS via Java API, python scripting.
Is there any similar way to access GPFS files?
Please suggest me the solution.
Vivek Nimje
- 21
- 2
1
vote
1 answer
HDFS-GPFS connector for using in Apache Spark
Is there a possibility to read data from IBM GPFS (Global Parallel Filesystem) in Apache Spark ?
My intention is to use something like this
sc.textFile("gfps://...")
instead of
sc.textFile("hdfs://...")
The environment that is intended to be used…
dumitru
- 2,068
- 14
- 23
1
vote
1 answer
Data Science Experience responds with an empty Hive table
From my Data Science Experience, I am able to make a connection to the Hive database in BigInsights and read the table schema. But Data Science Experience does not seem to be able to read the table contents as I get a count of zero! Here are some of…
Nitesh
- 51
- 4
1
vote
0 answers
Maven repo for BigInsights / IOP
I have a IBM BigInsights 4.2 cluster installed and I am trying to create native yarn jobs.
To compile the java code on my laptop (not on the cluster), I need the hadoop client libraries.
How can I get the hadoop java libraries for IBM BigInsigths…
Renaud Demarneffe
- 11
- 2
1
vote
1 answer
Spark Hive reporting ClassNotFoundException: com.ibm.biginsights.bigsql.sync.BIEventListener
I'm attempting to run a pyspark script on BigInsights on Cloud 4.2 Enterprise that accesses a Hive table.
First I create the hive table:
[biadmin@bi4c-xxxxx-mastermanager ~]$ hive
hive> CREATE TABLE pokes (foo INT, bar STRING);
OK
Time taken: 2.147…
Chris Snow
- 23,813
- 35
- 144
- 309
1
vote
3 answers
Spark Hive reporting pyspark.sql.utils.AnalysisException: u'Table not found: XXX' when run on yarn cluster
I'm attempting to run a pyspark script on BigInsights on Cloud 4.2 Enterprise that accesses a Hive table.
First I create the hive table:
[biadmin@bi4c-xxxxx-mastermanager ~]$ hive
hive> CREATE TABLE pokes (foo INT, bar STRING);
OK
Time taken: 2.147…
Chris Snow
- 23,813
- 35
- 144
- 309
1
vote
1 answer
hive spark yarn-cluster job fails with: "ClassNotFoundException: org.datanucleus.api.jdo.JDOPersistenceManagerFactory"
I'm attempting to run a pyspark script on BigInsights on Cloud 4.2 Enterprise that accesses a Hive table.
First I create the hive table:
[biadmin@bi4c-xxxxx-mastermanager ~]$ hive
hive> CREATE TABLE pokes (foo INT, bar STRING);
OK
Time taken: 2.147…
Chris Snow
- 23,813
- 35
- 144
- 309
1
vote
1 answer
How can I use python > 2.6.6 with spark on BigInsights on cloud Enterprise clusters?
The version of python with BigInsights is currently 2.6.6. How can I use a different version of Python with my spark jobs running on yarn?
Note that users of BigInsights on cloud do not have root access.
Chris Snow
- 23,813
- 35
- 144
- 309
1
vote
1 answer
BigInsights Example KnoxShell on Windows
Trying to use the KnoxShell example from Windows (using Cmder as shell emulator). When executing the following command, the shell locks, no prompt available and nothing happend. I can only close Shell.
Is that specific to Windows?
λ gradlew.bat -q…
Victor
- 243
- 1
- 3
- 9
1
vote
1 answer
BigInsights on cloud - Class org.apache.oozie.action.hadoop.SparkMain not found
I'm trying to execute the spark oozie example on the oozie_spark branch against a BigInsights for Apache Hadoop basic cluster.
The workflow.xml looks like this:
Chris Snow
- 23,813
- 35
- 144
- 309
1
vote
1 answer
SparkR job(R script) submit using spark-submit fails in BigInsights Hadoop cluster
I have created IBM BigInsights service with hadoop cluster of 5 nodes(including Apache Spark with SparkR). I trying to use SparkR to connect cloudant db and get some data and do some processing.
SparkR job(R script) submit using spark-submit fails…
Pari Margu
- 209
- 3
- 10
1
vote
2 answers
Message: RSA premaster secret error. ERRORCODE=-4499, SQLSTATE=08001
I'm trying to connect to BigSQL on a BigInsights on Cloud cluster from an iPython notebook on Bluemix Spark as a service.
My notebook looks like this:
import os
cwd = os.getcwd()
with open('certificate', 'w') as f:
f.write('''-----BEGIN…
Chris Snow
- 23,813
- 35
- 144
- 309
1
vote
3 answers
BigSheets - java.io.IOException: java.lang.reflect.UndeclaredThrowableException
I'm unable to browse data with BigSheets on BigInsights on Cloud. When I select a file and change the Reader type, I receive the exception (see screenshot, below).
I was previously able to browse data ok.
Chris Snow
- 23,813
- 35
- 144
- 309