Stack Overflow
Stack Overflow

Questions tagged [bigsql]

IBM® Big SQL is one of two connection environments to query data that is stored on an InfoSphere® BigInsights™ Hadoop cluster.

IBM® Db2® Big SQL is a hybrid SQL engine for Hadoop that delivers data querying for the enterprise. Use a single database connection or query to connect to disparate sources such as HDFS, RDMS, NoSQL databases, object stores and WebHDFS.

47 questions
0
votes
1 answer

Storing text more than 32762 chars in BigSQL Hadoop external table

I want to store a text of size 70000, but BigSQL Hadoop extrernal table restricts maximum field length to 32762. I do not want to trim or split into multiple columns. Is there any other datatype that let's me to load the full data.
Pari Sairam Mohan
  • 421
  • 1
  • 4
  • 11
0
votes
2 answers

Suppressing Headers while using DB2CLI

While using DB2CLI command,we get IBM copyright information and footer details. Is there any option to suppress header and footer information ? echo "select CURRENT USER from dual"|db2cli execsql -dsn 'SAMPLE' Output PS C:\Users\Desktop> echo…
SSE
  • 445
  • 2
  • 10
  • 29
0
votes
1 answer

How to store string data in BIGSQL table with length greater than VARCHAR(32k)

The data beyond 32762 characters is getting truncated when I am loading the data in my BigSQL table. My table definition is as below : CREATE hadoop TABLE schema_name.table_name ( column1 VARCHAR(50), column2 INTEGER, column3 STRING, …
tavishbhagat
  • 11
  • 3
0
votes
1 answer

How to read subdirectory data into bigsql table?

Bigsql is not able to read data from subdirectory like hive can by setting parameters. set tblproperties ( "hive.input.dir.recursive" = "TRUE", "hive.mapred.supports.subdirectories" = "TRUE", "hive.supports.subdirectories" = "TRUE", …
Pra
  • 31
  • 6
0
votes
1 answer

why there is a data mismatch in hive and bigSQL by 1 record?

I have created a hive table and integrated it to bigSQL. in the hive my count is proper, but in bigSQL, the record count is extra by 1. Below is the table properties that I have used to create the hive table. create table test(name string,age…
learner
  • 73
  • 2
  • 9
0
votes
2 answers

how to use Big SQL commands to automate the synchronization with HIVE through shell script?

I have written a small shell script to automate the Big SQL and HIVE synchronization. Code is as below echo "Login to BigSql" /jsqsh bigsql --user=abc --password=pwd echo "login succesfull" echo "Syncing hive table to Big…
learner
  • 73
  • 2
  • 9
0
votes
1 answer

With Pushdown query in spark, how to get parallelism in spark-HBASE (BIGSQL as SQL engine)?

In Spark PushdownQuery is processed by SQL Engine of the DB and with the result from it, dataframe is constructed. so, spark querying the results of that query. val pushdownQuery = """(SELECT DISTINCT(FLIGHT_NUMBER) blah blah blah )""" val…
Karthick
  • 662
  • 5
  • 14
0
votes
1 answer

java.lang.ClassNotFoundException: com.ibm.db2.jcc.DB2Driver exception for connecting BigSQL using Python

I'm new to pyspark.I'm using python 3.5 & spark2.2.0 on my Ubuntu 16.0. I wrote following code to connect BigSQL using pyspark from pyspark.sql.session import SparkSession spark = SparkSession.builder.getOrCreate() spark_train_df =…
Python Learner
  • 437
  • 2
  • 11
  • 28
0
votes
2 answers

How to connect from spark 1.6 to bigsql

I am newbie on Big Insights. I am working on BigInsigths on cloud 4.1, Ambari 2.2.0 and Spark 1.6.1 It doesn't matter if the connection is in scala or python, but I need to do data processing with spark and then persist it in BigSql. Is this…
JohanaAnez
  • 31
  • 5
0
votes
1 answer

Connect BigSQL using Python through Java

I'm trying to connect python to BigSQL.I have a java code which can connect python to BigSQL & retrieve data from BigSQL.Below is my sample code public class Hello { public static void main(String[] args) { try(Connection con =…
Python Learner
  • 437
  • 2
  • 11
  • 28
0
votes
0 answers

How to connect BigSQL using python

I'm new to BigSQL.I would like to know how can I connect BigSQL using python 3.5 Thanks in advance,
Python Learner
  • 437
  • 2
  • 11
  • 28
0
votes
2 answers

Syncing BigSQL data with Hive

I want to create a Hive table/view which will be accessing BigSQL (BigInsights 4.2) table. Data will be loaded in BigSQL table and I'm trying to fetch that data from Hive. Is there any procedure to sync data from BigSQL and Hive table?
Sourav Mallik
  • 3
  • 3
0
votes
0 answers

Remote access to EnterpriseDB PostgreSQL

When I install PostgreSQL from BigSQL I can connect from QGIS on other PC to my local PostgreSQL Server (both PCs are on the same local network 192.168.100.0/24). But then I can't install PostGIS because it always says $libdir/postgis-2-3 can't be…
Aleksandar
  • 1,163
  • 22
  • 41
0
votes
0 answers

HBase Secondary Indexing performance (BIGSQL VS PHOENIX)

Hi in my workplace we are using IBM BigInsights,we create secondary indexes on hbase table using BIGSQL, if the secondary indexes are present the row ingestion speed drops from 3000 rows /sec to 200 rows/sec. How is the performance of phoenix in…
siva krishna
  • 23
  • 5
0
votes
0 answers

BigSQL PostgreSQL CREATE EXTENSION errors

I have windows 8.1 and I want to try hadoop_fdw. Problem is with loading the extension with: CREATE EXTENSION hadoop_fdw; ddssdg ERROR: could not load library "D:/PROGRA~2/POSTGR~2/pg95/../pg95/lib/postgresql/hadoop_fdw.dll": unknown error…
dkocich
  • 221
  • 4
  • 7
1
2
3 4