Stack Overflow
Stack Overflow

Questions tagged [google-cloud-bigtable]

Google Cloud Bigtable is a fast, fully managed, massively scalable NoSQL database service designed for applications requiring terabytes to petabytes of data. Cloud Bigtable doesn't require you to sacrifice speed, scale, or cost efficiency when your applications grow.

Google Cloud Bigtable is a fast, fully managed, massively scalable NoSQL database service designed for applications requiring terabytes to petabytes of data. Cloud Bigtable doesn't require you to sacrifice speed, scale, or cost efficiency when your applications grow.

Cloud Bigtable is exposed to applications through a supported extension to the Apache HBase 1.0 Java library.

628 questions
0
votes
2 answers

Google Cloud Bigtable Instance

I am working on an IOT application which reads data from some sensors and I need to populate this data into a Google Cloud Bigtable instance. I am using Java 1.8 for development and I will be using HBase APIs for Bigtable related operations. Can…
jumping jack
  • 51
  • 5
0
votes
1 answer

What is the best practice for accessing Bigtable from streaming dataflow

I need to access the bigtable in one of the transformation from streaming dataflow job.As per my knowledge there are two ways : 1) we can create the conneciton to bigtable from startBundle method of DoFn and access data from bigtable in…
ganesh_patil
  • 356
  • 3
  • 18
0
votes
1 answer

Scanning entire bigtable (of a specific column family) via Dataflow

We use 50-100 bigtable nodes (depending on the amount of data we process, this number varies between 50 and 100 throughout the day). Every day we have a dataflow job which scans the entire bigtable (of one specific column family), and dumps the…
Haden Hooyeon Lee
  • 295
  • 2
  • 15
0
votes
1 answer

Sharing BigTable Connection object among DataFlow DoFn sub-classes

I am setting up a Java Pipeline in DataFlow to read a .csv file and to create a bunch of BigTable rows based on the content of the file. I see in the BigTable documentation the note that connecting to BigTable is an 'expensive' operation and that…
VS_FF
  • 2,353
  • 3
  • 16
  • 34
0
votes
1 answer

Exporting/importing specific column families of bigtable

My team is considering exporting/importing bigtable data, but just a subset of column families (not all) for technical issues. These docs seem to suggest that export/import is possible only for the entire…
Haden Hooyeon Lee
  • 295
  • 2
  • 15
0
votes
1 answer

From Bigtable To GCS (and vice versa) via Dataflow

We are trying to run a daily Dataflow pipeline that reads off Bigtable and dumps data into GCS (using HBase's Scan and BaseResultCoder as coder) as follows (just to highlight the idea): Pipeline pipeline = Pipeline.create(options); Scan scan =…
Haden Hooyeon Lee
  • 295
  • 2
  • 15
0
votes
3 answers

Bigtable/BigQuery pricing when inserts depend on lookups

I have a simple proof-of-concept application written in traditional SQL. I need to scale it to much larger size (potentially trillions of rows, multiple terabytes or possibly petabytes in size). I'm trying to come up with the pricing model of how…
VS_FF
  • 2,353
  • 3
  • 16
  • 34
0
votes
2 answers

Managing Google Cloud Bigtable from SQLAlchemy

It seems that SQL Alchemy can connect to MySQL table running on Google Cloud SQL. However, I spent time to look for wrapper of Google Cloud Bigtable, a NoSQL database, and could not find anything enough mature. Just wondering how to manage Google…
tensor
  • 3,088
  • 8
  • 37
  • 71
0
votes
2 answers

Does standard (non-flexible) Google App Engine support Google Cloud Bigtable?

Is there an official page, where this question is answered? In Nov 2016, I am trying to run some Java Bigtable code and it fails on both local dev server and on a real remote instance. I have tried different versions of both libraries. Appengine…
Aram Paronikyan
  • 1,598
  • 17
  • 22
0
votes
2 answers

Google Cloud Bigtable Python Client API

Google Bigtable Python Client Library is still in Alpha, which means future changes might not be backward compatible and no SLA. Does anyone know when will the library have production support? To give more context, I am debating Bigtable vs.…
YSC
  • 379
  • 1
  • 3
  • 13
0
votes
1 answer

Recommended nodes per TB of data in bigtable

I am looking to build a single (history) table in Bigtable that will be 30TB and growing. I expect a single daily batch load of 200GB, and occasional batch queries within a key range. What guidance is available for the number of TB per node in…
HFC
  • 1
0
votes
1 answer

google cloud dataflow cross project access for big table

I want to run dataflow job to migrate data from google-project-1-table to google-project-2-table. (Read from one and write to another). I am getting permission issue while doing that. I have set "GOOGLE_APPLICATION_CREDENTIALS" to point to my…
Omkar Joshi
  • 1
  • 2
0
votes
1 answer

com.google.cloud.dataflow.sdk.util.IllegalMutationException in DoFn converting to org.apache.hadoop.hbase.client.Put

I am trying to convert an object into a org.apache.hadoop.hbase.client.Put in a DoFn. I keep getting this error: com.google.cloud.dataflow.sdk.util.IllegalMutationException: DoFn DataBuckettingToBigTableConnectorFn mutated value…
user1568967
  • 1,816
  • 2
  • 16
  • 18
0
votes
1 answer

Google Big Table Vs Big Query for ad server data analysis

We are looking to build a solution on GCP for campaign/ad analytics (ingest doubleclik and other ad-server data into DW). Data is ingested as batch, with star schema but will have updates trickling in for up to a week, need trend analysis for…
Tim S
  • 185
  • 1
  • 13
0
votes
2 answers

Bigtable 'write-requests' is not consistent

I am writing data into BigTable from google storage using dataflow job, I am using 3 nodes BigTable cluster and has 25 workers working in parallel in my dataflow job When I check the 'Write-requests' graph of the Big table then what I observe it…
Amandeep
  • 325
  • 1
  • 3
  • 11
1 2 3
…
41 42