Stack Overflow
Stack Overflow

Questions tagged [google-cloud-bigtable]

Google Cloud Bigtable is a fast, fully managed, massively scalable NoSQL database service designed for applications requiring terabytes to petabytes of data. Cloud Bigtable doesn't require you to sacrifice speed, scale, or cost efficiency when your applications grow.

Google Cloud Bigtable is a fast, fully managed, massively scalable NoSQL database service designed for applications requiring terabytes to petabytes of data. Cloud Bigtable doesn't require you to sacrifice speed, scale, or cost efficiency when your applications grow.

Cloud Bigtable is exposed to applications through a supported extension to the Apache HBase 1.0 Java library.

628 questions
5
votes
3 answers

Detect Google Cloud Project Id from a container in Google hosted Kubernetes cluster

Detect Google Cloud Project Id from a container in Google hosted Kubernetes cluster. When connecting to BigTable; I need to provide the Google Project Id. Is there a way to detect this automatically from within K8s?
brent
  • 1,095
  • 1
  • 11
  • 27
5
votes
1 answer

How to read and write data in Google Cloud Bigtable in PySpark application?

I am using Spark on a Google Cloud Dataproc cluster and I would like to access Bigtable in a PySpark job. Do we have any Bigtable connector for Spark like Google BigQuery connector? How can we access Bigtable from a PySpark application?
Revan
  • 541
  • 1
  • 5
  • 13
5
votes
2 answers

Google Cloud Bigtable: query partial keys

So if I have the following data in Bigtable: DEL_6878 ..... DEL_6879 ..... BOM_5876 ..... SFO_8686 ..... SFO_8687 ..... How do I query for say SFO* records? I read the documentation; I know how to get single row; something similar to this:…
Amit
  • 1,836
  • 15
  • 24
5
votes
2 answers

Achieving declared Cloud Bigtable writes QPS

We've setup Bigtable cluster with 5 nodes, and GCP console states that it should support 50K QPS @ 6ms for reads and writes. We are trying to load a large dataset (~800M records) with ~50 fields containing mostly numeric data, and a few short…
Leon Stein
  • 63
  • 6
5
votes
2 answers

Does Cloud Bigtable have a local development server like Cloud Datastore?

If not, is there any document that list best practices to integration-test an application that uses Cloud Bigtable? Specifically the applications written in Java and Python. Ref: https://cloud.google.com/datastore/docs/tools/devserver
Vinay
  • 80
  • 4
4
votes
1 answer

Migration from DynamoDB to Spanner/BigTable

I have a use case where I need to migrate 70 TB of data from DynamoDB to BigTable and Spanner. Tables with a single index will go to BigTable else they will go to Spanner. I can easily handle the historical loads by exporting the data to S3 --> GCS…
rish0097
  • 1,024
  • 2
  • 18
  • 39
4
votes
1 answer

From Google Cloud Bigtable to AWS EMR (HBase)

How to export HBase table from Cloud Bigtable to AWS EMR? I know there's an official way to use Google Cloud Dataflow runner for exporting HBase tables to Google Storage. But the exported file is somehow encoded that I have no way to use it except…
Allen Huang
  • 61
  • 1
  • 5
4
votes
2 answers

BigTable: When should I enable Single-Row Transaction?

Cloud Bigtable docs on Single-row Transactions says: Cloud Bigtable also supports some write operations that would require a transaction in other databases: Read-modify-write operations, including increments and appends. A read-modify-write…
Gabriel
  • 809
  • 1
  • 10
  • 21
4
votes
1 answer

Connect to Bigtable emulator from localhost with Node.js client

Trying to connect to Cloud Bigtable emulator from localhost. Saw couple of posts on how to connect to localhost Bigtable emulator with Java. There is no documentation that specifies how to do so with Node.js. @google-cloud/bigtable client needs…
Aparna Kulkarni
  • 43
  • 5
4
votes
2 answers

Performance tuning tips for Google Cloud Bigtable

I'm using a collection of BT tables to store data that's being used for both batch and realtime operations, and want to optimize performance, especially around latency of random access reads. And while I do know the underlying BT codebase fairly…
Yonatan Zunger
  • 43
  • 3
4
votes
2 answers

How to design Bigtable Key

I would create my optimal key in Bigtable. I know the key design is crucial for query speed and optimization. My case is related to a time series information from a network machine. It is a tall and narrow table with 3 columns: id, info and…
lordcenzin
  • 339
  • 3
  • 15
4
votes
1 answer

Cloud Bigtable multi-prefix scan in dataflow

UPDATE: it seems that the recently released org.apache.beam.sdk.io.hbase-2.6.0 includes the HBaseIO.readAll() api. I tested in google dataflow, and it seems to be working. Will there be any issue or pitfall of using HBaseIO directly in Google Cloud…
CHZ
  • 41
  • 2
4
votes
0 answers

How to write data in Google Cloud Bigtable in PySpark application on dataproc?

I am using Spark on a Google Cloud Dataproc cluster and I would like to write in Bigtable in a PySpark job. As google connector for the same is not available, I am simply using google cloud bigtable client to insert the data and use spark for…
MANISH ZOPE
  • 1,181
  • 1
  • 11
  • 28
4
votes
1 answer

Why does BigTable have column families?

Why is BigTable structured as a two-level hierarchy of "family:qualifier"? Specifically, why is this enforced rather than just having columns and, say, recommending that users name their qualifiers "vertical:column"? I am interested in whether or…
user3038457
  • 185
  • 2
  • 11
4
votes
1 answer

How to delete a column of a single row in Google Cloud Bigtable with HBase API

I'm using the HBase API to access Google Cloud Bigtable, but whenever I try to delete a column: Delete delete = new Delete(r.getRow()); delete.addColumn(CF, Bytes.toBytes(d.seqid())); delete.addColumn(CF, COL_LEASE); tasksTable.delete(delete); I'm…
Chris
  • 470
  • 4
  • 11
1 2
3
…
41 42