Stack Overflow
Stack Overflow

Questions tagged [gcs]

237 questions
1
vote
2 answers

Where do I find my Google Cloud Storage Bucket access key?

I am trying to set up my Google Cloud Storage Bucket and need the JSON key structure, but I cannot find it. Where do I find the GCS access key?
user3877654
  • 1,045
  • 1
  • 16
  • 40
1
vote
0 answers

How do I link images I uploaded to GCS to my React front end?

I'm having a hard time figuring out how to link the files I upload to Google Cloud. I found that the link to the images I'm uploading are formatted like this: storage.googleapis.com/bucket/image.jpeg But the name of the image keeps getting…
Vikalp Malhotra
  • 11
  • 2
1
vote
1 answer

How reliable is GCP Transfer Service for transferring a lot of incoming files from on-premise to a bucket?

I've got a process, which constantly creates files. You can imaging some data stream and files are created by minutes from this stream. I've got hand-written code which reads files and upload them a to GCP bucket. Recently, I came across GCP…
Alexander Goida
  • 305
  • 1
  • 11
1
vote
2 answers

Google cloud storage equivalent for S3 multipart upload

We are migrating from S3 to GCS and one of the functionalities that needs to be supported in GCS is the ability to upload large files in a multi-part fashion. S3 boto provides us with functions like initiate_multipart_upload and copy_part_from_key…
Fremzy
  • 290
  • 2
  • 12
1
vote
0 answers

Python - Apache-beam outputs an empty file using dataflow runner, works fine with direct runner. Dataflow does not raise any errors

I've been trying to run this apache-beam script. This script runs nightly through an airflow DAG, and works perfectly fine that way so I'm (reasonably) confident that the script is correct. I think the relevant part of my apache script is summarized…
Nyrb
  • 11
  • 2
1
vote
1 answer

Reading data from GCS with BigQuery fails with "Not Found", but the date (files) exists

I have a service that is constantly updating files in GCS bucket with hive format: bucket device_id=aaaa month=01 part-0.parquet month=02 part-0.parquet .... device_id=bbbb month=01 …
shlomiLan
  • 659
  • 1
  • 9
  • 33
1
vote
0 answers

How to have multiple state files for a workspace in terraform gcs?

In AWS, when i set the backend provider to be S3, I'm able to set multiple keys. I can have one key for the database, another key for the cluster, etc. key - (Required) Path to the state file inside the S3 Bucket. When using a non-default workspace,…
jdkealy
  • 4,807
  • 6
  • 34
  • 56
1
vote
2 answers

Using external tables without granting access to GCS

We have 2 GCP projects project-a and project-b. And we want to give access to some external tables in the first project to users in second project using Authorized Views. Here's what we've done so far: create a couple of BigQuery tables in…
blackbishop
  • 30,945
  • 11
  • 55
  • 76
1
vote
2 answers

How to properly iterate Bigquery TableResult in Java

I am trying to iterate the rows from TableResult using getValues() as below. if I use getValues(), it's retrieving only the first page rows. I want to iterate all the rows using getValues() and NOT using iterateAll(). In the below code, the problem…
1stenjoydmoment
  • 229
  • 3
  • 14
1
vote
1 answer

Ways to using Google storage buckets on frontend by token without user creation

I have a client - server application that stores files on the server, however I would like to integrate google cloud. I must say right away that the user can log in by email, as well as by phone (in this case, he will not have an email). My main…
Роман Сергеевич
  • 65
  • 4
  • 12
1
vote
1 answer

Persisting Python error: No module named 'google'

I have 2 scripts for GCS (Google Cloud Storage) to upload and download files. I installed google.cloud using pip install google. My scripts were working, but now I see error: ModuleNotFoundError: No module named 'google' I read many questions here…
AnnaSh
  • 11
  • 1
  • 4
1
vote
1 answer

GCP Bucket name not visible/not available in metrics Explorer

I have a bucket which I created using Terraform. I have regular uploads on this bucket and everything is working fine. Now, I want to monitor the object count on this bucket, when I go to monitoring and select GCS Bucket and bucket name filter, it…
Sam
  • 161
  • 1
  • 9
1
vote
0 answers

Getting error while using google data transfer to transfer multiple Json file from GCSto Bigquery

I'm getting error while I'm trying to transfer file from Google cloud storage to google big query. This is the error : failed with error INVALID_ARGUMENT : Error while reading data, error message: JSON table encountered too many errors, giving up.…
Ernest Fokam
  • 31
  • 5
1
vote
1 answer

Image-processing gem creates resized file in GCS

I have Item object that #has_many_attached :photos. My Active storage service is GCS. I created a #thumbnail() instance method for Item. Like this: def thumbnail(photo) photo.variant(resize_and_pad: [400, 400, gravity: 'center', background:…
Murat Ogulcan Sahin
  • 112
  • 8
0
votes
1 answer

Terraform init puts an empty state file in my remove GCS backend but then creates a local state file

I'm running terraform from a GCP compute instance to create projects using https://github.com/terraform-google-modules/terraform-google-project-factory The definition seems correct--terraform creates a bare state file in the GCS bucket, but then…
ronin22
  • 1
  • 1
1 2 3
…
15 16