Questions tagged [gcs]
237 questions
-1
votes
1 answer
Google Cloud Storage - Perform resumable uploads - In multiple chunks
Using cURL to call the JSON API with a PUT Object request is not working
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
curl -i -X PUT --data-binary @CHUNK_LOCATION
-H "Content-Length: CHUNK_SIZE"
-H "Content-Range:…
Srikanth Koya
- 1
- 1
-1
votes
1 answer
How to calculate cost in for GCS rewrite operation
I recently switched the key of a bucket from being managed by Google to being managed by the customer. However, the existing files in the bucket are still using the old key. To resolve this, I need to rewrite the files so that they can utilize the…
vignesh
- 1,414
- 5
- 19
- 38
-1
votes
2 answers
Getting the filename with on_bad_line
I am trying to get the filename of where the bad line is happening. I have a function for getting all the bad lines and printing it to a .txt file, but when I pass in a parameter for the filename, it just prints all the filenames.
This is the bad…
DarrenC
- 11
- 3
-1
votes
1 answer
Removing entire string duplicates from a list
I am running into an issue when trying to removing duplicates from a list.
def my_list_bucket(self, bucketName, limit=sys.maxsize): #delimiter='/'):
a_bucket = self.storage_client.lookup_bucket(bucketName)
bucket_iterator =…
DarrenC
- 11
- 3
-1
votes
1 answer
GCS Notifications not shown in Pub/Sub Topic message
We have a GCS bucket named 'testfiles' and Pub/Sub topic 'testtopic' with a subscription 'testsubscription'. We have created a notification configuration to receive notifications on the Pub/Sub topic for any event happening on the GCS bucket. When…
Sabari
- 127
- 1
- 13
-1
votes
1 answer
Want to transfer file from private on-premise Instance (VM) to google cloud storage without internet
I want to transfer 2TB of files from my on-premise VM(which does not have internet connection) to google cloud storage. I have established shared VPC and my private connectivity are there. I don't want to enable internet on my on-premise VM , How…
-1
votes
1 answer
What is the difference?
When the total record count is higher (in millions ) then the iterator_1 is set with very minimal value like in a few thousand.
But when the data is less, both are having the same set of records. In iterator_1 fetching only the values and then…
1stenjoydmoment
- 229
- 3
- 14
-1
votes
1 answer
(Retool) Make a ".txt" file to GCS but I got the error "Failed to execute 'atob' on 'Window': The string to be decoded is not correctly encoded."
On Retool, I'm trying to make a ".txt" file with some words to Google Cloud Storage(GCS) but I got this error.
makeTXTFile: Failed to execute 'atob' on 'Window': The string to be
decoded is not correctly encoded.
This is the query settings on…
Super Kai - Kazuya Ito
- 22,221
- 10
- 124
- 129
-1
votes
2 answers
Delete multiple files(objects) listed in a ".txt" file with "gsutil" (GCS)
I could delete the 3 files(objects), "apple.jpg", "orange.png" and "kiwi.gif" on GCS with the "gsutil" command below on Cloud Shell:
(Hint: gs:///)
gsutil rm gs://test.com/apple.jpg gs://test.com/orange.png…
Super Kai - Kazuya Ito
- 22,221
- 10
- 124
- 129
-2
votes
1 answer
PubSub Publisher Ingestion
I am trying to ingest an avro file from a gcs to pubsub. Have some layman questions.
What are the options to send a file as a message in PubSub?
Like can we send the whole file as a message or only the contents of the file can be iterated and send…
mehere
- 1,487
- 5
- 28
- 50
-2
votes
1 answer
after typing gcsfuse gs://bucketname/* /home/admin/(directory in vm instance), it's showing successfully mounted. but when I am typing ls, it's 0
I also did 'df -h' to see if it's properly mounted or not and then it's showing it is:
gs://bucketname/* 1.0P 0 1.0P 0% /home/admin/(directory in vm instance)
But inside the directory in vm instance when I am typing ls -l , I am gettin total…
neo
- 55
- 8
-2
votes
1 answer
How to config GCS bucket permission with terraform?
I want to set permission for service account sa-email.com on bucket bucket_A.
How can I configure as below with terraform?
sa-email.com can create file in bucket_A
sa-email.com can not delete file in bucket_A
UPDATE:
Now in my project, sa-email is…
ender1986
- 95
- 1
- 9