Stack Overflow
Stack Overflow

Questions tagged [s3cmd]

S3CMD QUESTIONS MUST BE PROGRAMMING RELATED. s3cmd is a command line tool for uploading, retrieving and managing data in Amazon S3. It is best suited for power users who don't fear command line. It is also ideal for scripts, automated backups triggered from cron, etc.

S3CMD QUESTIONS MUST BE PROGRAMMING RELATED. s3cmd is a command line tool for uploading, retrieving and managing data in Amazon S3. It is best suited for power users who don't fear command line. It is also ideal for scripts, automated backups triggered from cron, etc.

See the s3cmd homepage for more information.

282 questions
0
votes
0 answers

How can I run s3cmd from anywhere, Windows 10

I pip installed s3cmd into my conda environment. I would like to be able to run this script the way the guide shows https://github.com/s3tools/s3cmd i.e s3cmd --configure The .py file installed itself…
GlaceCelery
  • 921
  • 1
  • 13
  • 30
0
votes
2 answers

AWS S3 Transfer file with expire days with s3cmd by perl

Can you help me with that script? I would like to transfer files to my bucket on S3 AWS. My code: $cmd = "s3cmd -v -c /root/.s3cfg put /var/project_db_" . $date . ".sql.xz s3://bucket600"; My second code - What I'm using. And doesn't work. $cmd =…
Małgorzata Urbańska
  • 11
  • 4
0
votes
1 answer

Using s3cmd, how do I retreived the newest folder by "Last modfied" date in an s3 directory

I have a directory containing folders whose folder names are created using timestamps. I want use s3cmd to find the file with the most recent "Last Modified" value. If that is not possible, are the solutions to these previous questions the way to…
weatherpattern
  • 3
  • 1
0
votes
1 answer

AWS JS SDK S3 use custom configuration file

I'm replacing s3cmd usage with AWS JS SDK. In s3cmd command there is an option to set a custom config file: s3cmd -c "/path/to/config". But I can't find how to use the same configuration file in "aws-sdk" (shared credentials file)? My code looks…
v.babak
  • 818
  • 11
  • 13
0
votes
1 answer

Github Action: s3cfg file is not found in /home/runner

In my github action workflow, I want to upload a text file to s3. in my projects root folder I have created .s3cfg file this is my action.yml file. name: Github Action on: push: branches: - fix/build jobs: test: runs-on:…
Ashik
  • 2,888
  • 8
  • 28
  • 53
0
votes
1 answer

AWS s3 sync is slow to copy from s3 to EMR

AWS s3 sync is slow. Also, AWS s3 sync silently fails. s3 mv deletes target files on failure and cleans up. However, it does not attempt to copy(needs error handling). Whats my best option to move files from s3 to EMR? - s3 cmd? - s3 mv? Any other…
user11099691
  • 1
0
votes
0 answers

What do PYTHONHOME and PYTHONPATH vars need to be set to for s3cmd on Mac?

I just installed s3cmd via homebrew on Mac High Sierra. Immediately when I go to run it, I get this error ... davea$ s3cmd Python path configuration: PYTHONHOME = '/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7' …
Dave
  • 15,639
  • 133
  • 442
  • 830
0
votes
1 answer

s3cmd failed after try to create a testbucket on ceph-cluster

Hello I am trying to create a testbucket on a Ceph Raspberry Pi cluster(local) and I get the following error message: OS:Debian Jessie Ceph: v12.2.12 Luminous s3cmd:2.0.2 [ceph_deploy.rgw][INFO ] The Ceph Object Gateway (RGW) is now running on host…
optionalt123
  • 39
  • 5
0
votes
2 answers

Transfer image files from MongoDB to S3

I have a NodeJS app that stores all of its data, even images, to MongoDB. Now I want to use an S3 to store my app's files (images, 43gb) and so, I need to transfer there all files that I have in my mongo. I found the s3cmd tool that can transfer my…
Sotiris Kaniras
  • 520
  • 1
  • 12
  • 30
0
votes
1 answer

s3cmd S3 Endpoint and DNS-Style bucket is missing on configure

I am using on s3cmd for reactJS web upload, I am doing configure and it able to access without issue but I notice there is missing on s3 endpoint and DNS bucket host it should show like this. as you can see s3 endpoint and DNS bucket host( this…
Steven Smith
  • 406
  • 5
  • 19
0
votes
2 answers

How to download S3-Bucket, compress on the fly and reupload to another s3 bucket without downloading locally?

I want to download the contents of a s3 bucket (hosted on wasabi, claims to be fully s3 compatible) to my VPS, tar and gzip and gpg it and reupload this archive to another s3 bucket on wasabi! My vps machine only has 30GB of storage, the whole…
Markus
  • 1
  • 1
  • 3
0
votes
1 answer

s3cmd ERROR: S3 error: 301 - Upload files

s3cmd ERROR: S3 error: 301 (PermanentRedirect). To files uploads or subpath list, using s3cmd command line. apt-get update && apt-get install -y python s3cmd Try specific region but not worked. Command line: s3cmd ls s3://bucket-my-name-sp/hello/…
reinaldo pinto
  • 117
  • 1
  • 6
0
votes
0 answers

Excluding the '#recycle' directory from s3cmd upload

I'm using s3cmd on a Synology NAS. I built an exclusion file including #recycle/* and the --exclude-from=/path/to/exc/file option but it doesn't work. I already tried this: '#recycle/*', "#recycle/*" and "\#recycle/*" but s3cmd still tries to upload…
brunobhr
  • 11
  • 4
0
votes
1 answer

I've been trying to use logrotate utility and move the compressed logs to s3, however, it rotates only when done forcefully

I've configured it to run hourly, it rotates logs hourly but not sending it to s3, what could be the reason? /var/log/newlog { rotate 5 hourly missingok notifempty compress dateext dateformat -%Y-%m-%d-%H%M%S postrotate …
KB11
  • 1
  • 1
0
votes
1 answer

How to get top 100 files inside a bucket

I have a bucket on aws and I wanna download top 100 of files inside the specified bucket. I can get one specific file using s3cmd likes the following: s3cmd get s3://bucket_name/file_name.m4a How to get 100 of them using a command? Is there any…
OmG
  • 18,337
  • 10
  • 57
  • 90
1 2 3
…
18 19