0

Is there any way to correctly upload unicode filenames to S3 using s3cmd from an OS X machine?

The problem is the following:

  1. OS X uses NFD filenames, Linux/BSD and Amazon uses NFC (for static file serving).
  2. When using rsync, I can specify the --iconv=UTF8-MAC,UTF-8 switch, which does the correct file-name conversion.
  3. However when using s3cmd, the file name conversion does not happen, thus non-ascii files cannot be accessed from the web (using their original URL), as their names are now in NFD form.

The only workaround I know for this issue is to upload the files to a Linux/BSD server and run s3cmd from there.

Is there any solution to this which does not involve using a Linux/BSD host?

hyperknot
  • 13,454
  • 24
  • 98
  • 153
  • Why do you think "Linux/BSD and Amazon uses NFC"? – 一二三 Sep 30 '15 at 00:50
  • I guess they use whatever is supplied, but for example if I wget --mirror a website from anywhere on the internet, I'll get NFC filenames, not NFD. So I cannot upload them from OS X, as the links in HTMLs will be broken. – hyperknot Sep 30 '15 at 07:53

1 Answers1

2

Alternatively you can take a look at mc at https://github.com/minio/mc.

mc provides minimal tools to work with Amazon S3 compatible cloud storage and filesystems. It has features like resumable uploads, progress bar, parallel copy. mc is written in Golang and released under Apache license v2.

Since 'mc' is written in Golang we get native UTF-8 support. 'mc' is available for OS X, Windows and Linux. 

$ mc cp 本語.txt s3/andoria/

Feel free to report any issues if you find at https://github.com/minio/mc/issues . Hope this helps.

Harshavardhana
  • 1,400
  • 8
  • 17