Basically I'm using the command ´distcp´ to put some data in S3, pretty small files. I am sending the data into a bucket, and a put the files inside a folder of the bucket.
It works fine and I can see on the logs that something has been written to S3 and if I do an ls
on the bucket I can see the size of the bucket is exactly the same of the data written on the logs:
S3A: Number of bytes read=0
S3A: Number of bytes written=609
S3A: Number of read operations=11
S3A: Number of large read operations=0
S3A: Number of write operations=5
DistCp Counters
Bytes Copied=609
Bytes Expected=609
Files Copied=1
The issue is that if I ls
on the folder inside the bucket it's empty, I can't see the file and unsure why.