2

I'm trying to count the number of lines in a file stored in an S3 bucket using AWS SELECT. Specifically, executing the following command (based upon AWS s3api documentation and this Java example for the count(*) query):

aws s3api select-object-content --bucket my_bucket_name --key my_file_name --expression "select count(*) from S3object s" --expression-type SQL --input-serialization CSV={}  --output-serialization CSV={}

The above returns:

{
    "Payload":
Circular reference detected

Even though the defaults work for my use case, I tried providing additional information with the CSV={} parameters in the command line and also tried the alternative JSON syntax '{"CSV":{}}'. Always receive the Circular reference detected error.

The file exists in the bucket; otherwise, you get a NoSuchKey error. Yeah, been there, done that... :)

Examples for AWS SELECT are either rare or my Google-fu is failing me.

Suggestions? Alternatives welcome, but they must be scriptable.

Steve Rosenberry
  • 31
  • 1
  • 4

1 Answers1

1

Not sure if someone still facing the error - updating awscli helped in my case - pip3 install awscli --upgrade

sandip
  • 11
  • 1