0

I have this C++ function that downloads S3 files as istreams using the AWS SDK C++:

std::istream& s3read(std::string bucket, std::string key) {
    Aws::Client::ClientConfiguration aws_conf;
    aws_conf.region = Aws::Environment::GetEnv("AWS_REGION");
    aws_conf.caFile = "/etc/pki/tls/certs/ca-bundle.crt";
    Aws::S3::S3Client s3_client(aws_conf);
    Aws::S3::Model::GetObjectRequest object_request;
    object_request.WithBucket(bucket.c_str()).WithKey(key.c_str());
    auto get_object_outcome = s3_client.GetObject(object_request);

    if (get_object_outcome.IsSuccess()) {
        std::istream& res = get_object_outcome.GetResult().GetBody();
        return res;
    } else {
        ...
    };
};

I call it from main.cpp and try to parse it with Jsoncpp:

std::istream& stream = s3read(bucket, key);
Json::Value json;
Json::Reader reader;
reader.parse(stream, json);

However, I keep getting segmentation fault. Why?

I think that the problem is that reader.parse needs binary data and the istream isn't. But, if I'm right, how can I parse the stream as binary?

Medical physicist
  • 2,510
  • 4
  • 34
  • 51
  • 1
    Inside the function `get_object_outcome` is a *local* variable. As such its life will end when the function ends, meaning all references to it or to members inside the object will become invalid. Using such references will lead to [*undefined behavior*](https://en.wikipedia.org/wiki/Undefined_behavior) and possible crashes. – Some programmer dude Feb 06 '19 at 11:31
  • 1
    You'r issue is `std::istream& res = get_object_outcome.GetResult().GetBody(); return res;` Check the compiler warning (enable them all). You are returning reference (address) to the local object, just after `s3read` call `get_object_outcome` is already destroyed so address to the stream object is no longer show into opened input stream. – Victor Gubin Feb 06 '19 at 11:32
  • 1
    What you can do - return parsed json object from the s3read – Victor Gubin Feb 06 '19 at 11:34

1 Answers1

0

The issue you'r have is classical returning reference to temporary

You can re-design your code a little, to avoid this. For example:

static Json::Value parse_json(std::istream& src) {
     Json::Value ret;
     Json::Reader reader;
     reader.parse(src, ret);
     return ret;  
}
// Aws::String is actually same thing to std::string except the allocator
// in case of Android, otherwise this is std::string as it is. 
// You can use function like s3read("foo","bar");  
Json::Value s3read_json(const Aws::String& bucket,const Aws::String& key) {
    static constexpr const char *FILE_NAME = "/etc/pki/tls/certs/ca-bundle.crt";

    Aws::Client::ClientConfiguration aws_conf;
    aws_conf.region = Aws::Environment::GetEnv("AWS_REGION");
    aws_conf.caFile = FILE_NAME;

    Aws::S3::S3Client s3_client(aws_conf);
    Aws::S3::Model::GetObjectRequest object_request;
    object_request.WithBucket( bucket ).WithKey( key );

    auto object_outcome = s3_client.GetObject(object_request);

    if (object_outcome.IsSuccess()) {
        auto result = object_outcome.GetResult();
        // destructor of object_outcome is not yet called
        return parse_json( result.GetBody() );
    } else {
        ...
        // throw std::runtime_error("S3 connection failed");
    };
};
Victor Gubin
  • 2,782
  • 10
  • 24