How to read file from s3 with uproot?

Question

Is there a way to open a ROOT file using s3?

ROOT support reading (but not writing) with TS3WebFile

Answer 1

Not with Uproot, at least not yet. Uproot has memmaped file (default), local file handle, HTTP, and XRootD protocols, and anything with a Python interface and the ability to read byte ranges (as opposed to the whole file) can be added.

This is the directory where backend protocols live.

Answer 2

Since uproots support http it is possible to convert s3 resource to http signing the URL. It can be done with boto3:

import boto3
s3_client = boto3.client('s3', endpoint_url=my_endpoint)
url = s3_client.generate_presigned_url('get_object', Params={'Bucket': my_bucket, 'Key': my_key}, ExpiresIn=10000)

f = uproot.open(url)

There are various authentication method using boto, e.g. via environment variables AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY.

The same procedure (thanks Chris) can be done without any special library, just implementing the algorithm provided by s3 (https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html, http://awsdocs.s3.amazonaws.com/S3/latest/s3-qrc.pdf)