3

is there a way to read a .ini configuration file from s3 without downloading it?

What I have tried:

config.ini:

[DEFAULT]
test = test1
test1 = test2

[ME]
me = you
you = he

code:

import boto3
import io
import configparser

s3_boto = boto3.client('s3')
configuration_file_bucket = "mybucket"
configuration_file_key = "config.ini"
obj = s3_boto.get_object(Bucket=configuration_file_bucket, Key=configuration_file_key)

config = configparser.ConfigParser()
config.read(io.BytesIO(obj['Body'].read()))

It returns [].

I have tried to make sure that 

obj['Body'].read()

is returning well a binary with content of config.ini. This is working. It breaks somewhere further.

Arhiliuc Cristina
  • 314
  • 2
  • 10

1 Answers1

5

The read method of ConfigParser takes a file name, and yet you're passing it a file object.

You can instead use the read_string method so that you can pass to it the content returned by the read method of the StreamingBody object:

config.read_string(obj['Body'].read().decode())
blhsing
  • 91,368
  • 6
  • 71
  • 106
  • 1
    I get an error: AttributeError: 'StreamingBody' object has no attribute 'readable' – Arhiliuc Cristina Feb 15 '20 at 07:19
  • Your comment that it has to be a binary stream had me look at the documentation. Finally, what worked is config.read_file(io.TextIOWrapper(io.BytesIO(obj['Body'].read()))) – Arhiliuc Cristina Feb 15 '20 at 07:22
  • No need to use a file-like object then. The point of using a file-like object is to avoid having to use the `read` method that loads the entire file into memory. But apparently `StreamingBody` doesn't implemented all the necessary attributes to make it compatible with `TextIOWrapper`, in which case you can simply use the `read_string` method instead. I've updated my answer accordingly. – blhsing Feb 15 '20 at 07:26
  • 1
    Thank you. Btw, the documentation of read() method says that it can take a bytes object. I have tried yesterday config.read(obj['Body'].read()), but it still didn't work. type(obj['Body'].read()) returns bytes. Do you know why it doesn't work? – Arhiliuc Cristina Feb 15 '20 at 07:29
  • 1
    That's because the documentation says that if it's a bytes object then it is treated as a filename, just as if it is a string. – blhsing Feb 15 '20 at 07:31
  • @blhsing, could you explain a bit more `obj['Body'].read().decode()` , please? – James Huang Sep 02 '22 at 08:50