I have a lambda reading csv's in an S3 bucket. The api gateway calls the lambda. The data in the csv is like this:
Ticker Exchange Date Open High Low Close Volume
6A BATS 12/2/2021 0.9 0.95 0.83 0.95 1200
6B BATS 12/3/2021 1 1.3 0.9 1.2 1500
6C BATS 12/4/2021 1.2 1.3 1.1 1.1 1300
here is my code:
import json
import awswrangler as wr
BUCKET_NAME = 'olaptrader-products'
DIRECTORY_PATH = '1'
FULL_PATH = f"s3://{BUCKET_NAME}/{DIRECTORY_PATH}"
def lambda_handler(event, context):
raw_df = wr.s3.read_csv(path=FULL_PATH, path_suffix=['.csv'], use_threads=True)
df = raw_df.to_json(orient="records")
parsed = json.loads(df)
return {
"statusCode": 200,
'body': json.dumps(parsed)
}
My output looks like this:
[{"Ticker": "6A", "Exchange": "BATS", "Date": "12/2/2021", "Open": 0.9, "High": 0.95, "Low": 0.83, "Close": 0.95, "Volume": 1200}, {"Ticker": "6B", "Exchange": "BATS", "Date": "12/3/2021", "Open": 1.0, "High": 1.3, "Low": 0.9, "Close": 1.2, "Volume": 1500}, {"Ticker": "6C", "Exchange": "BATS", "Date": "12/4/2021", "Open": 1.2, "High": 1.3, "Low": 1.1, "Close": 1.1, "Volume": 1300}]
How do I get rid of those extra spaces? I have tried "skipinitialspace" in the df.to_json, but that apparently is not for json, only csv and so, it didnt work. I dont want to get rid of ALL spaces, because there will be situations where I have a company name in there with a space and I want to keep that. Just need to get rid of these leading spaces. Thanks