2

Newbie to python here. I am changing some code which handles dynamodb write from sync to async, to reduce the latency. Current source code uses boto3 to connect to dynamodb:

import boto3


"""
Create dynamodb connection
"""
def _create_dynamo_connection(self):
   dynamodb = boto3.resource('dynamodb')
   return dynamodb 


"""
Write to dynamodb
"""
def _write_to_dynamodb(self, dynamodb, data):
   table = dynamodb.Table(DYNAMODB_TABLE)
   data_entry = json.loads(json.dumps(data))
   # The put_item takes time and we would want to execute the write step without wait
   aws_response = table.put_item(Item=data_entry)

In order to make the write function from synchronous to asynchronous approach, initially I was thinking of using aioboto3 (lib) replacing boto3, but after seeing the example code, it seems like the dynamodb is created with with clause and cannot be returned. This makes the syntax very different from the current

import asyncio
import aioboto3


async def main():
    session = aioboto3.Session()
    async with session.resource('dynamodb', region_name='eu-central-1') as dynamo_resource:
        table = await dynamo_resource.Table('test_table')

        await table.put_item(
            Item={'pk': 'test1', 'col1': 'some_data'}
        )

I am wondering: if the only need is to make the "write" async, can I just make minor modification of the _write_to_dynamodb() method? Will I be able to have a workaround without the aioboto3 but with asyncio. If so, what will the syntax look like?

dirn
  • 19,454
  • 5
  • 69
  • 74
jsh6303
  • 2,010
  • 3
  • 24
  • 50
  • If you don’t use aioboto3, you won’t be able to await `table.put_item`. – dirn Mar 06 '22 at 13:01
  • @dirn thanks. I was also wondering if I can get around by just using library `asyncio` ? – jsh6303 Mar 06 '22 at 14:21
  • I think you’d be doing yourself a disservice by not using libraries that know how to speak to AWS for you, but sure, you could do it without a third party library. I just wouldn’t recommend it. – dirn Mar 06 '22 at 17:33
  • @dirn the reason I am a little concerned about using `aioboto3` is that for each write operation, it requires a new connection (or session), which is different from what I currently have. The thought of breaking the existing logic into separate functions is to use the same dynamodb instance already created. – jsh6303 Mar 06 '22 at 17:41
  • @jsh6303 you can use `contextlib` for maintaining the session until you manually close it, that is, avoid the `async with` altogether. The author of `aioboto3` has mentioned this solution in one of the issues [here](https://github.com/terrycain/aioboto3/issues/279#issuecomment-1309061484) – D.B.K Nov 19 '22 at 21:01

0 Answers0