Using custom connector i am able to load data from mysql to s3 in AWS Appflow but when creating a new table in mysql, it is not showing up in Appflow console.
Asked
Active
Viewed 126 times
1 Answers
0
New entities are discovered during the ListEntities event that is called recursively when building a flow and selecting the source object. Since you are building a custom connector all the logic of how the connector discovers entities is in your custom code. Specifically the list_entities method of your MetadataHandler.
See the end-to-end flow of the readme of the Python custom connector SDK. I quote part here.
https://github.com/awslabs/aws-appflow-custom-connector-python
Step 5: AppFlow user then uses that ConnectorProfile to create Flow definitions. During the flow creation AppFlow invokes listEntity and describeEntity functions on the connector to retrieve the list of supported entities and the entity detail of the specific entity AppFlow user selected on the console / API.
With that said, if you can share I'm your question the code of your metadata handler as well as a description of the source database and why you think the new table should be discovered by your code.
I would also look at the CloudWatch logs or wherever you store logs for the Lambda hosting your custom connector. View the logs after trying to build a flow and not seeing the expected new entities listed. If there is an error trying to read a new entity you could see it here if you have the proper logging setup.
