Delete files from the SFTP folder

Question

I have an archive folder and i want to trigger workflow every day and check if there are some files which are older than 3 month. I tried to do it like this: run a trigger, list files in the folder, go into a for each and check if file is older than 3 month and just delete action. Are there some better/faster ways to do it? I am in doubt a bit because i don't think going thru the all files will be the faster options (i could have thousands files).

Answer 1

If you're looking for a more efficient way to delete files from an archive folder in an SFTP path that are older than 3 months, you can consider using Azure Functions in conjunction with Azure Logic App. Azure Functions provide more flexibility and control over the deletion process, allowing you to optimize performance and handle large numbers of files efficiently.

Here's an alternative approach using Azure Functions and Azure Logic App:

1. Create an Azure Function: Create a new Azure Function using your preferred language (such as C#, Python, or JavaScript). The function will handle the deletion of files from the archive folder.

2. Configure the Azure Function: Implement the logic within the Azure Function to connect to the SFTP server, list the files in the archive folder, and delete files that meet the criteria (i.e., older than 3 months). You can use existing SFTP libraries or modules for your chosen programming language to interact with the SFTP server.

3. Publish the Azure Function: Publish the Azure Function to Azure and ensure it is accessible via an HTTP endpoint.

4. Modify the Logic App: In the Logic App, replace the "Delete file" action with an "HTTP" action. Configure the HTTP action to make a request to the Azure Function endpoint you published in step 3. Pass any required parameters, such as the SFTP server details and the archive folder path, as part of the HTTP request.

5. Save and Test: Save the Logic App and test it by running it manually or waiting for the scheduled recurrence. The Logic App will trigger the Azure Function to delete files from the archive folder that are older than 3 months.

By leveraging Azure Functions, you can utilize more powerful scripting capabilities and fine-tune the deletion process based on your specific requirements. This approach allows for better scalability and performance when dealing with a large number of files, as Azure Functions can handle parallel processing and optimize resource allocation.