I'm trying to download big files that are bigger than 1GBytes and process them and store the results into the database. I would like to do this as a serverless application which is keeping an eye on the FTP server that is holding files and then compare it with the previous snapshot from then files. and process those files that have been changed.
I created this in 5 different azure functions but it almost does not work and it is extremely slow compared to the local version of it.
My question is what is the best technology or framework for me to have a fast real-time serverless program that does this for me