Is there a way to set up FTP using Azure Web App service? Is it even
possible?
As I known, it's not possible to set up FTP service via Azure Web App Service to reach your request. You can use Deployment Credentials to deploy your web App via FTP, but cannot use Web App to deploy FTP service.
According to your request, I suggest you can let your customers use Azure Storage Explorer .
For more convenient to your customers, using scripts to using Azure Blob to upload/download file is not recommended. Azure Storage Explorer can let you access Azure Blob like FTP. You can create different Access keys for customers and they can use their own keys to access the Blob.
If you don't want take this solution, I think that it's necessary to set up a FTP server via Azure VM.