I'm using AWS CLI tool to download hundreds of thousands of files. I have almost a million of these one-liners generated from SQL query with different file path that I need to go through:
aws s3 cp s3://[myS3FilePath]/17802c9-6d3b-4eef-855a-a6ae0039c7ff/ C:\[MyLocalFilePath]\17802c9-6d3b-4eef-855a-a6ae0039c7ff\ --recursive
I've been taking ~1000 lines at a time, pasting them to command prompt, and waiting for them to be iterated through. Works great! It's quite a time waste doing it in 1000 record batches though. What's the maximum amount of lines that I could paste to CMD without losing any of my download commands? Could I paste 1.000.000 lines into the command prompt for example and trust that it will iterate through all of them?