1

I'm using aria2 to download a bunch of files my problem now is that I have to postprocess the files within a VBA Application. Until now the postprocessing logic is already written in VBA so I'm looking for a solution to read the urls not just from a list (exported by VBA) rather than csv file to get them stored using a unique identifier. Otherwise, I'm not able to find the downloaded files in case of a name collision. Additionally to this, the content should be stored within a sqlite database to increase the processing speed. I'm talking about thousands of files with a total size of multipe GBs so the speed of my VBA application suffers from disk io.

Is there a solution for my problem?

Currently, I approach the Issue using a golang wrapper script which reads the csv file and spawns a bunch of aria2 instances using the --out parameter to control the output file naming.

Sample Code:

Export csv

path = ThisWorkbook.path & "\" & dataFile

If FileHandler.FileExists(path) Then
    FileHandler.DeleteFile (path)
End If
FileHandler.CreateFile (path)

fileHandle = FreeFile
Open path For Append As #fileHandle

For WksIndex = 3 To 37
    If SettingGet("Enable", WksIndex) Then ' check if worksheets should be processed
        For Row = 1 To Worksheets(WksIndex).Cells(9999, 1).End(xlUp).Row
            If Worksheets(WksIndex).Cells(Row, 1).value <> "" And Worksheets(WksIndex).Cells(Row, 3).value <> "" Then
                Call FileHandler.AppendToFile( _
                    path, _
                    Worksheets(WksIndex).Cells(Row, 1).value & ";" & _
                    Worksheets(WksIndex).Cells(Row, 3).value, _
                    fileHandle _
                ) ' append ID;URL to file
            End If
        Next Row
    End If
Next WksIndex

Close #fileHandle
Community
  • 1
  • 1
McNulty
  • 31
  • 5
  • Hi, welcome to Stackoverflow. Please read the [How to ask a good question section](https://stackoverflow.com/help/how-to-ask) and rephrase your question with sample code of what you have tried. – Roan Aug 27 '17 at 10:44
  • Thanks Roan! I'm not sure about the code you requested because it's a kind of 'conceptual' question – McNulty Aug 27 '17 at 11:27
  • If I would be able to redirect the aria2 file output to stdout I could process the data using my wrapper script but it seems like there is no way to achieve this. – McNulty Aug 27 '17 at 11:51

0 Answers0